From b24c9a8fdf89c50a8cb6f3c78fe742b56ba108ce Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 28 Sep 2026 01:16:40 +0000 Subject: [PATCH] Plan policy alignment and release workflow updates --- docs/roadmap/implementation.md | 795 +++++++++++++-------------------- docs/roadmap/policy.md | 7 +- 2 files changed, 320 insertions(+), 482 deletions(-) diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 4ef5b5e..8c4f956 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,482 +1,319 @@ -# Implementation Plan: Audita v2 and Shared Provider Credentials - -Status: Complete (2026-09-25). - -## Scope and working instructions - -Implement [the Audita roadmap](audita.md). The baseline is Narratio commit -`70f1dbf` (v2.2.1), with ConfigKit v0.4.0 and Audita v2.0.0 as the target -upstream contracts. This replaces the completed checkpoint-refactoring plan, -which remains in Git history. - -Complete Stage 1 through Stage 8 in order, one bounded implementation prompt per -stage for gpt-5.6-terra. Record completion and actual verification results under -each stage. These instructions apply to the implementation sprint; the planning -pass changes documentation only. Intermediate stages are not release candidates. -Do not add compatibility code to make intermediate states look like a supported -release. - -Before each stage, read `AGENTS.md`, `docs/development.md`, and all policies -under `docs/policy/`: agents, architecture, documentation, testing, and release. -Read this plan's selected design and that stage's preparation. Use Go 1.26 and graph-first code -discovery; inspect current files when graph results are incomplete. Preserve -unrelated changes. Test at behavioral owners rather than replicating every case -across configuration, adapter, stage, and command tests. - -The user approved a clean cutover: no Audita v1 runner, deprecated aliases, -compatibility shims, generated configuration conversion, or silently ignored -old settings. Do not broaden that decision into a purge of unrelated legacy -code or historical session data. - -## Selected design - -### ConfigKit and Narratio overlay terminology - -Upgrade ConfigKit from v0.3.1 to v0.4.0 before adapter work. Rename its -`Options.Profile`, `LayerProfile`, and any `CodeProfile*` uses to their overlay -equivalents. Selected pipeline files move from `profiles/.yml` to -`overlays/.yml`; no fallback directory or alias is permitted. - -Rename Narratio's pipeline-selection flag to `--overlay` everywhere it is -accepted, including convenience commands and configuration inspection. Rename -application-owned selection types/helpers, help, errors, and current inspection -labels consistently. `config diff` still takes two positional selections, -now described as overlays. Preserve omission, explicit empty, and `default` -as base selection; do not add a persistent default selector or new precedence. - -Rename newly written informational provenance to overlay terminology, including -`selected_overlay` in manifest configuration provenance. Do not add a dual-field -reader, automatically rewrite historical manifests, or introduce a manifest-version -rejection solely for this informational rename. Existing strict typed decoding -rejects retired provenance fields. Document backed-up operator migration of -that informational member without changing execution/checkpoint state; it does -not become resume authority. Preserve -semantic fingerprints and effective-value digest rules when values are unchanged. - -Only Narratio pipeline configuration selection is renamed. Keep model/profile -fields such as Scriptorium `profile_id`, Audita `proposal_profile` and -`validation_profile`, and Notarius `config_profile` unchanged. Explicit imports -from directories named `profiles` remain legal; only selected pipeline overlay -discovery changes. Do not globally replace the word “profile.” - -### Configuration and credentials - -- Both `pipeline.audita.credentials` and `pipeline.notarius.credentials` are - lists of ConfigKit secret references. Use `[]configkit.Secret`; each element - is an actual reference rather than a nullable pointer. Existing ConfigKit - validation owns name/value resolution. Omitted, null, or empty lists mean no - additional credential forwarding. -- Define each key once through existing environment/secrets-directory bootstrap. - Forward it under its original name to each adapter whose list includes it. - No alias mappings, provider registry, downstream config inspection, extra - bootstrap, or inheritance of the entire environment. -- Repeated names need no new rejection rule: forwarding the same captured - reference once suffices. Arbitrary valid variable names remain supported; - provider names in the roadmap are examples, not an allowlist. -- Add a narrow shared helper in `internal/adapters/subprocess` converting resolved - references to `EnvOverrides` and `SensitiveEnvNames`. It checks the existing - resolved/nonempty requirement for direct adapter callers but never resolves - secrets or rereads environment values. Errors expose no values. Both adapters - use it; the ConfigKit upgrade is owned by Stage 1, not this helper. No new - credential service is needed. -- Configuration projections and diagnostic metadata contain reference names - only. Register all forwarded names for existing captured-stream redaction. - Downstream tools remain responsible for files they write themselves. -- Remove both singular `llm_api_key` fields. For Audita also remove `base_url`, - `model`, `validation_model`, `total_llm_concurrency`, - `proposal_llm_concurrency`, `validation_llm_concurrency`, - `work_dir_retention`, and `report`, including defaults and projections. -- Retain Audita `binary`, `timeout`, `config_path`, `transcript_description`, - `output_schema`, and `modules`. Add optional string selectors `overlay`, - `proposal_profile`, and `validation_profile`: empty/unset means no CLI - override. Audita owns downstream file/profile validation and native config - discovery. Narratio overlay names are not implicitly Audita overlay names. -- Preserve nil versus non-nil empty `modules` through loading, projection, - cloning, composition, adapter defaults, and request overrides. Nil omits the - flag; empty passes `--modules` with an empty argument; nonempty joins in order - with repetitions intact. A nil request override uses the configured selection. - Avoid clones that collapse empty slices to nil. YAML null follows unset - behavior. Other adapters' module semantics do not change. - -### Invocation, reports, and failure - -Use `process INPUT --glossary PATH --output PATH` plus selected surviving -overrides. Production passes no reporting or debug switches, including -`--debug`, `--reporting=false`, `--report-dir`, or `--work-dir`. -Remove required `WorkDir` and legacy `ReportPath` request/result plumbing. -Document production external-config defaults without forcibly overriding or -rejecting deliberate external reporting/debug choices. - -Replace Audita's long positional config-values constructor with typed runner -configuration. Keep timeout parsing in composition and subprocess cancellation -through the existing runner. No version probe or automatic config-validation -subprocess is required. - -The corrected transcript remains the only canonical polish output, with existing -identity, path, validation, and bounded file handling. Discover exactly -`errors.yml`, `recoveries.yml`, and `report.md` beside the requested output. -Represent available paths in a small typed report bundle in `PolishResult`. -Fixed names belong to the adapter; do not parse stderr or recursively scan. - -Reports are diagnostics, not a new correctness gate: - -- Bound reads of present reports using the existing named 16 MiB report limit; - retain confined regular-file handling and errors identifying Audita, the file, - and limit. No new smaller limits or report schema/status/count validation is - necessary when Narratio only preserves bytes; Markdown needs no parser. -- Missing files are not independently fatal: external configuration can disable - reports and early failures may produce none. Default-reporting success tests - must nevertheless demonstrate capture of all three files. -- On process failure, preserve available diagnostics and the original error; - report-inspection errors must not hide it. Never promote an output merely - because it exists following a nonzero exit. -- Exit zero with a valid transcript may include recoverable errors or no model - corrections. Accept it without a fail-on-recovery or minimum-correction rule. - -Keep reports next to the run-local transcript, not in a second canonical -destination. Register present report paths in the existing `StageResult.Logs` -diagnostic collection alongside stdout/stderr, with safe bundle-path metadata. -Reuse current manifest-driven diagnostic publication rather than adding a -manifest schema or new publish source IDs. Physical placement does not define -canonical artifact identity. Existing diagnostic retention/cleanup rules apply. - -On failure, retain run-local files and expose the output/report directory in -contextual errors. Do not add runner-wide failed-result persistence merely to -register those files. Do not fabricate legacy JSON reports or delete historical -reports. Normal successful stage-record replacement removes obsolete current -output references. - -Generated invocation YAML becomes `audita.generated.v2`, recording actual -arguments, selections, paths, and credential names, never values. This remains -a diagnostic invocation record, not an Audita configuration file. - -### Resume boundary - -Bump only polish semantic-config version from 1 to 2. Record config path, -overlay/proposal/validation selectors, ordered modules with presence preserved, -transcript description, output schema, and existing output identity. Remove old -endpoint/model fields. Existing runner comparison reconsiders obsolete evidence -when polish is selected; do not synthesize equivalent fingerprints or expand -normal dependency/selection invalidation. - -Credentials, binary path, timeout, and diagnostics are not semantic keys. Do not -hash downstream config/profile contents; operators force polish after edits -behind unchanged selections. Notarius credential migration does not change -extraction fingerprints, references, force, resume, or chunk-cache behavior. - -## Stage 1: Adopt ConfigKit v0.4.0 and pipeline overlays - -Status: Complete (2026-09-25). - -Verification: `go test ./internal/config ./internal/app ./internal/manifest`, -`go test ./...`, `go test ./internal/doccheck`, -`go test ./internal/config -run '^TestExamplesLoadAndValidate$'`, and -`git diff --check` passed. - -Preparation: ConfigKit v0.4.0 release notes and selected-overlay migration guide -in `../ConfigKit/docs/`; Narratio `internal/config/pipeline_configkit.go`, -`artifact_catalog.go`, `load.go`, `effective_digest.go`; CLI flag parsing and -config commands in `internal/app/`; configuration provenance in -`internal/manifest/manifest.go`; current config/CLI docs and pipeline examples. - -1. Update `go.mod`/`go.sum` to ConfigKit v0.4.0 using normal module tooling; - do not leave a local replacement or change unrelated dependencies. - Replace removed ConfigKit APIs in loading, ownership, and artifact catalog - inspection. Account for source layer `overlay` and diagnostic prefix - `overlay.*` wherever they are consumed or asserted. -2. Rename Narratio's pipeline selector to `--overlay` across all commands and - aliases, load options, pair loading, inspection, provenance, and tests. - Preserve explicit-presence handling through convenience commands and existing - base/default selection. Remove `--profile` rather than retaining an alias. - Apply the selected design's informational manifest-provenance treatment. -3. Move maintained selected pipeline files and generated fixture paths to - `overlays/`. Update current canonical documentation, policy terminology, - examples, and user-visible migration guidance with the directory/flag break. - Do not rewrite historical release notes, rename downstream model profiles, - or prohibit ordinary explicit imports from a directory named `profiles`. -4. Adapt existing config/app tests to prove base and named-overlay selection, - imported defaults plus overrides, secret replacement/removal, source attribution - through character-family expansion, config diff, and convenient-command - forwarding. Add focused checks that the retired flag is rejected and the old - directory is not a discovery fallback. Do not freeze incidental error wording. -5. Verify unchanged effective values retain digest and stage-reuse behavior; - only diagnostic source labels/paths and selected provenance terminology change. - Test current provenance round trips and strict rejection of unknown/retired - provenance fields. Document deliberate operator migration; do not add a - compatibility decoder, semantic invalidation rule, or schema-version gate. -6. Run focused and full tests before proceeding so failures from the dependency - and terminology change are separated from adapter migrations. No Audita or - Notarius credential/CLI behavior changes in this stage. - -Verify: `go test ./internal/config ./internal/app ./internal/manifest`, -`go test ./...`, `go test ./internal/doccheck`, -`go test ./internal/config -run '^TestExamplesLoadAndValidate$'`, and -`git diff --check`. - -Done when Narratio uses ConfigKit v0.4.0, exposes pipeline overlays consistently, -has no old selector/directory fallback, and preserves configuration semantics and -downstream model-profile concepts. - -## Stage 2: Add shared resolved-credential forwarding - -Status: Complete (2026-09-25). - -Verification: `go test ./internal/adapters/subprocess`, -`go test -race ./internal/adapters/subprocess`, and `git diff --check` passed. - -Preparation: `internal/adapters/subprocess/run.go`, its environment/redaction -helpers and tests; `internal/adapters/notarius/subprocess.go`; -`internal/config/credential_projection.go`; the pinned ConfigKit secret API. - -1. Add the narrow helper described above. Return an independent environment map - and selected sensitive names. No logging/serialization of values and no - changes to minimal child environment inheritance. -2. Test multiple custom names, empty lists, repeated names, unresolved references, - and captured values after the parent environment changes. Reuse existing - split/overlapping-stream redaction coverage rather than repeating its matrix. -3. Leave production callers unchanged until their migrations. - -Verify: `go test ./internal/adapters/subprocess` and -`go test -race ./internal/adapters/subprocess`. - -Done when both adapters can reuse one small conversion mechanism without adding -a resolver, provider policy, or ambient-secret inheritance. - -## Stage 3: Migrate Notarius to credential lists - -Status: Complete (2026-09-25). - -Verification: `go test ./internal/config ./internal/adapters/notarius ./internal/stage ./internal/app`, -`go test -race ./internal/adapters/notarius`, and `git diff --check` passed. - -Preparation: `internal/config/config.go`, `credential_projection.go`, loading/ -inspection tests; `internal/adapters/notarius/runner.go`, `subprocess.go`; -`internal/stage/extract.go` and its request construction; -`docs/integrations/notarius.md` and secret handling in `docs/config.md`. - -1. Replace the singular field with the typed list across configuration, - projection, request types, stage wiring, and adapter execution. Use Stage 2's - helper and register all selected names for redaction. -2. Update name-only metadata and config-show/diff/effective projections. Test - real ConfigKit loading, overlay list replacement, and empty-list clearing. - Preserve bootstrap precedence and existing load-time reference resolution, - including references outside the selected stage range. -3. Migrate maintained examples, fixtures, and current Notarius configuration/ - integration documentation for this field. Ordinary strict decoding rejects - the old field; do not add a compatibility parser. -4. Cover multi-key forwarding, unselected-key exclusion, empty selection, - captured-value behavior, and custom-name redaction on success/failure. - Keep extraction cache/reference/output tests as regression protection, - not occasions to redesign extraction. - -Verify: `go test ./internal/config ./internal/adapters/notarius ./internal/stage ./internal/app` -and `go test -race ./internal/adapters/notarius`. - -Done when Notarius uses credential lists end to end and no active singular -credential path remains. - -## Stage 4: Cut Audita configuration and adapter over to v2 - -Status: Complete (2026-09-25). - -Verification: `go test ./internal/config ./internal/adapters/audita ./internal/stage ./internal/app`, -`go test -race ./internal/adapters/audita`, `go test ./...`, and -`git diff --check` passed. - -Preparation: Audita v2.0.0 `docs/migrations/v2.md`, `docs/cli.md`, -`docs/integrations/subprocess.md`, and `docs/integrations/output-files.md` in -`../audita/`; Narratio `internal/adapters/audita/`, -`internal/app/runner.go` (`buildDefaultAuditaRunner`), and config types, -defaults, validation, and credential projection. - -1. Replace the Audita configuration surface as selected above, updating defaults, - validation, safe projection, examples, and config fixtures. Prove nil/empty - modules survive real configuration loading and overlay overrides. -2. Migrate runner construction, selectors, and credential forwarding. Remove - retired flags, fixed credential alias, required work-directory plumbing, and - old JSON-report validation. Add the typed report bundle. -3. Implement adjacent report discovery, bounded reads, failure results, and - generated invocation v2. Share metadata construction between success/failure - instead of maintaining duplicate field maps. -4. Rewrite the Audita subprocess fixture to model v2 argument acceptance and - output placement. Cover omitted/selected selectors, ordered/repeated/empty - modules, no report/debug switches, malformed/missing transcript, ordinary - reports, externally disabled reporting, recoverable-error success, and - nonzero exit after transcript creation. Cover multiple custom credential names - and redaction; preserve timeout/cancellation coverage at its existing owner. -5. Make mechanical polish and app-fixture adjustments needed to compile: remove - obsolete request/metadata fields and old report materialization. Stage 5 owns - complete diagnostic integration. Remove retired fingerprint members as needed - to compile; Stage 6 completes the replacement semantic contract. These are - bounded call-site changes, not permission to introduce transitional shims. - -Verify: `go test ./internal/config ./internal/adapters/audita ./internal/stage ./internal/app` -and `go test -race ./internal/adapters/audita`. - -Done when configuration composes a v2-only adapter and subprocess tests prove its -credential/transcript/report boundary. Stages 5 and 6 complete diagnostic and -semantic integration; all stages are required before release. - -## Stage 5: Integrate polish reports with diagnostic lifecycle - -Status: Complete (2026-09-25). - -Verification: `go test ./internal/stage ./internal/app ./internal/manifest ./internal/artifacts`, -`go test -race ./internal/stage ./internal/app`, and `git diff --check` passed. - -Preparation: `internal/stage/polish.go`, `polish_test.go`, `run_local.go`, -`stage.go`, `publish_sources.go`; result application in `internal/app/runner.go`; -publish/restore diagnostic tests; `docs/internal/stage-polish.md`. - -1. Finish stage integration: materialize only the canonical polished transcript; - register available adjacent reports as diagnostics plus safe metadata and - existing logs/generated invocation references. Remove old `audita_report` - production and stale stage metadata. -2. Use existing authoritative-path checks for the requested transcript and fixed - adjacent report paths. Do not accept arbitrary fake-returned paths, scan - directories, or copy reports to a second destination. Preserve workspace - handling/modes. -3. On execution failure, preserve canonical transcript authority and run-local - diagnostics, with actionable directory/log context and original error identity. - Test failure after output creation. -4. Adapt the fake and app helper. With real manifest/store collaborators, prove - successful promotion, report references, absence of report canonical outputs, - and current-run diagnostic publication. Preserve restore's existing scope of - declared committed transcript/artifact objects: run-archive diagnostics do not - become automatically restored canonical objects. Test that report references - do not create new required canonical restore targets. Document remote diagnostic - inspection separately; no new diagnostic-restore feature is authorized. -5. Leave historical files untouched. Update tests expecting a second canonical - polish output without introducing a manifest-format migration. - -Verify: `go test ./internal/stage ./internal/app ./internal/manifest ./internal/artifacts` -and `go test -race ./internal/stage ./internal/app`. - -Done when reports use existing diagnostic lifecycle and failed execution cannot -promote an incidental transcript. - -## Stage 6: Complete semantic evidence and resume behavior - -Status: Complete (2026-09-25). - -Verification: `go test ./internal/stage ./internal/app`, -`go test -race ./internal/stage ./internal/app`, and `git diff --check` passed. - -Preparation: `internal/stage/semantic_contracts_refinement.go`, semantic -fingerprint tests, app runner semantic comparison tests, and extraction semantic -configuration tests (locate their current owners through graph discovery). - -1. Implement polish evidence version 2 and the selected payload. Remove sorting; - preserve nil/empty distinction in payload encoding and copies. -2. Test changes from module order/repetition/presence and external selectors. - Credentials and execution-only settings do not change evidence. Avoid - assertions on incidental digest literals. -3. Exercise old-version evidence through runner resume behavior: selected polish - is reconsidered and normal dependent invalidation applies. Downstream-only - bounded execution does not unexpectedly invoke polish. Unchanged new evidence - reuses valid completed output. -4. Confirm credential-list changes leave extraction semantic/cache behavior - unchanged. No downstream file-content scanning or eager fingerprint migration. - -Verify: `go test ./internal/stage ./internal/app` and -`go test -race ./internal/stage ./internal/app`. - -Done when selections affect reuse accurately without blanket credential/config -invalidation or changes to unselected-stage execution. - -## Stage 7: Complete operator migration documentation and examples - -Status: Complete (2026-09-25). - -Verification: `go test ./internal/doccheck`, -`go test ./internal/config -run '^TestExamplesLoadAndValidate$'`, and -`git diff --check` passed. - -Preparation: documentation policy; `docs/config.md`, `docs/operations.md`, -`docs/troubleshooting.md`, both integration guides, -`docs/internal/adapters.md`, `docs/internal/stage-polish.md`, and pipeline examples. - -1. Complete current-behavior documentation as implementation lands. Configuration - owns fields/defaults; integrations own external contracts; operations owns - placement and migration; internal docs explain ownership and link onward. -2. Provide an old-to-new migration table: endpoints/capacity move to Audita - backends, models to profiles, role selection to selectors, and credentials to - provider-named lists. Remove report/retention settings. Explain Audita's - all-configured-backend credential resolution and Notarius's profile lookup. -3. Describe preparing and validating Audita v2 config offline with its documented - command before switching executables. Link upstream examples rather than - building a downstream configuration generator. Explain ordinary reports, - recoverable-error success, and force after edits behind unchanged selectors. -4. Production examples omit modules when upstream defaults are intended; - `modules: []` deliberately disables correction modules. Production external - Audita config keeps reports enabled/debug disabled, without Narratio flag - overrides. Testing-profile debug integration remains deferred. -5. Document the configuration-breaking clean cutover for release-version - consideration. Do not edit historical release records, tag a release, or - remove unrelated roadmaps. - -Verify: `go test ./internal/doccheck`, -`go test ./internal/config -run '^TestExamplesLoadAndValidate$'`, and -`git diff --check`. - -Done when operators can migrate using valid examples without obsolete fields, -implicit credential aliases, or retired CLI/report expectations. - -## Stage 8: Verify assembled behavior and audit scope - -Status: Complete (2026-09-25). - -Verification: the offline app-level shared/Audita-only credential scenario -passed with config bootstrap, both subprocess fixtures, safe inspection/logging, -and ordinary polish reports. The tagged Audita v2.0.0 CLI validated its minimal -configuration and produced a transcript plus all three reports with an empty -module sequence and no provider request. Active-reference and scope audits found -no removed production fields or compatibility path; remaining old names occur in -migration guidance, explicit-rejection checks, or historical fixtures. `go test -./...`, `go test -race ./...`, `go vet ./...`, `go build ./...`, `go test -./internal/doccheck`, `go test ./internal/config -run -'^TestExamplesLoadAndValidate$'`, and `git diff --check` passed. - -Preparation: cumulative diff, this plan, feature roadmap, app configuration/ -subprocess integration tests, and all policies. - -1. Adapt or add one representative offline app-level scenario showing one - provider-named secret source supplying both adapters and another key selected - only for Audita. Use real config/bootstrap/composition with subprocess fixtures. - Prove safe inspection/logging and successful polish reports. Leave exhaustive - flag/redaction cases at lower layers. -2. Where the tagged Audita checkout is available, smoke-test CLI acceptance and - ordinary transcript/reports offline with an explicit minimal v2 config, - unauthenticated backend/profile fixtures, and an empty module sequence. - No provider requests or production credentials. This is optional local evidence, - not a default test dependent on sibling checkouts or installed tools. -3. Audit active references to removed fields/flags/aliases; historical release - notes and completed roadmaps are not compatibility code. Review tests - collectively for duplication while retaining consequential failure coverage. -4. Confirm no provider-name restrictions, duplicate-name rejection, downstream - content hashing, fail-on-recovery rule, mandatory explicit config path, debug - requirement, live-service gate, pipeline-order change, or Notarius cache change - slipped in. Do not require missing reports when disabled externally. Surface - additional policy choices rather than encoding them silently in tests. -5. Run every gate below and record actual results. Mark complete only after all - stages and gates pass. No release, tag, push, or unrelated cleanup in this plan. - -```sh -go test ./... -go test -race ./... -go vet ./... -go build ./... -go test ./internal/doccheck -go test ./internal/config -run '^TestExamplesLoadAndValidate$' -git diff --check -``` - -Done when the assembled v2 integration and shared credentials meet the roadmap, -verification passes, and no compatibility machinery remains within scope. +# Implementation Plan: Standard Repository Policy Alignment + +Status: Pending. + +## Scope and execution instructions + +Implement [the policy roadmap](policy.md). This replaces the completed Audita +migration plan, retained in Git history. This planning pass changes no executable +code or adopted policies. The instructions below govern the later implementation. + +Complete Stage 1 through Stage 7 in order, one bounded implementation prompt per +stage for gpt-5.6-terra. Record actual completion and verification under each stage +and commit each coherent completed unit with a plain-English message. Intermediate +states are not release candidates. Do not publish, tag, push, change dependencies, +or rewrite released material as part of this work. + +Before each stage, read root AGENTS.md, the development reading guide, the roadmap, +and the policies applicable to that stage. Reuse unchanged policy text already in +context. Agents and documentation apply throughout; meta governs adoption; OSS +governs incorporated material; architecture/testing/release apply to the script +and test stages. Do not impose whole-suite policy reading on unrelated tasks. + +Use upstream commit `d84a38ebab26f048cafbe59a2246700c35d29843` throughout. Start +with its [adoption guide](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/adoption.md). +Read source documents at that revision, not mutable sibling worktree content. +Public access or a verified Git object is sufficient. Do not reconstruct prior +adoption history. If an exact source/notice cannot be verified, report the concrete +gap rather than inventing a commit, copyright, or license expression. + +## Selected decisions + +### Common policy language and provenance + +Adopt non-project-specific upstream wording verbatim. Remove template-only +sections, customize required project content, and retain only the locally +significant exceptions identified in the roadmap. Preserve Narratio's actual +application contracts; this is not an architecture redesign or new safety audit. + +Every adopted policy receives its actual Base Policy Version, actual local +adoption date, and an Upstream Source URL with the full reviewed commit and its +own `docs/policy/.md` path. Use current branch URLs only for public +discovery/adoption-guide navigation. Preserve commit-pinned upstream example +links; do not duplicate the example collection locally. No sync checker or +metadata registry is required. + +### Attribution scope and presentation + +Use OSS v1.8's exact House Format. Attribution contains factual entries, not +research notes. Put concrete exception reasoning in OSS policy or an appropriate +decision record. Credit copied policy, agent-orientation, and reading-guide +material accurately, distinguishing adapted portions from unrelated local text. +Keep adapted material CC-BY-SA-4.0 and application code GPL-3.0-or-later. + +Inventory selected modules and shipped runtime material, not every historical +go.sum entry or development executable. Include the test-only go-cmp module as +part of the maintained module inventory with factual scope identification. Do not +include separately installed Audita, Notarius, PromptKit, or other subprocess +applications merely because Narratio invokes them. Include any actual copied +material discovered in the repository. Combine related modules into a component +entry only when their source revisions and notices can all be represented +accurately; do not invent a single revision for independently tagged modules. + +### Final release validation and mechanics + +Keep the current `scripts/release.sh VERSION` interface and source-push-before-tag +workflow. The optional `check-release-candidate.sh VERSION` remains diagnostic; +normal release instructions invoke only publication, which calls the checker once. +No persistent validation receipt/cache or skip-validation escape hatch is added. + +The final checker runs these checks once per invocation: + +| Check | Selected treatment and reason | +| --- | --- | +| Stable version and nonempty matching note/title | Retain existing identity checks. Remove fixed section-heading checks; content and footer are preparation responsibilities, not a new prose parser. | +| `go mod tidy -diff` | Retain: detects an incomplete checked-in module graph without modifying source. Module downloads are preparation, not application-service calls. | +| Tracked Go formatting and diff whitespace | Retain existing inexpensive source hygiene checks; do not add whole-history scans. | +| `go test -race -count=1 ./...` | The single full suite. It includes doccheck and maintained examples. Race instrumentation adds distinct coverage for Narratio's concurrent I/O, cancellation, and shared state. Remove the separate ordinary full-suite invocation. | +| `go vet ./...` | Retain distinct static checks not established by runtime tests. | +| Release asset builder | Retain three existing targets and embedded-version smoke check. Remove redundant `go build ./...`; package tests compile packages and the builder validates deliverables. | +| Extra focused doc/example test invocations | Remove: covered by the full suite. Remain available for development diagnosis. | + +Keep GOWORK=off for owned Go commands. Use explicit `-mod=readonly` on owned +test/vet/build commands, including the asset builder, so incidental vendor trees +are not selected and validation cannot silently edit the module requirements. +Do not overwrite all caller GOFLAGS just to set this option. Keep tidy's own +`-diff` mode. These choices preserve the existing module-based release model. + +Remove the blanket tracked-go.work/go.work.sum rejection: they are inert with +GOWORK=off. Remove the mere-existence vendor-directory rejection. Narrow the +existing all-replacements ban to filesystem replacements: release inputs remain +versioned modules rather than machine-local replacement paths; allow versioned +module replacements. Use Go's module metadata, not regex parsing of replace +blocks. A `go list -m` template inspecting `.Replace.Version` can report local +replacement module names. Do not build a new dependency-policy framework. +This narrows an existing prohibition; supporting repository-local replacement +modules is not introduced by this work. + +Retain exact-candidate guards, origin/main matching, clean tracked/untracked state +as reported by Git, fresh versions, lightweight tags, explicit-ref push, and +builder destination protections. Ignored local files alone do not fail release. +Do not add successful-push network read-back, CI polling, native macOS checks, +new artifacts, or application credential requirements. Cheap mutable publication +state checks before and after validation remain justified. + +## Stage 1: Adopt common policy, provenance, and orientation + +Status: Pending. + +Read the pinned adoption guide, all seven pinned policy templates, the attribution +example for policies, and current local counterparts. Review the roadmap's local +customizations, root AGENTS.md, and docs/development.md. + +1. Adopt agents v1.3, testing v1.3, documentation v1.11, architecture v1.3, + meta v1.0, and OSS v1.8. Preserve upstream common wording and apply only + necessary project customizations. Leave release policy adoption to Stage 5, + when executable behavior will match it. Do not falsely stamp that file now. +2. Preserve Narratio's architecture-specific stage, manifest, storage, credential, + group-workspace, and platform contracts. Retain troubleshooting ownership; + omit unused public API responsibilities. Remove redundant local general-agent + elaborations rather than preserving stylistic variants. +3. Add accurate per-file metadata and adapt upstream agents/AGENTS.md plus the + supplied selective policy-reading guidance. Preserve existing task routes. +4. Create attribution with the canonical policy entry and local CC-BY-SA-4.0 + text. Describe exactly the material adopted so far, including adapted reading + guidance. Add COPYRIGHT's limited document-license scope. Root LICENSE and + the application grant remain unchanged. Do not imply the not-yet-updated + release policy was adopted from this commit. +5. Keep public discovery links in meta and source/example links pinned. While + release policy still has its old headings, make the OSS correction-policy + cross-reference resolve to the existing heading; restore the standard target + when Stage 5 lands. Do not create dead local example links. + +Verification: compare each adopted file to its pinned source, checking that each +remaining substantive deviation is justified. Run doccheck and diff checks. +Record that dependency attribution and release alignment remain incomplete. + +## Stage 2: Attribute core modules and shipped Go material + +Status: Pending. + +Read adopted OSS/documentation policy, go.mod, the actual selected module source +licenses/notices, and the build toolchain declarations. Focus this prompt on +ConfigKit, both YAML module generations, go-cmp, x/sync, x/sys, and Go runtime +material. AWS modules are explicitly deferred to Stage 3. + +1. Resolve selected versions to exact upstream commits using module download + metadata or authoritative upstream Git records. Read licenses, notices, + copyright statements, exceptions, and companion-text requirements at those + commits. Do not rely on today's default branch or copy another project's + inventory without verifying the versions and material. +2. Add factual House Format entries and required standard texts. ConfigKit's + LGPLv3 entry must make both LGPLv3 and GPLv3 texts reachable. Preserve distinct + YAML v3/v4 notices and actual expressions rather than assuming alternatives. +3. Attribute the runtime/toolchain material incorporated in the declared binary + builds using the pinned release-CI toolchain source revision. Identify that + scope factually; do not claim every locally built binary uses that compiler. + Do not add a new toolchain pin or runtime scanner. +4. Inspect repository material for copied snippets/assets requiring attribution, + distinguishing consultation from incorporation. Include verified entries; + record any concrete unresolved material before claiming inventory completion. + +Verification: check statements against selected sources, exact-commit links, +verbatim text copies, companion links, and local doccheck. Do not add generated +SBOM tooling, research prose to attribution, or runtime dependency upgrades. + +## Stage 3: Complete AWS attribution and public navigation + +Status: Pending. + +Read adopted OSS policy and selected AWS SDK/smithy module license material. +Use go.mod plus Go's selected module graph, including indirect AWS modules. + +1. Resolve and document AWS components at their actual selected revisions. + Monorepo modules may share an entry only with accurate module/revision scope + and all necessary notice links. Include smithy separately when its source + and notices differ. Preserve Apache license and relevant NOTICE material. +2. Reconcile the complete attribution document against the module graph, runtime + scope, and copied-material review. Ensure no dependency was skipped merely + because it is indirect, and no external subprocess application was included + solely because it is invoked. Resolve concrete gaps; do not mark them covered + through generic wildcard prose. +3. End README with the two prescribed grant/attribution statements. Keep the + inventory factual and centralized; ensure all applicable local texts exist. + Do not copy unused reference licenses or example collections. + +Verification: review coverage and source identity, run doccheck, compare copied +texts to their authoritative sources, and check the diff for accidental secrets +or unrelated source changes. No new legal-approval or network release gate. + +## Stage 4: Simplify candidate validation and its behavioral tests + +Status: Pending. + +Read scripts/check-release-candidate.sh, scripts/build-release-assets.sh, +scripts/release-lib.sh, and internal/releasecheck/assets_test.go. Read testing, +architecture, OSS, and both current and target release policy. This stage changes +the checker; Stage 5 completes adopted release policy and operator documentation. + +1. Implement the exact selected validation matrix above. Remove fixed release + note sections, duplicate full/focused tests, and redundant generic build. + Retain existing note title/version checks without adding a footer parser. +2. Use the selected module mode/workspace handling in checker and builder. + Replace blanket file/directory guards and replacement parsing as specified. + Preserve output-directory protections and temporary cleanup ownership. +3. Update candidate test fixtures to accept a concise note without the old + headings; missing/empty notes and mismatched titles still fail. Test inert + workspace/vendor presence, local replacement rejection, and versioned + replacement acceptance using representative metadata through a small Go stub. +4. Update the existing owned-command tests to prove the new gate selection and + one invocation of each required suite/build, rather than retaining minimum + command-count assertions. Command observation is appropriate here because + duplicate validation is the specific regression being prevented. Exercise + failure propagation without duplicating every unrelated module-tool behavior. +5. Preserve builder tests for three named assets, version injection, GOWORK=off, + and destination protection. Cover readonly module selection without expanding + into a new cross-platform infrastructure matrix. + +Verification: focused internal/releasecheck tests, shell syntax checks, doccheck, +and diff checks. Do not run the real publication script against origin. Avoid +recursive real full-suite execution from within checker test fixtures. + +## Stage 5: Align publication behavior, policy, and operator procedure + +Status: Pending. + +Read pinned release v1.4, current docs/policy/release.md and docs/release.md, +scripts/release.sh, and internal/releasecheck/publication_test.go. + +1. Adopt release v1.4 verbatim except documented project choices. Record the + selected extra validation checks and their distinct rationale in a concise + local section; do not retain obsolete exception/compensating-control prose + for tag-push completion or authoritative-origin scope. +2. Keep the existing publication interface and pre/post-validation guards. + Fix push-failure reporting so it says publication failed or is unconfirmed, + not definitively unpublished: a failed transport can have an ambiguous + outcome. Keep the local tag for investigation; do not auto-delete or retry. +3. Rewrite the runbook around one publication command. Standalone validation is + optional diagnosis, not a prerequisite. Keep source commit push before tag, + exact candidate identity, current three artifacts/checksums, and lightweight + tagging. No real tag or release is created in this stage. +4. Include the exact three footer links using Narratio Gitea URLs at the intended + release tag. Preparing the footer and checking target paths locally is enough; + do not require the future tag to resolve remotely. Preserve hosted note reuse. +5. Explain ambiguous-push investigation and separate artifact delivery status. + Transient failures can be retried from the immutable tag; already-published + contents/checksums must not be overwritten. Source/content defects need a new + release. Do not add an automated retry subsystem. +6. Update policy attribution material scope for the now-adopted release policy + and the OSS correction-policy link. Update related development guidance so + it does not retain the old mandatory duplicate release sequence. +7. Preserve offline local-bare-remote tests. Add/adjust checks that the checker + is called once, ignored files alone do not block publication, a rejected push + leaves its local tag, and ambiguous failure wording makes no false success or + nonpublication claim. Keep exact-candidate, branch, dirty-state, uniqueness, + upstream-change, and explicit-ref protections. No live forge or CI tests. + +Verification: focused release/doc tests and shell syntax; inspect all current +release guidance for conflicting requirements. Historical release notes stay +unchanged. Adopt metadata only for the actual document revision incorporated. + +## Stage 6: Align CI and contributor validation references + +Status: Pending. + +Read .woodpecker/verify.yml, .woodpecker/release.yml, .woodpecker/shuffle.yml, +the doccheck workflow graph tests, and docs/development.md. + +1. Preserve asynchronous tag CI and its reuse of the candidate checker and + checked-in note. Revalidation on a separate CI runner is independent delivery + evidence, not a second synchronous release prerequisite. No CI-result polling. +2. Keep existing push/PR and scheduled race/shuffle responsibilities; broad CI + optimization is outside this scope. Remove only redundant standalone + documentation/example steps already covered by that workflow's full suite, + repairing depends_on edges and corresponding documentation/tests. Do not + change runner labels, platform targets, or toolchain versions. +3. Keep tag builder/publication dependencies and non-overwrite settings intact. + The tag checker may build temporary validation assets and the subsequent + build step creates upload assets; do not introduce cross-job artifact caching + or a new combined publisher just to remove this delivery separation. +4. Distinguish ordinary developer checks, optional focused diagnosis, and the + normal release entry point in documentation. Do not imply checks removed + from the release sequence are forbidden during development. + +Verification: doccheck workflow graph validation, relevant release tests, and +manual inspection that no execution path waits for CI or publishes broad refs. +No production CI configuration experiments or network publication are authorized. + +## Stage 7: Verify final alignment and close the roadmap + +Status: Pending. + +1. Compare all seven adopted policies against their exact pinned upstream files. + Confirm removed template instructions, correct source/version/date metadata, + verbatim common wording, and justified local content. Check orientation and + ownership, not just file presence. No metadata-only claim of full adoption. +2. Review attribution scope, modification descriptions, public source/notice + commits, license texts, and README/release navigation. Ensure upstream example + collections/guides were linked rather than copied. Do not expand the release + gate set to automate this one-time adoption review. +3. Run the complete race-enabled suite, vet, shell syntax checks, diff checks, + and the existing asset builder with its supported synthetic test version into + a temporary directory. Doccheck and maintained examples run through the suite; + reuse successful checks for unchanged content. Run tidy -diff and formatting + checks as selected above. This is verification, not publication: do not create + a fabricated release note or invoke release.sh on the real repository. +4. Audit test meaning: retained protections still cover candidate identity and + publication safety; removed checks are redundant or incidental; no new + contracts, approvals, provider calls, or live-service tests slipped in. +5. Mark this plan and the feature roadmap implemented with actual evidence and + note release pending. Do not remove unrelated roadmaps or historical audits. + Commit final coherent documentation/test corrections and report clean status. ## Open Questions -None requiring a product decision before implementation. The user settled the -clean cutover and shared provider-named lists. Reports use existing diagnostics -without new content-based success criteria; debug-profile work remains deferred. -Release version selection belongs to the later release procedure. +None requiring a product decision. The gate set and local customizations above +resolve the roadmap's planning choices. If source inspection reveals a concrete +rights question outside the adopted arrangements, stop only the affected item, +record the evidence, recommend a treatment and a viable alternative, and obtain +direction rather than inventing attribution or adding a blanket prohibition. diff --git a/docs/roadmap/policy.md b/docs/roadmap/policy.md index ba9a72b..883616f 100644 --- a/docs/roadmap/policy.md +++ b/docs/roadmap/policy.md @@ -10,7 +10,8 @@ only for substantive, locally significant requirements or decisions and explain their justification. Follow template adoption instructions and customize required project-specific sections rather than copying instructional placeholders. -This document defines the target and scope, not implementation stages. It +This document defines the target and scope; the [implementation plan](implementation.md) +owns sequencing and concrete validation choices. It supersedes the earlier licensing-only roadmap: public upstream discovery, copied-policy attribution, and coordinated release-workflow alignment are now included. The earlier prohibition on policy-project references and deferral of @@ -164,8 +165,8 @@ guidance together so rules and executable behavior agree: - Select additional gates by concrete risk and distinct protection. Do not keep race, static, module-hygiene, formatting, or other checks merely because they exist; document reasons for retained checks. Do not remove useful checks merely - to minimize command count. Resolve the exact set during implementation planning - against these criteria rather than leaving a blanket exemption or expansion. + to minimize command count. The implementation plan records the selected checks + and their distinct justification under these criteria. - Keep offline default tests, required build targets, fresh-version checks, clean candidate/exact source/tag guards, and explicit-ref publication. - Treat ordinary review and roadmap retirement as preparation, not new human