diff --git a/docs/roadmap/audita.md b/docs/roadmap/audita.md index 76f85c9..c042230 100644 --- a/docs/roadmap/audita.md +++ b/docs/roadmap/audita.md @@ -1,4 +1,4 @@ -# Adopt Audita v2 +# Adopt Audita v2 and ConfigKit v0.4.0 Status: agreed scope; implementation has not started. @@ -13,6 +13,9 @@ so operators define each secret once and explicitly share it with the downstream applications that need it. This is a clean cutover without backward-compatibility shims. +Adopt ConfigKit v0.4.0 and use “overlay” consistently for Narratio's selected +pipeline configuration, distinct from downstream LLM model profiles. + Production runs use Audita's default reporting configuration: Narratio passes neither `--debug` nor `--reporting=false`. Normal reports remain available without enabling debug diagnostics. Adding debug reporting through Narratio's testing @@ -73,6 +76,34 @@ duplicate Audita's complete validation logic. ## Configuration and invocation migration +### ConfigKit and pipeline overlays + +Upgrade Narratio's ConfigKit dependency from v0.3.1 to v0.4.0. ConfigKit's +selected-configuration terminology changes from profile to overlay: the Go API +uses `Options.Overlay`, `LayerOverlay`, and `CodeOverlay*`; source layers and +diagnostic codes use `overlay` and `overlay.*`. Composition, precedence, secret +resolution, and lifecycle semantics otherwise remain unchanged. + +Narratio's pipeline-selection interface becomes `--overlay`, selecting +`overlays/.yml` beside the root pipeline file. Remove `--profile` and old +selected-directory discovery without aliases or fallbacks. Preserve existing +base selection for omission, explicit empty selection, and `default`. +Configuration inspection, diff descriptions, internal selection names, and +newly written selection provenance use overlay terminology as well. + +Maintain existing value-based digest and stage-reuse semantics; renaming +informational selection provenance is not a reason to invalidate completed +work or rewrite historical manifests. Update maintained examples and operator +migration guidance for the directory and flag changes. + +This rename does not apply to downstream model profiles, Scriptorium +`profile_id`, Audita's proposal/validation profiles, or Notarius's +`config_profile`. Nor does it prohibit ordinary explicit imports from a directory +named `profiles`. Narratio overlays do not automatically select downstream +overlays or profiles. These are independently owned choices. + +### Audita v2 configuration + Keep binary, timeout, explicit config-path selection, and surviving explicit domain overrides such as transcript description, output schema, and modules. Allow Audita's native configuration discovery when no config path is selected; @@ -81,7 +112,7 @@ do not impose an additional requirement to supply an explicit path. Provide optional Audita overlay, proposal-profile, and validation-profile selectors, passed through to their corresponding CLI flags. Omission leaves Audita's own defaults and validation-profile inheritance in control. These are -Audita selectors, not automatic translations of Narratio profile names. +Audita selectors, not automatic translations of Narratio overlay names. Retire the Narratio settings that mirror removed Audita endpoint/model/capacity flags and work-directory retention. Move their operational meaning into the @@ -212,6 +243,10 @@ Update the canonical [configuration reference](../config.md), [polish internals](../internal/stage-polish.md), maintained examples, and relevant operations/troubleshooting guidance when implementation lands. Cover: +- Moving selected Narratio pipeline files to `overlays/`, replacing `--profile` + with `--overlay`, and distinguishing configuration overlays from model profiles. +- Updating ConfigKit API/source attribution and verifying unchanged effective + values retain existing semantic reuse behavior. - Preparing and validating an Audita v2 configuration and its profiles before switching the executable; copying an old v1 configuration is insufficient. - Moving endpoint, model, capacity, and credential selections to their v2 owners. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 1d10040..3a6e3ed 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -5,10 +5,11 @@ Status: Not started. ## Scope and working instructions Implement [the Audita roadmap](audita.md). The baseline is Narratio commit -`70f1dbf` (v2.2.1), with Audita v2.0.0 as the upstream contract. This replaces -the completed checkpoint-refactoring plan, which remains in Git history. +`70f1dbf` (v2.2.1), with ConfigKit v0.4.0 and Audita v2.0.0 as the target +upstream contracts. This replaces the completed checkpoint-refactoring plan, +which remains in Git history. -Complete Stage 1 through Stage 7 in order, one bounded implementation prompt per +Complete Stage 1 through Stage 8 in order, one bounded implementation prompt per stage for gpt-5.6-terra. Record completion and actual verification results under each stage. These instructions apply to the implementation sprint; the planning pass changes documentation only. Intermediate stages are not release candidates. @@ -29,6 +30,33 @@ code or historical session data. ## Selected design +### ConfigKit and Narratio overlay terminology + +Upgrade ConfigKit from v0.3.1 to v0.4.0 before adapter work. Rename its +`Options.Profile`, `LayerProfile`, and any `CodeProfile*` uses to their overlay +equivalents. Selected pipeline files move from `profiles/.yml` to +`overlays/.yml`; no fallback directory or alias is permitted. + +Rename Narratio's pipeline-selection flag to `--overlay` everywhere it is +accepted, including convenience commands and configuration inspection. Rename +application-owned selection types/helpers, help, errors, and current inspection +labels consistently. `config diff` still takes two positional selections, +now described as overlays. Preserve omission, explicit empty, and `default` +as base selection; do not add a persistent default selector or new precedence. + +Rename newly written informational provenance to overlay terminology, including +`selected_overlay` in manifest configuration provenance. Do not add a dual-field +reader, rewrite historical manifests, or introduce a manifest-version rejection +solely for this informational rename. Existing generic decoding behavior applies +to old optional provenance; it does not become resume authority. Preserve +semantic fingerprints and effective-value digest rules when values are unchanged. + +Only Narratio pipeline configuration selection is renamed. Keep model/profile +fields such as Scriptorium `profile_id`, Audita `proposal_profile` and +`validation_profile`, and Notarius `config_profile` unchanged. Explicit imports +from directories named `profiles` remain legal; only selected pipeline overlay +discovery changes. Do not globally replace the word “profile.” + ### Configuration and credentials - Both `pipeline.audita.credentials` and `pipeline.notarius.credentials` are @@ -47,7 +75,8 @@ code or historical session data. references to `EnvOverrides` and `SensitiveEnvNames`. It checks the existing resolved/nonempty requirement for direct adapter callers but never resolves secrets or rereads environment values. Errors expose no values. Both adapters - use it; no dependency upgrade or credential service is needed. + use it; the ConfigKit upgrade is owned by Stage 1, not this helper. No new + credential service is needed. - Configuration projections and diagnostic metadata contain reference names only. Register all forwarded names for existing captured-stream redaction. Downstream tools remain responsible for files they write themselves. @@ -59,7 +88,7 @@ code or historical session data. `output_schema`, and `modules`. Add optional string selectors `overlay`, `proposal_profile`, and `validation_profile`: empty/unset means no CLI override. Audita owns downstream file/profile validation and native config - discovery. Narratio profile names are not implicitly Audita overlay names. + discovery. Narratio overlay names are not implicitly Audita overlay names. - Preserve nil versus non-nil empty `modules` through loading, projection, cloning, composition, adapter defaults, and request overrides. Nil omits the flag; empty passes `--modules` with an empty argument; nonempty joins in order @@ -133,7 +162,54 @@ hash downstream config/profile contents; operators force polish after edits behind unchanged selections. Notarius credential migration does not change extraction fingerprints, references, force, resume, or chunk-cache behavior. -## Stage 1: Add shared resolved-credential forwarding +## Stage 1: Adopt ConfigKit v0.4.0 and pipeline overlays + +Status: Not started. + +Preparation: ConfigKit v0.4.0 release notes and selected-overlay migration guide +in `../ConfigKit/docs/`; Narratio `internal/config/pipeline_configkit.go`, +`artifact_catalog.go`, `load.go`, `effective_digest.go`; CLI flag parsing and +config commands in `internal/app/`; configuration provenance in +`internal/manifest/manifest.go`; current config/CLI docs and pipeline examples. + +1. Update `go.mod`/`go.sum` to ConfigKit v0.4.0 using normal module tooling; + do not leave a local replacement or change unrelated dependencies. + Replace removed ConfigKit APIs in loading, ownership, and artifact catalog + inspection. Account for source layer `overlay` and diagnostic prefix + `overlay.*` wherever they are consumed or asserted. +2. Rename Narratio's pipeline selector to `--overlay` across all commands and + aliases, load options, pair loading, inspection, provenance, and tests. + Preserve explicit-presence handling through convenience commands and existing + base/default selection. Remove `--profile` rather than retaining an alias. + Apply the selected design's informational manifest-provenance treatment. +3. Move maintained selected pipeline files and generated fixture paths to + `overlays/`. Update current canonical documentation, policy terminology, + examples, and user-visible migration guidance with the directory/flag break. + Do not rewrite historical release notes, rename downstream model profiles, + or prohibit ordinary explicit imports from a directory named `profiles`. +4. Adapt existing config/app tests to prove base and named-overlay selection, + imported defaults plus overrides, secret replacement/removal, source attribution + through character-family expansion, config diff, and convenient-command + forwarding. Add focused checks that the retired flag is rejected and the old + directory is not a discovery fallback. Do not freeze incidental error wording. +5. Verify unchanged effective values retain digest and stage-reuse behavior; + only diagnostic source labels/paths and selected provenance terminology change. + Test loading representative existing manifests without inventing a new + invalidation or rejection gate for old optional selection metadata. +6. Run focused and full tests before proceeding so failures from the dependency + and terminology change are separated from adapter migrations. No Audita or + Notarius credential/CLI behavior changes in this stage. + +Verify: `go test ./internal/config ./internal/app ./internal/manifest`, +`go test ./...`, `go test ./internal/doccheck`, +`go test ./internal/config -run '^TestExamplesLoadAndValidate$'`, and +`git diff --check`. + +Done when Narratio uses ConfigKit v0.4.0, exposes pipeline overlays consistently, +has no old selector/directory fallback, and preserves configuration semantics and +downstream model-profile concepts. + +## Stage 2: Add shared resolved-credential forwarding Status: Not started. @@ -155,7 +231,7 @@ Verify: `go test ./internal/adapters/subprocess` and Done when both adapters can reuse one small conversion mechanism without adding a resolver, provider policy, or ambient-secret inheritance. -## Stage 2: Migrate Notarius to credential lists +## Stage 3: Migrate Notarius to credential lists Status: Not started. @@ -165,10 +241,10 @@ inspection tests; `internal/adapters/notarius/runner.go`, `subprocess.go`; `docs/integrations/notarius.md` and secret handling in `docs/config.md`. 1. Replace the singular field with the typed list across configuration, - projection, request types, stage wiring, and adapter execution. Use Stage 1's + projection, request types, stage wiring, and adapter execution. Use Stage 2's helper and register all selected names for redaction. 2. Update name-only metadata and config-show/diff/effective projections. Test - real ConfigKit loading, profile list replacement, and empty-list clearing. + real ConfigKit loading, overlay list replacement, and empty-list clearing. Preserve bootstrap precedence and existing load-time reference resolution, including references outside the selected stage range. 3. Migrate maintained examples, fixtures, and current Notarius configuration/ @@ -185,7 +261,7 @@ and `go test -race ./internal/adapters/notarius`. Done when Notarius uses credential lists end to end and no active singular credential path remains. -## Stage 3: Cut Audita configuration and adapter over to v2 +## Stage 4: Cut Audita configuration and adapter over to v2 Status: Not started. @@ -197,7 +273,7 @@ defaults, validation, and credential projection. 1. Replace the Audita configuration surface as selected above, updating defaults, validation, safe projection, examples, and config fixtures. Prove nil/empty - modules survive real configuration loading and profile overrides. + modules survive real configuration loading and overlay overrides. 2. Migrate runner construction, selectors, and credential forwarding. Remove retired flags, fixed credential alias, required work-directory plumbing, and old JSON-report validation. Add the typed report bundle. @@ -211,18 +287,19 @@ defaults, validation, and credential projection. nonzero exit after transcript creation. Cover multiple custom credential names and redaction; preserve timeout/cancellation coverage at its existing owner. 5. Make mechanical polish and app-fixture adjustments needed to compile: remove - obsolete request/metadata fields and old report materialization. Stage 4 owns + obsolete request/metadata fields and old report materialization. Stage 5 owns complete diagnostic integration. Remove retired fingerprint members as needed - to compile; Stage 5 completes the replacement semantic contract. These are + to compile; Stage 6 completes the replacement semantic contract. These are bounded call-site changes, not permission to introduce transitional shims. Verify: `go test ./internal/config ./internal/adapters/audita ./internal/stage ./internal/app` and `go test -race ./internal/adapters/audita`. Done when configuration composes a v2-only adapter and subprocess tests prove its -credential/transcript/report boundary. Stages 4 and 5 are required before release. +credential/transcript/report boundary. Stages 5 and 6 complete diagnostic and +semantic integration; all stages are required before release. -## Stage 4: Integrate polish reports with diagnostic lifecycle +## Stage 5: Integrate polish reports with diagnostic lifecycle Status: Not started. @@ -257,7 +334,7 @@ and `go test -race ./internal/stage ./internal/app`. Done when reports use existing diagnostic lifecycle and failed execution cannot promote an incidental transcript. -## Stage 5: Complete semantic evidence and resume behavior +## Stage 6: Complete semantic evidence and resume behavior Status: Not started. @@ -283,7 +360,7 @@ Verify: `go test ./internal/stage ./internal/app` and Done when selections affect reuse accurately without blanket credential/config invalidation or changes to unselected-stage execution. -## Stage 6: Complete operator migration documentation and examples +## Stage 7: Complete operator migration documentation and examples Status: Not started. @@ -317,7 +394,7 @@ Verify: `go test ./internal/doccheck`, Done when operators can migrate using valid examples without obsolete fields, implicit credential aliases, or retired CLI/report expectations. -## Stage 7: Verify assembled behavior and audit scope +## Stage 8: Verify assembled behavior and audit scope Status: Not started.