narratio/docs/roadmap/implementation.md

Notarius Credential Forwarding Implementation Plan

Status: Not started

Implement the feature roadmap in the order below. Each stage is a bounded assignment for a gpt-5.6-terra coding agent. Record a short completion note and executed checks beside each stage's status; do not mark it complete until its acceptance criteria pass. The completed ConfigKit upgrade plan is preserved in Git history.

Instructions and fixed decisions

• Read docs/development.md, all documents under docs/policy/, the feature roadmap, and each stage's listed sources. Inspect callers and existing tests before editing; file lists are starting points, not edit allowlists.
• Implement one optional notarius.llm_api_key field. It contains an environment variable name, not a value. Omission and null mean no credential forwarding. Every surviving non-null reference resolves during ordinary configuration loading, even when extract is disabled, omitted from the range, or resumable.
• Reuse ConfigKit v0.3.0 and invocation-scoped secret bootstrap. Do not upgrade dependencies, modify upstream repositories, add flags, infer provider settings, or add a general environment map. Keep unrelated work out of scope.
• Keep the credential opaque across configuration and stage orchestration: both NotariusConfig.LLMAPIKey and notarius.RunRequest.LLMAPIKey use *configkit.Secret. This is the adapter's explicit typed-credential boundary contract, using only ConfigKit's public root API. It avoids another credential abstraction and avoids introducing raw-secret string fields in stage requests. Only the Notarius adapter calls Value() to construct the child environment.
• A supplied adapter credential must already be resolved. Reject unresolved or empty programmatic references before process launch with a value-free error. Do not resolve again from the process environment or silently drop an invalid credential. Nil remains supported by all existing callers and fakes.
• The adapter forwards EnvName() unchanged through EnvOverrides and includes that name in SensitiveEnvNames. Do not widen the shared inherited environment or hardcode a provider-specific destination variable.
• Neither credential name nor value participates in extract's semantic resume fingerprint. Safe configuration projections include at most the name: renaming can change the effective digest, value rotation cannot. Do not bump the extract fingerprint version or add digest-wide invalidation.
• Never serialize or log a raw subprocess request/environment map. Tests use disposable synthetic credentials and assertions that do not print values on failure. Environment-mutating tests are nonparallel and restore state.
• Retain the roadmap's downstream responsibility boundary: redact captured streams, but do not scan/rewrite arbitrary Notarius files or claim protection against encoded secrets or a malicious executable.
• Each stage must compile and keep existing tests green. Stage 1's configuration support is an intermediate state, not a releasable completed feature.
• No release, tag, push, upstream changes, or CI polling are part of this plan. Preserve published release notes and unrelated roadmaps.

Stage 1 — Add typed configuration and safe projections

Status: Not started

Read: internal/config/config.go, credential_projection.go, effective_projection.go, effective_pipeline.go, effective_digest.go, pipeline_configkit.go, pipeline_normalize.go, validate_notarius.go, notarius_test.go, load_validate_test.go, and pipeline_configkit_test.go. Also read docs/config.md and docs/internal/configuration.md.

1. Add LLMAPIKey *configkit.Secret to NotariusConfig, with yaml:"llm_api_key,omitempty" json:"llm_api_key,omitempty". Leave its default nil. Do not add a new environment lookup, codec, or secret-validation framework; ConfigKit already owns loaded reference syntax and resolution.
2. Add NotariusConfig.MarshalYAML in credential_projection.go, following the existing Audita/S3 pattern: preserve all other Notarius fields and their tags, and project the credential through secretReferenceName. This protects direct YAML serialization as well as effective inspection. JSON uses ConfigKit's safe secret representation. Preserve existing behavior of nil stage sections, reference/output maps, and logical path normalization.
3. Verify that the existing effective projection, schema-derived metadata paths, copy/normalization helpers, source reporting, and digest capture carry the new field correctly. Make only narrowly necessary changes. Do not add another projection or provenance store, and do not mutate or resolve the secret in normalization.
4. Extend package-boundary tests for:
   • omitted/null references; a valid custom reference; missing, empty, and invalid reference names rejected without exposing values;
   • selected profiles replacing or nulling an unavailable base reference, while references in disabled Notarius configuration still resolve;
   • safe effective YAML/JSON and direct pipeline YAML containing the reference name, never its value; nil references retain the previous projection shape;
   • correct root/profile provenance for the new field;
   • unchanged safe digest after rotating a value under the same name, and observable projection/diff identity changes after renaming the reference. Reuse existing credential-projection and profile regression test owners. Avoid asserting the entire configuration or complete diagnostic wording.

Acceptance:

• GOWORK=off go test ./internal/config
• GOWORK=off go test ./...
• Existing examples still load without adding new required credentials.
• No adapter delivery yet, but all serialization paths for the new field are safe.

Stage 2 — Deliver the selected credential and register redaction

Status: Not started

Read: internal/adapters/notarius/runner.go, subprocess.go, fake.go, subprocess_test.go; internal/adapters/subprocess/run.go, diagnostics.go, run_test.go, execution_test.go; internal/stage/extract.go, extract_test.go; and docs/internal/adapters.md, docs/internal/stage-extract.md.

1. Add the optional opaque field to notarius.RunRequest and document that it is an already-resolved reference. Pass notariusConfig.LLMAPIKey when extract constructs the request. Do not add credentials to stage metadata, receipts, output definitions, manifests, reference arguments, or generated files. Keep fakes compatible and avoid dumping credential-bearing requests in tests.
2. Validate any non-nil request credential before subprocess launch. Obtain the name and value only at this adapter boundary; a non-nil unresolved reference is an error, not an instruction to reread the environment.
3. In SubprocessRunner.Run, supply exactly the selected credential override and its sensitive name to the shared subprocess request. Nil adds no override or sensitive name. Preserve arguments, reference ordering, timeout, working directory, log/receipt destinations, process lifecycle, and bundle validation.
4. Extend the extract request test to show the selected resolved reference reaches the adapter without adding raw values to result metadata. Construct resolved test credentials through the real loader, not reflection or private ConfigKit state.
5. Extend adapter tests for nil, resolved custom name, and unresolved reference. Assert custom override/name wiring and unchanged argv without printing the environment map. A caller changing the parent environment after loading must not change the resolved credential delivered in this request.
6. Extend the existing real subprocess test using a controlled executable:
   • check that the selected value arrives under exactly the configured name, and an unrelated synthetic parent credential is absent;
   • report only success/failure of those checks, not an environment dump;
   • emit the selected canary through stdout/stderr in success and nonzero-exit cases, and assert its absence from persisted captured streams and returned errors, with the existing redaction marker present;
   • keep a valid successful bundle/receipt fixture; put any stdout canary in an ignorable JSON field rather than corrupting the expected receipt protocol. Use a custom name outside the built-in sensitive-name list. Verify actual redaction on disk, not only the request's SensitiveEnvNames.
7. Reuse shared subprocess timeout/cancellation and streaming-boundary coverage. Do not duplicate its whole test matrix or add a new redactor. If adapter control flow bypasses that shared path, add a focused regression for it.

Acceptance:

• GOWORK=off go test ./internal/adapters/notarius ./internal/adapters/subprocess ./internal/stage
• GOWORK=off go test -race ./internal/adapters/notarius ./internal/stage
• Delivery is opt-in and uses the loaded value. Known and custom names receive the same captured-stream protection, including subprocess failure diagnostics.

Stage 3 — Verify command lifecycle, inspection, and resume invariants

Status: Not started

Read: internal/app/secrets_bootstrap.go, secrets_bootstrap_test.go, config_commands_test.go, config_loader.go, extract_lifecycle_test.go, semantic_resume_test.go; internal/stage/semantic_contracts_delivery.go, semantic_contracts_delivery_test.go, prepare.go, and their focused tests.

1. Extend existing command credential fixtures with the Notarius reference. Exercise real pipeline loading/bootstrap with a controlled pipeline path: conventional and explicit secret directories, environment precedence, and an existing empty environment variable remaining an error rather than being replaced from a file. Reuse established fixture helpers and avoid a second exhaustive bootstrap suite.
2. Prove public config validation accepts the resolved field and rejects an unavailable one before any adapter execution. Exercise show, sources, and diff with a custom reference name and a synthetic canary: reference names and ownership are visible, raw values are absent, and profile null removal works.
3. Cover prepared/resolved pipeline diagnostics with the Notarius canary using the actual persistence path. Assert that manifests and Narratio-generated configuration contain no credential bytes. Do not assert sanitation of arbitrary files authored directly by the downstream fixture.
4. Extend extract semantic fingerprint and resume tests to cover credential addition/removal, reference-name changes, and value rotation after valid load. The semantic fingerprint and resume eligibility must remain unchanged. Verify a previously successful extraction is reused without invoking Notarius; retain normal selected-reference validation at command loading.
5. Keep effective digest/provenance distinct from semantic resume identity. Do not change the runner's invalidation relation, fingerprint version, or schema to satisfy these tests. No blanket cache key may include credentials.

Acceptance:

• GOWORK=off go test ./internal/app ./internal/config ./internal/stage
• GOWORK=off go test -race ./internal/app ./internal/config ./internal/stage
• Tests do not depend on real system configuration, installed Notarius, or a provider. No secret value is printed even if a non-disclosure assertion fails.
• Credential changes alone do not reexecute a completed extraction or stale dependent artifacts.

Stage 4 — Document the operator contract and validate the whole feature

Status: Not started

Read: docs/config.md, docs/cli.md, docs/integrations/notarius.md, docs/internal/configuration.md, docs/internal/adapters.md, docs/internal/stage-extract.md, docs/troubleshooting.md, examples/README.md, docs/release.md, and the feature roadmap.

1. Update canonical documentation with the field's type, absence/null behavior, environment-name matching with Notarius/PromptKit, existing bootstrap precedence, and the load-time requirement even when extraction is reusable. State the redaction/downstream responsibility boundary and operational-only resume treatment in their appropriate owners; link rather than duplicate. Do not advertise new CLI flags or automatic provider-name discovery.
2. Add a maintained, secret-free example of the optional reference in an appropriate existing Notarius example. Keep ordinary examples runnable without real keys: if a fixture actively declares the reference, its example test must supply and restore a synthetic environment value. Never add real secret files.
3. Preserve immutable release notes, including v2.0.0. Update an existing unreleased note if one exists; otherwise record a concise release-ready summary here for the separately authorized release task. Do not choose/tag a new version merely to complete this stage.
4. Run the repository checks below against normal module files. Inspect the diff for unintended dependency, platform, CLI, schema, and fingerprint changes. Mark the roadmap and all stages complete only after acceptance succeeds. Retain other roadmap and audit files.

Required validation:

GOWORK=off go mod tidy -diff
GOWORK=off go mod verify
GOWORK=off go test -count=1 ./...
GOWORK=off go test -race -count=1 ./...
GOWORK=off go vet ./...
GOWORK=off go build ./...
GOWORK=off go test ./internal/doccheck
GOWORK=off go test ./internal/config -run '^TestExamplesLoadAndValidate$'
git diff --check

Check changed Go files with gofmt -l. Run scripts/build-release-assets.sh v0.0.0-test OUTPUT_DIR with an empty absolute temporary directory created by mktemp -d. All three maintained targets must build; report macOS cross-compilation as such, not native runtime validation. No live model calls, tag publication, or CI monitoring are required.

Open Questions

None. The operator contract, optionality, delivery boundary, redaction scope, and resume policy are settled. Release version selection is a separate task.