diff --git a/docs/cli.md b/docs/cli.md index b90b04e..348c771 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -98,6 +98,8 @@ a session nor create a workspace, manifest, run, lock, adapter, or remote connection. Like every pipeline-loading command, they bootstrap the selected or conventional secrets directory before resolving configuration; `config show` never includes bootstrap reports or secret values in its YAML output. +Typed credential references, including `notarius.llm_api_key`, must therefore +resolve even though these inspection commands never construct an adapter. Campaign selection is optional for a pipeline without party-driven artifact families. A pipeline with `scriptorium.artifact_families` needs a selected or @@ -175,6 +177,9 @@ Behavior: - rejects unknown endpoints and a `--from` endpoint after `--through`; - runs `render` before `extract`; an omitted or disabled Notarius configuration records an explicit `notarius_disabled` self-skip; +- resolves every selected typed credential during the initial configuration + load, including a Notarius credential when extract is disabled, outside the + selected range, or later skipped as resumable; - skips already-succeeded stages unless `--force` is set or a stage-specific resume check finds its durable result obsolete; - applies `--force` only to stages in the selected range; diff --git a/docs/config.md b/docs/config.md index 715af0f..408c073 100644 --- a/docs/config.md +++ b/docs/config.md @@ -227,8 +227,12 @@ inputs: - Do not place raw secrets in YAML. - Credential fields contain environment-variable names, represented internally as ConfigKit typed secret references (for example - `pipeline.audita.llm_api_key: AUDITA_LLM_API_KEY`). A configured reference + `pipeline.notarius.llm_api_key: NOTARIUS_LLM_API_KEY`). A configured reference must resolve to a nonempty environment value during load. +- Every surviving reference is resolved during ordinary pipeline loading, + regardless of whether its integration is disabled, outside the selected + stage range, or eligible for resume. Omit the field or set it to `null` when + that deployment should not require the credential. - Every command that loads a pipeline accepts `--secrets-dir `. When the flag is omitted, Narratio uses a present `secrets/` directory beside the resolved root `pipeline.yml`; an absent conventional directory is ignored. @@ -353,6 +357,7 @@ Rules: | `pipeline.trim.seriatim.report` | bool | No | `false` | | `pipeline.notarius.enabled` | bool | No | `false` | | `pipeline.notarius.binary` | string | No | `notarius` | +| `pipeline.notarius.llm_api_key` | secret reference | No | unset; exact environment-variable name forwarded to Notarius | | `pipeline.notarius.config_path` | string | Conditional | required when enabled; relative paths resolve from the pipeline file directory | | `pipeline.notarius.pipeline_id` | string | Conditional | required when enabled | | `pipeline.notarius.timeout` | duration | No | `3h`; must be positive | @@ -373,6 +378,34 @@ Rules: | `pipeline.scriptorium.artifact_families` | map | No | empty; expands one ordinary artifact per canonical party character | | `pipeline.notification.mode` | string | No | `noop`; the only supported notification mode until a provider is implemented | +### Notarius Credential + +`pipeline.notarius.llm_api_key` optionally names the one environment variable +to forward to the Notarius child process: + +```yaml +notarius: + llm_api_key: OPENROUTER_API_KEY +``` + +The field contains the variable name, not a credential value. That name must +match `api_key_env` in the Notarius/PromptKit profile selected by the deployed +Notarius pipeline; Narratio does not infer or remap provider names. Omission or +`null` forwards nothing. A non-null selected reference must resolve at pipeline +load time through the process environment or existing secrets-directory +bootstrap, even when Notarius is disabled, extract is outside the requested +range, or a completed extraction can be reused. + +The Notarius child receives only Narratio's minimal execution environment plus +this exact selected name and loaded value. Other credentials are not forwarded. +Changing the value, name, presence, or absence does not invalidate extract's +semantic resume evidence. The public name remains part of safe effective +inspection, so renaming it may appear in `config diff` and change the effective +configuration digest; rotating only its value does neither. + +See the [Notarius integration contract](./integrations/notarius.md) for the +delivery and redaction boundary. + ### Notarius Reference Bindings `pipeline.notarius.references` maps a Notarius CLI selector to a prepared diff --git a/docs/integrations/notarius.md b/docs/integrations/notarius.md index d7b28eb..1a4bb06 100644 --- a/docs/integrations/notarius.md +++ b/docs/integrations/notarius.md @@ -75,6 +75,27 @@ It does not pass a Narratio session ID or run `notarius config validate` automatically; the configured working directory and Narratio's minimal child environment apply to the subprocess. +When `pipeline.notarius.llm_api_key` is absent or null, Narratio adds no +credential to that environment. When configured, the field names an environment +variable whose already-loaded nonempty value is forwarded under that exact +name. Operators must use the same name as the `api_key_env` in the selected +Notarius/PromptKit profile. Narratio does not inspect the Notarius configuration +to infer a provider or destination name, and it does not forward unrelated +credentials from its parent environment or secrets directory. + +The selected name is registered with Narratio's shared streaming redactor. +Credential bytes written by the child to captured stdout, captured stderr, or +stderr-derived failure diagnostics are replaced with ``, including +custom names not known to the built-in sensitive-name list. Narratio does not +serialize the credential into arguments, stage metadata, manifests, or +generated configuration. + +This captured-stream protection is not a filesystem sandbox. Notarius remains +responsible for keeping credentials out of files it writes directly, including +bundle artifacts and its own diagnostics. Narratio does not scan or rewrite +arbitrary downstream files and cannot promise redaction of transformed or +encoded credentials. + ## Accepted Result Narratio's supported invocation baseline is Notarius v0.6.0. The accepted diff --git a/docs/internal/adapters.md b/docs/internal/adapters.md index f0b9cdd..2d69462 100644 --- a/docs/internal/adapters.md +++ b/docs/internal/adapters.md @@ -65,6 +65,11 @@ Notarius is composed only when extraction is enabled; the extract stage owns prepared reference resolution, receipt, bundle, and configured-lane policy. The adapter validates the ordered selector/absolute-path pairs and is the sole owner of serializing them as repeated `--reference` arguments before `--json`. +Its request may also carry one already-resolved opaque ConfigKit secret +reference. Only the Notarius adapter reads that value, rejects an unresolved or +empty programmatic reference, and maps the exact configured name/value into the +shared subprocess override and sensitive-name fields. It never rereads the +parent environment. A nil reference preserves the previous invocation. Application-owned object-store construction loads configured filesystem secrets before invoking the invocation's storage factory. Production commands diff --git a/docs/internal/configuration.md b/docs/internal/configuration.md index 98ae142..27d74d8 100644 --- a/docs/internal/configuration.md +++ b/docs/internal/configuration.md @@ -168,6 +168,10 @@ Configuration stores credentials as ConfigKit typed references. A successful load resolves their values inside the opaque typed boundary, while this safe projection serializes only `EnvName()` identities. Credential-value changes are therefore never hashed; reference-name changes remain ordinary provenance. +The optional Notarius credential follows the same path even for disabled or +resumable extraction; normalization copies the opaque reference without reading +or resolving it again. Direct YAML serialization has a configuration-owned safe +projection so only the reference name can leave this boundary. `recomputePipelineEffectiveDigest` is the single package-owned refresh point for later runtime expansion. diff --git a/docs/internal/stage-extract.md b/docs/internal/stage-extract.md index 6a7e12f..2b42622 100644 --- a/docs/internal/stage-extract.md +++ b/docs/internal/stage-extract.md @@ -24,7 +24,8 @@ procedures belong in [Operations](../operations.md). 4. fingerprints the byte- and provenance-bearing Notarius invocation evidence, including sorted reference identities; 5. creates a run-local staging directory and invokes the injected - `notarius.Runner`; + `notarius.Runner`, carrying the optional already-resolved credential + reference without exposing its value; 6. revalidates the reference snapshots, then validates the v2 successful receipt, confined index, management documents, configured required lane descriptors, validation summaries, and regular payload files; @@ -61,6 +62,10 @@ semantic fingerprint. It covers enablement, Notarius pipeline identity, sorted reference selector/source mappings, sorted declared output contracts, and each canonical `narratio.extraction.` output identity. It excludes executable, timeout, working directory, config path, and private Notarius config contents. +Credential presence, reference name, and resolved value are likewise +operational-only and excluded. Ordinary command loading still resolves a +selected reference before resume evaluation, but a credential-only change does +not rerun a valid completed extraction or stale its dependents. `internal/stage/extract_resume.go` then permits a skip only when the existing stage record still matches the current byte- and provenance-bearing invocation diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 7fbe6cd..b1a5af5 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,6 +1,13 @@ # Notarius Credential Forwarding Implementation Plan -Status: In progress — 2026-09-16 +Status: Complete — 2026-09-16 + +Release-ready summary: Narratio now accepts an optional typed +`notarius.llm_api_key` environment reference, resolves it through the existing +command bootstrap, forwards only that selected credential to Notarius, and +redacts its value from captured subprocess streams. Existing configurations +require no migration, and credential-only changes do not invalidate completed +extraction work. Version selection and release publication remain separate. Implement the [feature roadmap](notarius-credential-forwarding.md) in the order below. Each stage is a bounded assignment for a gpt-5.6-terra coding agent. @@ -237,7 +244,28 @@ Acceptance: ## Stage 4 — Document the operator contract and validate the whole feature -Status: Not started +Status: Complete — 2026-09-16 + +Completion: Updated the canonical configuration, CLI, Notarius integration, +internal configuration/adapter/extract, troubleshooting, and example guidance. +The maintained annotated pipeline demonstrates the optional reference with a +synthetic-value test fixture, and published release notes remain unchanged. +The full repository validation and all three maintained asset builds succeeded; +the macOS arm64 result is cross-compilation evidence only. + +Checks: + +- `GOWORK=off go mod tidy -diff` — pass with no changes. +- `GOWORK=off go mod verify` — pass. +- `GOWORK=off go test -count=1 ./...` — pass. +- `GOWORK=off go test -race -count=1 ./...` — pass. +- `GOWORK=off go vet ./...` — pass. +- `GOWORK=off go build ./...` — pass. +- `GOWORK=off go test ./internal/doccheck` — pass. +- `GOWORK=off go test ./internal/config -run '^TestExamplesLoadAndValidate$'` — pass. +- `git diff --check` and `gofmt -l` on changed Go files — pass. +- `scripts/build-release-assets.sh v0.0.0-test ` — pass for + Linux amd64, Linux arm64, and macOS arm64 cross-compilation. Read: `docs/config.md`, `docs/cli.md`, `docs/integrations/notarius.md`, `docs/internal/configuration.md`, `docs/internal/adapters.md`, diff --git a/docs/roadmap/notarius-credential-forwarding.md b/docs/roadmap/notarius-credential-forwarding.md index b1175e5..fa33310 100644 --- a/docs/roadmap/notarius-credential-forwarding.md +++ b/docs/roadmap/notarius-credential-forwarding.md @@ -1,6 +1,6 @@ # Typed Notarius Credential Forwarding -Status: Accepted — implementation pending +Status: Complete — 2026-09-16 ## Summary diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 5f09fac..c220578 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -213,6 +213,40 @@ input validation. Relevant reference: [Operations: Stage Execution and Continuation Behavior](./operations.md#stage-execution-and-continuation-behavior). +## Notarius credential reference unavailable + +Symptom: + +- configuration loading reports `secret.unset`, `secret.empty`, or an invalid + name for `pipeline.notarius.llm_api_key`, including during inspection or a + run that would not execute extraction. + +Likely causes: + +- the named environment variable is absent or empty; +- the selected secrets directory lacks a file with that exact name; +- a process environment variable exists but is empty, so bootstrap correctly + leaves it in place; or +- the Narratio reference name does not match `api_key_env` in the selected + Notarius/PromptKit profile. + +Safe fix: + +- set the exact nonempty environment variable or provide an equivalently named + file through the conventional or explicit secrets directory; +- remove or set the field to `null` when that deployment needs no credential; + and +- use `config show`, `config sources`, and `config diff` to inspect the public + reference name and ownership without exposing its value. + +Narratio resolves selected credential references during ordinary pipeline +loading, before stage selection or resume. Credential changes alone do not +invalidate a completed extraction, but the current reference must still resolve +for the command to proceed. + +Relevant references: [Secrets](./config.md#secrets-handling) and +[Notarius credential](./config.md#notarius-credential). + ## Notarius executable missing Symptom: @@ -361,10 +395,10 @@ Symptom: Likely causes: -- the executable/config path, pipeline ID, timeout, working directory, or - configured output contracts changed; +- the pipeline ID or configured output contracts changed; - a configured prepared reference selector, source, path, checksum, or byte size changed; +- the direct trimmed transcript identity or bytes changed; or - the durable bundle, index, lane set, provenance, regular-file status, or checksum no longer validates. @@ -374,6 +408,11 @@ Safe fix: an unsafe path or bundle symlink error as filesystem corruption or tampering and investigate it rather than replacing files manually. +Executable, config path, timeout, working directory, and Notarius credential +changes are operational and do not by themselves invalidate reusable extraction +evidence. A newly selected credential must nevertheless resolve during command +configuration loading. + ## Notarius transitive configuration changed Symptom: @@ -726,7 +765,7 @@ Diagnostics: ```bash ls -la /path/to/secrets_dir -env | sed 's/=.*//' | grep -E 'OBJECT_STORAGE|AWS|AUDITA|SCRIPTORIUM' +env | sed 's/=.*//' | grep -E 'OBJECT_STORAGE|AWS|AUDITA|NOTARIUS|SCRIPTORIUM' ``` Safe fix: diff --git a/examples/README.md b/examples/README.md index cea4d70..534e565 100644 --- a/examples/README.md +++ b/examples/README.md @@ -67,10 +67,10 @@ with the sample campaign and a compatible local- or S3-audio session. FLAC file before running transcription. The examples contain environment-variable names but no credential values. Set -`OBJECT_STORAGE_KEY_ID`, `OBJECT_STORAGE_KEY`, and `AUDITA_LLM_API_KEY` to -nonempty synthetic or real values before loading the production-shaped and full -annotated pipelines. They use fictional campaign content and reserved example -domains. +`OBJECT_STORAGE_KEY_ID`, `OBJECT_STORAGE_KEY`, `AUDITA_LLM_API_KEY`, and +`NOTARIUS_LLM_API_KEY` to nonempty synthetic or real values before loading the +examples that reference them. They use fictional campaign content and reserved +example domains. For a deployed bundle, keep credential files out of source control in a `secrets/` directory beside the root `pipeline.yml`, one file per referenced diff --git a/examples/pipeline.full.annotated.yml b/examples/pipeline.full.annotated.yml index 466ce10..ad16add 100644 --- a/examples/pipeline.full.annotated.yml +++ b/examples/pipeline.full.annotated.yml @@ -131,6 +131,9 @@ notarius: # Optional structured extraction between trim and render. enabled: true binary: notarius + # Optional environment-variable reference. Match this name to api_key_env in + # the selected Notarius/PromptKit profile; never put the credential here. + llm_api_key: NOTARIUS_LLM_API_KEY config_path: /usr/local/etc/notarius/config.yml pipeline_id: dnd-session timeout: 3h diff --git a/internal/config/load_validate_test.go b/internal/config/load_validate_test.go index 82863c9..41b24d1 100644 --- a/internal/config/load_validate_test.go +++ b/internal/config/load_validate_test.go @@ -266,6 +266,7 @@ func TestExamplesLoadAndValidate(t *testing.T) { t.Setenv("OBJECT_STORAGE_KEY_ID", "synthetic-example-access-key") t.Setenv("OBJECT_STORAGE_KEY", "synthetic-example-secret-key") t.Setenv("AUDITA_LLM_API_KEY", "synthetic-example-audita-key") + t.Setenv("NOTARIUS_LLM_API_KEY", "synthetic-example-notarius-key") examplesDir := filepath.Join("..", "..", "examples") if got, want := maintainedPipelineRoots(t, examplesDir), []string{ "pipeline.extraction-subset.yml", @@ -453,6 +454,7 @@ func TestMaintainedExtractionExamplesPreservePublishedContracts(t *testing.T) { t.Setenv("OBJECT_STORAGE_KEY_ID", "synthetic-example-access-key") t.Setenv("OBJECT_STORAGE_KEY", "synthetic-example-secret-key") t.Setenv("AUDITA_LLM_API_KEY", "synthetic-example-audita-key") + t.Setenv("NOTARIUS_LLM_API_KEY", "synthetic-example-notarius-key") examplesDir := filepath.Join("..", "..", "examples") full, err := LoadPipeline(filepath.Join(examplesDir, "pipeline.full.annotated.yml")) if err != nil {