Adopt Notarius configuration overlays

This commit is contained in:
2026-09-28 03:09:55 +00:00
parent 3de5f9378d
commit 4c114b6319
19 changed files with 140 additions and 104 deletions

View File

@@ -358,7 +358,7 @@ Rules:
| `pipeline.notarius.enabled` | bool | No | `false` | | `pipeline.notarius.enabled` | bool | No | `false` |
| `pipeline.notarius.binary` | string | No | `notarius` | | `pipeline.notarius.binary` | string | No | `notarius` |
| `pipeline.notarius.credentials[]` | list[secret reference] | No | empty; exact environment-variable names forwarded to Notarius | | `pipeline.notarius.credentials[]` | list[secret reference] | No | empty; exact environment-variable names forwarded to Notarius |
| `pipeline.notarius.config_profile` | string | No | unset; exact Notarius ConfigKit profile selector | | `pipeline.notarius.config_overlay` | string | No | unset; exact Notarius ConfigKit overlay selector |
| `pipeline.notarius.config_path` | string | Conditional | required when enabled; relative paths resolve from the pipeline file directory | | `pipeline.notarius.config_path` | string | Conditional | required when enabled; relative paths resolve from the pipeline file directory |
| `pipeline.notarius.pipeline_id` | string | Conditional | required when enabled | | `pipeline.notarius.pipeline_id` | string | Conditional | required when enabled |
| `pipeline.notarius.timeout` | duration | No | `3h`; must be positive | | `pipeline.notarius.timeout` | duration | No | `3h`; must be positive |
@@ -407,22 +407,33 @@ or `[]` forwards nothing. A Narratio overlay replaces the entire list, and
`[]` clears it. See the [Audita integration contract](./integrations/audita.md) `[]` clears it. See the [Audita integration contract](./integrations/audita.md)
for the subprocess and redaction boundary. for the subprocess and redaction boundary.
### Notarius Configuration Profile <a id="notarius-configuration-profile"></a>
`pipeline.notarius.config_profile` optionally selects a Notarius ConfigKit ### Notarius Configuration Overlay
configuration profile when extraction executes:
Migration: rename `pipeline.notarius.config_profile` to `config_overlay`.
The old field is rejected; there is no compatibility alias. For Notarius v0.8+
move selected downstream files from `profiles/` to `overlays/` beside its root
configuration. This does not rename PromptKit LLM profiles.
The renamed selector also changes extraction resume evidence for named
selections: their existing extraction is regenerated on the next applicable
run, with normal dependent invalidation. Omitted/base selections do not acquire
a new invalidation solely from this rename.
`pipeline.notarius.config_overlay` optionally selects a Notarius ConfigKit
configuration overlay when extraction executes:
```yaml ```yaml
notarius: notarius:
config_profile: downstream-economy config_overlay: downstream-economy
``` ```
When the field is omitted or `null`, Narratio supplies no downstream profile When the field is omitted or `null`, Narratio supplies no downstream overlay
selector and preserves Notarius's existing configuration-selection behavior. selector and preserves Notarius's existing configuration-selection behavior.
When present, Narratio forwards the exact string to Notarius, including an When present, Narratio forwards the exact string to Notarius, including an
empty string or `default`, which explicitly select Notarius's composed base empty string or `default`, which explicitly select Notarius's composed base
configuration. Notarius owns selector interpretation and validation; Narratio configuration. Notarius owns selector interpretation and validation; Narratio
does not restrict profile names, resolve profile files, or rewrite the value. does not restrict overlay names, resolve overlay files, or rewrite the value.
This selection is independent of Narratio's own `--overlay` selection. A This selection is independent of Narratio's own `--overlay` selection. A
Narratio overlay may explicitly select a differently named Notarius overlay, Narratio overlay may explicitly select a differently named Notarius overlay,

View File

@@ -7,14 +7,14 @@ lanes from the final trimmed Seriatim transcript. Narratio owns invocation,
safe bundle discovery, lane selection, and its own artifact metadata. Notarius safe bundle discovery, lane selection, and its own artifact metadata. Notarius
owns pipeline definitions, lane schemas, the receipt, and bundle formats. owns pipeline definitions, lane schemas, the receipt, and bundle formats.
Canonical Notarius v0.7.0 references: Canonical Notarius v0.8.1 references:
- [CLI reference](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.7.0/docs/cli.md) - [CLI reference](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.8.1/docs/cli.md)
- [Subprocess consumer contract](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.7.0/docs/consumers/subprocess.md) - [Subprocess consumer contract](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.8.1/docs/consumers/subprocess.md)
- [D&D pipeline and lane contracts](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.7.0/docs/consumers/dnd-pipeline.md) - [D&D pipeline and lane contracts](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.8.1/docs/consumers/dnd-pipeline.md)
- [Run-result receipt](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.7.0/docs/integrations/run-result.md) - [Run-result receipt](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.8.1/docs/integrations/run-result.md)
- [JSON output bundle](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.7.0/docs/integrations/json-output.md) - [JSON output bundle](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.8.1/docs/integrations/json-output.md)
- [D&D spell-catalog overlay](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.7.0/docs/integrations/dnd-spell-catalog-overlays.md) - [D&D spell-catalog overlay](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.8.1/docs/integrations/dnd-spell-catalog-overlays.md)
The [complete Narratio example](../../examples/pipeline.full.annotated.yml) The [complete Narratio example](../../examples/pipeline.full.annotated.yml)
records the exact current constraints for all ten D&D lanes. Treat the linked records the exact current constraints for all ten D&D lanes. Treat the linked
@@ -27,17 +27,17 @@ When `pipeline.notarius.enabled` is true, Narratio resolves the executable,
configuration path, input path, output directory, and working directory to configuration path, input path, output directory, and working directory to
absolute paths. Configured references require support for the `--reference` absolute paths. Configured references require support for the `--reference`
contract introduced in Notarius v0.6.0. Cache controls require support for the contract introduced in Notarius v0.6.0. Cache controls require support for the
selected `--resume` and `--chunk_cache` flags. Notarius v0.7.0 is the tested selected `--resume` and `--chunk_cache` flags. Notarius v0.8.1 is the tested
cache integration baseline, not an unconditional minimum version; Narratio cache integration baseline, not an unconditional minimum version; Narratio
does not impose a runtime version gate. Each reference binding is a separate does not impose a runtime version gate. Each reference binding is a separate
argument before `--json`. When `pipeline.notarius.config_profile` is present, argument before `--json`. When `pipeline.notarius.config_overlay` is present,
its exact value is another argument pair before `--json`: its exact value is another argument pair before `--json`:
```text ```text
notarius run <pipeline_id> --config <config_path> --input <trimmed_json> --output-dir <staging_dir> [--resume] [--chunk_cache <mode>] [--reference <selector>=<verified_snapshot_path>]... [--config-profile <name>] --json notarius run <pipeline_id> --config <config_path> --input <trimmed_json> --output-dir <staging_dir> [--resume] [--chunk_cache <mode>] [--reference <selector>=<verified_snapshot_path>]... [--config-overlay <name>] --json
``` ```
Notarius v0.7.0 is a known supporting version for `--config-profile`; using the Notarius v0.8.1 is a known supporting version for `--config-overlay`; using the
option requires a compatible Notarius executable, but deployments that omit the option requires a compatible Notarius executable, but deployments that omit the
field acquire no additional version requirement. Omitted or null configuration field acquire no additional version requirement. Omitted or null configuration
adds no flag. Supplied empty, `default`, and named selectors are forwarded adds no flag. Supplied empty, `default`, and named selectors are forwarded
@@ -46,7 +46,7 @@ option selects a Notarius configuration overlay, not a PromptKit profile through
`--llm-profile`, and Narratio never substitutes its own selected overlay name. `--llm-profile`, and Narratio never substitutes its own selected overlay name.
A missing or invalid downstream overlay is an ordinary Notarius subprocess A missing or invalid downstream overlay is an ordinary Notarius subprocess
failure: Narratio does not retry without the selector or fall back to another failure: Narratio does not retry without the selector or fall back to another
profile. overlay.
Reference paths are absolute stable snapshots streamed from manifest-verified Reference paths are absolute stable snapshots streamed from manifest-verified
canonical files prepared inside the current Narratio session workspace. They canonical files prepared inside the current Narratio session workspace. They
@@ -129,7 +129,7 @@ encoded credentials.
## Accepted Result ## Accepted Result
Narratio's tested invocation baseline is Notarius v0.7.0. The accepted Narratio's tested invocation baseline is Notarius v0.8.1. The accepted
receipt remains `notarius.run-result.v2`; reference flags do not change the receipt remains `notarius.run-result.v2`; reference flags do not change the
receipt or ten-lane output contract. The receipt receipt or ten-lane output contract. The receipt
must identify the configured pipeline, and its `index_file` must be exactly must identify the configured pipeline, and its `index_file` must be exactly
@@ -188,7 +188,7 @@ for failure recovery.
## Opt-in Integration Test ## Opt-in Integration Test
The production-boundary cache test requires an absolute path to a locally The production-boundary cache test requires an absolute path to a locally
built Notarius v0.7.0-compatible executable. It uses only a local built Notarius v0.8.1-compatible executable. It uses only a local
OpenAI-compatible `httptest` provider and temporary cache/config/output roots: OpenAI-compatible `httptest` provider and temporary cache/config/output roots:
```sh ```sh

View File

@@ -24,11 +24,11 @@ procedures belong in [Operations](../operations.md).
and rejects any source change observed while copying; and rejects any source change observed while copying;
4. fingerprints the byte- and provenance-bearing Notarius invocation evidence, 4. fingerprints the byte- and provenance-bearing Notarius invocation evidence,
including sorted reference identities and the explicit downstream including sorted reference identities and the explicit downstream
configuration-profile selection; configuration-overlay selection;
5. creates a run-local staging directory and invokes the injected 5. creates a run-local staging directory and invokes the injected
`notarius.Runner`, carrying the optional already-resolved credential `notarius.Runner`, carrying the optional already-resolved credential
reference without exposing its value, the optional exact configuration reference without exposing its value, the optional exact configuration
profile, plus effective resume and chunk-cache controls after force overlay, plus effective resume and chunk-cache controls after force
precedence; precedence;
6. revalidates the reference snapshots, then validates the v2 successful 6. revalidates the reference snapshots, then validates the v2 successful
receipt, confined index, management documents, configured required lane receipt, confined index, management documents, configured required lane
@@ -73,7 +73,7 @@ Before the focused validator runs, the application compares extract's versioned
semantic fingerprint. It covers enablement, Notarius pipeline identity, sorted semantic fingerprint. It covers enablement, Notarius pipeline identity, sorted
reference selector/source mappings, sorted declared output contracts, and each reference selector/source mappings, sorted declared output contracts, and each
canonical `narratio.extraction.<key>` output identity. It also covers the canonical `narratio.extraction.<key>` output identity. It also covers the
explicit Notarius configuration-profile selection; omitted, empty, and exact explicit Notarius configuration-overlay selection; omitted, empty, and exact
`default` selectors share the base-selection identity. It excludes executable, `default` selectors share the base-selection identity. It excludes executable,
timeout, working directory, config path, and private Notarius config contents. timeout, working directory, config path, and private Notarius config contents.
Credential presence, reference name, and resolved value are likewise Credential presence, reference name, and resolved value are likewise
@@ -85,7 +85,7 @@ not rerun a valid completed extraction or stale its dependents.
stage record still matches the current byte- and provenance-bearing invocation stage record still matches the current byte- and provenance-bearing invocation
evidence. That evidence covers the current direct trimmed-transcript identity, evidence. That evidence covers the current direct trimmed-transcript identity,
sorted prepared-reference identities, pipeline identity, and configured output sorted prepared-reference identities, pipeline identity, and configured output
contracts, plus the same downstream profile-selection identity. The same contracts, plus the same downstream overlay-selection identity. The same
reference helper and transcript identity are resolved again for artifact reference helper and transcript identity are resolved again for artifact
evidence, so changing current transcript bytes, reference bytes, producer evidence, so changing current transcript bytes, reference bytes, producer
identity, or a named downstream selection makes the prior extraction obsolete. identity, or a named downstream selection makes the prior extraction obsolete.
@@ -107,7 +107,7 @@ obsolete results are non-resumable and run again; unsafe filesystem conditions
return an error rather than silently accepting or replacing data. return an error rather than silently accepting or replacing data.
This sensitivity is limited to the explicit downstream selector. Narratio's This sensitivity is limited to the explicit downstream selector. Narratio's
own profile label and whole effective-configuration digest are not blanket own overlay label and whole effective-configuration digest are not blanket
extraction invalidators. Neither contract can observe files imported by extraction invalidators. Neither contract can observe files imported by
Notarius configuration, profile contents, prompt/module definitions, or other Notarius configuration, profile contents, prompt/module definitions, or other
transitive inputs. Operators must force extraction after changing any such transitive inputs. Operators must force extraction after changing any such
@@ -144,5 +144,5 @@ recovery.
`internal/artifacts/extraction_catalog_test.go` `internal/artifacts/extraction_catalog_test.go`
- Composition and downstream behavior: `internal/app/runner_test.go`, - Composition and downstream behavior: `internal/app/runner_test.go`,
`internal/stage/analyze_test.go`, `internal/stage/publish_test.go` `internal/stage/analyze_test.go`, `internal/stage/publish_test.go`
- Opt-in Notarius v0.7.0 production-boundary coverage: - Opt-in Notarius v0.8.1 production-boundary coverage:
`internal/stage/notarius_cache_integration_test.go` `internal/stage/notarius_cache_integration_test.go`

View File

@@ -282,14 +282,21 @@ earlier checkpoints are reused and execution resumes with failed, stale, or
missing work in dependency order. `--force` is unnecessary unless the operator missing work in dependency order. `--force` is unnecessary unless the operator
intends to replace otherwise-current artifacts. intends to replace otherwise-current artifacts.
### Developing With A Notarius Configuration Profile ### Developing With A Notarius Configuration Overlay
Notarius v0.8+ defaults to two additional eligible transient provider attempts
per generation. Review downstream retry settings and Narratio's extraction
timeout together; retries can increase elapsed time and provider charges.
Notarius v0.8.1 refreshes its built-in profile checkpoint identity, so older
checkpoints may not be reused. Source-keyed chunk plans and already published
outputs are unaffected; no manual cache deletion is required.
A Narratio development overlay can explicitly select a Notarius overlay without A Narratio development overlay can explicitly select a Notarius overlay without
requiring the names to match. For example, `overlays/testing.yml` may contain: requiring the names to match. For example, `overlays/testing.yml` may contain:
```yaml ```yaml
notarius: notarius:
config_profile: downstream-economy config_overlay: downstream-economy
``` ```
Selecting Narratio's `testing` overlay then supplies `downstream-economy` only Selecting Narratio's `testing` overlay then supplies `downstream-economy` only
@@ -302,7 +309,7 @@ Narratio cannot observe edits inside Notarius configuration behind an unchanged
selector. Force extraction after such an edit when a replacement is intended. selector. Force extraction after such an edit when a replacement is intended.
Use `regenerate-artifacts` when both extraction and analysis must be rebuilt; Use `regenerate-artifacts` when both extraction and analysis must be rebuilt;
use `analyze` when the existing extraction bundle should be preserved. Field use `analyze` when the existing extraction bundle should be preserved. Field
semantics belong in [Configuration](./config.md#notarius-configuration-profile), semantics belong in [Configuration](./config.md#notarius-configuration-overlay),
and the exact downstream invocation belongs in the and the exact downstream invocation belongs in the
[Notarius integration contract](./integrations/notarius.md#invocation). [Notarius integration contract](./integrations/notarius.md#invocation).

View File

@@ -133,7 +133,7 @@ notarius:
# Optionally select a Notarius ConfigKit overlay when extraction runs. This # Optionally select a Notarius ConfigKit overlay when extraction runs. This
# is independent of Narratio's selected overlay and may use a different name. # is independent of Narratio's selected overlay and may use a different name.
# See docs/config.md and docs/integrations/notarius.md for the contracts. # See docs/config.md and docs/integrations/notarius.md for the contracts.
# config_profile: downstream-economy # config_overlay: downstream-economy
config_path: /usr/local/etc/notarius/config.yml config_path: /usr/local/etc/notarius/config.yml
pipeline_id: dnd-session pipeline_id: dnd-session
timeout: 3h timeout: 3h

View File

@@ -20,9 +20,9 @@ func (f *FakeRunner) Run(ctx context.Context, req RunRequest) (RunResult, error)
} }
copyRequest := req copyRequest := req
copyRequest.Credentials = append([]configkit.Secret(nil), req.Credentials...) copyRequest.Credentials = append([]configkit.Secret(nil), req.Credentials...)
if req.ConfigProfile != nil { if req.ConfigOverlay != nil {
profile := *req.ConfigProfile profile := *req.ConfigOverlay
copyRequest.ConfigProfile = &profile copyRequest.ConfigOverlay = &profile
} }
copyRequest.References = append([]ReferenceBinding(nil), req.References...) copyRequest.References = append([]ReferenceBinding(nil), req.References...)
f.Requests = append(f.Requests, copyRequest) f.Requests = append(f.Requests, copyRequest)

View File

@@ -27,7 +27,7 @@ type RunRequest struct {
Binary string Binary string
// Credentials are already-resolved environment references selected for Notarius. // Credentials are already-resolved environment references selected for Notarius.
Credentials []configkit.Secret Credentials []configkit.Secret
ConfigProfile *string ConfigOverlay *string
ConfigPath string ConfigPath string
PipelineID string PipelineID string
InputPath string InputPath string

View File

@@ -72,8 +72,8 @@ func (r *SubprocessRunner) Run(ctx context.Context, req RunRequest) (RunResult,
for _, reference := range references { for _, reference := range references {
args = append(args, "--reference", reference.Selector+"="+reference.Path) args = append(args, "--reference", reference.Selector+"="+reference.Path)
} }
if req.ConfigProfile != nil { if req.ConfigOverlay != nil {
args = append(args, "--config-profile", *req.ConfigProfile) args = append(args, "--config-overlay", *req.ConfigOverlay)
} }
args = append(args, "--json") args = append(args, "--json")
processResult, err := r.run(ctx, subprocess.RunRequest{ processResult, err := r.run(ctx, subprocess.RunRequest{

View File

@@ -118,7 +118,7 @@ func TestSubprocessRunnerMapsCacheControlsToArguments(t *testing.T) {
} }
} }
func TestSubprocessRunnerMapsConfigProfileToArgumentPair(t *testing.T) { func TestSubprocessRunnerMapsConfigOverlayToArgumentPair(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
supplied bool supplied bool
@@ -133,7 +133,7 @@ func TestSubprocessRunnerMapsConfigProfileToArgumentPair(t *testing.T) {
t.Run(test.name, func(t *testing.T) { t.Run(test.name, func(t *testing.T) {
req := validRunRequest(t) req := validRunRequest(t)
if test.supplied { if test.supplied {
req.ConfigProfile = &test.value req.ConfigOverlay = &test.value
} }
var captured sharedsubprocess.RunRequest var captured sharedsubprocess.RunRequest
runner := &SubprocessRunner{run: func(_ context.Context, processReq sharedsubprocess.RunRequest) (sharedsubprocess.RunResult, error) { runner := &SubprocessRunner{run: func(_ context.Context, processReq sharedsubprocess.RunRequest) (sharedsubprocess.RunResult, error) {
@@ -147,19 +147,19 @@ func TestSubprocessRunnerMapsConfigProfileToArgumentPair(t *testing.T) {
} }
flagIndex := -1 flagIndex := -1
for index, arg := range captured.Args { for index, arg := range captured.Args {
if arg == "--config-profile" { if arg == "--config-overlay" {
flagIndex = index flagIndex = index
break break
} }
} }
if !test.supplied { if !test.supplied {
if flagIndex >= 0 { if flagIndex >= 0 {
t.Fatalf("subprocess args unexpectedly contain config profile: %#v", captured.Args) t.Fatalf("subprocess args unexpectedly contain config overlay: %#v", captured.Args)
} }
return return
} }
if flagIndex < 0 || flagIndex+2 >= len(captured.Args) || captured.Args[flagIndex+1] != test.value || captured.Args[flagIndex+2] != "--json" { if flagIndex < 0 || flagIndex+2 >= len(captured.Args) || captured.Args[flagIndex+1] != test.value || captured.Args[flagIndex+2] != "--json" {
t.Fatalf("config profile argument boundary = %#v, want flag, exact value, then --json", captured.Args) t.Fatalf("config overlay argument boundary = %#v, want flag, exact value, then --json", captured.Args)
} }
}) })
} }
@@ -455,7 +455,7 @@ func TestSubprocessRunnerReturnsProcessFailuresWithoutParsingStdout(t *testing.T
req := validRunRequest(t) req := validRunRequest(t)
if test.name == "nonzero" { if test.name == "nonzero" {
profile := "missing-profile" profile := "missing-profile"
req.ConfigProfile = &profile req.ConfigOverlay = &profile
} }
req.Binary = writeShellScript(t, "#!/bin/sh\n"+test.scriptBody) req.Binary = writeShellScript(t, "#!/bin/sh\n"+test.scriptBody)
req.Timeout = test.timeout req.Timeout = test.timeout
@@ -754,7 +754,7 @@ func TestLoadDiagnosticSummariesValidateBoundsAndTolerateUnknownFields(t *testin
func TestFakeRunnerCapturesRequestsAndHonorsContextAndError(t *testing.T) { func TestFakeRunnerCapturesRequestsAndHonorsContextAndError(t *testing.T) {
profile := "testing" profile := "testing"
req := RunRequest{PipelineID: "pipeline", ConfigProfile: &profile, References: []ReferenceBinding{{Selector: "party", Path: "/references/party.json"}}} req := RunRequest{PipelineID: "pipeline", ConfigOverlay: &profile, References: []ReferenceBinding{{Selector: "party", Path: "/references/party.json"}}}
want := RunResult{BundleRoot: "/bundle"} want := RunResult{BundleRoot: "/bundle"}
fake := &FakeRunner{Result: want} fake := &FakeRunner{Result: want}
got, err := fake.Run(context.Background(), req) got, err := fake.Run(context.Background(), req)
@@ -766,8 +766,8 @@ func TestFakeRunnerCapturesRequestsAndHonorsContextAndError(t *testing.T) {
t.Fatalf("fake retained aliased request references: %#v", fake.Requests[0].References) t.Fatalf("fake retained aliased request references: %#v", fake.Requests[0].References)
} }
profile = "changed" profile = "changed"
if *fake.Requests[0].ConfigProfile != "testing" { if *fake.Requests[0].ConfigOverlay != "testing" {
t.Fatalf("fake retained aliased config profile: %q", *fake.Requests[0].ConfigProfile) t.Fatalf("fake retained aliased config overlay: %q", *fake.Requests[0].ConfigOverlay)
} }
wantErr := errors.New("configured failure") wantErr := errors.New("configured failure")

View File

@@ -691,17 +691,17 @@ func TestExtractLifecycleCredentialChangesRemainResumable(t *testing.T) {
} }
} }
func TestExtractLifecycleConfigProfileChangesRerunExtractionAndDependents(t *testing.T) { func TestExtractLifecycleConfigOverlayChangesRerunExtractionAndDependents(t *testing.T) {
cfg, env, runner := extractionLifecycleFixture(t, true) cfg, env, runner := extractionLifecycleFixture(t, true)
selected := "inexpensive" selected := "inexpensive"
cfg.Pipeline.Notarius.ConfigProfile = &selected cfg.Pipeline.Notarius.ConfigOverlay = &selected
analyzeRuns := 0 analyzeRuns := 0
plan := extractionLifecyclePlan(t, &analyzeRuns) plan := extractionLifecyclePlan(t, &analyzeRuns)
if _, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env}); err != nil { if _, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env}); err != nil {
t.Fatalf("initial executeStages() error = %v", err) t.Fatalf("initial executeStages() error = %v", err)
} }
if len(runner.requests) != 1 || runner.requests[0].ConfigProfile == nil || *runner.requests[0].ConfigProfile != selected || analyzeRuns != 1 { if len(runner.requests) != 1 || runner.requests[0].ConfigOverlay == nil || *runner.requests[0].ConfigOverlay != selected || analyzeRuns != 1 {
t.Fatalf("initial requests=%#v analyze=%d, want selected extraction and analysis", runner.requests, analyzeRuns) t.Fatalf("initial requests=%#v analyze=%d, want selected extraction and analysis", runner.requests, analyzeRuns)
} }
reused, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env}) reused, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env})
@@ -721,12 +721,12 @@ func TestExtractLifecycleConfigProfileChangesRerunExtractionAndDependents(t *tes
} }
changed := "experimental" changed := "experimental"
cfg.Pipeline.Notarius.ConfigProfile = &changed cfg.Pipeline.Notarius.ConfigOverlay = &changed
rerun, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env}) rerun, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env})
if err != nil { if err != nil {
t.Fatalf("changed-profile executeStages() error = %v", err) t.Fatalf("changed-profile executeStages() error = %v", err)
} }
if len(rerun.Executed) != 2 || len(runner.requests) != 2 || runner.requests[1].ConfigProfile == nil || *runner.requests[1].ConfigProfile != changed || analyzeRuns != 2 { if len(rerun.Executed) != 2 || len(runner.requests) != 2 || runner.requests[1].ConfigOverlay == nil || *runner.requests[1].ConfigOverlay != changed || analyzeRuns != 2 {
t.Fatalf("changed-profile summary=%#v requests=%#v analyze=%d", rerun, runner.requests, analyzeRuns) t.Fatalf("changed-profile summary=%#v requests=%#v analyze=%d", rerun, runner.requests, analyzeRuns)
} }
after := loadLifecycleManifest(t, cfg) after := loadLifecycleManifest(t, cfg)
@@ -741,12 +741,12 @@ func TestExtractLifecycleConfigProfileChangesRerunExtractionAndDependents(t *tes
t.Fatalf("changed-profile lifecycle render=%#v analyze=%#v publish=%#v", after.Stages["render"], after.Stages["analyze"], after.Stages["publish"]) t.Fatalf("changed-profile lifecycle render=%#v analyze=%#v publish=%#v", after.Stages["render"], after.Stages["analyze"], after.Stages["publish"])
} }
cfg.Pipeline.Notarius.ConfigProfile = nil cfg.Pipeline.Notarius.ConfigOverlay = nil
returned, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env}) returned, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env})
if err != nil { if err != nil {
t.Fatalf("return-to-base executeStages() error = %v", err) t.Fatalf("return-to-base executeStages() error = %v", err)
} }
if len(returned.Executed) != 2 || len(runner.requests) != 3 || runner.requests[2].ConfigProfile != nil || analyzeRuns != 3 { if len(returned.Executed) != 2 || len(runner.requests) != 3 || runner.requests[2].ConfigOverlay != nil || analyzeRuns != 3 {
t.Fatalf("return-to-base summary=%#v requests=%#v analyze=%d", returned, runner.requests, analyzeRuns) t.Fatalf("return-to-base summary=%#v requests=%#v analyze=%d", returned, runner.requests, analyzeRuns)
} }
final := loadLifecycleManifest(t, cfg) final := loadLifecycleManifest(t, cfg)
@@ -755,10 +755,10 @@ func TestExtractLifecycleConfigProfileChangesRerunExtractionAndDependents(t *tes
} }
} }
func TestExtractLifecycleConfigProfileFailureClearsCurrentResultAndDependents(t *testing.T) { func TestExtractLifecycleConfigOverlayFailureClearsCurrentResultAndDependents(t *testing.T) {
cfg, env, runner := extractionLifecycleFixture(t, true) cfg, env, runner := extractionLifecycleFixture(t, true)
selected := "inexpensive" selected := "inexpensive"
cfg.Pipeline.Notarius.ConfigProfile = &selected cfg.Pipeline.Notarius.ConfigOverlay = &selected
analyzeRuns := 0 analyzeRuns := 0
plan := extractionLifecyclePlan(t, &analyzeRuns) plan := extractionLifecyclePlan(t, &analyzeRuns)
if _, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env}); err != nil { if _, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env}); err != nil {
@@ -768,7 +768,7 @@ func TestExtractLifecycleConfigProfileFailureClearsCurrentResultAndDependents(t
markLifecycleStageSucceeded(t, cfg, "publish") markLifecycleStageSucceeded(t, cfg, "publish")
changed := "missing-profile" changed := "missing-profile"
cfg.Pipeline.Notarius.ConfigProfile = &changed cfg.Pipeline.Notarius.ConfigOverlay = &changed
runner.failuresRemaining = 1 runner.failuresRemaining = 1
if _, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env}); err == nil { if _, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env}); err == nil {
t.Fatal("changed-profile executeStages() error = nil, want downstream failure") t.Fatal("changed-profile executeStages() error = nil, want downstream failure")
@@ -783,12 +783,12 @@ func TestExtractLifecycleConfigProfileFailureClearsCurrentResultAndDependents(t
t.Fatalf("failed changed-profile lifecycle trim=%#v render=%#v analyze=%#v publish=%#v analyzeRuns=%d", t.Fatalf("failed changed-profile lifecycle trim=%#v render=%#v analyze=%#v publish=%#v analyzeRuns=%d",
failed.Stages["trim"], failed.Stages["render"], failed.Stages["analyze"], failed.Stages["publish"], analyzeRuns) failed.Stages["trim"], failed.Stages["render"], failed.Stages["analyze"], failed.Stages["publish"], analyzeRuns)
} }
if len(runner.requests) != 2 || runner.requests[1].ConfigProfile == nil || *runner.requests[1].ConfigProfile != changed { if len(runner.requests) != 2 || runner.requests[1].ConfigOverlay == nil || *runner.requests[1].ConfigOverlay != changed {
t.Fatalf("failed request = %#v, want changed selector", runner.requests) t.Fatalf("failed request = %#v, want changed selector", runner.requests)
} }
} }
func TestExtractLifecycleRealConfigProfilesSelectNotariusExplicitly(t *testing.T) { func TestExtractLifecycleRealConfigOverlaysSelectNotariusExplicitly(t *testing.T) {
seed, env, runner := extractionLifecycleFixture(t, true) seed, env, runner := extractionLifecycleFixture(t, true)
workspace := seed.Pipeline.Workspace.Root workspace := seed.Pipeline.Workspace.Root
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspace) pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspace)
@@ -822,7 +822,7 @@ func TestExtractLifecycleRealConfigProfilesSelectNotariusExplicitly(t *testing.T
t.Fatal(err) t.Fatal(err)
} }
for name, contents := range map[string]string{ for name, contents := range map[string]string{
"testing": "whisperx:\n language: fr\nnotarius:\n config_profile: downstream-economy\n", "testing": "whisperx:\n language: fr\nnotarius:\n config_overlay: downstream-economy\n",
"narratio-only": "whisperx:\n language: de\n", "narratio-only": "whisperx:\n language: de\n",
"narratio-only-two": "whisperx:\n language: es\n", "narratio-only-two": "whisperx:\n language: es\n",
} { } {
@@ -848,7 +848,7 @@ func TestExtractLifecycleRealConfigProfilesSelectNotariusExplicitly(t *testing.T
if _, err := executeStages(context.Background(), testingConfig, plan, RunOptions{Env: env}); err != nil { if _, err := executeStages(context.Background(), testingConfig, plan, RunOptions{Env: env}); err != nil {
t.Fatalf("execute testing profile: %v", err) t.Fatalf("execute testing profile: %v", err)
} }
if len(runner.requests) != 1 || runner.requests[0].ConfigProfile == nil || *runner.requests[0].ConfigProfile != "downstream-economy" { if len(runner.requests) != 1 || runner.requests[0].ConfigOverlay == nil || *runner.requests[0].ConfigOverlay != "downstream-economy" {
t.Fatalf("testing profile request = %#v, want explicit differently named downstream selector", runner.requests) t.Fatalf("testing profile request = %#v, want explicit differently named downstream selector", runner.requests)
} }
@@ -857,7 +857,7 @@ func TestExtractLifecycleRealConfigProfilesSelectNotariusExplicitly(t *testing.T
if _, err := executeStages(context.Background(), narratioOnly, plan, RunOptions{Env: env}); err != nil { if _, err := executeStages(context.Background(), narratioOnly, plan, RunOptions{Env: env}); err != nil {
t.Fatalf("execute Narratio-only profile: %v", err) t.Fatalf("execute Narratio-only profile: %v", err)
} }
if len(runner.requests) != 2 || runner.requests[1].ConfigProfile != nil { if len(runner.requests) != 2 || runner.requests[1].ConfigOverlay != nil {
t.Fatalf("Narratio-only profile request = %#v, want no automatic downstream selector", runner.requests) t.Fatalf("Narratio-only profile request = %#v, want no automatic downstream selector", runner.requests)
} }

View File

@@ -305,7 +305,7 @@ type NotariusConfig struct {
Enabled bool `yaml:"enabled" json:"enabled"` Enabled bool `yaml:"enabled" json:"enabled"`
Binary string `yaml:"binary" json:"binary"` Binary string `yaml:"binary" json:"binary"`
Credentials []configkit.Secret `yaml:"credentials,omitempty" json:"credentials,omitempty"` Credentials []configkit.Secret `yaml:"credentials,omitempty" json:"credentials,omitempty"`
ConfigProfile *string `yaml:"config_profile,omitempty" json:"config_profile,omitempty"` ConfigOverlay *string `yaml:"config_overlay,omitempty" json:"config_overlay,omitempty"`
Resume bool `yaml:"resume,omitempty" json:"resume,omitempty"` Resume bool `yaml:"resume,omitempty" json:"resume,omitempty"`
ChunkCache string `yaml:"chunk_cache,omitempty" json:"chunk_cache,omitempty"` ChunkCache string `yaml:"chunk_cache,omitempty" json:"chunk_cache,omitempty"`
ConfigPath string `yaml:"config_path" json:"config_path"` ConfigPath string `yaml:"config_path" json:"config_path"`

View File

@@ -54,7 +54,7 @@ func (cfg NotariusConfig) MarshalYAML() (any, error) {
Enabled bool `yaml:"enabled"` Enabled bool `yaml:"enabled"`
Binary string `yaml:"binary"` Binary string `yaml:"binary"`
Credentials []string `yaml:"credentials,omitempty"` Credentials []string `yaml:"credentials,omitempty"`
ConfigProfile *string `yaml:"config_profile,omitempty"` ConfigOverlay *string `yaml:"config_overlay,omitempty"`
Resume bool `yaml:"resume,omitempty"` Resume bool `yaml:"resume,omitempty"`
ChunkCache string `yaml:"chunk_cache,omitempty"` ChunkCache string `yaml:"chunk_cache,omitempty"`
ConfigPath string `yaml:"config_path"` ConfigPath string `yaml:"config_path"`
@@ -66,7 +66,7 @@ func (cfg NotariusConfig) MarshalYAML() (any, error) {
} }
return safeNotarius{ return safeNotarius{
Enabled: cfg.Enabled, Binary: cfg.Binary, Credentials: secretReferenceNames(cfg.Credentials), Enabled: cfg.Enabled, Binary: cfg.Binary, Credentials: secretReferenceNames(cfg.Credentials),
ConfigProfile: cfg.ConfigProfile, ConfigOverlay: cfg.ConfigOverlay,
Resume: cfg.Resume, ChunkCache: cfg.ChunkCache, Resume: cfg.Resume, ChunkCache: cfg.ChunkCache,
ConfigPath: cfg.ConfigPath, PipelineID: cfg.PipelineID, Timeout: cfg.Timeout, ConfigPath: cfg.ConfigPath, PipelineID: cfg.PipelineID, Timeout: cfg.Timeout,
WorkingDirectory: cfg.WorkingDirectory, References: cfg.References, Outputs: cfg.Outputs, WorkingDirectory: cfg.WorkingDirectory, References: cfg.References, Outputs: cfg.Outputs,

View File

@@ -103,7 +103,7 @@ storage:
notarius: notarius:
enabled: false enabled: false
credentials: [NARRATIO_TEST_NOTARIUS_KEY] credentials: [NARRATIO_TEST_NOTARIUS_KEY]
config_profile: "" config_overlay: ""
` `
pipelinePath, _ := writeConfigFiles(t, pipelineYAML, testSessionBaseYAML) pipelinePath, _ := writeConfigFiles(t, pipelineYAML, testSessionBaseYAML)
@@ -141,20 +141,20 @@ notarius:
} }
} }
} }
if !strings.Contains(string(firstYAML), "config_profile: \"\"\n") { if !strings.Contains(string(firstYAML), "config_overlay: \"\"\n") {
t.Fatalf("effective YAML omitted explicit empty Notarius config profile: %q", firstYAML) t.Fatalf("effective YAML omitted explicit empty Notarius config overlay: %q", firstYAML)
} }
var roundTrip struct { var roundTrip struct {
Notarius *struct { Notarius *struct {
Credentials []string `yaml:"credentials"` Credentials []string `yaml:"credentials"`
ConfigProfile *string `yaml:"config_profile"` ConfigOverlay *string `yaml:"config_overlay"`
} `yaml:"notarius"` } `yaml:"notarius"`
} }
if err := yaml.Unmarshal(firstYAML, &roundTrip); err != nil { if err := yaml.Unmarshal(firstYAML, &roundTrip); err != nil {
t.Fatalf("round-trip effective YAML: %v", err) t.Fatalf("round-trip effective YAML: %v", err)
} }
if roundTrip.Notarius == nil || roundTrip.Notarius.ConfigProfile == nil || *roundTrip.Notarius.ConfigProfile != "" { if roundTrip.Notarius == nil || roundTrip.Notarius.ConfigOverlay == nil || *roundTrip.Notarius.ConfigOverlay != "" {
t.Fatalf("round-trip config profile = %#v, want explicit empty", roundTrip.Notarius) t.Fatalf("round-trip config overlay = %#v, want explicit empty", roundTrip.Notarius)
} }
if len(roundTrip.Notarius.Credentials) != 1 || roundTrip.Notarius.Credentials[0] != "NARRATIO_TEST_NOTARIUS_KEY" { if len(roundTrip.Notarius.Credentials) != 1 || roundTrip.Notarius.Credentials[0] != "NARRATIO_TEST_NOTARIUS_KEY" {
t.Fatalf("round-trip credential reference = %#v, want safe public name", roundTrip.Notarius.Credentials) t.Fatalf("round-trip credential reference = %#v, want safe public name", roundTrip.Notarius.Credentials)

View File

@@ -238,7 +238,14 @@ func TestNotariusCredentialOverlayReplacementAndProvenance(t *testing.T) {
} }
} }
func TestNotariusConfigProfileComposition(t *testing.T) { func TestNotariusRetiredConfigProfileRejected(t *testing.T) {
root := writePipelineFixture(t, t.TempDir(), "pipeline.yml", "whisperx:\n transcribe_url: https://transcription.example.test/transcribe\nnotarius:\n config_profile: testing\n")
if _, err := LoadPipelineWithOptions(root, PipelineLoadOptions{}); err == nil {
t.Fatal("retired Notarius config_profile field was accepted")
}
}
func TestNotariusConfigOverlayComposition(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
baseValue string baseValue string
@@ -250,15 +257,15 @@ func TestNotariusConfigProfileComposition(t *testing.T) {
{name: "explicit empty", baseValue: `""`, want: stringPointer("")}, {name: "explicit empty", baseValue: `""`, want: stringPointer("")},
{name: "explicit default", baseValue: "default", want: stringPointer("default")}, {name: "explicit default", baseValue: "default", want: stringPointer("default")},
{name: "base named selection", baseValue: "production", want: stringPointer("production")}, {name: "base named selection", baseValue: "production", want: stringPointer("production")},
{name: "profile replaces named selection", baseValue: "production", overlayName: "testing", overlay: "notarius:\n config_profile: inexpensive\n", want: stringPointer("inexpensive")}, {name: "profile replaces named selection", baseValue: "production", overlayName: "testing", overlay: "notarius:\n config_overlay: inexpensive\n", want: stringPointer("inexpensive")},
{name: "profile clears named selection", baseValue: "production", overlayName: "testing", overlay: "notarius:\n config_profile: null\n"}, {name: "profile clears named selection", baseValue: "production", overlayName: "testing", overlay: "notarius:\n config_overlay: null\n"},
} }
for _, test := range tests { for _, test := range tests {
t.Run(test.name, func(t *testing.T) { t.Run(test.name, func(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
rootYAML := "whisperx:\n transcribe_url: https://transcription.example.test/transcribe\nnotarius:\n enabled: false\n" rootYAML := "whisperx:\n transcribe_url: https://transcription.example.test/transcribe\nnotarius:\n enabled: false\n"
if test.baseValue != "" { if test.baseValue != "" {
rootYAML += " config_profile: " + test.baseValue + "\n" rootYAML += " config_overlay: " + test.baseValue + "\n"
} }
root := writePipelineFixture(t, dir, "pipeline.yml", rootYAML) root := writePipelineFixture(t, dir, "pipeline.yml", rootYAML)
options := PipelineLoadOptions{} options := PipelineLoadOptions{}
@@ -270,8 +277,8 @@ func TestNotariusConfigProfileComposition(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
if cfg.Notarius == nil || !reflect.DeepEqual(cfg.Notarius.ConfigProfile, test.want) { if cfg.Notarius == nil || !reflect.DeepEqual(cfg.Notarius.ConfigOverlay, test.want) {
t.Fatalf("config profile = %#v, want %#v", cfg.Notarius.ConfigProfile, test.want) t.Fatalf("config overlay = %#v, want %#v", cfg.Notarius.ConfigOverlay, test.want)
} }
}) })
} }

View File

@@ -146,7 +146,7 @@ func (extractStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (re
chunkCache = "refresh" chunkCache = "refresh"
} }
request := notarius.RunRequest{ request := notarius.RunRequest{
Binary: resolvedBinary, Credentials: notariusConfig.Credentials, ConfigProfile: notariusConfig.ConfigProfile, Binary: resolvedBinary, Credentials: notariusConfig.Credentials, ConfigOverlay: notariusConfig.ConfigOverlay,
ConfigPath: configPath, PipelineID: notariusConfig.PipelineID, ConfigPath: configPath, PipelineID: notariusConfig.PipelineID,
InputPath: inputPath, OutputRoot: outputRoot, WorkingDirectory: workingDirectory, InputPath: inputPath, OutputRoot: outputRoot, WorkingDirectory: workingDirectory,
ReceiptPath: receiptPath, LogPath: logPath, Timeout: timeout, Resume: resume, ChunkCache: chunkCache, References: referenceBindings, ReceiptPath: receiptPath, LogPath: logPath, Timeout: timeout, Resume: resume, ChunkCache: chunkCache, References: referenceBindings,
@@ -451,7 +451,7 @@ type fingerprintOutput struct {
type fingerprintDocument struct { type fingerprintDocument struct {
PipelineID string `json:"pipeline_id"` PipelineID string `json:"pipeline_id"`
ConfigProfile string `json:"config_profile,omitempty"` ConfigOverlay string `json:"config_overlay,omitempty"`
Input artifacts.ExtractionInputIdentity `json:"input"` Input artifacts.ExtractionInputIdentity `json:"input"`
References []extractReferenceIdentity `json:"references"` References []extractReferenceIdentity `json:"references"`
Outputs []fingerprintOutput `json:"outputs"` Outputs []fingerprintOutput `json:"outputs"`
@@ -486,7 +486,7 @@ func extractionFingerprint(
return sortedReferences[i].Path < sortedReferences[j].Path return sortedReferences[i].Path < sortedReferences[j].Path
}) })
payload, err := json.Marshal(fingerprintDocument{ payload, err := json.Marshal(fingerprintDocument{
PipelineID: cfg.PipelineID, ConfigProfile: notariusConfigProfileReuseIdentity(cfg.ConfigProfile), PipelineID: cfg.PipelineID, ConfigOverlay: notariusConfigOverlayReuseIdentity(cfg.ConfigOverlay),
Input: input, References: sortedReferences, Outputs: outputs, Input: input, References: sortedReferences, Outputs: outputs,
}) })
if err != nil { if err != nil {

View File

@@ -72,8 +72,8 @@ notarius:
env, m, fake := setupExtractEnv(t) env, m, fake := setupExtractEnv(t)
env.Config.Pipeline.Notarius.Credentials = loadedPipeline.Notarius.Credentials env.Config.Pipeline.Notarius.Credentials = loadedPipeline.Notarius.Credentials
configProfile := "inexpensive" configOverlay := "inexpensive"
env.Config.Pipeline.Notarius.ConfigProfile = &configProfile env.Config.Pipeline.Notarius.ConfigOverlay = &configOverlay
result, err := (extractStage{}).Run(context.Background(), env, m) result, err := (extractStage{}).Run(context.Background(), env, m)
if err != nil { if err != nil {
t.Fatalf("Run() error = %v", err) t.Fatalf("Run() error = %v", err)
@@ -106,8 +106,8 @@ notarius:
if len(req.Credentials) != 1 || req.Credentials[0].EnvName() != credentialName || !req.Credentials[0].IsResolved() { if len(req.Credentials) != 1 || req.Credentials[0].EnvName() != credentialName || !req.Credentials[0].IsResolved() {
t.Fatal("request did not retain the loaded Notarius credential reference") t.Fatal("request did not retain the loaded Notarius credential reference")
} }
if req.ConfigProfile == nil || *req.ConfigProfile != configProfile { if req.ConfigOverlay == nil || *req.ConfigOverlay != configOverlay {
t.Fatalf("request config profile = %#v, want %q", req.ConfigProfile, configProfile) t.Fatalf("request config overlay = %#v, want %q", req.ConfigOverlay, configOverlay)
} }
if result.Metadata["reference_count"] != 0 || len(result.Metadata["references"].([]map[string]any)) != 0 { if result.Metadata["reference_count"] != 0 || len(result.Metadata["references"].([]map[string]any)) != 0 {
t.Fatalf("reference metadata = count %#v references %#v", result.Metadata["reference_count"], result.Metadata["references"]) t.Fatalf("reference metadata = count %#v references %#v", result.Metadata["reference_count"], result.Metadata["references"])
@@ -141,8 +141,8 @@ func TestExtractStageAppliesNotariusCacheControlsAndForcePrecedence(t *testing.T
for _, test := range tests { for _, test := range tests {
t.Run(test.name, func(t *testing.T) { t.Run(test.name, func(t *testing.T) {
env, m, fake := setupExtractEnv(t) env, m, fake := setupExtractEnv(t)
configProfile := "inexpensive" configOverlay := "inexpensive"
env.Config.Pipeline.Notarius.ConfigProfile = &configProfile env.Config.Pipeline.Notarius.ConfigOverlay = &configOverlay
env.Config.Pipeline.Notarius.Resume = test.configuredResume env.Config.Pipeline.Notarius.Resume = test.configuredResume
env.Config.Pipeline.Notarius.ChunkCache = test.configuredMode env.Config.Pipeline.Notarius.ChunkCache = test.configuredMode
env.Force = test.force env.Force = test.force
@@ -157,8 +157,8 @@ func TestExtractStageAppliesNotariusCacheControlsAndForcePrecedence(t *testing.T
if request.Resume != test.wantResume || request.ChunkCache != test.wantMode { if request.Resume != test.wantResume || request.ChunkCache != test.wantMode {
t.Fatalf("cache controls = resume %v, mode %q; want resume %v, mode %q", request.Resume, request.ChunkCache, test.wantResume, test.wantMode) t.Fatalf("cache controls = resume %v, mode %q; want resume %v, mode %q", request.Resume, request.ChunkCache, test.wantResume, test.wantMode)
} }
if request.ConfigProfile == nil || *request.ConfigProfile != configProfile { if request.ConfigOverlay == nil || *request.ConfigOverlay != configOverlay {
t.Fatalf("config profile = %#v, want %q", request.ConfigProfile, configProfile) t.Fatalf("config overlay = %#v, want %q", request.ConfigOverlay, configOverlay)
} }
}) })
} }
@@ -1018,7 +1018,7 @@ func TestExtractionFingerprintBaseSelectorsPreservePriorPayloadShape(t *testing.
legacyDigest := sha256.Sum256(legacyPayload) legacyDigest := sha256.Sum256(legacyPayload)
want := hex.EncodeToString(legacyDigest[:]) want := hex.EncodeToString(legacyDigest[:])
for _, profile := range []*string{nil, stringPointerForExtractTest(""), stringPointerForExtractTest("default")} { for _, profile := range []*string{nil, stringPointerForExtractTest(""), stringPointerForExtractTest("default")} {
cfg.ConfigProfile = profile cfg.ConfigOverlay = profile
got, err := extractionFingerprint(cfg, input, references) got, err := extractionFingerprint(cfg, input, references)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -1027,7 +1027,7 @@ func TestExtractionFingerprintBaseSelectorsPreservePriorPayloadShape(t *testing.
t.Fatalf("base selection fingerprint = %q, want prior no-profile payload digest %q", got, want) t.Fatalf("base selection fingerprint = %q, want prior no-profile payload digest %q", got, want)
} }
} }
cfg.ConfigProfile = stringPointerForExtractTest("inexpensive") cfg.ConfigOverlay = stringPointerForExtractTest("inexpensive")
named, err := extractionFingerprint(cfg, input, references) named, err := extractionFingerprint(cfg, input, references)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -1055,10 +1055,10 @@ func TestExtractStageResumeValidationAcceptsCurrentImmutableResult(t *testing.T)
} }
} }
func TestExtractStageResumeValidationTracksConfigProfile(t *testing.T) { func TestExtractStageResumeValidationTracksConfigOverlay(t *testing.T) {
env, m, _ := setupExtractEnv(t) env, m, _ := setupExtractEnv(t)
selected := "inexpensive" selected := "inexpensive"
env.Config.Pipeline.Notarius.ConfigProfile = &selected env.Config.Pipeline.Notarius.ConfigOverlay = &selected
seedSucceededExtractResult(t, env, m) seedSucceededExtractResult(t, env, m)
validation, err := (extractStage{}).ValidateResume(context.Background(), env, m) validation, err := (extractStage{}).ValidateResume(context.Background(), env, m)
@@ -1070,7 +1070,7 @@ func TestExtractStageResumeValidationTracksConfigProfile(t *testing.T) {
} }
changed := "experimental" changed := "experimental"
env.Config.Pipeline.Notarius.ConfigProfile = &changed env.Config.Pipeline.Notarius.ConfigOverlay = &changed
validation, err = (extractStage{}).ValidateResume(context.Background(), env, m) validation, err = (extractStage{}).ValidateResume(context.Background(), env, m)
if err != nil { if err != nil {
t.Fatalf("ValidateResume(changed selection) error = %v", err) t.Fatalf("ValidateResume(changed selection) error = %v", err)

View File

@@ -27,7 +27,7 @@ const notariusIntegrationBinaryEnv = "NARRATIO_TEST_NOTARIUS_BINARY"
func TestNotariusCacheReuseIntegration(t *testing.T) { func TestNotariusCacheReuseIntegration(t *testing.T) {
binary := strings.TrimSpace(os.Getenv(notariusIntegrationBinaryEnv)) binary := strings.TrimSpace(os.Getenv(notariusIntegrationBinaryEnv))
if binary == "" { if binary == "" {
t.Skipf("set %s to an absolute Notarius v0.7.0-compatible executable to run this integration test", notariusIntegrationBinaryEnv) t.Skipf("set %s to an absolute Notarius v0.8.1-compatible executable to run this integration test", notariusIntegrationBinaryEnv)
} }
if !filepath.IsAbs(binary) { if !filepath.IsAbs(binary) {
t.Fatalf("%s = %q, want an absolute path", notariusIntegrationBinaryEnv, binary) t.Fatalf("%s = %q, want an absolute path", notariusIntegrationBinaryEnv, binary)
@@ -235,6 +235,16 @@ pipelines:
if err := os.WriteFile(configPath, []byte(content), 0o600); err != nil { if err := os.WriteFile(configPath, []byte(content), 0o600); err != nil {
t.Fatalf("write Notarius integration config: %v", err) t.Fatalf("write Notarius integration config: %v", err)
} }
// Select a real downstream overlay. Disable provider retries so the controlled
// failure below tests checkpoint recovery rather than transient HTTP recovery.
overlay := "integration"
overlayDir := filepath.Join(root, "overlays")
if err := os.MkdirAll(overlayDir, 0o700); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(overlayDir, overlay+".yml"), []byte("promptkit:\n transient_retries: 0\n"), 0o600); err != nil {
t.Fatal(err)
}
transcriptPath := filepath.Join(sessionPathsForEnv(env, runManifest.SessionID).ArtifactsDir, "trimmed.from-manifest.json") transcriptPath := filepath.Join(sessionPathsForEnv(env, runManifest.SessionID).ArtifactsDir, "trimmed.from-manifest.json")
setNotariusIntegrationTranscript(t, runManifest, transcriptPath, notariusIntegrationTranscript("session-cache", "Aria casts Cure Wounds.")) setNotariusIntegrationTranscript(t, runManifest, transcriptPath, notariusIntegrationTranscript("session-cache", "Aria casts Cure Wounds."))
@@ -242,7 +252,8 @@ pipelines:
env.Notarius = notarius.NewSubprocessRunner() env.Notarius = notarius.NewSubprocessRunner()
env.Config.Pipeline.Notarius = &config.NotariusConfig{ env.Config.Pipeline.Notarius = &config.NotariusConfig{
Enabled: true, Binary: binary, ConfigPath: configPath, PipelineID: pipelineID, Enabled: true, Binary: binary, ConfigPath: configPath, PipelineID: pipelineID,
Timeout: "2m", WorkingDirectory: root, ChunkCache: "auto", ConfigOverlay: &overlay,
Timeout: "2m", WorkingDirectory: root, ChunkCache: "auto",
References: map[string]string{"party": artifactpolicy.SourceInputParty}, References: map[string]string{"party": artifactpolicy.SourceInputParty},
Outputs: map[string]config.NotariusOutputConfig{ Outputs: map[string]config.NotariusOutputConfig{
"spells_first": { "spells_first": {

View File

@@ -22,16 +22,16 @@ const (
type extractSemanticConfig struct { type extractSemanticConfig struct {
Enabled bool `json:"enabled"` Enabled bool `json:"enabled"`
PipelineID string `json:"pipeline_id,omitempty"` PipelineID string `json:"pipeline_id,omitempty"`
ConfigProfile string `json:"config_profile,omitempty"` ConfigOverlay string `json:"config_overlay,omitempty"`
References []extractSemanticReference `json:"references"` References []extractSemanticReference `json:"references"`
Outputs []extractSemanticOutput `json:"outputs"` Outputs []extractSemanticOutput `json:"outputs"`
} }
func notariusConfigProfileReuseIdentity(profile *string) string { func notariusConfigOverlayReuseIdentity(overlay *string) string {
if profile == nil || *profile == "" || *profile == "default" { if overlay == nil || *overlay == "" || *overlay == "default" {
return "" return ""
} }
return *profile return *overlay
} }
type extractSemanticReference struct { type extractSemanticReference struct {
@@ -85,7 +85,7 @@ func (extractStage) SemanticConfigFingerprint(env *Env) (manifest.SemanticConfig
} }
payload.Enabled = true payload.Enabled = true
payload.PipelineID = strings.TrimSpace(cfg.PipelineID) payload.PipelineID = strings.TrimSpace(cfg.PipelineID)
payload.ConfigProfile = notariusConfigProfileReuseIdentity(cfg.ConfigProfile) payload.ConfigOverlay = notariusConfigOverlayReuseIdentity(cfg.ConfigOverlay)
for rawSelector, rawSourceID := range cfg.References { for rawSelector, rawSourceID := range cfg.References {
selector, err := notariusref.NormalizeSelector(rawSelector) selector, err := notariusref.NormalizeSelector(rawSelector)
if err != nil { if err != nil {

View File

@@ -102,7 +102,7 @@ func TestExtractSemanticConfigSensitivityAndOperationalExclusions(t *testing.T)
baseFingerprints := make([]manifest.SemanticConfigFingerprint, 0, 3) baseFingerprints := make([]manifest.SemanticConfigFingerprint, 0, 3)
for _, profile := range []*string{nil, profilePointer(""), profilePointer("default")} { for _, profile := range []*string{nil, profilePointer(""), profilePointer("default")} {
env := downstreamSemanticEnv() env := downstreamSemanticEnv()
env.Config.Pipeline.Notarius.ConfigProfile = profile env.Config.Pipeline.Notarius.ConfigOverlay = profile
fingerprint, err := (extractStage{}).SemanticConfigFingerprint(env) fingerprint, err := (extractStage{}).SemanticConfigFingerprint(env)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -115,7 +115,7 @@ func TestExtractSemanticConfigSensitivityAndOperationalExclusions(t *testing.T)
} }
} }
profiled := downstreamSemanticEnv() profiled := downstreamSemanticEnv()
profiled.Config.Pipeline.Notarius.ConfigProfile = profilePointer("inexpensive") profiled.Config.Pipeline.Notarius.ConfigOverlay = profilePointer("inexpensive")
firstProfile, err := (extractStage{}).SemanticConfigFingerprint(profiled) firstProfile, err := (extractStage{}).SemanticConfigFingerprint(profiled)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -130,7 +130,7 @@ func TestExtractSemanticConfigSensitivityAndOperationalExclusions(t *testing.T)
if !firstProfile.Equal(unchangedProfile) { if !firstProfile.Equal(unchangedProfile) {
t.Fatal("unchanged Notarius selection changed semantic fingerprint") t.Fatal("unchanged Notarius selection changed semantic fingerprint")
} }
profiled.Config.Pipeline.Notarius.ConfigProfile = profilePointer("experimental") profiled.Config.Pipeline.Notarius.ConfigOverlay = profilePointer("experimental")
secondProfile, err := (extractStage{}).SemanticConfigFingerprint(profiled) secondProfile, err := (extractStage{}).SemanticConfigFingerprint(profiled)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -138,7 +138,7 @@ func TestExtractSemanticConfigSensitivityAndOperationalExclusions(t *testing.T)
if firstProfile.Equal(secondProfile) { if firstProfile.Equal(secondProfile) {
t.Fatal("named to different named Notarius selection retained semantic fingerprint") t.Fatal("named to different named Notarius selection retained semantic fingerprint")
} }
profiled.Config.Pipeline.Notarius.ConfigProfile = nil profiled.Config.Pipeline.Notarius.ConfigOverlay = nil
returnedToBase, err := (extractStage{}).SemanticConfigFingerprint(profiled) returnedToBase, err := (extractStage{}).SemanticConfigFingerprint(profiled)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -153,7 +153,7 @@ func TestExtractSemanticConfigSensitivityAndOperationalExclusions(t *testing.T)
disabledBase.Config.Pipeline.Notarius.Enabled = false disabledBase.Config.Pipeline.Notarius.Enabled = false
disabledNamed := downstreamSemanticEnv() disabledNamed := downstreamSemanticEnv()
disabledNamed.Config.Pipeline.Notarius.Enabled = false disabledNamed.Config.Pipeline.Notarius.Enabled = false
disabledNamed.Config.Pipeline.Notarius.ConfigProfile = profilePointer("ignored") disabledNamed.Config.Pipeline.Notarius.ConfigOverlay = profilePointer("ignored")
disabledBaseFingerprint, err := (extractStage{}).SemanticConfigFingerprint(disabledBase) disabledBaseFingerprint, err := (extractStage{}).SemanticConfigFingerprint(disabledBase)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)