eric/narratio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 27 Wiki Activity

Adopt Notarius configuration overlays

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-28 03:09:55 +00:00
parent 3de5f9378d
commit 4c114b6319
19 changed files with 140 additions and 104 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
docs/config.md
View File

@@ -358,7 +358,7 @@ Rules:
| `pipeline.notarius.enabled` | bool | No | `false` |
| `pipeline.notarius.binary` | string | No | `notarius` |
| `pipeline.notarius.credentials[]` | list[secret reference] | No | empty; exact environment-variable names forwarded to Notarius |
| `pipeline.notarius.config_profile` | string | No | unset; exact Notarius ConfigKit profile selector |
| `pipeline.notarius.config_overlay` | string | No | unset; exact Notarius ConfigKit overlay selector |
| `pipeline.notarius.config_path` | string | Conditional | required when enabled; relative paths resolve from the pipeline file directory |
| `pipeline.notarius.pipeline_id` | string | Conditional | required when enabled |
| `pipeline.notarius.timeout` | duration | No | `3h`; must be positive |
@@ -407,22 +407,33 @@ or `[]` forwards nothing. A Narratio overlay replaces the entire list, and
`[]` clears it. See the [Audita integration contract](./integrations/audita.md)
for the subprocess and redaction boundary.
### Notarius Configuration Profile
<a id="notarius-configuration-profile"></a>
`pipeline.notarius.config_profile` optionally selects a Notarius ConfigKit
configuration profile when extraction executes:
### Notarius Configuration Overlay
Migration: rename `pipeline.notarius.config_profile` to `config_overlay`.
The old field is rejected; there is no compatibility alias. For Notarius v0.8+
move selected downstream files from `profiles/` to `overlays/` beside its root
configuration. This does not rename PromptKit LLM profiles.
The renamed selector also changes extraction resume evidence for named
selections: their existing extraction is regenerated on the next applicable
run, with normal dependent invalidation. Omitted/base selections do not acquire
a new invalidation solely from this rename.
`pipeline.notarius.config_overlay` optionally selects a Notarius ConfigKit
configuration overlay when extraction executes:
```yaml
notarius:
config_profile: downstream-economy
config_overlay: downstream-economy
```
When the field is omitted or `null`, Narratio supplies no downstream profile
When the field is omitted or `null`, Narratio supplies no downstream overlay
selector and preserves Notarius's existing configuration-selection behavior.
When present, Narratio forwards the exact string to Notarius, including an
empty string or `default`, which explicitly select Notarius's composed base
configuration. Notarius owns selector interpretation and validation; Narratio
does not restrict profile names, resolve profile files, or rewrite the value.
does not restrict overlay names, resolve overlay files, or rewrite the value.
This selection is independent of Narratio's own `--overlay` selection. A
Narratio overlay may explicitly select a differently named Notarius overlay,

28
docs/integrations/notarius.md
View File

@@ -7,14 +7,14 @@ lanes from the final trimmed Seriatim transcript. Narratio owns invocation,
safe bundle discovery, lane selection, and its own artifact metadata. Notarius
owns pipeline definitions, lane schemas, the receipt, and bundle formats.
Canonical Notarius v0.7.0 references:
Canonical Notarius v0.8.1 references:
- [CLI reference](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.7.0/docs/cli.md)
- [Subprocess consumer contract](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.7.0/docs/consumers/subprocess.md)
- [D&D pipeline and lane contracts](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.7.0/docs/consumers/dnd-pipeline.md)
- [Run-result receipt](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.7.0/docs/integrations/run-result.md)
- [JSON output bundle](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.7.0/docs/integrations/json-output.md)
- [D&D spell-catalog overlay](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.7.0/docs/integrations/dnd-spell-catalog-overlays.md)
- [CLI reference](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.8.1/docs/cli.md)
- [Subprocess consumer contract](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.8.1/docs/consumers/subprocess.md)
- [D&D pipeline and lane contracts](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.8.1/docs/consumers/dnd-pipeline.md)
- [Run-result receipt](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.8.1/docs/integrations/run-result.md)
- [JSON output bundle](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.8.1/docs/integrations/json-output.md)
- [D&D spell-catalog overlay](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.8.1/docs/integrations/dnd-spell-catalog-overlays.md)
The [complete Narratio example](../../examples/pipeline.full.annotated.yml)
records the exact current constraints for all ten D&D lanes. Treat the linked
@@ -27,17 +27,17 @@ When `pipeline.notarius.enabled` is true, Narratio resolves the executable,
configuration path, input path, output directory, and working directory to
absolute paths. Configured references require support for the `--reference`
contract introduced in Notarius v0.6.0. Cache controls require support for the
selected `--resume` and `--chunk_cache` flags. Notarius v0.7.0 is the tested
selected `--resume` and `--chunk_cache` flags. Notarius v0.8.1 is the tested
cache integration baseline, not an unconditional minimum version; Narratio
does not impose a runtime version gate. Each reference binding is a separate
argument before `--json`. When `pipeline.notarius.config_profile` is present,
argument before `--json`. When `pipeline.notarius.config_overlay` is present,
its exact value is another argument pair before `--json`:
```text
notarius run <pipeline_id> --config <config_path> --input <trimmed_json> --output-dir <staging_dir> [--resume] [--chunk_cache <mode>] [--reference <selector>=<verified_snapshot_path>]... [--config-profile <name>] --json
notarius run <pipeline_id> --config <config_path> --input <trimmed_json> --output-dir <staging_dir> [--resume] [--chunk_cache <mode>] [--reference <selector>=<verified_snapshot_path>]... [--config-overlay <name>] --json
```
Notarius v0.7.0 is a known supporting version for `--config-profile`; using the
Notarius v0.8.1 is a known supporting version for `--config-overlay`; using the
option requires a compatible Notarius executable, but deployments that omit the
field acquire no additional version requirement. Omitted or null configuration
adds no flag. Supplied empty, `default`, and named selectors are forwarded
@@ -46,7 +46,7 @@ option selects a Notarius configuration overlay, not a PromptKit profile through
`--llm-profile`, and Narratio never substitutes its own selected overlay name.
A missing or invalid downstream overlay is an ordinary Notarius subprocess
failure: Narratio does not retry without the selector or fall back to another
profile.
overlay.
Reference paths are absolute stable snapshots streamed from manifest-verified
canonical files prepared inside the current Narratio session workspace. They
@@ -129,7 +129,7 @@ encoded credentials.
## Accepted Result
Narratio's tested invocation baseline is Notarius v0.7.0. The accepted
Narratio's tested invocation baseline is Notarius v0.8.1. The accepted
receipt remains `notarius.run-result.v2`; reference flags do not change the
receipt or ten-lane output contract. The receipt
must identify the configured pipeline, and its `index_file` must be exactly
@@ -188,7 +188,7 @@ for failure recovery.
## Opt-in Integration Test
The production-boundary cache test requires an absolute path to a locally
built Notarius v0.7.0-compatible executable. It uses only a local
built Notarius v0.8.1-compatible executable. It uses only a local
OpenAI-compatible `httptest` provider and temporary cache/config/output roots:
```sh

12
docs/internal/stage-extract.md
View File

@@ -24,11 +24,11 @@ procedures belong in [Operations](../operations.md).
and rejects any source change observed while copying;
4. fingerprints the byte- and provenance-bearing Notarius invocation evidence,
including sorted reference identities and the explicit downstream
configuration-profile selection;
configuration-overlay selection;
5. creates a run-local staging directory and invokes the injected
`notarius.Runner`, carrying the optional already-resolved credential
reference without exposing its value, the optional exact configuration
profile, plus effective resume and chunk-cache controls after force
overlay, plus effective resume and chunk-cache controls after force
precedence;
6. revalidates the reference snapshots, then validates the v2 successful
receipt, confined index, management documents, configured required lane
@@ -73,7 +73,7 @@ Before the focused validator runs, the application compares extract's versioned
semantic fingerprint. It covers enablement, Notarius pipeline identity, sorted
reference selector/source mappings, sorted declared output contracts, and each
canonical `narratio.extraction.<key>` output identity. It also covers the
explicit Notarius configuration-profile selection; omitted, empty, and exact
explicit Notarius configuration-overlay selection; omitted, empty, and exact
`default` selectors share the base-selection identity. It excludes executable,
timeout, working directory, config path, and private Notarius config contents.
Credential presence, reference name, and resolved value are likewise
@@ -85,7 +85,7 @@ not rerun a valid completed extraction or stale its dependents.
stage record still matches the current byte- and provenance-bearing invocation
evidence. That evidence covers the current direct trimmed-transcript identity,
sorted prepared-reference identities, pipeline identity, and configured output
contracts, plus the same downstream profile-selection identity. The same
contracts, plus the same downstream overlay-selection identity. The same
reference helper and transcript identity are resolved again for artifact
evidence, so changing current transcript bytes, reference bytes, producer
identity, or a named downstream selection makes the prior extraction obsolete.
@@ -107,7 +107,7 @@ obsolete results are non-resumable and run again; unsafe filesystem conditions
return an error rather than silently accepting or replacing data.
This sensitivity is limited to the explicit downstream selector. Narratio's
own profile label and whole effective-configuration digest are not blanket
own overlay label and whole effective-configuration digest are not blanket
extraction invalidators. Neither contract can observe files imported by
Notarius configuration, profile contents, prompt/module definitions, or other
transitive inputs. Operators must force extraction after changing any such
@@ -144,5 +144,5 @@ recovery.
`internal/artifacts/extraction_catalog_test.go`
- Composition and downstream behavior: `internal/app/runner_test.go`,
`internal/stage/analyze_test.go`, `internal/stage/publish_test.go`
- Opt-in Notarius v0.7.0 production-boundary coverage:
- Opt-in Notarius v0.8.1 production-boundary coverage:
`internal/stage/notarius_cache_integration_test.go`

13
docs/operations.md
View File

@@ -282,14 +282,21 @@ earlier checkpoints are reused and execution resumes with failed, stale, or
missing work in dependency order. `--force` is unnecessary unless the operator
intends to replace otherwise-current artifacts.
### Developing With A Notarius Configuration Profile
### Developing With A Notarius Configuration Overlay
Notarius v0.8+ defaults to two additional eligible transient provider attempts
per generation. Review downstream retry settings and Narratio's extraction
timeout together; retries can increase elapsed time and provider charges.
Notarius v0.8.1 refreshes its built-in profile checkpoint identity, so older
checkpoints may not be reused. Source-keyed chunk plans and already published
outputs are unaffected; no manual cache deletion is required.
A Narratio development overlay can explicitly select a Notarius overlay without
requiring the names to match. For example, `overlays/testing.yml` may contain:
```yaml
notarius:
config_profile: downstream-economy
config_overlay: downstream-economy
```
Selecting Narratio's `testing` overlay then supplies `downstream-economy` only
@@ -302,7 +309,7 @@ Narratio cannot observe edits inside Notarius configuration behind an unchanged
selector. Force extraction after such an edit when a replacement is intended.
Use `regenerate-artifacts` when both extraction and analysis must be rebuilt;
use `analyze` when the existing extraction bundle should be preserved. Field
semantics belong in [Configuration](./config.md#notarius-configuration-profile),
semantics belong in [Configuration](./config.md#notarius-configuration-overlay),
and the exact downstream invocation belongs in the
[Notarius integration contract](./integrations/notarius.md#invocation).

2
examples/pipeline.full.annotated.yml
View File

@@ -133,7 +133,7 @@ notarius:
# Optionally select a Notarius ConfigKit overlay when extraction runs. This
# is independent of Narratio's selected overlay and may use a different name.
# See docs/config.md and docs/integrations/notarius.md for the contracts.
# config_profile: downstream-economy
# config_overlay: downstream-economy
config_path: /usr/local/etc/notarius/config.yml
pipeline_id: dnd-session
timeout: 3h

6
internal/adapters/notarius/fake.go
View File

@@ -20,9 +20,9 @@ func (f *FakeRunner) Run(ctx context.Context, req RunRequest) (RunResult, error)
}
copyRequest := req
copyRequest.Credentials = append([]configkit.Secret(nil), req.Credentials...)
if req.ConfigProfile != nil {
profile := *req.ConfigProfile
copyRequest.ConfigProfile = &profile
if req.ConfigOverlay != nil {
profile := *req.ConfigOverlay
copyRequest.ConfigOverlay = &profile
}
copyRequest.References = append([]ReferenceBinding(nil), req.References...)
f.Requests = append(f.Requests, copyRequest)

2
internal/adapters/notarius/runner.go
View File

@@ -27,7 +27,7 @@ type RunRequest struct {
Binary string
// Credentials are already-resolved environment references selected for Notarius.
Credentials []configkit.Secret
ConfigProfile *string
ConfigOverlay *string
ConfigPath string
PipelineID string
InputPath string

4
internal/adapters/notarius/subprocess.go
View File

@@ -72,8 +72,8 @@ func (r *SubprocessRunner) Run(ctx context.Context, req RunRequest) (RunResult,
for _, reference := range references {
args = append(args, "--reference", reference.Selector+"="+reference.Path)
}
if req.ConfigProfile != nil {
args = append(args, "--config-profile", *req.ConfigProfile)
if req.ConfigOverlay != nil {
args = append(args, "--config-overlay", *req.ConfigOverlay)
}
args = append(args, "--json")
processResult, err := r.run(ctx, subprocess.RunRequest{

18
internal/adapters/notarius/subprocess_test.go
View File

@@ -118,7 +118,7 @@ func TestSubprocessRunnerMapsCacheControlsToArguments(t *testing.T) {
}
}
func TestSubprocessRunnerMapsConfigProfileToArgumentPair(t *testing.T) {
func TestSubprocessRunnerMapsConfigOverlayToArgumentPair(t *testing.T) {
tests := []struct {
name string
supplied bool
@@ -133,7 +133,7 @@ func TestSubprocessRunnerMapsConfigProfileToArgumentPair(t *testing.T) {
t.Run(test.name, func(t *testing.T) {
req := validRunRequest(t)
if test.supplied {
req.ConfigProfile = &test.value
req.ConfigOverlay = &test.value
}
var captured sharedsubprocess.RunRequest
runner := &SubprocessRunner{run: func(_ context.Context, processReq sharedsubprocess.RunRequest) (sharedsubprocess.RunResult, error) {
@@ -147,19 +147,19 @@ func TestSubprocessRunnerMapsConfigProfileToArgumentPair(t *testing.T) {
}
flagIndex := -1
for index, arg := range captured.Args {
if arg == "--config-profile" {
if arg == "--config-overlay" {
flagIndex = index
break
}
}
if !test.supplied {
if flagIndex >= 0 {
t.Fatalf("subprocess args unexpectedly contain config profile: %#v", captured.Args)
t.Fatalf("subprocess args unexpectedly contain config overlay: %#v", captured.Args)
}
return
}
if flagIndex < 0 || flagIndex+2 >= len(captured.Args) || captured.Args[flagIndex+1] != test.value || captured.Args[flagIndex+2] != "--json" {
t.Fatalf("config profile argument boundary = %#v, want flag, exact value, then --json", captured.Args)
t.Fatalf("config overlay argument boundary = %#v, want flag, exact value, then --json", captured.Args)
}
})
}
@@ -455,7 +455,7 @@ func TestSubprocessRunnerReturnsProcessFailuresWithoutParsingStdout(t *testing.T
req := validRunRequest(t)
if test.name == "nonzero" {
profile := "missing-profile"
req.ConfigProfile = &profile
req.ConfigOverlay = &profile
}
req.Binary = writeShellScript(t, "#!/bin/sh\n"+test.scriptBody)
req.Timeout = test.timeout
@@ -754,7 +754,7 @@ func TestLoadDiagnosticSummariesValidateBoundsAndTolerateUnknownFields(t *testin
func TestFakeRunnerCapturesRequestsAndHonorsContextAndError(t *testing.T) {
profile := "testing"
req := RunRequest{PipelineID: "pipeline", ConfigProfile: &profile, References: []ReferenceBinding{{Selector: "party", Path: "/references/party.json"}}}
req := RunRequest{PipelineID: "pipeline", ConfigOverlay: &profile, References: []ReferenceBinding{{Selector: "party", Path: "/references/party.json"}}}
want := RunResult{BundleRoot: "/bundle"}
fake := &FakeRunner{Result: want}
got, err := fake.Run(context.Background(), req)
@@ -766,8 +766,8 @@ func TestFakeRunnerCapturesRequestsAndHonorsContextAndError(t *testing.T) {
t.Fatalf("fake retained aliased request references: %#v", fake.Requests[0].References)
}
profile = "changed"
if *fake.Requests[0].ConfigProfile != "testing" {
t.Fatalf("fake retained aliased config profile: %q", *fake.Requests[0].ConfigProfile)
if *fake.Requests[0].ConfigOverlay != "testing" {
t.Fatalf("fake retained aliased config overlay: %q", *fake.Requests[0].ConfigOverlay)
}
wantErr := errors.New("configured failure")

30
internal/app/extract_lifecycle_test.go
View File

@@ -691,17 +691,17 @@ func TestExtractLifecycleCredentialChangesRemainResumable(t *testing.T) {
}
}
func TestExtractLifecycleConfigProfileChangesRerunExtractionAndDependents(t *testing.T) {
func TestExtractLifecycleConfigOverlayChangesRerunExtractionAndDependents(t *testing.T) {
cfg, env, runner := extractionLifecycleFixture(t, true)
selected := "inexpensive"
cfg.Pipeline.Notarius.ConfigProfile = &selected
cfg.Pipeline.Notarius.ConfigOverlay = &selected
analyzeRuns := 0
plan := extractionLifecyclePlan(t, &analyzeRuns)
if _, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env}); err != nil {
t.Fatalf("initial executeStages() error = %v", err)
}
if len(runner.requests) != 1 || runner.requests[0].ConfigProfile == nil || *runner.requests[0].ConfigProfile != selected || analyzeRuns != 1 {
if len(runner.requests) != 1 || runner.requests[0].ConfigOverlay == nil || *runner.requests[0].ConfigOverlay != selected || analyzeRuns != 1 {
t.Fatalf("initial requests=%#v analyze=%d, want selected extraction and analysis", runner.requests, analyzeRuns)
}
reused, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env})
@@ -721,12 +721,12 @@ func TestExtractLifecycleConfigProfileChangesRerunExtractionAndDependents(t *tes
}
changed := "experimental"
cfg.Pipeline.Notarius.ConfigProfile = &changed
cfg.Pipeline.Notarius.ConfigOverlay = &changed
rerun, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env})
if err != nil {
t.Fatalf("changed-profile executeStages() error = %v", err)
}
if len(rerun.Executed) != 2 || len(runner.requests) != 2 || runner.requests[1].ConfigProfile == nil || *runner.requests[1].ConfigProfile != changed || analyzeRuns != 2 {
if len(rerun.Executed) != 2 || len(runner.requests) != 2 || runner.requests[1].ConfigOverlay == nil || *runner.requests[1].ConfigOverlay != changed || analyzeRuns != 2 {
t.Fatalf("changed-profile summary=%#v requests=%#v analyze=%d", rerun, runner.requests, analyzeRuns)
}
after := loadLifecycleManifest(t, cfg)
@@ -741,12 +741,12 @@ func TestExtractLifecycleConfigProfileChangesRerunExtractionAndDependents(t *tes
t.Fatalf("changed-profile lifecycle render=%#v analyze=%#v publish=%#v", after.Stages["render"], after.Stages["analyze"], after.Stages["publish"])
}
cfg.Pipeline.Notarius.ConfigProfile = nil
cfg.Pipeline.Notarius.ConfigOverlay = nil
returned, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env})
if err != nil {
t.Fatalf("return-to-base executeStages() error = %v", err)
}
if len(returned.Executed) != 2 || len(runner.requests) != 3 || runner.requests[2].ConfigProfile != nil || analyzeRuns != 3 {
if len(returned.Executed) != 2 || len(runner.requests) != 3 || runner.requests[2].ConfigOverlay != nil || analyzeRuns != 3 {
t.Fatalf("return-to-base summary=%#v requests=%#v analyze=%d", returned, runner.requests, analyzeRuns)
}
final := loadLifecycleManifest(t, cfg)
@@ -755,10 +755,10 @@ func TestExtractLifecycleConfigProfileChangesRerunExtractionAndDependents(t *tes
}
}
func TestExtractLifecycleConfigProfileFailureClearsCurrentResultAndDependents(t *testing.T) {
func TestExtractLifecycleConfigOverlayFailureClearsCurrentResultAndDependents(t *testing.T) {
cfg, env, runner := extractionLifecycleFixture(t, true)
selected := "inexpensive"
cfg.Pipeline.Notarius.ConfigProfile = &selected
cfg.Pipeline.Notarius.ConfigOverlay = &selected
analyzeRuns := 0
plan := extractionLifecyclePlan(t, &analyzeRuns)
if _, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env}); err != nil {
@@ -768,7 +768,7 @@ func TestExtractLifecycleConfigProfileFailureClearsCurrentResultAndDependents(t
markLifecycleStageSucceeded(t, cfg, "publish")
changed := "missing-profile"
cfg.Pipeline.Notarius.ConfigProfile = &changed
cfg.Pipeline.Notarius.ConfigOverlay = &changed
runner.failuresRemaining = 1
if _, err := executeStages(context.Background(), cfg, plan, RunOptions{Env: env}); err == nil {
t.Fatal("changed-profile executeStages() error = nil, want downstream failure")
@@ -783,12 +783,12 @@ func TestExtractLifecycleConfigProfileFailureClearsCurrentResultAndDependents(t
t.Fatalf("failed changed-profile lifecycle trim=%#v render=%#v analyze=%#v publish=%#v analyzeRuns=%d",
failed.Stages["trim"], failed.Stages["render"], failed.Stages["analyze"], failed.Stages["publish"], analyzeRuns)
}
if len(runner.requests) != 2 || runner.requests[1].ConfigProfile == nil || *runner.requests[1].ConfigProfile != changed {
if len(runner.requests) != 2 || runner.requests[1].ConfigOverlay == nil || *runner.requests[1].ConfigOverlay != changed {
t.Fatalf("failed request = %#v, want changed selector", runner.requests)
}
}
func TestExtractLifecycleRealConfigProfilesSelectNotariusExplicitly(t *testing.T) {
func TestExtractLifecycleRealConfigOverlaysSelectNotariusExplicitly(t *testing.T) {
seed, env, runner := extractionLifecycleFixture(t, true)
workspace := seed.Pipeline.Workspace.Root
pipelinePath, campaignPath, sessionPath := writeValidConfigFiles(t, workspace)
@@ -822,7 +822,7 @@ func TestExtractLifecycleRealConfigProfilesSelectNotariusExplicitly(t *testing.T
t.Fatal(err)
}
for name, contents := range map[string]string{
"testing": "whisperx:\n language: fr\nnotarius:\n config_profile: downstream-economy\n",
"testing": "whisperx:\n language: fr\nnotarius:\n config_overlay: downstream-economy\n",
"narratio-only": "whisperx:\n language: de\n",
"narratio-only-two": "whisperx:\n language: es\n",
} {
@@ -848,7 +848,7 @@ func TestExtractLifecycleRealConfigProfilesSelectNotariusExplicitly(t *testing.T
if _, err := executeStages(context.Background(), testingConfig, plan, RunOptions{Env: env}); err != nil {
t.Fatalf("execute testing profile: %v", err)
}
if len(runner.requests) != 1 || runner.requests[0].ConfigProfile == nil || *runner.requests[0].ConfigProfile != "downstream-economy" {
if len(runner.requests) != 1 || runner.requests[0].ConfigOverlay == nil || *runner.requests[0].ConfigOverlay != "downstream-economy" {
t.Fatalf("testing profile request = %#v, want explicit differently named downstream selector", runner.requests)
}
@@ -857,7 +857,7 @@ func TestExtractLifecycleRealConfigProfilesSelectNotariusExplicitly(t *testing.T
if _, err := executeStages(context.Background(), narratioOnly, plan, RunOptions{Env: env}); err != nil {
t.Fatalf("execute Narratio-only profile: %v", err)
}
if len(runner.requests) != 2 || runner.requests[1].ConfigProfile != nil {
if len(runner.requests) != 2 || runner.requests[1].ConfigOverlay != nil {
t.Fatalf("Narratio-only profile request = %#v, want no automatic downstream selector", runner.requests)
}

2
internal/config/config.go
View File

@@ -305,7 +305,7 @@ type NotariusConfig struct {
Enabled bool `yaml:"enabled" json:"enabled"`
Binary string `yaml:"binary" json:"binary"`
Credentials []configkit.Secret `yaml:"credentials,omitempty" json:"credentials,omitempty"`
ConfigProfile *string `yaml:"config_profile,omitempty" json:"config_profile,omitempty"`
ConfigOverlay *string `yaml:"config_overlay,omitempty" json:"config_overlay,omitempty"`
Resume bool `yaml:"resume,omitempty" json:"resume,omitempty"`
ChunkCache string `yaml:"chunk_cache,omitempty" json:"chunk_cache,omitempty"`
ConfigPath string `yaml:"config_path" json:"config_path"`

4
internal/config/credential_projection.go
View File

@@ -54,7 +54,7 @@ func (cfg NotariusConfig) MarshalYAML() (any, error) {
Enabled bool `yaml:"enabled"`
Binary string `yaml:"binary"`
Credentials []string `yaml:"credentials,omitempty"`
ConfigProfile *string `yaml:"config_profile,omitempty"`
ConfigOverlay *string `yaml:"config_overlay,omitempty"`
Resume bool `yaml:"resume,omitempty"`
ChunkCache string `yaml:"chunk_cache,omitempty"`
ConfigPath string `yaml:"config_path"`
@@ -66,7 +66,7 @@ func (cfg NotariusConfig) MarshalYAML() (any, error) {
}
return safeNotarius{
Enabled: cfg.Enabled, Binary: cfg.Binary, Credentials: secretReferenceNames(cfg.Credentials),
ConfigProfile: cfg.ConfigProfile,
ConfigOverlay: cfg.ConfigOverlay,
Resume: cfg.Resume, ChunkCache: cfg.ChunkCache,
ConfigPath: cfg.ConfigPath, PipelineID: cfg.PipelineID, Timeout: cfg.Timeout,
WorkingDirectory: cfg.WorkingDirectory, References: cfg.References, Outputs: cfg.Outputs,

12
internal/config/load_validate_test.go
View File

@@ -103,7 +103,7 @@ storage:
notarius:
enabled: false
credentials: [NARRATIO_TEST_NOTARIUS_KEY]
config_profile: ""
config_overlay: ""
`
pipelinePath, _ := writeConfigFiles(t, pipelineYAML, testSessionBaseYAML)
@@ -141,20 +141,20 @@ notarius:
}
}
}
if !strings.Contains(string(firstYAML), "config_profile: \"\"\n") {
t.Fatalf("effective YAML omitted explicit empty Notarius config profile: %q", firstYAML)
if !strings.Contains(string(firstYAML), "config_overlay: \"\"\n") {
t.Fatalf("effective YAML omitted explicit empty Notarius config overlay: %q", firstYAML)
}
var roundTrip struct {
Notarius *struct {
Credentials []string `yaml:"credentials"`
ConfigProfile *string `yaml:"config_profile"`
ConfigOverlay *string `yaml:"config_overlay"`
} `yaml:"notarius"`
}
if err := yaml.Unmarshal(firstYAML, &roundTrip); err != nil {
t.Fatalf("round-trip effective YAML: %v", err)
}
if roundTrip.Notarius == nil || roundTrip.Notarius.ConfigProfile == nil || *roundTrip.Notarius.ConfigProfile != "" {
t.Fatalf("round-trip config profile = %#v, want explicit empty", roundTrip.Notarius)
if roundTrip.Notarius == nil || roundTrip.Notarius.ConfigOverlay == nil || *roundTrip.Notarius.ConfigOverlay != "" {
t.Fatalf("round-trip config overlay = %#v, want explicit empty", roundTrip.Notarius)
}
if len(roundTrip.Notarius.Credentials) != 1 || roundTrip.Notarius.Credentials[0] != "NARRATIO_TEST_NOTARIUS_KEY" {
t.Fatalf("round-trip credential reference = %#v, want safe public name", roundTrip.Notarius.Credentials)

19
internal/config/pipeline_configkit_test.go
View File

@@ -238,7 +238,14 @@ func TestNotariusCredentialOverlayReplacementAndProvenance(t *testing.T) {
}
}
func TestNotariusConfigProfileComposition(t *testing.T) {
func TestNotariusRetiredConfigProfileRejected(t *testing.T) {
root := writePipelineFixture(t, t.TempDir(), "pipeline.yml", "whisperx:\n transcribe_url: https://transcription.example.test/transcribe\nnotarius:\n config_profile: testing\n")
if _, err := LoadPipelineWithOptions(root, PipelineLoadOptions{}); err == nil {
t.Fatal("retired Notarius config_profile field was accepted")
}
}
func TestNotariusConfigOverlayComposition(t *testing.T) {
tests := []struct {
name string
baseValue string
@@ -250,15 +257,15 @@ func TestNotariusConfigProfileComposition(t *testing.T) {
{name: "explicit empty", baseValue: `""`, want: stringPointer("")},
{name: "explicit default", baseValue: "default", want: stringPointer("default")},
{name: "base named selection", baseValue: "production", want: stringPointer("production")},
{name: "profile replaces named selection", baseValue: "production", overlayName: "testing", overlay: "notarius:\n config_profile: inexpensive\n", want: stringPointer("inexpensive")},
{name: "profile clears named selection", baseValue: "production", overlayName: "testing", overlay: "notarius:\n config_profile: null\n"},
{name: "profile replaces named selection", baseValue: "production", overlayName: "testing", overlay: "notarius:\n config_overlay: inexpensive\n", want: stringPointer("inexpensive")},
{name: "profile clears named selection", baseValue: "production", overlayName: "testing", overlay: "notarius:\n config_overlay: null\n"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
dir := t.TempDir()
rootYAML := "whisperx:\n transcribe_url: https://transcription.example.test/transcribe\nnotarius:\n enabled: false\n"
if test.baseValue != "" {
rootYAML += " config_profile: " + test.baseValue + "\n"
rootYAML += " config_overlay: " + test.baseValue + "\n"
}
root := writePipelineFixture(t, dir, "pipeline.yml", rootYAML)
options := PipelineLoadOptions{}
@@ -270,8 +277,8 @@ func TestNotariusConfigProfileComposition(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if cfg.Notarius == nil || !reflect.DeepEqual(cfg.Notarius.ConfigProfile, test.want) {
t.Fatalf("config profile = %#v, want %#v", cfg.Notarius.ConfigProfile, test.want)
if cfg.Notarius == nil || !reflect.DeepEqual(cfg.Notarius.ConfigOverlay, test.want) {
t.Fatalf("config overlay = %#v, want %#v", cfg.Notarius.ConfigOverlay, test.want)
}
})
}

6
internal/stage/extract.go
View File

@@ -146,7 +146,7 @@ func (extractStage) Run(ctx context.Context, env *Env, m *manifest.Manifest) (re
chunkCache = "refresh"
}
request := notarius.RunRequest{
Binary: resolvedBinary, Credentials: notariusConfig.Credentials, ConfigProfile: notariusConfig.ConfigProfile,
Binary: resolvedBinary, Credentials: notariusConfig.Credentials, ConfigOverlay: notariusConfig.ConfigOverlay,
ConfigPath: configPath, PipelineID: notariusConfig.PipelineID,
InputPath: inputPath, OutputRoot: outputRoot, WorkingDirectory: workingDirectory,
ReceiptPath: receiptPath, LogPath: logPath, Timeout: timeout, Resume: resume, ChunkCache: chunkCache, References: referenceBindings,
@@ -451,7 +451,7 @@ type fingerprintOutput struct {
type fingerprintDocument struct {
PipelineID string `json:"pipeline_id"`
ConfigProfile string `json:"config_profile,omitempty"`
ConfigOverlay string `json:"config_overlay,omitempty"`
Input artifacts.ExtractionInputIdentity `json:"input"`
References []extractReferenceIdentity `json:"references"`
Outputs []fingerprintOutput `json:"outputs"`
@@ -486,7 +486,7 @@ func extractionFingerprint(
return sortedReferences[i].Path < sortedReferences[j].Path
})
payload, err := json.Marshal(fingerprintDocument{
PipelineID: cfg.PipelineID, ConfigProfile: notariusConfigProfileReuseIdentity(cfg.ConfigProfile),
PipelineID: cfg.PipelineID, ConfigOverlay: notariusConfigOverlayReuseIdentity(cfg.ConfigOverlay),
Input: input, References: sortedReferences, Outputs: outputs,
})
if err != nil {

26
internal/stage/extract_test.go
View File

@@ -72,8 +72,8 @@ notarius:
env, m, fake := setupExtractEnv(t)
env.Config.Pipeline.Notarius.Credentials = loadedPipeline.Notarius.Credentials
configProfile := "inexpensive"
env.Config.Pipeline.Notarius.ConfigProfile = &configProfile
configOverlay := "inexpensive"
env.Config.Pipeline.Notarius.ConfigOverlay = &configOverlay
result, err := (extractStage{}).Run(context.Background(), env, m)
if err != nil {
t.Fatalf("Run() error = %v", err)
@@ -106,8 +106,8 @@ notarius:
if len(req.Credentials) != 1 || req.Credentials[0].EnvName() != credentialName || !req.Credentials[0].IsResolved() {
t.Fatal("request did not retain the loaded Notarius credential reference")
}
if req.ConfigProfile == nil || *req.ConfigProfile != configProfile {
t.Fatalf("request config profile = %#v, want %q", req.ConfigProfile, configProfile)
if req.ConfigOverlay == nil || *req.ConfigOverlay != configOverlay {
t.Fatalf("request config overlay = %#v, want %q", req.ConfigOverlay, configOverlay)
}
if result.Metadata["reference_count"] != 0 || len(result.Metadata["references"].([]map[string]any)) != 0 {
t.Fatalf("reference metadata = count %#v references %#v", result.Metadata["reference_count"], result.Metadata["references"])
@@ -141,8 +141,8 @@ func TestExtractStageAppliesNotariusCacheControlsAndForcePrecedence(t *testing.T
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
env, m, fake := setupExtractEnv(t)
configProfile := "inexpensive"
env.Config.Pipeline.Notarius.ConfigProfile = &configProfile
configOverlay := "inexpensive"
env.Config.Pipeline.Notarius.ConfigOverlay = &configOverlay
env.Config.Pipeline.Notarius.Resume = test.configuredResume
env.Config.Pipeline.Notarius.ChunkCache = test.configuredMode
env.Force = test.force
@@ -157,8 +157,8 @@ func TestExtractStageAppliesNotariusCacheControlsAndForcePrecedence(t *testing.T
if request.Resume != test.wantResume || request.ChunkCache != test.wantMode {
t.Fatalf("cache controls = resume %v, mode %q; want resume %v, mode %q", request.Resume, request.ChunkCache, test.wantResume, test.wantMode)
}
if request.ConfigProfile == nil || *request.ConfigProfile != configProfile {
t.Fatalf("config profile = %#v, want %q", request.ConfigProfile, configProfile)
if request.ConfigOverlay == nil || *request.ConfigOverlay != configOverlay {
t.Fatalf("config overlay = %#v, want %q", request.ConfigOverlay, configOverlay)
}
})
}
@@ -1018,7 +1018,7 @@ func TestExtractionFingerprintBaseSelectorsPreservePriorPayloadShape(t *testing.
legacyDigest := sha256.Sum256(legacyPayload)
want := hex.EncodeToString(legacyDigest[:])
for _, profile := range []*string{nil, stringPointerForExtractTest(""), stringPointerForExtractTest("default")} {
cfg.ConfigProfile = profile
cfg.ConfigOverlay = profile
got, err := extractionFingerprint(cfg, input, references)
if err != nil {
t.Fatal(err)
@@ -1027,7 +1027,7 @@ func TestExtractionFingerprintBaseSelectorsPreservePriorPayloadShape(t *testing.
t.Fatalf("base selection fingerprint = %q, want prior no-profile payload digest %q", got, want)
}
}
cfg.ConfigProfile = stringPointerForExtractTest("inexpensive")
cfg.ConfigOverlay = stringPointerForExtractTest("inexpensive")
named, err := extractionFingerprint(cfg, input, references)
if err != nil {
t.Fatal(err)
@@ -1055,10 +1055,10 @@ func TestExtractStageResumeValidationAcceptsCurrentImmutableResult(t *testing.T)
}
}
func TestExtractStageResumeValidationTracksConfigProfile(t *testing.T) {
func TestExtractStageResumeValidationTracksConfigOverlay(t *testing.T) {
env, m, _ := setupExtractEnv(t)
selected := "inexpensive"
env.Config.Pipeline.Notarius.ConfigProfile = &selected
env.Config.Pipeline.Notarius.ConfigOverlay = &selected
seedSucceededExtractResult(t, env, m)
validation, err := (extractStage{}).ValidateResume(context.Background(), env, m)
@@ -1070,7 +1070,7 @@ func TestExtractStageResumeValidationTracksConfigProfile(t *testing.T) {
}
changed := "experimental"
env.Config.Pipeline.Notarius.ConfigProfile = &changed
env.Config.Pipeline.Notarius.ConfigOverlay = &changed
validation, err = (extractStage{}).ValidateResume(context.Background(), env, m)
if err != nil {
t.Fatalf("ValidateResume(changed selection) error = %v", err)

15
internal/stage/notarius_cache_integration_test.go
View File

@@ -27,7 +27,7 @@ const notariusIntegrationBinaryEnv = "NARRATIO_TEST_NOTARIUS_BINARY"
func TestNotariusCacheReuseIntegration(t *testing.T) {
binary := strings.TrimSpace(os.Getenv(notariusIntegrationBinaryEnv))
if binary == "" {
t.Skipf("set %s to an absolute Notarius v0.7.0-compatible executable to run this integration test", notariusIntegrationBinaryEnv)
t.Skipf("set %s to an absolute Notarius v0.8.1-compatible executable to run this integration test", notariusIntegrationBinaryEnv)
}
if !filepath.IsAbs(binary) {
t.Fatalf("%s = %q, want an absolute path", notariusIntegrationBinaryEnv, binary)
@@ -235,6 +235,16 @@ pipelines:
if err := os.WriteFile(configPath, []byte(content), 0o600); err != nil {
t.Fatalf("write Notarius integration config: %v", err)
}
// Select a real downstream overlay. Disable provider retries so the controlled
// failure below tests checkpoint recovery rather than transient HTTP recovery.
overlay := "integration"
overlayDir := filepath.Join(root, "overlays")
if err := os.MkdirAll(overlayDir, 0o700); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(overlayDir, overlay+".yml"), []byte("promptkit:\n transient_retries: 0\n"), 0o600); err != nil {
t.Fatal(err)
}
transcriptPath := filepath.Join(sessionPathsForEnv(env, runManifest.SessionID).ArtifactsDir, "trimmed.from-manifest.json")
setNotariusIntegrationTranscript(t, runManifest, transcriptPath, notariusIntegrationTranscript("session-cache", "Aria casts Cure Wounds."))
@@ -242,7 +252,8 @@ pipelines:
env.Notarius = notarius.NewSubprocessRunner()
env.Config.Pipeline.Notarius = &config.NotariusConfig{
Enabled: true, Binary: binary, ConfigPath: configPath, PipelineID: pipelineID,
Timeout: "2m", WorkingDirectory: root, ChunkCache: "auto",
ConfigOverlay: &overlay,
Timeout: "2m", WorkingDirectory: root, ChunkCache: "auto",
References: map[string]string{"party": artifactpolicy.SourceInputParty},
Outputs: map[string]config.NotariusOutputConfig{
"spells_first": {

10
internal/stage/semantic_contracts_delivery.go
View File

@@ -22,16 +22,16 @@ const (
type extractSemanticConfig struct {
Enabled bool `json:"enabled"`
PipelineID string `json:"pipeline_id,omitempty"`
ConfigProfile string `json:"config_profile,omitempty"`
ConfigOverlay string `json:"config_overlay,omitempty"`
References []extractSemanticReference `json:"references"`
Outputs []extractSemanticOutput `json:"outputs"`
}
func notariusConfigProfileReuseIdentity(profile *string) string {
if profile == nil || *profile == "" || *profile == "default" {
func notariusConfigOverlayReuseIdentity(overlay *string) string {
if overlay == nil || *overlay == "" || *overlay == "default" {
return ""
}
return *profile
return *overlay
}
type extractSemanticReference struct {
@@ -85,7 +85,7 @@ func (extractStage) SemanticConfigFingerprint(env *Env) (manifest.SemanticConfig
}
payload.Enabled = true
payload.PipelineID = strings.TrimSpace(cfg.PipelineID)
payload.ConfigProfile = notariusConfigProfileReuseIdentity(cfg.ConfigProfile)
payload.ConfigOverlay = notariusConfigOverlayReuseIdentity(cfg.ConfigOverlay)
for rawSelector, rawSourceID := range cfg.References {
selector, err := notariusref.NormalizeSelector(rawSelector)
if err != nil {

10
internal/stage/semantic_contracts_delivery_test.go
View File

@@ -102,7 +102,7 @@ func TestExtractSemanticConfigSensitivityAndOperationalExclusions(t *testing.T)
baseFingerprints := make([]manifest.SemanticConfigFingerprint, 0, 3)
for _, profile := range []*string{nil, profilePointer(""), profilePointer("default")} {
env := downstreamSemanticEnv()
env.Config.Pipeline.Notarius.ConfigProfile = profile
env.Config.Pipeline.Notarius.ConfigOverlay = profile
fingerprint, err := (extractStage{}).SemanticConfigFingerprint(env)
if err != nil {
t.Fatal(err)
@@ -115,7 +115,7 @@ func TestExtractSemanticConfigSensitivityAndOperationalExclusions(t *testing.T)
}
}
profiled := downstreamSemanticEnv()
profiled.Config.Pipeline.Notarius.ConfigProfile = profilePointer("inexpensive")
profiled.Config.Pipeline.Notarius.ConfigOverlay = profilePointer("inexpensive")
firstProfile, err := (extractStage{}).SemanticConfigFingerprint(profiled)
if err != nil {
t.Fatal(err)
@@ -130,7 +130,7 @@ func TestExtractSemanticConfigSensitivityAndOperationalExclusions(t *testing.T)
if !firstProfile.Equal(unchangedProfile) {
t.Fatal("unchanged Notarius selection changed semantic fingerprint")
}
profiled.Config.Pipeline.Notarius.ConfigProfile = profilePointer("experimental")
profiled.Config.Pipeline.Notarius.ConfigOverlay = profilePointer("experimental")
secondProfile, err := (extractStage{}).SemanticConfigFingerprint(profiled)
if err != nil {
t.Fatal(err)
@@ -138,7 +138,7 @@ func TestExtractSemanticConfigSensitivityAndOperationalExclusions(t *testing.T)
if firstProfile.Equal(secondProfile) {
t.Fatal("named to different named Notarius selection retained semantic fingerprint")
}
profiled.Config.Pipeline.Notarius.ConfigProfile = nil
profiled.Config.Pipeline.Notarius.ConfigOverlay = nil
returnedToBase, err := (extractStage{}).SemanticConfigFingerprint(profiled)
if err != nil {
t.Fatal(err)
@@ -153,7 +153,7 @@ func TestExtractSemanticConfigSensitivityAndOperationalExclusions(t *testing.T)
disabledBase.Config.Pipeline.Notarius.Enabled = false
disabledNamed := downstreamSemanticEnv()
disabledNamed.Config.Pipeline.Notarius.Enabled = false
disabledNamed.Config.Pipeline.Notarius.ConfigProfile = profilePointer("ignored")
disabledNamed.Config.Pipeline.Notarius.ConfigOverlay = profilePointer("ignored")
disabledBaseFingerprint, err := (extractStage{}).SemanticConfigFingerprint(disabledBase)
if err != nil {
t.Fatal(err)