Adopt ConfigKit overlays for pipeline selection

This commit is contained in:
2026-09-25 15:24:03 +00:00
parent e27c49e946
commit 83eaa960ec
38 changed files with 394 additions and 320 deletions

View File

@@ -44,7 +44,7 @@ Most session-aware commands accept:
- `--session <session.yml>`
- `--session-id <session_id>`
- `--previous-session-id <session_id>`
- `--profile <name>`
- `--overlay <name>`
- `--secrets-dir <directory>`
Rules:
@@ -54,9 +54,9 @@ Rules:
- if both positional `<session_id>` and `--session-id` are provided, values must match.
- `--previous-session-id` is a strict expectation: the selected session file
must contain the same `previous_session_id`.
- `--profile` selects `profiles/<name>.yml` relative to the pipeline root. It
- `--overlay` selects `overlays/<name>.yml` relative to the pipeline root. It
may be supplied once; omitted, empty, and the exact name `default` select the
base configuration, while an unknown named profile fails. The same selection
base configuration, while an unknown named overlay fails. The same selection
applies to all common-flag commands, including `regenerate-artifacts`.
- `--secrets-dir` selects ConfigKit credential-file bootstrap explicitly. A
relative value uses caller CWD. Without the flag, a present `secrets/`
@@ -85,14 +85,14 @@ Commands with additional positionals keep their command-specific order:
### `config validate`, `config show`, `config sources`, `config artifacts`, and `config diff`
```bash
narratio config validate [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--profile <name>] [--secrets-dir <directory>]
narratio config show [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--profile <name>] [--secrets-dir <directory>]
narratio config sources [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--profile <name>] [--secrets-dir <directory>]
narratio config artifacts [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--profile <name>] [--secrets-dir <directory>]
narratio config diff <left-profile> <right-profile> [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--secrets-dir <directory>]
narratio config validate [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--overlay <name>] [--secrets-dir <directory>]
narratio config show [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--overlay <name>] [--secrets-dir <directory>]
narratio config sources [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--overlay <name>] [--secrets-dir <directory>]
narratio config artifacts [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--overlay <name>] [--secrets-dir <directory>]
narratio config diff <left-overlay> <right-overlay> [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--secrets-dir <directory>]
```
These commands resolve the selected profile, defaults, ordinary paths, and—if
These commands resolve the selected overlay, defaults, ordinary paths, and—if
a campaign is selected—the campaign-owned party. They neither discover or load
a session nor create a workspace, manifest, run, lock, adapter, or remote
connection. Like every pipeline-loading command, they bootstrap the selected or
@@ -107,7 +107,7 @@ configured default campaign so Narratio can expand its concrete artifacts and
publish rules. `--campaign` and `--campaign-file` remain mutually exclusive.
Session, range, force, and artifact-execution flags are not accepted.
`config validate` writes a concise root-path, selected-profile (or `none`), and
`config validate` writes a concise root-path, selected-overlay (or `none`), and
effective-digest summary after successful complete validation. `config show`
writes one deterministic, secret-free YAML document containing defaulted and
expanded concrete configuration. It omits ConfigKit directives, artifact
@@ -115,9 +115,9 @@ family declarations, and runtime provenance.
`config sources` reports the same fully validated resolution without printing
effective values. Its header identifies the root, ordered imports, selected
profile and overlay, selected campaign, party source, and digest. The
overlay and its file, selected campaign, party source, and digest. The
remaining tab-separated records are sorted as `path`, `role`, and `source`.
Roles distinguish root, import, profile, centralized default, campaign, party,
Roles distinguish root, import, overlay, centralized default, campaign, party,
and generated family ownership. A generated party member has
one family record and one party record at the same logical path. The output
never reads or prints secret values.
@@ -133,14 +133,14 @@ party-driven families retain the campaign requirement. See
[Scriptorium Artifact Entries](./config.md#scriptorium-artifact-entries) for the
configuration layout.
`config diff` resolves both supplied profile names independently and compares
`config diff` resolves both supplied overlay names independently and compares
their fully resolved, secret-free effective mappings. It does
not accept `--profile`; the two positional names must be distinct, declared
profiles. When party-driven families are present, both profiles must resolve to
the same selected campaign and party. Use `--campaign-file` if profile-specific
not accept `--overlay`; the two positional names must be distinct, valid
overlays. When party-driven families are present, both overlays must resolve to
the same selected campaign and party. Use `--campaign-file` if overlay-specific
campaign configuration would otherwise select different files.
Equal profiles print `no differences`. Otherwise, sorted tab-separated records
Equal overlays print `no differences`. Otherwise, sorted tab-separated records
use one of these forms, with compact JSON values:
```text
@@ -151,7 +151,7 @@ changed <path> <left-value> <right-value>
Mappings are flattened to their logical field paths; lists remain one atomic
value. The command compares defaulted concrete artifacts and publish rules, not
profile names, source-file layout, or formatting. It succeeds when differences
overlay names, source-file layout, or formatting. It succeeds when differences
are found, making it suitable for review and migration checks.
### `version`
@@ -189,7 +189,7 @@ Behavior:
- writes the session manifest as the sole durable progress ledger. Each
invocation also creates its run-local directory and returns an immediate
command summary; neither creates another durable progress record.
- reports the resolved profile (or `none`) and effective configuration digest.
- reports the resolved overlay (or `none`) and effective configuration digest.
When `--artifacts` is present, the selected range must contain `analyze` or
`publish`. Either consumer is sufficient, including a one-stage range.
@@ -304,7 +304,7 @@ Uses the same inclusive bounds, endpoint validation, force scope, and artifact
selection contract as `run`. It validates config and prints run/skip decisions
for selected stages only without creating the local workdir or changing the
manifest. Resume-capable selected stages are checked against durable evidence.
The output includes the resolved profile (or `none`) and effective configuration
The output includes the resolved overlay (or `none`) and effective configuration
digest without writing provenance or any manifest state.
For `analyze`, the preview also lists explicit targets, prerequisite-only work,
execution order, and reusable current artifacts with concise reasons. These

View File

@@ -47,24 +47,24 @@ the durable copied session input.
`narratio config validate`, `narratio config show`, `narratio config sources`,
and `narratio config artifacts` use the same `--config`, `--campaign`,
`--campaign-file`, and
`--profile` selection rules as pipeline commands, but do not select, discover,
`--overlay` selection rules as pipeline commands, but do not select, discover,
or load a session. They resolve configured typed credential references as part
of the ordinary ConfigKit load but expose only their environment-variable names;
they do not create runtime state.
`narratio config diff <left-profile> <right-profile>` uses the same pipeline and
campaign selectors, resolves each named profile with an independent ConfigKit
load, and does not accept a separate `--profile` flag.
`narratio config diff <left-overlay> <right-overlay>` uses the same pipeline and
campaign selectors, resolves each named overlay with an independent ConfigKit
load, and does not accept a separate `--overlay` flag.
Campaign selection is optional only when the resolved pipeline has no
`scriptorium.artifact_families`. When families are declared, Narratio selects a
campaign through an explicit flag or `pipeline.campaigns.default_campaign_id`,
then parses the campaign-owned party and expands concrete artifacts and any
family publish rules before validation. `config validate` prints the resulting
root, profile, and effective digest. `config show` emits the normalized
root, overlay, and effective digest. `config show` emits the normalized
effective pipeline YAML, with defaults and concrete expansion included but
ConfigKit directives and family declarations omitted. This review projection is
not promised to be a reloadable authoring document. `config sources` prints a stable
source projection instead of effective values: root/import/profile/default
source projection instead of effective values: root/import/overlay/default
ownership plus campaign/party and generated-family records. Canonical derived
players trace to the party. `config artifacts` lists the concrete artifact map,
effective dependencies, and defining sources. The [CLI reference](cli.md#config-validate-config-show-config-sources-config-artifacts-and-config-diff)
@@ -83,7 +83,7 @@ difference.
Use the [production/testing bundle](../examples/production-testing/pipeline.yml)
as the complete copyable migration reference. Keep production settings in the
base, split them into explicit additive imports, and place testing differences
in the fixed `profiles/testing.yml` overlay. Put ordinary artifacts and families
in the fixed `overlays/testing.yml` overlay. Put ordinary artifacts and families
in the recommended one-definition files shown by that bundle. Convert campaign
rosters to [canonical party input](integrations/party.md), then express character
work as families. Inspect the result with `config validate`, `config show`,
@@ -108,11 +108,14 @@ remote state with an unsafe legacy identity must be migrated before use.
root-only `configkit.imports` list. Imported files contribute fields to one
logical pipeline document; they do not override fields supplied by the root
or another import.
- An omitted profile, an empty profile option, and the exact name `default`
- An omitted overlay, an empty overlay option, and the exact name `default`
select the base configuration. Any other name selects
`profiles/<name>.yml` relative to the pipeline root.
`overlays/<name>.yml` relative to the pipeline root. Move previously selected
`profiles/<name>.yml` files there; the old directory is not searched, and
`--profile` is no longer accepted for Narratio pipeline selection. Explicit
imports from a directory named `profiles` remain valid.
- Campaign, session, and canonical-party documents are strict single-source
ConfigKit schemas. They do not enable imports, profiles, or `configkit`
ConfigKit schemas. They do not enable imports, overlays, or `configkit`
directives.
- Configured timeout and retry-delay durations must be positive. An omitted
artifact timeout continues to inherit its configured Scriptorium timeout.
@@ -137,10 +140,10 @@ remote state with an unsafe legacy identity must be migrated before use.
- local (`audio_dir` or `audio_files`), or
- S3 (`audio_s3.prefix`).
### Pipeline imports and profiles
### Pipeline imports and overlays
Large pipeline configurations may be split into explicitly named fragments and
may use fixed-path profile overlays:
may use fixed-path selected overlays:
```yaml
configkit:
@@ -170,12 +173,12 @@ name the full field path and every source that claimed it. Narratio additionally
rejects an artifact or artifact-family key declared by more than one base file,
even when those files contribute disjoint children.
Profile names are case-sensitive and use ConfigKit's safe fixed-path naming
Overlay names are case-sensitive and use ConfigKit's safe fixed-path naming
rules. Only the selected overlay is opened; a malformed unselected overlay does
not affect a base or different-profile load. Maps merge recursively, overlay
not affect a base or different-overlay load. Maps merge recursively, overlay
scalars replace base scalars, and overlay lists replace base lists completely.
Explicit `false`, zero, empty strings, empty containers, and null values remain
explicit inputs and are not refilled by defaults. Profiles cannot inherit,
explicit inputs and are not refilled by defaults. Overlays cannot inherit,
stack, import files, or declare another ConfigKit control section.
During ConfigKit loading, Narratio applies fresh typed defaults, recorded
@@ -183,7 +186,7 @@ entry-level defaults and normalization, read-only file-local validation, and
ordinary path resolution before computing a deterministic effective
configuration digest. The digest represents the normalized, secret-free
runtime pipeline mapping; it excludes ConfigKit directives, source
provenance, profile identity, and raw environment secret values. Equivalent
provenance, overlay identity, and raw environment secret values. Equivalent
effective mappings therefore have the same digest regardless of how fields are
split among the root and imports.
@@ -400,9 +403,9 @@ empty string or `default`, which explicitly select Notarius's composed base
configuration. Notarius owns selector interpretation and validation; Narratio
does not restrict profile names, resolve profile files, or rewrite the value.
This selection is independent of Narratio's own `--profile` selection. A
This selection is independent of Narratio's own `--overlay` selection. A
Narratio overlay may explicitly select a differently named Notarius overlay,
but Narratio never forwards its own profile name automatically. It is also
but Narratio never forwards its own overlay name automatically. It is also
independent of `pipeline.notarius.llm_api_key`: an overlay may change either or
both fields according to the deployed Notarius configuration. See the
[Notarius integration contract](./integrations/notarius.md) for invocation and

View File

@@ -43,7 +43,7 @@ field acquire no additional version requirement. Omitted or null configuration
adds no flag. Supplied empty, `default`, and named selectors are forwarded
without normalization so Notarius can apply its own ConfigKit semantics. This
option selects a Notarius configuration overlay, not a PromptKit profile through
`--llm-profile`, and Narratio never substitutes its own selected profile name.
`--llm-profile`, and Narratio never substitutes its own selected overlay name.
A missing or invalid downstream overlay is an ordinary Notarius subprocess
failure: Narratio does not retry without the selector or fall back to another
profile.

View File

@@ -9,9 +9,9 @@ pipeline-loading boundary implemented by `internal/config`.
`LoadPipeline` assembles and validates a pipeline in this order:
1. Give ConfigKit a fresh typed `PipelineConfig` default value, the root file,
enabled imports, and the caller's profile selection. Omitted, empty, and
exact `default` selections use the base; named profiles use the fixed
`profiles/<name>.yml` convention.
enabled imports, and the caller's overlay selection. Omitted, empty, and
exact `default` selections use the base; named overlays use the fixed
`overlays/<name>.yml` convention.
2. ConfigKit parses the root and declared imports, additively merges base
sources, applies only the selected overlay, and strictly decodes the result.
3. The definition resolver applies recorded entry-level and inherited defaults,
@@ -45,7 +45,7 @@ a temporary downloaded session file therefore cannot invalidate the resolved
pipeline or campaign context.
Campaign and session files use independent ConfigKit definitions with imports
and profiles disabled. Campaign files normally load directly from their source;
and overlays disabled. Campaign files normally load directly from their source;
session files are first read through the 8 MiB bounded-read boundary because the
application must reject unresolved template placeholders before passing an exact
byte snapshot to ConfigKit. Local and remotely downloaded session bytes then use
@@ -53,8 +53,9 @@ the same typed loader and retain the existing post-load identity expectations,
stable-input precedence, and source-relative path semantics.
The application also has a separate read-only inspection resolver for `config
validate`, `config show`, `config sources`, and `config artifacts`. It uses the same production root/profile and
campaign selection functions, but never routes through session discovery,
validate`, `config show`, `config sources`, and `config artifacts`. It uses the
same production root, overlay, and campaign selection functions, but never routes
through session discovery,
remote-session download, adapter composition, workspace
initialization, manifest access, or cleanup. A pipeline with retained artifact
family declarations must resolve its selected campaign before ordinary pipeline
@@ -73,7 +74,7 @@ commands do not marshal runtime objects directly.
`EffectivePipelineSources` and `EffectiveCampaignSources` provide the separate
safe provenance projection for `config sources`. Pipeline ownership begins with
the complete logical field paths projected from the schema and classifies
each contributor as root, import, profile, or centralized default. The
each contributor as root, import, overlay, or centralized default. The
projection replaces generated concrete member paths with paired family and
canonical-party records, and does the same for generated publish rules.
Campaign records identify campaign-owned fields and party inputs; canonical
@@ -106,7 +107,7 @@ configured roster authority; player and session-party fields are unsupported.
## Canonical Party Domain
`ParseParty` is the package-owned boundary for parsing a party source. Its
single-source ConfigKit definition disables imports and profiles, requires
single-source ConfigKit definition disables imports and overlays, requires
`schema_version` through `VersionRule`, and strictly decodes the
character-keyed typed schema before applying Narratio's party-domain checks.
Successful metadata coordinates recover authored character-map order without a
@@ -144,14 +145,14 @@ not compatibility contracts.
## Diagnostics And Runtime Metadata
Syntax, duplicate-key, merge, profile, and schema failures include the
Syntax, duplicate-key, merge, overlay, and schema failures include the
relevant source name and full field path. Additive conflicts report every
claiming source so operators can repair the split without repeatedly
rediscovering additional conflicts.
The loaded pipeline retains private runtime metadata for the absolute root
path, ordered imports, selected profile name and selection source, selected
overlay, original ConfigKit metadata and source snapshots, effective digest,
path, ordered imports, selected overlay name, source, and path, original
ConfigKit metadata and source snapshots, effective digest,
and leaf ownership. Base
leaves retain their root/import owners, replaced leaves belong to the selected
overlay, and centrally supplied values use the synthetic `default` owner. This
@@ -163,7 +164,7 @@ the same typed safe projection used by show and diff. Runtime Notarius paths rem
execution, but the digest substitutes their normalized logical values captured
before root-relative resolution, so relocating an equivalent configuration
bundle does not change provenance. Because ConfigKit and resolution metadata
are private, the digest excludes source layout, profile name, and ownership.
are private, the digest excludes source layout, overlay name, and ownership.
Configuration stores credentials as ConfigKit typed references. A successful
load resolves their values inside the opaque typed boundary, while this safe
projection serializes only `EnvName()` identities. Credential-value changes are
@@ -177,7 +178,7 @@ for later runtime expansion.
## Test Surfaces
`pipeline_configkit_test.go` exercises real imports and fixed-path profiles,
`pipeline_configkit_test.go` exercises real imports and fixed-path overlays,
base selection, unselected-overlay isolation, source attribution, and Narratio's
unique artifact/family base-declaration rule. Existing focused configuration
tests protect defaults, explicit-value semantics, strict decoding, validation,

View File

@@ -23,9 +23,11 @@ portable opaque segments. Unsafe legacy identities are rejected with migration
guidance rather than being normalized into a different workspace or remote
namespace.
Decoding is strict at every typed manifest field. Unknown fields are rejected;
the schema version remains `narratio.session-manifest.v1` and there is no
compatibility reader that silently drops retired state. Cleanup progress uses
Decoding is strict at typed execution-state fields. Unknown informational fields
inside `effective_config` are ignored so older selection labels can be read;
unknown fields elsewhere are rejected. The schema version remains
`narratio.session-manifest.v1` and there is no compatibility reader that
silently drops retired execution state. Cleanup progress uses
only the cleanup entry in `stages`.
Every persisted session manifest has a non-nil `stages` map. Each entry is
@@ -306,10 +308,10 @@ after such a private content change.
The session manifest is the authoritative stage-progress ledger across
invocations. It retains the most recently resolved invocation's bounded
configuration provenance. It identifies the selected profile name and source
(`default` or `cli`) plus the effective configuration digest, but never a raw
secret or profile content. This provenance is informational: it does not
participate in stage resume or cache decisions. A profile change therefore
configuration provenance. It identifies the selected overlay name and source
(`explicit` when selected) plus the effective configuration digest, but never a raw
secret or overlay content. This provenance is informational: it does not
participate in stage resume or cache decisions. An overlay change therefore
invalidates only stages whose semantic configuration changed. When a private
external-tool model, module, prompt, or profile changes behind an unchanged
configured identifier, use `--force` for the affected work.

View File

@@ -70,19 +70,19 @@ narratio run 2026-04-04
narratio session status 2026-04-04
```
Run, plan, and status output identify the resolved pipeline profile (or `none`)
Run, plan, and status output identify the resolved pipeline overlay (or `none`)
and effective configuration digest. Status distinguishes the current resolved
value from the last value persisted in the session manifest, which helps
diagnose profile switches without changing resume authority.
diagnose overlay switches without changing resume authority.
Before switching an operational profile, compare its effective meaning with the
production base through `narratio config diff default <profile>`.
Before switching an operational overlay, compare its effective meaning with the
production base through `narratio config diff default <overlay>`.
The command is read-only and succeeds whether it finds differences or not. Its
sorted records describe defaulted, expanded concrete configuration—not source
file layout—so it can be used to review model, artifact, and publish changes
without creating a session or run. Use `narratio config artifacts` to review the
concrete artifact identities, definitions, outputs, and dependencies. Select the
same campaign explicitly when profiles could resolve different campaign paths;
same campaign explicitly when overlays could resolve different campaign paths;
see the [CLI reference](cli.md#config-validate-config-show-config-sources-config-artifacts-and-config-diff)
for syntax and record format.
@@ -284,17 +284,17 @@ intends to replace otherwise-current artifacts.
### Developing With A Notarius Configuration Profile
A Narratio development overlay can explicitly select a Notarius overlay without
requiring the names to match. For example, `profiles/testing.yml` may contain:
requiring the names to match. For example, `overlays/testing.yml` may contain:
```yaml
notarius:
config_profile: downstream-economy
```
Selecting Narratio's `testing` profile then supplies `downstream-economy` only
Selecting Narratio's `testing` overlay then supplies `downstream-economy` only
when extraction executes. Switching that explicit selector makes a recorded
extraction non-current and reruns it when `extract` is in the selected stage
range; the usual dependent invalidation then applies. Selecting a profile does
range; the usual dependent invalidation then applies. Selecting an overlay does
not create a separate workspace or retain a separate current extraction result.
Narratio cannot observe edits inside Notarius configuration behind an unchanged

View File

@@ -152,13 +152,13 @@ Configuration is strict, explicit, centralized, and operator-oriented.
- Empty configured values do not silently replace meaningful defaults.
- Validation rejects invalid configuration before stage execution where
practical.
- Root-owned imports and one selected profile resolve deterministically through
- Root-owned imports and one selected overlay resolve deterministically through
the configuration owner; commands do not implement their own merge rules.
- Canonical party rosters are campaign-owned. Their derived players projection
and concrete character-family artifacts are resolved before runtime stages
or adapters receive configuration.
- Resume uses stage- or artifact-owned semantic evidence for observable
result-affecting configuration; profile identity and an effective digest are
result-affecting configuration; overlay identity and an effective digest are
provenance, never blanket cache keys.
- Session templating remains narrow and deterministic rather than becoming a
general configuration language.

View File

@@ -1,6 +1,6 @@
# Implementation Plan: Audita v2 and Shared Provider Credentials
Status: Not started.
Status: In progress.
## Scope and working instructions
@@ -164,7 +164,12 @@ extraction fingerprints, references, force, resume, or chunk-cache behavior.
## Stage 1: Adopt ConfigKit v0.4.0 and pipeline overlays
Status: Not started.
Status: Complete (2026-09-25).
Verification: `go test ./internal/config ./internal/app ./internal/manifest`,
`go test ./...`, `go test ./internal/doccheck`,
`go test ./internal/config -run '^TestExamplesLoadAndValidate$'`, and
`git diff --check` passed.
Preparation: ConfigKit v0.4.0 release notes and selected-overlay migration guide
in `../ConfigKit/docs/`; Narratio `internal/config/pipeline_configkit.go`,

View File

@@ -102,7 +102,7 @@ Safe fix:
Relevant reference: [Canonical Party Input](./integrations/party.md).
## Unexpected imported or profile value
## Unexpected imported or overlay value
Symptom:
@@ -112,21 +112,21 @@ Symptom:
Diagnostics:
```bash
narratio config sources --config /path/pipeline.yml --profile testing
narratio config sources --config /path/pipeline.yml --overlay testing
```
Add `--campaign` or `--campaign-file` when the pipeline has party-driven
artifact families. The output identifies each effective logical field's root,
import, profile, default, campaign, party, or family source without printing
import, overlay, default, campaign, party, or family source without printing
the field value or credential contents.
Safe fix:
- move a duplicated base field so it has one owner;
- correct the selected profile or its overlay; or
- correct the selected overlay; or
- correct the campaign party/family declaration that owns generated values.
To review what would actually change before switching profiles, run `config
To review what would actually change before switching overlays, run `config
diff` with the same pipeline and campaign selectors. It compares normalized
effective values rather than YAML formatting or source-file layout.