Prepare version 0.7.0 and retire completed configuration plans

AGENTS.md

@@ -1,2 +1,7 @@
 Please review `docs/development.md` for initial orientation in this repository
 and follow its task-specific reading guide.
+
+For every planning and implementation task, follow the
+[`Implementation Discipline`](docs/policy/architecture.md#implementation-discipline):
+keep machinery proportionate, do not invent restrictions, and do not elevate
+incidental implementation details into repository policy.

docs/adr/0017-defer-cache-access-policy-to-deployment.md

@@ -0,0 +1,59 @@
+# ADR-0017: Defer cache access policy to the deployment
+
+**Status:** Accepted
+**Date:** 2026-09-11
+
+## Context
+
+Canonical chunk plans and invocation checkpoints are reconstructible cache
+state. They may include source-derived content, so access and retention matter,
+but Notarius cannot determine whether a deployment needs a private cache or a
+cache shared by a trusted group. The current application-enforced `0700` and
+`0600` modes prevent ordinary Unix provisioning from making that choice.
+
+[ADR-0005](0005-cache-canonical-chunk-plans-by-source.md) governs canonical
+chunk-plan reuse, while [ADR-0006](0006-separate-output-cache-and-debug-state.md)
+established the output, cache, and debug separation refined by
+[ADR-0007](0007-separate-checkpoint-recording-from-reuse.md). None of those
+decisions requires Notarius to own exact cache access modes.
+
+## Decision
+
+Deployments own cache access policy through provisioned directories, process
+umask, setgid inheritance, and default ACLs. Notarius requests conventional
+maximum creation modes for cache entries and leaves the kernel to select their
+effective permissions.
+
+Notarius continues to own confined paths, symlink rejection, validation,
+atomic publication, cleanup, and recovery. Those correctness boundaries are
+independent from deciding which accounts may access cache state.
+
+Source-derived cache contents can be sensitive. Operators own their access and
+retention policy, while API credentials remain outside cache payloads under the
+existing secret boundary. Existing cache inode modes are not migrated
+automatically.
+
+This decision does not supersede ADR-0005, ADR-0006, or ADR-0007.
+
+## Alternatives considered
+
+- Continue enforcing private `0700` directory and `0600` file modes. This
+  makes one deployment choice for every operator and prevents trusted-group
+  provisioning.
+- Add configurable Notarius permission modes or a private/group cache setting.
+  This preserves application ownership of a deployment concern and expands the
+  public configuration surface without a durable policy reason.
+- Recursively converge existing cache ownership, modes, or ACLs. This would
+  make upgrades mutate operator-managed state and cannot safely infer an
+  intended policy.
+
+## Consequences
+
+Operators can use a restrictive umask for private cache state or provision a
+setgid/default-ACL cache root for trusted-group access. They must deliberately
+recreate or administratively migrate pre-existing cache trees when their
+existing modes do not match the desired deployment policy.
+
+Notarius does not add user or group management, ACL mutation, permission
+configuration, or recursive cache migration. Output and debug permission
+contracts remain separate from this decision.

docs/adr/0018-adopt-configkit.md

@@ -0,0 +1,53 @@
+# ADR-0018: Adopt ConfigKit for configuration loading
+
+**Status:** Accepted
+**Date:** 2026-09-13
+
+## Context
+
+Notarius's former configuration boundary owned strict YAML parsing, input
+presence bookkeeping, defaults, normalization, environment overlays, and
+configuration inspection. Those general-purpose responsibilities had grown
+alongside application-specific pipeline and module semantics.
+
+ConfigKit v0.3.0 provides strict typed YAML loading, imports, configuration
+profiles, explicit CLI overrides, recorded defaults and normalization, safe
+inspection rendering, and released source-origin APIs. The application needs
+those source records to resolve filesystem paths according to the file that
+supplied a surviving declaration.
+
+## Decision
+
+Notarius will adopt the root public `gitea.maximumdirect.net/eric/ConfigKit`
+package at v0.3.0 for its version 5 configuration contract. ConfigKit owns
+source composition, strict decoding, input presence, metadata, and safe
+effective/explanation rendering. Notarius retains its typed document schema,
+field semantics, runtime conversion, catalog-dependent pipeline resolution,
+PromptKit profile handling, and physical run preparation.
+
+Version 5 is a deliberate breaking file-contract change. It uses canonical
+object module bindings and reference sources, and enables ConfigKit imports
+and configuration profiles. The selected configuration file's canonical
+directory is the base for its declared application paths; ConfigKit's released
+`Resolution.InputInfo` and `Resolution.Source` APIs provide their origins.
+
+`NOTARIUS_CONFIG` remains file discovery, while field-level operational
+environment overlays are retired. Explicit configuration-valued CLI flags are
+represented as ConfigKit overrides; invocation actions remain outside that
+boundary.
+
+## Alternatives considered
+
+- Continue extending the custom YAML loader. This preserves duplicated
+  general-purpose parsing, presence, composition, and inspection machinery.
+- Retain version 4 alongside version 5. This would leave a supported dual
+  configuration boundary and delay removal of the machinery being replaced.
+- Wrap ConfigKit in a compatibility pre-parser. This weakens ConfigKit's
+  strict contract and recreates the application-owned parser complexity.
+
+## Consequences
+
+The released application has one ConfigKit-backed loader and one version 5
+contract. Operator documentation and maintained examples move with the cutover.
+ConfigKit metadata is retained only at the configuration publication boundary;
+its types do not become module or runtime pipeline interfaces.

docs/cli.md

@@ -12,8 +12,8 @@ defined in [Operations](operations.md).
 notarius help
 notarius --version
 notarius run <pipeline-id> --input path/to/source.json [--json] [flags]
-notarius config validate [--config path/to/config.yml] [--pipeline pipeline-id] [--only lane-a,lane-b]
+notarius config validate [--config path/to/config.yml] [--config-profile name] [--pipeline pipeline-id] [--only lane-a,lane-b]
-notarius pipelines list [--config path/to/config.yml] [--json]
+notarius pipelines list [--config path/to/config.yml] [--config-profile name] [--json]
 ~~~
 
 Running Notarius without arguments, or with **help**, **--help**, or **-h**,
@@ -42,6 +42,7 @@ pipeline ID and **--input** are required.
 | Flag | Meaning |
 | --- | --- |
 | **--config path** | Use this configuration file. When omitted, configuration discovery applies; see [Configuration](config.md). |
+| **--config-profile name** | Select a ConfigKit configuration overlay. Empty and **default** select the composed base configuration. |
 | **--input path** | Source input file to process. Required. |
 | **--output-dir path** | Override the configured output root for this run. |
 | **--json** | Write the successful run-result receipt as JSON to standard output. |
@@ -154,10 +155,10 @@ OPENROUTER_API_KEY=your-api-key \
 ## config validate
 
 ~~~
-notarius config validate [--config path/to/config.yml] [--pipeline pipeline-id] [--only lane-a,lane-b]
+notarius config validate [--config path/to/config.yml] [--config-profile name] [--pipeline pipeline-id] [--only lane-a,lane-b]
 ~~~
 
-This command loads and validates a configuration. With **--pipeline**, it also
+This command loads and validates the selected composed configuration. With **--pipeline**, it also
 resolves that pipeline against the production module catalog. **--only** selects
 lanes during that resolution and requires **--pipeline**.
 
@@ -180,10 +181,32 @@ OPENROUTER_API_KEY=validation-placeholder \
 The placeholder in the second command is sufficient only for offline
 validation; it cannot run a provider-backed pipeline.
 
+## config effective
+
+~~~
+notarius config effective [--config path/to/config.yml] [--config-profile name]
+~~~
+
+This command writes the selected composed configuration as deterministic,
+redacted YAML. Standard output contains only that YAML: it does not validate a
+pipeline, prepare modules, materialize references, or contact a provider.
+
+## config explain
+
+~~~
+notarius config explain [--config path/to/config.yml] [--config-profile name] <json-pointer>
+~~~
+
+This command writes ConfigKit's redacted, human-readable explanation for one
+canonical JSON Pointer. Supply `""` explicitly to explain the root. A malformed
+pointer is a syntax error; a well-formed unknown path is a command failure.
+The explanation can describe a value displaced by a selected profile. Standard
+output contains only the explanation.
+
 ## pipelines list
 
 ~~~
-notarius pipelines list [--config path/to/config.yml] [--json]
+notarius pipelines list [--config path/to/config.yml] [--config-profile name] [--json]
 ~~~
 
 This command lists configured pipeline IDs in sorted order. By default, it
@@ -200,6 +223,10 @@ go run ./cmd/notarius pipelines list \
 Successful commands write their primary result to standard output. Warnings and
 errors are written to standard error.
 
+Configuration load errors include ConfigKit's field diagnostics and source
+coordinates, followed by a source-ID legend identifying the acquired files or
+source labels. This includes imported files and the selected configuration profile.
+
 For **run --json**, actionable process warnings remain on standard error and standard output is a
 machine-readable success result only. Syntax and runtime diagnostics remain on
 standard error. Parse the result only after the process exits with status 0.
@@ -211,6 +238,6 @@ standard error. Parse the result only after the process exits with status 0.
 | 2 | The command or flag syntax was invalid, including unknown commands, missing required arguments, invalid flag values, or invalid flag combinations. |
 
 The root help spellings are the supported help path. Invoking **--help** on
-**run**, **config validate**, or **pipelines list** is handled by the flag
+**run**, a **config** subcommand, or **pipelines list** is handled by the flag
 parser as a usage error: it writes an error to standard error and exits with
 status 2.

docs/config.md

@@ -1,7 +1,7 @@
 # Configuration
 
 This is the canonical reference for Notarius configuration. Configuration files
-are YAML and must declare version 4. They select pipelines and their modules;
+are YAML and must declare version 5. They select pipelines and their modules;
 the [CLI reference](cli.md) owns invocation syntax, and
 [Operations](operations.md) owns run-state procedures.
 
@@ -19,12 +19,15 @@ The command fails if none of these paths provides a configuration file.
 For configuration values, precedence is:
 
 1. built-in defaults;
-2. the selected YAML file;
+2. composed YAML sources, including imports;
-3. supported operational environment variables; and
+3. the selected ConfigKit profile; and
-4. the CLI run overrides that apply to a command.
+4. explicitly supplied configuration-valued run flags.
 
-Environment variables do not provide a second configuration schema. They only
+`--config-profile` selects a ConfigKit overlay from `profiles/<name>.yml`
-override the fields listed below.
+beneath the selected configuration root. Empty and `default` select no overlay.
+Configuration profiles are distinct from PromptKit LLM profiles selected by
+`llm_profile` or `--llm-profile`. The retired operational environment variables
+have no effect; `NOTARIUS_CONFIG` remains a source-discovery input.
 
 ## Maintained Examples
 
@@ -33,7 +36,8 @@ override the fields listed below.
 - [Complete D&D configuration](../examples/dnd-complete.config.yml) uses
   ordered steps, all implemented D&D lanes, generated references, state
   settings, bounded LLM concurrency, and the maintained
-  [operator profile](../examples/profiles/dnd-extraction.yml).
+  [LLM profile](../examples/llm-profiles/dnd-extraction.yml) and imports its
+  pipeline from [a fragment](../examples/pipelines/dnd-session.yml).
 
 Use these complete files as starting points rather than combining the
 illustrative fragments in this reference.
@@ -46,7 +50,7 @@ other than **version** is optional.
 
 | Field | Type | Default | Rules |
 | --- | --- | --- | --- |
-| **version** | integer | none | Required; must be 4. |
+| **version** | integer or quoted string | none | Required; must be 5. Maintained files use `version: 5`. |
 | **promptkit** | object | none | Profile source and optional local-backend configuration. |
 | **pipelines** | map | empty | Maps pipeline IDs to pipeline definitions. |
 | **concurrency** | object | see below | Global LLM and extraction limits. |
@@ -69,6 +73,22 @@ Built-in defaults are:
 
 An empty cache directory in YAML deliberately selects the corresponding
 per-user root. An explicit empty output or debug directory is invalid.
+Selecting a chunk-plan or checkpoint directory does not configure filesystem
+access policy. Cache permissions come from deployment provisioning, process
+umask, setgid inheritance, and default ACLs; see [Operations](operations.md#cache-provisioning).
+
+Every nonempty relative configured destination or reference path resolves from
+the canonical directory of the file that supplies its surviving declaration.
+This includes imported fragments and selected configuration profiles. Built-in
+relative destinations use the selected main configuration file's directory;
+explicit `--output-dir` and `--debug-dir` values use the invocation working
+directory. Absolute values remain unchanged.
+
+The root file can use ConfigKit imports to compose disjoint definitions. Imports
+and selected profiles remain under the configuration root; they do not merge or
+append a pipeline's `steps` sequence. A profile replaces sequences as defined
+by ConfigKit, so changing an individual ordered binding requires supplying the
+whole replacement sequence.
 
 ## PromptKit Profiles
 
@@ -76,7 +96,7 @@ The optional **promptkit** object selects one source of profile definitions and
 may register one conventional local OpenAI-compatible backend:
 
 ~~~yaml
-version: 4
+version: 5
 
 promptkit:
   profile_dir: ./profiles
@@ -94,17 +114,18 @@ promptkit:
 | **local_backend.endpoint** | string | Required when **local_backend** is present; absolute HTTP or HTTPS URL with a host. |
 | **local_backend.concurrency_limit** | integer | Optional non-negative limit; defaults to 0. |
 
-Set at most one of **profile_dir** and **profile_file**. Relative values use
+Set at most one of **profile_dir** and **profile_file**. Every nonempty relative
-the process working directory, not the configuration file's directory. The
+configured filesystem path is resolved from the canonical directory of the
-complete example's `./examples/profiles/dnd-extraction.yml` value is therefore
+file that supplies it. The complete example's
-valid when Notarius is launched from the repository root; use an absolute path
+`./llm-profiles/dnd-extraction.yml` value therefore resolves beside that example.
-for services and containers.
+Use absolute paths for services and containers when their deployment layout is
+not configuration-relative.
 
 An operator source is optional. For a requested ID, PromptKit checks the
 configured operator source first, then Notarius's embedded fallback profiles,
 then its own built-in catalog. A matching profile is complete: it replaces a
 lower-precedence definition rather than merging with it. The maintained
-[`dnd-extraction` operator profile](../examples/profiles/dnd-extraction.yml)
+[`dnd-extraction` LLM profile](../examples/llm-profiles/dnd-extraction.yml)
 is a secret-free deployment artifact; production, development, and local
 deployments can each provide a complete definition with that same workload ID.
 Use workload-oriented IDs for new profiles instead of model names.
@@ -147,30 +168,15 @@ requiring credential values. It rejects absent, malformed, or incompatible
 profiles before a run prepares modules. Credential availability is checked only
 when a generation is prepared.
 
-## Migrating Version 3 Configuration
+## Version 5 Migration
 
-Version 3 files are not decoded or rewritten. Change **version: 3** to
+Version 3 and version 4 files are rejected and are not rewritten at load time.
-**version: 4** and rename the top-level **scriptorium:** section to
+Use `version: 5`, rename the former `scriptorium` section to `promptkit`, and
-**promptkit:**. Version 4 decoding is strict, so a remaining **scriptorium**
+convert every module binding to an object containing `module`. Convert every
-field is rejected as unknown.
+external reference to `{path: ...}` and move any lane-level references to the
-
+corresponding extraction binding. Put durable operational values in YAML or
-## Operational Environment Variables
+use the documented explicit run flags; the former field-level environment
-
+variables are not read.
-These variables are applied after YAML values:
-
-| Variable | Overrides | Rules |
-| --- | --- | --- |
-| **NOTARIUS_TOTAL_LLM_CONCURRENCY** | **concurrency.total_llm** | Integer. |
-| **NOTARIUS_STAGE_WORKERS_EXTRACT** | **concurrency.stage_workers.extract** | Integer. |
-| **NOTARIUS_OUTPUT_DIR** | **output.directory** | Non-empty path. |
-| **NOTARIUS_CACHE_CHUNK_PLANS_MODE** | **cache.chunk_plans.mode** | **auto**, **bypass**, or **refresh**. |
-| **NOTARIUS_CACHE_CHUNK_PLANS_DIR** | **cache.chunk_plans.directory** | Non-empty path. |
-| **NOTARIUS_CACHE_CHECKPOINTS_DIR** | **cache.checkpoints.directory** | Non-empty path. |
-| **NOTARIUS_DEBUG_DIR** | **debug.directory** | Non-empty path. |
-
-Integer values are trimmed then parsed as base-10 integers. Directory and
-output values reject NUL characters. **NOTARIUS_CONFIG** participates only in
-configuration discovery.
 
 ## Concurrency, Output, Cache, And Debug
 
@@ -195,11 +201,14 @@ debug:
 **concurrency.total_llm** must be greater than zero. The only supported
 **concurrency.stage_workers** key is **extract**; its value must be from 1
 through **total_llm**. When omitted, it is recalculated from the effective
-**total_llm** after YAML and environment precedence.
+**total_llm** after imports, profile selection, and explicit ConfigKit
+overrides.
 
 **cache.chunk_plans.mode** accepts **auto**, **bypass**, or **refresh**.
 **cache.checkpoints.enabled** is a boolean. The CLI can override the output
 directory and chunk-plan mode for one run; see [CLI reference](cli.md#run).
+Cache directory fields select only locations; Notarius has no cache permission
+field or operational environment override.
 
 ## Pipelines
 
@@ -209,14 +218,20 @@ Each **pipelines** entry has a unique, non-empty ID and the following shape:
 pipelines:
   dnd-session:
     llm_profile: dnd-extraction
-    input: seriatim
+    input:
-    chunk: generic
+      module: seriatim
-    output: json
+    chunk:
+      module: generic
+    output:
+      module: json
     artifacts:
       spells:
-        extract: dnd/spells
+        extract:
-        merge: appendorder
+          module: dnd/spells
-        normalize: dnd/spells
+        merge:
+          module: appendorder
+        normalize:
+          module: dnd/spells
 ~~~
 
 | Field | Type | Default | Rules |
@@ -286,19 +301,20 @@ A lane has these fields:
 | **extract** | module binding | none | Required. |
 | **merge** | module binding | **appendorder** | Optional. |
 | **normalize** | module binding | **noop** | Optional. |
-| **references** | map | none | Supported compatibility alias for **extract.references**. |
+| **references** | map | none | Not supported at lane scope; place references on the eligible binding. |
 | **validators** | list | none | Non-empty lane-level lists are rejected. Set validator overrides on a binding instead. |
 
-The lane-level **references** alias remains accepted. When the alias and
+Lane-level **references** are not supported. Put a lane extraction reference
-**extract.references** bind the same slot, **extract.references** wins. Use
+under **extract.references**.
-the binding-local form in new configurations.
 
 ## Module Bindings And Validators
 
-Use a module key directly when no other binding fields are needed:
+Every module and validator binding uses an object, even when it has no fields
+beyond its module key:
 
 ~~~yaml
-input: seriatim
+input:
+  module: seriatim
 ~~~
 
 Use an object for fields:
@@ -309,7 +325,8 @@ extract:
   llm_profile: dnd-extraction
   retries: 2
   references:
-    spell_catalog: ./dnd-spell-catalog.json
+    spell_catalog:
+      path: ./dnd-spell-catalog.json
 ~~~
 
 | Binding field | Type | Default | Rules |
@@ -381,18 +398,20 @@ When enabled, it publishes the selected source-unit excerpt defined by the
 
 ## References And Ordered Handoffs
 
-Reference maps bind named slots that the selected target declares. A scalar is
+Reference maps bind named slots that the selected target declares. An external
-an external path. Pipeline-level maps accept only external paths; step-local
+reference is an object with `path`. Pipeline-level maps accept only external
-and binding-local maps may also select a normalized artifact from an earlier
+paths; step-local and binding-local maps may also select a normalized artifact
-step:
+from an earlier step:
 
 ~~~yaml
 steps:
   - id: describe-session
     artifacts:
       npc-registry:
-        extract: dnd/npc-registry
+        extract:
-        normalize: dnd/npc-registry
+          module: dnd/npc-registry
+        normalize:
+          module: dnd/npc-registry
   - id: extract-events
     references:
       npc_registry:
@@ -401,8 +420,10 @@ steps:
           lane: npc-registry
     artifacts:
       spells:
-        extract: dnd/spells
+        extract:
-        normalize: dnd/spells
+          module: dnd/spells
+        normalize:
+          module: dnd/spells
 ~~~
 
 An artifact selector contains only **step** and **lane**. The producer must be
@@ -417,9 +438,10 @@ bindings are then operational overrides of configured external paths; their
 pipeline, lane, and binding scopes and precedence are defined by the
 [CLI reference](cli.md#reference-selectors). A CLI file reference cannot
 replace a configured generated artifact handoff. Required slots must be bound
-after configuration and CLI reference actions are applied. Reference paths in
+after configuration and CLI reference actions are applied. A YAML reference
-YAML are resolved relative to the configuration file; CLI reference paths are
+path resolves from the file that supplies its surviving declaration, including
-resolved relative to the process working directory.
+an imported fragment or selected profile. CLI reference paths resolve relative
+to the process working directory.
 
 ### D&D Reference Slots
 
@@ -521,11 +543,11 @@ extract:
   module: dnd/scene-descriptions
   retries: 1
   validators:
-    - generic/valid_json
+    - module: generic/valid_json
-    - extract/dnd/scene-descriptions/shape
+    - module: extract/dnd/scene-descriptions/shape
-    - extract/dnd/scene-descriptions/source_refs
+    - module: extract/dnd/scene-descriptions/source_refs
-    - generic/valid_json_schema
+    - module: generic/valid_json_schema
-    - extract/dnd/scene-descriptions/source_relatedness
+    - module: extract/dnd/scene-descriptions/source_relatedness
     - module: extract/dnd/scene-descriptions/combat_semantics
       retries: 1
 ~~~
@@ -568,7 +590,16 @@ go run ./cmd/notarius config validate \
 ~~~
 
 Configuration validation rejects invalid YAML, unsupported fields, invalid
-defaults or environment overrides, incompatible module keys, unknown options,
+defaults or explicit CLI overrides, incompatible module keys, unknown options,
 invalid reference bindings, missing required reference slots, invalid validator
 overrides, and incompatible generated artifact handoffs. Use
 [pipelines list](cli.md#pipelines-list) to inspect configured IDs.
+
+## Inspection
+
+Use [config effective](cli.md#config-effective) to write the selected composed
+configuration as deterministic, redacted YAML. Use
+[config explain](cli.md#config-explain) with a canonical JSON Pointer to inspect
+the effective or historical provenance for one path, including a value displaced
+by a selected profile. Neither command resolves a pipeline, prepares modules,
+materializes references, or requires provider credentials.

docs/consumers/dnd-pipeline.md

@@ -23,16 +23,14 @@ A deployment must provide its own PromptKit profile and campaign reference
 files. Use absolute paths for service and subprocess deployments. In
 particular, observe these different resolution rules:
 
-- reference paths in YAML are resolved relative to the Notarius configuration
+- reference and PromptKit profile paths in YAML are resolved relative to the
-  file;
+  file that supplies their surviving declaration;
 - reference paths passed with `--reference` are resolved relative to the
-  Notarius process working directory; and
+  Notarius process working directory.
-- `promptkit.profile_file` is resolved relative to the Notarius process working
-  directory.
 
 Do not copy the repository example's relative profile path into a deployment
-without also controlling that working directory. The complete path and profile
+without preserving its configuration-relative layout. The complete path and
-rules are defined in [Configuration](../config.md).
+profile rules are defined in [Configuration](../config.md).
 
 Preflight the deployed configuration before processing sessions and whenever
 it changes:

docs/consumers/subprocess.md

@@ -15,9 +15,12 @@ configured lane inventory, and downstream acceptance checklist, see
 Optionally preflight a selected configuration and pipeline before work starts:
 
 ```sh
-notarius config validate --config /path/to/notarius.yml --pipeline pipeline-id
+notarius config validate --config /path/to/notarius.yml --config-profile production --pipeline pipeline-id
 ```
 
+Omit `--config-profile` when the composed base configuration is the intended
+selection; it is independent from any `--llm-profile` run override.
+
 Invoke the run with explicit paths and machine-readable output. Capture
 standard output and standard error separately; do not combine them before
 processing the result.
@@ -25,6 +28,7 @@ processing the result.
 ```sh
 notarius run pipeline-id \
   --config /path/to/notarius.yml \
+  --config-profile production \
   --input /path/to/source.json \
   --output-dir /path/to/output-root \
   --json

docs/integrations/pkg-configkit.md

@@ -0,0 +1,13 @@
+# ConfigKit Integration
+
+Notarius uses the public root package
+[`gitea.maximumdirect.net/eric/ConfigKit`](https://gitea.maximumdirect.net/eric/ConfigKit)
+at v0.3.0 for version 5 configuration acquisition, imports, profiles, explicit
+overrides, diagnostics, and safe inspection rendering.
+
+Notarius owns the typed document, domain defaults and validation, runtime
+conversion, catalog-dependent pipeline resolution, source-aware configured
+paths, and command discovery. It uses only ConfigKit's public path, result,
+diagnostic, and resolution-source APIs. Upstream documentation owns general
+ConfigKit semantics; [Configuration](../config.md) owns Notarius's file and
+operator contract.

docs/internal/cli.md

@@ -25,7 +25,8 @@ preparation, and runner mechanics after their inputs are supplied.
 ## Dispatch And Configuration Handoff
 
 The root dispatcher handles help, version reporting, configuration validation,
-pipeline listing, and a pipeline run. Version reporting resolves build
+effective-configuration inspection, explanation inspection, pipeline listing,
+and a pipeline run. Version reporting resolves build
 information through `internal/buildinfo` before production composition, so the
 diagnostic remains available without configuration or runtime collaborators.
 The public syntax, streams, exit classes, and version semantics are defined by
@@ -33,14 +34,15 @@ the [CLI reference](../cli.md). Other root commands normalize injectable
 options before dispatch so that a missing production dependency fails as a
 command error rather than reaching execution.
 
-Commands that need configuration use one shared loader. The CLI discovers the
+Commands that need configuration use one shared ConfigKit-backed loader. The
-file, parses it through **internal/core/config**, starts from defaults, applies
+CLI discovers the file, passes the requested profile and only explicit
-the file and supported environment overrides, and then validates it for the
+configuration-valued run overrides, and receives a **LoadedConfig**. The
-command. The configured discovery and precedence contract is in
+configured discovery and precedence contract is in
 [Configuration](../config.md), while the loading and resolution mechanics are
 in [Configuration Internals](configuration.md).
 
-Configuration validation without a selected pipeline checks structural
+`config effective` and `config explain` only render the loaded ConfigKit view;
+they do not resolve pipelines or construct providers. Configuration validation without a selected pipeline checks structural
 configuration only. Validation with a selected pipeline also builds the
 effective catalog, resolves the pipeline, and verifies every explicit effective
 PromptKit profile. Selected LLM-backed input, chunk, lane, output, and validator
@@ -84,8 +86,7 @@ field.
 After parsing and validating a run invocation, the CLI performs this ordered
 handoff:
 
-1. load and validate configuration, then apply command-level operational
+1. load and validate configuration with ConfigKit command overrides;
-   overrides;
 2. create and validate a safe run identity, then allocate a debug bundle only
    when requested;
 3. build the effective catalog, resolve requested reference changes, resolve

docs/internal/configuration.md

@@ -8,17 +8,17 @@ either interface.
 
 ## Boundary
 
-The configuration package turns a selected YAML file and supported environment
+The configuration package turns a selected ConfigKit-composed YAML document
-values into a validated, independently owned configuration. It then resolves a
+into a validated, independently owned configuration. It then resolves a
 requested pipeline against a module catalog before the framework prepares or
 runs anything.
 
 | Boundary | Inputs | Outputs | Does not own |
 | --- | --- | --- | --- |
-| Loading | Selected file path and environment lookup | Parsed file model and a populated **Config** | Choosing the file path or reporting a command result. |
+| Loading | Selected file path, profile, overrides, and invocation directory | **LoadedConfig** with ConfigKit inspection metadata and a populated **Config** | Choosing the file path or reporting a command result. |
 | Validation | **Config** | Structural configuration errors with pipeline, lane, or binding context | Module availability, capabilities, or construction. |
 | Resolution | Valid **Config**, selected pipeline and lanes, runtime reference changes, LLM override, and module catalog | **EffectiveConfig** with a **ResolvedPipeline** | Materializing reference bytes, preparing modules, execution, or filesystem state. |
-| Summary | **Config** or **EffectiveConfig** | Detached redacted payload suitable for debug summaries | Redacting arbitrary process state or provider traffic. |
+| Inspection | **LoadedConfig** and an optional JSON Pointer | Redacted effective YAML or explanation text | Pipeline selection, module preparation, or provider work. |
 
 The CLI discovers a configuration file, invokes this package, and supplies the
 result to the framework. Configuration never reads an input file, constructs a
@@ -28,23 +28,15 @@ at their respective [CLI](cli.md), [pipeline](pipeline.md), and
 
 ## Loading And Validation
 
-The CLI loads configuration in this order:
+ConfigKit acquires the selected root, imports, optional selected profile, and
-
+explicit CLI overrides in precedence order. Its document definition supplies
-1. parse the selected YAML file strictly into the file model;
+typed defaults, source-aware path normalization, and structural validation
-2. start from **Default**;
+before conversion to the independently owned runtime **Config**. Only
-3. apply the file model; and
+`NOTARIUS_CONFIG` participates in discovery; retired field-level environment
-4. apply the supported environment overrides.
+variables do not participate in loading. Version 5 decoding rejects scalar
-
+module bindings and scalar external references. ConfigKit retains redacted,
-This establishes the public precedence order without giving environment input a
+historical inspection metadata while **LoadedConfig** keeps that implementation
-second file schema. Loading and application reject malformed YAML, unsupported
+detail private behind effective and explanation writers.
-file versions, unknown fields, invalid values, and identifiers that are empty
-or collide after whitespace normalization. The file application also makes the
-effective extraction-worker default follow the effective LLM limit. A present
-PromptKit local-backend object requires and trims its endpoint, defaults its
-omitted concurrency limit to zero, and is copied so the parsed file model
-cannot alias the populated **Config**. A pipeline `llm_profile` is
-presence-aware: omission remains empty, while a present blank value is
-rejected and a non-empty file value is trimmed before it reaches **Config**.
 
 **Config.Validate** checks configuration-only invariants before resolution. It
 rejects incompatible profile sources, invalid state-surface values, unsupported
@@ -57,13 +49,14 @@ It deliberately does not require modules to be registered: that requires a
 catalog and belongs to resolution.
 
 The exact user-selectable values and validation rules are defined in
-[Configuration](../config.md). Keep additions to the file model, an
+[Configuration](../config.md). Keep document schema, source-aware normalization,
-environment override, its validation, and that reference in the same change.
+runtime conversion, and that reference aligned in the same change.
 
 ## Effective Resolution
 
-**Config.Resolve** first recomputes derived concurrency defaults and validates
+**Config.Resolve** derives an absent extraction-worker value in an owned copy
-the configuration. It normalizes the requested pipeline ID, copies the selected
+and validates the configuration. A programmatically supplied worker value is
+always explicit. It normalizes the requested pipeline ID, copies the selected
 profile, and passes the non-empty command-level LLM profile override, requested
 lane selection, and reference changes to the framework resolver.
 
@@ -110,43 +103,37 @@ bytes, a debug payload, or all runtime state. The CLI records it as invocation
 provenance before execution; cache and checkpoint identity have additional
 owners in [Run State Internals](state.md).
 
-Configuration summaries must use **Redacted**, **RedactedSummaryPayload**, or
+Configuration publication uses ConfigKit's redacted effective YAML. Resolved
-**RedactedResolvedPipelinePayload**, never a direct configuration marshal.
+pipeline publication uses **RedactedResolvedPipelinePayload**, which copies
-Those methods copy every binding and nested option container, replace values
+bindings and nested option containers, replaces credential-shaped values with
-whose key is credential-shaped with **[REDACTED]**, and omit materialized
+**[REDACTED]**, and omits materialized reference content while retaining safe
-reference content while retaining safe binding and reference provenance. The
+provenance. This redaction is deliberately narrow and does not authorize
-payload must not alias the source configuration or resolved pipeline.
+recording arbitrary environment values or provider requests.
-PromptKit's local endpoint and concurrency limit are preserved as non-secret
-configuration metadata in the independently owned summary; the object contains
-no credential value. This redaction is deliberately narrow: it protects
-configuration summaries and does not authorize recording arbitrary environment
-values or provider requests.
 
 ## Invariants To Preserve
 
-- Defaults, YAML values, and environment values are applied in one direction;
+- ConfigKit composition applies defaults, imports, an optional profile, and
-  later sources may override only their supported operational settings.
+  explicit overrides in one direction with retained origin metadata.
 - A configuration is structurally valid before it is resolved, and a resolved
   pipeline is compatible with the supplied catalog before preparation begins.
 - Whitespace-normalized identifiers are unique wherever they identify a
   pipeline, step, lane, worker, or reference slot.
-- Resolution and summary generation return detached data. Redaction must cover
+- Runtime conversion and resolved-pipeline publication return detached data.
-  every configured and resolved binding, including nested validator bindings.
 - The resolved digest changes when resolved composition changes and never
   includes itself.
 
 ## Focused Tests
 
-- **internal/core/config/file_config_contract_test.go** covers strict file
+- **internal/core/config/document_test.go** and
-  parsing, normalization, file application, and structural rejection.
+  **operational_document_test.go** cover ConfigKit document loading,
-- **internal/core/config/env_contract_test.go** covers supported operational
+  normalization, imports, profiles, and overrides.
-  overrides and their precedence.
 - **internal/core/config/validation_contract_test.go** covers configuration
   invariants and contextual failures.
 - **internal/core/config/effective_config_contract_test.go** covers defaults,
   selections, overrides, resolution context, digest changes, and ownership.
-- **internal/core/config/redaction_test.go** covers recursive credential
+- **internal/core/config/loaded_config_test.go** covers safe effective and
-  redaction, reference-content exclusion, and non-aliasing payloads.
+  explanation rendering; **redaction_test.go** covers resolved-pipeline
+  redaction and reference-content exclusion.
 
 Run **go test ./internal/core/config** after changing this boundary. Changes to
 the handoff or resolved-composition semantics also need the focused framework

docs/internal/overview.md

@@ -26,7 +26,7 @@ physical state roots.
 | --- | --- | --- |
 | Executable and command boundary | **cmd/notarius**, **internal/cli** | Process entry, command dispatch, configuration discovery, production composition, runtime collaborator setup, durable file placement, and user-facing reporting. |
 | Build information | **internal/buildinfo** | Resolves a stable linked release tag or build metadata for the diagnostic root version command. |
-| Configuration | **internal/core/config** | Defaults, strict YAML parsing, environment overrides, structural validation, effective resolution, redaction, and resolved-composition summaries. |
+| Configuration | **internal/core/config** | ConfigKit document definition, source-aware normalization, runtime conversion, structural validation, safe inspection, effective resolution, and resolved-composition redaction. |
 | Generic models | **internal/core/source**, **internal/core/artifacts**, **internal/framework/contracts** | Source documents and chunks, manifests and provenance, plus typed artifact, reference, validation, output, and structured-completion contracts. |
 | Pipeline framework | **internal/framework/pipeline** | Registries, profile and reference resolution, typed preparation, validation, retry coordination, ordered execution, handoff, and result assembly. |
 | LLM and prompt runtime | **internal/framework/llm**, **internal/framework/promptfs** | Provider-neutral structured completions, scheduling, profile recording, prompt assets, schema registration, and credential-shaped-value redaction. |

docs/internal/state.md

@@ -32,11 +32,18 @@ owned by the [Accepted Chunk Map contract](../integrations/chunk-map.md).
 
 `internal/framework/chunkplan` owns source-addressed plan storage, validation,
 and atomic publication. Its store is constructed only when the selected mode is
-not `bypass`.
+not `bypass`. It preserves rooted entry checks while creating new cache roots
+and source-digest directories with requested mode `0777`, then creates an
+atomic temporary file through rooted `O_CREATE|O_EXCL` with requested mode
+`0666` before sync and rename.
 
 `internal/framework/checkpoint` owns checkpoint identity, manifests, payload
 codecs, loader, and recorder. The CLI constructs a recorder whenever checkpoint
 recording is enabled and constructs a loader only for a `--resume` invocation.
+The recorder writes manifest and payload JSON through
+`fileio.WriteJSONWithProcessPermissions`, whose confined atomic writer creates
+new directories with requested mode `0777` and temporary files with requested
+mode `0666` before publication.
 Identity incorporates explicit stable semantic fingerprints collected from
 prepared modules and validators in addition to configuration, input,
 references, runtime overrides, observed LLM profiles, and the LLM runtime's
@@ -86,7 +93,10 @@ and debug-summary JSON.
 canonical operator-facing reason-code reference.
 
 `internal/core/fileio` provides confined atomic file writes used by state
-collaborators. The chunk-plan store retains its stronger entry validation.
+collaborators. Its exact-mode writer remains available to output and debug
+callers; its process-permission writer is used by checkpoints. The chunk-plan
+store retains its stronger rooted entry validation. [Operations](../operations.md#cache-provisioning)
+owns cache mode and provisioning guidance.
 
 The CLI constructs selective-recomputation policy from resolved generated
 artifact dependencies. It forces the selected step and transitive consumers,

docs/operations.md

@@ -50,11 +50,45 @@ directory selects a per-user root:
 
 The field definitions and configuration examples are in [Configuration](config.md).
 On supported Unix systems, output directories and files are created with
-requested modes **0755** and **0644**. Chunk-plan, checkpoint, and debug
+requested modes **0755** and **0644**; debug directories and files use
-directories and files use **0700** and **0600**. The operating system's umask
+**0700** and **0600**. The operating system's umask may impose stricter output
-may impose stricter output modes. Cache and debug roots may contain sensitive
+or debug modes. Chunk-plan and checkpoint caches instead request maximum modes
-source-derived data, so provision them for one trusted account or service. An
+of **0777** for new directories and **0666** for new files, including atomic
-output bundle can also contain source content when its JSON output enables
+temporary files. The kernel applies the service's umask and applicable default
+ACLs when those entries are created.
+
+## Cache Provisioning
+
+Cache locations select where Notarius writes, not who can access them. A
+private service can use a restrictive umask:
+
+~~~sh
+umask 0077
+notarius run example --config /etc/notarius/config.yml --input /srv/input.txt
+~~~
+
+A trusted-group deployment can provision a setgid cache root and use a
+group-oriented umask (or an equivalent default ACL):
+
+~~~sh
+install -d -m 2770 -g notarius-operators /srv/notarius/chunk-plans /srv/notarius/checkpoints
+umask 0007
+notarius run example --config /etc/notarius/config.yml --input /srv/input.txt
+~~~
+
+Setgid and default ACLs control inherited group ownership and access policy;
+Notarius does not copy a parent directory's literal mode bits. Cache state is
+reconstructible but can contain source-derived data, so operators own its
+access and retention policy. Existing cache inodes retain their prior modes
+after an upgrade. Recreate a cache beneath correctly provisioned parents, or
+stop writers and perform a deliberate administrative migration when a
+whole-tree change is required.
+
+Cache publication remains atomic and stored records remain validated. Those
+guarantees do not add cross-process locking or transactional coordination for
+simultaneous writers to one checkpoint identity.
+
+Output bundles can also contain source content when JSON output enables
 evidence publication. Apply an appropriate umask and output-root access policy
 before enabling that option; the requested output modes alone may not be
 suitable for transcript-bearing bundles.
@@ -81,7 +115,7 @@ precedence rules are owned by [Configuration](config.md#promptkit-profiles).
 Use a profile source owned by the service account, keep it readable only by
 the intended operator, and supply provider credentials through the service
 environment—not in the Notarius configuration or profile YAML. The maintained
-[operator profile](../examples/profiles/dnd-extraction.yml) is secret-free and
+[LLM profile](../examples/llm-profiles/dnd-extraction.yml) is secret-free and
 can be copied as a format starting point. Validate a deployment without a
 provider call or credentials:
 
@@ -92,11 +126,11 @@ notarius config validate --config /etc/notarius/config.yml --pipeline dnd-sessio
 An unset optional `api_key_env` reaches the provider without authorization and
 may receive a 401 or 403 response.
 
-Profile paths are currently resolved from the process working directory, not
+Configured PromptKit profile paths resolve from the file that supplies their
-from the configuration file. The complete example's
+surviving declaration. The complete example's
-`./examples/profiles/dnd-extraction.yml` path is valid for a repository-root
+`./llm-profiles/dnd-extraction.yml` path is therefore relative to that example.
-invocation only. Use absolute paths such as
+Use absolute paths such as `/etc/notarius/profiles/dnd-extraction.yml` for
-`/etc/notarius/profiles/dnd-extraction.yml` for services and containers.
+services and containers when that is clearer for the deployment.
 
 ## Run Lifecycle
 
@@ -274,7 +308,8 @@ Only a [debug-enabled run](cli.md#run) creates a bundle:
   trace/
 ~~~
 
-The summary contains redacted invocation and resolution information plus run,
+The summary contains redacted invocation and resolution information, including
+`effective-config.yml` and the separately redacted resolved pipeline, plus run,
 final grouped diagnostic, checkpoint, chunk-plan, and terminal reporting artifacts. Attempt
 terminal records contain bounded attempt kinds, validator outcomes, policy,
 decision, PromptKit repair count, and usage; they do not contain assistant
@@ -284,7 +319,8 @@ content. Neither surface is a cache input. Do not treat a debug bundle as safe
 to share merely because its configuration summary is redacted. Invocation
 metadata omits reasoning effort when it is inherited, records the replacement
 value when one is supplied, and records an empty value when inherited reasoning
-was explicitly cleared.
+was explicitly cleared. It records `config_profile` only when a non-base
+configuration overlay was selected.
 
 Notarius never creates debug state without an explicit request and never
 automatically deletes a requested bundle. If allocation succeeds, the command

docs/policy/architecture.md

@@ -38,6 +38,36 @@ Notarius is contract-first without being abstraction-heavy. Interfaces and
 extension points should protect demonstrated boundaries. New abstraction is not
 itself an architectural goal.
 
+## Implementation Discipline
+
+Simplify the mechanism, not the user's choices.
+
+**Keep implementation machinery proportionate.** Use straightforward,
+idiomatic code and introduce only the machinery needed to support the requested
+behavior and flexibility. Optimize for conceptual simplicity and
+maintainability, not for the fewest lines, fewest options, or narrowest product
+surface.
+
+**Do not invent restrictions.** Do not reject, prohibit, constrain, cap,
+harden, or otherwise narrow behavior unless required by the user, an explicit
+repository contract, or an unavoidable correctness constraint. Preserve
+operator and caller choice where policy has not been established. If an
+additional restriction appears advisable, present it as a decision rather than
+silently implementing it.
+
+**An implementation detail is not repository policy.** Tests and documentation
+must not elevate incidental behavior into a durable guarantee or restriction
+unless that behavior was expressly selected as part of the contract. Existing
+explicit contracts remain authoritative; changing or weakening one requires a
+deliberate decision rather than being inferred from this principle.
+
+[ADR-0017](../adr/0017-defer-cache-access-policy-to-deployment.md) illustrates
+these principles. Notarius previously elevated private cache creation modes
+into application policy even though Unix deployments already express access
+policy through directory permissions, process umask, setgid inheritance, and
+default ACLs. The revised design retains application-owned correctness
+boundaries while leaving deployment security policy to the operator.
+
 ## Layers And Dependency Direction
 
 The application boundary is the composition root and may depend on concrete
@@ -261,9 +291,11 @@ beneath this mandatory application-wide scheduler.
 
 ## Configuration And Provenance
 
-Configuration loading, precedence, defaults, environment overrides, redaction,
+Configuration loading, precedence, defaults, redaction, and validation are
-and validation are centralized. Named pipeline definitions make structural
+centralized through the ConfigKit-backed boundary. `NOTARIUS_CONFIG` is limited
-composition explicit and discoverable. Operational overrides are permitted
+to discovery; field-level environment overlays are not configuration input.
+Named pipeline definitions make structural composition explicit and discoverable.
+Explicit command overrides are permitted
 when they do not obscure the configured pipeline structure.
 
 Run preparation fails before stage execution when statically discoverable
@@ -293,6 +325,15 @@ an invocation that explicitly requests resume. Debug is never a cache input and
 is never created without an explicit request. Pipeline modules receive
 collaborator interfaces and never physical roots.
 
+Deployments own cache access policy. Cache writers request conventional maximum
+creation modes and let the process umask, setgid inheritance, and default ACLs
+determine effective access. Notarius owns the separate safety boundary:
+confined paths, symlink rejection, atomic publication, validation, and
+recovery. It does not chmod, chown, chgrp, or migrate cache entries to impose
+an application policy. This rule applies to chunk-plan and checkpoint cache
+only; output and debug retain their separate contracts. [Operations](../operations.md)
+owns deployment provisioning and retention guidance.
+
 Only accepted, completely validated producer output is reusable checkpoint or
 chunk-plan state. Rejected, structurally invalid, and validation-incomplete
 results cannot become cache or checkpoint inputs, even when a

docs/policy/documentation.md

@@ -30,6 +30,18 @@ that the behavior exists. The roadmap owns implementation status and sequencing
 until the decision is implemented. Current architecture, user, operator,
 integration, and internal documentation are updated when the behavior lands.
 
+### Implementation Details And Policy
+
+Documentation must distinguish an explicit contract from an incidental
+implementation choice. Do not describe a restriction, guarantee, security
+policy, default, or limit as authoritative merely because the current code
+implements it. Documentation may accurately describe current behavior without
+making it a permanent requirement; distinguish descriptive behavior from
+normative policy. The
+[implementation-discipline rules](architecture.md#implementation-discipline)
+own the criteria for adopting product restrictions and for elevating behavior
+into repository policy.
+
 ### Audience And Detail
 
 Write for the document's stated audience and include only the detail needed for

docs/policy/testing.md

@@ -87,6 +87,13 @@ Do not force all behavior through oversized end-to-end tests. Do not test every
 
 A test should protect a decision, contract, or invariant—not memorialize the current implementation.
 
+Tests must not create a stronger product contract than the requirement they
+protect. In particular, do not turn an incidental limit, restriction,
+permission choice, validation rule, or implementation default into policy by
+asserting it without an expressly selected contract. Follow the
+[implementation-discipline rules](architecture.md#implementation-discipline)
+when deciding what behavior deserves durable protection.
+
 Before adding or retaining a test, ask:
 
 > What realistic defect would this test catch?

docs/releases/v0.7.0.md

@@ -0,0 +1,88 @@
+# Notarius v0.7.0
+
+This release simplifies configuration management with ConfigKit and makes
+cache access permissions a deployment responsibility.
+
+## Summary
+
+Notarius adopts ConfigKit v0.3.0 for strict configuration loading, imports,
+configuration profiles, provenance, and redacted inspection. Operators can
+compose whole pipeline definitions from separate files and select development
+overlays without duplicating the base configuration.
+
+Configured relative filesystem paths now resolve against their declaring
+files, including imported definitions and profile replacements. The effective
+configuration and its explanation reflect the paths used at runtime.
+
+## Compatibility
+
+- Configuration version 5 replaces version 4; there is no compatibility loader.
+  Numeric `version: 5` is the canonical spelling; quoted `"5"` also works.
+- Module and validator bindings require objects with `module:`; external
+  references require objects with `path:`. Lane-level reference aliases are
+  removed; use binding-local references instead.
+- Strict ConfigKit YAML parsing rejects aliases, merge keys, and unsupported
+  scalar forms. See the [configuration reference](../config.md) for the
+  complete contract and migration guidance.
+- The seven operational environment overrides are removed:
+  `NOTARIUS_TOTAL_LLM_CONCURRENCY`, `NOTARIUS_STAGE_WORKERS_EXTRACT`,
+  `NOTARIUS_OUTPUT_DIR`, `NOTARIUS_CACHE_CHUNK_PLANS_MODE`,
+  `NOTARIUS_CACHE_CHUNK_PLANS_DIR`, `NOTARIUS_CACHE_CHECKPOINTS_DIR`, and
+  `NOTARIUS_DEBUG_DIR`. `NOTARIUS_CONFIG` still selects the configuration
+  file; PromptKit credential environment variables remain supported.
+- Configured relative reference, PromptKit profile, output, cache, and debug
+  paths use the canonical declaring file's directory. Built-in relative
+  defaults use the main configuration directory. CLI-relative paths still use
+  the invocation directory; absolute paths are unchanged.
+- Requested debug bundles now contain `effective-config.yml` instead of
+  `effective-config.json`. Inspection output is redacted and is not promised
+  to be reloadable configuration.
+- Chunk-plan and checkpoint cache writers respect deployment permissions
+  through ordinary creation modes, umask, setgid inheritance, and default ACLs;
+  they no longer impose private permissions. Existing entries are not chmodded
+  or migrated. Output and debug access contracts are unchanged.
+- Durable extraction artifact schemas and subprocess receipt formats are
+  unchanged by this release.
+
+## Upgrade
+
+1. Migrate deployed configurations to version 5 using the
+   [configuration migration guide](../config.md#version-5-migration). Expand
+   shorthand bindings and references, and move retired environment settings
+   into YAML or supported CLI overrides.
+2. Review relative paths, particularly output/cache/debug directories and
+   PromptKit profile sources. Use absolute paths when preserving an existing
+   deployment location is important.
+3. Review deployment access settings for cache directories. Provision umask,
+   groups, and ACLs as appropriate; Notarius does not choose the installation's
+   access policy. See [Operations](../operations.md).
+4. Use `config validate --config <path> --pipeline <id>` for each pipeline,
+   including selected configuration profiles. Inspect resolved settings with
+   `config effective` and their provenance with `config explain`; see the
+   [CLI reference](../cli.md).
+5. Update any debug-bundle tooling that consumes the former effective JSON
+   file. Subprocess callers may optionally select an overlay through
+   `--config-profile`; this is separate from PromptKit's `--llm-profile`.
+
+The maintained [minimal](../../examples/dnd-minimal.config.yml) and
+[complete](../../examples/dnd-complete.config.yml) D&D examples demonstrate
+self-contained configuration and composed pipelines with a development profile.
+Build from the source tag using the [source-release procedure](../release.md).
+
+## Changes
+
+- Replaced custom YAML parsing and environment-overlay machinery with a
+  ConfigKit-backed typed document boundary and detached runtime conversion.
+- Added imports, configuration profiles, source-aware normalization, safe
+  effective/explanation commands, and source-file legends for load errors.
+- Preserved explicit zeroes and empty validator overrides at supported stages,
+  rejected invalid reference branches before conversion, and retained
+  whitespace normalization for worker identifiers.
+- Corrected normalization recording for ordered steps and validator sequences,
+  with regression coverage for runtime values and provenance.
+- Accepted ConfigKit's integer representation at module-option boundaries
+  through checked shared conversion.
+- Updated examples, configuration and subprocess guidance, and debug rendering.
+- Reworked cache creation to respect deployment-selected permissions while
+  retaining confined paths, symlink rejection, atomic publication, validation,
+  and recovery.

examples/dnd-complete.config.yml

@@ -1,6 +1,9 @@
-version: 4
+version: 5
+configkit:
+  imports:
+    - path: pipelines/dnd-session.yml
 promptkit:
-  profile_file: ./examples/profiles/dnd-extraction.yml
+  profile_file: ./llm-profiles/dnd-extraction.yml
 concurrency:
   total_llm: 2
   stage_workers:
@@ -16,145 +19,3 @@ cache:
     directory: ./notarius-cache/checkpoints
 debug:
   directory: ./notarius-debug
-pipelines:
-  dnd-session:
-    llm_profile: dnd-extraction
-    input: seriatim
-    # Stable campaign context is shared by every module that accepts these slots.
-    references:
-      party: ./dnd-party.txt
-      glossary: ./dnd-glossary.txt
-    chunk:
-      module: dnd/scenes
-      retries: 2
-    output:
-      module: json
-      options:
-        include_chunk_map: true
-        evidence_context:
-          enabled: true
-          window_units: 3
-          lanes:
-            - item-occurrences
-            - item-registry
-            - location-registry
-            - location-occurrences
-            - npc-registry
-            - spells
-            - combat-turns
-            - npc-occurrences
-            - enemy-events
-    steps:
-      # Establish session-wide reference artifacts before their consumers.
-      - id: describe-session
-        artifacts:
-          item-registry:
-            extract:
-              module: dnd/item-registry
-              retries: 2
-            merge: appendorder
-            normalize: dnd/item-registry
-          npc-registry:
-            extract:
-              module: dnd/npc-registry
-              retries: 2
-            merge: appendorder
-            normalize:
-              module: dnd/npc-registry
-              retries: 2
-          location-registry:
-            extract:
-              module: dnd/location-registry
-              retries: 2
-            merge: appendorder
-            normalize:
-              module: dnd/location-registry
-              retries: 2
-          scene-descriptions:
-            extract:
-              module: dnd/scene-descriptions
-              retries: 2
-            merge: appendorder
-            normalize: dnd/scene-descriptions
-      - id: extract-events
-        # Accepted registry artifacts and scene-description eligibility artifacts
-        # are supplied in memory to their compatible consumers in this step.
-        references:
-          location_registry:
-            artifact:
-              step: describe-session
-              lane: location-registry
-          npc_registry:
-            artifact:
-              step: describe-session
-              lane: npc-registry
-          scene_descriptions:
-            artifact:
-              step: describe-session
-              lane: scene-descriptions
-          item_registry:
-            artifact:
-              step: describe-session
-              lane: item-registry
-        artifacts:
-          item-occurrences:
-            extract:
-              module: dnd/item-occurrences
-              retries: 2
-            merge: appendorder
-            normalize: dnd/item-occurrences
-          spells:
-            extract:
-              module: dnd/spells
-              retries: 2
-              references:
-                spell_catalog: ./dnd-spell-catalog.json
-            merge: appendorder
-            # Stage-local file references are intentionally bound at each stage.
-            normalize:
-              module: dnd/spells
-              references:
-                spell_catalog: ./dnd-spell-catalog.json
-          combat-turns:
-            extract:
-              module: dnd/combat-turns
-              retries: 2
-            merge: appendorder
-            normalize: dnd/combat-turns
-          npc-occurrences:
-            extract:
-              module: dnd/npc-occurrences
-              retries: 2
-            merge: appendorder
-            normalize: dnd/npc-occurrences
-          location-occurrences:
-            extract:
-              module: dnd/location-occurrences
-              retries: 2
-            merge: appendorder
-            normalize: dnd/location-occurrences
-      - id: track-enemies
-        references:
-          npc_registry:
-            artifact:
-              step: describe-session
-              lane: npc-registry
-          scene_descriptions:
-            artifact:
-              step: describe-session
-              lane: scene-descriptions
-          combat_turns:
-            artifact:
-              step: extract-events
-              lane: combat-turns
-          npc_occurrences:
-            artifact:
-              step: extract-events
-              lane: npc-occurrences
-        artifacts:
-          enemy-events:
-            extract:
-              module: dnd/enemy-events
-              retries: 2
-            merge: appendorder
-            normalize: dnd/enemy-events

examples/dnd-minimal.config.yml

@@ -1,9 +1,12 @@
-version: 4
+version: 5
 pipelines:
   dnd-session:
     llm_profile: dnd-extraction
-    input: seriatim
+    input:
+      module: seriatim
     artifacts:
       spells:
-        extract: dnd/spells
+        extract:
-        normalize: dnd/spells
+          module: dnd/spells
+        normalize:
+          module: dnd/spells

examples/pipelines/dnd-session.yml

@@ -0,0 +1,164 @@
+pipelines:
+  dnd-session:
+    llm_profile: dnd-extraction
+    input:
+      module: seriatim
+    # Stable campaign context is shared by every module that accepts these slots.
+    references:
+      party:
+        path: ../dnd-party.txt
+      glossary:
+        path: ../dnd-glossary.txt
+    chunk:
+      module: dnd/scenes
+      retries: 2
+    output:
+      module: json
+      options:
+        include_chunk_map: true
+        evidence_context:
+          enabled: true
+          window_units: 3
+          lanes:
+            - item-occurrences
+            - item-registry
+            - location-registry
+            - location-occurrences
+            - npc-registry
+            - spells
+            - combat-turns
+            - npc-occurrences
+            - enemy-events
+    steps:
+      # Establish session-wide reference artifacts before their consumers.
+      - id: describe-session
+        artifacts:
+          item-registry:
+            extract:
+              module: dnd/item-registry
+              retries: 2
+            merge:
+              module: appendorder
+            normalize:
+              module: dnd/item-registry
+          npc-registry:
+            extract:
+              module: dnd/npc-registry
+              retries: 2
+            merge:
+              module: appendorder
+            normalize:
+              module: dnd/npc-registry
+              retries: 2
+          location-registry:
+            extract:
+              module: dnd/location-registry
+              retries: 2
+            merge:
+              module: appendorder
+            normalize:
+              module: dnd/location-registry
+              retries: 2
+          scene-descriptions:
+            extract:
+              module: dnd/scene-descriptions
+              retries: 2
+            merge:
+              module: appendorder
+            normalize:
+              module: dnd/scene-descriptions
+      - id: extract-events
+        # Accepted registry artifacts and scene-description eligibility artifacts
+        # are supplied in memory to their compatible consumers in this step.
+        references:
+          location_registry:
+            artifact:
+              step: describe-session
+              lane: location-registry
+          npc_registry:
+            artifact:
+              step: describe-session
+              lane: npc-registry
+          scene_descriptions:
+            artifact:
+              step: describe-session
+              lane: scene-descriptions
+          item_registry:
+            artifact:
+              step: describe-session
+              lane: item-registry
+        artifacts:
+          item-occurrences:
+            extract:
+              module: dnd/item-occurrences
+              retries: 2
+            merge:
+              module: appendorder
+            normalize:
+              module: dnd/item-occurrences
+          spells:
+            extract:
+              module: dnd/spells
+              retries: 2
+              references:
+                spell_catalog:
+                  path: ../dnd-spell-catalog.json
+            merge:
+              module: appendorder
+            # Stage-local file references are intentionally bound at each stage.
+            normalize:
+              module: dnd/spells
+              references:
+                spell_catalog:
+                  path: ../dnd-spell-catalog.json
+          combat-turns:
+            extract:
+              module: dnd/combat-turns
+              retries: 2
+            merge:
+              module: appendorder
+            normalize:
+              module: dnd/combat-turns
+          npc-occurrences:
+            extract:
+              module: dnd/npc-occurrences
+              retries: 2
+            merge:
+              module: appendorder
+            normalize:
+              module: dnd/npc-occurrences
+          location-occurrences:
+            extract:
+              module: dnd/location-occurrences
+              retries: 2
+            merge:
+              module: appendorder
+            normalize:
+              module: dnd/location-occurrences
+      - id: track-enemies
+        references:
+          npc_registry:
+            artifact:
+              step: describe-session
+              lane: npc-registry
+          scene_descriptions:
+            artifact:
+              step: describe-session
+              lane: scene-descriptions
+          combat_turns:
+            artifact:
+              step: extract-events
+              lane: combat-turns
+          npc_occurrences:
+            artifact:
+              step: extract-events
+              lane: npc-occurrences
+        artifacts:
+          enemy-events:
+            extract:
+              module: dnd/enemy-events
+              retries: 2
+            merge:
+              module: appendorder
+            normalize:
+              module: dnd/enemy-events

examples/profiles/development.yml

@@ -0,0 +1,8 @@
+# This selects operational settings only; choose a different LLM profile with
+# the pipeline's llm_profile field or the run command's --llm-profile flag.
+concurrency:
+  total_llm: 1
+  stage_workers:
+    extract: 1
+output:
+  directory: ./notarius-output-development

go.mod

@@ -3,9 +3,10 @@ module gitea.maximumdirect.net/eric/notarius
 go 1.25.5
 
 require (
+	gitea.maximumdirect.net/eric/ConfigKit v0.3.0
 	gitea.maximumdirect.net/eric/promptkit v0.9.0
 	github.com/santhosh-tekuri/jsonschema/v6 v6.0.2
-	gopkg.in/yaml.v3 v3.0.1
+	gopkg.in/yaml.v3 v3.0.1 // indirect
 )
 
 require golang.org/x/text v0.40.0
@@ -13,4 +14,5 @@ require golang.org/x/text v0.40.0
 require (
 	gitea.maximumdirect.net/eric/promptkit-backend-openrouter v1.0.0 // indirect
 	gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome v1.0.0 // indirect
+	go.yaml.in/yaml/v4 v4.0.0-rc.6 // indirect
 )

go.sum

@@ -1,3 +1,5 @@
+gitea.maximumdirect.net/eric/ConfigKit v0.3.0 h1:LgXEUjR0KsgwGBznOWa8FJTQ+iElctHtI1via5vKdu4=
+gitea.maximumdirect.net/eric/ConfigKit v0.3.0/go.mod h1:tYWpsrbLyu05r8+1MapWp5fN6ZrX9I8hU2xEPedYmYk=
 gitea.maximumdirect.net/eric/promptkit v0.9.0 h1:IpvDRC8L6xRxQ9hpuyKOmMc5b6MeLTKYyx+h1YAjy08=
 gitea.maximumdirect.net/eric/promptkit v0.9.0/go.mod h1:oMJ/WUJImUtwJ5e+6MAGECPYAErAkOaKel0G+3T/b4E=
 gitea.maximumdirect.net/eric/promptkit-backend-openrouter v1.0.0 h1:lc062euk2qseO//D762i3JaFyulDNML3eQQX7DkYTho=
@@ -8,6 +10,8 @@ github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxK
 github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
 github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
 github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
+go.yaml.in/yaml/v4 v4.0.0-rc.6 h1:1h7H1ohdUh93/FyE4YaDa1Zh64K6VVbjF4K6WUxMtH4=
+go.yaml.in/yaml/v4 v4.0.0-rc.6/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=
 golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
 golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=

internal/cli/cache_contract_test.go

@@ -23,7 +23,7 @@ func TestRunChunkPlanModePrecedenceAndValidation(t *testing.T) {
 	}{
 		{name: "default", wantStores: 1},
 		{name: "file", fileMode: "bypass"},
-		{name: "environment", envMode: "bypass"},
+		{name: "retired environment has no effect", envMode: "bypass", wantStores: 1},
 		{name: "cli", envMode: "refresh", cliMode: "bypass"},
 	}
 	for _, tt := range tests {
@@ -80,25 +80,14 @@ func TestRunChunkPlanModePrecedenceAndValidation(t *testing.T) {
 	})
 
 	for _, tt := range []struct {
-		name       string
+		name string
-		fileConfig bool
 	}{
-		{name: "invalid environment mode"},
+		{name: "invalid file mode"},
-		{name: "invalid file mode", fileConfig: true},
 	} {
 		t.Run(tt.name, func(t *testing.T) {
 			roots := newStateTestRoots(t)
 			opts := newStateTestHarness().options()
-			if tt.fileConfig {
+			replaceStateTestConfigLine(t, roots.config, "    mode: auto\n", "    mode: invalid\n")
-				replaceStateTestConfigLine(t, roots.config, "    mode: auto\n", "    mode: invalid\n")
-			} else {
-				opts.LookupEnv = func(name string) (string, bool) {
-					if name == "NOTARIUS_CACHE_CHUNK_PLANS_MODE" {
-						return "invalid", true
-					}
-					return "", false
-				}
-			}
 			var stdout, stderr bytes.Buffer
 			code := RunWithOptions([]string{"run", "sample", "--config", roots.config, "--input", roots.input}, &stdout, &stderr, opts)
 			if code != 1 || stdout.Len() != 0 || stderr.Len() == 0 {
@@ -207,16 +196,20 @@ func TestRunAutoReusesPlanWhenRunInputsChange(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	configText := replaceRequiredOnce(t, string(data), "    chunk: test/chunk\n", `    chunk:
+	configText := replaceRequiredOnce(t, string(data), "    chunk:\n      module: test/chunk\n", `    chunk:
       module: test/chunk
       options:
         strategy: first
 `)
-	configText = replaceRequiredOnce(t, configText, "    output: test/output\n", `      other:
+	configText = replaceRequiredOnce(t, configText, "    output:\n      module: test/output\n", `      other:
-        extract: test/extract
+        extract:
-        merge: test/merge
+          module: test/extract
-        normalize: test/normalize
+        merge:
-    output: test/output
+          module: test/merge
+        normalize:
+          module: test/normalize
+    output:
+      module: test/output
 `)
 	if err := os.WriteFile(roots.config, []byte(configText), 0o600); err != nil {
 		t.Fatal(err)

internal/cli/command_contract_test.go

@@ -140,7 +140,7 @@ func TestConfigLoadingFailuresReturnOneWithPathContext(t *testing.T) {
 	stdout.Reset()
 	stderr.Reset()
 	code = RunWithOptions([]string{"config", "validate", "--config", malformed}, &stdout, &stderr, commandContractOptions(t))
-	if code != 1 || !strings.Contains(stderr.String(), malformed) || !strings.Contains(stderr.String(), "parse config file") || stdout.Len() != 0 {
+	if code != 1 || !strings.Contains(stderr.String(), "configuration contains invalid YAML") || stdout.Len() != 0 {
 		t.Fatalf("malformed config: code=%d stdout=%q stderr=%q", code, stdout.String(), stderr.String())
 	}
 }
@@ -185,14 +185,16 @@ func TestConfigValidateResolvesPipelineAndChecksSelection(t *testing.T) {
 }
 
 func TestConfigValidatePipelineDefaultProfileIsOffline(t *testing.T) {
-	configPath := writeCommandConfigContent(t, `version: 4
+	configPath := writeCommandConfigContent(t, `version: 5
 pipelines:
   demo:
     llm_profile: dnd-extraction
-    input: seriatim
+    input:
+      module: seriatim
     artifacts:
       spells:
-        extract: dnd/spells
+        extract:
+          module: dnd/spells
 `)
 	var stdout, stderr bytes.Buffer
 	code := RunWithOptions([]string{"config", "validate", "--config", configPath, "--pipeline", "demo"}, &stdout, &stderr, Options{})
@@ -261,19 +263,21 @@ func commandContractOptionsWithLookup(t *testing.T, lookup func(string) (string,
 
 func writeCommandConfig(t *testing.T, firstID, secondID string) string {
 	t.Helper()
-	content := fmt.Sprintf("version: 4\npipelines:\n  %q:\n    input: seriatim\n  %q:\n    input: seriatim\n", firstID, secondID)
+	content := fmt.Sprintf("version: 5\npipelines:\n  %q:\n    input:\n      module: seriatim\n  %q:\n    input:\n      module: seriatim\n", firstID, secondID)
 	return writeCommandConfigContent(t, content)
 }
 
 func writeResolvableCommandConfig(t *testing.T) string {
 	t.Helper()
-	return writeCommandConfigContent(t, `version: 4
+	return writeCommandConfigContent(t, `version: 5
 pipelines:
   demo:
-    input: seriatim
+    input:
+      module: seriatim
     artifacts:
       spells:
-        extract: dnd/spells
+        extract:
+          module: dnd/spells
 `)
 }
 

internal/cli/configkit_commands.go

@@ -0,0 +1,157 @@
+package cli
+
+import (
+	"context"
+	"encoding/json"
+	"flag"
+	"fmt"
+	"io"
+	"strings"
+
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
+	"gitea.maximumdirect.net/eric/notarius/internal/core/config"
+)
+
+func runConfigEffective(args []string, stdout, stderr io.Writer, opts Options) int {
+	fs := flag.NewFlagSet("config effective", flag.ContinueOnError)
+	fs.SetOutput(io.Discard)
+	configPath := fs.String("config", "", "config file path")
+	profile := fs.String("config-profile", "", "configuration profile")
+	if err := fs.Parse(args); err != nil {
+		fmt.Fprintf(stderr, "notarius: %v\n", err)
+		return 2
+	}
+	if fs.NArg() != 0 {
+		fmt.Fprintf(stderr, "notarius: unexpected argument %q\n", fs.Arg(0))
+		return 2
+	}
+	loaded, _, err := loadConfigKit(*configPath, *profile, nil, opts)
+	if err != nil {
+		writeConfigLoadError(stderr, err)
+		return 1
+	}
+	if err := loaded.WriteEffective(stdout); err != nil {
+		fmt.Fprintf(stderr, "notarius: write effective configuration: %v\n", err)
+		return 1
+	}
+	return 0
+}
+
+func runConfigExplain(args []string, stdout, stderr io.Writer, opts Options) int {
+	fs := flag.NewFlagSet("config explain", flag.ContinueOnError)
+	fs.SetOutput(io.Discard)
+	configPath := fs.String("config", "", "config file path")
+	profile := fs.String("config-profile", "", "configuration profile")
+	if err := fs.Parse(args); err != nil {
+		fmt.Fprintf(stderr, "notarius: %v\n", err)
+		return 2
+	}
+	if fs.NArg() != 1 {
+		fmt.Fprintln(stderr, "notarius: config explain requires exactly one JSON Pointer")
+		return 2
+	}
+	pointer := fs.Arg(0)
+	parsed, err := configkit.ParsePath(pointer)
+	if err != nil || parsed.String() != pointer {
+		fmt.Fprintf(stderr, "notarius: %q is not a canonical JSON Pointer\n", pointer)
+		return 2
+	}
+	loaded, _, err := loadConfigKit(*configPath, *profile, nil, opts)
+	if err != nil {
+		writeConfigLoadError(stderr, err)
+		return 1
+	}
+	if err := loaded.WriteExplanation(stdout, pointer); err != nil {
+		fmt.Fprintf(stderr, "notarius: write configuration explanation: %v\n", err)
+		return 1
+	}
+	return 0
+}
+
+func runConfigValidate(args []string, stdout, stderr io.Writer, opts Options) int {
+	fs := flag.NewFlagSet("config validate", flag.ContinueOnError)
+	fs.SetOutput(io.Discard)
+	configPath := fs.String("config", "", "config file path")
+	profile := fs.String("config-profile", "", "configuration profile")
+	pipelineID := fs.String("pipeline", "", "pipeline ID")
+	onlyRaw := fs.String("only", "", "comma-separated artifact lanes")
+	if err := fs.Parse(args); err != nil {
+		fmt.Fprintf(stderr, "notarius: %v\n", err)
+		return 2
+	}
+	if fs.NArg() != 0 {
+		fmt.Fprintf(stderr, "notarius: unexpected argument %q\n", fs.Arg(0))
+		return 2
+	}
+	if strings.TrimSpace(*onlyRaw) != "" && strings.TrimSpace(*pipelineID) == "" {
+		fmt.Fprintln(stderr, "notarius: --only requires --pipeline")
+		return 2
+	}
+	only, err := parseOnly(*onlyRaw)
+	if err != nil {
+		fmt.Fprintf(stderr, "notarius: %v\n", err)
+		return 2
+	}
+	loaded, path, err := loadConfigKit(*configPath, *profile, nil, opts)
+	if err != nil {
+		writeConfigLoadError(stderr, err)
+		return 1
+	}
+	if strings.TrimSpace(*pipelineID) == "" {
+		fmt.Fprintf(stdout, "config %q is valid\n", path)
+		return 0
+	}
+	catalog, err := effectiveCatalog(opts)
+	if err != nil {
+		fmt.Fprintf(stderr, "notarius: %v\n", err)
+		return 1
+	}
+	effective, err := loaded.Config.Resolve(config.ResolveInput{PipelineID: *pipelineID, Only: only, Catalog: catalog})
+	if err != nil {
+		fmt.Fprintf(stderr, "notarius: %v\n", err)
+		return 1
+	}
+	if err := validateExplicitPromptKitProfiles(context.Background(), effective.Config, effectiveLLMProfileIDs(effective.ResolvedPipeline), opts.promptKitAssets); err != nil {
+		fmt.Fprintf(stderr, "notarius: %v\n", err)
+		return 1
+	}
+	fmt.Fprintf(stdout, "config %q is valid for pipeline %q\n", path, strings.TrimSpace(*pipelineID))
+	return 0
+}
+
+func runPipelinesList(args []string, stdout, stderr io.Writer, opts Options) int {
+	fs := flag.NewFlagSet("pipelines list", flag.ContinueOnError)
+	fs.SetOutput(io.Discard)
+	configPath := fs.String("config", "", "config file path")
+	profile := fs.String("config-profile", "", "configuration profile")
+	jsonOutput := fs.Bool("json", false, "write JSON output")
+	if err := fs.Parse(args); err != nil {
+		fmt.Fprintf(stderr, "notarius: %v\n", err)
+		return 2
+	}
+	if fs.NArg() != 0 {
+		fmt.Fprintf(stderr, "notarius: unexpected argument %q\n", fs.Arg(0))
+		return 2
+	}
+	loaded, _, err := loadConfigKit(*configPath, *profile, nil, opts)
+	if err != nil {
+		writeConfigLoadError(stderr, err)
+		return 1
+	}
+	ids := sortedPipelineIDs(loaded.Config)
+	if *jsonOutput {
+		payload, err := json.Marshal(struct {
+			Pipelines []string `json:"pipelines"`
+		}{Pipelines: ids})
+		if err != nil {
+			fmt.Fprintf(stderr, "notarius: marshal pipeline list: %v\n", err)
+			return 1
+		}
+		fmt.Fprintf(stdout, "%s\n", payload)
+		return 0
+	}
+	for _, id := range ids {
+		fmt.Fprintln(stdout, id)
+	}
+	return 0
+}

internal/cli/configkit_commands_test.go

@@ -0,0 +1,132 @@
+package cli
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"io"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
+	"gitea.maximumdirect.net/eric/notarius/internal/core/config"
+	"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
+)
+
+func TestConfigKitReadOnlyCommandsLoadImportsAndProfiles(t *testing.T) {
+	t.Parallel()
+	root := t.TempDir()
+	path := filepath.Join(root, "notarius.yml")
+	writeConfigKitFile(t, filepath.Join(root, "fragment.yml"), "pipelines:\n  imported:\n    input: {module: input}\n")
+	writeConfigKitFile(t, filepath.Join(root, "profiles", "development.yml"), "pipelines:\n  profiled:\n    input: {module: input}\n")
+	writeConfigKitFile(t, path, "version: 5\nconfigkit:\n  imports:\n    - path: fragment.yml\npipelines:\n  base:\n    input: {module: input}\n")
+	var stdout, stderr bytes.Buffer
+	if code := RunWithOptions([]string{"pipelines", "list", "--config", path, "--config-profile", "development"}, &stdout, &stderr, commandContractOptions(t)); code != 0 || stdout.String() != "base\nimported\nprofiled\n" {
+		t.Fatalf("list = %d, %q, %q", code, stdout.String(), stderr.String())
+	}
+	stdout.Reset()
+	stderr.Reset()
+	if code := RunWithOptions([]string{"config", "validate", "--config", path, "--config-profile", "development"}, &stdout, &stderr, commandContractOptions(t)); code != 0 || !strings.Contains(stdout.String(), "is valid") {
+		t.Fatalf("validate = %d, %q, %q", code, stdout.String(), stderr.String())
+	}
+}
+
+func TestConfigKitReadOnlyCommandsRejectVersionFour(t *testing.T) {
+	t.Parallel()
+	path := filepath.Join(t.TempDir(), "notarius.yml")
+	writeConfigKitFile(t, path, "version: 4\n")
+	var stdout, stderr bytes.Buffer
+	if code := runPipelinesList([]string{"--config", path}, &stdout, &stderr, commandContractOptions(t)); code != 1 || !strings.Contains(stderr.String(), "version") {
+		t.Fatalf("list = %d, %q", code, stderr.String())
+	}
+}
+
+func TestConfigurationInspectionCommandsRenderSafeConfigKitViews(t *testing.T) {
+	t.Parallel()
+	root := t.TempDir()
+	path := filepath.Join(root, "notarius.yml")
+	writeConfigKitFile(t, path, `version: 5
+concurrency:
+  total_llm: 4
+pipelines:
+  "0":
+    input: {module: input}
+`)
+	writeConfigKitFile(t, filepath.Join(root, "profiles", "development.yml"), "concurrency:\n  total_llm: 3\n")
+	writeConfigKitFile(t, filepath.Join(root, "profiles", "retired.yml"), "pipelines: null\n")
+
+	providerCalled := false
+	opts := commandContractOptions(t)
+	opts.LLMClientFactory = func(context.Context, config.Config, string, LLMRuntimeOverrides) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error) {
+		providerCalled = true
+		return nil, nil, errors.New("LLM client must not be constructed")
+	}
+
+	var stdout, stderr bytes.Buffer
+	if code := RunWithOptions([]string{"config", "effective", "--config", path, "--config-profile", "development"}, &stdout, &stderr, opts); code != 0 || !strings.Contains(stdout.String(), "total_llm: 3") || stderr.Len() != 0 {
+		t.Fatalf("effective = %d, %q, %q", code, stdout.String(), stderr.String())
+	}
+	if providerCalled {
+		t.Fatal("config effective constructed an LLM client")
+	}
+
+	stdout.Reset()
+	stderr.Reset()
+	if code := RunWithOptions([]string{"config", "explain", "--config", path, "--config-profile", "development", "/concurrency/total_llm"}, &stdout, &stderr, opts); code != 0 || !strings.Contains(stdout.String(), `path: "/concurrency/total_llm"`) || stderr.Len() != 0 {
+		t.Fatalf("explain effective = %d, %q, %q", code, stdout.String(), stderr.String())
+	}
+
+	stdout.Reset()
+	stderr.Reset()
+	if code := RunWithOptions([]string{"config", "explain", "--config", path, ""}, &stdout, &stderr, opts); code != 0 || !strings.Contains(stdout.String(), `path: ""`) || stderr.Len() != 0 {
+		t.Fatalf("explain root = %d, %q, %q", code, stdout.String(), stderr.String())
+	}
+
+	stdout.Reset()
+	stderr.Reset()
+	if code := RunWithOptions([]string{"config", "explain", "--config", path, "--config-profile", "retired", "/pipelines/0"}, &stdout, &stderr, opts); code != 0 || !strings.Contains(stdout.String(), `path: "/pipelines/0"`) || stderr.Len() != 0 {
+		t.Fatalf("explain removed mapping key = %d, %q, %q", code, stdout.String(), stderr.String())
+	}
+}
+
+func TestConfigExplainRejectsInvalidAndUnknownPointers(t *testing.T) {
+	t.Parallel()
+	path := writeCommandConfig(t, "session", "other")
+	for _, test := range []struct {
+		name     string
+		pointer  string
+		wantCode int
+	}{
+		{name: "missing", wantCode: 2},
+		{name: "malformed escape", pointer: "/pipelines~2session", wantCode: 2},
+		{name: "unknown", pointer: "/unknown", wantCode: 1},
+		{name: "invalid sequence index", pointer: "/pipelines/session/steps/+0", wantCode: 1},
+	} {
+		t.Run(test.name, func(t *testing.T) {
+			var stdout, stderr bytes.Buffer
+			args := []string{"config", "explain", "--config", path}
+			if test.name != "missing" {
+				args = append(args, test.pointer)
+			}
+			if code := RunWithOptions(args, &stdout, &stderr, commandContractOptions(t)); code != test.wantCode || stdout.Len() != 0 || stderr.Len() == 0 {
+				t.Fatalf("code=%d stdout=%q stderr=%q", code, stdout.String(), stderr.String())
+			}
+		})
+	}
+}
+
+func TestConfigInspectionCommandsReportWriterFailures(t *testing.T) {
+	t.Parallel()
+	path := writeCommandConfig(t, "session", "other")
+	var stderr bytes.Buffer
+	if code := RunWithOptions([]string{"config", "effective", "--config", path}, failingInspectionWriter{}, &stderr, commandContractOptions(t)); code != 1 || !strings.Contains(stderr.String(), "write effective configuration") {
+		t.Fatalf("effective writer failure = %d, %q", code, stderr.String())
+	}
+}
+
+type failingInspectionWriter struct{}
+
+func (failingInspectionWriter) Write([]byte) (int, error) {
+	return 0, io.ErrClosedPipe
+}

internal/cli/configkit_load.go

@@ -0,0 +1,49 @@
+package cli
+
+import (
+	"errors"
+	"fmt"
+	"io"
+	"os"
+
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
+	"gitea.maximumdirect.net/eric/notarius/internal/core/config"
+)
+
+func loadConfigKit(configPath, profile string, overrides []configkit.Override, opts Options) (*config.LoadedConfig, string, error) {
+	path, err := discoverConfigPath(configPath, opts)
+	if err != nil {
+		return nil, "", err
+	}
+	workingDirectory, err := os.Getwd()
+	if err != nil {
+		return nil, "", fmt.Errorf("determine invocation directory: %w", err)
+	}
+	loaded, err := config.Load(path, config.LoadOptions{Profile: profile, Overrides: overrides, InvocationDirectory: workingDirectory})
+	if err != nil {
+		return nil, "", err
+	}
+	return loaded, path, nil
+}
+
+func writeConfigLoadError(output io.Writer, err error) {
+	var loadError *configkit.LoadError
+	if errors.As(err, &loadError) {
+		if writeErr := configkit.WriteDiagnostics(output, loadError.Diagnostics()); writeErr == nil {
+			// ConfigKit's locations contain source IDs. Keep its renderer and
+			// supply the acquired-source legend so imported/profile errors can
+			// be traced to a file. Quoting also escapes terminal controls.
+			for _, source := range loadError.Sources() {
+				label := source.Path
+				if label == "" {
+					label = source.Label
+				}
+				if _, err := fmt.Fprintf(output, "source %q: %q\n", source.ID, label); err != nil {
+					return
+				}
+			}
+			return
+		}
+	}
+	fmt.Fprintf(output, "notarius: %v\n", err)
+}

internal/cli/configkit_load_test.go

@@ -0,0 +1,85 @@
+package cli
+
+import (
+	"bytes"
+	"os"
+	"path/filepath"
+	"strconv"
+	"strings"
+	"testing"
+
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
+)
+
+func TestConfigLoadDiagnosticsIdentifyComposedSources(t *testing.T) {
+	t.Parallel()
+	for _, profile := range []bool{false, true} {
+		t.Run(strconv.FormatBool(profile), func(t *testing.T) {
+			root := t.TempDir()
+			main := filepath.Join(root, "config.yml")
+			fragment := filepath.Join(root, "pipeline\n\x1b.yml")
+			args := []string{"config", "validate", "--config", main}
+			if profile {
+				fragment = filepath.Join(root, "profiles", "development.yml")
+				writeConfigKitFile(t, main, "version: 5\n")
+				args = append(args, "--config-profile", "development")
+			} else {
+				writeConfigKitFile(t, main, "version: 5\nconfigkit:\n  imports:\n    - path: "+strconv.Quote(filepath.Base(fragment))+"\n")
+			}
+			writeConfigKitFile(t, fragment, "concurrency: {total_llm: invalid-private-value}\n")
+			var stdout, stderr bytes.Buffer
+			if code := RunWithOptions(args, &stdout, &stderr, commandContractOptions(t)); code != 1 {
+				t.Fatalf("status = %d; %s", code, stderr.String())
+			}
+			got := stderr.String()
+			if stdout.Len() != 0 || !strings.Contains(got, "decode.type") ||
+				!strings.Contains(got, "source-2:1:") ||
+				!strings.Contains(got, strconv.Quote(fragment)) ||
+				strings.Contains(got, "invalid-private-value") ||
+				strings.ContainsRune(got, '\x1b') {
+				t.Fatalf("stdout = %q; stderr = %q", stdout.String(), got)
+			}
+		})
+	}
+}
+
+func TestLoadConfigKitUsesDiscoveryProfileAndOverrides(t *testing.T) {
+	t.Parallel()
+	root := t.TempDir()
+	path := filepath.Join(root, "notarius.yml")
+	writeConfigKitFile(t, path, "version: 5\npipelines:\n  base:\n    input: {module: input}\n")
+	profiles := filepath.Join(root, "profiles")
+	writeConfigKitFile(t, filepath.Join(profiles, "development.yml"), "pipelines:\n  profiled:\n    input: {module: input}\n")
+	loaded, gotPath, err := loadConfigKit(path, "development", []configkit.Override{{Path: configkit.Keys("output", "directory"), Value: "generated", Flag: "--output"}}, commandContractOptions(t))
+	if err != nil {
+		t.Fatalf("loadConfigKit() error = %v", err)
+	}
+	if gotPath != path || loaded.Config.Pipelines["profiled"].Input.Module != "input" || loaded.Config.Output.Directory == "generated" {
+		t.Fatalf("loaded = %#v, path=%q", loaded.Config, gotPath)
+	}
+}
+
+func TestWriteConfigLoadErrorUsesSafeDiagnostics(t *testing.T) {
+	t.Parallel()
+	path := filepath.Join(t.TempDir(), "invalid.yml")
+	writeConfigKitFile(t, path, "version: 4\n")
+	_, _, err := loadConfigKit(path, "", nil, commandContractOptions(t))
+	if err == nil {
+		t.Fatal("loadConfigKit() error = nil")
+	}
+	var output bytes.Buffer
+	writeConfigLoadError(&output, err)
+	if !strings.Contains(output.String(), "version") || strings.Contains(output.String(), "notarius: configkit") {
+		t.Fatalf("diagnostics = %q", output.String())
+	}
+}
+
+func writeConfigKitFile(t *testing.T, path, contents string) {
+	t.Helper()
+	if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(path, []byte(contents), 0o600); err != nil {
+		t.Fatal(err)
+	}
+}

internal/cli/dnd_enemy_events_contract_test.go

@@ -264,13 +264,29 @@ func completeExampleConfigWithTemporaryCache(t *testing.T) string {
 	updated := strings.Replace(string(content), "directory: ./notarius-cache/chunk-plans", fmt.Sprintf("directory: %q", filepath.Join(cacheRoot, "chunk-plans")), 1)
 	updated = strings.Replace(updated, "directory: ./notarius-cache/checkpoints", fmt.Sprintf("directory: %q", filepath.Join(cacheRoot, "checkpoints")), 1)
 	for relative, absolute := range map[string]string{
-		"./dnd-party.txt":          repositoryPath("examples", "dnd-party.txt"),
+		"./llm-profiles/dnd-extraction.yml": repositoryPath("examples", "llm-profiles", "dnd-extraction.yml"),
-		"./dnd-glossary.txt":       repositoryPath("examples", "dnd-glossary.txt"),
-		"./dnd-spell-catalog.json": repositoryPath("examples", "dnd-spell-catalog.json"),
 	} {
 		updated = strings.ReplaceAll(updated, relative, fmt.Sprintf("%q", absolute))
 	}
-	path := filepath.Join(t.TempDir(), "dnd-complete.config.yml")
+	root := t.TempDir()
+	fragment, err := os.ReadFile(repositoryPath("examples", "pipelines", "dnd-session.yml"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	for relative, absolute := range map[string]string{
+		"../dnd-party.txt":          repositoryPath("examples", "dnd-party.txt"),
+		"../dnd-glossary.txt":       repositoryPath("examples", "dnd-glossary.txt"),
+		"../dnd-spell-catalog.json": repositoryPath("examples", "dnd-spell-catalog.json"),
+	} {
+		fragment = []byte(strings.ReplaceAll(string(fragment), relative, fmt.Sprintf("%q", absolute)))
+	}
+	if err := os.MkdirAll(filepath.Join(root, "pipelines"), 0o700); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(filepath.Join(root, "pipelines", "dnd-session.yml"), fragment, 0o600); err != nil {
+		t.Fatal(err)
+	}
+	path := filepath.Join(root, "dnd-complete.config.yml")
 	if err := os.WriteFile(path, []byte(updated), 0o600); err != nil {
 		t.Fatal(err)
 	}

internal/cli/example_contract_test.go

@@ -147,8 +147,15 @@ func TestMaintainedConfigurationExampleSet(t *testing.T) {
 		}
 	}
 	sort.Strings(names)
-	if got := strings.Join(names, ","); got != "dnd-extraction.yml" {
+	if got := strings.Join(names, ","); got != "development.yml" {
-		t.Fatalf("maintained operator profiles = %q, want dnd-extraction.yml", got)
+		t.Fatalf("maintained configuration profiles = %q, want development.yml", got)
+	}
+	llmProfiles, err := os.ReadDir(repositoryPath("examples", "llm-profiles"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(llmProfiles) != 1 || llmProfiles[0].Name() != "dnd-extraction.yml" {
+		t.Fatalf("maintained LLM profiles = %#v, want dnd-extraction.yml", llmProfiles)
 	}
 }
 

internal/cli/npc_registry_contract_test.go

@@ -23,7 +23,7 @@ func TestOversizedNPCRegistryFailsBeforeRuntimeAndCheckpointConstruction(t *test
 		t.Fatal(err)
 	}
 	checkpointRoot := filepath.Join(t.TempDir(), "checkpoints")
-	content := fmt.Sprintf(`version: 4
+	content := fmt.Sprintf(`version: 5
 cache:
   chunk_plans:
     mode: bypass
@@ -32,14 +32,17 @@ cache:
     directory: %q
 pipelines:
   dnd-session:
-    input: seriatim
+    input:
+      module: seriatim
     artifacts:
       spells:
         extract:
           module: dnd/spells
           references:
-            npc_registry: %q
+            npc_registry:
-        normalize: dnd/spells
+              path: %q
+        normalize:
+          module: dnd/spells
 `, checkpointRoot, npcPath)
 	configPath := filepath.Join(t.TempDir(), "config.yml")
 	if err := os.WriteFile(configPath, []byte(content), 0o600); err != nil {

internal/cli/production_contract_test.go

@@ -19,8 +19,6 @@ import (
 	"testing/fstest"
 	"time"
 
-	"gopkg.in/yaml.v3"
-
 	"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
 	"gitea.maximumdirect.net/eric/notarius/internal/core/config"
 	"gitea.maximumdirect.net/eric/notarius/internal/framework/chunkmap"
@@ -333,61 +331,8 @@ func TestProductionPromptAssetsPrepareWithoutProviderCredentials(t *testing.T) {
 	if err != nil {
 		t.Fatalf("production prompt engine: %v", err)
 	}
-	promptFS, err := components.assets.PromptFS()
+	if engine == nil {
-	if err != nil {
+		t.Fatal("production PromptKit engine is nil")
-		t.Fatalf("production prompt assets: %v", err)
-	}
-	type manifest struct {
-		ID      string `yaml:"id"`
-		Version string `yaml:"version"`
-		Inputs  []struct {
-			Name string `yaml:"name"`
-		} `yaml:"inputs"`
-		Messages []struct {
-			Role string `yaml:"role"`
-		} `yaml:"messages"`
-	}
-	preparedPrompts := 0
-	if err := fs.WalkDir(promptFS, ".", func(path string, entry fs.DirEntry, walkErr error) error {
-		if walkErr != nil {
-			return walkErr
-		}
-		if entry.IsDir() || filepath.Base(path) != "prompt.yaml" {
-			return nil
-		}
-		data, err := fs.ReadFile(promptFS, path)
-		if err != nil {
-			return err
-		}
-		var prompt manifest
-		if err := yaml.Unmarshal(data, &prompt); err != nil {
-			return err
-		}
-		for _, message := range prompt.Messages {
-			if message.Role != promptkit.RoleSystem && message.Role != promptkit.RoleUser {
-				return fmt.Errorf("production prompt %q uses role %q, want system or user", prompt.ID, message.Role)
-			}
-		}
-		inputs := make(map[string]promptkit.ArtifactRef, len(prompt.Inputs))
-		for _, input := range prompt.Inputs {
-			inputs[input.Name] = promptkit.Inline(`{}`)
-		}
-		prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
-			PromptID: prompt.ID, PromptVersion: prompt.Version, ProfileID: "assembled-prompt-test", Inputs: inputs,
-		})
-		if err != nil {
-			return fmt.Errorf("prepare production prompt %q: %w", prompt.ID, err)
-		}
-		if prepared.OutputContract.RepairAttempts != 1 {
-			return fmt.Errorf("prompt %q repair attempts = %d, want 1", prompt.ID, prepared.OutputContract.RepairAttempts)
-		}
-		preparedPrompts++
-		return nil
-	}); err != nil {
-		t.Fatal(err)
-	}
-	if preparedPrompts == 0 {
-		t.Fatal("prepared no production prompts")
 	}
 }
 
@@ -737,13 +682,13 @@ func TestProductionConfigValidationCoversModuleAndVariantFailures(t *testing.T)
 	}{
 		{
 			name:      "unknown module",
-			content:   replaceRequiredOnce(t, base, "    input: seriatim\n", "    input: missing/input\n"),
+			content:   replaceRequiredOnce(t, base, "    input:\n      module: seriatim\n", "    input:\n      module: missing/input\n"),
 			options:   productionCLIOptions(t),
 			fragments: []string{"pipeline \"dnd-session\"", "input", "missing/input"},
 		},
 		{
 			name:      "unknown validator",
-			content:   replaceRequiredOnce(t, base, "        extract: dnd/spells\n", "        extract:\n          module: dnd/spells\n          validators:\n            - module: missing/validator\n"),
+			content:   replaceRequiredOnce(t, base, "        extract:\n          module: dnd/spells\n", "        extract:\n          module: dnd/spells\n          validators:\n            - module: missing/validator\n"),
 			options:   productionCLIOptions(t),
 			fragments: []string{"validator", "missing/validator"},
 		},
@@ -755,7 +700,7 @@ func TestProductionConfigValidationCoversModuleAndVariantFailures(t *testing.T)
 		},
 		{
 			name:      "deterministic validator with profile",
-			content:   replaceRequiredOnce(t, base, "        extract: dnd/spells\n", "        extract:\n          module: dnd/spells\n          validators:\n            - module: generic/valid_json\n              llm_profile: forbidden-profile\n"),
+			content:   replaceRequiredOnce(t, base, "        extract:\n          module: dnd/spells\n", "        extract:\n          module: dnd/spells\n          validators:\n            - module: generic/valid_json\n              llm_profile: forbidden-profile\n"),
 			options:   productionCLIOptions(t),
 			fragments: []string{"deterministic validator", "llm_profile"},
 		},
@@ -779,7 +724,7 @@ func TestProductionConfigValidationCoversModuleAndVariantFailures(t *testing.T)
 
 func TestProductionNormalizeValidatorOverrideRemainsAuthoritative(t *testing.T) {
 	base := string(readRepositoryFile(t, "examples", "dnd-minimal.config.yml"))
-	content := replaceRequiredOnce(t, base, "        normalize: dnd/spells\n", "        normalize:\n          module: dnd/spells\n          validators:\n            - module: generic/always_accept\n            - module: generic/valid_json\n")
+	content := replaceRequiredOnce(t, base, "        normalize:\n          module: dnd/spells\n", "        normalize:\n          module: dnd/spells\n          validators:\n            - module: generic/always_accept\n            - module: generic/valid_json\n")
 	path := writeProductionContractConfig(t, content)
 	components := productionTestComponents(t)
 	effective, err := loadMaintainedExample(t, path).Resolve(resolveInputForMaintainedExample(components, "dnd-session"))
@@ -879,7 +824,7 @@ func maintainedExampleFiles(t *testing.T) []maintainedExample {
 
 func productionSpellCatalogContractConfig(t *testing.T) string {
 	t.Helper()
-	return fmt.Sprintf(`version: 4
+	return fmt.Sprintf(`version: 5
 cache:
   chunk_plans:
     mode: bypass
@@ -888,21 +833,26 @@ cache:
     directory: ""
 pipelines:
   dnd-session:
-    input: seriatim
+    input:
+      module: seriatim
     references:
-      party: %q
+      party:
-      glossary: %q
+        path: %q
+      glossary:
+        path: %q
     artifacts:
       spells:
         extract:
           module: dnd/spells
           retries: 2
           references:
-            spell_catalog: %q
+            spell_catalog:
+              path: %q
         normalize:
           module: dnd/spells
           references:
-            spell_catalog: %q
+            spell_catalog:
+              path: %q
 `, repositoryPath("examples", "dnd-party.txt"), repositoryPath("examples", "dnd-glossary.txt"), repositoryPath("examples", "dnd-spell-catalog.json"), repositoryPath("examples", "dnd-spell-catalog.json"))
 }
 
@@ -913,14 +863,11 @@ func writeProductionSpellCatalogContractConfig(t *testing.T) string {
 
 func loadMaintainedExample(t *testing.T, path string) config.Config {
 	t.Helper()
-	fileConfig, err := config.LoadFileConfig(path)
+	loaded, err := config.Load(path, config.LoadOptions{})
 	if err != nil {
 		t.Fatalf("load maintained config %q: %v", path, err)
 	}
-	cfg := config.Default()
+	cfg := loaded.RuntimeConfig()
-	if err := cfg.ApplyFileConfig(fileConfig); err != nil {
-		t.Fatalf("apply maintained config %q: %v", path, err)
-	}
 	if err := cfg.Validate(); err != nil {
 		t.Fatalf("validate maintained config %q: %v", path, err)
 	}
@@ -977,7 +924,7 @@ func productionRunOptions(t *testing.T, fake *productionFakeLLMClient) Options {
 }
 
 func productionRunConfig(outputRoot, chunkModule string) string {
-	return fmt.Sprintf(`version: 4
+	return fmt.Sprintf(`version: 5
 output:
   directory: %q
 cache:
@@ -988,8 +935,10 @@ debug:
   directory: %q
 pipelines:
   dnd-session:
-    input: seriatim
+    input:
-    chunk: %s
+      module: seriatim
+    chunk:
+      module: %s
     output:
       module: json
       options:
@@ -998,8 +947,10 @@ pipelines:
       - id: identify-items
         artifacts:
           item-registry:
-            extract: dnd/item-registry
+            extract:
-            normalize: dnd/item-registry
+              module: dnd/item-registry
+            normalize:
+              module: dnd/item-registry
       - id: extract-occurrences
         references:
           item_registry:
@@ -1008,9 +959,11 @@ pipelines:
               lane: item-registry
         artifacts:
           spells:
-            extract: dnd/spells
+            extract:
+              module: dnd/spells
           item-occurrences:
-            extract: dnd/item-occurrences
+            extract:
+              module: dnd/item-occurrences
 `, outputRoot, filepath.Join(filepath.Dir(outputRoot), "debug"), chunkModule)
 }
 

internal/cli/recompute_execution_contract_test.go

@@ -194,7 +194,7 @@ func (recomputeTestOutput) Encode(_ context.Context, req contracts.OutputRequest
 func newRecomputeTestRoots(t *testing.T) stateTestRoots {
 	t.Helper()
 	roots := newStateTestRoots(t)
-	config := fmt.Sprintf(`version: 4
+	config := fmt.Sprintf(`version: 5
 output:
   directory: %q
 cache:
@@ -208,19 +208,27 @@ debug:
   directory: %q
 pipelines:
   sample:
-    input: test/input
+    input:
-    chunk: test/chunk
+      module: test/input
+    chunk:
+      module: test/chunk
     steps:
       - id: first
         artifacts:
           producer:
-            extract: test/extract/producer
+            extract:
-            merge: test/merge
+              module: test/extract/producer
-            normalize: test/normalize
+            merge:
+              module: test/merge
+            normalize:
+              module: test/normalize
           unrelated:
-            extract: test/extract/unrelated
+            extract:
-            merge: test/merge
+              module: test/extract/unrelated
-            normalize: test/normalize
+            merge:
+              module: test/merge
+            normalize:
+              module: test/normalize
       - id: second
         references:
           upstream:
@@ -229,9 +237,12 @@ pipelines:
               lane: producer
         artifacts:
           middle:
-            extract: test/extract/middle
+            extract:
-            merge: test/merge
+              module: test/extract/middle
-            normalize: test/normalize
+            merge:
+              module: test/merge
+            normalize:
+              module: test/normalize
       - id: third
         references:
           upstream:
@@ -240,10 +251,14 @@ pipelines:
               lane: middle
         artifacts:
           dependent:
-            extract: test/extract/dependent
+            extract:
-            merge: test/merge
+              module: test/extract/dependent
-            normalize: test/normalize
+            merge:
-    output: test/recompute-output
+              module: test/merge
+            normalize:
+              module: test/normalize
+    output:
+      module: test/recompute-output
 `, roots.output, roots.plans, roots.checkpoints, roots.debug)
 	if err := os.WriteFile(roots.config, []byte(config), 0o600); err != nil {
 		t.Fatal(err)

internal/cli/reference_contract_test.go

@@ -289,7 +289,7 @@ func TestReferenceMaterializationSeparatesCLIAndConfigPathOrigins(t *testing.T)
 	workingDir := t.TempDir()
 	cfg := referenceContractConfig()
 	configPath := filepath.Join(configDir, "config.yml")
-	if err := os.WriteFile(configPath, []byte("version: 4\n"), 0o600); err != nil {
+	if err := os.WriteFile(configPath, []byte("version: 5\n"), 0o600); err != nil {
 		t.Fatal(err)
 	}
 	if err := os.WriteFile(filepath.Join(configDir, "required.txt"), []byte("config reference"), 0o600); err != nil {

internal/cli/run.go

@@ -4,7 +4,6 @@ import (
 	"context"
 	"crypto/sha256"
 	"encoding/hex"
-	"encoding/json"
 	"errors"
 	"flag"
 	"fmt"
@@ -16,6 +15,7 @@ import (
 	"strings"
 	"time"
 
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
 	"gitea.maximumdirect.net/eric/notarius/internal/buildinfo"
 	"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
 	"gitea.maximumdirect.net/eric/notarius/internal/core/config"
@@ -34,8 +34,10 @@ const usage = `Usage:
   notarius help
   notarius --version
   notarius run <pipeline-id> --input path/to/source.json [--json] [flags]
-  notarius config validate --config path/to/config.yml [--pipeline pipeline-id] [--only lane-a,lane-b]
+  notarius config validate --config path/to/config.yml [--config-profile name] [--pipeline pipeline-id] [--only lane-a,lane-b]
-  notarius pipelines list --config path/to/config.yml [--json]
+  notarius config effective --config path/to/config.yml [--config-profile name]
+  notarius config explain --config path/to/config.yml [--config-profile name] <json-pointer>
+  notarius pipelines list --config path/to/config.yml [--config-profile name] [--json]
 `
 
 type Options struct {
@@ -158,6 +160,7 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
 	fs := flag.NewFlagSet("run", flag.ContinueOnError)
 	fs.SetOutput(io.Discard)
 	configPath := fs.String("config", "", "config file path")
+	configProfile := fs.String("config-profile", "", "configuration profile")
 	inputPath := fs.String("input", "", "source input file path")
 	onlyRaw := fs.String("only", "", "comma-separated artifact lanes")
 	outputDir := fs.String("output-dir", "", "output directory")
@@ -265,20 +268,12 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
 		return 2
 	}
 
-	cfg, loadedConfigPath, err := loadConfig(*configPath, opts)
+	loaded, loadedConfigPath, err := loadConfigKit(*configPath, *configProfile, runConfigurationOverrides(chunkCache, *outputDir, *debugDir), opts)
 	if err != nil {
-		fmt.Fprintf(stderr, "notarius: %v\n", err)
+		writeConfigLoadError(stderr, err)
 		return 1
 	}
-	if chunkCache.set {
+	cfg := loaded.RuntimeConfig()
-		cfg.Cache.ChunkPlans.Mode = chunkCache.value
-	}
-	if dir := strings.TrimSpace(*outputDir); dir != "" {
-		cfg.Output.Directory = dir
-	}
-	if dir := strings.TrimSpace(*debugDir); dir != "" {
-		cfg.Debug.Directory = dir
-	}
 	if err := cfg.Validate(); err != nil {
 		fmt.Fprintf(stderr, "notarius: %v\n", err)
 		return 1
@@ -336,6 +331,7 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
 		InputPath:               strings.TrimSpace(*inputPath),
 		ConfigPath:              loadedConfigPath,
 		ConfigSource:            configSource(*configPath),
+		ConfigProfile:           effectiveConfigProfile(*configProfile),
 		OnlyLanes:               append([]string(nil), only...),
 		ChunkCacheOverride:      chunkCache.explicitValue(),
 		ReasoningEffortOverride: runtimeOverrides.ReasoningEffort,
@@ -394,7 +390,7 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
 	if err := writeSummary(summary, func() error { return summary.WriteInvocation(invocation) }); err != nil {
 		return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("write debug invocation metadata: %w", err))
 	}
-	if err := writeSummary(summary, func() error { return summary.WriteRedactedEffectiveConfig(effective) }); err != nil {
+	if err := writeSummary(summary, func() error { return summary.WriteEffectiveConfig(loaded) }); err != nil {
 		return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("write debug effective config: %w", err))
 	}
 	if err := writeSummary(summary, func() error { return summary.WriteResolvedPipeline(effective) }); err != nil {
@@ -881,13 +877,39 @@ func reorderRunArgs(args []string) []string {
 
 func runFlagTakesValue(arg string) bool {
 	switch arg {
-	case "--config", "--input", "--only", "--output-dir", "--debug-dir", "--llm-profile", "--session-id", "--reasoning-effort", "--chunk_cache", "--reference", "--without-reference", "--recompute-step":
+	case "--config", "--config-profile", "--input", "--only", "--output-dir", "--debug-dir", "--llm-profile", "--session-id", "--reasoning-effort", "--chunk_cache", "--reference", "--without-reference", "--recompute-step":
 		return true
 	default:
 		return false
 	}
 }
 
+func runConfigurationOverrides(chunkCache chunkCacheFlag, outputDir, debugDir string) []configkit.Override {
+	overrides := make([]configkit.Override, 0, 3)
+	if chunkCache.set {
+		overrides = append(overrides, configkit.Override{
+			Path:  configkit.Keys("cache", "chunk_plans", "mode"),
+			Value: string(chunkCache.value),
+			Flag:  "--chunk_cache",
+		})
+	}
+	if outputDir = strings.TrimSpace(outputDir); outputDir != "" {
+		overrides = append(overrides, configkit.Override{
+			Path:  configkit.Keys("output", "directory"),
+			Value: outputDir,
+			Flag:  "--output-dir",
+		})
+	}
+	if debugDir = strings.TrimSpace(debugDir); debugDir != "" {
+		overrides = append(overrides, configkit.Override{
+			Path:  configkit.Keys("debug", "directory"),
+			Value: debugDir,
+			Flag:  "--debug-dir",
+		})
+	}
+	return overrides
+}
+
 type chunkCacheFlag struct {
 	value pipeline.ChunkCacheMode
 	set   bool
@@ -941,7 +963,7 @@ func chunkPlanStoreForRun(cfg config.ChunkPlanCacheConfig, opts Options) (pipeli
 
 func validateRunFlagValues(args []string) error {
 	for i, arg := range args {
-		if arg != "--session-id" && arg != "--reasoning-effort" {
+		if arg != "--config-profile" && arg != "--session-id" && arg != "--reasoning-effort" {
 			continue
 		}
 		if i+1 >= len(args) || strings.HasPrefix(args[i+1], "-") {
@@ -951,6 +973,13 @@ func validateRunFlagValues(args []string) error {
 	return nil
 }
 
+func effectiveConfigProfile(profile string) string {
+	if profile == "" || profile == "default" {
+		return ""
+	}
+	return profile
+}
+
 func flagWasProvided(args []string, name string) bool {
 	for _, arg := range args {
 		if arg == name || strings.HasPrefix(arg, name+"=") {
@@ -1026,6 +1055,10 @@ func runConfig(args []string, stdout, stderr io.Writer, opts Options) int {
 	switch args[0] {
 	case "validate":
 		return runConfigValidate(args[1:], stdout, stderr, opts)
+	case "effective":
+		return runConfigEffective(args[1:], stdout, stderr, opts)
+	case "explain":
+		return runConfigExplain(args[1:], stdout, stderr, opts)
 	default:
 		fmt.Fprintf(stderr, "notarius: unknown config subcommand %q\n", args[0])
 		writeUsage(stderr)
@@ -1033,67 +1066,6 @@ func runConfig(args []string, stdout, stderr io.Writer, opts Options) int {
 	}
 }
 
-func runConfigValidate(args []string, stdout, stderr io.Writer, opts Options) int {
-	fs := flag.NewFlagSet("config validate", flag.ContinueOnError)
-	fs.SetOutput(io.Discard)
-	configPath := fs.String("config", "", "config file path")
-	pipelineID := fs.String("pipeline", "", "pipeline ID")
-	onlyRaw := fs.String("only", "", "comma-separated artifact lanes")
-	if err := fs.Parse(args); err != nil {
-		fmt.Fprintf(stderr, "notarius: %v\n", err)
-		return 2
-	}
-	if fs.NArg() != 0 {
-		fmt.Fprintf(stderr, "notarius: unexpected argument %q\n", fs.Arg(0))
-		return 2
-	}
-	if strings.TrimSpace(*onlyRaw) != "" && strings.TrimSpace(*pipelineID) == "" {
-		fmt.Fprintln(stderr, "notarius: --only requires --pipeline")
-		return 2
-	}
-	only, err := parseOnly(*onlyRaw)
-	if err != nil {
-		fmt.Fprintf(stderr, "notarius: %v\n", err)
-		return 2
-	}
-
-	cfg, path, err := loadConfig(*configPath, opts)
-	if err != nil {
-		fmt.Fprintf(stderr, "notarius: %v\n", err)
-		return 1
-	}
-
-	if strings.TrimSpace(*pipelineID) != "" {
-		catalog, err := effectiveCatalog(opts)
-		if err != nil {
-			fmt.Fprintf(stderr, "notarius: %v\n", err)
-			return 1
-		}
-		effective, err := cfg.Resolve(config.ResolveInput{
-			PipelineID: *pipelineID,
-			Only:       only,
-			Catalog:    catalog,
-		})
-		if err != nil {
-			fmt.Fprintf(stderr, "notarius: %v\n", err)
-			return 1
-		}
-		if err := validateExplicitPromptKitProfiles(context.Background(), effective.Config, effectiveLLMProfileIDs(effective.ResolvedPipeline), opts.promptKitAssets); err != nil {
-			fmt.Fprintf(stderr, "notarius: %v\n", err)
-			return 1
-		}
-		fmt.Fprintf(stdout, "config %q is valid for pipeline %q\n", path, strings.TrimSpace(*pipelineID))
-		return 0
-	}
-
-	if err := cfg.Validate(); err != nil {
-		fmt.Fprintf(stderr, "notarius: %v\n", err)
-		return 1
-	}
-	fmt.Fprintf(stdout, "config %q is valid\n", path)
-	return 0
-}
-
 func runPipelines(args []string, stdout, stderr io.Writer, opts Options) int {
 	if len(args) == 0 {
 		fmt.Fprintln(stderr, "notarius: pipelines requires a subcommand")
@@ -1110,70 +1082,6 @@ func runPipelines(args []string, stdout, stderr io.Writer, opts Options) int {
 	}
 }
 
-func runPipelinesList(args []string, stdout, stderr io.Writer, opts Options) int {
-	fs := flag.NewFlagSet("pipelines list", flag.ContinueOnError)
-	fs.SetOutput(io.Discard)
-	configPath := fs.String("config", "", "config file path")
-	jsonOutput := fs.Bool("json", false, "write JSON output")
-	if err := fs.Parse(args); err != nil {
-		fmt.Fprintf(stderr, "notarius: %v\n", err)
-		return 2
-	}
-	if fs.NArg() != 0 {
-		fmt.Fprintf(stderr, "notarius: unexpected argument %q\n", fs.Arg(0))
-		return 2
-	}
-
-	cfg, _, err := loadConfig(*configPath, opts)
-	if err != nil {
-		fmt.Fprintf(stderr, "notarius: %v\n", err)
-		return 1
-	}
-	if err := cfg.Validate(); err != nil {
-		fmt.Fprintf(stderr, "notarius: %v\n", err)
-		return 1
-	}
-
-	ids := sortedPipelineIDs(cfg)
-	if *jsonOutput {
-		payload := struct {
-			Pipelines []string `json:"pipelines"`
-		}{Pipelines: ids}
-		encoded, err := json.Marshal(payload)
-		if err != nil {
-			fmt.Fprintf(stderr, "notarius: marshal pipeline list: %v\n", err)
-			return 1
-		}
-		fmt.Fprintf(stdout, "%s\n", encoded)
-		return 0
-	}
-
-	for _, id := range ids {
-		fmt.Fprintln(stdout, id)
-	}
-	return 0
-}
-
-func loadConfig(configPath string, opts Options) (config.Config, string, error) {
-	path, err := discoverConfigPath(configPath, opts)
-	if err != nil {
-		return config.Config{}, "", err
-	}
-
-	fileCfg, err := config.LoadFileConfig(path)
-	if err != nil {
-		return config.Config{}, "", err
-	}
-	cfg := config.Default()
-	if err := cfg.ApplyFileConfigWithLookup(fileCfg, opts.LookupEnv); err != nil {
-		return config.Config{}, "", err
-	}
-	if err := cfg.ApplyEnvOverridesWithLookup(opts.LookupEnv); err != nil {
-		return config.Config{}, "", err
-	}
-	return cfg, path, nil
-}
-
 func discoverConfigPath(configPath string, opts Options) (string, error) {
 	if path := strings.TrimSpace(configPath); path != "" {
 		if err := requireConfigFile(path); err != nil {

internal/cli/run_contract_test.go

@@ -12,6 +12,7 @@ import (
 
 	"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
 	"gitea.maximumdirect.net/eric/notarius/internal/core/config"
+	"gitea.maximumdirect.net/eric/notarius/internal/core/debugbundle"
 	"gitea.maximumdirect.net/eric/notarius/internal/framework/checkpoint"
 	"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
 	"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
@@ -37,6 +38,9 @@ func TestRunControlsRejectSyntaxWithoutAllocatingState(t *testing.T) {
 		{name: "blank debug directory", args: func(roots stateTestRoots) []string {
 			return []string{"run", "sample", "--config", roots.config, "--input", roots.input, "--debug-dir", ""}
 		}},
+		{name: "missing configuration profile", args: func(roots stateTestRoots) []string {
+			return []string{"run", "sample", "--config", roots.config, "--input", roots.input, "--config-profile"}
+		}},
 		{name: "debug directory without debug", args: func(roots stateTestRoots) []string {
 			return []string{"run", "sample", "--config", roots.config, "--input", roots.input, "--debug-dir", filepath.Join(filepath.Dir(roots.debug), "requested-debug")}
 		}},
@@ -79,7 +83,7 @@ func TestRecomputeStepCLIContract(t *testing.T) {
 		{
 			name: "explicit step",
 			configure: func(t *testing.T, roots stateTestRoots) {
-				replaceStateTestConfigLine(t, roots.config, "    artifacts:\n      items:\n        extract: test/extract\n        merge: test/merge\n        normalize: test/normalize\n", "    steps:\n      - id: chosen\n        artifacts:\n          items:\n            extract: test/extract\n            merge: test/merge\n            normalize: test/normalize\n")
+				replaceStateTestConfigLine(t, roots.config, "    artifacts:\n      items:\n        extract:\n          module: test/extract\n        merge:\n          module: test/merge\n        normalize:\n          module: test/normalize\n", "    steps:\n      - id: chosen\n        artifacts:\n          items:\n            extract:\n              module: test/extract\n            merge:\n              module: test/merge\n            normalize:\n              module: test/normalize\n")
 			},
 			flags:      []string{"--resume", "--recompute-step", "chosen"},
 			wantCode:   0,
@@ -161,7 +165,7 @@ func TestRunOnlyExecutesSelectedLanes(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	data = []byte(replaceRequiredOnce(t, string(data), "    output: test/output\n", "      other:\n        extract: test/extract\n    output: test/output\n"))
+	data = []byte(replaceRequiredOnce(t, string(data), "    output:\n      module: test/output\n", "      other:\n        extract:\n          module: test/extract\n    output:\n      module: test/output\n"))
 	if err := os.WriteFile(roots.config, data, 0o600); err != nil {
 		t.Fatal(err)
 	}
@@ -180,7 +184,7 @@ func TestRunOnlyExecutesSelectedLanes(t *testing.T) {
 }
 
 func TestRunStateRootsHonorEnvironmentFlagsAndDefaults(t *testing.T) {
-	t.Run("environment roots", func(t *testing.T) {
+	t.Run("retired environment roots have no effect", func(t *testing.T) {
 		roots := newStateTestRoots(t)
 		environmentOutput := filepath.Join(t.TempDir(), "environment-output")
 		environmentDebug := filepath.Join(t.TempDir(), "environment-debug")
@@ -193,10 +197,10 @@ func TestRunStateRootsHonorEnvironmentFlagsAndDefaults(t *testing.T) {
 		if result.code != 0 {
 			t.Fatalf("code=%d stdout=%q stderr=%q", result.code, result.stdout, result.stderr)
 		}
-		assertFile(t, filepath.Join(environmentOutput, filepath.Base(onlyChildDir(t, environmentOutput)), "result.json"))
+		assertFile(t, filepath.Join(roots.output, filepath.Base(onlyChildDir(t, roots.output)), "result.json"))
-		onlyChildDir(t, environmentDebug)
+		onlyChildDir(t, roots.debug)
-		assertAbsent(t, roots.output)
+		assertAbsent(t, environmentOutput)
-		assertAbsent(t, roots.debug)
+		assertAbsent(t, environmentDebug)
 	})
 
 	t.Run("command flags override environment", func(t *testing.T) {
@@ -227,23 +231,57 @@ func TestRunStateRootsHonorEnvironmentFlagsAndDefaults(t *testing.T) {
 			t.Fatal(err)
 		}
 		text := string(data)
-		text = replaceRequiredOnce(t, text, fmt.Sprintf("  directory: %q\n", roots.output), "")
+		text = replaceRequiredOnce(t, text, fmt.Sprintf("output:\n  directory: %q\n", roots.output), "")
-		text = replaceRequiredOnce(t, text, fmt.Sprintf("  directory: %q\n", roots.debug), "")
+		text = replaceRequiredOnce(t, text, fmt.Sprintf("debug:\n  directory: %q\n", roots.debug), "")
 		if err := os.WriteFile(roots.config, []byte(text), 0o600); err != nil {
 			t.Fatal(err)
 		}
-		workDir := t.TempDir()
-		t.Chdir(workDir)
 		opts := newStateTestHarness().options()
 		result := runWithStateRoots(t, roots, opts, nil)
 		if result.code != 0 {
 			t.Fatalf("code=%d stdout=%q stderr=%q", result.code, result.stdout, result.stderr)
 		}
-		assertFile(t, filepath.Join(workDir, "notarius-output", filepath.Base(onlyChildDir(t, filepath.Join(workDir, "notarius-output"))), "result.json"))
+		configDirectory := filepath.Dir(roots.config)
-		onlyChildDir(t, filepath.Join(workDir, "notarius-debug"))
+		assertFile(t, filepath.Join(configDirectory, "notarius-output", filepath.Base(onlyChildDir(t, filepath.Join(configDirectory, "notarius-output"))), "result.json"))
+		onlyChildDir(t, filepath.Join(configDirectory, "notarius-debug"))
 	})
 }
 
+func TestRunAppliesConfigurationProfileAndRecordsSelection(t *testing.T) {
+	roots := newStateTestRoots(t)
+	profileDirectory := filepath.Join(filepath.Dir(roots.config), "profiles")
+	profileOutput := filepath.Join(profileDirectory, "profile-output")
+	profileDebug := filepath.Join(profileDirectory, "profile-debug")
+	writeConfigKitFile(t, filepath.Join(profileDirectory, "development.yml"), "output:\n  directory: ./profile-output\ndebug:\n  directory: ./profile-debug\n")
+
+	var stdout, stderr bytes.Buffer
+	code := RunWithOptions([]string{
+		"run", "sample", "--config", roots.config, "--config-profile", "development",
+		"--input", roots.input, "--chunk_cache", "bypass", "--debug",
+	}, &stdout, &stderr, newStateTestHarness().options())
+	if code != 0 || stderr.Len() != 0 {
+		t.Fatalf("code=%d stdout=%q stderr=%q", code, stdout.String(), stderr.String())
+	}
+	assertFile(t, filepath.Join(profileOutput, filepath.Base(onlyChildDir(t, profileOutput)), "result.json"))
+	bundle := onlyChildDir(t, profileDebug)
+	var invocation debugbundle.Invocation
+	readStateTestSummaryJSON(t, bundle, "invocation.json", &invocation)
+	if invocation.ConfigProfile != "development" {
+		t.Fatalf("config profile = %q, want development", invocation.ConfigProfile)
+	}
+}
+
+func TestEffectiveConfigProfileOmitsBaseSelection(t *testing.T) {
+	for _, profile := range []string{"", "default"} {
+		if got := effectiveConfigProfile(profile); got != "" {
+			t.Fatalf("effectiveConfigProfile(%q) = %q, want empty", profile, got)
+		}
+	}
+	if got := effectiveConfigProfile("development"); got != "development" {
+		t.Fatalf("effectiveConfigProfile(development) = %q", got)
+	}
+}
+
 func TestRunLLMProfileOverrideAndValidationUseInjectedBoundaries(t *testing.T) {
 	t.Run("one effective profile reaches the factory and modules", func(t *testing.T) {
 		roots := newStateTestRoots(t)
@@ -567,7 +605,7 @@ func TestRunFactoryAndPreparationFailuresAreProcessFailures(t *testing.T) {
 		if err != nil {
 			t.Fatal(err)
 		}
-		data = []byte(replaceRequiredOnce(t, string(data), "extract: test/extract", "extract: test/failing-extract"))
+		data = []byte(replaceRequiredOnce(t, string(data), "module: test/extract", "module: test/failing-extract"))
 		if err := os.WriteFile(roots.config, data, 0o600); err != nil {
 			t.Fatal(err)
 		}

internal/cli/run_result_command_test.go

@@ -92,7 +92,7 @@ func TestRunResultReportsSuccessfulRejection(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	configBytes = []byte(replaceRequiredOnce(t, string(configBytes), "        normalize: test/normalize\n", "        normalize:\n          module: test/normalize\n          validators:\n            - generic/always_reject\n          validation_policy:\n            semantic_rejection: reject_output\n"))
+	configBytes = []byte(replaceRequiredOnce(t, string(configBytes), "        normalize:\n          module: test/normalize\n", "        normalize:\n          module: test/normalize\n          validators:\n            - module: generic/always_reject\n          validation_policy:\n            semantic_rejection: reject_output\n"))
 	if err := os.WriteFile(roots.config, configBytes, 0o600); err != nil {
 		t.Fatal(err)
 	}

internal/cli/state_hardening_test.go

@@ -57,7 +57,6 @@ func TestRunStateSurfaceMatrix(t *testing.T) {
 						}
 					}
 					assertAnyFile(t, roots.checkpoints)
-					assertRestrictedTree(t, roots.checkpoints)
 					if debug {
 						bundle := onlyChildDir(t, roots.debug)
 						assertFile(t, filepath.Join(bundle, "summary", "invocation.json"))
@@ -259,7 +258,7 @@ func TestRunRedactsSensitiveModuleOptionsFromConfigAndPipelineSummaries(t *testi
 	if err != nil {
 		t.Fatal(err)
 	}
-	configText := replaceRequiredOnce(t, string(data), "    input: test/input\n", `    input:
+	configText := replaceRequiredOnce(t, string(data), "    input:\n      module: test/input\n", `    input:
       module: test/input
       options:
         api_key: CONFIG_SUMMARY_SECRET_SENTINEL
@@ -277,7 +276,7 @@ func TestRunRedactsSensitiveModuleOptionsFromConfigAndPipelineSummaries(t *testi
 		t.Fatalf("code=%d stderr=%q", result.code, result.stderr)
 	}
 	summaryRoot := filepath.Join(onlyChildDir(t, roots.debug), "summary")
-	for _, name := range []string{"effective-config.json", "resolved-pipeline.json"} {
+	for _, name := range []string{"effective-config.yml", "resolved-pipeline.json"} {
 		contents, err := os.ReadFile(filepath.Join(summaryRoot, name))
 		if err != nil {
 			t.Fatal(err)
@@ -688,7 +687,7 @@ func newStateTestRoots(t *testing.T) stateTestRoots {
 		t.Fatal(err)
 	}
 	roots.config = filepath.Join(base, "config.yml")
-	config := fmt.Sprintf("version: 4\noutput:\n  directory: %q\ncache:\n  chunk_plans:\n    directory: %q\n    mode: auto\n  checkpoints:\n    enabled: true\n    directory: %q\ndebug:\n  directory: %q\npipelines:\n  sample:\n    input: test/input\n    chunk: test/chunk\n    artifacts:\n      items:\n        extract: test/extract\n        merge: test/merge\n        normalize: test/normalize\n    output: test/output\n", roots.output, roots.plans, roots.checkpoints, roots.debug)
+	config := fmt.Sprintf("version: 5\noutput:\n  directory: %q\ncache:\n  chunk_plans:\n    directory: %q\n    mode: auto\n  checkpoints:\n    enabled: true\n    directory: %q\ndebug:\n  directory: %q\npipelines:\n  sample:\n    input:\n      module: test/input\n    chunk:\n      module: test/chunk\n    artifacts:\n      items:\n        extract:\n          module: test/extract\n        merge:\n          module: test/merge\n        normalize:\n          module: test/normalize\n    output:\n      module: test/output\n", roots.output, roots.plans, roots.checkpoints, roots.debug)
 	if err := os.WriteFile(roots.config, []byte(config), 0o600); err != nil {
 		t.Fatal(err)
 	}

internal/cli/state_permissions_unix_test.go

@@ -0,0 +1,53 @@
+//go:build linux || darwin
+
+package cli
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"syscall"
+	"testing"
+)
+
+func TestRunCreatesReusableGroupAccessibleCacheState(t *testing.T) {
+	previousUmask := syscall.Umask(0o007)
+	defer syscall.Umask(previousUmask)
+
+	roots := newStateTestRoots(t)
+	harness := newStateTestHarness()
+	result := runStateTest(t, roots, harness.options(), false, false, "auto")
+	if result.code != 0 {
+		t.Fatalf("initial run code=%d stderr=%q", result.code, result.stderr)
+	}
+	assertGroupAccessibleCacheTree(t, roots.plans)
+	assertGroupAccessibleCacheTree(t, roots.checkpoints)
+
+	extractCalls := harness.extractCalls
+	result = runStateTest(t, roots, harness.options(), false, true, "auto")
+	if result.code != 0 {
+		t.Fatalf("resume run code=%d stderr=%q", result.code, result.stderr)
+	}
+	if harness.extractCalls != extractCalls {
+		t.Fatalf("extract calls after reuse = %d, want %d", harness.extractCalls, extractCalls)
+	}
+}
+
+func assertGroupAccessibleCacheTree(t *testing.T, root string) {
+	t.Helper()
+	if err := filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
+		if err != nil {
+			return err
+		}
+		want := os.FileMode(0o660)
+		if info.IsDir() {
+			want = 0o770
+		}
+		if got := info.Mode().Perm(); got != want {
+			return fmt.Errorf("%s mode = %#o, want %#o", path, got, want)
+		}
+		return nil
+	}); err != nil {
+		t.Fatal(err)
+	}
+}

internal/core/config/config.go

@@ -4,8 +4,6 @@ import (
 	"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
 )
 
-const SupportedFileConfigVersion = 4
-
 type Config struct {
 	PromptKit   PromptKitConfig                     `json:"promptkit,omitempty"`
 	Pipelines   map[string]pipeline.PipelineProfile `json:"pipelines"`
@@ -29,9 +27,6 @@ type PromptKitLocalBackendConfig struct {
 type ConcurrencyConfig struct {
 	TotalLLM     int            `json:"total_llm"`
 	StageWorkers map[string]int `json:"stage_workers"`
-
-	extractWorkersConfigured bool
-	defaultedExtractWorkers  int
 }
 
 const defaultLLMConcurrency = 16
@@ -62,9 +57,8 @@ func Default() Config {
 	return Config{
 		Pipelines: map[string]pipeline.PipelineProfile{},
 		Concurrency: ConcurrencyConfig{
-			TotalLLM:                defaultLLMConcurrency,
+			TotalLLM:     defaultLLMConcurrency,
-			StageWorkers:            map[string]int{"extract": defaultLLMConcurrency},
+			StageWorkers: map[string]int{"extract": defaultLLMConcurrency},
-			defaultedExtractWorkers: defaultLLMConcurrency,
 		},
 		Output: OutputConfig{Directory: "./notarius-output"},
 		Cache:  CacheConfig{ChunkPlans: ChunkPlanCacheConfig{Mode: pipeline.ChunkCacheAuto}},
@@ -97,21 +91,15 @@ func cloneIntMap(in map[string]int) map[string]int {
 	return out
 }
 
-func (c *ConcurrencyConfig) recomputeStageWorkerDefaults() {
+func (c Config) withDerivedStageWorkers() Config {
-	if c == nil {
+	c = cloneConfig(c)
-		return
+	if _, exists := c.Concurrency.StageWorkers["extract"]; !exists {
-	}
+		if c.Concurrency.StageWorkers == nil {
-	if c.StageWorkers == nil {
+			c.Concurrency.StageWorkers = make(map[string]int)
-		c.StageWorkers = make(map[string]int)
-	}
-	if !c.extractWorkersConfigured {
-		if value, ok := c.StageWorkers["extract"]; ok && (c.defaultedExtractWorkers == 0 || value != c.defaultedExtractWorkers) {
-			c.extractWorkersConfigured = true
-			return
 		}
-		c.StageWorkers["extract"] = c.TotalLLM
+		c.Concurrency.StageWorkers["extract"] = c.Concurrency.TotalLLM
-		c.defaultedExtractWorkers = c.TotalLLM
 	}
+	return c
 }
 
 func clonePipelineProfile(in pipeline.PipelineProfile) pipeline.PipelineProfile {

internal/core/config/document.go

@@ -0,0 +1,122 @@
+package config
+
+import "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
+
+// Document is the version 5 configuration shape decoded by ConfigKit. It is
+// deliberately separate from Config so YAML presence and composition concerns
+// do not reach runtime pipeline types.
+type Document struct {
+	Version     string                      `yaml:"version"`
+	PromptKit   *PromptKitDocument          `yaml:"promptkit,omitempty"`
+	Pipelines   map[string]PipelineDocument `yaml:"pipelines,omitempty"`
+	Concurrency *ConcurrencyDocument        `yaml:"concurrency,omitempty"`
+	Output      *OutputDocument             `yaml:"output,omitempty"`
+	Cache       *CacheDocument              `yaml:"cache,omitempty"`
+	Debug       *DebugDocument              `yaml:"debug,omitempty"`
+}
+
+type PromptKitDocument struct {
+	ProfileDir   *string                        `yaml:"profile_dir,omitempty"`
+	ProfileFile  *string                        `yaml:"profile_file,omitempty"`
+	LocalBackend *PromptKitLocalBackendDocument `yaml:"local_backend,omitempty"`
+}
+
+type PromptKitLocalBackendDocument struct {
+	Endpoint         *string `yaml:"endpoint,omitempty"`
+	ConcurrencyLimit *int    `yaml:"concurrency_limit,omitempty"`
+}
+
+type PipelineDocument struct {
+	LLMProfile                     *string                         `yaml:"llm_profile,omitempty"`
+	StructuredOutputRepairAttempts *int                            `yaml:"structured_output_repair_attempts,omitempty"`
+	ValidationPolicy               *ValidationPolicyDocument       `yaml:"validation_policy,omitempty"`
+	Input                          BindingDocument                 `yaml:"input"`
+	Chunk                          *BindingDocument                `yaml:"chunk,omitempty"`
+	Artifacts                      map[string]ArtifactLaneDocument `yaml:"artifacts,omitempty"`
+	Steps                          []PipelineStepDocument          `yaml:"steps,omitempty"`
+	Output                         *BindingDocument                `yaml:"output,omitempty"`
+	References                     map[string]ReferenceDocument    `yaml:"references,omitempty"`
+}
+
+type PipelineStepDocument struct {
+	ID         string                          `yaml:"id"`
+	Artifacts  map[string]ArtifactLaneDocument `yaml:"artifacts"`
+	References map[string]ReferenceDocument    `yaml:"references,omitempty"`
+}
+
+type ArtifactLaneDocument struct {
+	Extract    BindingDocument              `yaml:"extract"`
+	Merge      *BindingDocument             `yaml:"merge,omitempty"`
+	Normalize  *BindingDocument             `yaml:"normalize,omitempty"`
+	Validators []ValidatorDocument          `yaml:"validators,omitempty"`
+	References map[string]ReferenceDocument `yaml:"references,omitempty"`
+}
+
+// BindingDocument represents a producer binding. ValidatorDocument is a
+// deliberately finite subset so nested validator-only fields are rejected by
+// ConfigKit's strict decoder.
+type BindingDocument struct {
+	Module                         string                       `yaml:"module"`
+	LLMProfile                     *string                      `yaml:"llm_profile,omitempty"`
+	StructuredOutputRepairAttempts *int                         `yaml:"structured_output_repair_attempts,omitempty"`
+	ValidationPolicy               *ValidationPolicyDocument    `yaml:"validation_policy,omitempty"`
+	Retries                        int                          `yaml:"retries,omitempty"`
+	Options                        map[string]any               `yaml:"options,omitempty"`
+	References                     map[string]ReferenceDocument `yaml:"references,omitempty"`
+	Validators                     []ValidatorDocument          `yaml:"validators,omitempty"`
+}
+
+type ValidatorDocument struct {
+	Module                         string         `yaml:"module"`
+	LLMProfile                     *string        `yaml:"llm_profile,omitempty"`
+	StructuredOutputRepairAttempts *int           `yaml:"structured_output_repair_attempts,omitempty"`
+	Retries                        int            `yaml:"retries,omitempty"`
+	Options                        map[string]any `yaml:"options,omitempty"`
+}
+
+// ReferenceDocument is the explicit version 5 union. Later document
+// validation establishes which locations permit generated references and
+// requires exactly one populated branch.
+type ReferenceDocument struct {
+	Path     *string                    `yaml:"path,omitempty"`
+	Artifact *ArtifactReferenceDocument `yaml:"artifact,omitempty"`
+}
+
+type ArtifactReferenceDocument struct {
+	Step string `yaml:"step"`
+	Lane string `yaml:"lane"`
+}
+
+type ValidationPolicyDocument struct {
+	ProducerStructuralFailure *pipeline.ProducerStructuralFailureAction `yaml:"producer_structural_failure,omitempty"`
+	SemanticRejection         *pipeline.SemanticRejectionAction         `yaml:"semantic_rejection,omitempty"`
+	ValidatorFailure          *pipeline.ValidatorFailureAction          `yaml:"validator_failure,omitempty"`
+}
+
+type ConcurrencyDocument struct {
+	TotalLLM     *int           `yaml:"total_llm,omitempty"`
+	StageWorkers map[string]int `yaml:"stage_workers,omitempty"`
+}
+
+type OutputDocument struct {
+	Directory *string `yaml:"directory,omitempty"`
+}
+
+type CacheDocument struct {
+	ChunkPlans  *ChunkPlanCacheDocument  `yaml:"chunk_plans,omitempty"`
+	Checkpoints *CheckpointCacheDocument `yaml:"checkpoints,omitempty"`
+}
+
+type ChunkPlanCacheDocument struct {
+	Directory *string `yaml:"directory,omitempty"`
+	Mode      *string `yaml:"mode,omitempty"`
+}
+
+type CheckpointCacheDocument struct {
+	Enabled   *bool   `yaml:"enabled,omitempty"`
+	Directory *string `yaml:"directory,omitempty"`
+}
+
+type DebugDocument struct {
+	Directory *string `yaml:"directory,omitempty"`
+}

internal/core/config/document_normalizer.go

@@ -0,0 +1,26 @@
+package config
+
+import configkit "gitea.maximumdirect.net/eric/ConfigKit"
+
+// documentNormalizer records leaf provenance and, while visiting a sequence,
+// covers its atomic mutation boundary. ConfigKit audits sequences as whole
+// values, so a record for only a descendant is insufficient. The outermost
+// sequence covers nested sequences as well; unchanged visits record nothing.
+type documentNormalizer struct {
+	*configkit.Resolution
+	sequence *configkit.Path
+}
+
+func (n documentNormalizer) inSequence(path configkit.Path) documentNormalizer {
+	if n.sequence == nil {
+		n.sequence = &path
+	}
+	return n
+}
+
+func (n documentNormalizer) Record(path configkit.Path, kind configkit.ChangeKind, reason string, dependencies ...configkit.Path) {
+	n.Resolution.Record(path, kind, reason, dependencies...)
+	if n.sequence != nil {
+		n.Resolution.Record(*n.sequence, kind, reason, append([]configkit.Path{path}, dependencies...)...)
+	}
+}

internal/core/config/document_paths.go

@@ -0,0 +1,167 @@
+package config
+
+import (
+	"path/filepath"
+
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
+)
+
+func normalizeDocumentPaths(document *Document, context *configkit.Resolution, state *documentLoadState) {
+	resolution := documentNormalizer{Resolution: context}
+	if document == nil {
+		return
+	}
+	initializeDocumentMainDirectory(resolution, state)
+	if document.PromptKit != nil {
+		normalizeDocumentPath(resolution, state, configkit.Keys("promptkit", "profile_dir"), &document.PromptKit.ProfileDir, false, "resolved PromptKit profile directory")
+		normalizeDocumentPath(resolution, state, configkit.Keys("promptkit", "profile_file"), &document.PromptKit.ProfileFile, false, "resolved PromptKit profile file")
+	}
+	if document.Output != nil {
+		normalizeDocumentPath(resolution, state, configkit.Keys("output", "directory"), &document.Output.Directory, false, "resolved output directory")
+	}
+	if document.Cache != nil {
+		if document.Cache.ChunkPlans != nil {
+			normalizeDocumentPath(resolution, state, configkit.Keys("cache", "chunk_plans", "directory"), &document.Cache.ChunkPlans.Directory, true, "resolved chunk-plan cache directory")
+		}
+		if document.Cache.Checkpoints != nil {
+			normalizeDocumentPath(resolution, state, configkit.Keys("cache", "checkpoints", "directory"), &document.Cache.Checkpoints.Directory, true, "resolved checkpoint cache directory")
+		}
+	}
+	if document.Debug != nil {
+		normalizeDocumentPath(resolution, state, configkit.Keys("debug", "directory"), &document.Debug.Directory, false, "resolved debug directory")
+	}
+	for _, id := range sortedPipelineDocumentKeys(document.Pipelines) {
+		profile := document.Pipelines[id]
+		path := configkit.Keys("pipelines").Key(id)
+		normalizeDocumentReferencePaths(profile.References, path.Key("references"), resolution, state)
+		normalizeDocumentBindingReferencePaths(&profile.Input, path.Key("input"), resolution, state)
+		if profile.Chunk != nil {
+			normalizeDocumentBindingReferencePaths(profile.Chunk, path.Key("chunk"), resolution, state)
+		}
+		if profile.Output != nil {
+			normalizeDocumentBindingReferencePaths(profile.Output, path.Key("output"), resolution, state)
+		}
+		for laneID, lane := range profile.Artifacts {
+			normalizeDocumentLaneReferencePaths(&lane, path.Key("artifacts").Key(laneID), resolution, state)
+			profile.Artifacts[laneID] = lane
+		}
+		stepResolution := resolution.inSequence(path.Key("steps"))
+		for index := range profile.Steps {
+			step := &profile.Steps[index]
+			stepPath := path.Key("steps").Index(index)
+			normalizeDocumentReferencePaths(step.References, stepPath.Key("references"), stepResolution, state)
+			for laneID, lane := range step.Artifacts {
+				normalizeDocumentLaneReferencePaths(&lane, stepPath.Key("artifacts").Key(laneID), stepResolution, state)
+				step.Artifacts[laneID] = lane
+			}
+		}
+		document.Pipelines[id] = profile
+	}
+}
+
+func initializeDocumentMainDirectory(resolution documentNormalizer, state *documentLoadState) {
+	if state == nil || state.mainDirectory != "" {
+		return
+	}
+	info, ok := resolution.InputInfo(configkit.Keys("version"))
+	if ok && len(info.Origin.Locations) == 1 {
+		if source, sourceOK := resolution.Source(info.Origin.Locations[0].SourceID); sourceOK && source.Path != "" {
+			state.mainDirectory = filepath.Dir(source.Path)
+			return
+		}
+	}
+	// Byte and reader sources are test-oriented private entry points. They have
+	// no declaring file, so preserve their established invocation-directory
+	// semantics without extending that fallback to file-backed production loads.
+	state.mainDirectory = cleanInvocationDirectory(".")
+}
+
+func normalizeDocumentBindingReferencePaths(binding *BindingDocument, path configkit.Path, resolution documentNormalizer, state *documentLoadState) {
+	if binding != nil {
+		normalizeDocumentReferencePaths(binding.References, path.Key("references"), resolution, state)
+	}
+}
+
+func normalizeDocumentLaneReferencePaths(lane *ArtifactLaneDocument, path configkit.Path, resolution documentNormalizer, state *documentLoadState) {
+	if lane == nil {
+		return
+	}
+	normalizeDocumentBindingReferencePaths(&lane.Extract, path.Key("extract"), resolution, state)
+	normalizeDocumentBindingReferencePaths(lane.Merge, path.Key("merge"), resolution, state)
+	normalizeDocumentBindingReferencePaths(lane.Normalize, path.Key("normalize"), resolution, state)
+}
+
+func normalizeDocumentReferencePaths(references map[string]ReferenceDocument, path configkit.Path, resolution documentNormalizer, state *documentLoadState) {
+	for _, name := range sortedReferenceDocumentKeys(references) {
+		source := references[name]
+		sourcePath := path.Key(name).Key("path")
+		if source.Path != nil && *source.Path != "" && !filepath.IsAbs(*source.Path) {
+			base, ok := documentPathBase(resolution, state, sourcePath)
+			if !ok {
+				resolution.Error(sourcePath, "relative path has no unambiguous source base")
+			} else {
+				value := filepath.Clean(filepath.Join(base, *source.Path))
+				if value != *source.Path {
+					resolution.Record(sourcePath, configkit.ChangeNormalization, "resolved external reference path")
+					source.Path = documentValue(value)
+				}
+			}
+			references[name] = source
+		}
+	}
+}
+
+func normalizeDocumentPath(resolution documentNormalizer, state *documentLoadState, path configkit.Path, target **string, optional bool, reason string) {
+	if target == nil || *target == nil || **target == "" || filepath.IsAbs(**target) {
+		return
+	}
+	base, ok := documentPathBase(resolution, state, path)
+	if !ok {
+		resolution.Error(path, "relative path has no unambiguous source base")
+		return
+	}
+	value := filepath.Clean(filepath.Join(base, **target))
+	if optional && **target == "" {
+		return
+	}
+	if *target != nil && **target != value {
+		configkit.Normalize(resolution.Resolution, path, target, documentValue(value), reason)
+	}
+}
+
+func documentPathBase(resolution documentNormalizer, state *documentLoadState, path configkit.Path) (string, bool) {
+	original := path
+	if state != nil {
+		original = state.originalPath(path)
+	}
+	info, ok := resolution.InputInfo(original)
+	if !ok {
+		if state != nil && state.mainDirectory != "" {
+			return state.mainDirectory, true
+		}
+		return "", false
+	}
+	switch info.Origin.Layer {
+	case configkit.LayerCLI:
+		if state != nil && state.invocationDirectory != "" {
+			return state.invocationDirectory, true
+		}
+	case configkit.LayerDefault:
+		if state != nil && state.mainDirectory != "" {
+			return state.mainDirectory, true
+		}
+	default:
+		if len(info.Origin.Locations) != 1 {
+			return "", false
+		}
+		source, ok := resolution.Source(info.Origin.Locations[0].SourceID)
+		if !ok || source.Path == "" {
+			if state != nil && state.mainDirectory != "" {
+				return state.mainDirectory, true
+			}
+			return "", false
+		}
+		return filepath.Dir(source.Path), true
+	}
+	return "", false
+}

internal/core/config/document_paths_test.go

@@ -0,0 +1,123 @@
+package config
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
+)
+
+func TestDocumentPathsUseDeclaringFilesAcrossImports(t *testing.T) {
+	t.Parallel()
+
+	root := t.TempDir()
+	main := filepath.Join(root, "notarius.yml")
+	fragmentDirectory := filepath.Join(root, "fragments")
+	if err := os.MkdirAll(fragmentDirectory, 0o700); err != nil {
+		t.Fatal(err)
+	}
+	fragment := filepath.Join(fragmentDirectory, "pipelines.yml")
+	if err := os.WriteFile(fragment, []byte(`pipelines:
+  imported:
+    input: {module: input}
+    references:
+      source: {path: ./from-fragment.txt}
+`), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(main, []byte(`version: 5
+configkit:
+  imports:
+    - path: fragments/pipelines.yml
+output: {directory: ./output}
+debug: {directory: ./debug}
+pipelines:
+  main:
+    input: {module: input}
+    references:
+      source: {path: ./from-main.txt}
+`), 0o600); err != nil {
+		t.Fatal(err)
+	}
+
+	loaded, err := loadConfig(main, LoadOptions{})
+	if err != nil {
+		t.Fatalf("loadConfig() error = %v", err)
+	}
+	if got := loaded.Config.Pipelines["main"].References["source"].Path; got != filepath.Join(root, "from-main.txt") {
+		t.Fatalf("main reference = %q", got)
+	}
+	if got := loaded.Config.Pipelines["imported"].References["source"].Path; got != filepath.Join(fragmentDirectory, "from-fragment.txt") {
+		t.Fatalf("fragment reference = %q", got)
+	}
+	if loaded.Config.Output.Directory != filepath.Join(root, "output") || loaded.Config.Debug.Directory != filepath.Join(root, "debug") {
+		t.Fatalf("destinations = %#v %#v", loaded.Config.Output, loaded.Config.Debug)
+	}
+}
+
+func TestDocumentPathsUseInvocationDirectoryForOverrides(t *testing.T) {
+	t.Parallel()
+
+	root := t.TempDir()
+	configPath := filepath.Join(root, "notarius.yml")
+	if err := os.WriteFile(configPath, []byte("version: 5\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	invocation := t.TempDir()
+	loaded, err := loadConfig(configPath, LoadOptions{
+		InvocationDirectory: invocation,
+		Overrides:           []configkit.Override{{Path: configkit.Keys("output", "directory"), Value: "generated", Flag: "--output"}},
+	})
+	if err != nil {
+		t.Fatalf("loadConfig() error = %v", err)
+	}
+	if got := loaded.Config.Output.Directory; got != filepath.Join(invocation, "generated") {
+		t.Fatalf("override directory = %q", got)
+	}
+}
+
+func TestDocumentPathsUseProfileDeclarationDirectory(t *testing.T) {
+	t.Parallel()
+
+	root := t.TempDir()
+	profiles := filepath.Join(root, "profiles")
+	if err := os.MkdirAll(profiles, 0o700); err != nil {
+		t.Fatal(err)
+	}
+	main := filepath.Join(root, "notarius.yml")
+	if err := os.WriteFile(main, []byte("version: 5\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(filepath.Join(profiles, "development.yml"), []byte("output: {directory: ./profile-output}\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	loaded, err := loadConfig(main, LoadOptions{Profile: "development"})
+	if err != nil {
+		t.Fatalf("loadConfig() error = %v", err)
+	}
+	if got := loaded.Config.Output.Directory; got != filepath.Join(profiles, "profile-output") {
+		t.Fatalf("profile directory = %q", got)
+	}
+}
+
+func TestDocumentPathsKeepEmptyCacheDirectoriesAndAbsolutePaths(t *testing.T) {
+	t.Parallel()
+
+	root := t.TempDir()
+	configPath := filepath.Join(root, "notarius.yml")
+	abs := filepath.Join(t.TempDir(), "outside.txt")
+	if err := os.WriteFile(configPath, []byte("version: 5\ncache:\n  chunk_plans: {directory: ''}\n  checkpoints: {directory: ''}\npipelines:\n  main:\n    input: {module: input}\n    references:\n      outside: {path: "+abs+"}\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	loaded, err := loadConfig(configPath, LoadOptions{})
+	if err != nil {
+		t.Fatalf("loadConfig() error = %v", err)
+	}
+	if loaded.Config.Cache.ChunkPlans.Directory != "" || loaded.Config.Cache.Checkpoints.Directory != "" {
+		t.Fatalf("cache directories = %#v", loaded.Config.Cache)
+	}
+	if got := loaded.Config.Pipelines["main"].References["outside"].Path; got != abs {
+		t.Fatalf("absolute reference = %q", got)
+	}
+}

internal/core/config/document_test.go

@@ -0,0 +1,154 @@
+package config
+
+import (
+	"errors"
+	"strings"
+	"testing"
+
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
+)
+
+func TestDocumentLoadAcceptsVersion5AndCanonicalShapes(t *testing.T) {
+	t.Parallel()
+
+	for _, source := range []string{
+		`version: 5
+pipelines:
+  session:
+    input:
+      module: seriatim
+    references:
+      party:
+        path: ./party.txt
+    steps:
+      - id: describe
+        artifacts:
+          spells:
+            extract:
+              module: dnd/spells
+              validators:
+                - module: generic/valid_json
+                  options:
+                    strict: true
+      - id: use-artifact
+        references:
+          spells:
+            artifact:
+              step: describe
+              lane: spells
+        artifacts:
+          locations:
+            extract:
+              module: dnd/locations
+`,
+		`version: "5"
+pipelines:
+  session:
+    input:
+      module: seriatim
+`,
+	} {
+		loaded, err := loadDocumentSource(configkit.Bytes("document.yml", []byte(source)), LoadOptions{})
+		if err != nil {
+			t.Fatalf("loadDocumentSource() error = %v", err)
+		}
+		profile, ok := loaded.Config.Pipelines["session"]
+		if !ok || profile.Input.Module != "seriatim" {
+			t.Fatalf("pipeline = %#v", profile)
+		}
+	}
+}
+
+func TestDocumentLoadRejectsUnsupportedVersionsBeforeDocumentDecoding(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		name   string
+		source string
+		code   configkit.Code
+	}{
+		{name: "missing", source: "unknown: value\n", code: configkit.CodeVersionError},
+		{name: "null", source: "version: null\nunknown: value\n", code: configkit.CodeVersionError},
+		{name: "version four", source: "version: 4\nunknown: value\n", code: configkit.CodeVersionError},
+		{name: "unsupported", source: "version: 6\nunknown: value\n", code: configkit.CodeVersionError},
+		{name: "noninteger", source: "version: 5.0\n", code: configkit.CodeVersionType},
+	}
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			_, err := loadDocumentSource(configkit.Bytes("document.yml", []byte(test.source)), LoadOptions{})
+			var loadError *configkit.LoadError
+			if !errors.As(err, &loadError) {
+				t.Fatalf("error = %v, want ConfigKit load error", err)
+			}
+			diagnostics := loadError.Diagnostics()
+			if len(diagnostics) != 1 || diagnostics[0].Code != test.code || diagnostics[0].Path.String() != "/version" {
+				t.Fatalf("diagnostics = %#v", diagnostics)
+			}
+			if strings.Contains(diagnostics[0].Message, "unknown") {
+				t.Fatalf("document decoding ran before version validation: %#v", diagnostics)
+			}
+		})
+	}
+}
+
+func TestDocumentLoadRejectsShorthandAndUnknownFields(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		name   string
+		source string
+		code   configkit.Code
+		path   string
+	}{
+		{
+			name:   "binding shorthand",
+			source: "version: 5\npipelines:\n  session:\n    input: seriatim\n",
+			code:   configkit.CodeDecodeType,
+			path:   "/pipelines/session/input",
+		},
+		{
+			name:   "validator nested producer field",
+			source: "version: 5\npipelines:\n  session:\n    input:\n      module: seriatim\n      validators:\n        - module: generic/valid_json\n          validators: []\n",
+			code:   configkit.CodeDecodeUnknownField,
+			path:   "/pipelines/session/input/validators/0/validators",
+		},
+		{
+			name:   "unknown field",
+			source: "version: 5\npipelines:\n  session:\n    input:\n      module: seriatim\n      unknown: true\n",
+			code:   configkit.CodeDecodeUnknownField,
+			path:   "/pipelines/session/input/unknown",
+		},
+	}
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			_, err := loadDocumentSource(configkit.Bytes("document.yml", []byte(test.source)), LoadOptions{})
+			var loadError *configkit.LoadError
+			if !errors.As(err, &loadError) {
+				t.Fatalf("error = %v, want ConfigKit load error", err)
+			}
+			diagnostics := loadError.Diagnostics()
+			if len(diagnostics) != 1 || diagnostics[0].Code != test.code || diagnostics[0].Path.String() != test.path {
+				t.Fatalf("diagnostics = %#v", diagnostics)
+			}
+		})
+	}
+}
+
+func TestDocumentDefinitionReturnsIndependentDefaults(t *testing.T) {
+	t.Parallel()
+
+	first, err := loadDocumentSource(configkit.Bytes("first.yml", []byte("version: 5\n")), LoadOptions{})
+	if err != nil {
+		t.Fatal(err)
+	}
+	first.Config.Pipelines["changed"] = PipelineDocument{}
+	first.Config.Concurrency.StageWorkers["extract"] = 3
+
+	second, err := loadDocumentSource(configkit.Bytes("second.yml", []byte("version: 5\n")), LoadOptions{})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(second.Config.Pipelines) != 0 || second.Config.Concurrency.StageWorkers["extract"] != defaultLLMConcurrency || len(second.Config.Concurrency.StageWorkers) != 1 {
+		t.Fatalf("defaults aliased across loads: %#v", second.Config)
+	}
+}

internal/core/config/effective_config.go

@@ -26,7 +26,7 @@ type EffectiveConfig struct {
 }
 
 func (c Config) Resolve(input ResolveInput) (EffectiveConfig, error) {
-	c.Concurrency.recomputeStageWorkerDefaults()
+	c = c.withDerivedStageWorkers()
 	if err := c.Validate(); err != nil {
 		return EffectiveConfig{}, err
 	}

internal/core/config/env.go

@@ -1,103 +0,0 @@
-package config
-
-import (
-	"fmt"
-	"os"
-	"strconv"
-	"strings"
-
-	"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
-)
-
-func LoadFromEnv() (Config, error) {
-	cfg := Default()
-	if err := cfg.applyEnvOverridesWithLookup(os.LookupEnv); err != nil {
-		return Config{}, err
-	}
-	return cfg, nil
-}
-
-func (c *Config) ApplyEnvOverrides() error {
-	return c.applyEnvOverridesWithLookup(os.LookupEnv)
-}
-
-func (c *Config) ApplyEnvOverridesWithLookup(lookup func(string) (string, bool)) error {
-	return c.applyEnvOverridesWithLookup(lookup)
-}
-
-func (c *Config) applyEnvOverridesWithLookup(lookup func(string) (string, bool)) error {
-	if c == nil {
-		return fmt.Errorf("config must not be nil")
-	}
-	if raw, ok := lookup("NOTARIUS_TOTAL_LLM_CONCURRENCY"); ok {
-		value, err := parseIntEnv("NOTARIUS_TOTAL_LLM_CONCURRENCY", raw)
-		if err != nil {
-			return err
-		}
-		c.Concurrency.TotalLLM = value
-	}
-	if raw, ok := lookup("NOTARIUS_STAGE_WORKERS_EXTRACT"); ok {
-		value, err := parseIntEnv("NOTARIUS_STAGE_WORKERS_EXTRACT", raw)
-		if err != nil {
-			return err
-		}
-		if c.Concurrency.StageWorkers == nil {
-			c.Concurrency.StageWorkers = make(map[string]int)
-		}
-		c.Concurrency.StageWorkers["extract"] = value
-		c.Concurrency.extractWorkersConfigured = true
-	}
-	c.Concurrency.recomputeStageWorkerDefaults()
-	if raw, ok := lookup("NOTARIUS_OUTPUT_DIR"); ok {
-		c.Output.Directory = strings.TrimSpace(raw)
-		if c.Output.Directory == "" {
-			return fmt.Errorf("NOTARIUS_OUTPUT_DIR: must not be empty")
-		}
-		if strings.ContainsRune(c.Output.Directory, '\x00') {
-			return fmt.Errorf("NOTARIUS_OUTPUT_DIR: must not contain NUL")
-		}
-	}
-	if raw, ok := lookup("NOTARIUS_CACHE_CHUNK_PLANS_MODE"); ok {
-		mode, err := pipeline.ParseChunkCacheMode(raw)
-		if err != nil {
-			return fmt.Errorf("NOTARIUS_CACHE_CHUNK_PLANS_MODE: %w", err)
-		}
-		c.Cache.ChunkPlans.Mode = mode
-	}
-	if raw, ok := lookup("NOTARIUS_CACHE_CHUNK_PLANS_DIR"); ok {
-		c.Cache.ChunkPlans.Directory = cleanOptionalPath(raw)
-		if c.Cache.ChunkPlans.Directory == "" {
-			return fmt.Errorf("NOTARIUS_CACHE_CHUNK_PLANS_DIR: must not be empty")
-		}
-		if strings.ContainsRune(c.Cache.ChunkPlans.Directory, '\x00') {
-			return fmt.Errorf("NOTARIUS_CACHE_CHUNK_PLANS_DIR: must not contain NUL")
-		}
-	}
-	if raw, ok := lookup("NOTARIUS_CACHE_CHECKPOINTS_DIR"); ok {
-		c.Cache.Checkpoints.Directory = cleanOptionalPath(raw)
-		if c.Cache.Checkpoints.Directory == "" {
-			return fmt.Errorf("NOTARIUS_CACHE_CHECKPOINTS_DIR: must not be empty")
-		}
-		if strings.ContainsRune(c.Cache.Checkpoints.Directory, '\x00') {
-			return fmt.Errorf("NOTARIUS_CACHE_CHECKPOINTS_DIR: must not contain NUL")
-		}
-	}
-	if raw, ok := lookup("NOTARIUS_DEBUG_DIR"); ok {
-		c.Debug.Directory = strings.TrimSpace(raw)
-		if c.Debug.Directory == "" {
-			return fmt.Errorf("NOTARIUS_DEBUG_DIR: must not be empty")
-		}
-		if strings.ContainsRune(c.Debug.Directory, '\x00') {
-			return fmt.Errorf("NOTARIUS_DEBUG_DIR: must not contain NUL")
-		}
-	}
-	return nil
-}
-
-func parseIntEnv(name string, raw string) (int, error) {
-	value, err := strconv.Atoi(strings.TrimSpace(raw))
-	if err != nil {
-		return 0, fmt.Errorf("%s: must be an integer", name)
-	}
-	return value, nil
-}

internal/core/config/env_contract_test.go

@@ -1,239 +0,0 @@
-package config
-
-import (
-	"errors"
-	"reflect"
-	"strings"
-	"testing"
-
-	"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
-)
-
-func TestPrecedenceFileValuesOverrideBuiltInDefaults(t *testing.T) {
-	cfg := applyFileConfig(t, `version: 4
-concurrency:
-  total_llm: 4
-  stage_workers:
-    extract: 2
-output:
-  directory: ./file-output
-cache:
-  chunk_plans:
-    directory: ./file-plans
-    mode: refresh
-  checkpoints:
-    directory: ./file-checkpoints
-debug:
-  directory: ./file-debug
-`)
-	if cfg.Concurrency.TotalLLM != 4 || cfg.Concurrency.StageWorkers["extract"] != 2 ||
-		cfg.Output.Directory != "./file-output" || cfg.Cache.ChunkPlans.Directory != "file-plans" ||
-		cfg.Cache.ChunkPlans.Mode != pipeline.ChunkCacheRefresh || cfg.Cache.Checkpoints.Directory != "file-checkpoints" ||
-		cfg.Debug.Directory != "./file-debug" {
-		t.Fatalf("file values did not override defaults: %#v", cfg)
-	}
-}
-
-func TestPrecedenceOperationalEnvironmentOverridesFileValues(t *testing.T) {
-	cfg := applyFileConfig(t, `version: 4
-concurrency:
-  total_llm: 2
-  stage_workers:
-    extract: 1
-output:
-  directory: ./file-output
-cache:
-  chunk_plans:
-    directory: ./file-plans
-    mode: refresh
-  checkpoints:
-    directory: ./file-checkpoints
-debug:
-  directory: ./file-debug
-`)
-	env := map[string]string{
-		"NOTARIUS_TOTAL_LLM_CONCURRENCY":  "8",
-		"NOTARIUS_STAGE_WORKERS_EXTRACT":  "6",
-		"NOTARIUS_OUTPUT_DIR":             "/env/output",
-		"NOTARIUS_CACHE_CHUNK_PLANS_MODE": "bypass",
-		"NOTARIUS_CACHE_CHUNK_PLANS_DIR":  "/env/plans",
-		"NOTARIUS_CACHE_CHECKPOINTS_DIR":  "/env/checkpoints",
-		"NOTARIUS_DEBUG_DIR":              "/env/debug",
-	}
-	if err := cfg.ApplyEnvOverridesWithLookup(lookupValues(env)); err != nil {
-		t.Fatal(err)
-	}
-	if cfg.Concurrency.TotalLLM != 8 || cfg.Concurrency.StageWorkers["extract"] != 6 ||
-		cfg.Output.Directory != "/env/output" || cfg.Cache.ChunkPlans.Directory != "/env/plans" ||
-		cfg.Cache.ChunkPlans.Mode != pipeline.ChunkCacheBypass || cfg.Cache.Checkpoints.Directory != "/env/checkpoints" ||
-		cfg.Debug.Directory != "/env/debug" {
-		t.Fatalf("environment values did not override file values: %#v", cfg)
-	}
-}
-
-func TestPrecedenceExtractWorkersFollowEffectiveConcurrencyUnlessExplicit(t *testing.T) {
-	tests := []struct {
-		name       string
-		file       string
-		env        map[string]string
-		wantTotal  int
-		wantWorker int
-	}{
-		{
-			name:       "default follows environment total",
-			file:       "version: 4\n",
-			env:        map[string]string{"NOTARIUS_TOTAL_LLM_CONCURRENCY": "5"},
-			wantTotal:  5,
-			wantWorker: 5,
-		},
-		{
-			name:       "file worker is retained",
-			file:       "version: 4\nconcurrency:\n  total_llm: 3\n  stage_workers:\n    extract: 2\n",
-			env:        map[string]string{"NOTARIUS_TOTAL_LLM_CONCURRENCY": "6"},
-			wantTotal:  6,
-			wantWorker: 2,
-		},
-		{
-			name: "environment worker is retained",
-			file: "version: 4\nconcurrency:\n  total_llm: 2\n",
-			env: map[string]string{
-				"NOTARIUS_TOTAL_LLM_CONCURRENCY": "6",
-				"NOTARIUS_STAGE_WORKERS_EXTRACT": "4",
-			},
-			wantTotal:  6,
-			wantWorker: 4,
-		},
-	}
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			cfg := applyFileConfig(t, tt.file)
-			if err := cfg.ApplyEnvOverridesWithLookup(lookupValues(tt.env)); err != nil {
-				t.Fatal(err)
-			}
-			if cfg.Concurrency.TotalLLM != tt.wantTotal || cfg.Concurrency.StageWorkers["extract"] != tt.wantWorker {
-				t.Fatalf("concurrency = %#v, want total %d and extract %d", cfg.Concurrency, tt.wantTotal, tt.wantWorker)
-			}
-		})
-	}
-}
-
-func TestPrecedenceEmptyFileCacheDirectoriesDeferPerUserResolution(t *testing.T) {
-	cfg := applyFileConfig(t, `version: 4
-cache:
-  chunk_plans:
-    directory: ""
-  checkpoints:
-    directory: ""
-`)
-	if err := cfg.Validate(); err != nil {
-		t.Fatalf("empty file cache directories should be valid: %v", err)
-	}
-	if cfg.Cache.ChunkPlans.Directory != "" || cfg.Cache.Checkpoints.Directory != "" {
-		t.Fatalf("empty cache directories were not preserved for deferred resolution: %#v", cfg.Cache)
-	}
-	resolver := func() (string, error) { return "/user/cache", nil }
-	chunkPlans, err := DefaultChunkPlanRoot(resolver)
-	if err != nil {
-		t.Fatal(err)
-	}
-	checkpoints, err := DefaultCheckpointRoot(resolver)
-	if err != nil {
-		t.Fatal(err)
-	}
-	if chunkPlans != "/user/cache/notarius/chunk-plans" || checkpoints != "/user/cache/notarius/checkpoints" {
-		t.Fatalf("deferred cache roots = %q, %q", chunkPlans, checkpoints)
-	}
-}
-
-func TestDefaultCacheRootsRejectInvalidUserCacheResolvers(t *testing.T) {
-	tests := []struct {
-		name     string
-		resolver func() (string, error)
-		want     string
-	}{
-		{name: "nil resolver", want: "must not be nil"},
-		{
-			name: "resolver failure",
-			resolver: func() (string, error) {
-				return "", errors.New("cache home unavailable")
-			},
-			want: "resolve user cache directory",
-		},
-		{name: "empty directory", resolver: func() (string, error) { return "  ", nil }, want: "must not be empty"},
-	}
-	families := []struct {
-		name string
-		root func(func() (string, error)) (string, error)
-	}{
-		{name: "chunk plans", root: DefaultChunkPlanRoot},
-		{name: "checkpoints", root: DefaultCheckpointRoot},
-	}
-
-	for _, family := range families {
-		for _, tt := range tests {
-			t.Run(family.name+"/"+tt.name, func(t *testing.T) {
-				_, err := family.root(tt.resolver)
-				if err == nil || !strings.Contains(err.Error(), tt.want) {
-					t.Fatalf("error = %v, want substring %q", err, tt.want)
-				}
-			})
-		}
-	}
-}
-
-func TestEnvEmptyDirectoryOverridesAreErrors(t *testing.T) {
-	tests := []string{
-		"NOTARIUS_OUTPUT_DIR",
-		"NOTARIUS_CACHE_CHUNK_PLANS_DIR",
-		"NOTARIUS_CACHE_CHECKPOINTS_DIR",
-		"NOTARIUS_DEBUG_DIR",
-	}
-	for _, name := range tests {
-		t.Run(name, func(t *testing.T) {
-			cfg := Default()
-			err := cfg.ApplyEnvOverridesWithLookup(lookupValues(map[string]string{name: " \t"}))
-			if err == nil || !strings.Contains(err.Error(), name) {
-				t.Fatalf("error = %v, want responsible environment variable", err)
-			}
-		})
-	}
-}
-
-func TestEnvInvalidIntegersAndChunkCacheModesReportTheirNames(t *testing.T) {
-	tests := map[string]string{
-		"NOTARIUS_TOTAL_LLM_CONCURRENCY":  "not-an-integer",
-		"NOTARIUS_STAGE_WORKERS_EXTRACT":  "not-an-integer",
-		"NOTARIUS_CACHE_CHUNK_PLANS_MODE": "not-a-cache-mode",
-	}
-	for name, value := range tests {
-		t.Run(name, func(t *testing.T) {
-			cfg := Default()
-			err := cfg.ApplyEnvOverridesWithLookup(lookupValues(map[string]string{name: value}))
-			if err == nil || !strings.Contains(err.Error(), name) {
-				t.Fatalf("error = %v, want responsible environment variable", err)
-			}
-		})
-	}
-}
-
-func TestEnvRemovedProviderVariablesAreIgnored(t *testing.T) {
-	before := Default()
-	cfg := Default()
-	removed := map[string]string{
-		"NOTARIUS_LLM_DEFAULT_ENDPOINT": "ignored-provider-setting",
-		"NOTARIUS_LLM_DEFAULT_MODEL":    "ignored-provider-setting",
-	}
-	if err := cfg.ApplyEnvOverridesWithLookup(lookupValues(removed)); err != nil {
-		t.Fatal(err)
-	}
-	if !reflect.DeepEqual(cfg, before) {
-		t.Fatalf("removed provider variables changed configuration: %#v", cfg)
-	}
-}
-
-func lookupValues(values map[string]string) func(string) (string, bool) {
-	return func(name string) (string, bool) {
-		value, ok := values[name]
-		return value, ok
-	}
-}

internal/core/config/file_config.go

@@ -1,902 +0,0 @@
-package config
-
-import (
-	"bytes"
-	"fmt"
-	"io"
-	"os"
-	"path/filepath"
-	"sort"
-	"strings"
-
-	"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
-	"gopkg.in/yaml.v3"
-)
-
-type FileConfig struct {
-	Version     int                            `yaml:"version"`
-	PromptKit   *FilePromptKitConfig           `yaml:"promptkit,omitempty"`
-	Pipelines   map[string]FilePipelineProfile `yaml:"pipelines,omitempty"`
-	Concurrency *FileConcurrencyConfig         `yaml:"concurrency,omitempty"`
-	Output      *FileOutputConfig              `yaml:"output,omitempty"`
-	Cache       *FileCacheConfig               `yaml:"cache,omitempty"`
-	Debug       *FileDebugConfig               `yaml:"debug,omitempty"`
-}
-
-type FilePromptKitConfig struct {
-	ProfileDir   *string                          `yaml:"profile_dir,omitempty"`
-	ProfileFile  *string                          `yaml:"profile_file,omitempty"`
-	LocalBackend *FilePromptKitLocalBackendConfig `yaml:"local_backend,omitempty"`
-}
-
-type FilePromptKitLocalBackendConfig struct {
-	Endpoint         *string `yaml:"endpoint,omitempty"`
-	ConcurrencyLimit *int    `yaml:"concurrency_limit,omitempty"`
-}
-
-type FilePipelineProfile struct {
-	LLMProfile                     *string                            `yaml:"llm_profile,omitempty"`
-	StructuredOutputRepairAttempts *int                               `yaml:"structured_output_repair_attempts,omitempty"`
-	ValidationPolicy               *pipeline.ValidationPolicyOverride `yaml:"validation_policy,omitempty"`
-	Input                          fileModuleBinding                  `yaml:"input"`
-	Chunk                          *fileModuleBinding                 `yaml:"chunk,omitempty"`
-	Artifacts                      map[string]FileArtifactLaneProfile `yaml:"artifacts,omitempty"`
-	Steps                          []FilePipelineStepProfile          `yaml:"steps,omitempty"`
-	Output                         *fileModuleBinding                 `yaml:"output,omitempty"`
-	References                     map[string]fileReferenceSource     `yaml:"references,omitempty"`
-	artifactsSet                   bool                               `yaml:"-"`
-	stepsSet                       bool                               `yaml:"-"`
-	llmProfileSet                  bool                               `yaml:"-"`
-}
-
-func (p *FilePipelineProfile) UnmarshalYAML(node *yaml.Node) error {
-	if err := validateStructuredOutputRepairAttemptsNode(node, "pipeline profile"); err != nil {
-		return err
-	}
-	type plainFilePipelineProfile FilePipelineProfile
-	var decoded plainFilePipelineProfile
-	seen, err := decodeKnownMapping(node, &decoded, map[string]struct{}{
-		"llm_profile": {}, "structured_output_repair_attempts": {}, "validation_policy": {}, "input": {}, "chunk": {}, "artifacts": {}, "steps": {}, "output": {}, "references": {},
-	}, "pipeline profile")
-	if err != nil {
-		return err
-	}
-	*p = FilePipelineProfile(decoded)
-	if validationPolicyNode, ok := mappingValue(node, "validation_policy"); ok {
-		policy, err := parseValidationPolicy(validationPolicyNode, "pipeline profile")
-		if err != nil {
-			return err
-		}
-		p.ValidationPolicy = policy
-	}
-	_, p.artifactsSet = seen["artifacts"]
-	_, p.stepsSet = seen["steps"]
-	_, p.llmProfileSet = seen["llm_profile"]
-	return nil
-}
-
-func (s *FilePipelineStepProfile) UnmarshalYAML(node *yaml.Node) error {
-	type plainFilePipelineStepProfile FilePipelineStepProfile
-	var decoded plainFilePipelineStepProfile
-	if _, err := decodeKnownMapping(node, &decoded, map[string]struct{}{
-		"id": {}, "artifacts": {}, "references": {},
-	}, "pipeline step"); err != nil {
-		return err
-	}
-	*s = FilePipelineStepProfile(decoded)
-	return nil
-}
-
-func (l *FileArtifactLaneProfile) UnmarshalYAML(node *yaml.Node) error {
-	type plainFileArtifactLaneProfile FileArtifactLaneProfile
-	var decoded plainFileArtifactLaneProfile
-	if _, err := decodeKnownMapping(node, &decoded, map[string]struct{}{
-		"extract": {}, "merge": {}, "normalize": {}, "validators": {}, "references": {},
-	}, "artifact lane"); err != nil {
-		return err
-	}
-	*l = FileArtifactLaneProfile(decoded)
-	return nil
-}
-
-func decodeKnownMapping(node *yaml.Node, target any, allowed map[string]struct{}, context string) (map[string]struct{}, error) {
-	if node.Kind != yaml.MappingNode {
-		return nil, fmt.Errorf("%s must be an object", context)
-	}
-	if err := node.Decode(target); err != nil {
-		return nil, err
-	}
-	seen := make(map[string]struct{}, len(node.Content)/2)
-	for i := 0; i < len(node.Content); i += 2 {
-		key := node.Content[i].Value
-		if _, exists := seen[key]; exists {
-			return nil, fmt.Errorf("%s field %q is duplicated", context, key)
-		}
-		if _, ok := allowed[key]; !ok {
-			return nil, fmt.Errorf("field %s not found in %s", key, context)
-		}
-		seen[key] = struct{}{}
-	}
-	return seen, nil
-}
-
-type FilePipelineStepProfile struct {
-	ID         string                             `yaml:"id"`
-	Artifacts  map[string]FileArtifactLaneProfile `yaml:"artifacts"`
-	References map[string]fileReferenceSource     `yaml:"references,omitempty"`
-}
-
-type FileArtifactLaneProfile struct {
-	Extract    fileModuleBinding              `yaml:"extract"`
-	Merge      *fileModuleBinding             `yaml:"merge,omitempty"`
-	Normalize  *fileModuleBinding             `yaml:"normalize,omitempty"`
-	Validators []fileModuleBinding            `yaml:"validators,omitempty"`
-	References map[string]fileReferenceSource `yaml:"references,omitempty"`
-}
-
-type FileConcurrencyConfig struct {
-	TotalLLM     *int           `yaml:"total_llm,omitempty"`
-	StageWorkers map[string]int `yaml:"stage_workers,omitempty"`
-}
-
-type FileOutputConfig struct {
-	Directory *string `yaml:"directory,omitempty"`
-}
-type FileCacheConfig struct {
-	ChunkPlans  *FileChunkPlanCacheConfig  `yaml:"chunk_plans,omitempty"`
-	Checkpoints *FileCheckpointCacheConfig `yaml:"checkpoints,omitempty"`
-}
-type FileChunkPlanCacheConfig struct {
-	Directory *string `yaml:"directory,omitempty"`
-	Mode      *string `yaml:"mode,omitempty"`
-}
-type FileCheckpointCacheConfig struct {
-	Enabled   *bool   `yaml:"enabled,omitempty"`
-	Directory *string `yaml:"directory,omitempty"`
-}
-type FileDebugConfig struct {
-	Directory *string `yaml:"directory,omitempty"`
-}
-
-type fileModuleBinding struct {
-	Module                         string
-	LLMProfile                     string
-	StructuredOutputRepairAttempts *int
-	ValidationPolicy               *pipeline.ValidationPolicyOverride
-	Retries                        int
-	Options                        map[string]any
-	References                     map[string]fileReferenceSource
-	Validators                     pipeline.ValidatorOverride
-}
-
-type fileReferenceSource struct {
-	path     string
-	artifact *pipeline.ArtifactReference
-}
-
-func (source *fileReferenceSource) UnmarshalYAML(node *yaml.Node) error {
-	if source == nil {
-		return fmt.Errorf("reference source must not be nil")
-	}
-	switch node.Kind {
-	case yaml.ScalarNode:
-		if node.Tag != "!!str" {
-			return fmt.Errorf("external reference path must be a string")
-		}
-		path := strings.TrimSpace(node.Value)
-		if path == "" {
-			return fmt.Errorf("external reference path must not be empty")
-		}
-		source.path = path
-		source.artifact = nil
-		return nil
-	case yaml.MappingNode:
-		if len(node.Content) != 2 || node.Content[0].Value != "artifact" {
-			return fmt.Errorf("reference source mapping must contain only artifact")
-		}
-		artifactNode := node.Content[1]
-		if artifactNode.Kind != yaml.MappingNode {
-			return fmt.Errorf("artifact reference must be an object")
-		}
-		var step, lane string
-		seen := map[string]bool{}
-		for i := 0; i < len(artifactNode.Content); i += 2 {
-			key := artifactNode.Content[i].Value
-			value := artifactNode.Content[i+1]
-			if seen[key] {
-				return fmt.Errorf("artifact reference field %q is duplicated", key)
-			}
-			seen[key] = true
-			if value.Tag != "!!str" {
-				return fmt.Errorf("artifact reference field %q must be a string", key)
-			}
-			switch key {
-			case "step":
-				step = strings.TrimSpace(value.Value)
-			case "lane":
-				lane = strings.TrimSpace(value.Value)
-			default:
-				return fmt.Errorf("field %s not found in artifact reference", key)
-			}
-		}
-		if step == "" || lane == "" {
-			return fmt.Errorf("artifact reference step and lane must not be empty")
-		}
-		source.path = ""
-		source.artifact = &pipeline.ArtifactReference{Step: step, Lane: lane}
-		return nil
-	default:
-		return fmt.Errorf("reference source must be a string or object")
-	}
-}
-
-func (source fileReferenceSource) toPipelineSource() pipeline.ReferenceSource {
-	if source.artifact != nil {
-		artifact := *source.artifact
-		return pipeline.ReferenceSource{Artifact: &artifact}
-	}
-	return pipeline.ExternalReference(source.path)
-}
-
-func fileReferenceSourcesToPipeline(values map[string]fileReferenceSource) map[string]pipeline.ReferenceSource {
-	if len(values) == 0 {
-		return nil
-	}
-	out := make(map[string]pipeline.ReferenceSource, len(values))
-	for key, value := range values {
-		out[strings.TrimSpace(key)] = value.toPipelineSource()
-	}
-	return out
-}
-
-func (b *fileModuleBinding) UnmarshalYAML(node *yaml.Node) error {
-	switch node.Kind {
-	case yaml.ScalarNode:
-		var module string
-		if err := node.Decode(&module); err != nil {
-			return fmt.Errorf("module binding must be a string or object")
-		}
-		b.Module = strings.TrimSpace(module)
-		return nil
-	case yaml.MappingNode:
-		seen := make(map[string]struct{}, len(node.Content)/2)
-		for i := 0; i < len(node.Content); i += 2 {
-			keyNode := node.Content[i]
-			valueNode := node.Content[i+1]
-			if _, exists := seen[keyNode.Value]; exists {
-				return fmt.Errorf("module binding field %q is duplicated", keyNode.Value)
-			}
-			seen[keyNode.Value] = struct{}{}
-			switch keyNode.Value {
-			case "module":
-				var module string
-				if err := valueNode.Decode(&module); err != nil {
-					return err
-				}
-				b.Module = strings.TrimSpace(module)
-			case "llm_profile":
-				var llmProfile string
-				if err := valueNode.Decode(&llmProfile); err != nil {
-					return err
-				}
-				b.LLMProfile = strings.TrimSpace(llmProfile)
-				if b.LLMProfile == "" {
-					return fmt.Errorf("llm_profile must not be empty when set")
-				}
-			case "structured_output_repair_attempts":
-				attempts, err := parseStructuredOutputRepairAttempts(valueNode, "module binding")
-				if err != nil {
-					return err
-				}
-				b.StructuredOutputRepairAttempts = attempts
-			case "validation_policy":
-				policy, err := parseValidationPolicy(valueNode, "module binding")
-				if err != nil {
-					return err
-				}
-				b.ValidationPolicy = policy
-			case "retries":
-				var retries int
-				if err := valueNode.Decode(&retries); err != nil {
-					return err
-				}
-				b.Retries = retries
-			case "options":
-				var options map[string]any
-				if err := valueNode.Decode(&options); err != nil {
-					return err
-				}
-				b.Options = normalizeOptions(options)
-			case "references":
-				var references map[string]fileReferenceSource
-				if err := valueNode.Decode(&references); err != nil {
-					return err
-				}
-				b.References = references
-			case "validators":
-				b.Validators.Set = true
-				var validators []fileModuleBinding
-				if err := valueNode.Decode(&validators); err != nil {
-					return err
-				}
-				b.Validators.Validators = make([]pipeline.ModuleBinding, len(validators))
-				for i, validator := range validators {
-					b.Validators.Validators[i] = validator.toPipelineBinding()
-				}
-			default:
-				return fmt.Errorf("field %s not found in module binding", keyNode.Value)
-			}
-		}
-		return nil
-	default:
-		return fmt.Errorf("module binding must be a string or object")
-	}
-}
-
-func (b fileModuleBinding) toPipelineBinding() pipeline.ModuleBinding {
-	return pipeline.ModuleBinding{
-		Module:                         strings.TrimSpace(b.Module),
-		LLMProfile:                     strings.TrimSpace(b.LLMProfile),
-		StructuredOutputRepairAttempts: cloneStructuredOutputRepairAttempts(b.StructuredOutputRepairAttempts),
-		ValidationPolicy:               cloneValidationPolicyOverride(b.ValidationPolicy),
-		Retries:                        b.Retries,
-		Options:                        cloneOptions(b.Options),
-		References:                     fileReferenceSourcesToPipeline(b.References),
-		Validators:                     cloneValidatorOverride(b.Validators),
-	}
-}
-
-func mappingValue(node *yaml.Node, key string) (*yaml.Node, bool) {
-	for i := 0; i < len(node.Content); i += 2 {
-		if node.Content[i].Value == key {
-			return node.Content[i+1], true
-		}
-	}
-	return nil, false
-}
-
-func parseValidationPolicy(node *yaml.Node, context string) (*pipeline.ValidationPolicyOverride, error) {
-	if node == nil || node.Tag == "!!null" || node.Kind != yaml.MappingNode {
-		return nil, fmt.Errorf("%s validation_policy must be an object", context)
-	}
-	policy := &pipeline.ValidationPolicyOverride{}
-	seen := make(map[string]struct{}, len(node.Content)/2)
-	for i := 0; i < len(node.Content); i += 2 {
-		key := node.Content[i].Value
-		value := node.Content[i+1]
-		if _, exists := seen[key]; exists {
-			return nil, fmt.Errorf("%s validation_policy field %q is duplicated", context, key)
-		}
-		seen[key] = struct{}{}
-		if value.Tag == "!!null" || value.Kind != yaml.ScalarNode || value.Tag != "!!str" {
-			return nil, fmt.Errorf("%s validation_policy.%s must be a string", context, key)
-		}
-		switch key {
-		case "producer_structural_failure":
-			value := pipeline.ProducerStructuralFailureAction(value.Value)
-			policy.ProducerStructuralFailure = &value
-		case "semantic_rejection":
-			value := pipeline.SemanticRejectionAction(value.Value)
-			policy.SemanticRejection = &value
-		case "validator_failure":
-			value := pipeline.ValidatorFailureAction(value.Value)
-			policy.ValidatorFailure = &value
-		default:
-			return nil, fmt.Errorf("field %s not found in %s validation_policy", key, context)
-		}
-	}
-	if err := policy.Validate(); err != nil {
-		return nil, fmt.Errorf("%s validation_policy: %w", context, err)
-	}
-	return policy, nil
-}
-
-func validateStructuredOutputRepairAttemptsNode(node *yaml.Node, context string) error {
-	if node.Kind != yaml.MappingNode {
-		return fmt.Errorf("%s must be an object", context)
-	}
-	for i := 0; i < len(node.Content); i += 2 {
-		if node.Content[i].Value != "structured_output_repair_attempts" {
-			continue
-		}
-		if _, err := parseStructuredOutputRepairAttempts(node.Content[i+1], context); err != nil {
-			return err
-		}
-	}
-	return nil
-}
-
-func parseStructuredOutputRepairAttempts(node *yaml.Node, context string) (*int, error) {
-	if node.Tag == "!!null" {
-		return nil, fmt.Errorf("%s structured_output_repair_attempts must not be null", context)
-	}
-	if node.Kind != yaml.ScalarNode || node.Tag != "!!int" {
-		return nil, fmt.Errorf("%s structured_output_repair_attempts must be an integer", context)
-	}
-	var attempts int
-	if err := node.Decode(&attempts); err != nil {
-		return nil, fmt.Errorf("%s structured_output_repair_attempts must be an integer: %w", context, err)
-	}
-	if attempts < 0 || attempts > 3 {
-		return nil, fmt.Errorf("%s structured_output_repair_attempts must be between zero and three", context)
-	}
-	return &attempts, nil
-}
-
-func cloneStructuredOutputRepairAttempts(attempts *int) *int {
-	if attempts == nil {
-		return nil
-	}
-	value := *attempts
-	return &value
-}
-
-func LoadFileConfig(path string) (FileConfig, error) {
-	data, err := os.ReadFile(path)
-	if err != nil {
-		return FileConfig{}, fmt.Errorf("read config file %q: %w", path, err)
-	}
-	cfg, err := ParseFileConfigYAML(data)
-	if err != nil {
-		return FileConfig{}, fmt.Errorf("parse config file %q: %w", path, err)
-	}
-	return cfg, nil
-}
-
-func ParseFileConfigYAML(data []byte) (FileConfig, error) {
-	var header struct {
-		Version int `yaml:"version"`
-	}
-	if err := yaml.Unmarshal(data, &header); err != nil {
-		return FileConfig{}, fmt.Errorf("decode yaml version header: %w", err)
-	}
-	if header.Version == 0 {
-		return FileConfig{}, fmt.Errorf("config version is required")
-	}
-	if header.Version == 2 {
-		return FileConfig{}, fmt.Errorf("config version 2 is no longer supported; migrate the file using the version 2-to-3 migration in docs/config.md")
-	}
-	if header.Version == 3 {
-		return FileConfig{}, fmt.Errorf("config version 3 is no longer supported; change \"version: 3\" to \"version: 4\" and rename \"scriptorium:\" to \"promptkit:\"")
-	}
-	if header.Version != SupportedFileConfigVersion {
-		return FileConfig{}, fmt.Errorf("unsupported config version %d (supported version is %d)", header.Version, SupportedFileConfigVersion)
-	}
-	var fileCfg FileConfig
-	decoder := yaml.NewDecoder(bytes.NewReader(data))
-	decoder.KnownFields(true)
-	if err := decoder.Decode(&fileCfg); err != nil {
-		return FileConfig{}, fmt.Errorf("decode yaml: %w", err)
-	}
-	var trailing any
-	if err := decoder.Decode(&trailing); err == nil {
-		return FileConfig{}, fmt.Errorf("config must contain exactly one YAML document")
-	} else if err != io.EOF {
-		return FileConfig{}, fmt.Errorf("decode trailing yaml document: %w", err)
-	}
-	return fileCfg, nil
-}
-
-func (c *Config) ApplyFileConfig(fileCfg FileConfig) error {
-	return c.applyFileConfigWithLookup(fileCfg, os.LookupEnv)
-}
-
-func (c *Config) ApplyFileConfigWithLookup(fileCfg FileConfig, lookup func(string) (string, bool)) error {
-	return c.applyFileConfigWithLookup(fileCfg, lookup)
-}
-
-func (c *Config) applyFileConfigWithLookup(fileCfg FileConfig, lookup func(string) (string, bool)) error {
-	_ = lookup
-	if c == nil {
-		return fmt.Errorf("config must not be nil")
-	}
-	if fileCfg.Version != SupportedFileConfigVersion {
-		return fmt.Errorf("unsupported config version %d", fileCfg.Version)
-	}
-	if c.Pipelines == nil {
-		c.Pipelines = map[string]pipeline.PipelineProfile{}
-	}
-
-	pipelineIDs, rawPipelineIDs, err := normalizedMapKeys(fileCfg.Pipelines, "pipeline id")
-	if err != nil {
-		return err
-	}
-	for _, pipelineID := range pipelineIDs {
-		filePipeline := fileCfg.Pipelines[rawPipelineIDs[pipelineID]]
-		hasArtifacts := filePipeline.artifactsSet || filePipeline.Artifacts != nil
-		hasSteps := filePipeline.stepsSet || filePipeline.Steps != nil
-		if hasArtifacts && hasSteps {
-			return fmt.Errorf("pipeline %q must not declare both artifacts and steps", pipelineID)
-		}
-		if hasSteps && len(filePipeline.Steps) == 0 {
-			return fmt.Errorf("pipeline %q must declare at least one ordered step", pipelineID)
-		}
-		if hasSteps {
-			seenSteps := make(map[string]struct{}, len(filePipeline.Steps))
-			seenLanes := make(map[string]struct{})
-			for index, step := range filePipeline.Steps {
-				stepID := strings.TrimSpace(step.ID)
-				if stepID == "" {
-					return fmt.Errorf("pipeline %q step[%d] id must not be empty", pipelineID, index)
-				}
-				if _, ok := seenSteps[stepID]; ok {
-					return fmt.Errorf("pipeline %q step id %q is duplicated after trimming", pipelineID, stepID)
-				}
-				seenSteps[stepID] = struct{}{}
-				laneIDs, rawLaneIDs, err := normalizedMapKeys(step.Artifacts, fmt.Sprintf("pipeline %q step %q artifact lane id", pipelineID, stepID))
-				if err != nil {
-					return err
-				}
-				for _, laneID := range laneIDs {
-					if _, ok := seenLanes[laneID]; ok {
-						return fmt.Errorf("pipeline %q artifact lane id %q is duplicated across steps", pipelineID, laneID)
-					}
-					seenLanes[laneID] = struct{}{}
-					fileLane := step.Artifacts[rawLaneIDs[laneID]]
-					if err := validateFileLaneReferences(pipelineID, stepID, laneID, fileLane); err != nil {
-						return err
-					}
-				}
-				if err := validateFileReferenceSources(step.References, fmt.Sprintf("pipeline %q step %q reference slot", pipelineID, stepID)); err != nil {
-					return err
-				}
-			}
-		}
-		if _, _, err := normalizedMapKeys(filePipeline.Artifacts, fmt.Sprintf("pipeline %q artifact lane id", pipelineID)); err != nil {
-			return err
-		}
-		if err := validateFileReferenceSources(filePipeline.References, fmt.Sprintf("pipeline %q reference slot", pipelineID)); err != nil {
-			return err
-		}
-		if filePipeline.Chunk != nil {
-			if _, _, err := normalizedMapKeys(filePipeline.Chunk.References, fmt.Sprintf("pipeline %q chunk reference slot", pipelineID)); err != nil {
-				return err
-			}
-		}
-		if _, _, err := normalizedMapKeys(filePipeline.Input.References, fmt.Sprintf("pipeline %q input reference slot", pipelineID)); err != nil {
-			return err
-		}
-		if filePipeline.Output != nil {
-			if _, _, err := normalizedMapKeys(filePipeline.Output.References, fmt.Sprintf("pipeline %q output reference slot", pipelineID)); err != nil {
-				return err
-			}
-		}
-		for rawLaneID, fileLane := range filePipeline.Artifacts {
-			laneID := strings.TrimSpace(rawLaneID)
-			if laneID == "" {
-				continue
-			}
-			if _, _, err := normalizedMapKeys(fileLane.References, fmt.Sprintf("pipeline %q lane %q reference slot", pipelineID, laneID)); err != nil {
-				return err
-			}
-			if _, _, err := normalizedMapKeys(fileLane.Extract.References, fmt.Sprintf("pipeline %q lane %q extract reference slot", pipelineID, laneID)); err != nil {
-				return err
-			}
-			if fileLane.Merge != nil {
-				if _, _, err := normalizedMapKeys(fileLane.Merge.References, fmt.Sprintf("pipeline %q lane %q merge reference slot", pipelineID, laneID)); err != nil {
-					return err
-				}
-			}
-			if fileLane.Normalize != nil {
-				if _, _, err := normalizedMapKeys(fileLane.Normalize.References, fmt.Sprintf("pipeline %q lane %q normalize reference slot", pipelineID, laneID)); err != nil {
-					return err
-				}
-			}
-			for i, validator := range fileLane.Validators {
-				if _, _, err := normalizedMapKeys(validator.References, fmt.Sprintf("pipeline %q lane %q validator[%d] reference slot", pipelineID, laneID, i)); err != nil {
-					return err
-				}
-			}
-		}
-	}
-
-	if fileCfg.PromptKit != nil {
-		if fileCfg.PromptKit.ProfileDir != nil {
-			value := strings.TrimSpace(*fileCfg.PromptKit.ProfileDir)
-			if value == "" {
-				return fmt.Errorf("promptkit.profile_dir must not be empty when set")
-			}
-			c.PromptKit.ProfileDir = value
-		}
-		if fileCfg.PromptKit.ProfileFile != nil {
-			value := strings.TrimSpace(*fileCfg.PromptKit.ProfileFile)
-			if value == "" {
-				return fmt.Errorf("promptkit.profile_file must not be empty when set")
-			}
-			c.PromptKit.ProfileFile = value
-		}
-		if fileCfg.PromptKit.LocalBackend != nil {
-			if fileCfg.PromptKit.LocalBackend.Endpoint == nil {
-				return fmt.Errorf("promptkit.local_backend.endpoint must not be empty when set")
-			}
-			endpoint := strings.TrimSpace(*fileCfg.PromptKit.LocalBackend.Endpoint)
-			if endpoint == "" {
-				return fmt.Errorf("promptkit.local_backend.endpoint must not be empty when set")
-			}
-			localBackend := PromptKitLocalBackendConfig{Endpoint: endpoint}
-			if fileCfg.PromptKit.LocalBackend.ConcurrencyLimit != nil {
-				localBackend.ConcurrencyLimit = *fileCfg.PromptKit.LocalBackend.ConcurrencyLimit
-			}
-			c.PromptKit.LocalBackend = &localBackend
-		}
-	}
-
-	for _, pipelineID := range pipelineIDs {
-		filePipeline := fileCfg.Pipelines[rawPipelineIDs[pipelineID]]
-		llmProfile := ""
-		if filePipeline.llmProfileSet || filePipeline.LLMProfile != nil {
-			if filePipeline.LLMProfile == nil || strings.TrimSpace(*filePipeline.LLMProfile) == "" {
-				return fmt.Errorf("pipeline %q llm_profile must not be empty when set", pipelineID)
-			}
-			llmProfile = strings.TrimSpace(*filePipeline.LLMProfile)
-		}
-		hasSteps := filePipeline.stepsSet || filePipeline.Steps != nil
-		laneIDs, rawLaneIDs, err := normalizedMapKeys(filePipeline.Artifacts, fmt.Sprintf("pipeline %q artifact lane id", pipelineID))
-		if err != nil {
-			return err
-		}
-		profile := pipeline.PipelineProfile{
-			ID:                             pipelineID,
-			LLMProfile:                     llmProfile,
-			StructuredOutputRepairAttempts: cloneStructuredOutputRepairAttempts(filePipeline.StructuredOutputRepairAttempts),
-			ValidationPolicy:               cloneValidationPolicyOverride(filePipeline.ValidationPolicy),
-			Input:                          filePipeline.Input.toPipelineBinding(),
-			Artifacts:                      make(map[string]pipeline.ArtifactLaneProfile, len(filePipeline.Artifacts)),
-			References:                     fileReferenceSourcesToPipeline(filePipeline.References),
-		}
-		if filePipeline.Chunk != nil {
-			profile.Chunk = filePipeline.Chunk.toPipelineBinding()
-		}
-		if filePipeline.Output != nil {
-			profile.Output = filePipeline.Output.toPipelineBinding()
-		}
-		for _, laneID := range laneIDs {
-			fileLane := filePipeline.Artifacts[rawLaneIDs[laneID]]
-			extract := fileLane.Extract.toPipelineBinding()
-			extract.References = mergeReferenceSources(fileReferenceSourcesToPipeline(fileLane.References), extract.References)
-			lane := pipeline.ArtifactLaneProfile{
-				Extract:    extract,
-				References: fileReferenceSourcesToPipeline(fileLane.References),
-			}
-			if fileLane.Merge != nil {
-				lane.Merge = fileLane.Merge.toPipelineBinding()
-			}
-			if fileLane.Normalize != nil {
-				lane.Normalize = fileLane.Normalize.toPipelineBinding()
-			}
-			if len(fileLane.Validators) > 0 {
-				lane.Validators = make([]pipeline.ModuleBinding, len(fileLane.Validators))
-				for i, validator := range fileLane.Validators {
-					lane.Validators[i] = validator.toPipelineBinding()
-				}
-			}
-			profile.Artifacts[laneID] = lane
-		}
-		if hasSteps {
-			profile.Artifacts = nil
-			profile.Steps = make([]pipeline.PipelineStepProfile, len(filePipeline.Steps))
-			for i, fileStep := range filePipeline.Steps {
-				stepID := strings.TrimSpace(fileStep.ID)
-				step := pipeline.PipelineStepProfile{
-					ID:         stepID,
-					Artifacts:  make(map[string]pipeline.ArtifactLaneProfile, len(fileStep.Artifacts)),
-					References: fileReferenceSourcesToPipeline(fileStep.References),
-				}
-				stepLaneIDs, stepRawLaneIDs, err := normalizedMapKeys(fileStep.Artifacts, fmt.Sprintf("pipeline %q step %q artifact lane id", pipelineID, stepID))
-				if err != nil {
-					return err
-				}
-				for _, laneID := range stepLaneIDs {
-					fileLane := fileStep.Artifacts[stepRawLaneIDs[laneID]]
-					extract := fileLane.Extract.toPipelineBinding()
-					extract.References = mergeReferenceSources(fileReferenceSourcesToPipeline(fileLane.References), extract.References)
-					lane := pipeline.ArtifactLaneProfile{Extract: extract, References: fileReferenceSourcesToPipeline(fileLane.References)}
-					if fileLane.Merge != nil {
-						lane.Merge = fileLane.Merge.toPipelineBinding()
-					}
-					if fileLane.Normalize != nil {
-						lane.Normalize = fileLane.Normalize.toPipelineBinding()
-					}
-					if len(fileLane.Validators) > 0 {
-						lane.Validators = make([]pipeline.ModuleBinding, len(fileLane.Validators))
-						for index, validator := range fileLane.Validators {
-							lane.Validators[index] = validator.toPipelineBinding()
-						}
-					}
-					step.Artifacts[laneID] = lane
-				}
-				profile.Steps[i] = step
-			}
-		}
-		c.Pipelines[pipelineID] = profile
-	}
-
-	if fileCfg.Concurrency != nil && fileCfg.Concurrency.TotalLLM != nil {
-		c.Concurrency.TotalLLM = *fileCfg.Concurrency.TotalLLM
-	}
-	if fileCfg.Concurrency != nil && fileCfg.Concurrency.StageWorkers != nil {
-		workers, configured, err := normalizeStageWorkers(fileCfg.Concurrency.StageWorkers)
-		if err != nil {
-			return err
-		}
-		c.Concurrency.StageWorkers = workers
-		c.Concurrency.extractWorkersConfigured = configured
-	}
-	c.Concurrency.recomputeStageWorkerDefaults()
-	if fileCfg.Output != nil && fileCfg.Output.Directory != nil {
-		c.Output.Directory = strings.TrimSpace(*fileCfg.Output.Directory)
-		if c.Output.Directory == "" {
-			return fmt.Errorf("output.directory must not be empty")
-		}
-		if strings.ContainsRune(c.Output.Directory, '\x00') {
-			return fmt.Errorf("output.directory must not contain NUL")
-		}
-	}
-	if fileCfg.Cache != nil {
-		if fileCfg.Cache.ChunkPlans != nil {
-			if fileCfg.Cache.ChunkPlans.Mode != nil {
-				mode, err := pipeline.ParseChunkCacheMode(*fileCfg.Cache.ChunkPlans.Mode)
-				if err != nil {
-					return fmt.Errorf("cache.chunk_plans.mode: %w", err)
-				}
-				c.Cache.ChunkPlans.Mode = mode
-			}
-			if fileCfg.Cache.ChunkPlans.Directory != nil {
-				c.Cache.ChunkPlans.Directory = cleanOptionalPath(*fileCfg.Cache.ChunkPlans.Directory)
-				if strings.ContainsRune(c.Cache.ChunkPlans.Directory, '\x00') {
-					return fmt.Errorf("cache.chunk_plans.directory must not contain NUL")
-				}
-			}
-		}
-		if fileCfg.Cache.Checkpoints != nil {
-			if fileCfg.Cache.Checkpoints.Enabled != nil {
-				c.Cache.Checkpoints.Enabled = *fileCfg.Cache.Checkpoints.Enabled
-			}
-			if fileCfg.Cache.Checkpoints.Directory != nil {
-				c.Cache.Checkpoints.Directory = cleanOptionalPath(*fileCfg.Cache.Checkpoints.Directory)
-				if strings.ContainsRune(c.Cache.Checkpoints.Directory, '\x00') {
-					return fmt.Errorf("cache.checkpoints.directory must not contain NUL")
-				}
-			}
-		}
-	}
-	if fileCfg.Debug != nil && fileCfg.Debug.Directory != nil {
-		c.Debug.Directory = strings.TrimSpace(*fileCfg.Debug.Directory)
-		if c.Debug.Directory == "" {
-			return fmt.Errorf("debug.directory must not be empty")
-		}
-		if strings.ContainsRune(c.Debug.Directory, '\x00') {
-			return fmt.Errorf("debug.directory must not contain NUL")
-		}
-	}
-
-	return nil
-}
-
-func cleanOptionalPath(value string) string {
-	value = strings.TrimSpace(value)
-	if value == "" {
-		return ""
-	}
-	return filepath.Clean(value)
-}
-
-func normalizeStageWorkers(values map[string]int) (map[string]int, bool, error) {
-	workers := make(map[string]int, len(values))
-	configured := false
-	for rawKey, value := range values {
-		key := strings.TrimSpace(rawKey)
-		if key == "" {
-			return nil, false, fmt.Errorf("concurrency.stage_workers key must not be empty")
-		}
-		if key != "extract" {
-			return nil, false, fmt.Errorf("concurrency.stage_workers key %q is not supported", rawKey)
-		}
-		if _, exists := workers[key]; exists {
-			return nil, false, fmt.Errorf("concurrency.stage_workers key %q is duplicated after trimming", key)
-		}
-		workers[key] = value
-		configured = true
-	}
-	return workers, configured, nil
-}
-
-func normalizedMapKeys[T any](values map[string]T, keyName string) ([]string, map[string]string, error) {
-	keys := make([]string, 0, len(values))
-	rawByNormalized := make(map[string]string, len(values))
-	for rawID := range values {
-		id := strings.TrimSpace(rawID)
-		if id == "" {
-			return nil, nil, fmt.Errorf("%s must not be empty", keyName)
-		}
-		if _, ok := rawByNormalized[id]; ok {
-			return nil, nil, fmt.Errorf("%s %q is duplicated after trimming", keyName, id)
-		}
-		rawByNormalized[id] = rawID
-		keys = append(keys, id)
-	}
-	sort.Strings(keys)
-	return keys, rawByNormalized, nil
-}
-
-func validateFileReferenceSources(values map[string]fileReferenceSource, context string) error {
-	seen := make(map[string]struct{}, len(values))
-	for rawSlot, source := range values {
-		slot := strings.TrimSpace(rawSlot)
-		if slot == "" {
-			return fmt.Errorf("%s must not be empty", context)
-		}
-		if _, ok := seen[slot]; ok {
-			return fmt.Errorf("%s %q is duplicated after trimming", context, slot)
-		}
-		seen[slot] = struct{}{}
-		if source.artifact != nil {
-			if strings.TrimSpace(source.artifact.Step) == "" || strings.TrimSpace(source.artifact.Lane) == "" {
-				return fmt.Errorf("%s %q artifact selector step and lane must not be empty", context, slot)
-			}
-			if strings.TrimSpace(source.path) != "" {
-				return fmt.Errorf("%s %q must contain either an external path or artifact selector", context, slot)
-			}
-			continue
-		}
-		if strings.TrimSpace(source.path) == "" {
-			return fmt.Errorf("%s %q source must not be empty", context, slot)
-		}
-	}
-	return nil
-}
-
-func validateFileLaneReferences(pipelineID, stepID, laneID string, lane FileArtifactLaneProfile) error {
-	prefix := fmt.Sprintf("pipeline %q step %q lane %q", pipelineID, stepID, laneID)
-	references := []struct {
-		label  string
-		values map[string]fileReferenceSource
-	}{
-		{label: "reference slot", values: lane.References},
-		{label: "extract reference slot", values: lane.Extract.References},
-	}
-	if lane.Merge != nil {
-		references = append(references, struct {
-			label  string
-			values map[string]fileReferenceSource
-		}{label: "merge reference slot", values: lane.Merge.References})
-	}
-	if lane.Normalize != nil {
-		references = append(references, struct {
-			label  string
-			values map[string]fileReferenceSource
-		}{label: "normalize reference slot", values: lane.Normalize.References})
-	}
-	for _, item := range references {
-		if err := validateFileReferenceSources(item.values, prefix+" "+item.label); err != nil {
-			return err
-		}
-	}
-	for index, validator := range lane.Validators {
-		if err := validateFileReferenceSources(validator.References, fmt.Sprintf("%s validator[%d] reference slot", prefix, index)); err != nil {
-			return err
-		}
-	}
-	return nil
-}
-
-func mergeReferenceSources(base map[string]pipeline.ReferenceSource, override map[string]pipeline.ReferenceSource) map[string]pipeline.ReferenceSource {
-	if len(base) == 0 && len(override) == 0 {
-		return nil
-	}
-	out := make(map[string]pipeline.ReferenceSource, len(base)+len(override))
-	for key, value := range base {
-		out[key] = value
-	}
-	for key, value := range override {
-		out[key] = value
-	}
-	return out
-}
-
-func normalizeOptions(options map[string]any) map[string]any {
-	if len(options) == 0 {
-		return nil
-	}
-	return cloneOptions(options)
-}

internal/core/config/file_config_contract_test.go

@@ -1,928 +0,0 @@
-package config
-
-import (
-	"encoding/json"
-	"fmt"
-	"os"
-	"path/filepath"
-	"reflect"
-	"strings"
-	"testing"
-
-	"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
-)
-
-func TestDefaultReturnsDocumentedValuesAndIndependentMaps(t *testing.T) {
-	first := Default()
-	if first.Concurrency.TotalLLM != 16 || first.Concurrency.StageWorkers["extract"] != 16 {
-		t.Fatalf("concurrency defaults = %#v", first.Concurrency)
-	}
-	if first.Output.Directory != "./notarius-output" || first.Debug.Directory != "./notarius-debug" {
-		t.Fatalf("output/debug defaults = %#v, %#v", first.Output, first.Debug)
-	}
-	if first.Cache.ChunkPlans.Mode != pipeline.ChunkCacheAuto || first.Cache.ChunkPlans.Directory != "" || first.Cache.Checkpoints.Enabled || first.Cache.Checkpoints.Directory != "" {
-		t.Fatalf("cache defaults = %#v", first.Cache)
-	}
-	if len(first.Pipelines) != 0 {
-		t.Fatalf("pipeline defaults = %#v", first.Pipelines)
-	}
-
-	first.Concurrency.StageWorkers["extract"] = 99
-	first.Concurrency.StageWorkers["other"] = 100
-	first.Pipelines["changed"] = pipeline.PipelineProfile{}
-	second := Default()
-	if second.Concurrency.StageWorkers["extract"] != 16 || len(second.Concurrency.StageWorkers) != 1 || len(second.Pipelines) != 0 {
-		t.Fatalf("Default() returned state shared with an earlier result: %#v", second)
-	}
-}
-
-func TestFileConfigMinimalVersion4AppliesOverDefaults(t *testing.T) {
-	file := parseFileConfig(t, "version: 4\n")
-	cfg := Default()
-	if err := cfg.ApplyFileConfig(file); err != nil {
-		t.Fatal(err)
-	}
-	if cfg.Output.Directory != "./notarius-output" || cfg.Debug.Directory != "./notarius-debug" || cfg.Cache.ChunkPlans.Mode != pipeline.ChunkCacheAuto {
-		t.Fatalf("minimal file changed unrelated defaults: %#v", cfg)
-	}
-	if cfg.Concurrency.TotalLLM != 16 || cfg.Concurrency.StageWorkers["extract"] != 16 || len(cfg.Pipelines) != 0 {
-		t.Fatalf("minimal file did not retain defaults: %#v", cfg)
-	}
-}
-
-func TestParseFileConfigYAMLRejectsAdditionalDocuments(t *testing.T) {
-	tests := []struct {
-		name    string
-		source  string
-		wantErr bool
-	}{
-		{name: "trailing whitespace", source: "version: 4\n\n  \t", wantErr: false},
-		{name: "trailing comment", source: "version: 4\n# trailing comment\n", wantErr: false},
-		{name: "second valid document", source: "version: 4\n---\nversion: 4\n", wantErr: true},
-		{name: "second empty document", source: "version: 4\n---\n", wantErr: true},
-		{name: "second malformed document", source: "version: 4\n---\nversion: [\n", wantErr: true},
-	}
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			_, err := ParseFileConfigYAML([]byte(tt.source))
-			if (err != nil) != tt.wantErr {
-				t.Fatalf("ParseFileConfigYAML() error = %v, want error=%t", err, tt.wantErr)
-			}
-		})
-	}
-}
-
-func TestFilePipelineLLMProfileIsPresenceAwareAndDetached(t *testing.T) {
-	const pipelineYAML = `version: 4
-pipelines:
-  main:
-    %s
-    input: input
-    artifacts:
-      lane:
-        extract: extract
-`
-
-	t.Run("omitted", func(t *testing.T) {
-		file := parseFileConfig(t, fmt.Sprintf(pipelineYAML, ""))
-		if file.Pipelines["main"].LLMProfile != nil || file.Pipelines["main"].llmProfileSet {
-			t.Fatalf("parsed pipeline profile = %#v, want omitted llm profile", file.Pipelines["main"])
-		}
-		cfg := Default()
-		if err := cfg.ApplyFileConfig(file); err != nil {
-			t.Fatal(err)
-		}
-		if got := cfg.Pipelines["main"].LLMProfile; got != "" {
-			t.Fatalf("pipeline llm profile = %q, want empty", got)
-		}
-	})
-
-	t.Run("trimmed and detached", func(t *testing.T) {
-		file := parseFileConfig(t, fmt.Sprintf(pipelineYAML, "llm_profile: ' configured-profile '"))
-		cfg := Default()
-		if err := cfg.ApplyFileConfig(file); err != nil {
-			t.Fatal(err)
-		}
-		if got := cfg.Pipelines["main"].LLMProfile; got != "configured-profile" {
-			t.Fatalf("pipeline llm profile = %q, want trimmed value", got)
-		}
-		*file.Pipelines["main"].LLMProfile = "changed-profile"
-		if got := cfg.Pipelines["main"].LLMProfile; got != "configured-profile" {
-			t.Fatalf("effective config aliases parsed file: %q", got)
-		}
-		if got := cloneConfig(cfg).Pipelines["main"].LLMProfile; got != "configured-profile" {
-			t.Fatalf("cloned pipeline llm profile = %q", got)
-		}
-		data, err := json.Marshal(cfg)
-		if err != nil {
-			t.Fatal(err)
-		}
-		var roundTripped Config
-		if err := json.Unmarshal(data, &roundTripped); err != nil {
-			t.Fatal(err)
-		}
-		if got := roundTripped.Pipelines["main"].LLMProfile; got != "configured-profile" {
-			t.Fatalf("round-tripped pipeline llm profile = %q", got)
-		}
-	})
-
-	for _, value := range []string{"''", "'   '", "null"} {
-		t.Run("explicit empty "+value, func(t *testing.T) {
-			file := parseFileConfig(t, fmt.Sprintf(pipelineYAML, "llm_profile: "+value))
-			cfg := Default()
-			err := cfg.ApplyFileConfig(file)
-			if err == nil || !strings.Contains(err.Error(), `pipeline "main" llm_profile must not be empty`) {
-				t.Fatalf("ApplyFileConfig() error = %v, want explicit-empty rejection", err)
-			}
-		})
-	}
-}
-
-func TestStructuredOutputRepairAttemptsFileConfigurationPreservesPresenceAndOwnership(t *testing.T) {
-	const pipelineYAML = `version: 4
-pipelines:
-  main:
-    %s
-    input: input
-    artifacts:
-      lane:
-        extract:
-          module: extract
-          structured_output_repair_attempts: 2
-          validators:
-            - module: validator
-              structured_output_repair_attempts: 3
-`
-
-	t.Run("omitted pipeline value remains absent", func(t *testing.T) {
-		file := parseFileConfig(t, fmt.Sprintf(pipelineYAML, ""))
-		if got := file.Pipelines["main"].StructuredOutputRepairAttempts; got != nil {
-			t.Fatalf("file pipeline repair attempts = %v, want nil", *got)
-		}
-		cfg := Default()
-		if err := cfg.ApplyFileConfig(file); err != nil {
-			t.Fatal(err)
-		}
-		if got := cfg.Pipelines["main"].StructuredOutputRepairAttempts; got != nil {
-			t.Fatalf("pipeline repair attempts = %v, want nil", *got)
-		}
-		if got := cfg.Pipelines["main"].Input.StructuredOutputRepairAttempts; got != nil {
-			t.Fatalf("scalar input binding repair attempts = %v, want nil", *got)
-		}
-	})
-
-	t.Run("zero is explicit and survives configuration boundaries", func(t *testing.T) {
-		file := parseFileConfig(t, fmt.Sprintf(pipelineYAML, "structured_output_repair_attempts: 0"))
-		cfg := Default()
-		if err := cfg.ApplyFileConfig(file); err != nil {
-			t.Fatal(err)
-		}
-		profile := cfg.Pipelines["main"]
-		if profile.StructuredOutputRepairAttempts == nil || *profile.StructuredOutputRepairAttempts != 0 {
-			t.Fatalf("pipeline repair attempts = %v, want explicit zero", profile.StructuredOutputRepairAttempts)
-		}
-		lane := profile.Artifacts["lane"]
-		if lane.Extract.StructuredOutputRepairAttempts == nil || *lane.Extract.StructuredOutputRepairAttempts != 2 {
-			t.Fatalf("extract repair attempts = %v, want 2", lane.Extract.StructuredOutputRepairAttempts)
-		}
-		if lane.Extract.Validators.Validators[0].StructuredOutputRepairAttempts == nil || *lane.Extract.Validators.Validators[0].StructuredOutputRepairAttempts != 3 {
-			t.Fatalf("validator repair attempts = %v, want 3", lane.Extract.Validators.Validators[0].StructuredOutputRepairAttempts)
-		}
-
-		*file.Pipelines["main"].StructuredOutputRepairAttempts = 1
-		if got := *cfg.Pipelines["main"].StructuredOutputRepairAttempts; got != 0 {
-			t.Fatalf("applied config aliased file configuration: got %d, want 0", got)
-		}
-		fileProfile := file.Pipelines["main"]
-		fileLane := fileProfile.Artifacts["lane"]
-		*fileLane.Extract.StructuredOutputRepairAttempts = 1
-		*fileLane.Extract.Validators.Validators[0].StructuredOutputRepairAttempts = 1
-		fileProfile.Artifacts["lane"] = fileLane
-		file.Pipelines["main"] = fileProfile
-		configuredLane := cfg.Pipelines["main"].Artifacts["lane"]
-		if got := *configuredLane.Extract.StructuredOutputRepairAttempts; got != 2 {
-			t.Fatalf("configured extract aliased file configuration: got %d, want 2", got)
-		}
-		if got := *configuredLane.Extract.Validators.Validators[0].StructuredOutputRepairAttempts; got != 3 {
-			t.Fatalf("configured validator aliased file configuration: got %d, want 3", got)
-		}
-		cloned := cloneConfig(cfg)
-		*cloned.Pipelines["main"].StructuredOutputRepairAttempts = 1
-		if got := *cfg.Pipelines["main"].StructuredOutputRepairAttempts; got != 0 {
-			t.Fatalf("cloned config aliased source configuration: got %d, want 0", got)
-		}
-		redacted := cfg.Redacted()
-		*redacted.Pipelines["main"].StructuredOutputRepairAttempts = 1
-		if got := *cfg.Pipelines["main"].StructuredOutputRepairAttempts; got != 0 {
-			t.Fatalf("redacted config aliased source configuration: got %d, want 0", got)
-		}
-		summary := cfg.RedactedSummaryPayload().(Config)
-		if summary.Pipelines["main"].StructuredOutputRepairAttempts == nil || *summary.Pipelines["main"].StructuredOutputRepairAttempts != 0 {
-			t.Fatalf("redacted summary pipeline repair attempts = %v, want explicit zero", summary.Pipelines["main"].StructuredOutputRepairAttempts)
-		}
-
-		data, err := json.Marshal(cfg)
-		if err != nil {
-			t.Fatal(err)
-		}
-		var roundTripped Config
-		if err := json.Unmarshal(data, &roundTripped); err != nil {
-			t.Fatal(err)
-		}
-		roundTrippedProfile := roundTripped.Pipelines["main"]
-		if roundTrippedProfile.StructuredOutputRepairAttempts == nil || *roundTrippedProfile.StructuredOutputRepairAttempts != 0 {
-			t.Fatalf("round-tripped pipeline repair attempts = %v, want explicit zero", roundTrippedProfile.StructuredOutputRepairAttempts)
-		}
-		roundTrippedLane := roundTrippedProfile.Artifacts["lane"]
-		if roundTrippedLane.Extract.StructuredOutputRepairAttempts == nil || *roundTrippedLane.Extract.StructuredOutputRepairAttempts != 2 {
-			t.Fatalf("round-tripped extract repair attempts = %v, want 2", roundTrippedLane.Extract.StructuredOutputRepairAttempts)
-		}
-		if roundTrippedLane.Extract.Validators.Validators[0].StructuredOutputRepairAttempts == nil || *roundTrippedLane.Extract.Validators.Validators[0].StructuredOutputRepairAttempts != 3 {
-			t.Fatalf("round-tripped validator repair attempts = %v, want 3", roundTrippedLane.Extract.Validators.Validators[0].StructuredOutputRepairAttempts)
-		}
-	})
-}
-
-func TestStructuredOutputRepairAttemptsFileConfigurationRejectsInvalidValues(t *testing.T) {
-	const pipelineYAML = `version: 4
-pipelines:
-  main:
-    input: input
-    artifacts:
-      lane:
-        extract: extract
-    %s
-`
-	const bindingYAML = `version: 4
-pipelines:
-  main:
-    input:
-      module: input
-      %s
-    artifacts:
-      lane:
-        extract: extract
-`
-
-	for _, tt := range []struct {
-		name   string
-		source string
-		want   string
-	}{
-		{name: "null pipeline value", source: "structured_output_repair_attempts: null", want: "pipeline profile structured_output_repair_attempts must not be null"},
-		{name: "fractional pipeline value", source: "structured_output_repair_attempts: 1.5", want: "pipeline profile structured_output_repair_attempts must be an integer"},
-		{name: "quoted pipeline value", source: "structured_output_repair_attempts: '1'", want: "pipeline profile structured_output_repair_attempts must be an integer"},
-		{name: "out of range pipeline value", source: "structured_output_repair_attempts: 4", want: "pipeline profile structured_output_repair_attempts must be between zero and three"},
-	} {
-		t.Run(tt.name, func(t *testing.T) {
-			_, err := ParseFileConfigYAML([]byte(fmt.Sprintf(pipelineYAML, tt.source)))
-			if err == nil || !strings.Contains(err.Error(), tt.want) {
-				t.Fatalf("ParseFileConfigYAML() error = %v, want %q", err, tt.want)
-			}
-		})
-	}
-
-	for _, tt := range []struct {
-		name   string
-		source string
-		want   string
-	}{
-		{name: "null binding value", source: "structured_output_repair_attempts: null", want: "module binding structured_output_repair_attempts must not be null"},
-		{name: "noninteger binding value", source: "structured_output_repair_attempts: true", want: "module binding structured_output_repair_attempts must be an integer"},
-		{name: "out of range binding value", source: "structured_output_repair_attempts: -1", want: "module binding structured_output_repair_attempts must be between zero and three"},
-	} {
-		t.Run(tt.name, func(t *testing.T) {
-			_, err := ParseFileConfigYAML([]byte(fmt.Sprintf(bindingYAML, tt.source)))
-			if err == nil || !strings.Contains(err.Error(), tt.want) {
-				t.Fatalf("ParseFileConfigYAML() error = %v, want %q", err, tt.want)
-			}
-		})
-	}
-}
-
-func TestFileModuleBindingRejectsExplicitEmptyLLMProfile(t *testing.T) {
-	const configYAML = `version: 4
-pipelines:
-  main:
-    input:
-      module: input
-      llm_profile: %s
-    artifacts:
-      lane:
-        extract: extract
-`
-	for _, value := range []string{"''", "'   '", "null"} {
-		t.Run(value, func(t *testing.T) {
-			_, err := ParseFileConfigYAML([]byte(fmt.Sprintf(configYAML, value)))
-			if err == nil || !strings.Contains(err.Error(), "llm_profile must not be empty when set") {
-				t.Fatalf("ParseFileConfigYAML() error = %v, want explicit-empty binding profile rejection", err)
-			}
-		})
-	}
-}
-
-func TestFilePromptKitProfileSourcesSurviveConfigBoundaries(t *testing.T) {
-	tests := []struct {
-		name string
-		yaml string
-		want PromptKitConfig
-	}{
-		{
-			name: "profile directory",
-			yaml: "version: 4\npromptkit:\n  profile_dir: ' ./profiles '\n",
-			want: PromptKitConfig{ProfileDir: "./profiles"},
-		},
-		{
-			name: "profile file",
-			yaml: "version: 4\npromptkit:\n  profile_file: ' ./profiles.yml '\n",
-			want: PromptKitConfig{ProfileFile: "./profiles.yml"},
-		},
-	}
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			cfg := applyFileConfig(t, tt.yaml)
-			if cfg.PromptKit != tt.want {
-				t.Fatalf("PromptKit config = %#v, want %#v", cfg.PromptKit, tt.want)
-			}
-			if got := cloneConfig(cfg).PromptKit; got != tt.want {
-				t.Fatalf("cloned PromptKit config = %#v, want %#v", got, tt.want)
-			}
-			if got := cfg.Redacted().PromptKit; got != tt.want {
-				t.Fatalf("redacted PromptKit config = %#v, want %#v", got, tt.want)
-			}
-
-			data, err := json.Marshal(cfg)
-			if err != nil {
-				t.Fatalf("json.Marshal() error = %v", err)
-			}
-			var payload map[string]json.RawMessage
-			if err := json.Unmarshal(data, &payload); err != nil {
-				t.Fatalf("json.Unmarshal() error = %v", err)
-			}
-			if _, ok := payload["promptkit"]; !ok {
-				t.Fatalf("runtime JSON keys = %v, want promptkit", payload)
-			}
-			if _, ok := payload["scriptorium"]; ok {
-				t.Fatalf("runtime JSON keys = %v, must not contain removed section", payload)
-			}
-		})
-	}
-}
-
-func TestFilePromptKitLocalBackendSurvivesConfigBoundaries(t *testing.T) {
-	tests := []struct {
-		name            string
-		concurrencyYAML string
-		wantConcurrency int
-	}{
-		{name: "omitted concurrency defaults to zero"},
-		{name: "positive concurrency is preserved", concurrencyYAML: "    concurrency_limit: 2\n", wantConcurrency: 2},
-	}
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			file := parseFileConfig(t, "version: 4\npromptkit:\n  local_backend:\n    endpoint: ' http://localhost:8000/v1 '\n"+tt.concurrencyYAML)
-			cfg := Default()
-			if err := cfg.ApplyFileConfig(file); err != nil {
-				t.Fatalf("ApplyFileConfig() error = %v", err)
-			}
-			want := PromptKitLocalBackendConfig{
-				Endpoint:         "http://localhost:8000/v1",
-				ConcurrencyLimit: tt.wantConcurrency,
-			}
-			if cfg.PromptKit.LocalBackend == nil || *cfg.PromptKit.LocalBackend != want {
-				t.Fatalf("local backend config = %#v, want %#v", cfg.PromptKit.LocalBackend, want)
-			}
-
-			*file.PromptKit.LocalBackend.Endpoint = "http://changed.example/v1"
-			if file.PromptKit.LocalBackend.ConcurrencyLimit != nil {
-				*file.PromptKit.LocalBackend.ConcurrencyLimit = 99
-			}
-			if *cfg.PromptKit.LocalBackend != want {
-				t.Fatalf("effective config aliases parsed file model: %#v", cfg.PromptKit.LocalBackend)
-			}
-
-			cloned := cloneConfig(cfg)
-			if cloned.PromptKit.LocalBackend == cfg.PromptKit.LocalBackend || *cloned.PromptKit.LocalBackend != want {
-				t.Fatalf("cloned local backend = %#v, want detached %#v", cloned.PromptKit.LocalBackend, want)
-			}
-			cloned.PromptKit.LocalBackend.Endpoint = "http://clone.example/v1"
-			if *cfg.PromptKit.LocalBackend != want {
-				t.Fatalf("mutating clone changed source config: %#v", cfg.PromptKit.LocalBackend)
-			}
-
-			redacted := cfg.Redacted()
-			if redacted.PromptKit.LocalBackend == cfg.PromptKit.LocalBackend || *redacted.PromptKit.LocalBackend != want {
-				t.Fatalf("redacted local backend = %#v, want detached %#v", redacted.PromptKit.LocalBackend, want)
-			}
-
-			data, err := json.Marshal(cfg)
-			if err != nil {
-				t.Fatalf("json.Marshal() error = %v", err)
-			}
-			var payload struct {
-				PromptKit map[string]json.RawMessage `json:"promptkit"`
-			}
-			if err := json.Unmarshal(data, &payload); err != nil {
-				t.Fatalf("json.Unmarshal() error = %v", err)
-			}
-			localJSON, ok := payload.PromptKit["local_backend"]
-			if !ok {
-				t.Fatalf("runtime PromptKit JSON keys = %v, want local_backend", payload.PromptKit)
-			}
-			var localPayload map[string]json.RawMessage
-			if err := json.Unmarshal(localJSON, &localPayload); err != nil {
-				t.Fatalf("unmarshal local_backend JSON: %v", err)
-			}
-			if _, ok := localPayload["endpoint"]; !ok {
-				t.Fatalf("runtime local_backend JSON keys = %v, want endpoint", localPayload)
-			}
-			if _, ok := localPayload["concurrency_limit"]; !ok {
-				t.Fatalf("runtime local_backend JSON keys = %v, want concurrency_limit", localPayload)
-			}
-		})
-	}
-}
-
-func TestFilePromptKitLocalBackendRequiresEndpoint(t *testing.T) {
-	for _, tt := range []struct {
-		name string
-		yaml string
-	}{
-		{name: "missing", yaml: "version: 4\npromptkit:\n  local_backend: {}\n"},
-		{name: "empty", yaml: "version: 4\npromptkit:\n  local_backend:\n    endpoint: ''\n"},
-		{name: "blank", yaml: "version: 4\npromptkit:\n  local_backend:\n    endpoint: '   '\n"},
-	} {
-		t.Run(tt.name, func(t *testing.T) {
-			file := parseFileConfig(t, tt.yaml)
-			cfg := Default()
-			err := cfg.ApplyFileConfig(file)
-			if err == nil || !strings.Contains(err.Error(), "promptkit.local_backend.endpoint") {
-				t.Fatalf("ApplyFileConfig() error = %v, want endpoint field context", err)
-			}
-		})
-	}
-}
-
-func TestFilePromptKitExplicitEmptyProfileSourcesAreRejected(t *testing.T) {
-	for _, field := range []string{"profile_dir", "profile_file"} {
-		t.Run(field, func(t *testing.T) {
-			file := parseFileConfig(t, "version: 4\npromptkit:\n  "+field+": ''\n")
-			cfg := Default()
-			err := cfg.ApplyFileConfig(file)
-			if err == nil || !strings.Contains(err.Error(), "promptkit."+field+" must not be empty") {
-				t.Fatalf("ApplyFileConfig() error = %v, want explicit-empty rejection", err)
-			}
-		})
-	}
-}
-
-func TestFilePromptKitProfileSourcesRemainMutuallyExclusive(t *testing.T) {
-	cfg := applyFileConfig(t, `version: 4
-promptkit:
-  profile_dir: ./profiles
-  profile_file: ./profiles.yml
-`)
-	if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "promptkit profile_dir and profile_file are mutually exclusive") {
-		t.Fatalf("Validate() error = %v, want mutually exclusive profile sources", err)
-	}
-}
-
-func TestFileConfigMissingVersionIsReportedBeforeFieldDecoding(t *testing.T) {
-	_, err := ParseFileConfigYAML([]byte("workspace:\n  directory: /tmp/old\n"))
-	if err == nil || !strings.Contains(err.Error(), "config version is required") {
-		t.Fatalf("missing version error = %v", err)
-	}
-}
-
-func TestFileConfigVersion3ReportsPromptKitMigration(t *testing.T) {
-	_, err := ParseFileConfigYAML([]byte("version: 3\nscriptorium:\n  profile_dir: ./profiles\n"))
-	if err == nil ||
-		!strings.Contains(err.Error(), `change "version: 3" to "version: 4"`) ||
-		!strings.Contains(err.Error(), `rename "scriptorium:" to "promptkit:"`) {
-		t.Fatalf("version 3 error = %v, want actionable version and section migration", err)
-	}
-}
-
-func TestFileConfigRejectsUnknownCurrentAndRemovedFields(t *testing.T) {
-	tests := []struct {
-		name string
-		yaml string
-		want string
-	}{
-		{
-			name: "removed diagnostics",
-			yaml: "version: 4\ndiagnostics: {}\n",
-			want: "field diagnostics not found",
-		},
-		{
-			name: "removed llm profiles",
-			yaml: "version: 4\nllm_profiles: {}\n",
-			want: "field llm_profiles not found",
-		},
-		{
-			name: "removed scriptorium section",
-			yaml: "version: 4\nscriptorium: {}\n",
-			want: "field scriptorium not found",
-		},
-		{
-			name: "version 2 migration",
-			yaml: "version: 2\nworkspace:\n  directory: /tmp/old\n",
-			want: "version 2-to-3 migration",
-		},
-		{
-			name: "pipeline field",
-			yaml: "version: 4\npipelines:\n  main:\n    unknown: true\n",
-			want: "field unknown not found",
-		},
-		{
-			name: "lane field",
-			yaml: "version: 4\npipelines:\n  main:\n    artifacts:\n      spells:\n        unknown: true\n",
-			want: "field unknown not found",
-		},
-		{
-			name: "module binding field",
-			yaml: "version: 4\npipelines:\n  main:\n    input:\n      module: seriatim\n      unknown: true\n",
-			want: "field unknown not found in module binding",
-		},
-		{
-			name: "checkpoint field",
-			yaml: "version: 4\ncache:\n  checkpoints:\n    unknown: true\n",
-			want: "field unknown not found",
-		},
-		{
-			name: "checkpoint enabled type",
-			yaml: "version: 4\ncache:\n  checkpoints:\n    enabled: definitely\n",
-			want: "cannot unmarshal",
-		},
-		{
-			name: "local backend field",
-			yaml: "version: 4\npromptkit:\n  local_backend:\n    endpoint: http://localhost:8000/v1\n    unknown: true\n",
-			want: "field unknown not found",
-		},
-	}
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			_, err := ParseFileConfigYAML([]byte(tt.yaml))
-			if err == nil || !strings.Contains(err.Error(), tt.want) {
-				t.Fatalf("error = %v, want context %q", err, tt.want)
-			}
-		})
-	}
-}
-
-func TestFileConfigModuleBindingsPreserveFormsAndValidatorPresence(t *testing.T) {
-	cfg := applyFileConfig(t, `version: 4
-pipelines:
-  main:
-    input: seriatim
-    chunk:
-      module: generic
-      llm_profile: chunk-profile
-      retries: 2
-      options:
-        max_units: 25
-      references:
-        glossary: ./glossary.md
-      validators: []
-    artifacts:
-      spells:
-        extract:
-          module: dnd/spells
-          options:
-            nested:
-              enabled: true
-        merge: appendorder
-        normalize: noop
-`)
-	profile := cfg.Pipelines["main"]
-	if profile.Input.Module != "seriatim" || profile.Input.Validators.Set {
-		t.Fatalf("shorthand binding = %#v", profile.Input)
-	}
-	if profile.Chunk.Module != "generic" || profile.Chunk.LLMProfile != "chunk-profile" || profile.Chunk.Retries != 2 ||
-		!reflect.DeepEqual(profile.Chunk.Options, map[string]any{"max_units": 25}) ||
-		!reflect.DeepEqual(profile.Chunk.References, pipeline.ExternalReferenceMap(map[string]string{"glossary": "./glossary.md"})) {
-		t.Fatalf("object binding = %#v", profile.Chunk)
-	}
-	if !profile.Chunk.Validators.Set || len(profile.Chunk.Validators.Validators) != 0 {
-		t.Fatalf("explicit empty validators = %#v", profile.Chunk.Validators)
-	}
-	if profile.Artifacts["spells"].Extract.Module != "dnd/spells" ||
-		!reflect.DeepEqual(profile.Artifacts["spells"].Extract.Options, map[string]any{
-			"nested": map[string]any{"enabled": true},
-		}) {
-		t.Fatalf("extract binding = %#v", profile.Artifacts["spells"].Extract)
-	}
-	if profile.Artifacts["spells"].Merge.Module != "appendorder" || profile.Artifacts["spells"].Normalize.Module != "noop" {
-		t.Fatalf("stage shorthand bindings = %#v", profile.Artifacts["spells"])
-	}
-}
-
-func TestFileConfigReferencePrecedenceIsRetained(t *testing.T) {
-	cfg := applyFileConfig(t, `version: 4
-pipelines:
-  main:
-    input: seriatim
-    references:
-      pipeline-only: ./pipeline.txt
-      shared: ./pipeline-shared.txt
-    chunk:
-      module: generic
-      references:
-        chunk-only: ./chunk.txt
-    artifacts:
-      spells:
-        references:
-          lane-only: ./lane.txt
-          shared: ./lane-shared.txt
-          overridden: ./lane.txt
-        extract:
-          module: dnd/spells
-          references:
-            extract-only: ./extract.txt
-            overridden: ./extract-overridden.txt
-        merge:
-          module: appendorder
-          references:
-            merge-only: ./merge.txt
-        normalize:
-          module: noop
-          references:
-            normalize-only: ./normalize.txt
-`)
-	profile := cfg.Pipelines["main"]
-	if !reflect.DeepEqual(profile.References, pipeline.ExternalReferenceMap(map[string]string{
-		"pipeline-only": "./pipeline.txt",
-		"shared":        "./pipeline-shared.txt",
-	})) {
-		t.Fatalf("pipeline references = %#v", profile.References)
-	}
-	if !reflect.DeepEqual(profile.Chunk.References, pipeline.ExternalReferenceMap(map[string]string{"chunk-only": "./chunk.txt"})) {
-		t.Fatalf("chunk references = %#v", profile.Chunk.References)
-	}
-	lane := profile.Artifacts["spells"]
-	if !reflect.DeepEqual(lane.References, pipeline.ExternalReferenceMap(map[string]string{
-		"lane-only":  "./lane.txt",
-		"shared":     "./lane-shared.txt",
-		"overridden": "./lane.txt",
-	})) {
-		t.Fatalf("lane compatibility references = %#v", lane.References)
-	}
-	if !reflect.DeepEqual(lane.Extract.References, pipeline.ExternalReferenceMap(map[string]string{
-		"lane-only":    "./lane.txt",
-		"shared":       "./lane-shared.txt",
-		"overridden":   "./extract-overridden.txt",
-		"extract-only": "./extract.txt",
-	})) {
-		t.Fatalf("extract references = %#v", lane.Extract.References)
-	}
-	if !reflect.DeepEqual(lane.Merge.References, pipeline.ExternalReferenceMap(map[string]string{"merge-only": "./merge.txt"})) ||
-		!reflect.DeepEqual(lane.Normalize.References, pipeline.ExternalReferenceMap(map[string]string{"normalize-only": "./normalize.txt"})) {
-		t.Fatalf("merge/normalize references = %#v, %#v", lane.Merge.References, lane.Normalize.References)
-	}
-}
-
-func TestFileConfigStageLocalValidatorsPreserveOrderAndFields(t *testing.T) {
-	cfg := applyFileConfig(t, `version: 4
-pipelines:
-  main:
-    input: seriatim
-    chunk:
-      module: generic
-      validators:
-        - generic/always_accept
-        - module: generic/valid_json
-          llm_profile: validator-profile
-          options:
-            schema: compact
-    artifacts:
-      spells:
-        extract:
-          module: dnd/spells
-          validators:
-            - module: extract/dnd/spells/shape
-              options:
-                strict: true
-        merge:
-          module: appendorder
-          validators:
-            - generic/always_accept
-        normalize:
-          module: noop
-          validators:
-            - module: generic/valid_json
-              options:
-                mode: normalized
-`)
-	profile := cfg.Pipelines["main"]
-	chunkValidators := profile.Chunk.Validators.Validators
-	if !profile.Chunk.Validators.Set || len(chunkValidators) != 2 || chunkValidators[0].Module != "generic/always_accept" ||
-		chunkValidators[1].Module != "generic/valid_json" || chunkValidators[1].LLMProfile != "validator-profile" ||
-		!reflect.DeepEqual(chunkValidators[1].Options, map[string]any{"schema": "compact"}) {
-		t.Fatalf("chunk validators = %#v", profile.Chunk.Validators)
-	}
-	lane := profile.Artifacts["spells"]
-	if len(lane.Extract.Validators.Validators) != 1 || lane.Extract.Validators.Validators[0].Module != "extract/dnd/spells/shape" ||
-		!reflect.DeepEqual(lane.Extract.Validators.Validators[0].Options, map[string]any{"strict": true}) {
-		t.Fatalf("extract validators = %#v", lane.Extract.Validators)
-	}
-	if len(lane.Merge.Validators.Validators) != 1 || lane.Merge.Validators.Validators[0].Module != "generic/always_accept" {
-		t.Fatalf("merge validators = %#v", lane.Merge.Validators)
-	}
-	if len(lane.Normalize.Validators.Validators) != 1 || lane.Normalize.Validators.Validators[0].Module != "generic/valid_json" ||
-		!reflect.DeepEqual(lane.Normalize.Validators.Validators[0].Options, map[string]any{"mode": "normalized"}) {
-		t.Fatalf("normalize validators = %#v", lane.Normalize.Validators)
-	}
-}
-
-func TestFileConfigStateSectionsApplyIndependently(t *testing.T) {
-	cfg := applyFileConfig(t, `version: 4
-promptkit:
-  profile_dir: ./profiles
-concurrency:
-  total_llm: 7
-output:
-  directory: ./output
-cache:
-  chunk_plans:
-    directory: ./plans
-    mode: bypass
-  checkpoints:
-    enabled: true
-    directory: ./checkpoints
-debug:
-  directory: ./debug
-`)
-	if cfg.PromptKit.ProfileDir != "./profiles" || cfg.PromptKit.ProfileFile != "" {
-		t.Fatalf("promptkit = %#v", cfg.PromptKit)
-	}
-	if cfg.Concurrency.TotalLLM != 7 || cfg.Concurrency.StageWorkers["extract"] != 7 {
-		t.Fatalf("concurrency = %#v", cfg.Concurrency)
-	}
-	if cfg.Output.Directory != "./output" || cfg.Cache.ChunkPlans.Directory != "plans" || cfg.Cache.ChunkPlans.Mode != pipeline.ChunkCacheBypass ||
-		!cfg.Cache.Checkpoints.Enabled || cfg.Cache.Checkpoints.Directory != "checkpoints" || cfg.Debug.Directory != "./debug" {
-		t.Fatalf("state sections = %#v, %#v, %#v, %#v", cfg.Output, cfg.Cache, cfg.Debug, cfg.PromptKit)
-	}
-	if cfg.Output.Directory == cfg.Cache.ChunkPlans.Directory || cfg.Cache.ChunkPlans.Directory == cfg.Cache.Checkpoints.Directory || cfg.Cache.Checkpoints.Directory == cfg.Debug.Directory {
-		t.Fatal("state roots were coupled")
-	}
-}
-
-func TestFileConfigCheckpointEnabledCanBeExplicitlyDisabled(t *testing.T) {
-	cfg := applyFileConfig(t, "version: 4\ncache:\n  checkpoints:\n    enabled: true\n")
-	if !cfg.Cache.Checkpoints.Enabled || !cloneConfig(cfg).Cache.Checkpoints.Enabled {
-		t.Fatalf("enabled checkpoint config was not retained: %#v", cfg.Cache.Checkpoints)
-	}
-	file := parseFileConfig(t, "version: 4\ncache:\n  checkpoints:\n    enabled: false\n")
-	if err := cfg.ApplyFileConfig(file); err != nil {
-		t.Fatal(err)
-	}
-	if cfg.Cache.Checkpoints.Enabled {
-		t.Fatalf("explicit false checkpoint config was not applied: %#v", cfg.Cache.Checkpoints)
-	}
-}
-
-func TestFileConfigRejectsTrimmedKeyCollisions(t *testing.T) {
-	tests := []struct {
-		name string
-		yaml string
-		want string
-	}{
-		{
-			name: "pipeline ids",
-			yaml: "version: 4\npipelines:\n  main: {}\n  ' main ': {}\n",
-			want: "pipeline id \"main\" is duplicated after trimming",
-		},
-		{
-			name: "lane ids",
-			yaml: "version: 4\npipelines:\n  main:\n    artifacts:\n      spells: {}\n      ' spells ': {}\n",
-			want: "artifact lane id \"spells\" is duplicated after trimming",
-		},
-		{
-			name: "reference slots",
-			yaml: "version: 4\npipelines:\n  main:\n    references:\n      slot: ./one.txt\n      ' slot ': ./two.txt\n",
-			want: "reference slot \"slot\" is duplicated after trimming",
-		},
-	}
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			file := parseFileConfig(t, tt.yaml)
-			cfg := Default()
-			err := cfg.ApplyFileConfig(file)
-			if err == nil || !strings.Contains(err.Error(), tt.want) {
-				t.Fatalf("ApplyFileConfig() error = %v, want context %q", err, tt.want)
-			}
-		})
-	}
-}
-
-func TestFileConfigParsesOrderedStepsAndReferenceSources(t *testing.T) {
-	file := parseFileConfig(t, `version: 4
-pipelines:
-  session:
-    input: seriatim
-    steps:
-      - id: identify-npcs
-        artifacts:
-          npc_registry:
-            extract: dnd/npc-registry
-      - id: grounded-events
-        references:
-          npc_registry:
-            artifact:
-              step: identify-npcs
-              lane: npc_registry
-        artifacts:
-          spells:
-            extract: dnd/spells
-`)
-	cfg := Default()
-	if err := cfg.ApplyFileConfig(file); err != nil {
-		t.Fatal(err)
-	}
-	profile := cfg.Pipelines["session"]
-	if len(profile.Steps) != 2 || profile.Steps[0].ID != "identify-npcs" || profile.Steps[1].ID != "grounded-events" {
-		t.Fatalf("steps = %#v", profile.Steps)
-	}
-	source := profile.Steps[1].References["npc_registry"]
-	if source.Artifact == nil || source.Artifact.Step != "identify-npcs" || source.Artifact.Lane != "npc_registry" {
-		t.Fatalf("generated source = %#v", source)
-	}
-}
-
-func TestFileConfigRejectsAmbiguousReferenceSourceForms(t *testing.T) {
-	for _, source := range []string{
-		"artifact: {step: a, lane: b, extra: c}",
-		"artifact: {step: 1, lane: b}",
-		"1",
-	} {
-		_, err := ParseFileConfigYAML([]byte("version: 4\npipelines:\n  p:\n    input: text\n    references:\n      slot: " + source + "\n"))
-		if err == nil {
-			t.Fatalf("ParseFileConfigYAML(%q) error = nil", source)
-		}
-	}
-}
-
-func TestFileConfigRejectsEmptyAndAmbiguousPipelineShapes(t *testing.T) {
-	tests := []struct {
-		name string
-		yaml string
-		want string
-	}{
-		{
-			name: "empty steps",
-			yaml: "version: 4\npipelines:\n  p:\n    input: text\n    steps: []\n",
-			want: "at least one ordered step",
-		},
-		{
-			name: "both forms",
-			yaml: "version: 4\npipelines:\n  p:\n    input: text\n    artifacts: {}\n    steps: []\n",
-			want: "both artifacts and steps",
-		},
-	}
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			file := parseFileConfig(t, tt.yaml)
-			cfg := Default()
-			err := cfg.ApplyFileConfig(file)
-			if err == nil || !strings.Contains(err.Error(), tt.want) {
-				t.Fatalf("ApplyFileConfig() error = %v, want context %q", err, tt.want)
-			}
-		})
-	}
-}
-
-func TestLoadFileConfigReportsPathAndOperationContext(t *testing.T) {
-	dir := t.TempDir()
-	missing := filepath.Join(dir, "missing.yml")
-	_, err := LoadFileConfig(missing)
-	if err == nil || !strings.Contains(err.Error(), "read config file") || !strings.Contains(err.Error(), missing) {
-		t.Fatalf("missing-file error = %v", err)
-	}
-
-	malformed := filepath.Join(dir, "malformed.yml")
-	if err := os.WriteFile(malformed, []byte("version: [\n"), 0o600); err != nil {
-		t.Fatal(err)
-	}
-	_, err = LoadFileConfig(malformed)
-	if err == nil || !strings.Contains(err.Error(), "parse config file") || !strings.Contains(err.Error(), malformed) {
-		t.Fatalf("malformed-file error = %v", err)
-	}
-}
-
-func parseFileConfig(t *testing.T, source string) FileConfig {
-	t.Helper()
-	file, err := ParseFileConfigYAML([]byte(source))
-	if err != nil {
-		t.Fatalf("ParseFileConfigYAML() error = %v", err)
-	}
-	return file
-}
-
-func applyFileConfig(t *testing.T, source string) Config {
-	t.Helper()
-	cfg := Default()
-	if err := cfg.ApplyFileConfig(parseFileConfig(t, source)); err != nil {
-		t.Fatalf("ApplyFileConfig() error = %v", err)
-	}
-	return cfg
-}

internal/core/config/load.go

@@ -0,0 +1,101 @@
+package config
+
+import (
+	"path/filepath"
+
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
+)
+
+// LoadOptions contains configuration selections supplied by an application
+// boundary. It is intentionally narrow: configuration discovery and CLI flag
+// parsing remain outside this package.
+type LoadOptions struct {
+	Profile             string
+	Overrides           []configkit.Override
+	InvocationDirectory string
+}
+
+// loadDocument composes and validates a version 5 file with ConfigKit before
+// runtime conversion, retaining source provenance and frozen inspection output.
+func loadDocument(path string, options LoadOptions) (*configkit.Result[Document], error) {
+	return loadDocumentWithSource(configkit.File(path), options, true)
+}
+
+func loadDocumentSource(source configkit.Source, options LoadOptions) (*configkit.Result[Document], error) {
+	return loadDocumentWithSource(source, options, false)
+}
+
+func loadDocumentWithSource(source configkit.Source, options LoadOptions, imports bool) (*configkit.Result[Document], error) {
+	state := newDocumentLoadState(options.InvocationDirectory)
+	return configkit.Load(source, configkit.Options[Document]{
+		Definition: documentDefinition(state),
+		Imports:    imports,
+		Profile:    options.Profile,
+		Overrides:  options.Overrides,
+	})
+}
+
+func cleanInvocationDirectory(value string) string {
+	if value == "" {
+		return ""
+	}
+	absolute, err := filepath.Abs(value)
+	if err != nil {
+		return ""
+	}
+	return filepath.Clean(absolute)
+}
+
+func documentDefinition(state *documentLoadState) configkit.Definition[Document] {
+	return configkit.Definition[Document]{
+		Defaults: documentDefaults,
+		Resolve: func(document *Document, resolution *configkit.Resolution) error {
+			return resolveDocumentOperations(document, resolution, state)
+		},
+		Validate: validateDocumentOperations,
+		Version: &configkit.VersionRule{
+			Path:  configkit.Keys("version"),
+			Check: validateDocumentVersion,
+		},
+		Secret: sensitiveDocumentPath,
+	}
+}
+
+func documentDefaults() Document {
+	return Document{
+		Pipelines: map[string]PipelineDocument{},
+		Concurrency: &ConcurrencyDocument{
+			TotalLLM:     documentValue(defaultLLMConcurrency),
+			StageWorkers: map[string]int{},
+		},
+		Output: &OutputDocument{Directory: documentValue("./notarius-output")},
+		Cache: &CacheDocument{
+			ChunkPlans: &ChunkPlanCacheDocument{
+				Directory: documentValue(""),
+				Mode:      documentValue("auto"),
+			},
+			Checkpoints: &CheckpointCacheDocument{
+				Enabled:   documentValue(false),
+				Directory: documentValue(""),
+			},
+		},
+		Debug: &DebugDocument{Directory: documentValue("./notarius-debug")},
+	}
+}
+
+func documentValue[T any](value T) *T { return &value }
+
+func validateDocumentVersion(version configkit.Version, validation *configkit.Validation) error {
+	if !version.Present || version.Null {
+		validation.Error(configkit.Keys("version"), "configuration version is required")
+		return nil
+	}
+	if version.Text == "4" {
+		validation.Error(configkit.Keys("version"), "configuration version 4 is no longer supported; migrate to version 5")
+		return nil
+	}
+	if version.Text != "5" {
+		validation.Error(configkit.Keys("version"), "unsupported configuration version")
+	}
+	return nil
+}

internal/core/config/loaded_config.go

@@ -0,0 +1,323 @@
+package config
+
+import (
+	"fmt"
+	"io"
+
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
+	"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
+)
+
+// LoadedConfig keeps the finalized runtime configuration together with the
+// private ConfigKit inspection snapshot that produced it.
+type LoadedConfig struct {
+	Config Config
+
+	result *configkit.Result[Document]
+}
+
+// RuntimeConfig returns an independently owned runtime configuration.
+func (loaded *LoadedConfig) RuntimeConfig() Config {
+	if loaded == nil {
+		return Config{}
+	}
+	return cloneConfig(loaded.Config)
+}
+
+// WriteEffective renders ConfigKit's frozen, redacted effective document.
+func (loaded *LoadedConfig) WriteEffective(output io.Writer) error {
+	if loaded == nil || loaded.result == nil {
+		return fmt.Errorf("loaded configuration is unavailable")
+	}
+	return loaded.result.WriteEffective(output)
+}
+
+// WriteExplanation renders ConfigKit's safe explanation for a JSON-pointer
+// path without exposing ConfigKit types at this package boundary.
+func (loaded *LoadedConfig) WriteExplanation(output io.Writer, pointer string) error {
+	if loaded == nil || loaded.result == nil {
+		return fmt.Errorf("loaded configuration is unavailable")
+	}
+	path, err := configkit.ParsePath(pointer)
+	if err != nil {
+		return fmt.Errorf("parse configuration explanation path: %w", err)
+	}
+	return loaded.result.WriteExplanation(output, path)
+}
+
+func loadConfig(path string, options LoadOptions) (*LoadedConfig, error) {
+	result, err := loadDocument(path, options)
+	if err != nil {
+		return nil, err
+	}
+	return newLoadedConfig(result)
+}
+
+// Load reads a version 5 configuration file into runtime configuration and a
+// private inspection snapshot.
+func Load(path string, options LoadOptions) (*LoadedConfig, error) {
+	return loadConfig(path, options)
+}
+
+func loadConfigSource(source configkit.Source, options LoadOptions) (*LoadedConfig, error) {
+	result, err := loadDocumentSource(source, options)
+	if err != nil {
+		return nil, err
+	}
+	return newLoadedConfig(result)
+}
+
+func newLoadedConfig(result *configkit.Result[Document]) (*LoadedConfig, error) {
+	if result == nil {
+		return nil, fmt.Errorf("configuration result is unavailable")
+	}
+	runtime, err := documentToConfig(result.Config)
+	if err != nil {
+		return nil, err
+	}
+	return &LoadedConfig{Config: runtime, result: result}, nil
+}
+
+func documentToConfig(document Document) (Config, error) {
+	config := Default()
+	config.Pipelines = documentPipelinesToRuntime(document.Pipelines)
+	if document.Concurrency != nil {
+		config.Concurrency.StageWorkers = documentIntMap(document.Concurrency.StageWorkers)
+		if document.Concurrency.TotalLLM != nil {
+			config.Concurrency.TotalLLM = *document.Concurrency.TotalLLM
+		}
+	}
+	if document.PromptKit != nil {
+		if document.PromptKit.ProfileDir != nil {
+			config.PromptKit.ProfileDir = *document.PromptKit.ProfileDir
+		}
+		if document.PromptKit.ProfileFile != nil {
+			config.PromptKit.ProfileFile = *document.PromptKit.ProfileFile
+		}
+		if document.PromptKit.LocalBackend != nil {
+			backend := PromptKitLocalBackendConfig{}
+			if document.PromptKit.LocalBackend.Endpoint != nil {
+				backend.Endpoint = *document.PromptKit.LocalBackend.Endpoint
+			}
+			if document.PromptKit.LocalBackend.ConcurrencyLimit != nil {
+				backend.ConcurrencyLimit = *document.PromptKit.LocalBackend.ConcurrencyLimit
+			}
+			config.PromptKit.LocalBackend = &backend
+		}
+	}
+	if document.Output != nil && document.Output.Directory != nil {
+		config.Output.Directory = *document.Output.Directory
+	}
+	if document.Cache != nil {
+		if document.Cache.ChunkPlans != nil {
+			if document.Cache.ChunkPlans.Directory != nil {
+				config.Cache.ChunkPlans.Directory = *document.Cache.ChunkPlans.Directory
+			}
+			if document.Cache.ChunkPlans.Mode != nil {
+				mode, err := pipeline.ParseChunkCacheMode(*document.Cache.ChunkPlans.Mode)
+				if err != nil {
+					return Config{}, fmt.Errorf("convert chunk-plan cache mode: %w", err)
+				}
+				config.Cache.ChunkPlans.Mode = mode
+			}
+		}
+		if document.Cache.Checkpoints != nil {
+			if document.Cache.Checkpoints.Enabled != nil {
+				config.Cache.Checkpoints.Enabled = *document.Cache.Checkpoints.Enabled
+			}
+			if document.Cache.Checkpoints.Directory != nil {
+				config.Cache.Checkpoints.Directory = *document.Cache.Checkpoints.Directory
+			}
+		}
+	}
+	if document.Debug != nil && document.Debug.Directory != nil {
+		config.Debug.Directory = *document.Debug.Directory
+	}
+	return config, nil
+}
+
+func documentPipelinesToRuntime(documents map[string]PipelineDocument) map[string]pipeline.PipelineProfile {
+	if documents == nil {
+		return nil
+	}
+	profiles := make(map[string]pipeline.PipelineProfile, len(documents))
+	for id, document := range documents {
+		profiles[id] = documentPipelineToRuntime(id, document)
+	}
+	return profiles
+}
+
+func documentPipelineToRuntime(id string, document PipelineDocument) pipeline.PipelineProfile {
+	profile := pipeline.PipelineProfile{
+		ID:                             id,
+		StructuredOutputRepairAttempts: cloneDocumentInt(document.StructuredOutputRepairAttempts),
+		ValidationPolicy:               documentPolicyToRuntime(document.ValidationPolicy),
+		Input:                          documentBindingToRuntime(document.Input),
+		Artifacts:                      documentLanesToRuntime(document.Artifacts),
+		References:                     documentReferencesToRuntime(document.References),
+	}
+	if document.LLMProfile != nil {
+		profile.LLMProfile = *document.LLMProfile
+	}
+	if document.Chunk != nil {
+		profile.Chunk = documentBindingToRuntime(*document.Chunk)
+	}
+	if document.Output != nil {
+		profile.Output = documentBindingToRuntime(*document.Output)
+	}
+	if document.Steps != nil {
+		profile.Steps = make([]pipeline.PipelineStepProfile, len(document.Steps))
+		for index, step := range document.Steps {
+			profile.Steps[index] = pipeline.PipelineStepProfile{
+				ID:         step.ID,
+				Artifacts:  documentLanesToRuntime(step.Artifacts),
+				References: documentReferencesToRuntime(step.References),
+			}
+		}
+	}
+	return profile
+}
+
+func documentLanesToRuntime(documents map[string]ArtifactLaneDocument) map[string]pipeline.ArtifactLaneProfile {
+	if documents == nil {
+		return nil
+	}
+	lanes := make(map[string]pipeline.ArtifactLaneProfile, len(documents))
+	for id, document := range documents {
+		lane := pipeline.ArtifactLaneProfile{
+			Extract:    documentBindingToRuntime(document.Extract),
+			References: documentReferencesToRuntime(document.References),
+		}
+		if document.Merge != nil {
+			lane.Merge = documentBindingToRuntime(*document.Merge)
+		}
+		if document.Normalize != nil {
+			lane.Normalize = documentBindingToRuntime(*document.Normalize)
+		}
+		if document.Validators != nil {
+			lane.Validators = make([]pipeline.ModuleBinding, len(document.Validators))
+			for index, validator := range document.Validators {
+				lane.Validators[index] = documentValidatorToRuntime(validator)
+			}
+		}
+		lanes[id] = lane
+	}
+	return lanes
+}
+
+func documentBindingToRuntime(document BindingDocument) pipeline.ModuleBinding {
+	binding := pipeline.ModuleBinding{
+		Module:                         document.Module,
+		StructuredOutputRepairAttempts: cloneDocumentInt(document.StructuredOutputRepairAttempts),
+		ValidationPolicy:               documentPolicyToRuntime(document.ValidationPolicy),
+		Retries:                        document.Retries,
+		Options:                        documentOptions(document.Options),
+		References:                     documentReferencesToRuntime(document.References),
+	}
+	if document.LLMProfile != nil {
+		binding.LLMProfile = *document.LLMProfile
+	}
+	if document.Validators != nil {
+		binding.Validators.Set = true
+		binding.Validators.Validators = make([]pipeline.ModuleBinding, len(document.Validators))
+		for index, validator := range document.Validators {
+			binding.Validators.Validators[index] = documentValidatorToRuntime(validator)
+		}
+	}
+	return binding
+}
+
+func documentValidatorToRuntime(document ValidatorDocument) pipeline.ModuleBinding {
+	binding := pipeline.ModuleBinding{
+		Module:                         document.Module,
+		StructuredOutputRepairAttempts: cloneDocumentInt(document.StructuredOutputRepairAttempts),
+		Retries:                        document.Retries,
+		Options:                        documentOptions(document.Options),
+	}
+	if document.LLMProfile != nil {
+		binding.LLMProfile = *document.LLMProfile
+	}
+	return binding
+}
+
+func documentReferencesToRuntime(documents map[string]ReferenceDocument) map[string]pipeline.ReferenceSource {
+	if documents == nil {
+		return nil
+	}
+	references := make(map[string]pipeline.ReferenceSource, len(documents))
+	for name, document := range documents {
+		if document.Artifact != nil {
+			references[name] = pipeline.GeneratedReference(document.Artifact.Step, document.Artifact.Lane)
+			continue
+		}
+		if document.Path != nil {
+			references[name] = pipeline.ExternalReference(*document.Path)
+		}
+	}
+	return references
+}
+
+func documentPolicyToRuntime(document *ValidationPolicyDocument) *pipeline.ValidationPolicyOverride {
+	if document == nil {
+		return nil
+	}
+	policy := pipeline.ValidationPolicyOverride{}
+	if document.ProducerStructuralFailure != nil {
+		value := *document.ProducerStructuralFailure
+		policy.ProducerStructuralFailure = &value
+	}
+	if document.SemanticRejection != nil {
+		value := *document.SemanticRejection
+		policy.SemanticRejection = &value
+	}
+	if document.ValidatorFailure != nil {
+		value := *document.ValidatorFailure
+		policy.ValidatorFailure = &value
+	}
+	return &policy
+}
+
+func documentIntMap(values map[string]int) map[string]int {
+	if values == nil {
+		return nil
+	}
+	result := make(map[string]int, len(values))
+	for key, value := range values {
+		result[key] = value
+	}
+	return result
+}
+
+func cloneDocumentInt(value *int) *int {
+	if value == nil {
+		return nil
+	}
+	result := *value
+	return &result
+}
+
+func documentOptions(values map[string]any) map[string]any {
+	if values == nil {
+		return nil
+	}
+	result := make(map[string]any, len(values))
+	for key, value := range values {
+		result[key] = cloneDocumentOption(value)
+	}
+	return result
+}
+
+func cloneDocumentOption(value any) any {
+	switch typed := value.(type) {
+	case map[string]any:
+		return documentOptions(typed)
+	case []any:
+		result := make([]any, len(typed))
+		for index, item := range typed {
+			result[index] = cloneDocumentOption(item)
+		}
+		return result
+	default:
+		return typed
+	}
+}

internal/core/config/loaded_config_test.go

@@ -0,0 +1,184 @@
+package config
+
+import (
+	"bytes"
+	"strings"
+	"testing"
+
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
+	"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
+	units "gitea.maximumdirect.net/eric/notarius/internal/modules/generic/chunk/units"
+	jsonoutput "gitea.maximumdirect.net/eric/notarius/internal/modules/generic/output/json"
+)
+
+func TestLoadedConfigConvertsDocumentsAndResolvesRuntimePipeline(t *testing.T) {
+	t.Parallel()
+
+	loaded, err := loadConfigSource(configkit.Bytes("document.yml", []byte(`version: 5
+pipelines:
+  session:
+    llm_profile: local
+    structured_output_repair_attempts: 0
+    input:
+      module: input
+    chunk:
+      module: generic
+      validators: []
+    artifacts:
+      score:
+        extract:
+          module: extract
+          validators:
+            - module: configured-a
+              options:
+                nested:
+                  level: 1
+`)), LoadOptions{})
+	if err != nil {
+		t.Fatalf("loadConfigSource() error = %v", err)
+	}
+	profile := loaded.Config.Pipelines["session"]
+	if profile.LLMProfile != "local" || profile.StructuredOutputRepairAttempts == nil || *profile.StructuredOutputRepairAttempts != 0 {
+		t.Fatalf("pipeline profile = %#v", profile)
+	}
+	if !profile.Chunk.Validators.Set || profile.Chunk.Validators.Validators == nil || len(profile.Chunk.Validators.Validators) != 0 {
+		t.Fatalf("empty validator chain was not preserved: %#v", profile.Chunk.Validators)
+	}
+	validator := profile.Artifacts["score"].Extract.Validators.Validators[0]
+	if validator.Module != "configured-a" || validator.Options["nested"].(map[string]any)["level"] == nil {
+		t.Fatalf("validator conversion = %#v", validator)
+	}
+
+	effective, err := loaded.Config.Resolve(ResolveInput{PipelineID: "session", Catalog: effectiveCatalog(t)})
+	if err != nil {
+		t.Fatalf("Resolve() error = %v", err)
+	}
+	if effective.ResolvedPipeline.ID != "session" || effective.ResolvedPipeline.Digest == "" {
+		t.Fatalf("resolved pipeline = %#v", effective.ResolvedPipeline)
+	}
+}
+
+func TestLoadedConfigOwnsRuntimeValuesAndKeepsFrozenInspection(t *testing.T) {
+	t.Parallel()
+
+	loaded, err := loadConfigSource(configkit.Bytes("document.yml", []byte(`version: 5
+pipelines:
+  session:
+    input: {module: input}
+    artifacts:
+      score:
+        extract:
+          module: extract
+          options:
+            nested: {value: original}
+`)), LoadOptions{})
+	if err != nil {
+		t.Fatalf("loadConfigSource() error = %v", err)
+	}
+
+	first := loaded.RuntimeConfig()
+	first.Pipelines["session"] = pipelineProfileWithChangedOption(first.Pipelines["session"])
+	second := loaded.RuntimeConfig()
+	if got := second.Pipelines["session"].Artifacts["score"].Extract.Options["nested"].(map[string]any)["value"]; got != "original" {
+		t.Fatalf("runtime configuration aliases caller mutation: %#v", got)
+	}
+
+	loaded.Config.Pipelines["session"] = pipelineProfileWithChangedOption(loaded.Config.Pipelines["session"])
+	var effective bytes.Buffer
+	if err := loaded.WriteEffective(&effective); err != nil {
+		t.Fatalf("WriteEffective() error = %v", err)
+	}
+	if !strings.Contains(effective.String(), "original") || strings.Contains(effective.String(), "changed") {
+		t.Fatalf("effective document changed with runtime mutation: %q", effective.String())
+	}
+}
+
+func TestLoadedConfigPreservesConfigKitIntegerOptionsForModules(t *testing.T) {
+	t.Parallel()
+
+	loaded, err := loadConfigSource(configkit.Bytes("document.yml", []byte(`version: 5
+pipelines:
+  session:
+    input: {module: input}
+    chunk:
+      module: generic
+      options:
+        max_units: 3
+        overlap_units: 1
+    output:
+      module: json
+      options:
+        evidence_context:
+          enabled: true
+          lanes: [score]
+          window_units: 2
+`)), LoadOptions{})
+	if err != nil {
+		t.Fatalf("loadConfigSource() error = %v", err)
+	}
+	profile := loaded.Config.Pipelines["session"]
+	chunkOptions, err := units.DecodeOptions(profile.Chunk.Options)
+	if err != nil || chunkOptions.MaxUnits != 3 || chunkOptions.OverlapUnits != 1 {
+		t.Fatalf("chunk options = %#v, %v", chunkOptions, err)
+	}
+	outputOptions, err := jsonoutput.DecodeOptions(profile.Output.Options)
+	if err != nil || !outputOptions.EvidenceContext.Enabled || outputOptions.EvidenceContext.WindowUnits != 2 {
+		t.Fatalf("output options = %#v, %v", outputOptions, err)
+	}
+}
+
+func TestLoadedConfigRedactsSensitiveNestedOptions(t *testing.T) {
+	t.Parallel()
+
+	loaded, err := loadConfigSource(configkit.Bytes("document.yml", []byte(`version: 5
+pipelines:
+  session:
+    input:
+      module: input
+      options:
+        nested:
+          api_key: input-secret
+    artifacts:
+      score:
+        extract:
+          module: extract
+          validators:
+            - module: configured-a
+              options:
+                credentials:
+                  token: validator-secret
+`)), LoadOptions{})
+	if err != nil {
+		t.Fatalf("loadConfigSource() error = %v", err)
+	}
+	var effective bytes.Buffer
+	if err := loaded.WriteEffective(&effective); err != nil {
+		t.Fatalf("WriteEffective() error = %v", err)
+	}
+	for _, secret := range []string{"input-secret", "validator-secret"} {
+		if strings.Contains(effective.String(), secret) {
+			t.Fatalf("effective configuration exposes %q: %q", secret, effective.String())
+		}
+	}
+	if !strings.Contains(effective.String(), "[REDACTED]") {
+		t.Fatalf("effective configuration did not mark sensitive values: %q", effective.String())
+	}
+
+	var explanation bytes.Buffer
+	if err := loaded.WriteExplanation(&explanation, "/pipelines/session/artifacts/score/extract/validators/0/options/credentials/token"); err != nil {
+		t.Fatalf("WriteExplanation() error = %v", err)
+	}
+	if strings.Contains(explanation.String(), "validator-secret") || !strings.Contains(explanation.String(), "[REDACTED]") {
+		t.Fatalf("explanation exposes sensitive value: %q", explanation.String())
+	}
+}
+
+func pipelineProfileWithChangedOption(profile pipeline.PipelineProfile) pipeline.PipelineProfile {
+	profile = clonePipelineProfile(profile)
+	nested := profile.Artifacts["score"].Extract.Options["nested"].(map[string]any)
+	nested["value"] = "changed"
+	lane := profile.Artifacts["score"]
+	lane.Extract.Options["nested"] = nested
+	profile.Artifacts["score"] = lane
+	return profile
+}

internal/core/config/operational_document.go

@@ -0,0 +1,305 @@
+package config
+
+import (
+	"fmt"
+	"net/url"
+	"sort"
+	"strings"
+
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
+	"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
+)
+
+func resolveDocumentOperations(document *Document, resolution *configkit.Resolution, state *documentLoadState) error {
+	if document == nil {
+		return nil
+	}
+	normalizeDocumentPromptKit(document.PromptKit, resolution)
+	normalizeDocumentState(document, resolution)
+	normalizeDocumentPipelines(document, resolution, state)
+	normalizeDocumentPaths(document, resolution, state)
+	if document.Concurrency != nil {
+		document.Concurrency.StageWorkers = normalizeDocumentMap(document.Concurrency.StageWorkers,
+			configkit.Keys("concurrency", "stage_workers"), "stage worker identifier",
+			documentNormalizer{Resolution: resolution}, state)
+	}
+	deriveDocumentExtractWorkers(document, resolution, state)
+	return nil
+}
+
+func normalizeDocumentPromptKit(promptKit *PromptKitDocument, resolution *configkit.Resolution) {
+	if promptKit == nil {
+		return
+	}
+	profileDirPath := configkit.Keys("promptkit", "profile_dir")
+	profileFilePath := configkit.Keys("promptkit", "profile_file")
+	if promptKit.ProfileDir == nil && resolution.IsExplicit(profileDirPath) {
+		resolution.Error(profileDirPath, "must not be null")
+	}
+	if promptKit.ProfileFile == nil && resolution.IsExplicit(profileFilePath) {
+		resolution.Error(profileFilePath, "must not be null")
+	}
+	normalizeDocumentString(resolution, profileDirPath, &promptKit.ProfileDir, "trimmed PromptKit profile directory")
+	normalizeDocumentString(resolution, profileFilePath, &promptKit.ProfileFile, "trimmed PromptKit profile file")
+
+	if promptKit.LocalBackend == nil {
+		return
+	}
+	endpointPath := configkit.Keys("promptkit", "local_backend", "endpoint")
+	limitPath := configkit.Keys("promptkit", "local_backend", "concurrency_limit")
+	normalizeDocumentString(resolution, endpointPath, &promptKit.LocalBackend.Endpoint, "trimmed PromptKit local backend endpoint")
+	if promptKit.LocalBackend.ConcurrencyLimit == nil && !resolution.IsExplicit(limitPath) {
+		configkit.Default(resolution, limitPath, &promptKit.LocalBackend.ConcurrencyLimit,
+			documentValue(0), "defaulted PromptKit local backend concurrency limit")
+	}
+}
+
+func normalizeDocumentState(document *Document, resolution *configkit.Resolution) {
+	if document.Output != nil {
+		normalizeDocumentString(resolution, configkit.Keys("output", "directory"), &document.Output.Directory, "trimmed output directory")
+	}
+	if document.Cache != nil {
+		if document.Cache.ChunkPlans != nil {
+			normalizeDocumentString(resolution, configkit.Keys("cache", "chunk_plans", "directory"), &document.Cache.ChunkPlans.Directory, "trimmed chunk-plan cache directory")
+			normalizeDocumentString(resolution, configkit.Keys("cache", "chunk_plans", "mode"), &document.Cache.ChunkPlans.Mode, "trimmed chunk-plan cache mode")
+		}
+		if document.Cache.Checkpoints != nil {
+			normalizeDocumentString(resolution, configkit.Keys("cache", "checkpoints", "directory"), &document.Cache.Checkpoints.Directory, "trimmed checkpoint cache directory")
+		}
+	}
+	if document.Debug != nil {
+		normalizeDocumentString(resolution, configkit.Keys("debug", "directory"), &document.Debug.Directory, "trimmed debug directory")
+	}
+}
+
+func normalizeDocumentString(resolution *configkit.Resolution, path configkit.Path, target **string, reason string) {
+	if target == nil || *target == nil || !resolution.IsExplicit(path) {
+		return
+	}
+	configkit.Normalize(resolution, path, target, documentValue(strings.TrimSpace(**target)), reason)
+}
+
+func deriveDocumentExtractWorkers(document *Document, resolution *configkit.Resolution, state *documentLoadState) {
+	if document.Concurrency == nil || document.Concurrency.TotalLLM == nil {
+		return
+	}
+	if document.Concurrency.StageWorkers == nil {
+		document.Concurrency.StageWorkers = make(map[string]int)
+	}
+	path := configkit.Keys("concurrency", "stage_workers", "extract")
+	// Presence belongs to the input spelling, not the key produced by trimming.
+	// A moved explicit count must never be replaced by a derived default.
+	if resolution.IsExplicit(state.originalPath(path)) {
+		return
+	}
+	workers := document.Concurrency.StageWorkers
+	value := workers["extract"]
+	configkit.Default(resolution, path, &value, *document.Concurrency.TotalLLM,
+		"derived extraction workers from total LLM concurrency", configkit.Keys("concurrency", "total_llm"))
+	workers["extract"] = value
+}
+
+func validateDocumentOperations(document *Document, validation *configkit.Validation) {
+	if document == nil {
+		return
+	}
+	validateDocumentConcurrency(document.Concurrency, validation)
+	validateDocumentPromptKit(document.PromptKit, validation)
+	validateDocumentState(document, validation)
+	validateDocumentPipelines(document, validation)
+}
+
+func validateDocumentConcurrency(concurrency *ConcurrencyDocument, validation *configkit.Validation) {
+	if concurrency == nil {
+		validation.Error(configkit.Keys("concurrency"), "must not be null")
+		return
+	}
+	totalPath := configkit.Keys("concurrency", "total_llm")
+	if concurrency.TotalLLM == nil {
+		validation.Error(totalPath, "must not be null")
+		return
+	}
+	totalValid := true
+	if err := validateTotalLLMConcurrency(*concurrency.TotalLLM); err != nil {
+		validation.Error(totalPath, err.Error())
+		totalValid = false
+	}
+	keys := make([]string, 0, len(concurrency.StageWorkers))
+	for key := range concurrency.StageWorkers {
+		keys = append(keys, key)
+	}
+	sort.Strings(keys)
+	for _, key := range keys {
+		if err := validateStageWorkerKey(key); err != nil {
+			validation.Error(configkit.Keys("concurrency", "stage_workers").Key(key), err.Error())
+		}
+	}
+	if !totalValid {
+		return
+	}
+	if workers, ok := concurrency.StageWorkers["extract"]; ok {
+		if err := validateExtractWorkerCount(workers, *concurrency.TotalLLM); err != nil {
+			validation.Error(configkit.Keys("concurrency", "stage_workers", "extract"), err.Error())
+		}
+	}
+}
+
+func validateDocumentPromptKit(promptKit *PromptKitDocument, validation *configkit.Validation) {
+	if promptKit == nil {
+		return
+	}
+	profileDirPath := configkit.Keys("promptkit", "profile_dir")
+	profileFilePath := configkit.Keys("promptkit", "profile_file")
+	profileDir := ""
+	if promptKit.ProfileDir != nil {
+		profileDir = *promptKit.ProfileDir
+		if strings.TrimSpace(profileDir) == "" {
+			validation.Error(profileDirPath, "must not be empty when set")
+		}
+	}
+	profileFile := ""
+	if promptKit.ProfileFile != nil {
+		profileFile = *promptKit.ProfileFile
+		if strings.TrimSpace(profileFile) == "" {
+			validation.Error(profileFilePath, "must not be empty when set")
+		}
+	}
+	if err := validatePromptKitProfileSources(profileDir, profileFile); err != nil {
+		validation.Error(configkit.Keys("promptkit"), err.Error())
+	}
+	if promptKit.LocalBackend == nil {
+		return
+	}
+	endpointPath := configkit.Keys("promptkit", "local_backend", "endpoint")
+	limitPath := configkit.Keys("promptkit", "local_backend", "concurrency_limit")
+	if promptKit.LocalBackend.Endpoint == nil {
+		validation.Error(endpointPath, "must be nonempty when local backend is configured")
+	}
+	if promptKit.LocalBackend.ConcurrencyLimit == nil {
+		validation.Error(limitPath, "must be an integer when local backend is configured")
+	}
+	if promptKit.LocalBackend.Endpoint != nil {
+		if err := validatePromptKitLocalBackendEndpoint(*promptKit.LocalBackend.Endpoint); err != nil {
+			validation.Error(endpointPath, err.Error())
+		}
+	}
+	if promptKit.LocalBackend.ConcurrencyLimit != nil {
+		if err := validatePromptKitLocalBackendConcurrencyLimit(*promptKit.LocalBackend.ConcurrencyLimit); err != nil {
+			validation.Error(limitPath, err.Error())
+		}
+	}
+}
+
+func validateDocumentState(document *Document, validation *configkit.Validation) {
+	if document.Output == nil || document.Output.Directory == nil {
+		validation.Error(configkit.Keys("output", "directory"), "must not be empty")
+	} else if err := validateRequiredPath("output.directory", *document.Output.Directory); err != nil {
+		validation.Error(configkit.Keys("output", "directory"), err.Error())
+	}
+
+	if document.Cache == nil || document.Cache.ChunkPlans == nil || document.Cache.ChunkPlans.Mode == nil {
+		validation.Error(configkit.Keys("cache", "chunk_plans", "mode"), "must be a supported cache mode")
+	} else if err := validateChunkPlanCacheMode(*document.Cache.ChunkPlans.Mode); err != nil {
+		validation.Error(configkit.Keys("cache", "chunk_plans", "mode"), err.Error())
+	}
+	if document.Cache != nil && document.Cache.ChunkPlans != nil && document.Cache.ChunkPlans.Directory != nil {
+		if err := validateOptionalPath("cache.chunk_plans.directory", *document.Cache.ChunkPlans.Directory); err != nil {
+			validation.Error(configkit.Keys("cache", "chunk_plans", "directory"), err.Error())
+		}
+	}
+	if document.Cache != nil && document.Cache.Checkpoints != nil && document.Cache.Checkpoints.Directory != nil {
+		if err := validateOptionalPath("cache.checkpoints.directory", *document.Cache.Checkpoints.Directory); err != nil {
+			validation.Error(configkit.Keys("cache", "checkpoints", "directory"), err.Error())
+		}
+	}
+
+	if document.Debug == nil || document.Debug.Directory == nil {
+		validation.Error(configkit.Keys("debug", "directory"), "must not be empty")
+	} else if err := validateRequiredPath("debug.directory", *document.Debug.Directory); err != nil {
+		validation.Error(configkit.Keys("debug", "directory"), err.Error())
+	}
+}
+
+func validateTotalLLMConcurrency(total int) error {
+	if total <= 0 {
+		return fmt.Errorf("total LLM concurrency must be greater than zero")
+	}
+	return nil
+}
+
+func validateStageWorkerKey(key string) error {
+	if strings.TrimSpace(key) == "" {
+		return fmt.Errorf("concurrency.stage_workers key must not be empty")
+	}
+	if key != "extract" {
+		return fmt.Errorf("concurrency.stage_workers key %q is not supported", key)
+	}
+	return nil
+}
+
+func validateExtractWorkerCount(workers, total int) error {
+	if workers < 1 || workers > total {
+		return fmt.Errorf("concurrency.stage_workers.extract must be between 1 and concurrency.total_llm (%d)", total)
+	}
+	return nil
+}
+
+func validatePromptKitProfileSources(profileDir, profileFile string) error {
+	if strings.TrimSpace(profileDir) != "" && strings.TrimSpace(profileFile) != "" {
+		return fmt.Errorf("promptkit profile_dir and profile_file are mutually exclusive")
+	}
+	return nil
+}
+
+func validatePromptKitLocalBackend(endpoint string, concurrencyLimit int) error {
+	if err := validatePromptKitLocalBackendEndpoint(endpoint); err != nil {
+		return err
+	}
+	return validatePromptKitLocalBackendConcurrencyLimit(concurrencyLimit)
+}
+
+func validatePromptKitLocalBackendEndpoint(endpoint string) error {
+	if strings.TrimSpace(endpoint) == "" {
+		return fmt.Errorf("promptkit.local_backend.endpoint must not be empty when set")
+	}
+	parsed, err := url.Parse(strings.TrimSpace(endpoint))
+	if err != nil ||
+		(!strings.EqualFold(parsed.Scheme, "http") && !strings.EqualFold(parsed.Scheme, "https")) ||
+		!parsed.IsAbs() ||
+		parsed.Hostname() == "" ||
+		parsed.User != nil ||
+		parsed.RawQuery != "" ||
+		parsed.ForceQuery ||
+		strings.Contains(endpoint, "#") {
+		return fmt.Errorf("promptkit.local_backend.endpoint must be an absolute HTTP or HTTPS URL with a host and no user information, query, or fragment")
+	}
+	return nil
+}
+
+func validatePromptKitLocalBackendConcurrencyLimit(concurrencyLimit int) error {
+	if concurrencyLimit < 0 {
+		return fmt.Errorf("promptkit.local_backend.concurrency_limit must not be negative")
+	}
+	return nil
+}
+
+func validateRequiredPath(name, value string) error {
+	if strings.TrimSpace(value) == "" {
+		return fmt.Errorf("%s must not be empty", name)
+	}
+	return validateOptionalPath(name, value)
+}
+
+func validateOptionalPath(name, value string) error {
+	if strings.ContainsRune(value, '\x00') {
+		return fmt.Errorf("%s must not contain NUL", name)
+	}
+	return nil
+}
+
+func validateChunkPlanCacheMode(value string) error {
+	if _, err := pipeline.ParseChunkCacheMode(value); err != nil {
+		return fmt.Errorf("cache.chunk_plans.mode: %w", err)
+	}
+	return nil
+}

internal/core/config/operational_document_test.go

@@ -0,0 +1,228 @@
+package config
+
+import (
+	"bytes"
+	"errors"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
+)
+
+func TestWorkerKeyNormalizationPreservesExplicitCount(t *testing.T) {
+	t.Parallel()
+	for _, test := range []struct {
+		name, workers string
+		want          int
+		errorPath     string
+	}{
+		{name: "omitted", workers: "{}", want: 4},
+		{name: "padded", workers: "{' extract ': 2}", want: 2},
+		{name: "explicit invalid", workers: "{' extract ': 0}", errorPath: "/concurrency/stage_workers/extract"},
+		{name: "collision", workers: "{' extract ': 2, extract: 3}", errorPath: "/concurrency/stage_workers/extract"},
+	} {
+		t.Run(test.name, func(t *testing.T) {
+			loaded, err := loadConfigSource(configkit.Bytes("workers.yml", []byte(
+				"version: 5\nconcurrency:\n  total_llm: 4\n  stage_workers: "+test.workers+"\n")), LoadOptions{})
+			if test.errorPath != "" {
+				assertDocumentDiagnosticAt(t, err, test.errorPath)
+				return
+			}
+			if err != nil {
+				t.Fatal(err)
+			}
+			if got := loaded.Config.Concurrency.StageWorkers["extract"]; got != test.want {
+				t.Fatalf("workers = %d, want %d", got, test.want)
+			}
+			var explanation bytes.Buffer
+			if err := loaded.WriteExplanation(&explanation, "/concurrency/stage_workers"); err != nil {
+				t.Fatal(err)
+			}
+			if test.name == "padded" && (!strings.Contains(explanation.String(), "normalization") ||
+				!strings.Contains(explanation.String(), " extract ") ||
+				strings.Contains(explanation.String(), "derived extraction workers")) {
+				t.Fatalf("explicit worker provenance = %s", explanation.String())
+			}
+		})
+	}
+}
+
+func TestDocumentLoadAppliesOperationalDefaultsAndNormalization(t *testing.T) {
+	t.Parallel()
+
+	loaded, err := loadDocumentSource(configkit.Bytes("document.yml", []byte(`version: 5
+concurrency:
+  total_llm: 4
+promptkit:
+  profile_dir: " ./profiles "
+  local_backend:
+    endpoint: " https://localhost:8080/v1 "
+output:
+  directory: " ./output "
+cache:
+  chunk_plans:
+    mode: " refresh "
+    directory: " "
+  checkpoints:
+    directory: " "
+debug:
+  directory: " ./debug "
+`)), LoadOptions{})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if loaded.Config.Concurrency.TotalLLM == nil || *loaded.Config.Concurrency.TotalLLM != 4 || loaded.Config.Concurrency.StageWorkers["extract"] != 4 {
+		t.Fatalf("concurrency = %#v", loaded.Config.Concurrency)
+	}
+	workingDirectory, err := os.Getwd()
+	if err != nil {
+		t.Fatal(err)
+	}
+	if loaded.Config.PromptKit.ProfileDir == nil || *loaded.Config.PromptKit.ProfileDir != filepath.Join(workingDirectory, "profiles") ||
+		loaded.Config.PromptKit.LocalBackend == nil || loaded.Config.PromptKit.LocalBackend.Endpoint == nil || *loaded.Config.PromptKit.LocalBackend.Endpoint != "https://localhost:8080/v1" ||
+		loaded.Config.PromptKit.LocalBackend.ConcurrencyLimit == nil || *loaded.Config.PromptKit.LocalBackend.ConcurrencyLimit != 0 {
+		t.Fatalf("promptkit = %#v", loaded.Config.PromptKit)
+	}
+	if loaded.Config.Output.Directory == nil || *loaded.Config.Output.Directory != filepath.Join(workingDirectory, "output") ||
+		loaded.Config.Cache.ChunkPlans.Mode == nil || *loaded.Config.Cache.ChunkPlans.Mode != "refresh" ||
+		loaded.Config.Cache.ChunkPlans.Directory == nil || *loaded.Config.Cache.ChunkPlans.Directory != "" ||
+		loaded.Config.Cache.Checkpoints.Directory == nil || *loaded.Config.Cache.Checkpoints.Directory != "" ||
+		loaded.Config.Debug.Directory == nil || *loaded.Config.Debug.Directory != filepath.Join(workingDirectory, "debug") {
+		t.Fatalf("state = %#v, %#v, %#v", loaded.Config.Output, loaded.Config.Cache, loaded.Config.Debug)
+	}
+	if diagnostics := loaded.Diagnostics(); len(diagnostics) != 0 {
+		t.Fatalf("diagnostics = %#v", diagnostics)
+	}
+	extractHistory := loaded.Metadata().History(configkit.Keys("concurrency", "stage_workers", "extract"))
+	if len(extractHistory) == 0 || extractHistory[len(extractHistory)-1].Origin.Operation != configkit.ChangeDerivedDefault {
+		t.Fatalf("extract history = %#v", extractHistory)
+	}
+	outputHistory := loaded.Metadata().History(configkit.Keys("output", "directory"))
+	if len(outputHistory) == 0 || outputHistory[len(outputHistory)-1].Origin.Operation != configkit.ChangeNormalization {
+		t.Fatalf("output history = %#v", outputHistory)
+	}
+}
+
+func TestDocumentLoadDerivesWorkersAfterProfileAndCLIOverrides(t *testing.T) {
+	t.Parallel()
+
+	directory := t.TempDir()
+	main := filepath.Join(directory, "config.yml")
+	profiles := filepath.Join(directory, "profiles")
+	if err := os.Mkdir(profiles, 0o755); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(main, []byte("version: 5\nconcurrency:\n  total_llm: 4\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(filepath.Join(profiles, "development.yml"), []byte("concurrency:\n  total_llm: 3\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+
+	profiled, err := loadDocument(main, LoadOptions{Profile: "development"})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if profiled.Config.Concurrency.TotalLLM == nil || *profiled.Config.Concurrency.TotalLLM != 3 || profiled.Config.Concurrency.StageWorkers["extract"] != 3 {
+		t.Fatalf("profiled concurrency = %#v", profiled.Config.Concurrency)
+	}
+
+	overridden, err := loadDocument(main, LoadOptions{Overrides: []configkit.Override{{
+		Path: configkit.Keys("concurrency", "total_llm"), Value: 2, Flag: "--total-llm",
+	}}})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if overridden.Config.Concurrency.TotalLLM == nil || *overridden.Config.Concurrency.TotalLLM != 2 || overridden.Config.Concurrency.StageWorkers["extract"] != 2 {
+		t.Fatalf("overridden concurrency = %#v", overridden.Config.Concurrency)
+	}
+}
+
+func TestDocumentLoadRetainsExplicitWorkerAndRejectsInvalidOperationalValues(t *testing.T) {
+	t.Parallel()
+
+	loaded, err := loadDocumentSource(configkit.Bytes("explicit.yml", []byte(`version: 5
+concurrency:
+  total_llm: 4
+  stage_workers:
+    extract: 0
+`)), LoadOptions{})
+	if loaded != nil {
+		t.Fatalf("loaded = %#v, want validation failure", loaded.Config)
+	}
+	assertDocumentDiagnostic(t, err, configkit.CodeValidateError, "/concurrency/stage_workers/extract")
+
+	_, err = loadDocumentSource(configkit.Bytes("invalid.yml", []byte(`version: 5
+concurrency:
+  total_llm: 0
+output:
+  directory: ""
+promptkit:
+  local_backend:
+    endpoint: "ftp://localhost"
+    concurrency_limit: -1
+`)), LoadOptions{})
+	var loadError *configkit.LoadError
+	if !errors.As(err, &loadError) {
+		t.Fatalf("error = %v, want ConfigKit load error", err)
+	}
+	seen := make(map[string]bool)
+	for _, diagnostic := range loadError.Diagnostics() {
+		if diagnostic.Code != configkit.CodeValidateError || diagnostic.Primary == nil {
+			t.Fatalf("diagnostic = %#v", diagnostic)
+		}
+		seen[diagnostic.Path.String()] = true
+	}
+	for _, path := range []string{"/concurrency/total_llm", "/output/directory", "/promptkit/local_backend/endpoint", "/promptkit/local_backend/concurrency_limit"} {
+		if !seen[path] {
+			t.Fatalf("diagnostics missing %s: %#v", path, loadError.Diagnostics())
+		}
+	}
+}
+
+func TestDocumentLoadDistinguishesRequiredAndOptionalPathValues(t *testing.T) {
+	t.Parallel()
+
+	loaded, err := loadDocumentSource(configkit.Bytes("cache.yml", []byte(`version: 5
+cache:
+  chunk_plans:
+    directory: " "
+  checkpoints:
+    directory: " "
+`)), LoadOptions{})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if loaded.Config.Cache.ChunkPlans.Directory == nil || *loaded.Config.Cache.ChunkPlans.Directory != "" ||
+		loaded.Config.Cache.Checkpoints.Directory == nil || *loaded.Config.Cache.Checkpoints.Directory != "" {
+		t.Fatalf("cache directories = %#v", loaded.Config.Cache)
+	}
+
+	_, err = loadDocumentSource(configkit.Bytes("output.yml", []byte("version: 5\noutput:\n  directory: ' '\n")), LoadOptions{})
+	assertDocumentDiagnostic(t, err, configkit.CodeValidateError, "/output/directory")
+
+	_, err = loadDocumentSource(configkit.Bytes("promptkit.yml", []byte("version: 5\npromptkit:\n  profile_file: null\n  local_backend: {}\n")), LoadOptions{})
+	assertDocumentDiagnostic(t, err, configkit.CodeResolveError, "/promptkit/profile_file")
+
+	_, err = loadDocumentSource(configkit.Bytes("empty-source.yml", []byte("version: 5\npromptkit:\n  profile_dir: ' '\n")), LoadOptions{})
+	assertDocumentDiagnostic(t, err, configkit.CodeValidateError, "/promptkit/profile_dir")
+
+	_, err = loadDocumentSource(configkit.Bytes("missing-endpoint.yml", []byte("version: 5\npromptkit:\n  local_backend: {}\n")), LoadOptions{})
+	assertDocumentDiagnostic(t, err, configkit.CodeValidateError, "/promptkit/local_backend/endpoint")
+}
+
+func assertDocumentDiagnostic(t *testing.T, err error, code configkit.Code, path string) {
+	t.Helper()
+	var loadError *configkit.LoadError
+	if !errors.As(err, &loadError) {
+		t.Fatalf("error = %v, want ConfigKit load error", err)
+	}
+	for _, diagnostic := range loadError.Diagnostics() {
+		if diagnostic.Code == code && diagnostic.Path.String() == path {
+			return
+		}
+	}
+	t.Fatalf("diagnostics = %#v, want %s at %s", loadError.Diagnostics(), code, path)
+}

internal/core/config/pipeline_document.go

@@ -0,0 +1,513 @@
+package config
+
+import (
+	"sort"
+	"strings"
+
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
+	"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
+)
+
+// documentLoadState keeps the spelling supplied by the user while resolver
+// operations move map entries to their canonical keys. It is intentionally
+// load-local: source-location handling consumes it during a later resolver
+// pass and it never becomes runtime configuration state.
+type documentLoadState struct {
+	originalPaths       map[string]configkit.Path
+	mainDirectory       string
+	invocationDirectory string
+}
+
+func newDocumentLoadState(invocationDirectory string) *documentLoadState {
+	return &documentLoadState{originalPaths: make(map[string]configkit.Path), invocationDirectory: cleanInvocationDirectory(invocationDirectory)}
+}
+
+func (state *documentLoadState) rememberPath(normalized, original configkit.Path) {
+	if state != nil {
+		state.originalPaths[normalized.String()] = original
+	}
+}
+
+// originalPath follows recorded map-key moves back to the spelling present in
+// the source. Resolver code that needs a source coordinate must use this
+// rather than reconstructing a path from a normalized identifier.
+func (state *documentLoadState) originalPath(path configkit.Path) configkit.Path {
+	if state == nil || len(state.originalPaths) == 0 {
+		return path
+	}
+	value := path.String()
+	for {
+		changed := false
+		segments := strings.Split(strings.TrimPrefix(value, "/"), "/")
+		for length := 1; length <= len(segments); length++ {
+			prefix := "/" + strings.Join(segments[:length], "/")
+			original, ok := state.originalPaths[prefix]
+			if !ok {
+				continue
+			}
+			value = original.String() + strings.TrimPrefix(value, prefix)
+			changed = true
+			break
+		}
+		if !changed {
+			break
+		}
+	}
+	if original, err := configkit.ParsePath(value); err == nil {
+		return original
+	}
+	return path
+}
+
+func normalizeDocumentPipelines(document *Document, resolution *configkit.Resolution, state *documentLoadState) {
+	if document == nil {
+		return
+	}
+	validateDocumentPipelinePresence(document, resolution)
+	normalizer := documentNormalizer{Resolution: resolution}
+
+	pipelinesPath := configkit.Keys("pipelines")
+	for _, id := range sortedPipelineDocumentKeys(document.Pipelines) {
+		profile := document.Pipelines[id]
+		normalizePipelineDocument(&profile, pipelinesPath.Key(id), normalizer, state)
+		document.Pipelines[id] = profile
+	}
+	document.Pipelines = normalizeDocumentMap(document.Pipelines, pipelinesPath, "pipeline identifier", normalizer, state)
+}
+
+func validateDocumentPipelinePresence(document *Document, resolution *configkit.Resolution) {
+	pipelinesPath := configkit.Keys("pipelines")
+	for _, id := range sortedPipelineDocumentKeys(document.Pipelines) {
+		profile := document.Pipelines[id]
+		path := pipelinesPath.Key(id)
+		validatePipelineDocumentPresence(profile, path, resolution)
+	}
+}
+
+func validatePipelineDocumentPresence(profile PipelineDocument, path configkit.Path, resolution *configkit.Resolution) {
+	rejectDocumentNull(resolution, path.Key("llm_profile"), profile.LLMProfile == nil)
+	rejectDocumentNull(resolution, path.Key("structured_output_repair_attempts"), profile.StructuredOutputRepairAttempts == nil)
+	rejectDocumentNull(resolution, path.Key("validation_policy"), profile.ValidationPolicy == nil)
+	validateDocumentPolicyPresence(profile.ValidationPolicy, path.Key("validation_policy"), resolution)
+
+	stepsPath := path.Key("steps")
+	if resolution.IsExplicit(stepsPath) && profile.Steps == nil {
+		resolution.Error(stepsPath, "must not be null")
+	}
+	if resolution.IsExplicit(stepsPath) && len(profile.Steps) == 0 {
+		resolution.Error(stepsPath, "must contain at least one ordered step")
+	}
+	if resolution.IsExplicit(stepsPath) && resolution.IsExplicit(path.Key("artifacts")) {
+		resolution.Error(stepsPath, "must not be declared with artifacts", path.Key("artifacts"))
+	}
+
+	validateBindingDocumentPresence(profile.Input, path.Key("input"), resolution)
+	validateOptionalBindingDocumentPresence(profile.Chunk, path.Key("chunk"), resolution)
+	validateOptionalBindingDocumentPresence(profile.Output, path.Key("output"), resolution)
+	validateReferenceDocumentMapPresence(profile.References, path.Key("references"), resolution)
+	for _, laneID := range sortedLaneDocumentKeys(profile.Artifacts) {
+		validateArtifactLaneDocumentPresence(profile.Artifacts[laneID], path.Key("artifacts").Key(laneID), resolution)
+	}
+	for index, step := range profile.Steps {
+		stepPath := stepsPath.Index(index)
+		validateReferenceDocumentMapPresence(step.References, stepPath.Key("references"), resolution)
+		for _, laneID := range sortedLaneDocumentKeys(step.Artifacts) {
+			validateArtifactLaneDocumentPresence(step.Artifacts[laneID], stepPath.Key("artifacts").Key(laneID), resolution)
+		}
+	}
+}
+
+func validateArtifactLaneDocumentPresence(lane ArtifactLaneDocument, path configkit.Path, resolution *configkit.Resolution) {
+	if resolution.IsExplicit(path.Key("references")) {
+		resolution.Error(path.Key("references"), "artifact-lane references are not supported; use binding references")
+	}
+	validateBindingDocumentPresence(lane.Extract, path.Key("extract"), resolution)
+	validateOptionalBindingDocumentPresence(lane.Merge, path.Key("merge"), resolution)
+	validateOptionalBindingDocumentPresence(lane.Normalize, path.Key("normalize"), resolution)
+}
+
+func validateOptionalBindingDocumentPresence(binding *BindingDocument, path configkit.Path, resolution *configkit.Resolution) {
+	if binding == nil {
+		if resolution.IsExplicit(path) {
+			resolution.Error(path, "must not be null when configured")
+		}
+		return
+	}
+	validateBindingDocumentPresence(*binding, path, resolution)
+}
+
+func validateBindingDocumentPresence(binding BindingDocument, path configkit.Path, resolution *configkit.Resolution) {
+	rejectDocumentNull(resolution, path.Key("llm_profile"), binding.LLMProfile == nil)
+	rejectDocumentNull(resolution, path.Key("structured_output_repair_attempts"), binding.StructuredOutputRepairAttempts == nil)
+	rejectDocumentNull(resolution, path.Key("validation_policy"), binding.ValidationPolicy == nil)
+	validateDocumentPolicyPresence(binding.ValidationPolicy, path.Key("validation_policy"), resolution)
+	if binding.Validators == nil && resolution.IsExplicit(path.Key("validators")) {
+		resolution.Error(path.Key("validators"), "must not be null")
+	}
+	validateReferenceDocumentMapPresence(binding.References, path.Key("references"), resolution)
+	for index, validator := range binding.Validators {
+		validatorPath := path.Key("validators").Index(index)
+		rejectDocumentNull(resolution, validatorPath.Key("llm_profile"), validator.LLMProfile == nil)
+		rejectDocumentNull(resolution, validatorPath.Key("structured_output_repair_attempts"), validator.StructuredOutputRepairAttempts == nil)
+	}
+}
+
+func validateReferenceDocumentMapPresence(references map[string]ReferenceDocument, path configkit.Path, resolution *configkit.Resolution) {
+	for _, name := range sortedReferenceDocumentKeys(references) {
+		source := references[name]
+		sourcePath := path.Key(name)
+		rejectDocumentNull(resolution, sourcePath.Key("path"), source.Path == nil)
+		rejectDocumentNull(resolution, sourcePath.Key("artifact"), source.Artifact == nil)
+	}
+}
+
+func rejectDocumentNull(resolution *configkit.Resolution, path configkit.Path, isNil bool) {
+	if isNil && resolution.IsExplicit(path) {
+		resolution.Error(path, "must not be null")
+	}
+}
+
+func validateDocumentPolicyPresence(policy *ValidationPolicyDocument, path configkit.Path, resolution *configkit.Resolution) {
+	if policy == nil {
+		return
+	}
+	rejectDocumentNull(resolution, path.Key("producer_structural_failure"), policy.ProducerStructuralFailure == nil)
+	rejectDocumentNull(resolution, path.Key("semantic_rejection"), policy.SemanticRejection == nil)
+	rejectDocumentNull(resolution, path.Key("validator_failure"), policy.ValidatorFailure == nil)
+}
+
+func normalizePipelineDocument(profile *PipelineDocument, path configkit.Path, resolution documentNormalizer, state *documentLoadState) {
+	normalizePipelineDocumentString(resolution, path.Key("llm_profile"), &profile.LLMProfile, "trimmed pipeline LLM profile")
+	normalizeDocumentBinding(&profile.Input, path.Key("input"), resolution, state)
+	if profile.Chunk != nil {
+		normalizeDocumentBinding(profile.Chunk, path.Key("chunk"), resolution, state)
+	}
+	if profile.Output != nil {
+		normalizeDocumentBinding(profile.Output, path.Key("output"), resolution, state)
+	}
+	profile.References = normalizeDocumentReferences(profile.References, path.Key("references"), resolution, state)
+	for _, laneID := range sortedLaneDocumentKeys(profile.Artifacts) {
+		lane := profile.Artifacts[laneID]
+		normalizeArtifactLaneDocument(&lane, path.Key("artifacts").Key(laneID), resolution, state)
+		profile.Artifacts[laneID] = lane
+	}
+	profile.Artifacts = normalizeDocumentMap(profile.Artifacts, path.Key("artifacts"), "artifact lane identifier", resolution, state)
+	resolution = resolution.inSequence(path.Key("steps"))
+	for index := range profile.Steps {
+		step := &profile.Steps[index]
+		stepPath := path.Key("steps").Index(index)
+		normalizeDocumentValue(resolution, stepPath.Key("id"), &step.ID, strings.TrimSpace(step.ID), "trimmed step identifier")
+		step.References = normalizeDocumentReferences(step.References, stepPath.Key("references"), resolution, state)
+		for _, laneID := range sortedLaneDocumentKeys(step.Artifacts) {
+			lane := step.Artifacts[laneID]
+			normalizeArtifactLaneDocument(&lane, stepPath.Key("artifacts").Key(laneID), resolution, state)
+			step.Artifacts[laneID] = lane
+		}
+		step.Artifacts = normalizeDocumentMap(step.Artifacts, stepPath.Key("artifacts"), "artifact lane identifier", resolution, state)
+	}
+}
+
+func normalizeArtifactLaneDocument(lane *ArtifactLaneDocument, path configkit.Path, resolution documentNormalizer, state *documentLoadState) {
+	normalizeDocumentBinding(&lane.Extract, path.Key("extract"), resolution, state)
+	if lane.Merge != nil {
+		normalizeDocumentBinding(lane.Merge, path.Key("merge"), resolution, state)
+	}
+	if lane.Normalize != nil {
+		normalizeDocumentBinding(lane.Normalize, path.Key("normalize"), resolution, state)
+	}
+	resolution = resolution.inSequence(path.Key("validators"))
+	for index := range lane.Validators {
+		normalizeValidatorDocument(&lane.Validators[index], path.Key("validators").Index(index), resolution)
+	}
+}
+
+func normalizeDocumentBinding(binding *BindingDocument, path configkit.Path, resolution documentNormalizer, state *documentLoadState) {
+	normalizeDocumentValue(resolution, path.Key("module"), &binding.Module, strings.TrimSpace(binding.Module), "trimmed module identifier")
+	normalizePipelineDocumentString(resolution, path.Key("llm_profile"), &binding.LLMProfile, "trimmed binding LLM profile")
+	binding.References = normalizeDocumentReferences(binding.References, path.Key("references"), resolution, state)
+	resolution = resolution.inSequence(path.Key("validators"))
+	for index := range binding.Validators {
+		normalizeValidatorDocument(&binding.Validators[index], path.Key("validators").Index(index), resolution)
+	}
+}
+
+func normalizeValidatorDocument(validator *ValidatorDocument, path configkit.Path, resolution documentNormalizer) {
+	normalizeDocumentValue(resolution, path.Key("module"), &validator.Module, strings.TrimSpace(validator.Module), "trimmed validator module identifier")
+	normalizePipelineDocumentString(resolution, path.Key("llm_profile"), &validator.LLMProfile, "trimmed validator LLM profile")
+}
+
+func normalizeDocumentReferences(references map[string]ReferenceDocument, path configkit.Path, resolution documentNormalizer, state *documentLoadState) map[string]ReferenceDocument {
+	for _, name := range sortedReferenceDocumentKeys(references) {
+		source := references[name]
+		sourcePath := path.Key(name)
+		normalizePipelineDocumentString(resolution, sourcePath.Key("path"), &source.Path, "trimmed reference path")
+		if source.Artifact != nil {
+			normalizeDocumentValue(resolution, sourcePath.Key("artifact").Key("step"), &source.Artifact.Step, strings.TrimSpace(source.Artifact.Step), "trimmed artifact step selector")
+			normalizeDocumentValue(resolution, sourcePath.Key("artifact").Key("lane"), &source.Artifact.Lane, strings.TrimSpace(source.Artifact.Lane), "trimmed artifact lane selector")
+		}
+		references[name] = source
+	}
+	return normalizeDocumentMap(references, path, "reference name", resolution, state)
+}
+
+func normalizeDocumentValue[T comparable](resolution documentNormalizer, path configkit.Path, target *T, value T, reason string) {
+	if target == nil || !resolution.IsExplicit(path) {
+		return
+	}
+	if *target != value {
+		resolution.Record(path, configkit.ChangeNormalization, reason)
+		*target = value
+	}
+}
+
+func normalizePipelineDocumentString(resolution documentNormalizer, path configkit.Path, target **string, reason string) {
+	if target == nil || *target == nil || !resolution.IsExplicit(path) {
+		return
+	}
+	value := strings.TrimSpace(**target)
+	if **target != value {
+		resolution.Record(path, configkit.ChangeNormalization, reason)
+		**target = value
+	}
+}
+
+func normalizeDocumentMap[T any](values map[string]T, path configkit.Path, kind string, resolution documentNormalizer, state *documentLoadState) map[string]T {
+	if len(values) == 0 {
+		return values
+	}
+	keys := make([]string, 0, len(values))
+	for key := range values {
+		keys = append(keys, key)
+	}
+	sort.Strings(keys)
+	canonical := make(map[string]string, len(values))
+	valid := true
+	for _, key := range keys {
+		trimmed := strings.TrimSpace(key)
+		if trimmed == "" {
+			resolution.Error(path.Key(key), kind+" must not be empty")
+			valid = false
+			continue
+		}
+		if first, exists := canonical[trimmed]; exists {
+			resolution.Error(path.Key(key), kind+" collides after whitespace normalization", path.Key(first))
+			valid = false
+			continue
+		}
+		canonical[trimmed] = key
+	}
+	if !valid {
+		return values
+	}
+	result := make(map[string]T, len(values))
+	changed := false
+	dependencies := make([]configkit.Path, 0, len(keys))
+	for _, key := range keys {
+		trimmed := strings.TrimSpace(key)
+		result[trimmed] = values[key]
+		dependencies = append(dependencies, path.Key(key))
+		if trimmed != key {
+			changed = true
+			state.rememberPath(path.Key(trimmed), path.Key(key))
+		}
+	}
+	if changed {
+		resolution.Record(path, configkit.ChangeNormalization, "normalized "+kind+" keys", dependencies...)
+		values = result
+	}
+	return values
+}
+
+func validateDocumentPipelines(document *Document, validation *configkit.Validation) {
+	if document == nil {
+		return
+	}
+	pipelinesPath := configkit.Keys("pipelines")
+	for _, id := range sortedPipelineDocumentKeys(document.Pipelines) {
+		profile := document.Pipelines[id]
+		path := pipelinesPath.Key(id)
+		validatePipelineDocument(profile, path, validation)
+	}
+}
+
+func validatePipelineDocument(profile PipelineDocument, path configkit.Path, validation *configkit.Validation) {
+	validateSelectedDocumentProfile(profile.LLMProfile, path.Key("llm_profile"), validation)
+	validateDocumentRepairAttempts(profile.StructuredOutputRepairAttempts, path.Key("structured_output_repair_attempts"), validation)
+	validateDocumentPolicy(profile.ValidationPolicy, path.Key("validation_policy"), validation)
+	validateDocumentBinding(profile.Input, path.Key("input"), "input", false, false, validation)
+	if profile.Chunk != nil {
+		validateDocumentBinding(*profile.Chunk, path.Key("chunk"), "chunk", true, true, validation)
+	}
+	if profile.Output != nil {
+		validateDocumentBinding(*profile.Output, path.Key("output"), "output", false, false, validation)
+	}
+	validateDocumentReferences(profile.References, path.Key("references"), false, validation)
+
+	seenLanes := make(map[string]configkit.Path)
+	if profile.Steps != nil {
+		seenSteps := make(map[string]configkit.Path)
+		for index, step := range profile.Steps {
+			stepPath := path.Key("steps").Index(index)
+			if strings.TrimSpace(step.ID) == "" {
+				validation.Error(stepPath.Key("id"), "must not be empty")
+			} else if first, exists := seenSteps[step.ID]; exists {
+				validation.Error(stepPath.Key("id"), "must be unique", first)
+			} else {
+				seenSteps[step.ID] = stepPath.Key("id")
+			}
+			validateDocumentReferences(step.References, stepPath.Key("references"), true, validation)
+			validateDocumentLanes(step.Artifacts, stepPath.Key("artifacts"), seenLanes, validation)
+		}
+		return
+	}
+	validateDocumentLanes(profile.Artifacts, path.Key("artifacts"), seenLanes, validation)
+}
+
+func validateDocumentLanes(lanes map[string]ArtifactLaneDocument, path configkit.Path, seen map[string]configkit.Path, validation *configkit.Validation) {
+	for _, id := range sortedLaneDocumentKeys(lanes) {
+		lanePath := path.Key(id)
+		if first, exists := seen[id]; exists {
+			validation.Error(lanePath, "artifact lane identifier must be unique across steps", first)
+		} else {
+			seen[id] = lanePath
+		}
+		lane := lanes[id]
+		validateDocumentBinding(lane.Extract, lanePath.Key("extract"), "extract", true, true, validation)
+		if lane.Merge != nil {
+			validateDocumentBinding(*lane.Merge, lanePath.Key("merge"), "merge", true, true, validation)
+		}
+		if lane.Normalize != nil {
+			validateDocumentBinding(*lane.Normalize, lanePath.Key("normalize"), "normalize", true, true, validation)
+		}
+		if len(lane.Validators) > 0 {
+			validation.Error(lanePath.Key("validators"), "artifact-lane validators are not supported; attach validators to a binding")
+		}
+	}
+}
+
+func validateDocumentBinding(binding BindingDocument, path configkit.Path, slot string, referencesAllowed, validatorsAllowed bool, validation *configkit.Validation) {
+	if strings.TrimSpace(binding.Module) == "" {
+		validation.Error(path.Key("module"), "must not be empty")
+	}
+	validateSelectedDocumentProfile(binding.LLMProfile, path.Key("llm_profile"), validation)
+	validateDocumentRepairAttempts(binding.StructuredOutputRepairAttempts, path.Key("structured_output_repair_attempts"), validation)
+	if binding.Retries < 0 {
+		validation.Error(path.Key("retries"), "must be greater than or equal to zero")
+	}
+	if binding.ValidationPolicy != nil {
+		if slot != "chunk" && slot != "extract" && slot != "merge" && slot != "normalize" {
+			validation.Error(path.Key("validation_policy"), "is not supported for this binding")
+		}
+		validateDocumentPolicy(binding.ValidationPolicy, path.Key("validation_policy"), validation)
+	}
+	if len(binding.References) > 0 && !referencesAllowed {
+		validation.Error(path.Key("references"), "are not supported for this binding")
+	}
+	if referencesAllowed {
+		validateDocumentReferences(binding.References, path.Key("references"), true, validation)
+	}
+	if binding.Validators != nil && !validatorsAllowed {
+		validation.Error(path.Key("validators"), "are not supported for this binding")
+	}
+	for index, validator := range binding.Validators {
+		validateValidatorDocument(validator, path.Key("validators").Index(index), validation)
+	}
+}
+
+func validateValidatorDocument(validator ValidatorDocument, path configkit.Path, validation *configkit.Validation) {
+	if strings.TrimSpace(validator.Module) == "" {
+		validation.Error(path.Key("module"), "must not be empty")
+	}
+	validateSelectedDocumentProfile(validator.LLMProfile, path.Key("llm_profile"), validation)
+	validateDocumentRepairAttempts(validator.StructuredOutputRepairAttempts, path.Key("structured_output_repair_attempts"), validation)
+	if validator.Retries < 0 {
+		validation.Error(path.Key("retries"), "must be greater than or equal to zero")
+	}
+}
+
+func validateDocumentReferences(references map[string]ReferenceDocument, path configkit.Path, generatedAllowed bool, validation *configkit.Validation) {
+	for _, name := range sortedReferenceDocumentKeys(references) {
+		source := references[name]
+		sourcePath := path.Key(name)
+		hasPath := source.Path != nil
+		hasArtifact := source.Artifact != nil
+		if hasPath == hasArtifact {
+			validation.Error(sourcePath, "must select exactly one reference source")
+			continue
+		}
+		if hasPath {
+			if strings.TrimSpace(*source.Path) == "" {
+				validation.Error(sourcePath.Key("path"), "must not be empty")
+			}
+			continue
+		}
+		if !generatedAllowed {
+			validation.Error(sourcePath.Key("artifact"), "is not supported here; use an external path")
+			continue
+		}
+		if strings.TrimSpace(source.Artifact.Step) == "" {
+			validation.Error(sourcePath.Key("artifact").Key("step"), "must not be empty")
+		}
+		if strings.TrimSpace(source.Artifact.Lane) == "" {
+			validation.Error(sourcePath.Key("artifact").Key("lane"), "must not be empty")
+		}
+	}
+}
+
+func validateSelectedDocumentProfile(value *string, path configkit.Path, validation *configkit.Validation) {
+	if value != nil && strings.TrimSpace(*value) == "" {
+		validation.Error(path, "must not be empty when set")
+	}
+}
+
+func validateDocumentRepairAttempts(value *int, path configkit.Path, validation *configkit.Validation) {
+	if value != nil && (*value < 0 || *value > 3) {
+		validation.Error(path, "must be between zero and three")
+	}
+}
+
+func validateDocumentPolicy(policy *ValidationPolicyDocument, path configkit.Path, validation *configkit.Validation) {
+	if policy == nil {
+		return
+	}
+	if policy.ProducerStructuralFailure != nil {
+		switch *policy.ProducerStructuralFailure {
+		case pipeline.ProducerStructuralFailureFailRun, pipeline.ProducerStructuralFailureRejectOutput:
+		default:
+			validation.Error(path.Key("producer_structural_failure"), "must be fail_run or reject_output")
+		}
+	}
+	if policy.SemanticRejection != nil {
+		switch *policy.SemanticRejection {
+		case pipeline.SemanticRejectionFailRun, pipeline.SemanticRejectionRejectOutput:
+		default:
+			validation.Error(path.Key("semantic_rejection"), "must be fail_run or reject_output")
+		}
+	}
+	if policy.ValidatorFailure != nil {
+		switch *policy.ValidatorFailure {
+		case pipeline.ValidatorFailureWarnContinue, pipeline.ValidatorFailureFailRun:
+		default:
+			validation.Error(path.Key("validator_failure"), "must be warn_continue or fail_run")
+		}
+	}
+}
+
+func sortedPipelineDocumentKeys(values map[string]PipelineDocument) []string {
+	return sortedDocumentKeys(values)
+}
+func sortedLaneDocumentKeys(values map[string]ArtifactLaneDocument) []string {
+	return sortedDocumentKeys(values)
+}
+func sortedReferenceDocumentKeys(values map[string]ReferenceDocument) []string {
+	return sortedDocumentKeys(values)
+}
+
+func sortedDocumentKeys[T any](values map[string]T) []string {
+	keys := make([]string, 0, len(values))
+	for key := range values {
+		keys = append(keys, key)
+	}
+	sort.Strings(keys)
+	return keys
+}

internal/core/config/pipeline_document_test.go

@@ -0,0 +1,263 @@
+package config
+
+import (
+	"errors"
+	"fmt"
+	"testing"
+
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
+)
+
+func TestDocumentRejectsExplicitInvalidBranches(t *testing.T) {
+	t.Parallel()
+	for _, slot := range []string{"input", "output"} {
+		t.Run(slot+" empty validators", func(t *testing.T) {
+			body := "input: {module: input, validators: []}"
+			if slot == "output" {
+				body = "input: {module: input}\n    output: {module: output, validators: []}"
+			}
+			_, err := loadConfigSource(configkit.Bytes("document.yml", []byte("version: 5\npipelines:\n  session:\n    "+body+"\n")), LoadOptions{})
+			assertDocumentDiagnosticAt(t, err, "/pipelines/session/"+slot+"/validators")
+		})
+	}
+	for _, value := range []string{
+		"{path: ''}", "{path: '   '}",
+		"{path: '', artifact: {step: previous, lane: registry}}",
+		"{path: '   ', artifact: {step: previous, lane: registry}}",
+		"{path: null, artifact: {step: previous, lane: registry}}",
+	} {
+		t.Run(value, func(t *testing.T) {
+			content := fmt.Sprintf("version: 5\npipelines:\n  session:\n    input: {module: input}\n    artifacts:\n      lane:\n        extract:\n          module: extract\n          references:\n            party: %s\n", value)
+			_, err := loadConfigSource(configkit.Bytes("document.yml", []byte(content)), LoadOptions{})
+			path := "/pipelines/session/artifacts/lane/extract/references/party"
+			if value == "{path: ''}" || value == "{path: '   '}" || value == "{path: null, artifact: {step: previous, lane: registry}}" {
+				path += "/path"
+			}
+			assertDocumentDiagnosticAt(t, err, path)
+		})
+	}
+}
+
+func TestDocumentPreservesSupportedEmptyValidatorsAndReferenceForms(t *testing.T) {
+	t.Parallel()
+	loaded, err := loadConfigSource(configkit.Bytes("document.yml", []byte(`version: 5
+pipelines:
+  session:
+    input: {module: input}
+    chunk: {module: chunk, validators: []}
+    artifacts:
+      lane:
+        validators: []
+        extract:
+          module: extract
+          validators: []
+          references:
+            external: {path: /data/party.txt}
+            generated: {artifact: {step: previous, lane: registry}}
+        merge: {module: merge, validators: []}
+        normalize: {module: normalize, validators: []}
+`)), LoadOptions{})
+	if err != nil {
+		t.Fatal(err)
+	}
+	profile := loaded.Config.Pipelines["session"]
+	lane := profile.Artifacts["lane"]
+	for _, override := range []bool{profile.Chunk.Validators.Set, lane.Extract.Validators.Set, lane.Merge.Validators.Set, lane.Normalize.Validators.Set} {
+		if !override {
+			t.Fatal("supported empty validator override lost explicitness")
+		}
+	}
+	if len(lane.Validators) != 0 || lane.Extract.References["external"].Path != "/data/party.txt" ||
+		lane.Extract.References["generated"].Artifact.Lane != "registry" {
+		t.Fatalf("lane = %#v", lane)
+	}
+}
+
+func TestDocumentPipelineNormalizesCanonicalIdentifiers(t *testing.T) {
+	t.Parallel()
+
+	loaded, err := loadDocumentSource(configkit.Bytes("document.yml", []byte(`version: 5
+pipelines:
+  " session ":
+    llm_profile: " local "
+    input:
+      module: " seriatim "
+    artifacts:
+      " spells ":
+        extract:
+          module: " dnd/spells "
+          structured_output_repair_attempts: 0
+          validators: []
+        merge:
+          module: dnd/merge
+          references:
+            previous:
+              artifact:
+                step: " default "
+                lane: " spells "
+`)), LoadOptions{})
+	if err != nil {
+		var loadError *configkit.LoadError
+		if errors.As(err, &loadError) {
+			t.Fatalf("loadDocumentSource() diagnostics = %#v", loadError.Diagnostics())
+		}
+		t.Fatalf("loadDocumentSource() error = %v", err)
+	}
+	profile, ok := loaded.Config.Pipelines["session"]
+	if !ok || profile.Input.Module != "seriatim" || profile.LLMProfile == nil || *profile.LLMProfile != "local" {
+		t.Fatalf("normalized pipeline = %#v", loaded.Config.Pipelines)
+	}
+	lane, ok := profile.Artifacts["spells"]
+	if !ok || lane.Extract.StructuredOutputRepairAttempts == nil || *lane.Extract.StructuredOutputRepairAttempts != 0 || lane.Extract.Validators == nil || len(lane.Extract.Validators) != 0 {
+		t.Fatalf("normalized lane = %#v", profile.Artifacts)
+	}
+	selector := lane.Merge.References["previous"].Artifact
+	if selector == nil || selector.Step != "default" || selector.Lane != "spells" {
+		t.Fatalf("artifact selector = %#v", selector)
+	}
+}
+
+func TestDocumentPipelineRejectsInvalidShapes(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		name   string
+		source string
+		path   string
+	}{
+		{
+			name: "both pipeline forms",
+			source: `version: 5
+pipelines:
+  session:
+    input: {module: seriatim}
+    artifacts: {}
+    steps:
+      - id: first
+        artifacts: {}
+`,
+			path: "/pipelines/session/steps",
+		},
+		{
+			name: "empty ordered steps",
+			source: `version: 5
+pipelines:
+  session:
+    input: {module: seriatim}
+    steps: []
+`,
+			path: "/pipelines/session/steps",
+		},
+		{
+			name: "normalized pipeline collision",
+			source: `version: 5
+pipelines:
+  session: {input: {module: seriatim}}
+  " session ": {input: {module: seriatim}}
+`,
+			path: "/pipelines/session",
+		},
+		{
+			name: "duplicate lane across ordered steps",
+			source: `version: 5
+pipelines:
+  session:
+    input: {module: seriatim}
+    steps:
+      - id: first
+        artifacts: {summary: {extract: {module: dnd/summary}}}
+      - id: second
+        artifacts: {summary: {extract: {module: dnd/summary}}}
+`,
+			path: "/pipelines/session/steps/1/artifacts/summary",
+		},
+		{
+			name: "invalid reference union",
+			source: `version: 5
+pipelines:
+  session:
+    input: {module: seriatim}
+    references:
+      party: {path: ./party.txt, artifact: {step: one, lane: summary}}
+`,
+			path: "/pipelines/session/references/party",
+		},
+	}
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			_, err := loadDocumentSource(configkit.Bytes("document.yml", []byte(test.source)), LoadOptions{})
+			assertDocumentDiagnosticAt(t, err, test.path)
+		})
+	}
+}
+
+func TestDocumentPipelineRejectsInvalidBindingPlacementAndPresence(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		name   string
+		source string
+		path   string
+	}{
+		{
+			name: "input validators",
+			source: `version: 5
+pipelines:
+  session:
+    input:
+      module: seriatim
+      validators: [{module: generic/valid_json}]
+`,
+			path: "/pipelines/session/input/validators",
+		},
+		{
+			name: "blank selected profile",
+			source: `version: 5
+pipelines:
+  session:
+    llm_profile: "   "
+    input: {module: seriatim}
+`,
+			path: "/pipelines/session/llm_profile",
+		},
+		{
+			name: "null validators",
+			source: `version: 5
+pipelines:
+  session:
+    input: {module: seriatim}
+    chunk: {module: generic/chunk, validators: null}
+`,
+			path: "/pipelines/session/chunk/validators",
+		},
+		{
+			name: "negative retry budget",
+			source: `version: 5
+pipelines:
+  session:
+    input: {module: seriatim, retries: -1}
+`,
+			path: "/pipelines/session/input/retries",
+		},
+	}
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			_, err := loadDocumentSource(configkit.Bytes("document.yml", []byte(test.source)), LoadOptions{})
+			assertDocumentDiagnosticAt(t, err, test.path)
+		})
+	}
+}
+
+func assertDocumentDiagnosticAt(t *testing.T, err error, path string) {
+	t.Helper()
+	var loadError *configkit.LoadError
+	if !errors.As(err, &loadError) {
+		t.Fatalf("error = %v, want ConfigKit load error", err)
+	}
+	for _, diagnostic := range loadError.Diagnostics() {
+		if diagnostic.Path.String() == path {
+			return
+		}
+	}
+	t.Fatalf("diagnostics = %#v, want a diagnostic at %s", loadError.Diagnostics(), path)
+}

internal/core/config/redaction.go

@@ -4,28 +4,10 @@ import (
 	"reflect"
 	"strings"
 
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
 	"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
 )
 
-func (c Config) Redacted() Config {
-	return redactConfig(cloneConfig(c))
-}
-
-func (c Config) RedactedSummaryPayload() any {
-	return c.Redacted()
-}
-
-func (e EffectiveConfig) RedactedSummaryPayload() any {
-	return EffectiveConfig{
-		Config:             e.Config.Redacted(),
-		PipelineID:         e.PipelineID,
-		Only:               append([]string(nil), e.Only...),
-		ReferenceOverrides: append([]pipeline.ReferenceBinding(nil), e.ReferenceOverrides...),
-		ReferenceUnbinds:   append([]pipeline.ReferenceUnbind(nil), e.ReferenceUnbinds...),
-		ResolvedPipeline:   cloneResolvedPipeline(e.ResolvedPipeline),
-	}
-}
-
 func (e EffectiveConfig) RedactedResolvedPipelinePayload() pipeline.ResolvedPipeline {
 	return cloneResolvedPipeline(e.ResolvedPipeline)
 }
@@ -91,31 +73,6 @@ func cloneResolvedArtifactLane(in pipeline.ResolvedArtifactLane) pipeline.Resolv
 	return out
 }
 
-func redactConfig(cfg Config) Config {
-	for id, profile := range cfg.Pipelines {
-		profile.Input = redactBinding(profile.Input)
-		profile.Chunk = redactBinding(profile.Chunk)
-		profile.Output = redactBinding(profile.Output)
-		redactLanes := func(lanes map[string]pipeline.ArtifactLaneProfile) {
-			for laneID, lane := range lanes {
-				lane.Extract = redactBinding(lane.Extract)
-				lane.Merge = redactBinding(lane.Merge)
-				lane.Normalize = redactBinding(lane.Normalize)
-				for i := range lane.Validators {
-					lane.Validators[i] = redactBinding(lane.Validators[i])
-				}
-				lanes[laneID] = lane
-			}
-		}
-		redactLanes(profile.Artifacts)
-		for i := range profile.Steps {
-			redactLanes(profile.Steps[i].Artifacts)
-		}
-		cfg.Pipelines[id] = profile
-	}
-	return cfg
-}
-
 func redactBinding(binding pipeline.ModuleBinding) pipeline.ModuleBinding {
 	binding.Options = redactOptions(binding.Options)
 	for i := range binding.Validators.Validators {
@@ -188,3 +145,21 @@ func sensitiveConfigKey(key string) bool {
 	key = strings.ToLower(key)
 	return strings.Contains(key, "api_key") || strings.Contains(key, "apikey") || strings.Contains(key, "authorization") || strings.Contains(key, "bearer") || strings.Contains(key, "password") || strings.Contains(key, "secret") || strings.Contains(key, "token")
 }
+
+// sensitiveDocumentPath applies the existing option-key classification to the
+// fully typed ConfigKit document. Dynamic option maps can be nested below any
+// producer or validator, so every key after an options container participates.
+func sensitiveDocumentPath(path configkit.Path) bool {
+	segments := strings.Split(strings.TrimPrefix(path.String(), "/"), "/")
+	withinOptions := false
+	for _, segment := range segments {
+		segment = strings.ReplaceAll(strings.ReplaceAll(segment, "~1", "/"), "~0", "~")
+		if withinOptions && sensitiveConfigKey(segment) {
+			return true
+		}
+		if segment == "options" {
+			withinOptions = true
+		}
+	}
+	return false
+}

internal/core/config/redaction_test.go

@@ -114,83 +114,6 @@ func TestRedactedResolvedPipelinePayloadRedactsEveryBinding(t *testing.T) {
 	}
 }
 
-func TestRedactedEffectiveConfigPayloadDoesNotAliasSource(t *testing.T) {
-	binding := redactionTestBinding("effective")
-	effective := EffectiveConfig{
-		Config: Config{Pipelines: map[string]pipeline.PipelineProfile{
-			"redaction-test": {Input: binding},
-		}},
-		ResolvedPipeline: pipeline.ResolvedPipeline{Input: binding},
-	}
-
-	payload := effective.RedactedSummaryPayload().(EffectiveConfig)
-	payload.Config.Pipelines["redaction-test"].Input.Options["safe"] = "mutated"
-	payload.ResolvedPipeline.Input.Options["safe"] = "mutated"
-
-	assertRedactionTestBindingUnchanged(t, effective.Config.Pipelines["redaction-test"].Input, "effective")
-	assertRedactionTestBindingUnchanged(t, effective.ResolvedPipeline.Input, "effective")
-}
-
-func TestRedactedSummaryPayloadsCoverEveryEffectiveConfigBinding(t *testing.T) {
-	bindings := map[string]pipeline.ModuleBinding{}
-	for _, name := range []string{"input", "chunk", "output", "extract", "merge", "normalize", "lane-validator"} {
-		bindings[name] = redactionTestBinding("summary-" + name)
-	}
-	effective := EffectiveConfig{
-		Config: Config{Pipelines: map[string]pipeline.PipelineProfile{
-			"redaction-test": {
-				Input:  bindings["input"],
-				Chunk:  bindings["chunk"],
-				Output: bindings["output"],
-				Artifacts: map[string]pipeline.ArtifactLaneProfile{
-					"safe-lane": {
-						Extract:    bindings["extract"],
-						Merge:      bindings["merge"],
-						Normalize:  bindings["normalize"],
-						Validators: []pipeline.ModuleBinding{bindings["lane-validator"]},
-					},
-				},
-			},
-		}},
-		ResolvedPipeline: pipeline.ResolvedPipeline{
-			Input:  bindings["input"],
-			Chunk:  bindings["chunk"],
-			Output: bindings["output"],
-			Steps: []pipeline.ResolvedPipelineStep{{
-				ID: "default",
-				ArtifactLanes: []pipeline.ResolvedArtifactLane{{
-					ID:         "safe-lane",
-					Extract:    bindings["extract"],
-					Merge:      bindings["merge"],
-					Normalize:  bindings["normalize"],
-					Validators: []pipeline.ModuleBinding{bindings["lane-validator"]},
-				}},
-			}},
-		},
-	}
-
-	payload, ok := effective.RedactedSummaryPayload().(EffectiveConfig)
-	if !ok {
-		t.Fatal("RedactedSummaryPayload() returned an unexpected type")
-	}
-	encoded, err := json.Marshal(payload)
-	if err != nil {
-		t.Fatal(err)
-	}
-	text := string(encoded)
-	for name := range bindings {
-		if strings.Contains(text, "summary-"+name+"-secret") || strings.Contains(text, "summary-"+name+"-nested-secret") {
-			t.Fatalf("summary payload contains sensitive option for %q: %s", name, text)
-		}
-		if !strings.Contains(text, "summary-"+name+"-safe") {
-			t.Fatalf("summary payload omitted safe option for %q: %s", name, text)
-		}
-	}
-	if !strings.Contains(text, "[REDACTED]") {
-		t.Fatalf("summary payload contains no redaction marker: %s", text)
-	}
-}
-
 func TestRedactedResolvedPipelinePayloadHandlesTypedOptionContainers(t *testing.T) {
 	type optionMap map[string]string
 	type optionList []optionMap
@@ -225,39 +148,6 @@ func TestRedactedResolvedPipelinePayloadHandlesTypedOptionContainers(t *testing.
 	}
 }
 
-func TestRedactedEffectiveConfigPayloadOwnsValidationPolicies(t *testing.T) {
-	semantic := pipeline.SemanticRejectionRejectOutput
-	validator := pipeline.ValidatorFailureFailRun
-	configured := &pipeline.ValidationPolicyOverride{SemanticRejection: &semantic, ValidatorFailure: &validator}
-	effective := EffectiveConfig{
-		Config: Config{Pipelines: map[string]pipeline.PipelineProfile{
-			"main": {ValidationPolicy: configured},
-		}},
-		ResolvedPipeline: pipeline.ResolvedPipeline{
-			ConfiguredValidationPolicy: configured,
-			ChunkValidationPolicy: pipeline.ValidationPolicy{
-				ProducerStructuralFailure: pipeline.ProducerStructuralFailureFailRun,
-				SemanticRejection:         pipeline.SemanticRejectionRejectOutput,
-				ValidatorFailure:          pipeline.ValidatorFailureFailRun,
-			},
-		},
-	}
-
-	payload := effective.RedactedSummaryPayload().(EffectiveConfig)
-	encoded, err := json.Marshal(payload)
-	if err != nil {
-		t.Fatal(err)
-	}
-	if !strings.Contains(string(encoded), `"configured_validation_policy":{"semantic_rejection":"reject_output","validator_failure":"fail_run"}`) || !strings.Contains(string(encoded), `"chunk_validation_policy":{"producer_structural_failure":"fail_run","semantic_rejection":"reject_output","validator_failure":"fail_run"}`) {
-		t.Fatalf("redacted payload omitted validation policy: %s", encoded)
-	}
-	*payload.Config.Pipelines["main"].ValidationPolicy.SemanticRejection = pipeline.SemanticRejectionFailRun
-	*payload.ResolvedPipeline.ConfiguredValidationPolicy.ValidatorFailure = pipeline.ValidatorFailureWarnContinue
-	if *effective.Config.Pipelines["main"].ValidationPolicy.SemanticRejection != pipeline.SemanticRejectionRejectOutput || *effective.ResolvedPipeline.ConfiguredValidationPolicy.ValidatorFailure != pipeline.ValidatorFailureFailRun {
-		t.Fatal("redacted payload aliases validation policy")
-	}
-}
-
 func redactionTestBinding(name string) pipeline.ModuleBinding {
 	return pipeline.ModuleBinding{
 		Module: "safe-" + name,

internal/core/config/sequence_normalization_test.go

@@ -0,0 +1,92 @@
+package config
+
+import (
+	"bytes"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+func TestSequenceNormalizationPreservesRuntimeAndProvenance(t *testing.T) {
+	t.Parallel()
+	for _, test := range []struct {
+		name, body, pointer, want string
+	}{
+		{
+			name:    "step identifier",
+			body:    "steps:\n      - id: ' first '\n        artifacts: {spells: {extract: {module: dnd/spells}}}",
+			pointer: "/pipelines/session/steps/0/id",
+			want:    "first",
+		},
+		{
+			name:    "standalone validator",
+			body:    "artifacts:\n      spells:\n        extract:\n          module: dnd/spells\n          validators: [{module: ' generic/valid_json '}]",
+			pointer: "/pipelines/session/artifacts/spells/extract/validators/0/module",
+			want:    "generic/valid_json",
+		},
+		{
+			name: "nested fields and source paths",
+			body: `steps:
+      - id: first
+        artifacts:
+          " spells ":
+            extract:
+              module: " dnd/spells "
+              llm_profile: " local "
+              validators: [{module: " generic/valid_json "}]
+              references:
+                " party ": {path: "./party.txt"}
+                registry: {artifact: {step: " previous ", lane: " nouns "}}`,
+			// Map-key moves are explained at their enclosing container, where
+			// the original declaration and the new canonical keys coexist.
+			pointer: "/pipelines/session/steps",
+			want:    "party.txt",
+		},
+	} {
+		t.Run(test.name, func(t *testing.T) {
+			root := t.TempDir()
+			path := filepath.Join(root, "config.yml")
+			if err := os.WriteFile(path, []byte("version: 5\npipelines:\n  session:\n    input: {module: seriatim}\n    "+test.body+"\n"), 0o600); err != nil {
+				t.Fatal(err)
+			}
+			loaded, err := Load(path, LoadOptions{})
+			if err != nil {
+				t.Fatal(err)
+			}
+			profile := loaded.Config.Pipelines["session"]
+			switch test.name {
+			case "step identifier":
+				if profile.Steps[0].ID != "first" {
+					t.Fatalf("step = %#v", profile.Steps[0])
+				}
+			case "standalone validator":
+				if profile.Artifacts["spells"].Extract.Validators.Validators[0].Module != test.want {
+					t.Fatal("validator was not normalized")
+				}
+			default:
+				binding := profile.Steps[0].Artifacts["spells"].Extract
+				if binding.Module != "dnd/spells" || binding.LLMProfile != "local" ||
+					binding.Validators.Validators[0].Module != "generic/valid_json" ||
+					binding.References["party"].Path != filepath.Join(root, "party.txt") ||
+					binding.References["registry"].Artifact.Step != "previous" ||
+					binding.References["registry"].Artifact.Lane != "nouns" {
+					t.Fatalf("binding = %#v", binding)
+				}
+			}
+			var effective, explanation bytes.Buffer
+			if err := loaded.WriteEffective(&effective); err != nil {
+				t.Fatal(err)
+			}
+			if err := loaded.WriteExplanation(&explanation, test.pointer); err != nil {
+				t.Fatal(err)
+			}
+			if !strings.Contains(effective.String(), test.want) ||
+				!strings.Contains(explanation.String(), test.want) ||
+				!strings.Contains(explanation.String(), path) ||
+				!strings.Contains(explanation.String(), "normalization") {
+				t.Fatalf("missing effective value or provenance:\n%s\n%s", effective.String(), explanation.String())
+			}
+		})
+	}
+}

internal/core/config/validation.go

@@ -2,7 +2,6 @@ package config
 
 import (
 	"fmt"
-	"net/url"
 	"sort"
 	"strings"
 
@@ -10,15 +9,14 @@ import (
 )
 
 func (c Config) Validate() error {
-	c.Concurrency.recomputeStageWorkerDefaults()
 	if err := validatePromptKit(c.PromptKit); err != nil {
 		return err
 	}
 	if err := validateStateSurfaces(c); err != nil {
 		return err
 	}
-	if c.Concurrency.TotalLLM <= 0 {
+	if err := validateTotalLLMConcurrency(c.Concurrency.TotalLLM); err != nil {
-		return fmt.Errorf("total LLM concurrency must be greater than zero")
+		return err
 	}
 	if err := validateStageWorkers(c.Concurrency); err != nil {
 		return err
@@ -33,70 +31,42 @@ func validateStageWorkers(cfg ConcurrencyConfig) error {
 	}
 	sort.Strings(keys)
 	for _, key := range keys {
-		if strings.TrimSpace(key) == "" {
+		if err := validateStageWorkerKey(key); err != nil {
-			return fmt.Errorf("concurrency.stage_workers key must not be empty")
+			return err
-		}
-		if key != "extract" {
-			return fmt.Errorf("concurrency.stage_workers key %q is not supported", key)
 		}
 	}
 	extractWorkers, ok := cfg.StageWorkers["extract"]
 	if !ok {
 		extractWorkers = cfg.TotalLLM
 	}
-	if extractWorkers < 1 || extractWorkers > cfg.TotalLLM {
+	return validateExtractWorkerCount(extractWorkers, cfg.TotalLLM)
-		return fmt.Errorf("concurrency.stage_workers.extract must be between 1 and concurrency.total_llm (%d)", cfg.TotalLLM)
-	}
-	return nil
 }
 
 func validatePromptKit(cfg PromptKitConfig) error {
-	if strings.TrimSpace(cfg.ProfileDir) != "" && strings.TrimSpace(cfg.ProfileFile) != "" {
+	if err := validatePromptKitProfileSources(cfg.ProfileDir, cfg.ProfileFile); err != nil {
-		return fmt.Errorf("promptkit profile_dir and profile_file are mutually exclusive")
+		return err
 	}
 	if cfg.LocalBackend == nil {
 		return nil
 	}
-	endpoint := strings.TrimSpace(cfg.LocalBackend.Endpoint)
+	return validatePromptKitLocalBackend(cfg.LocalBackend.Endpoint, cfg.LocalBackend.ConcurrencyLimit)
-	if endpoint == "" {
-		return fmt.Errorf("promptkit.local_backend.endpoint must not be empty when set")
-	}
-	parsed, err := url.Parse(endpoint)
-	if err != nil ||
-		(!strings.EqualFold(parsed.Scheme, "http") && !strings.EqualFold(parsed.Scheme, "https")) ||
-		!parsed.IsAbs() ||
-		parsed.Hostname() == "" ||
-		parsed.User != nil ||
-		parsed.RawQuery != "" ||
-		parsed.ForceQuery ||
-		strings.Contains(endpoint, "#") {
-		return fmt.Errorf("promptkit.local_backend.endpoint must be an absolute HTTP or HTTPS URL with a host and no user information, query, or fragment")
-	}
-	if cfg.LocalBackend.ConcurrencyLimit < 0 {
-		return fmt.Errorf("promptkit.local_backend.concurrency_limit must not be negative")
-	}
-	return nil
 }
 
 func validateStateSurfaces(cfg Config) error {
-	if strings.TrimSpace(cfg.Output.Directory) == "" {
+	if err := validateRequiredPath("output.directory", cfg.Output.Directory); err != nil {
-		return fmt.Errorf("output.directory must not be empty")
+		return err
 	}
-	if strings.TrimSpace(cfg.Debug.Directory) == "" {
+	if err := validateRequiredPath("debug.directory", cfg.Debug.Directory); err != nil {
-		return fmt.Errorf("debug.directory must not be empty")
+		return err
 	}
-	if err := cfg.Cache.ChunkPlans.Mode.Validate(); err != nil {
+	if err := validateChunkPlanCacheMode(string(cfg.Cache.ChunkPlans.Mode)); err != nil {
-		return fmt.Errorf("cache.chunk_plans.mode: %w", err)
+		return err
 	}
-	for name, value := range map[string]string{
+	if err := validateOptionalPath("cache.chunk_plans.directory", cfg.Cache.ChunkPlans.Directory); err != nil {
-		"output.directory":            cfg.Output.Directory,
+		return err
-		"cache.chunk_plans.directory": cfg.Cache.ChunkPlans.Directory,
+	}
-		"cache.checkpoints.directory": cfg.Cache.Checkpoints.Directory,
+	if err := validateOptionalPath("cache.checkpoints.directory", cfg.Cache.Checkpoints.Directory); err != nil {
-		"debug.directory":             cfg.Debug.Directory,
+		return err
-	} {
-		if strings.ContainsRune(value, '\x00') {
-			return fmt.Errorf("%s must not contain NUL", name)
-		}
 	}
 	return nil
 }

internal/core/config/validation_contract_test.go

@@ -25,7 +25,6 @@ func TestValidateConcurrencyRules(t *testing.T) {
 			setup: func(cfg *Config) {
 				cfg.Concurrency.TotalLLM = 3
 				cfg.Concurrency.StageWorkers = map[string]int{"extract": 0}
-				cfg.Concurrency.extractWorkersConfigured = true
 			},
 			want: "stage_workers.extract must be between 1",
 		},
@@ -34,7 +33,6 @@ func TestValidateConcurrencyRules(t *testing.T) {
 			setup: func(cfg *Config) {
 				cfg.Concurrency.TotalLLM = 3
 				cfg.Concurrency.StageWorkers = map[string]int{"extract": 4}
-				cfg.Concurrency.extractWorkersConfigured = true
 			},
 			want: "stage_workers.extract must be between 1",
 		},
@@ -43,7 +41,6 @@ func TestValidateConcurrencyRules(t *testing.T) {
 			setup: func(cfg *Config) {
 				cfg.Concurrency.TotalLLM = 3
 				cfg.Concurrency.StageWorkers = map[string]int{"extract": 1}
-				cfg.Concurrency.extractWorkersConfigured = true
 			},
 		},
 		{
@@ -51,7 +48,6 @@ func TestValidateConcurrencyRules(t *testing.T) {
 			setup: func(cfg *Config) {
 				cfg.Concurrency.TotalLLM = 3
 				cfg.Concurrency.StageWorkers = map[string]int{"extract": 3}
-				cfg.Concurrency.extractWorkersConfigured = true
 			},
 		},
 		{

internal/core/config/validation_policy_contract_test.go

@@ -4,17 +4,19 @@ import (
 	"strings"
 	"testing"
 
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
 	"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
 )
 
 func TestValidationPolicyFileConfigurationIsStrictAndPresenceAware(t *testing.T) {
-	const valid = `version: 4
+	const valid = `version: 5
 pipelines:
   main:
     validation_policy:
       producer_structural_failure: reject_output
       semantic_rejection: fail_run
-    input: seriatim
+    input:
+      module: seriatim
     chunk:
       module: generic
       validation_policy:
@@ -26,7 +28,11 @@ pipelines:
           validation_policy:
             semantic_rejection: reject_output
 `
-	cfg := applyFileConfig(t, valid)
+	loaded, err := loadDocumentSource(configkit.Bytes("valid.yml", []byte(valid)), LoadOptions{})
+	if err != nil {
+		t.Fatal(err)
+	}
+	cfg := loaded.Config
 	profile := cfg.Pipelines["main"]
 	if profile.ValidationPolicy == nil || profile.ValidationPolicy.ProducerStructuralFailure == nil || *profile.ValidationPolicy.ProducerStructuralFailure != pipeline.ProducerStructuralFailureRejectOutput || profile.ValidationPolicy.SemanticRejection == nil || *profile.ValidationPolicy.SemanticRejection != pipeline.SemanticRejectionFailRun || profile.ValidationPolicy.ValidatorFailure != nil {
 		t.Fatalf("pipeline validation policy = %#v", profile.ValidationPolicy)
@@ -50,8 +56,8 @@ pipelines:
 		{"invalid enum", strings.Replace(valid, "semantic_rejection: fail_run", "semantic_rejection: continue", 1)},
 	} {
 		t.Run(test.name, func(t *testing.T) {
-			if _, err := ParseFileConfigYAML([]byte(test.yaml)); err == nil {
+			if _, err := loadDocumentSource(configkit.Bytes("invalid.yml", []byte(test.yaml)), LoadOptions{}); err == nil {
-				t.Fatal("ParseFileConfigYAML() error = nil, want strict validation-policy rejection")
+				t.Fatal("loadDocumentSource() error = nil, want strict validation-policy rejection")
 			}
 		})
 	}

internal/core/debugbundle/bundle_test.go

@@ -3,6 +3,8 @@ package debugbundle
 import (
 	"bytes"
 	"encoding/json"
+	"errors"
+	"io"
 	"os"
 	"path/filepath"
 	"strings"
@@ -89,7 +91,7 @@ func TestSummaryWriterWritesEverySummaryArtifact(t *testing.T) {
 	if err := summary.WriteInvocation(Invocation{Operation: "run"}); err != nil {
 		t.Fatal(err)
 	}
-	if err := summary.WriteRedactedEffectiveConfig(testRedactedSummaryPayload{}); err != nil {
+	if err := summary.WriteEffectiveConfig(testEffectiveConfigWriter{content: "version: 5\n"}); err != nil {
 		t.Fatal(err)
 	}
 	if err := summary.WriteResolvedPipeline(testRedactedResolvedPipelinePayload{}); err != nil {
@@ -119,7 +121,7 @@ func TestSummaryWriterWritesEverySummaryArtifact(t *testing.T) {
 
 	for _, name := range []string{
 		ArtifactInvocationMetadata,
-		ArtifactEffectiveConfig,
+		ArtifactEffectiveConfigYAML,
 		ArtifactResolvedPipeline,
 		ArtifactResolvedReferences,
 		ArtifactCheckpointEvents,
@@ -139,6 +141,20 @@ func TestSummaryWriterWritesEverySummaryArtifact(t *testing.T) {
 	}
 }
 
+func TestWriteEffectiveConfigDoesNotPublishPartialRendering(t *testing.T) {
+	bundle, err := Allocate(t.TempDir(), testBundleRunID, time.Unix(0, 42))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if err := bundle.Summary().WriteEffectiveConfig(testEffectiveConfigWriter{err: errors.New("render failed")}); err == nil {
+		t.Fatal("WriteEffectiveConfig() error = nil, want render failure")
+	}
+	path := filepath.Join(bundle.SummaryRoot(), ArtifactEffectiveConfigYAML)
+	if _, err := os.Stat(path); !os.IsNotExist(err) {
+		t.Fatalf("effective configuration artifact exists after rendering failure: %v", err)
+	}
+}
+
 func TestWriteInvocationPreservesReasoningEffortOverrideStates(t *testing.T) {
 	replacement := "focused"
 	cleared := ""
@@ -216,10 +232,17 @@ func TestSummaryWriterInternalWritesConfineArtifacts(t *testing.T) {
 	}
 }
 
-type testRedactedSummaryPayload struct{}
+type testEffectiveConfigWriter struct {
+	content string
+	err     error
+}
 
-func (testRedactedSummaryPayload) RedactedSummaryPayload() any {
+func (writer testEffectiveConfigWriter) WriteEffective(output io.Writer) error {
-	return map[string]any{"redacted": true}
+	if writer.err != nil {
+		return writer.err
+	}
+	_, err := io.WriteString(output, writer.content)
+	return err
 }
 
 type testRedactedResolvedPipelinePayload struct{}

internal/core/debugbundle/summary.go

@@ -1,7 +1,9 @@
 package debugbundle
 
 import (
+	"bytes"
 	"fmt"
+	"io"
 	"strings"
 	"time"
 
@@ -12,19 +14,19 @@ import (
 )
 
 const (
-	ArtifactInvocationMetadata = "invocation.json"
+	ArtifactInvocationMetadata  = "invocation.json"
-	ArtifactEffectiveConfig    = "effective-config.json"
+	ArtifactEffectiveConfigYAML = "effective-config.yml"
-	ArtifactResolvedPipeline   = "resolved-pipeline.json"
+	ArtifactResolvedPipeline    = "resolved-pipeline.json"
-	ArtifactResolvedReferences = "resolved-references.json"
+	ArtifactResolvedReferences  = "resolved-references.json"
-	ArtifactCheckpointEvents   = "checkpoint-events.json"
+	ArtifactCheckpointEvents    = "checkpoint-events.json"
-	ArtifactRunManifest        = "run-manifest.json"
+	ArtifactRunManifest         = "run-manifest.json"
-	ArtifactChunkPlan          = "chunk-plan.json"
+	ArtifactChunkPlan           = "chunk-plan.json"
-	ArtifactRunReport          = "run-report.json"
+	ArtifactRunReport           = "run-report.json"
-	ArtifactDiagnostics        = "final-diagnostics.json"
+	ArtifactDiagnostics         = "final-diagnostics.json"
-	ArtifactErrorLog           = "error.log"
+	ArtifactErrorLog            = "error.log"
 )
 
-type RedactedSummaryPayload interface{ RedactedSummaryPayload() any }
+type EffectiveConfigWriter interface{ WriteEffective(io.Writer) error }
 type RedactedResolvedPipelinePayload interface {
 	RedactedResolvedPipelinePayload() pipeline.ResolvedPipeline
 }
@@ -37,6 +39,7 @@ type Invocation struct {
 	InputPath               string    `json:"input_path,omitempty"`
 	ConfigPath              string    `json:"config_path,omitempty"`
 	ConfigSource            string    `json:"config_source,omitempty"`
+	ConfigProfile           string    `json:"config_profile,omitempty"`
 	OnlyLanes               []string  `json:"only_lanes,omitempty"`
 	ChunkCacheOverride      string    `json:"chunk_cache_override,omitempty"`
 	ReasoningEffortOverride *string   `json:"reasoning_effort_override,omitempty"`
@@ -80,11 +83,15 @@ func (w *SummaryWriter) WriteInvocation(payload Invocation) error {
 	}
 	return w.writeJSON(ArtifactInvocationMetadata, payload)
 }
-func (w *SummaryWriter) WriteRedactedEffectiveConfig(payload RedactedSummaryPayload) error {
+func (w *SummaryWriter) WriteEffectiveConfig(payload EffectiveConfigWriter) error {
 	if payload == nil {
-		return fmt.Errorf("redacted summary payload must not be nil")
+		return fmt.Errorf("effective configuration writer must not be nil")
 	}
-	return w.writeJSON(ArtifactEffectiveConfig, payload.RedactedSummaryPayload())
+	var rendered bytes.Buffer
+	if err := payload.WriteEffective(&rendered); err != nil {
+		return fmt.Errorf("render effective configuration: %w", err)
+	}
+	return w.writeBytes(ArtifactEffectiveConfigYAML, rendered.Bytes())
 }
 func (w *SummaryWriter) WriteResolvedPipeline(payload RedactedResolvedPipelinePayload) error {
 	if payload == nil {

internal/core/fileio/fileio.go

@@ -2,7 +2,10 @@
 package fileio
 
 import (
+	"crypto/rand"
+	"encoding/hex"
 	"encoding/json"
+	"errors"
 	"fmt"
 	"os"
 	"path"
@@ -92,6 +95,16 @@ func WriteJSON(root, name string, payload any, dirMode, fileMode os.FileMode) er
 	return WriteBytes(root, name, append(data, '\n'), dirMode, fileMode)
 }
 
+// WriteJSONWithProcessPermissions writes indented JSON beneath root using the
+// process and filesystem creation policy for new directories and files.
+func WriteJSONWithProcessPermissions(root, name string, payload any) error {
+	data, err := json.MarshalIndent(payload, "", "  ")
+	if err != nil {
+		return fmt.Errorf("marshal artifact %q: %w", name, err)
+	}
+	return WriteBytesWithProcessPermissions(root, name, append(data, '\n'))
+}
+
 func WriteBytes(root, name string, data []byte, dirMode, fileMode os.FileMode) error {
 	target, err := SafePath(root, name)
 	if err != nil {
@@ -109,6 +122,26 @@ func WriteBytes(root, name string, data []byte, dirMode, fileMode os.FileMode) e
 	return nil
 }
 
+// WriteBytesWithProcessPermissions atomically writes data beneath root while
+// allowing the process umask and filesystem default ACLs to select effective
+// permissions for newly created entries.
+func WriteBytesWithProcessPermissions(root, name string, data []byte) error {
+	target, err := SafePath(root, name)
+	if err != nil {
+		return err
+	}
+	if err := os.MkdirAll(filepath.Dir(target), 0o777); err != nil {
+		return fmt.Errorf("write artifact %q: %w", name, err)
+	}
+	if err := rejectSymlinkComponents(root, name); err != nil {
+		return err
+	}
+	if err := writeAtomicWithProcessPermissions(target, data); err != nil {
+		return fmt.Errorf("write artifact %q: %w", name, err)
+	}
+	return nil
+}
+
 func rejectSymlinkComponents(root, name string) error {
 	absRoot, err := filepath.Abs(root)
 	if err != nil {
@@ -163,3 +196,51 @@ func writeAtomic(target string, data []byte, fileMode os.FileMode) error {
 	keep = false
 	return nil
 }
+
+func writeAtomicWithProcessPermissions(target string, data []byte) error {
+	directory := filepath.Dir(target)
+	base := filepath.Base(target)
+	seen := make(map[string]struct{}, 32)
+	for attempt := 0; attempt < 32; attempt++ {
+		suffix := make([]byte, 16)
+		if _, err := rand.Read(suffix); err != nil {
+			return fmt.Errorf("generate temporary file name: %w", err)
+		}
+		temporaryPath := filepath.Join(directory, "."+base+".tmp-"+hex.EncodeToString(suffix))
+		if _, exists := seen[temporaryPath]; exists {
+			continue
+		}
+		seen[temporaryPath] = struct{}{}
+
+		temporary, err := os.OpenFile(temporaryPath, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o666)
+		if errors.Is(err, os.ErrExist) {
+			continue
+		}
+		if err != nil {
+			return err
+		}
+		return writeAndPublish(temporary, temporaryPath, target, data)
+	}
+	return fmt.Errorf("create temporary file for %q: exhausted unique names", target)
+}
+
+func writeAndPublish(temporary *os.File, temporaryPath, target string, data []byte) error {
+	keep := true
+	defer func() {
+		if keep {
+			_ = os.Remove(temporaryPath)
+		}
+	}()
+	if _, err := temporary.Write(data); err != nil {
+		_ = temporary.Close()
+		return err
+	}
+	if err := temporary.Close(); err != nil {
+		return err
+	}
+	if err := os.Rename(temporaryPath, target); err != nil {
+		return err
+	}
+	keep = false
+	return nil
+}

internal/core/fileio/fileio_process_permissions_unix_test.go

@@ -0,0 +1,131 @@
+//go:build linux || darwin
+
+package fileio
+
+import (
+	"os"
+	"path/filepath"
+	"strings"
+	"syscall"
+	"testing"
+)
+
+func TestWriteBytesWithProcessPermissionsHonorsUmask(t *testing.T) {
+	for _, test := range []struct {
+		name     string
+		umask    int
+		dirMode  os.FileMode
+		fileMode os.FileMode
+	}{
+		{name: "private", umask: 0o077, dirMode: 0o700, fileMode: 0o600},
+		{name: "group", umask: 0o007, dirMode: 0o770, fileMode: 0o660},
+	} {
+		t.Run(test.name, func(t *testing.T) {
+			previousUmask := syscall.Umask(test.umask)
+			defer syscall.Umask(previousUmask)
+
+			root := filepath.Join(t.TempDir(), "cache")
+			if err := WriteBytesWithProcessPermissions(root, "nested/value", []byte("value")); err != nil {
+				t.Fatal(err)
+			}
+			for path, want := range map[string]os.FileMode{
+				root:                                   test.dirMode,
+				filepath.Join(root, "nested"):          test.dirMode,
+				filepath.Join(root, "nested", "value"): test.fileMode,
+			} {
+				assertPermissions(t, path, want)
+			}
+			assertNoTemporarySiblings(t, filepath.Join(root, "nested"))
+		})
+	}
+}
+
+func TestWriteBytesWithProcessPermissionsLeavesExistingRootUntouched(t *testing.T) {
+	previousUmask := syscall.Umask(0o077)
+	defer syscall.Umask(previousUmask)
+
+	root := filepath.Join(t.TempDir(), "cache")
+	syscall.Umask(0)
+	if err := os.Mkdir(root, 0o751); err != nil {
+		t.Fatal(err)
+	}
+	syscall.Umask(0o077)
+	if err := WriteBytesWithProcessPermissions(root, "nested/value", []byte("value")); err != nil {
+		t.Fatal(err)
+	}
+	assertPermissions(t, root, 0o751)
+}
+
+func TestWriteBytesWithProcessPermissionsReplacementUsesCurrentPolicy(t *testing.T) {
+	root := filepath.Join(t.TempDir(), "cache")
+	previousUmask := syscall.Umask(0o077)
+	defer syscall.Umask(previousUmask)
+	if err := WriteBytesWithProcessPermissions(root, "value", []byte("before")); err != nil {
+		t.Fatal(err)
+	}
+	before, err := os.Stat(filepath.Join(root, "value"))
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	syscall.Umask(0o007)
+	if err := WriteBytesWithProcessPermissions(root, "value", []byte("after")); err != nil {
+		t.Fatal(err)
+	}
+	after, err := os.Stat(filepath.Join(root, "value"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if before.Sys().(*syscall.Stat_t).Ino == after.Sys().(*syscall.Stat_t).Ino {
+		t.Fatal("replacement retained the old inode")
+	}
+	assertPermissions(t, filepath.Join(root, "value"), 0o660)
+	data, err := os.ReadFile(filepath.Join(root, "value"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if string(data) != "after" {
+		t.Fatalf("replacement content = %q, want %q", data, "after")
+	}
+}
+
+func TestWriteJSONWithProcessPermissionsFormatsJSON(t *testing.T) {
+	previousUmask := syscall.Umask(0o077)
+	defer syscall.Umask(previousUmask)
+
+	root := filepath.Join(t.TempDir(), "cache")
+	if err := WriteJSONWithProcessPermissions(root, "value.json", map[string]string{"value": "ok"}); err != nil {
+		t.Fatal(err)
+	}
+	data, err := os.ReadFile(filepath.Join(root, "value.json"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if string(data) != "{\n  \"value\": \"ok\"\n}\n" {
+		t.Fatalf("JSON = %q", data)
+	}
+}
+
+func assertPermissions(t *testing.T, path string, want os.FileMode) {
+	t.Helper()
+	info, err := os.Stat(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got := info.Mode().Perm(); got != want {
+		t.Fatalf("%s mode = %#o, want %#o", path, got, want)
+	}
+}
+
+func assertNoTemporarySiblings(t *testing.T, directory string) {
+	t.Helper()
+	entries, err := os.ReadDir(directory)
+	if err != nil {
+		t.Fatal(err)
+	}
+	for _, entry := range entries {
+		if strings.Contains(entry.Name(), ".tmp-") {
+			t.Fatalf("temporary file remains: %s", entry.Name())
+		}
+	}
+}

internal/framework/checkpoint/filesystem_permissions_unix_test.go

@@ -0,0 +1,98 @@
+//go:build linux || darwin
+
+package checkpoint
+
+import (
+	"os"
+	"path/filepath"
+	"syscall"
+	"testing"
+
+	"gitea.maximumdirect.net/eric/notarius/internal/core/source"
+)
+
+func TestFilesystemCheckpointPermissionsFollowUmask(t *testing.T) {
+	for _, test := range []struct {
+		name     string
+		umask    int
+		dirMode  os.FileMode
+		fileMode os.FileMode
+	}{
+		{name: "private", umask: 0o077, dirMode: 0o700, fileMode: 0o600},
+		{name: "group", umask: 0o007, dirMode: 0o770, fileMode: 0o660},
+	} {
+		t.Run(test.name, func(t *testing.T) {
+			previousUmask := syscall.Umask(test.umask)
+			defer syscall.Umask(previousUmask)
+
+			root := filepath.Join(t.TempDir(), "checkpoints")
+			identity, err := NewIdentity(representativeIdentityInput())
+			if err != nil {
+				t.Fatal(err)
+			}
+			recorder, err := NewFilesystemRecorder(root, identity)
+			if err != nil {
+				t.Fatal(err)
+			}
+			if err := recorder.SourceSucceeded("source-module", sourceDocumentForPermissions()); err != nil {
+				t.Fatal(err)
+			}
+
+			relative, err := identity.RelativePath()
+			if err != nil {
+				t.Fatal(err)
+			}
+			identityRoot := filepath.Join(root, filepath.FromSlash(relative))
+			sourceRoot := filepath.Join(identityRoot, "source")
+			for path, want := range map[string]os.FileMode{
+				root:         test.dirMode,
+				identityRoot: test.dirMode,
+				sourceRoot:   test.dirMode,
+				filepath.Join(sourceRoot, "manifest.json"):        test.fileMode,
+				filepath.Join(sourceRoot, "source-document.json"): test.fileMode,
+			} {
+				assertCheckpointPermissions(t, path, want)
+			}
+		})
+	}
+}
+
+func TestFilesystemCheckpointLeavesExistingRootUntouched(t *testing.T) {
+	previousUmask := syscall.Umask(0o077)
+	defer syscall.Umask(previousUmask)
+
+	root := filepath.Join(t.TempDir(), "checkpoints")
+	syscall.Umask(0)
+	if err := os.Mkdir(root, 0o751); err != nil {
+		t.Fatal(err)
+	}
+	syscall.Umask(0o077)
+	identity, err := NewIdentity(representativeIdentityInput())
+	if err != nil {
+		t.Fatal(err)
+	}
+	recorder, err := NewFilesystemRecorder(root, identity)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if err := recorder.SourceSucceeded("source-module", sourceDocumentForPermissions()); err != nil {
+		t.Fatal(err)
+	}
+	assertCheckpointPermissions(t, root, 0o751)
+}
+
+func sourceDocumentForPermissions() *source.SourceDocument {
+	document := checkpointDocument()
+	return &document
+}
+
+func assertCheckpointPermissions(t *testing.T, path string, want os.FileMode) {
+	t.Helper()
+	info, err := os.Stat(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got := info.Mode().Perm(); got != want {
+		t.Fatalf("%s mode = %#o, want %#o", path, got, want)
+	}
+}

internal/framework/checkpoint/filesystem_test.go

@@ -5,7 +5,6 @@ import (
 	"encoding/json"
 	"os"
 	"path/filepath"
-	"runtime"
 	"strings"
 	"testing"
 	"unicode/utf8"
@@ -106,27 +105,6 @@ func TestFilesystemCheckpointRoundTripsAllStages(t *testing.T) {
 		})
 	}
 
-	if runtime.GOOS != "windows" {
-		t.Run("restrictive permissions", func(t *testing.T) {
-			root := filepath.Join(fixture.root, mustRelativePath(t, fixture.identity))
-			err := filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
-				if err != nil {
-					return err
-				}
-				want := os.FileMode(0o600)
-				if info.IsDir() {
-					want = 0o700
-				}
-				if got := info.Mode().Perm(); got != want {
-					t.Errorf("%s permissions = %o, want %o", path, got, want)
-				}
-				return nil
-			})
-			if err != nil {
-				t.Fatal(err)
-			}
-		})
-	}
 }
 
 func TestFilesystemCheckpointRejectsMissingAndCorruptState(t *testing.T) {

internal/framework/checkpoint/recorder.go

@@ -217,7 +217,7 @@ func (r *FilesystemRecorder) writeJSON(name string, payload any) error {
 	if r == nil || strings.TrimSpace(r.root) == "" {
 		return nil
 	}
-	return fileio.WriteJSON(r.root, name, payload, 0o700, 0o600)
+	return fileio.WriteJSONWithProcessPermissions(r.root, name, payload)
 }
 
 func (r *FilesystemRecorder) timestamp() time.Time {

internal/framework/chunkplan/store.go

@@ -140,7 +140,7 @@ func (s *filesystemStore) openRoot(create bool) (*os.Root, error) {
 		return nil, fmt.Errorf("chunk plan store must not be nil")
 	}
 	if create {
-		if err := os.MkdirAll(s.root, 0o700); err != nil {
+		if err := os.MkdirAll(s.root, 0o777); err != nil {
 			return nil, err
 		}
 	}
@@ -148,19 +148,6 @@ func (s *filesystemStore) openRoot(create bool) (*os.Root, error) {
 	if err != nil {
 		return nil, err
 	}
-	if !create {
-		return root, nil
-	}
-	rootDirectory, err := root.Open(".")
-	if err != nil {
-		_ = root.Close()
-		return nil, err
-	}
-	defer rootDirectory.Close()
-	if err := rootDirectory.Chmod(0o700); err != nil {
-		_ = root.Close()
-		return nil, err
-	}
 	return root, nil
 }
 
@@ -207,7 +194,7 @@ func openDigestRootWithHooks(root *os.Root, digestDir string, create bool, hooks
 			if !create {
 				return nil, entryMissing, nil
 			}
-			if err := root.Mkdir(digestDir, 0o700); err != nil && !errors.Is(err, os.ErrExist) {
+			if err := root.Mkdir(digestDir, 0o777); err != nil && !errors.Is(err, os.ErrExist) {
 				return nil, entryPresent, err
 			}
 			continue
@@ -238,22 +225,6 @@ func openDigestRootWithHooks(root *os.Root, digestDir string, create bool, hooks
 			}
 			continue
 		}
-		if create {
-			directory, openErr := digestRoot.Open(".")
-			if openErr != nil {
-				_ = digestRoot.Close()
-				return nil, entryPresent, openErr
-			}
-			chmodErr := directory.Chmod(0o700)
-			closeErr := directory.Close()
-			if chmodErr != nil || closeErr != nil {
-				_ = digestRoot.Close()
-				if chmodErr != nil {
-					return nil, entryPresent, chmodErr
-				}
-				return nil, entryPresent, closeErr
-			}
-		}
 		return digestRoot, entryPresent, nil
 	}
 }
@@ -433,10 +404,6 @@ func writeAtomicWithHooks(root *os.Root, target string, data []byte, hooks atomi
 			_ = root.Remove(tempPath)
 		}
 	}()
-	if err := temp.Chmod(0o600); err != nil {
-		_ = temp.Close()
-		return err
-	}
 	if _, err := temp.Write(data); err != nil {
 		_ = temp.Close()
 		return err
@@ -467,7 +434,7 @@ func createTemporaryFile(root *os.Root, target string) (*os.File, string, error)
 			return nil, "", err
 		}
 		path := strings.TrimSuffix(target, planFileName) + ".plan.json.tmp-" + hex.EncodeToString(suffix[:])
-		file, err := root.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600)
+		file, err := root.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o666)
 		if errors.Is(err, os.ErrExist) {
 			continue
 		}

internal/framework/chunkplan/store_permissions_unix_test.go

@@ -0,0 +1,76 @@
+//go:build linux || darwin
+
+package chunkplan
+
+import (
+	"os"
+	"path/filepath"
+	"strings"
+	"syscall"
+	"testing"
+)
+
+func TestFilesystemStorePermissions(t *testing.T) {
+	for _, test := range []struct {
+		name     string
+		umask    int
+		dirMode  os.FileMode
+		fileMode os.FileMode
+	}{
+		{name: "private", umask: 0o077, dirMode: 0o700, fileMode: 0o600},
+		{name: "group", umask: 0o007, dirMode: 0o770, fileMode: 0o660},
+	} {
+		t.Run(test.name, func(t *testing.T) {
+			previousUmask := syscall.Umask(test.umask)
+			defer syscall.Umask(previousUmask)
+
+			root := filepath.Join(t.TempDir(), "plans")
+			store := newStore(t, root)
+			if err := store.Save(testRecord(t, 1)); err != nil {
+				t.Fatal(err)
+			}
+			digestDir := filepath.Join(root, strings.TrimPrefix(testSourceDigest, "sha256:"))
+			for path, want := range map[string]os.FileMode{
+				root:                                  test.dirMode,
+				digestDir:                             test.dirMode,
+				filepath.Join(digestDir, "plan.json"): test.fileMode,
+			} {
+				assertStorePermissions(t, path, want)
+			}
+		})
+	}
+}
+
+func TestFilesystemStoreLeavesExistingDirectoriesUntouched(t *testing.T) {
+	previousUmask := syscall.Umask(0o077)
+	defer syscall.Umask(previousUmask)
+
+	root := filepath.Join(t.TempDir(), "plans")
+	digestDir := filepath.Join(root, strings.TrimPrefix(testSourceDigest, "sha256:"))
+	syscall.Umask(0)
+	if err := os.Mkdir(root, 0o751); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.Mkdir(digestDir, 0o753); err != nil {
+		t.Fatal(err)
+	}
+	syscall.Umask(0o077)
+
+	store := newStore(t, root)
+	if err := store.Save(testRecord(t, 1)); err != nil {
+		t.Fatal(err)
+	}
+	assertStorePermissions(t, root, 0o751)
+	assertStorePermissions(t, digestDir, 0o753)
+}
+
+func assertStorePermissions(t *testing.T, path string, want os.FileMode) {
+	t.Helper()
+	info, err := os.Stat(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got := info.Mode().Perm(); got != want {
+		t.Fatalf("%s mode = %#o, want %#o", path, got, want)
+	}
+}

internal/framework/chunkplan/store_test.go

@@ -61,28 +61,6 @@ func TestFilesystemStoreRoundTripAndExactPath(t *testing.T) {
 	}
 }
 
-func TestFilesystemStorePermissions(t *testing.T) {
-	root := filepath.Join(t.TempDir(), "plans")
-	store := newStore(t, root)
-	if err := store.Save(testRecord(t, 1)); err != nil {
-		t.Fatal(err)
-	}
-	digestDir := filepath.Join(root, strings.TrimPrefix(testSourceDigest, "sha256:"))
-	for path, want := range map[string]os.FileMode{
-		root:                                  0o700,
-		digestDir:                             0o700,
-		filepath.Join(digestDir, "plan.json"): 0o600,
-	} {
-		info, err := os.Stat(path)
-		if err != nil {
-			t.Fatal(err)
-		}
-		if got := info.Mode().Perm(); got != want {
-			t.Fatalf("%s mode = %04o, want %04o", path, got, want)
-		}
-	}
-}
-
 func TestFilesystemStoreMissingAndOperationalErrors(t *testing.T) {
 	root := filepath.Join(t.TempDir(), "plans")
 	store := newStore(t, root)

internal/framework/pipeline/construction.go

@@ -69,3 +69,22 @@ func RejectUnknownOptions(options map[string]any, allowed ...string) error {
 	}
 	return fmt.Errorf("unknown options %q", unknown)
 }
+
+// IntOption accepts the two integer representations admitted at the module
+// boundary: native int values from programmatic bindings and int64 values from
+// ConfigKit dynamic option maps. Callers remain responsible for their own
+// domain constraints.
+func IntOption(value any) (int, bool) {
+	switch typed := value.(type) {
+	case int:
+		return typed, true
+	case int64:
+		converted := int(typed)
+		if int64(converted) != typed {
+			return 0, false
+		}
+		return converted, true
+	default:
+		return 0, false
+	}
+}

internal/modules/generic/chunk/units/chunker.go

@@ -155,7 +155,7 @@ func nonNegativeIntOption(name string, value any) (int, error) {
 }
 
 func intOption(name string, value any) (int, error) {
-	got, ok := value.(int)
+	got, ok := pipeline.IntOption(value)
 	if !ok {
 		return 0, chunkerErrorf("%s must be an integer", name)
 	}

internal/modules/generic/chunk/units/chunker_test.go

@@ -121,9 +121,19 @@ func TestPlanRejectsInvalidOptions(t *testing.T) {
 	}
 }
 
-func TestDecodeOptionsRejectsNonNativeIntegers(t *testing.T) {
+func TestDecodeOptionsAcceptsConfigKitIntegers(t *testing.T) {
+	options, err := DecodeOptions(map[string]any{"max_units": int64(2), "overlap_units": int64(1)})
+	if err != nil {
+		t.Fatalf("DecodeOptions() error = %v", err)
+	}
+	if options.MaxUnits != 2 || options.OverlapUnits != 1 {
+		t.Fatalf("DecodeOptions() = %#v", options)
+	}
+}
+
+func TestDecodeOptionsRejectsUnsupportedIntegerRepresentations(t *testing.T) {
 	for _, value := range []any{
-		int8(2), int16(2), int32(2), int64(2),
+		int8(2), int16(2), int32(2),
 		uint(2), uint8(2), uint16(2), uint32(2), uint64(2),
 		float32(2), float64(2), json.Number("2"),
 	} {

internal/modules/generic/output/json/encoder.go

@@ -182,7 +182,7 @@ func decodeEvidenceContextPolicy(value any) (pipeline.EvidenceContextPolicy, err
 	}
 	windowUnits := 3
 	if rawWindow, ok := object["window_units"]; ok {
-		value, ok := rawWindow.(int)
+		value, ok := pipeline.IntOption(rawWindow)
 		if !ok {
 			return pipeline.EvidenceContextPolicy{}, encoderErrorf("evidence_context option %q must be an integer", "window_units")
 		}

internal/modules/generic/output/json/encoder_test.go

@@ -87,6 +87,7 @@ func TestDecodeEvidenceContextOptions(t *testing.T) {
 		{name: "disabled", options: map[string]any{"evidence_context": map[string]any{"enabled": false}}},
 		{name: "enabled default window", options: map[string]any{"evidence_context": map[string]any{"enabled": true, "lanes": []any{"npcs"}}}, want: pipeline.EvidenceContextPolicy{Enabled: true, WindowUnits: 3, LaneIDs: []string{"npcs"}}},
 		{name: "explicit zero window and normalized lanes", options: map[string]any{"evidence_context": map[string]any{"enabled": true, "lanes": []any{" spells ", "npcs"}, "window_units": 0}}, want: pipeline.EvidenceContextPolicy{Enabled: true, WindowUnits: 0, LaneIDs: []string{"npcs", "spells"}}},
+		{name: "ConfigKit window integer", options: map[string]any{"evidence_context": map[string]any{"enabled": true, "lanes": []any{"npcs"}, "window_units": int64(2)}}, want: pipeline.EvidenceContextPolicy{Enabled: true, WindowUnits: 2, LaneIDs: []string{"npcs"}}},
 		{name: "duplicate lanes", options: map[string]any{"evidence_context": map[string]any{"enabled": true, "lanes": []any{"npcs", " npcs "}}}, wantErr: "duplicated"},
 		{name: "unknown nested option", options: map[string]any{"evidence_context": map[string]any{"enabled": false, "extra": true}}, wantErr: "unknown option"},
 		{name: "disabled nested fields", options: map[string]any{"evidence_context": map[string]any{"enabled": false, "lanes": []any{"npcs"}}}, wantErr: "not allowed"},

internal/modules/integration/config_test.go

@@ -0,0 +1,43 @@
+package integration_test
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+
+	"gitea.maximumdirect.net/eric/notarius/internal/core/config"
+)
+
+func loadIntegrationConfig(t *testing.T, path string) config.Config {
+	t.Helper()
+	loaded, err := config.Load(path, config.LoadOptions{})
+	if err != nil {
+		t.Fatalf("load configuration %q: %v", path, err)
+	}
+	return loaded.RuntimeConfig()
+}
+
+func loadIntegrationConfigText(t *testing.T, content string) config.Config {
+	t.Helper()
+	path := filepath.Join(t.TempDir(), "config.yml")
+	if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	return loadIntegrationConfig(t, path)
+}
+
+func configFromYAML(content string) (config.Config, error) {
+	directory, err := os.MkdirTemp("", "notarius-integration-config-")
+	if err != nil {
+		return config.Config{}, err
+	}
+	path := filepath.Join(directory, "config.yml")
+	if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
+		return config.Config{}, err
+	}
+	loaded, err := config.Load(path, config.LoadOptions{})
+	if err != nil {
+		return config.Config{}, err
+	}
+	return loaded.RuntimeConfig(), nil
+}

internal/modules/integration/dnd_itemoccurrences_runner_test.go

@@ -10,18 +10,18 @@ import (
 func TestItemOccurrenceConfigurationRequiresEarlierItemRegistry(t *testing.T) {
 	registries := productionNPCRegistries(t)
 	cfg, err := configFromYAML(`
-version: 4
+version: 5
 pipelines:
   items:
-    input: seriatim
+    input: {module: seriatim}
-    chunk: generic
+    chunk: {module: generic}
     steps:
       - id: identify-items
         artifacts:
           registry:
-            extract: dnd/item-registry
+            extract: {module: dnd/item-registry}
-            merge: appendorder
+            merge: {module: appendorder}
-            normalize: dnd/item-registry
+            normalize: {module: dnd/item-registry}
       - id: record-occurrences
         references:
           item_registry:
@@ -30,9 +30,9 @@ pipelines:
               lane: registry
         artifacts:
           occurrences:
-            extract: dnd/item-occurrences
+            extract: {module: dnd/item-occurrences}
-            merge: appendorder
+            merge: {module: appendorder}
-            normalize: dnd/item-occurrences
+            normalize: {module: dnd/item-occurrences}
 `)
 	if err != nil {
 		t.Fatal(err)
@@ -42,16 +42,16 @@ pipelines:
 	}
 
 	broken, err := configFromYAML(strings.Replace(`
-version: 4
+version: 5
 pipelines:
   items:
-    input: seriatim
+    input: {module: seriatim}
-    chunk: generic
+    chunk: {module: generic}
     steps:
       - id: record-occurrences
         artifacts:
           occurrences:
-            extract: dnd/item-occurrences
+            extract: {module: dnd/item-occurrences}
 `, "record-occurrences", "record-occurrences", 1))
 	if err != nil {
 		t.Fatal(err)

internal/modules/integration/dnd_location_registry_runner_test.go

@@ -99,11 +99,8 @@ func TestLocationOccurrenceConsumerDoesNotRunAfterRejectedRegistry(t *testing.T)
 
 func TestLocationOccurrenceConfigurationRequiresEarlierRegistry(t *testing.T) {
 	registries := productionNPCRegistries(t)
-	cfg, err := configFromYAML(strings.Replace(locationPipelineConfig, "step: identify-locations", "step: track-location-occurrences", 1))
+	cfg := loadIntegrationConfigText(t, strings.Replace(locationPipelineConfig, "step: identify-locations", "step: track-location-occurrences", 1))
-	if err != nil {
+	_, err := cfg.Resolve(config.ResolveInput{PipelineID: "dnd-locations-fixture", Catalog: moduleCatalog(registries)})
-		t.Fatal(err)
-	}
-	_, err = cfg.Resolve(config.ResolveInput{PipelineID: "dnd-locations-fixture", Catalog: moduleCatalog(registries)})
 	if err == nil || !strings.Contains(err.Error(), "earlier step") {
 		t.Fatalf("Resolve() error = %v, want an earlier generated registry requirement", err)
 	}
@@ -129,10 +126,7 @@ func TestLocationOccurrenceCheckpointTracksGeneratedRegistry(t *testing.T) {
 
 func resolveLocationPipeline(t *testing.T, registries pipeline.Registries) pipeline.ResolvedPipeline {
 	t.Helper()
-	cfg, err := configFromYAML(locationPipelineConfig)
+	cfg := loadIntegrationConfigText(t, locationPipelineConfig)
-	if err != nil {
-		t.Fatal(err)
-	}
 	effective, err := cfg.Resolve(config.ResolveInput{PipelineID: "dnd-locations-fixture", Catalog: moduleCatalog(registries)})
 	if err != nil {
 		t.Fatalf("Resolve() error = %v", err)
@@ -144,26 +138,15 @@ func resolveLocationPipeline(t *testing.T, registries pipeline.Registries) pipel
 	return resolved
 }
 
-func configFromYAML(content string) (config.Config, error) {
+const locationPipelineConfig = `version: 5
-	fileConfig, err := config.ParseFileConfigYAML([]byte(content))
-	if err != nil {
-		return config.Config{}, err
-	}
-	cfg := config.Default()
-	if err := cfg.ApplyFileConfig(fileConfig); err != nil {
-		return config.Config{}, err
-	}
-	return cfg, nil
-}
-
-const locationPipelineConfig = `version: 4
 cache:
   chunk_plans:
     mode: bypass
   checkpoints: {}
 pipelines:
   dnd-locations-fixture:
-    input: seriatim
+    input:
+      module: seriatim
     output:
       module: json
       options:
@@ -174,9 +157,9 @@ pipelines:
       - id: identify-locations
         artifacts:
           locations:
-            extract: dnd/location-registry
+            extract: {module: dnd/location-registry}
-            merge: appendorder
+            merge: {module: appendorder}
-            normalize: dnd/location-registry
+            normalize: {module: dnd/location-registry}
       - id: track-location-occurrences
         references:
           location_registry:
@@ -185,9 +168,9 @@ pipelines:
               lane: locations
         artifacts:
           occurrences:
-            extract: dnd/location-occurrences
+            extract: {module: dnd/location-occurrences}
-            merge: appendorder
+            merge: {module: appendorder}
-            normalize: dnd/location-occurrences
+            normalize: {module: dnd/location-occurrences}
 `
 
 type locationHandoffLLMClient struct {

internal/modules/integration/dnd_npc_grounded_test.go

@@ -429,15 +429,7 @@ func assertCurrentEvidence(t *testing.T, references []source.SourceRef) {
 
 func loadGroundedPipelineConfig(t *testing.T) config.Config {
 	t.Helper()
-	fileConfig, err := config.LoadFileConfig(repositoryPathForIntegration("internal", "modules", "integration", "testdata", "dnd_npc_grounded_pipeline.yml"))
+	return loadIntegrationConfig(t, repositoryPathForIntegration("internal", "modules", "integration", "testdata", "dnd_npc_grounded_pipeline.yml"))
-	if err != nil {
-		t.Fatalf("LoadFileConfig() error = %v", err)
-	}
-	cfg := config.Default()
-	if err := cfg.ApplyFileConfig(fileConfig); err != nil {
-		t.Fatalf("ApplyFileConfig() error = %v", err)
-	}
-	return cfg
 }
 
 func configureGroundedCampaignReferences(t *testing.T, cfg *config.Config) map[string]string {

internal/modules/integration/dnd_npc_occurrences_runner_test.go

@@ -4,7 +4,6 @@ import (
 	"context"
 	"encoding/json"
 	"fmt"
-	"os"
 	"reflect"
 	"strings"
 	"sync"
@@ -168,19 +167,7 @@ func resolveNPCOccurrencePipeline(t *testing.T, registries pipeline.Registries)
 
 func loadNPCOccurrencePipelineConfig(t *testing.T) config.Config {
 	t.Helper()
-	data, err := os.ReadFile("testdata/dnd_npc_occurrences_pipeline.yml")
+	return loadIntegrationConfig(t, "testdata/dnd_npc_occurrences_pipeline.yml")
-	if err != nil {
-		t.Fatalf("ReadFile(dnd_npc_occurrences_pipeline.yml) error = %v", err)
-	}
-	fileConfig, err := config.ParseFileConfigYAML(data)
-	if err != nil {
-		t.Fatalf("ParseFileConfigYAML() error = %v", err)
-	}
-	result := config.Default()
-	if err := result.ApplyFileConfig(fileConfig); err != nil {
-		t.Fatalf("ApplyFileConfig() error = %v", err)
-	}
-	return result
 }
 
 func normalizedLane(t *testing.T, output pipeline.RunOutput, laneID string) contracts.SerializedOutput {

internal/modules/integration/dnd_npc_registry_runner_test.go

@@ -331,19 +331,7 @@ func moduleCatalog(registries pipeline.Registries) pipeline.ModuleCatalog {
 
 func loadNPCPipelineConfig(t *testing.T) config.Config {
 	t.Helper()
-	data, err := os.ReadFile("testdata/dnd_npc_registry_pipeline.yml")
+	return loadIntegrationConfig(t, "testdata/dnd_npc_registry_pipeline.yml")
-	if err != nil {
-		t.Fatalf("ReadFile(dnd_npc_registry_pipeline.yml) error = %v", err)
-	}
-	fileConfig, err := config.ParseFileConfigYAML(data)
-	if err != nil {
-		t.Fatalf("ParseFileConfigYAML() error = %v", err)
-	}
-	cfg := config.Default()
-	if err := cfg.ApplyFileConfig(fileConfig); err != nil {
-		t.Fatalf("ApplyFileConfig() error = %v", err)
-	}
-	return cfg
 }
 
 func readNPCFixture(t *testing.T) []byte {

internal/modules/integration/dnd_spells_config_test.go

@@ -2,7 +2,6 @@ package integration_test
 
 import (
 	"context"
-	"os"
 	"strings"
 	"testing"
 
@@ -121,19 +120,7 @@ func dndCapabilityCatalog(t *testing.T, inputSpec, extractorSpec pipeline.Module
 func loadDNDSpellsPipelineConfig(t *testing.T) config.Config {
 	t.Helper()
 
-	data, err := os.ReadFile("testdata/pipeline.yml")
+	return loadIntegrationConfig(t, "testdata/pipeline.yml")
-	if err != nil {
-		t.Fatalf("ReadFile(pipeline.yml) error = %v, want nil", err)
-	}
-	fileCfg, err := config.ParseFileConfigYAML(data)
-	if err != nil {
-		t.Fatalf("ParseFileConfigYAML() error = %v, want nil", err)
-	}
-	cfg := config.Default()
-	if err := cfg.ApplyFileConfig(fileCfg); err != nil {
-		t.Fatalf("ApplyFileConfig() error = %v, want nil", err)
-	}
-	return cfg
 }
 
 type dndSpellsCatalogSpecs struct {

internal/modules/integration/testdata/dnd_npc_grounded_pipeline.yml

@@ -1,4 +1,4 @@
-version: 4
+version: 5
 output:
   directory: ./notarius-output
 cache:
@@ -9,16 +9,16 @@ debug:
   directory: ./notarius-debug
 pipelines:
   dnd-npc-grounded:
-    input: seriatim
+    input: {module: seriatim}
     steps:
       - id: identify-npcs
         artifacts:
           npc_registry:
-            extract: dnd/npc-registry
+            extract: {module: dnd/npc-registry}
-            normalize: dnd/npc-registry
+            normalize: {module: dnd/npc-registry}
           scene-descriptions:
-            extract: dnd/scene-descriptions
+            extract: {module: dnd/scene-descriptions}
-            normalize: dnd/scene-descriptions
+            normalize: {module: dnd/scene-descriptions}
       - id: grounded-events
         references:
           npc_registry:
@@ -31,8 +31,8 @@ pipelines:
               lane: scene-descriptions
         artifacts:
           spells:
-            extract: dnd/spells
+            extract: {module: dnd/spells}
-            normalize: dnd/spells
+            normalize: {module: dnd/spells}
           combat:
-            extract: dnd/combat-turns
+            extract: {module: dnd/combat-turns}
-            normalize: dnd/combat-turns
+            normalize: {module: dnd/combat-turns}

internal/modules/integration/testdata/dnd_npc_occurrences_pipeline.yml

@@ -1,4 +1,4 @@
-version: 4
+version: 5
 output:
   directory: ./notarius-output
 cache:
@@ -9,13 +9,13 @@ debug:
   directory: ./notarius-debug
 pipelines:
   dnd-npc-occurrences-fixture:
-    input: seriatim
+    input: {module: seriatim}
     steps:
       - id: identify-npcs
         artifacts:
           npc_registry:
-            extract: dnd/npc-registry
+            extract: {module: dnd/npc-registry}
-            normalize: dnd/npc-registry
+            normalize: {module: dnd/npc-registry}
       - id: extract-occurrences
         references:
           npc_registry:
@@ -24,5 +24,5 @@ pipelines:
               lane: npc_registry
         artifacts:
           occurrences:
-            extract: dnd/npc-occurrences
+            extract: {module: dnd/npc-occurrences}
-            normalize: dnd/npc-occurrences
+            normalize: {module: dnd/npc-occurrences}

internal/modules/integration/testdata/dnd_npc_registry_pipeline.yml

@@ -1,4 +1,4 @@
-version: 4
+version: 5
 output:
   directory: ./notarius-output
 cache:
@@ -9,12 +9,12 @@ debug:
   directory: ./notarius-debug
 pipelines:
   dnd-npc-registry-fixture:
-    input: seriatim
+    input: {module: seriatim}
-    chunk: generic
+    chunk: {module: generic}
     artifacts:
       npc_registry:
         extract:
           module: dnd/npc-registry
           retries: 2
-        normalize: dnd/npc-registry
+        normalize: {module: dnd/npc-registry}
-    output: json
+    output: {module: json}

internal/modules/integration/testdata/pipeline.yml

@@ -1,4 +1,4 @@
-version: 4
+version: 5
 output:
   directory: ./notarius-output
 cache:
@@ -9,11 +9,11 @@ debug:
   directory: ./notarius-debug
 pipelines:
   dnd-spells-fixture:
-    input: seriatim
+    input: {module: seriatim}
-    chunk: fake/chunk
+    chunk: {module: fake/chunk}
     artifacts:
       spells:
-        extract: dnd/spells
+        extract: {module: dnd/spells}
-        merge: appendorder
+        merge: {module: appendorder}
-        normalize: noop
+        normalize: {module: noop}
-    output: json
+    output: {module: json}