Adopt ConfigKit for pipeline loading

docs/cli.md

@@ -53,10 +53,10 @@ Rules:
 - if both positional `<session_id>` and `--session-id` are provided, values must match.
 - `--previous-session-id` is a strict expectation: the selected session file
   must contain the same `previous_session_id`.
-- `--profile` selects a declared pipeline profile. It may be supplied once;
-  an explicit empty or unknown value fails configuration resolution. When it is
-  omitted, a declared `default_profile` is used. The same selection applies to
-  all common-flag commands, including `regenerate-artifacts`.
+- `--profile` selects `profiles/<name>.yml` relative to the pipeline root. It
+  may be supplied once; omitted, empty, and the exact name `default` select the
+  base configuration, while an unknown named profile fails. The same selection
+  applies to all common-flag commands, including `regenerate-artifacts`.
 - `clean --all` cannot be combined with campaign/session selectors.
 - notification delivery is currently limited to the configured `noop` mode; see
   the [configuration reference](./config.md#notifications).
@@ -101,7 +101,7 @@ Session, range, force, and artifact-execution flags are not accepted.
 `config validate` writes a concise root-path, selected-profile (or `none`), and
 effective-digest summary after successful complete validation. `config show`
 writes one deterministic, secret-free YAML document containing defaulted and
-expanded concrete configuration. It omits composition declarations, artifact
+expanded concrete configuration. It omits ConfigKit directives, artifact
 family declarations, and runtime provenance.
 
 `config sources` reports the same fully validated resolution without printing
@@ -113,8 +113,8 @@ and generated family ownership. A generated party member has
 one family record and one party record at the same logical path. The output
 never reads or prints secret values.
 
-`config diff` resolves both supplied profile names from one parsed root source
-set and compares their fully resolved, secret-free effective mappings. It does
+`config diff` resolves both supplied profile names independently and compares
+their fully resolved, secret-free effective mappings. It does
 not accept `--profile`; the two positional names must be distinct, declared
 profiles. When party-driven families are present, both profiles must resolve to
 the same selected campaign and party. Use `--campaign-file` if profile-specific

docs/config.md

@@ -49,8 +49,8 @@ sources` use the same `--config`, `--campaign`, `--campaign-file`, and
 `--profile` selection rules as pipeline commands, but do not select, discover,
 or load a session. They do not read credential values or create runtime state.
 `narratio config diff <left-profile> <right-profile>` uses the same pipeline and
-campaign selectors, resolves each named profile independently from one parsed
-root source set, and does not accept a separate `--profile` flag.
+campaign selectors, resolves each named profile with an independent ConfigKit
+load, and does not accept a separate `--profile` flag.
 
 Campaign selection is optional only when the resolved pipeline has no
 `scriptorium.artifact_families`. When families are declared, Narratio selects a
@@ -59,7 +59,7 @@ then parses the campaign-owned party and expands concrete artifacts and any
 family publish rules before validation. `config validate` prints the resulting
 root, profile, and effective digest. `config show` emits the normalized
 effective pipeline YAML, with defaults and concrete expansion included but
-composition and family declarations omitted. `config sources` prints a stable
+ConfigKit directives and family declarations omitted. `config sources` prints a stable
 source projection instead of effective values: root/import/profile/default
 ownership plus campaign/party and generated-family records. Canonical derived
 players trace to the party. The [CLI reference](cli.md#config-validate-config-show-config-sources-and-config-diff)
@@ -77,8 +77,9 @@ difference.
 
 Use the [production/testing bundle](../examples/production-testing/pipeline.yml)
 as the complete copyable migration reference. Split stable pipeline settings
-into explicit additive imports, place production/testing differences in one
-selected overlay, and retain a production `default_profile`. Convert campaign
+into explicit additive imports and place production/testing differences in the
+fixed `profiles/production.yml` and `profiles/testing.yml` overlays. Convert
+campaign
 rosters to [canonical party input](integrations/party.md), then express
 character work as families. Inspect the result
 with `config validate`, `config show`, and `config sources`; use `config diff`
@@ -96,16 +97,15 @@ remote state with an unsafe legacy identity must be migrated before use.
 
 ## Validation and Merge Rules
 
-- YAML decode is strict (`KnownFields(true)`) and accepts exactly one document:
-  unknown fields or trailing documents fail load.
+- ConfigKit performs strict typed YAML decoding and accepts exactly one document;
+  unknown fields, duplicate keys, and trailing documents fail load.
 - A pipeline file may explicitly import additive YAML fragments through the
-  root-only `composition.imports` list. Imported files contribute fields to one
+  root-only `configkit.imports` list. Imported files contribute fields to one
   logical pipeline document; they do not override fields supplied by the root
   or another import.
-- A root pipeline may declare named profiles. Exactly one profile is selected
-  by an option-aware caller or by `composition.default_profile`; a caller's
-  explicit selection takes precedence. Declaring profiles without either form
-  of selection is an error.
+- An omitted profile, an empty profile option, and the exact name `default`
+  select the base configuration. Any other name selects
+  `profiles/<name>.yml` relative to the pipeline root.
 - Configured timeout and retry-delay durations must be positive. An omitted
   artifact timeout continues to inherit its configured Scriptorium timeout.
 - Session files must be concrete; unresolved `{{ ... }}` placeholders fail load.
@@ -123,22 +123,16 @@ remote state with an unsafe legacy identity must be migrated before use.
   - local (`audio_dir` or `audio_files`), or
   - S3 (`audio_s3.prefix`).
 
-### Pipeline composition
+### Pipeline imports and profiles
 
 Large pipeline configurations may be split into explicitly named fragments and
-may declare one overlay per selectable profile:
+may use fixed-path profile overlays:
 
 ```yaml
-composition:
+configkit:
   imports:
-    - config/storage.yml
-    - config/integrations.yaml
-  default_profile: production
-  profiles:
-    production:
-      overlay: profiles/production.yml
-    testing:
-      overlay: profiles/testing.yml
+    - path: config/storage.yml
+    - path: config/integrations.yaml
 
 campaigns:
   root: /usr/local/share/narratio/campaigns
@@ -152,36 +146,29 @@ Imports are resolved relative to the directory containing the root pipeline
 file and are loaded in declaration order. Narratio does not scan directories or
 infer fragments. Each import must be a confined regular `.yml` or `.yaml` file:
 absolute paths, traversal, symlinks, directories, duplicate files, and an
-import of the root pipeline itself are rejected. Only the root pipeline may
-contain `composition`; nested composition is rejected.
+import of the root pipeline itself are rejected. Imported files cannot declare
+another `configkit` control section.
 
-Composition is additive. A map may be extended by multiple files when every
+Base merging is additive. A map may be extended by multiple files when every
 leaf is distinct, but a scalar, list, or map/list/scalar kind cannot be claimed
 more than once, even when the repeated values are identical. Conflict errors
-name the full field path and every source that claimed it. The assembled YAML is
-then decoded against the normal strict pipeline schema and defaults are applied
-once.
+name the full field path and every source that claimed it. Narratio additionally
+rejects an artifact or artifact-family key declared by more than one base file,
+even when those files contribute disjoint children.
 
-Profile names are case-sensitive, non-empty, trimmed, and cannot contain
-control characters. If `profiles` is present, it must contain at least one
-entry and every entry must contain only an `overlay` path. An explicit profile
-selection overrides `default_profile`; unknown and explicitly empty selections
-fail. Narratio never selects the first profile implicitly and does not read a
-profile selection from the environment.
+Profile names are case-sensitive and use ConfigKit's safe fixed-path naming
+rules. Only the selected overlay is opened; a malformed unselected overlay does
+not affect a base or different-profile load. Maps merge recursively, overlay
+scalars replace base scalars, and overlay lists replace base lists completely.
+Explicit `false`, zero, empty strings, empty containers, and null values remain
+explicit inputs and are not refilled by defaults. Profiles cannot inherit,
+stack, import files, or declare another ConfigKit control section.
 
-Every declared overlay is resolved relative to the root pipeline directory and
-must satisfy the same confined regular-YAML-file rules as an import. Narratio
-parses every declared overlay even when it is not selected, then applies only
-the selected one. Maps merge recursively, overlay scalars replace base scalars,
-and overlay lists replace base lists completely. Explicit `false`, zero, empty
-lists, and empty maps remain meaningful. YAML null cannot delete a value, and
-kind changes are rejected. Profiles cannot inherit from or stack with other
-profiles, and overlays cannot import files or declare profiles.
-
-After composition, Narratio strictly decodes the result, applies centralized
-defaults once, resolves ordinary paths, and computes a deterministic effective
+During ConfigKit loading, Narratio applies fresh typed defaults, recorded
+entry-level defaults and normalization, read-only file-local validation, and
+ordinary path resolution before computing a deterministic effective
 configuration digest. The digest represents the normalized, secret-free
-runtime pipeline mapping; it excludes composition declarations, source
+runtime pipeline mapping; it excludes ConfigKit directives, source
 provenance, profile identity, and raw environment secret values. Equivalent
 effective mappings therefore have the same digest regardless of how fields are
 split among the root and imports.
@@ -274,9 +261,7 @@ Rules:
 
 | Field | Type | Required | Default / Rule |
 | --- | --- | --- | --- |
-| `composition.imports[]` | list of strings | No | explicit additive pipeline fragments relative to the root pipeline directory; `.yml` or `.yaml` regular files only |
-| `composition.default_profile` | string | Conditional | selected when profiles exist and no caller explicitly selects one; must name a declared profile |
-| `composition.profiles.<name>.overlay` | string | Conditional | required for every declared profile; one confined `.yml` or `.yaml` overlay relative to the root pipeline directory |
+| `configkit.imports[].path` | string | No | explicit additive pipeline fragment relative to the root pipeline directory; `.yml` or `.yaml` regular files only |
 | `pipeline.workspace.root` | string | No | `/var/lib/narratio` |
 | `pipeline.workspace.cleanup_after_publish` | bool | No | `false`; selects the published run's workspace target for final cleanup and requires effective `pipeline.publish.enabled=true` and `pipeline.publish.upload_run=true` |
 | `pipeline.campaigns.root` | string | No | `/usr/local/share/narratio/campaigns` |

docs/internal/configuration.md

@@ -8,31 +8,21 @@ pipeline-loading boundary implemented by `internal/config`.
 
 `LoadPipeline` assembles and validates a pipeline in this order:
 
-1. Parse the root YAML into a presence-aware composition tree. The tree retains
-   source names, full field paths, node kinds, declaration order, and explicit
-   zero, false, empty-map, and empty-list values.
-2. Decode the root-only `composition` envelope, validate its declaration shapes
-   and cross-field profile semantics, then clone the root without that envelope
-   as the ordinary pipeline body. The parsed source document is not mutated. A
-   load option retains the difference between omitted and explicitly empty
-   profile selection.
-3. Open each import relative to the root pipeline directory through the
-   confined regular-file boundary. Imports must use a `.yml` or `.yaml`
-   extension and cannot traverse, use symlinks, repeat a file, import the root,
-   or contain another composition envelope.
-4. Resolve and structurally parse every declared profile overlay through the
-   same confined regular-file boundary. Missing or malformed unselected
-   overlays fail the load. Overlays cannot contain a composition envelope.
-5. Additively merge the root body and imports. Distinct map leaves compose;
-   repeated scalar or list paths and node-kind disagreements are conflicts.
-6. Select exactly one declared profile from an explicit option or the default,
-   then recursively merge its overlay. Overlay leaves replace base leaves,
-   lists are atomic replacements, and null or kind changes fail.
-7. Emit deterministic canonical YAML and strictly decode it into
-   `PipelineConfig`.
-8. Apply pipeline defaults once, explicitly normalize reusable domain values,
-   resolve ordinary relative pipeline paths from the root pipeline file, and
-   digest the normalized effective mapping.
+1. Give ConfigKit a fresh typed `PipelineConfig` default value, the root file,
+   enabled imports, and the caller's profile selection. Omitted, empty, and
+   exact `default` selections use the base; named profiles use the fixed
+   `profiles/<name>.yml` convention.
+2. ConfigKit parses the root and declared imports, additively merges base
+   sources, applies only the selected overlay, and strictly decodes the result.
+3. The definition resolver applies recorded entry-level and inherited defaults,
+   runs deterministic domain normalization, records derived publish destinations,
+   and resolves ordinary paths relative to the root pipeline file.
+4. The definition validator performs read-only checks that require only the
+   pipeline file. Checks requiring canonical-party family expansion remain at
+   the existing explicit pipeline/campaign resolution boundary.
+5. Narratio inspects public ConfigKit metadata to reject artifact and family keys
+   declared by multiple base files, adapts source ownership records, attaches
+   runtime-only state, and computes the normalized effective digest.
 
 This ordering preserves monolithic configuration behavior. Moving a field to
 an imported fragment changes its source ownership, not its path base, default,
@@ -71,7 +61,7 @@ commands do not marshal runtime objects directly.
 
 `EffectivePipelineSources` and `EffectiveCampaignSources` provide the separate
 safe provenance projection for `config sources`. Pipeline ownership begins with
-the complete logical field paths retained during composition and classifies
+the complete logical field paths projected from the schema and classifies
 each contributor as root, import, profile, or centralized default. The
 projection replaces generated concrete member paths with paired family and
 canonical-party records, and does the same for generated publish rules.
@@ -79,9 +69,8 @@ Campaign records identify campaign-owned fields and party inputs; canonical
 derived players point to the party source. The application command only joins these sorted
 records with selection metadata and never reparses configuration files.
 
-`config diff` uses a paired profile loader that parses the root, imports, and
-declared overlays once, then clones the additive base before independently
-selecting, decoding, defaulting, and finalizing each profile. When campaign
+`config diff` uses a thin paired loader that performs two independent ConfigKit
+loads. When campaign
 resolution is needed, the command loads one selected campaign and party and
 expands both effective pipelines from that same party value. The configuration
 owner projects each normalized effective mapping directly from that same typed
@@ -128,21 +117,22 @@ that candidate without changing it. Scriptorium validates artifact declarations
 and outputs, dependencies and inputs, variables, referenced outputs, and the final
 dependency graph in deterministic artifact order.
 
-Diagnostics identify the relevant semantic field. Composition conflicts also
+Diagnostics identify the relevant semantic field. Merge conflicts also
 retain every claiming source needed to correct an additive ownership conflict.
 Complete prose, punctuation, formatting, and private validation choreography are
 not compatibility contracts.
 
 ## Diagnostics And Runtime Metadata
 
-Syntax, duplicate-key, composition, conflict, and schema failures include the
+Syntax, duplicate-key, merge, profile, and schema failures include the
 relevant source name and full field path. Additive conflicts report every
 claiming source so operators can repair the split without repeatedly
 rediscovering additional conflicts.
 
 The loaded pipeline retains private runtime metadata for the absolute root
 path, ordered imports, selected profile name and selection source, selected
-overlay, contributing sources, effective digest, and leaf ownership. Base
+overlay, original ConfigKit metadata and source snapshots, effective digest,
+and leaf ownership. Base
 leaves retain their root/import owners, replaced leaves belong to the selected
 overlay, and centrally supplied values use the synthetic `default` owner. This
 metadata does not participate in YAML decoding or alter the public
@@ -152,7 +142,7 @@ The effective digest is SHA-256 over a versioned deterministic JSON encoding of
 the same typed safe projection used by show and diff. Runtime Notarius paths remain absolute for
 execution, but the digest substitutes their normalized logical values captured
 before root-relative resolution, so relocating an equivalent configuration
-bundle does not change provenance. Because composition and resolution metadata
+bundle does not change provenance. Because ConfigKit and resolution metadata
 are private, the digest excludes source layout, profile name, and ownership.
 Configuration stores environment variable names rather than resolving raw
 credentials, so raw secret values are neither loaded nor hashed.
@@ -161,12 +151,11 @@ for later runtime expansion.
 
 ## Test Surfaces
 
-`composition_test.go` protects the presence and merge algebra independently of
-the public schema. `pipeline_composition_test.go` exercises explicit imports,
-confinement, conflicts, strict decoding, metadata, and root-relative path
-behavior through `LoadPipeline`. `pipeline_profiles_test.go` covers selection,
-all-overlay validation, overlay behavior, provenance, option propagation, and
-effective-digest stability. Application configuration-loader tests protect the
+`pipeline_configkit_test.go` exercises real imports and fixed-path profiles,
+base selection, unselected-overlay isolation, source attribution, and Narratio's
+unique artifact/family base-declaration rule. Existing focused configuration
+tests protect defaults, explicit-value semantics, strict decoding, validation,
+metadata, and root-relative path behavior. Application configuration-loader tests protect the
 single-read boundary by changing the pipeline file after its initial load and
 confirming local session resolution retains the original pipeline. Other
 configuration tests continue to protect defaults and validation after assembly.

docs/internal/overview.md

@@ -37,7 +37,7 @@ failed stage or required manifest save prevents later stages from starting.
 | --- | --- | --- |
 | Executable | `cmd/narratio` | Process entry, standard stream wiring, argument handoff, and exit status. |
 | Application orchestration | `internal/app` | Command dispatch, configuration selection, secret-file environment loading, production composition, session locking, planning, execution, restore, and user-facing reporting. |
-| Configuration | [`internal/config`](configuration.md) | Presence-aware root/import/profile composition, canonical party and family expansion, strict YAML loading, defaults, normalization, session templating, and validation. |
+| Configuration | [`internal/config`](configuration.md) | ConfigKit-backed pipeline imports and profiles, canonical party and family expansion, strict YAML loading, defaults, normalization, session templating, and validation. |
 | Pipeline stages | `internal/stage` | Canonical stage registry, shared stage contract, execution dependencies, and implemented stage behavior. |
 | External boundaries | `internal/adapters`, `internal/audio` | WhisperX HTTP, downstream subprocesses, notification, object storage, and S3 audio materialization behind Narratio contracts. |
 | Manifests | `internal/manifest` | Durable session progress, stage transitions, validation, and atomic persistence. |
@@ -105,8 +105,8 @@ so any earlier replacement can stale a prior cleanup result.
 
 ## Focused Documentation
 
-- [Configuration Internals](configuration.md): pipeline composition, import
-  confinement, field ownership, decoding, and root-relative path semantics.
+- [Configuration Internals](configuration.md): pipeline imports and profiles,
+  field ownership, decoding, and root-relative path semantics.
 - [Adapter Internals](adapters.md): external adapter boundaries, composition,
   failure behavior, and test surfaces.
 - [Artifact Internals](artifacts.md): source identities, runtime catalog,

docs/policy/architecture.md

@@ -146,7 +146,7 @@ Configuration is strict, explicit, centralized, and operator-oriented.
 - YAML decoding rejects unknown fields.
 - Defaults are centralized and testable.
 - Empty configured values do not silently replace meaningful defaults.
-- Validation rejects invalid composition before stage execution where
+- Validation rejects invalid configuration before stage execution where
   practical.
 - Root-owned imports and one selected profile resolve deterministically through
   the configuration owner; commands do not implement their own merge rules.

docs/roadmap/implementation.md

@@ -189,7 +189,10 @@ projection consumer needs the custom composition parser/renderer.
 
 ## Stage 4 — Cut pipeline loading over to ConfigKit
 
-Status: Pending
+Status: Complete — Replaced pipeline composition with ConfigKit v0.1.1 imports,
+fixed-path profile selection, fresh typed defaults, recorded resolution, source
+metadata adaptation, and duplicate base artifact/family enforcement; removed the
+legacy parser and verified all tests plus the configuration race suite.
 
 Read: ConfigKit v0.1.1 `docs/config.md`, `docs/consumers/README.md`, and its
 Narratio-shaped compiled example; Narratio `internal/config/load.go`,