narratio/docs/internal/configuration.md

Configuration Internals

User-visible fields, defaults, and selection behavior belong in the Configuration Reference. This document describes the internal pipeline-loading boundary implemented by internal/config.

Pipeline Loading

LoadPipeline assembles and validates a pipeline in this order:

1. Parse the root YAML into a presence-aware composition tree. The tree retains source names, full field paths, node kinds, declaration order, and explicit zero, false, empty-map, and empty-list values.
2. Remove the root-only composition envelope and validate its explicit imports, default_profile, and named profiles declarations. A load option retains the difference between omitted and explicitly empty profile selection.
3. Open each import relative to the root pipeline directory through the confined regular-file boundary. Imports must use a .yml or .yaml extension and cannot traverse, use symlinks, repeat a file, import the root, or contain another composition envelope.
4. Resolve and structurally parse every declared profile overlay through the same confined regular-file boundary. Missing or malformed unselected overlays fail the load. Overlays cannot contain a composition envelope.
5. Additively merge the root body and imports. Distinct map leaves compose; repeated scalar or list paths and node-kind disagreements are conflicts.
6. Select exactly one declared profile from an explicit option or the default, then recursively merge its overlay. Overlay leaves replace base leaves, lists are atomic replacements, and null or kind changes fail.
7. Emit deterministic canonical YAML and strictly decode it into PipelineConfig.
8. Apply pipeline defaults once, resolve ordinary relative pipeline paths from the root pipeline file, and digest the normalized effective mapping.

This ordering preserves monolithic configuration behavior. Moving a field to an imported fragment changes its source ownership, not its path base, default, or schema semantics.

Diagnostics And Runtime Metadata

Syntax, duplicate-key, composition, conflict, and schema failures include the relevant source name and full field path. Additive conflicts report every claiming source so operators can repair the split without repeatedly rediscovering additional conflicts.

The loaded pipeline retains private runtime metadata for the absolute root path, ordered imports, selected profile name and selection source, selected overlay, contributing sources, effective digest, and leaf ownership. Base leaves retain their root/import owners, replaced leaves belong to the selected overlay, and centrally supplied values use the synthetic default owner. This metadata does not participate in YAML decoding or alter the public configuration model.

The effective digest is SHA-256 over deterministic canonical YAML produced from the defaulted and path-resolved PipelineConfig. Because composition and resolution metadata are private, the digest excludes source layout, profile name, and ownership. Configuration stores environment variable names rather than resolving raw credentials, so raw secret values are neither loaded nor hashed. recomputePipelineEffectiveDigest is the single package-owned refresh point for later runtime expansion.

Test Surfaces

composition_test.go protects the presence and merge algebra independently of the public schema. pipeline_composition_test.go exercises explicit imports, confinement, conflicts, strict decoding, metadata, and root-relative path behavior through LoadPipeline. pipeline_profiles_test.go covers selection, all-overlay validation, overlay behavior, provenance, option propagation, and effective-digest stability. Other configuration tests continue to protect defaults and validation after assembly.