narratio/docs/internal/configuration.md

Configuration Internals

User-visible fields, defaults, and selection behavior belong in the Configuration Reference. This document describes the internal pipeline-loading boundary implemented by internal/config.

Pipeline Loading

LoadPipeline assembles and validates a pipeline in this order:

1. Parse the root YAML into a presence-aware composition tree. The tree retains source names, full field paths, node kinds, declaration order, and explicit zero, false, empty-map, and empty-list values.
2. Decode the root-only composition envelope, validate its declaration shapes and cross-field profile semantics, then clone the root without that envelope as the ordinary pipeline body. The parsed source document is not mutated. A load option retains the difference between omitted and explicitly empty profile selection.
3. Open each import relative to the root pipeline directory through the confined regular-file boundary. Imports must use a .yml or .yaml extension and cannot traverse, use symlinks, repeat a file, import the root, or contain another composition envelope.
4. Resolve and structurally parse every declared profile overlay through the same confined regular-file boundary. Missing or malformed unselected overlays fail the load. Overlays cannot contain a composition envelope.
5. Additively merge the root body and imports. Distinct map leaves compose; repeated scalar or list paths and node-kind disagreements are conflicts.
6. Select exactly one declared profile from an explicit option or the default, then recursively merge its overlay. Overlay leaves replace base leaves, lists are atomic replacements, and null or kind changes fail.
7. Emit deterministic canonical YAML and strictly decode it into PipelineConfig.
8. Apply pipeline defaults once, explicitly normalize reusable domain values, resolve ordinary relative pipeline paths from the root pipeline file, and digest the normalized effective mapping.

This ordering preserves monolithic configuration behavior. Moving a field to an imported fragment changes its source ownership, not its path base, default, or schema semantics.

Loaded Context Resolution

LoadedPipelineCampaign carries one already composed pipeline and its selected campaign into session resolution. LoadSessionWithPipelineCampaignOptions loads a local session against that context, while ResolveLoadedPipelineCampaign also accepts an already loaded remote session or no session while a caller retrieves one. Compatibility loaders route through these functions after their initial pipeline and campaign reads.

Application commands own pipeline and campaign discovery, campaign-file versus registry selection, and the corresponding mutual-exclusion rules. Once they have a LoadedPipelineCampaign, local session discovery and remote-session download retain that exact pipeline object and its private provenance. Removing a temporary downloaded session file therefore cannot invalidate the resolved pipeline or campaign context.

The application also has a separate read-only inspection resolver for config validate, config show, and config sources. It uses the same production root/profile and campaign selection functions, but never routes through session discovery, remote-session download, secret loading, adapter composition, workspace initialization, manifest access, or cleanup. A pipeline with retained artifact family declarations must resolve its selected campaign before ordinary pipeline validation, which expands its canonical-party members and generated publish rules. A pipeline without those declarations may be validated by itself.

MarshalEffectivePipeline is the configuration-owned projection for config show. It serializes one explicit typed safe projection with the existing YAML encoder. The projection has no slots for resolution-only artifact family declarations or private provenance. The result has one trailing newline; commands do not marshal runtime objects directly.

EffectivePipelineSources and EffectiveCampaignSources provide the separate safe provenance projection for config sources. Pipeline ownership begins with the complete logical field paths retained during composition and classifies each contributor as root, import, profile, or centralized default. The projection replaces generated concrete member paths with paired family and canonical-party records, and does the same for generated publish rules. Campaign records identify campaign-owned fields and party inputs; canonical derived players point to the party source. The application command only joins these sorted records with selection metadata and never reparses configuration files.

config diff uses a paired profile loader that parses the root, imports, and declared overlays once, then clones the additive base before independently selecting, decoding, defaulting, and finalizing each profile. When campaign resolution is needed, the command loads one selected campaign and party and expands both effective pipelines from that same party value. The configuration owner projects each normalized effective mapping directly from that same typed projection into sorted logical paths using JSON number preservation; mapping leaves are compared individually while sequence values remain atomic. Values are compact deterministic JSON representations for command output, not raw YAML fragments, ownership records, or secret material. A differing digest with no projected difference is treated as an internal consistency error.

Campaign context construction reads the campaign-owned party source through ParseParty. The required canonical party retains its raw bytes and normalized roster in runtime-only ResolvedParty provenance. Canonical resolution creates a deterministic derived_from_party players projection. The party is the only configured roster authority; player and session-party fields are unsupported.

Canonical Party Domain

ParseParty is the package-owned boundary for parsing a party source. It requires a top-level schema_version and strictly validates the narratio.party.v1 contract into ordered character domain values. The canonical value retains a separate exact byte copy of its source so consumers can materialize the authored party document without reserializing it. Its PlayersYAML method deterministically derives the versioned players-only projection.

An unversioned source is rejected before resolution, family expansion, or prepare-stage materialization.

Validation Ownership

Configuration validation remains within internal/config, with the public entry points and pipeline-domain sequencing separated from the private owners for core pipeline/session rules, external adapters, publish/storage, Notarius, Scriptorium, and shared scalar/path checks. These are package-local ownership boundaries rather than extension points or a general validation framework.

Validation is read-only, fails fast, and does not define which of several independent invalid fields is reported first. Map-backed declarations are nevertheless traversed deterministically. Reusable domain normalization is an explicit preceding operation for both loaded and programmatically constructed pipelines. Notarius normalization commits canonical scalar, reference, and output values only after the complete normalization succeeds; its validator then checks that candidate without changing it. Scriptorium validates artifact declarations and outputs, dependencies and inputs, variables, referenced outputs, and the final dependency graph in deterministic artifact order.

Diagnostics identify the relevant semantic field. Composition conflicts also retain every claiming source needed to correct an additive ownership conflict. Complete prose, punctuation, formatting, and private validation choreography are not compatibility contracts.

Diagnostics And Runtime Metadata

Syntax, duplicate-key, composition, conflict, and schema failures include the relevant source name and full field path. Additive conflicts report every claiming source so operators can repair the split without repeatedly rediscovering additional conflicts.

The loaded pipeline retains private runtime metadata for the absolute root path, ordered imports, selected profile name and selection source, selected overlay, contributing sources, effective digest, and leaf ownership. Base leaves retain their root/import owners, replaced leaves belong to the selected overlay, and centrally supplied values use the synthetic default owner. This metadata does not participate in YAML decoding or alter the public configuration model.

The effective digest is SHA-256 over a versioned deterministic JSON encoding of the same typed safe projection used by show and diff. Runtime Notarius paths remain absolute for execution, but the digest substitutes their normalized logical values captured before root-relative resolution, so relocating an equivalent configuration bundle does not change provenance. Because composition and resolution metadata are private, the digest excludes source layout, profile name, and ownership. Configuration stores environment variable names rather than resolving raw credentials, so raw secret values are neither loaded nor hashed. recomputePipelineEffectiveDigest is the single package-owned refresh point for later runtime expansion.

Test Surfaces

composition_test.go protects the presence and merge algebra independently of the public schema. pipeline_composition_test.go exercises explicit imports, confinement, conflicts, strict decoding, metadata, and root-relative path behavior through LoadPipeline. pipeline_profiles_test.go covers selection, all-overlay validation, overlay behavior, provenance, option propagation, and effective-digest stability. Application configuration-loader tests protect the single-read boundary by changing the pipeline file after its initial load and confirming local session resolution retains the original pipeline. Other configuration tests continue to protect defaults and validation after assembly. party_test.go protects the versioned party schema, domain invariants, and deterministic players projection without involving campaign or runtime wiring. party_resolution_test.go protects campaign-owned party loading, canonical input restrictions, source provenance, and virtual players input selection.

Artifact Family Resolution

Pipeline loading retains scriptorium.artifact_families as a resolution-only declaration. Once campaign party resolution establishes a canonical roster, configuration expands families in sorted family-key and character-ID order into ordinary ScriptoriumArtifactConfig values. The expansion owns the narrow {character_id} output substitution, closed member-variable selectors, key and output collision checks, and the runtime-only family-origin catalog. It then removes family declarations from ScriptoriumConfig, runs ordinary Scriptorium validation, and refreshes the effective pipeline digest. Stages and adapters therefore receive only concrete artifact maps.

The catalog retains sorted family member keys plus family/character/source origins and the typed dependency/publish declarations for their later owners. member_dependencies add corresponding ordinary concrete dependencies, while the family-only narratio.member_artifact.<family> input form is rewritten to the matching ordinary configured-artifact source. The catalog records those resolved dependency and input identities with their declaring family and party member. No member-artifact source is registered as a runtime policy source. An enabled family publish declaration expands to ordinary configured-artifact publish rules before the existing publish and lock validators run. Runtime publication consequently receives no family wildcard or special matcher.