narratio/docs/internal/configuration.md

Configuration Internals

User-visible fields, defaults, and selection behavior belong in the Configuration Reference. This document describes the internal pipeline-loading boundary implemented by internal/config.

Pipeline Loading

LoadPipeline assembles and validates a pipeline in this order:

1. Give ConfigKit a fresh typed PipelineConfig default value, the root file, enabled imports, and the caller's profile selection. Omitted, empty, and exact default selections use the base; named profiles use the fixed profiles/<name>.yml convention.
2. ConfigKit parses the root and declared imports, additively merges base sources, applies only the selected overlay, and strictly decodes the result.
3. The definition resolver applies recorded entry-level and inherited defaults, runs deterministic domain normalization, records derived publish destinations, and resolves ordinary paths relative to the root pipeline file.
4. The definition validator performs read-only checks that require only the pipeline file. Checks requiring canonical-party family expansion remain at the existing explicit pipeline/campaign resolution boundary.
5. Narratio inspects public ConfigKit metadata to reject artifact and family keys declared by multiple base files, adapts source ownership records, attaches runtime-only state, and computes the normalized effective digest.

This ordering preserves monolithic configuration behavior. Moving a field to an imported fragment changes its source ownership, not its path base, default, or schema semantics.

Loaded Context Resolution

LoadedPipelineCampaign carries one already composed pipeline and its selected campaign into session resolution. LoadSessionWithPipelineCampaignOptions loads a local session against that context, while ResolveLoadedPipelineCampaign also accepts an already loaded remote session or no session while a caller retrieves one. Compatibility loaders route through these functions after their initial pipeline and campaign reads.

Application commands own pipeline and campaign discovery, campaign-file versus registry selection, and the corresponding mutual-exclusion rules. Once they have a LoadedPipelineCampaign, local session discovery and remote-session download retain that exact pipeline object and its private provenance. Removing a temporary downloaded session file therefore cannot invalidate the resolved pipeline or campaign context.

The application also has a separate read-only inspection resolver for config validate, config show, and config sources. It uses the same production root/profile and campaign selection functions, but never routes through session discovery, remote-session download, secret loading, adapter composition, workspace initialization, manifest access, or cleanup. A pipeline with retained artifact family declarations must resolve its selected campaign before ordinary pipeline validation, which expands its canonical-party members and generated publish rules. A pipeline without those declarations may be validated by itself.

MarshalEffectivePipeline is the configuration-owned projection for config show. It serializes one explicit typed safe projection with the existing YAML encoder. The projection has no slots for resolution-only artifact family declarations or private provenance. The result has one trailing newline; commands do not marshal runtime objects directly.

EffectivePipelineSources and EffectiveCampaignSources provide the separate safe provenance projection for config sources. Pipeline ownership begins with the complete logical field paths projected from the schema and classifies each contributor as root, import, profile, or centralized default. The projection replaces generated concrete member paths with paired family and canonical-party records, and does the same for generated publish rules. Campaign records identify campaign-owned fields and party inputs; canonical derived players point to the party source. The application command only joins these sorted records with selection metadata and never reparses configuration files.

config diff uses a thin paired loader that performs two independent ConfigKit loads. When campaign resolution is needed, the command loads one selected campaign and party and expands both effective pipelines from that same party value. The configuration owner projects each normalized effective mapping directly from that same typed projection into sorted logical paths using JSON number preservation; mapping leaves are compared individually while sequence values remain atomic. Values are compact deterministic JSON representations for command output, not raw YAML fragments, ownership records, or secret material. A differing digest with no projected difference is treated as an internal consistency error.

Campaign context construction reads the campaign-owned party source through ParseParty. The required canonical party retains its raw bytes and normalized roster in runtime-only ResolvedParty provenance. Canonical resolution creates a deterministic derived_from_party players projection. The party is the only configured roster authority; player and session-party fields are unsupported.

Canonical Party Domain

ParseParty is the package-owned boundary for parsing a party source. It requires a top-level schema_version and strictly validates the narratio.party.v1 contract into ordered character domain values. The canonical value retains a separate exact byte copy of its source so consumers can materialize the authored party document without reserializing it. Its PlayersYAML method deterministically derives the versioned players-only projection.

An unversioned source is rejected before resolution, family expansion, or prepare-stage materialization.

Validation Ownership

Configuration validation remains within internal/config, with the public entry points and pipeline-domain sequencing separated from the private owners for core pipeline/session rules, external adapters, publish/storage, Notarius, Scriptorium, and shared scalar/path checks. These are package-local ownership boundaries rather than extension points or a general validation framework.

Validation is read-only, fails fast, and does not define which of several independent invalid fields is reported first. Map-backed declarations are nevertheless traversed deterministically. Reusable domain normalization is an explicit preceding operation for both loaded and programmatically constructed pipelines. Notarius normalization commits canonical scalar, reference, and output values only after the complete normalization succeeds; its validator then checks that candidate without changing it. Scriptorium validates artifact declarations and outputs, dependencies and inputs, variables, referenced outputs, and the final dependency graph in deterministic artifact order.

Diagnostics identify the relevant semantic field. Merge conflicts also retain every claiming source needed to correct an additive ownership conflict. Complete prose, punctuation, formatting, and private validation choreography are not compatibility contracts.

Diagnostics And Runtime Metadata

Syntax, duplicate-key, merge, profile, and schema failures include the relevant source name and full field path. Additive conflicts report every claiming source so operators can repair the split without repeatedly rediscovering additional conflicts.

The loaded pipeline retains private runtime metadata for the absolute root path, ordered imports, selected profile name and selection source, selected overlay, original ConfigKit metadata and source snapshots, effective digest, and leaf ownership. Base leaves retain their root/import owners, replaced leaves belong to the selected overlay, and centrally supplied values use the synthetic default owner. This metadata does not participate in YAML decoding or alter the public configuration model.

The effective digest is SHA-256 over a versioned deterministic JSON encoding of the same typed safe projection used by show and diff. Runtime Notarius paths remain absolute for execution, but the digest substitutes their normalized logical values captured before root-relative resolution, so relocating an equivalent configuration bundle does not change provenance. Because ConfigKit and resolution metadata are private, the digest excludes source layout, profile name, and ownership. Configuration stores environment variable names rather than resolving raw credentials, so raw secret values are neither loaded nor hashed. recomputePipelineEffectiveDigest is the single package-owned refresh point for later runtime expansion.

Test Surfaces

pipeline_configkit_test.go exercises real imports and fixed-path profiles, base selection, unselected-overlay isolation, source attribution, and Narratio's unique artifact/family base-declaration rule. Existing focused configuration tests protect defaults, explicit-value semantics, strict decoding, validation, metadata, and root-relative path behavior. Application configuration-loader tests protect the single-read boundary by changing the pipeline file after its initial load and confirming local session resolution retains the original pipeline. Other configuration tests continue to protect defaults and validation after assembly. party_test.go protects the versioned party schema, domain invariants, and deterministic players projection without involving campaign or runtime wiring. party_resolution_test.go protects campaign-owned party loading, canonical input restrictions, source provenance, and virtual players input selection.

Artifact Family Resolution

Pipeline loading retains scriptorium.artifact_families as a resolution-only declaration. Once campaign party resolution establishes a canonical roster, configuration expands families in sorted family-key and character-ID order into ordinary ScriptoriumArtifactConfig values. The expansion owns the narrow {character_id} output substitution, closed member-variable selectors, key and output collision checks, and the runtime-only family-origin catalog. It then removes family declarations from ScriptoriumConfig, runs ordinary Scriptorium validation, and refreshes the effective pipeline digest. Stages and adapters therefore receive only concrete artifact maps.

The catalog retains sorted family member keys plus family/character/source origins and the typed dependency/publish declarations for their later owners. member_dependencies add corresponding ordinary concrete dependencies, while the family-only narratio.member_artifact.<family> input form is rewritten to the matching ordinary configured-artifact source. The catalog records those resolved dependency and input identities with their declaring family and party member. No member-artifact source is registered as a runtime policy source. An enabled family publish declaration expands to ordinary configured-artifact publish rules before the existing publish and lock validators run. Runtime publication consequently receives no family wildcard or special matcher.