Plan filesystem sources and underlays

docs/adr/0007-source-backed-underlays.md

@@ -0,0 +1,118 @@
+# ADR 0007: Add Source-Backed Underlays with Consistent Recursive Precedence
+
+## Status
+
+Proposed draft for review. The
+[filesystem sources and underlays roadmap](../roadmap/sources-and-layering.md)
+owns implementation status, detailed target behavior, and remaining API choices.
+
+This extends [ADR 0001](0001-separate-composition-and-overlay.md) and preserves
+its distinction between strict composition and intentional overlay.
+
+## Date
+
+2026-09-29
+
+## Context
+
+Configuration-backed defaults provide lower-priority settings that applications
+can override. These sources carry imports, locations, and relative-resource
+contexts. Integrating them into ConfigKit's loading lifecycle gives their values
+the same presence tracking and inspectable provenance as other configuration
+inputs.
+
+The [architecture goals](../policy/architecture.md#success-criteria) prioritize
+one coherent, predictable configuration model with consistent rules across
+source representations and loading interfaces. Consumers should load once,
+receive a complete validated configuration, and inspect its final values' origins.
+Reducing integration code is important subject to those principles.
+
+## Decision
+
+### Add underlays to the existing precedence model
+
+Introduce explicitly supplied source-backed underlays between typed defaults
+and main configuration. The proposed precedence, from lowest to highest, is:
+
+**Typed application defaults < source underlays < strictly composed main
+configuration < selected overlay < explicit CLI overrides.**
+
+Typed defaults remain the Go-value baseline. Underlays supply lower-priority
+configuration with source provenance and, where applicable, imports and resource
+context. This ordering describes value precedence; typed defaults fill fields
+absent from the assembled input under the existing defaulting rules.
+
+Preserve strict composition of the main document and its imports before applying
+the resulting configuration over underlays. Independent contributions within an
+underlay tier also compose strictly. Explicitly ordered tiers express intentional
+precedence. Strict composition reports conflicting contributions within each tier.
+
+Across precedence tiers, reuse the existing recursive mapping-overlay semantics.
+Omitted children retain lower-priority input; existing scalar, sequence, null,
+and cross-kind replacement semantics remain unchanged. Reusing these operations
+keeps the combination rules consistent across the loading model.
+
+### Keep source representation separate from composition semantics
+
+Native `fs.FS` and programmatic inputs join the shared loading model.
+Equivalent inputs using the same
+operation have the same presence and precedence semantics regardless of medium.
+Programmatic values enter directly as labeled configuration sources.
+
+Retain the contributing source's identity and resource context through layering.
+A retained underlay value keeps its originating source's resource context.
+Consumers remain responsible for opening and interpreting their domain resources
+under the supplied filesystem's lifetime and access semantics. Exact
+context-access and input APIs remain roadmap decisions.
+
+### Deliver one final configuration and its inspection metadata
+
+Assemble effective input, then run the typed loading lifecycle once for the load.
+Preserve explicit zero,
+false, empty, and null input versus omission through configuration and metadata
+together.
+
+Extend existing provenance and history through underlays, defaults, and recorded
+application resolution. The returned result includes the completed combination,
+defaulting, and provenance work. Domain resolution and validation
+remain application-owned within the recorded lifecycle, consistent with
+[ADR 0003](0003-preserve-presence-and-resolution-history.md).
+
+Preserve the existing opt-in schema-version ownership, path protections, and
+check timing before precedence
+assembly and typed decoding. The roadmap owns how layered input identifies that
+owner when version checking is configured.
+
+Favor near-linear processing in input, retained history, and output size through
+incremental assembly of load-owned state. This is an engineering objective
+subordinate to clarity and semantic consistency. Preserve the root-owned facade and
+private implementation boundary established by [ADR 0006](0006-public-facade.md).
+
+## Alternatives considered
+
+- **Leave bundled defaults to consumers.** Avoids extending loading, but makes
+  consumers reproduce merging, presence tracking, and provenance reconciliation.
+  It does not meet the goal of obtaining a complete configuration in one load.
+- **Offer recursive or atomic combination per layer.** Makes both behaviors
+  explicit and could serve more consumers, but expands the semantic surface and
+  its interactions with presence and provenance without a demonstrated need that
+  outweighs a single consistent rule. A future concrete requirement may justify
+  reconsideration; this feature does not add the option.
+- **Flatten all documents into last-wins layers.** Simplifies ordering but turns
+  accidental main/import conflicts into overrides, weakening the intentional
+  distinction established by ADR 0001.
+
+## Consequences
+
+Consumers gain source-backed defaults within the shared loading lifecycle.
+Strict contributions and intentional precedence remain distinct, with consistent
+combination rules across source media. Presence and provenance remain
+part of the same loading lifecycle as the final values.
+
+Higher-priority mappings retain omitted lower-priority children, so application
+validation assesses the final combined configuration.
+
+Implementation must integrate source context and underlay history into the shared
+engine with attention to assembly and history-retention costs. Current behavior
+documentation will change when the feature lands. The roadmap retains ownership
+of the remaining API decisions.