diff --git a/docs/roadmap/sources-and-layering.md b/docs/roadmap/sources-and-layering.md index 66c055d..77934cf 100644 --- a/docs/roadmap/sources-and-layering.md +++ b/docs/roadmap/sources-and-layering.md @@ -52,8 +52,8 @@ these proposals. clear existing children; null is an explicit value, not omission. - An explicitly configured schema-version path currently belongs to the selected source. Its checker runs before overlay/CLI input, which cannot alter that path; - imported fragments cannot contribute it. The target below deliberately revises - those ownership and timing rules. + imported fragments cannot contribute it. This roadmap preserves those ownership + and timing rules. - CLI overrides already convert supported programmatic scalars, lists, and string-keyed maps directly to the private document representation. They have CLI-specific paths, labels, restrictions, and origins, not a general source API. @@ -69,8 +69,7 @@ these proposals. Existing source/lifecycle contracts, ownership, secret handling, and diagnostics remain the compatibility baseline. This work is not permission to silently reinterpret current `Load`, imports, selected overlays, or CLI overrides. -Effective-version validation is an explicit exception described below, not an -unannounced consequence of refactoring. +Redesigning schema-version policy is outside this roadmap's scope. ## Target source model @@ -138,6 +137,9 @@ input uses the same presence and combination rules. Explicitly choosing compose, overlay, or replacement expresses different intentions; merely changing the input medium must not switch between them. This applies to both ordinary `Load` and the new layered-loading interface wherever they offer the same operation. +The existing opt-in version-path ownership rule remains an explicit metadata +constraint, independent of whether its owning source is a file or programmatic +input; it is not a source-format-dependent merge rule. Keep strict composition and intentional overriding visibly distinct: @@ -153,8 +155,8 @@ precedence is applied. An overlay must not hide malformed YAML or a composition conflict in an earlier contribution. There is no automatic matching by an `id` field, implicit sequence merge, universal profile registry, or merge callback. -Assemble the effective input, including selected-overlay and CLI contributions, -then check its configured schema version before typed defaults/decoding, +Preserve version checking after base composition and before intentional overlays +and CLI contributions. Assemble the effective input before typed defaults/decoding, application resolution, secret resolution, semantic validation, and final snapshotting. Each lifecycle phase runs once per load, not once per layer. Higher layers may replace values that would not fit the final typed schema; @@ -181,40 +183,34 @@ supply the final value. The metadata then attributes `retries` to defaults, not to the replaced input. An empty replacement entry likewise removes lower-input children without suppressing applicable typed defaults. -### Effective-configuration schema version +### Preserve existing schema-version semantics An application opts into version checking by configuring a version path. A key -spelled `version` has no automatic special status. The configured path follows -the selected combination operation like any other setting: +spelled `version` has no automatic special status. When checking is configured, +one designated input owns that declaration; fragments cannot contribute it and +overlays or CLI overrides cannot alter it, even to an equal value. New layering +operations must respect that ownership rather than bypass it through subtree +replacement. Preserve the current version-path and ancestor protections. -- Strict composition rejects conflicting scalar definitions, even if equal; - it is not last-wins. -- An intentional overlay can replace an earlier version value. Imports may - contribute the version subject to ordinary composition rules; selected overlays - and CLI input are not excluded solely because a path is the version path. -- Any selected subtree replacement has the same effect on version presence as - on other values in that subtree, including removing a lower-input version. +Keep the checker at its existing lifecycle point: after base composition and +schema construction, before intentional overlays, CLI overrides, defaults, and +typed decoding. Preserve supported scalar forms, absent/null distinction, +diagnostics, failure propagation, and version-path integer-to-string decoding. +An absent declaration remains absent for the checker; an invalid version type +fails without invoking it. Later input cannot mask a version-check failure. -After all input precedence operations, inspect the effective version and invoke -the configured checker once, before typed decoding. Preserve the existing -supported scalar forms, absent/null distinction, version diagnostics, and -version-path integer-to-string decoding behavior. As today, an invalid effective -version type produces a diagnostic rather than invoking the checker. Absent -effective input is reported as absent, not manufactured from typed defaults. -Version origins and replacement history follow the same rules as other fields. +Filesystem sources and layered loading do not require a different version +policy. The layered API needs a way to identify the version-owning input when a +`VersionRule` is configured; Decision 4 concerns that integration only. Without +version checking, no version owner or synthetic primary document is required. +Named-overlay lookup context is a separate concern, not a reason to impose +version ownership on unversioned loads. -Do not check each source's version independently or fail merely because a -superseded version value would fail effective-version validation. Invalid YAML, -source failures, and strict-composition conflicts still fail at their owning -phases; precedence does not conceal those errors. A later version value does not -migrate earlier data. Consumers supply compatible inputs and own schema meaning; -automatic migration and per-document format interpretation remain out of scope. - -Apply this model consistently to existing and new loading interfaces. It -deliberately changes the current selected-source ownership restrictions and moves -checking after overlay/CLI application. Document the resulting compatibility -impact and callback timing in the feature's minor release rather than retaining -different version semantics for old and new entry points. +Consumers continue to define version meaning and acceptance. Effective-version +last-wins validation, independent per-source version checks, schema selection, +and migration are not added by this feature. A future concrete need can motivate +a separate version-policy proposal; preserving the current contract here does +not settle that broader question permanently. ### Programmatic presence @@ -303,9 +299,9 @@ These are responsibility areas, not implementation stages: with ordinary loading rather than building a parallel configuration system. - Resolve the fixed operation and ownership decisions below, then extend source records, histories, diagnostics, and rendering consistently across input kinds. -- Replace selected-source version ownership with the effective-version lifecycle - across both loading interfaces, including import, overlay, and CLI handling. - Keep filesystem-context selection independent of version checking. +- Integrate version-owner selection into layered loading while preserving the + existing version lifecycle and protections. Keep filesystem-context selection + independent of version checking. - Refactor mapping assembly and provenance updates where current algorithms would scale poorly under layered workloads. Keep optimization scope tied to the new workflow rather than rewriting unrelated library subsystems. @@ -317,8 +313,9 @@ These are responsibility areas, not implementation stages: - Compare equivalent logical inputs across supported source kinds and loading interfaces. Assert consistent values, presence, operation outcomes, and origin meaning while allowing genuine location/label differences. Cover defaulted and - derived values, replacement followed by typed defaults, version conflicts under - composition, and effective-version checking after intentional overrides. + derived values, replacement followed by typed defaults, and preservation of + version ownership/check timing across both loading interfaces. Verify that + unversioned layered loads need no synthetic version-owning root. - Provide maintained simple and PromptKit-shaped examples: embedded defaults, operator configuration from another context, and labeled programmatic input. Demonstrate exact operations and resource origins without provider calls or @@ -330,10 +327,10 @@ These are responsibility areas, not implementation stages: as behavior lands. Record significant boundary changes in an ADR without rewriting accepted ADR history. Keep named inheritance explicitly separate. -Plan a minor release for the new public capability and the deliberate change to -schema-version ownership and checking order. Explain the version-checking change -and provide migration guidance; this release is not wholly additive. Identify any -other compatibility changes expressly rather than calling them optimizations. +Plan an additive minor release for the new public capability, preserving current +version semantics. If another design choice requires changing documented behavior, +identify that change expressly and provide migration guidance rather than calling +it an internal optimization. No release, implementation change, or current-policy revision occurs in this pass. ## Completion criteria @@ -343,8 +340,8 @@ with independently located operator configuration and explicit programmatic values, and obtain one typed result with accurate presence, sources, history, and resource context. No temporary configuration files or serialization round trip or subsequent generic configuration processing is needed. Simple `Load` -usage retains its behavior except for the explicitly selected version-policy -change, which is shared by both entry points. Final-value provenance covers input, +usage retains its current behavior, including version ownership and validation +timing. Layered loading uses the same version policy. Final-value provenance covers input, defaults, and recorded domain resolution, and metadata distinguishes explicit zero/false from absence even when the typed values are equal. Benchmarks demonstrate that adding inputs does not entail avoidable full-collection copying @@ -400,3 +397,20 @@ registry mapping IDs to their filesystems. Metadata stays capability-free and serializable, but every consumer must maintain and correctly reconcile another registry. This is viable if limiting retained filesystem capabilities proves more important than the additional integration machinery. + +### 4. Selecting the version-owning input for layered loads + +**Recommendation:** let callers explicitly designate one of the supplied base +inputs as the version owner when configuring a `VersionRule`. It need not be a +host file or a newly manufactured root document. Other contributions retain +fragment/override roles with respect to that path. Explicit designation avoids +making version ownership depend on incidental list order and preserves the +existing selected-source model without changing version acceptance or timing. +No designation is needed when version checking is not configured. + +**Alternative:** make the first base input the version owner when checking is +configured. This reduces API ceremony and naturally extends a main-source-plus- +fragments model, but couples ownership to ordering and can make reordering +otherwise independent contributions consequential. Either choice must preserve +the existing version contract; this decision does not reopen effective-version +or per-document version validation.