From 2c91e41d10a74b7007cb48832ae09a77e1183a4e Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Wed, 14 Oct 2026 20:30:58 +0000 Subject: [PATCH] Record source version decisions and remaining edge cases --- docs/roadmap/source-version-checks.md | 181 ++++++++++++++++---------- 1 file changed, 109 insertions(+), 72 deletions(-) diff --git a/docs/roadmap/source-version-checks.md b/docs/roadmap/source-version-checks.md index 19bbb6b..5ed7507 100644 --- a/docs/roadmap/source-version-checks.md +++ b/docs/roadmap/source-version-checks.md @@ -16,7 +16,10 @@ is one consumer-owned configuration specification and one acceptance policy. Different accepted declarations may coexist: ConfigKit does not require exact equality with the main declaration, impose SemVer, or invent an expected value. -The consumer selects one of three modes through the supported loading API: +The consumer selects a load-wide enum on ordinary and layered options. Its zero +value retains current behavior; exact exported names remain implementation design +work. The same `Definition.Version` can be reused across loads with different +strictness. The three modes are: | Mode | Missing non-main declaration | Present non-main declaration | | --- | --- | --- | @@ -77,16 +80,18 @@ declarations need an explicit absence state and an appropriate source-location fallback for diagnostics, not an invented authored header coordinate. Preserve normal safe diagnostics, warning behavior, callback lifetime, and -disclosure rules. The public evidence representation and treatment of classified -declarations need the design decision below; retaining evidence does not authorize +disclosure rules. Evidence will be attached to existing source inspection as a +detached declaration record, distinguishing unchecked sources, checked absence, +present null/text, and redaction, with local path and physical location. The +remaining sensitivity decision below governs disclosure; retaining evidence does not authorize unredacted exposure of sensitive values or arbitrary malformed input. ### Participation and unchanged boundaries Document participation must be explicit, independent of filesystem naming or -collection identity. The recommended scope is: +collection identity. The agreed scope is: -| Contribution | Recommended treatment | +| Contribution | Treatment | | --- | --- | | Selected main document | Shared checker; declaration retained; no duplicate main callback | | Direct underlay documents | Check under the selected mode; accepted headers consumed | @@ -99,8 +104,7 @@ collection identity. The recommended scope is: A present empty document mapping is still a document. Reuse of a source in allowed separate tiers retains existing acquisition semantics; do not introduce -a global once-only check or a new uniqueness policy. The participation boundary -and mode API remain recommendations to settle in stage 1. +a global once-only check or a new uniqueness policy. Existing source acquisition, containment, import destinations, strict composition, precedence, inheritance, and complete-on-return loading continue to apply. @@ -113,6 +117,26 @@ per-input compatibility policies, automatic schema migration, YAML-selected callbacks, or general source-validation hooks. These are outside this delivery, not new permanent prohibitions on future work. +### Agreed main-path and consumption behavior + +In enabled modes, locate main's header source-locally, retain it, and derive its +effective protected path by prefixing main's destination. Use that effective +path consistently for protection, existing typed version conversion, and +inspection. Invoke the shared checker once for main. Preserve its existing +acceptance policy for absence; required mode additionally rejects a missing +header when an actual main document participates. `NoSource` retains the existing +absent-main checker behavior because it is not a document. + +Default mode keeps the existing effective-path interpretation. Root-mounted main +inputs need no path adjustment; mounted-main callers opting in must account for +the enabled mode's source-local interpretation. No separate main-version path or +root-only main restriction is introduced. + +For nested mapping headers, consume only the selected non-main declaration and +retain authored ancestor containers, including newly empty mappings. This avoids +removing additional input presence or changing unrelated default/composition +behavior. Sequence-element and structural-directive cases remain open below. + ## Current baseline Reviewed worktree baseline: `b3fc6a5`. Input destinations and relative import @@ -144,13 +168,13 @@ destinations are implemented; the ### 1. Finalize the contract and integration design -Resolve the Open Decisions below. Select public mode names and placement, source -participation, main-path handling, and evidence types. Define header lookup for +Resolve the Open Decisions below. Select exact public mode and evidence type +names within the agreed load-wide API, participation, main-path, and inspection +design. Define header lookup for nested paths and distinguish a missing member from an obstructing non-container ancestor. Specify consumption without broadening it into a generic transformation -API. Recommend removing only the selected header node and preserving its authored -ancestor containers, including newly empty mappings; decide handling of overlaps -with structural directives before implementation. +API. Preserve authored ancestors when consuming nested mapping headers as agreed; +settle sequence-element consumption and structural-directive overlaps below. Record the change to main-only version ownership in an ADR and review public API compatibility under repository policy. Do not rewrite accepted ADR decisions or @@ -245,74 +269,87 @@ required to complete this feature. ## Open Decisions -### Mode API and document participation +These are recommendations for further discussion, not accepted restrictions or +implemented behavior. The mode API location, source participation, main-path +handling, source-record evidence, and nested mapping ancestor preservation above +have been agreed. -Recommendation: expose a load-wide enum on ordinary and layered options, with -zero retaining current behavior, and use the participation table above. This -allows one reusable `Definition.Version` policy to be used with different adoption -strictness on different loads. Include mapping sources and selected overlays: -they contribute documents even though they are not necessarily imported files. -Exclude synthetic field operations and absent sources as described above. +### Sensitivity of consumed declaration evidence -Alternative: place the mode on `VersionRule` so policy and enforcement travel -together. This reduces option plumbing but requires changing/copying the rule -to vary enforcement. A narrower first implementation checking imports and -underlays but exempting overlays is also viable; it reduces integration work but -offers weaker assurance about all contributing documents and needs an explicit -documented exception. Neither choice should imply per-source independent policies. +Recommendation: classify declaration evidence using its original source shape +at its mounted application path, including classified ancestors. Preserve the +header as private evidence long enough to apply applicable existing field-tag, +predicate, historical, and recorded-dependency sensitivity rules; consumption +must not make a classified declaration public. Do not require the consumed path +to survive in final effective metadata. Use explicit redaction state and omit the +classified text from public evidence while retaining presence, source attribution, +and the fact that it was checked. The checker still receives the declaration as +trusted application code, as the existing version checker does. -### Mounted main documents and the protected application path +For example, a header `meta.version` mounted at `collections.weather` is masked +if either that header path or an ancestor such as `collections.weather.meta` is +classified. An unclassified header remains inspectable. A later override must +not erase applicable historical classification. Sequence evidence uses original +source coordinates and segment kinds, not indexes shifted by consumption. -Recommendation: in enabled modes, locate main's header source-locally as requested, -retain it, and derive its effective protected path by prefixing main's destination. -Use that derived path consistently for protection, existing typed version -conversion, and inspection. Invoke the shared checker once for main, preserving -its existing acceptance policy for absence; required mode additionally rejects a -missing declaration when an actual main document participates. `NoSource` retains -the existing absent-main checker behavior, since it is not a document. -Default mode keeps today's effective-path interpretation unchanged. This supports -mounted main input without contradicting source-local lookup or losing protection. +On failed loads, retain the existing safe diagnostic boundary: do not include raw +header text in generated failure messages. If checked evidence is exposed through +load errors before full sensitivity is available, mask its text unless applicable +classification can already be established safely; this does not require a new +failed-load evidence API or continuing a failed load to finish classification. -Alternative: separate the public source-header path from the existing effective -main-version path while still using one checker. It makes both coordinate systems -explicit and can ease migration for mounted main callers, but adds configuration -and two paths that must be kept coherent. Root-only main input would simplify the -problem but would restrict an existing capability; do not impose that restriction -as an implementation shortcut. +Alternative: always redact declaration text in public source evidence. This +simplifies classification and prevents disclosure, but hides ordinary version +identifiers that the consumer explicitly wants to inspect. Conversely, treating +headers as inherently public would be simpler but would bypass existing +classification where the configured header path or its ancestor is sensitive. +Prefer normal classification rather than either blanket rule. -### Evidence representation and sensitivity +### Headers that are sequence elements -Recommendation: attach a detached declaration record to existing source -inspection, with local path, presence/null/text state, physical location, and -explicit redaction state. Reuse existing sensitivity machinery where applicable -to the source's mounted header path, retaining classification evidence before -consumption so removing a node does not erase its sensitivity. Main evidence and -source evidence should use consistent disclosure. Retain actual declaration text -privately when needed; inspectable sensitive evidence can be redacted without -denying that a declaration was checked. Specify handling for historical tags and -dependency classification before choosing storage, rather than exposing raw -`Version.Text` unconditionally. +Recommendation: support consumption of a selected sequence element by deleting +that element, retaining the sequence and its order among remaining elements. +Later indexes shift in the remaining payload; an emptied sequence remains `[]`. +This is ordinary removal, not a merge strategy. Retain the header's original +index and physical location in source evidence; downstream payload paths reflect +the post-consumption sequence. Main's retained declaration is not deleted. +Removing a mapping member inside a sequence element instead retains that element, +including an empty mapping, under the agreed ancestor-preservation rule. -Alternative: expose declarations through a dedicated source query returning safe -snapshots instead of extending source records. This better accommodates on-demand -redaction and richer evidence, but adds an API and lookup semantics. Both choices -must preserve physical attribution and the requested declaration evidence; neither -should expose arbitrary malformed input or require reopening the source. +This gives exact-node consumption a concrete meaning without restricting existing +path forms. Verify protected-path and typed behavior where main retains an element +but non-main payloads have shifted indexes; do not invent alignment or bypass +normal sequence replacement, decoding, or ownership rules to make such inputs fit. -### Nested consumption and structural directives +Alternative: reject headers whose selected node is a sequence element in enabled +modes, while permitting mapping-member headers nested inside sequences. This +avoids index shifts and may be easier for consumers to reason about, but narrows +supported header locations and is not required for correctness if deletion is +specified. Replacing the element with null would preserve indexes but fabricate a +payload value and could change decoding; it is not the recommended interpretation +of consumption. -Recommendation: remove only the checked non-main declaration and retain authored -ancestor containers, even when empty. Validate source structure and interpret -existing directives so header consumption cannot conceal an invalid directive or -silently cancel an import. Preserve the existing path vocabulary where its -semantics apply; explicitly work through sequence-index and structural-path -interactions before implementation instead of adding a blanket top-level-only -header restriction. Retaining ancestors best preserves input presence outside -the requested consumed value. +### Overlap with structural directives -Alternative: prune ancestor containers emptied by consumption. This can make a -nested metadata-only header disappear completely from application payload, but -also removes authored empty-container presence and can affect defaults and -composition. A dedicated top-level header location would simplify these cases, -but narrows the existing configurable-path model and would need an explicit -consumer requirement or design decision rather than an implementation shortcut. +Recommendation: reject an actual overlap between a checked header and an existing +structural directive node with a structured error in enabled modes. A value must +not simultaneously serve as a consumed specification declaration and as import +or inheritance control. Include a header selected inside an import declaration +or at an `extends` control; preserve ordinary structural validation so malformed +controls cannot disappear through consumption. Use source-local paths and real +locations for these errors. This is a proposed semantic restriction to approve, +not an existing general prohibition on nested header paths. + +Limit the rule to structural overlap, not substrings in ordinary keys. A header +at `metadata.version` is not blocked merely because its source also contains +`configkit.imports`, and a key such as `extends_version` is not an `extends` +control. Missing headers remain governed by the selected mode; existing invalid +directives still fail independently. + +Alternative: interpret structural directives first, then allow the same node to +serve as a checked and consumed header. This can be defined and avoids a new +path restriction, but gives one value two unrelated roles: for example, an +accepted version string could also name an import file or inheritance parent. +It also needs extra ordering and retained-control rules to keep consumption from +canceling behavior. Prefer an explicit overlap error over that dual interpretation; +there is no identified consumer need for sharing these roles.