ConfigKit/docs/roadmap/source-version-checks.md

Shared Source Version Checks Roadmap

Status: design decisions settled; implementation has not started. The implementation plan owns execution order, concrete API and integration decisions, and completion evidence. This roadmap develops the partially accepted source-version request in PromptKit's proposal using the consumer's single Definition.Version policy. The design direction below incorporates the maintainer's subsequent decisions on lookup, consumption, inspection, and declaration handling; it supersedes the proposal's earlier open-ended discussion of those subjects for this work.

Targeted end state

A consumer can require that source documents contributing to a configuration declare a version accepted by its existing Definition.Version checker. There is one consumer-owned configuration specification and one acceptance policy. Different accepted declarations may coexist: ConfigKit does not require exact equality with the main declaration, impose SemVer, or invent an expected value.

The consumer selects a load-wide enum on ordinary and layered options. Its zero value retains current behavior; the implementation plan selects exported names. The same Definition.Version can be reused across loads with different strictness. The three modes are:

Mode	Missing non-main declaration	Present non-main declaration
Current behavior, the default	Existing behavior	Existing behavior, including protected-path restrictions
Check when present	Proceed without a source-version callback for that omission	Apply the shared version vocabulary and checker
Require declarations	Structured load error	Apply the shared version vocabulary and checker

The main declaration remains part of the configuration and retains its existing checker policy. The treatment of missing main input and mounted main paths is addressed explicitly below rather than silently replacing existing behavior. An enabled mode requires a configured version rule and checker; select a normal options error for an unusable rule, rather than silently ignoring the mode.

Source-local lookup and shared policy

In enabled modes, interpret the configured header path relative to each source document before mounting. Read that document's own declaration, not a composed or overlaid value. An import cannot supply its parent's missing declaration, and another source or later tier cannot hide a rejected declaration. Check acquired participating documents even if their application payload is later superseded. A rejection fails the load with no usable partial result.

Reuse the existing version vocabulary: absence, explicit null, strings, and strict decimal integers. Null is present and reaches the checker as null; it is not an omission. A required declaration need not be non-null if the consumer's checker accepts null. Unsupported scalar/container kinds remain present but fail the existing kind of version-type validation before invoking the checker. Do not coerce malformed values into strings or treat them as missing. Preserve exact string/integer text and the existing absence of a machine-integer range requirement. No new string-versus-integer discriminator is required by this scope.

This checks declared compatibility. Final decoding and domain validation still validate the assembled application; fragments are not independently decoded as complete applications, and declarations do not prove schema conformance.

Header consumption and evidence

After a non-main declaration passes its check, consume that declaration as structural source metadata before its application content participates in composition. Preserve the main declaration and existing main value semantics. Do not consume unrelated nested versions, imported payload fields at other paths, or declarations in the default current-behavior mode.

For example, with a source-local header path version, a non-main document's version: "1" is checked and consumed even when that document will be mounted under collections/weather. A local release label at that same path cannot simultaneously remain ordinary payload in an enabled mode. Applications must choose a distinct location for release metadata or for their specification header. A prompt's version below a separate prompt key is unaffected unless that is the configured header path.

Retain each checked declaration, its source-local path, and its physical source location as inspectable evidence. Keep this evidence separate from effective domain values and their composition history; consumed headers should not appear as surviving fields or create effective-value diff changes. Source destination and resource context remain available through existing inspection. Missing declarations need an explicit absence state and an appropriate source-location fallback for diagnostics, not an invented authored header coordinate.

Preserve normal safe diagnostics, warning behavior, callback lifetime, and disclosure rules. Evidence will be attached to existing source inspection as a detached declaration record, distinguishing unchecked sources, checked absence, present null/text, and redaction, with local path and physical location. The agreed sensitivity rules below govern disclosure; retaining evidence does not authorize unredacted exposure of sensitive values or arbitrary malformed input.

Participation and unchanged boundaries

Document participation must be explicit, independent of filesystem naming or collection identity. The agreed scope is:

Contribution	Treatment
Selected main document	Shared checker; declaration retained; no duplicate main callback
Direct underlay documents	Check under the selected mode; accepted headers consumed
Imported documents, including transitive imports and glob matches	Same policy and mode; accepted headers consumed
Selected overlay document	Same policy and mode; accepted header consumed before overlay application
Host files, native filesystem files, bytes, readers, and mapping sources	Equivalent document participation under their existing source contracts
CLI field overrides	Synthetic field operations; no document header requirement
Typed defaults, resolution operations, inheritance copies, and placement wrappers	Not newly acquired documents; no additional header checks
NoSource, unmatched optional imports, and empty groups	No acquired document and no new header requirement

A present empty document mapping is still a document. Reuse of a source in allowed separate tiers retains existing acquisition semantics; do not introduce a global once-only check or a new uniqueness policy.

Existing source acquisition, containment, import destinations, strict composition, precedence, inheritance, and complete-on-return loading continue to apply. Consumption permits the checked header to serve as metadata; it must not allow remaining payload to bypass main-version ownership. Synthetic overrides remain subject to their existing protected-path restrictions.

This feature does not add independently versioned collections or subtrees, per-input compatibility policies, automatic schema migration, YAML-selected callbacks, or general source-validation hooks. These are outside this delivery, not new permanent prohibitions on future work.

Agreed main-path and consumption behavior

In enabled modes, locate main's header source-locally, retain it, and derive its effective protected path by prefixing main's destination. Use that effective path consistently for protection, existing typed version conversion, and inspection. Invoke the shared checker once for main. Preserve its existing acceptance policy for absence; required mode additionally rejects a missing header when an actual main document participates. NoSource retains the existing absent-main checker behavior because it is not a document.

Default mode keeps the existing effective-path interpretation. Root-mounted main inputs need no path adjustment; mounted-main callers opting in must account for the enabled mode's source-local interpretation. No separate main-version path or root-only main restriction is introduced.

For nested mapping headers, consume only the selected non-main declaration and retain authored ancestor containers, including newly empty mappings. This avoids removing additional input presence or changing unrelated default/composition behavior. Sequence-element and structural-directive behavior is settled below.

Current baseline

Reviewed worktree baseline: b3fc6a5. Input destinations and relative import destinations are implemented; the relative import roadmap records completion.

• VersionRule and Version provide a path, a checker, and absent/null/text states. There is no expected-version string or source-check mode. Root facade conversions preserve the public/internal boundary.
• The version contract checks the main version before defaults and typed decoding. The current path is an effective application path; changing its interpretation in enabled modes needs explicit handling for mounted main documents.
• Loading invokes the main check after main acquisition/composition. Version validation already handles scalar vocabulary, checker diagnostics, and safe callback errors. It is a useful shared mechanism, not a ready-made source-check API.
• Imports, underlays, and selected overlays currently reject contributions to the protected version path. Enabled source checks need to extract and consume headers before payload protection and composition, without weakening the default behavior or checking only the eventual winning values.
• Existing source records carry identity, locations, and destination, but no dedicated checked-version evidence. Existing sensitivity and inspection are primarily organized around application paths and retained application trees.

Acceptance outcomes

The completed feature supports all three modes through ordinary and layered public loading, with these observable outcomes:

• One checker can accept several distinct source declarations. Rejected acquired declarations fail the load even when later tiers would replace their payload.
• Absence, explicit null, strings, exact decimal integers, and malformed selected values follow the shared version vocabulary and selected mode.
• Main retains its own declaration and is checked once; mounted main uses the enabled mode's source-local interpretation, while NoSource and default mode retain their documented behavior.
• Every participating document role and supported acquisition kind follows the same selected policy. Synthetic field operations and absent sources add no document-header requirement.
• Accepted non-main headers are consumed exactly, preserving authored empty containers and ordinary sequence-removal semantics. Remaining payload retains strict composition, precedence, protected-path rules, and ordinary versions outside the selected header path.
• Source evidence is detached, physically attributed, available without reopening files, and subject to the agreed sensitivity rules. Warnings survive successful checks; generated failure messages remain safe.
• Effective values, presence, provenance, history, inheritance, rendering, diffs, and resource contexts agree with remaining application payload. Consumed declarations remain separate source evidence; main history stays ordinary.
• Maintained public examples demonstrate gradual and required adoption with one policy. Current API/configuration documentation describes the delivered contract, and existing default-mode behavior remains covered.

Independent specification policies, release preparation, and publication are outside this feature's completion criteria.

Agreed Edge-Case Decisions

The maintainer has accepted all three decisions below. Together with the mode API location, source participation, main-path handling, source-record evidence, and nested mapping ancestor preservation above, these settle the discussed product decisions. The implementation plan selects public names and integration mechanics; the feature is not yet implemented.

Sensitivity of consumed declaration evidence

Classify declaration evidence using its original source shape at its mounted application path, including classified ancestors. Preserve the header as private evidence long enough to apply applicable existing field-tag, predicate, historical, and recorded-dependency sensitivity rules; consumption must not make a classified declaration public. Do not require the consumed path to survive in final effective metadata. Use explicit redaction state and omit the classified text from public evidence while retaining presence, source attribution, and the fact that it was checked. The checker still receives the declaration as trusted application code, as the existing version checker does.

For example, a header meta.version mounted at collections.weather is masked if either that header path or an ancestor such as collections.weather.meta is classified. An unclassified header remains inspectable. A later override must not erase applicable historical classification. Sequence evidence uses original source coordinates and segment kinds, not indexes shifted by consumption.

On failed loads, retain the existing safe diagnostic boundary: do not include raw header text in generated failure messages. If checked evidence is exposed through load errors before full sensitivity is available, mask its text unless applicable classification can already be established safely; this does not require a new failed-load evidence API or continuing a failed load to finish classification.

Headers that are sequence elements

Support consumption of a selected sequence element by deleting that element, retaining the sequence and its order among remaining elements. Later indexes shift in the remaining payload; an emptied sequence remains []. This is ordinary removal, not a merge strategy. Retain the header's original index and physical location in source evidence; downstream payload paths reflect the post-consumption sequence. Main's retained declaration is not deleted. Removing a mapping member inside a sequence element instead retains that element, including an empty mapping, under the agreed ancestor-preservation rule.

This gives exact-node consumption a concrete meaning without restricting existing path forms. Verify protected-path and typed behavior where main retains an element but non-main payloads have shifted indexes; do not invent alignment or bypass normal sequence replacement, decoding, or ownership rules to make such inputs fit.

Overlap with structural directives

Reject an actual overlap between a checked header and an existing structural directive node with a structured error in enabled modes. A value must not simultaneously serve as a consumed specification declaration and as import or inheritance control. Include a header selected inside an import declaration or at an extends control; preserve ordinary structural validation so malformed controls cannot disappear through consumption. Use source-local paths and real locations for these errors. This is an explicitly accepted, narrow restriction for enabled source checks, not a general prohibition on nested header paths.

Limit the rule to structural overlap, not substrings in ordinary keys. A header at metadata.version is not blocked merely because its source also contains configkit.imports, and a key such as extends_version is not an extends control. Missing headers remain governed by the selected mode; existing invalid directives still fail independently.