ConfigKit/docs/roadmap/source-version-checks.md

# Shared Source Version Checks Roadmap

Status: design decisions settled; implementation has not started. The
[implementation plan](implementation.md) owns execution order, concrete API and
integration decisions, and completion evidence. This roadmap develops the
partially accepted source-version request in
[PromptKit's proposal](promptkit-feature-request.md#partially-accepted-addition-shared-source-specification-checks)
using the consumer's single `Definition.Version` policy. The design direction
below incorporates the maintainer's subsequent decisions on lookup, consumption,
inspection, and declaration handling; it supersedes the proposal's earlier
open-ended discussion of those subjects for this work.

## Targeted end state

A consumer can require that source documents contributing to a configuration
declare a version accepted by its existing `Definition.Version` checker. There
is one consumer-owned configuration specification and one acceptance policy.
Different accepted declarations may coexist: ConfigKit does not require exact
equality with the main declaration, impose SemVer, or invent an expected value.

The consumer selects a load-wide enum on ordinary and layered options. Its zero
value retains current behavior; the implementation plan selects exported names.
The same `Definition.Version` can be reused across loads with different
strictness. The three modes are:

| Mode | Missing non-main declaration | Present non-main declaration |
| --- | --- | --- |
| Current behavior, the default | Existing behavior | Existing behavior, including protected-path restrictions |
| Check when present | Proceed without a source-version callback for that omission | Apply the shared version vocabulary and checker |
| Require declarations | Structured load error | Apply the shared version vocabulary and checker |

The main declaration remains part of the configuration and retains its existing
checker policy. The treatment of missing main input and mounted main paths is
addressed explicitly below rather than silently replacing existing behavior.
An enabled mode requires a configured version rule and checker; select a normal
options error for an unusable rule, rather than silently ignoring the mode.

### Source-local lookup and shared policy

In enabled modes, interpret the configured header path relative to each source
document before mounting. Read that document's own declaration, not a composed
or overlaid value. An import cannot supply its parent's missing declaration,
and another source or later tier cannot hide a rejected declaration. Check
acquired participating documents even if their application payload is later
superseded. A rejection fails the load with no usable partial result.

Reuse the existing version vocabulary: absence, explicit null, strings, and
strict decimal integers. Null is present and reaches the checker as null; it is
not an omission. A required declaration need not be non-null if the consumer's
checker accepts null. Unsupported scalar/container kinds remain present but
fail the existing kind of version-type validation before invoking the checker.
Do not coerce malformed values into strings or treat them as missing. Preserve
exact string/integer text and the existing absence of a machine-integer range
requirement. No new string-versus-integer discriminator is required by this scope.

This checks declared compatibility. Final decoding and domain validation still
validate the assembled application; fragments are not independently decoded as
complete applications, and declarations do not prove schema conformance.

### Header consumption and evidence

After a non-main declaration passes its check, consume that declaration as
structural source metadata before its application content participates in
composition. Preserve the main declaration and existing main value semantics.
Do not consume unrelated nested versions, imported payload fields at other
paths, or declarations in the default current-behavior mode.

For example, with a source-local header path `version`, a non-main document's
`version: "1"` is checked and consumed even when that document will be mounted
under `collections/weather`. A local release label at that same path cannot
simultaneously remain ordinary payload in an enabled mode. Applications must
choose a distinct location for release metadata or for their specification
header. A prompt's `version` below a separate prompt key is unaffected unless
that is the configured header path.

Retain each checked declaration, its source-local path, and its physical source
location as inspectable evidence. Keep this evidence separate from effective
domain values and their composition history; consumed headers should not appear
as surviving fields or create effective-value diff changes. Source destination
and resource context remain available through existing inspection. Missing
declarations need an explicit absence state and an appropriate source-location
fallback for diagnostics, not an invented authored header coordinate.

Preserve normal safe diagnostics, warning behavior, callback lifetime, and
disclosure rules. Evidence will be attached to existing source inspection as a
detached declaration record, distinguishing unchecked sources, checked absence,
present null/text, and redaction, with local path and physical location. The
agreed sensitivity rules below govern disclosure; retaining evidence does not
authorize unredacted exposure of sensitive values or arbitrary malformed input.

### Participation and unchanged boundaries

Document participation must be explicit, independent of filesystem naming or
collection identity. The agreed scope is:

| Contribution | Treatment |
| --- | --- |
| Selected main document | Shared checker; declaration retained; no duplicate main callback |
| Direct underlay documents | Check under the selected mode; accepted headers consumed |
| Imported documents, including transitive imports and glob matches | Same policy and mode; accepted headers consumed |
| Selected overlay document | Same policy and mode; accepted header consumed before overlay application |
| Host files, native filesystem files, bytes, readers, and mapping sources | Equivalent document participation under their existing source contracts |
| CLI field overrides | Synthetic field operations; no document header requirement |
| Typed defaults, resolution operations, inheritance copies, and placement wrappers | Not newly acquired documents; no additional header checks |
| `NoSource`, unmatched optional imports, and empty groups | No acquired document and no new header requirement |

A present empty document mapping is still a document. Reuse of a source in
allowed separate tiers retains existing acquisition semantics; do not introduce
a global once-only check or a new uniqueness policy.

Existing source acquisition, containment, import destinations, strict composition,
precedence, inheritance, and complete-on-return loading continue to apply.
Consumption permits the checked header to serve as metadata; it must not allow
remaining payload to bypass main-version ownership. Synthetic overrides remain
subject to their existing protected-path restrictions.

This feature does not add independently versioned collections or subtrees,
per-input compatibility policies, automatic schema migration, YAML-selected
callbacks, or general source-validation hooks. These are outside this delivery,
not new permanent prohibitions on future work.

### Agreed main-path and consumption behavior

In enabled modes, locate main's header source-locally, retain it, and derive its
effective protected path by prefixing main's destination. Use that effective
path consistently for protection, existing typed version conversion, and
inspection. Invoke the shared checker once for main. Preserve its existing
acceptance policy for absence; required mode additionally rejects a missing
header when an actual main document participates. `NoSource` retains the existing
absent-main checker behavior because it is not a document.

Default mode keeps the existing effective-path interpretation. Root-mounted main
inputs need no path adjustment; mounted-main callers opting in must account for
the enabled mode's source-local interpretation. No separate main-version path or
root-only main restriction is introduced.

For nested mapping headers, consume only the selected non-main declaration and
retain authored ancestor containers, including newly empty mappings. This avoids
removing additional input presence or changing unrelated default/composition
behavior. Sequence-element and structural-directive behavior is settled below.

## Current baseline

Reviewed worktree baseline: `b3fc6a5`. Input destinations and relative import
destinations are implemented; the
[relative import roadmap](relative-import-destinations.md) records completion.

- [VersionRule and Version](../../lifecycle.go) provide a path, a checker, and
  absent/null/text states. There is no expected-version string or source-check
  mode. Root facade conversions preserve the public/internal boundary.
- The [version contract](../config.md#application-schema-versions) checks the
  main version before defaults and typed decoding. The current path is an
  effective application path; changing its interpretation in enabled modes
  needs explicit handling for mounted main documents.
- [Loading](../../internal/engine/load.go) invokes the main check after main
  acquisition/composition. [Version validation](../../internal/engine/validation.go)
  already handles scalar vocabulary, checker diagnostics, and safe callback
  errors. It is a useful shared mechanism, not a ready-made source-check API.
- [Imports](../../internal/engine/imports.go),
  [underlays](../../internal/engine/load_request.go), and
  [selected overlays](../../internal/engine/selected_overlay.go) currently reject
  contributions to the protected version path. Enabled source checks need to
  extract and consume headers before payload protection and composition, without
  weakening the default behavior or checking only the eventual winning values.
- Existing source records carry identity, locations, and destination, but no
  dedicated checked-version evidence. Existing sensitivity and inspection are
  primarily organized around application paths and retained application trees.

## Acceptance outcomes

The completed feature supports all three modes through ordinary and layered
public loading, with these observable outcomes:

- One checker can accept several distinct source declarations. Rejected acquired
  declarations fail the load even when later tiers would replace their payload.
- Absence, explicit null, strings, exact decimal integers, and malformed selected
  values follow the shared version vocabulary and selected mode.
- Main retains its own declaration and is checked once; mounted main uses the
  enabled mode's source-local interpretation, while NoSource and default mode
  retain their documented behavior.
- Every participating document role and supported acquisition kind follows the
  same selected policy. Synthetic field operations and absent sources add no
  document-header requirement.
- Accepted non-main headers are consumed exactly, preserving authored empty
  containers and ordinary sequence-removal semantics. Remaining payload retains
  strict composition, precedence, protected-path rules, and ordinary versions
  outside the selected header path.
- Source evidence is detached, physically attributed, available without reopening
  files, and subject to the agreed sensitivity rules. Warnings survive successful
  checks; generated failure messages remain safe.
- Effective values, presence, provenance, history, inheritance, rendering, diffs,
  and resource contexts agree with remaining application payload. Consumed
  declarations remain separate source evidence; main history stays ordinary.
- Maintained public examples demonstrate gradual and required adoption with one
  policy. Current API/configuration documentation describes the delivered
  contract, and existing default-mode behavior remains covered.

Independent specification policies, release preparation, and publication are
outside this feature's completion criteria.

## Agreed Edge-Case Decisions

The maintainer has accepted all three decisions below. Together with the mode
API location, source participation, main-path handling, source-record evidence,
and nested mapping ancestor preservation above, these settle the discussed
product decisions. The [implementation plan](implementation.md) selects public
names and integration mechanics; the feature is not yet implemented.

### Sensitivity of consumed declaration evidence

Classify declaration evidence using its original source shape
at its mounted application path, including classified ancestors. Preserve the
header as private evidence long enough to apply applicable existing field-tag,
predicate, historical, and recorded-dependency sensitivity rules; consumption
must not make a classified declaration public. Do not require the consumed path
to survive in final effective metadata. Use explicit redaction state and omit the
classified text from public evidence while retaining presence, source attribution,
and the fact that it was checked. The checker still receives the declaration as
trusted application code, as the existing version checker does.

For example, a header `meta.version` mounted at `collections.weather` is masked
if either that header path or an ancestor such as `collections.weather.meta` is
classified. An unclassified header remains inspectable. A later override must
not erase applicable historical classification. Sequence evidence uses original
source coordinates and segment kinds, not indexes shifted by consumption.

On failed loads, retain the existing safe diagnostic boundary: do not include raw
header text in generated failure messages. If checked evidence is exposed through
load errors before full sensitivity is available, mask its text unless applicable
classification can already be established safely; this does not require a new
failed-load evidence API or continuing a failed load to finish classification.

### Headers that are sequence elements

Support consumption of a selected sequence element by deleting
that element, retaining the sequence and its order among remaining elements.
Later indexes shift in the remaining payload; an emptied sequence remains `[]`.
This is ordinary removal, not a merge strategy. Retain the header's original
index and physical location in source evidence; downstream payload paths reflect
the post-consumption sequence. Main's retained declaration is not deleted.
Removing a mapping member inside a sequence element instead retains that element,
including an empty mapping, under the agreed ancestor-preservation rule.

This gives exact-node consumption a concrete meaning without restricting existing
path forms. Verify protected-path and typed behavior where main retains an element
but non-main payloads have shifted indexes; do not invent alignment or bypass
normal sequence replacement, decoding, or ownership rules to make such inputs fit.

### Overlap with structural directives

Reject an actual overlap between a checked header and an existing
structural directive node with a structured error in enabled modes. A value must
not simultaneously serve as a consumed specification declaration and as import
or inheritance control. Include a header selected inside an import declaration
or at an `extends` control; preserve ordinary structural validation so malformed
controls cannot disappear through consumption. Use source-local paths and real
locations for these errors. This is an explicitly accepted, narrow restriction
for enabled source checks, not a general prohibition on nested header paths.

Limit the rule to structural overlap, not substrings in ordinary keys. A header
at `metadata.version` is not blocked merely because its source also contains
`configkit.imports`, and a key such as `extends_version` is not an `extends`
control. Missing headers remain governed by the selected mode; existing invalid
directives still fail independently.