eric/ConfigKit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Plan shared source version checks

Browse Source
This commit is contained in:
Eric Rakestraw
2026-10-14 19:03:25 +00:00
parent b3fc6a5950
commit e3860036f8
1 changed files with 318 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

318
docs/roadmap/source-version-checks.md Normal file
View File

@@ -0,0 +1,318 @@
# Shared Source Version Checks Roadmap
Status: planned; implementation has not started. Execute the stages below in
order. This roadmap develops the partially accepted source-version request in
[PromptKit's proposal](promptkit-feature-request.md#partially-accepted-addition-shared-source-specification-checks)
using the consumer's single `Definition.Version` policy. The design direction
below incorporates the maintainer's subsequent decisions on lookup, consumption,
inspection, and declaration handling; it supersedes the proposal's earlier
open-ended discussion of those subjects for this work.
## Targeted end state
A consumer can require that source documents contributing to a configuration
declare a version accepted by its existing `Definition.Version` checker. There
is one consumer-owned configuration specification and one acceptance policy.
Different accepted declarations may coexist: ConfigKit does not require exact
equality with the main declaration, impose SemVer, or invent an expected value.
The consumer selects one of three modes through the supported loading API:
| Mode | Missing non-main declaration | Present non-main declaration |
| --- | --- | --- |
| Current behavior, the default | Existing behavior | Existing behavior, including protected-path restrictions |
| Check when present | Proceed without a source-version callback for that omission | Apply the shared version vocabulary and checker |
| Require declarations | Structured load error | Apply the shared version vocabulary and checker |
The main declaration remains part of the configuration and retains its existing
checker policy. The treatment of missing main input and mounted main paths is
addressed explicitly below rather than silently replacing existing behavior.
An enabled mode requires a configured version rule and checker; select a normal
options error for an unusable rule, rather than silently ignoring the mode.
### Source-local lookup and shared policy
In enabled modes, interpret the configured header path relative to each source
document before mounting. Read that document's own declaration, not a composed
or overlaid value. An import cannot supply its parent's missing declaration,
and another source or later tier cannot hide a rejected declaration. Check
acquired participating documents even if their application payload is later
superseded. A rejection fails the load with no usable partial result.
Reuse the existing version vocabulary: absence, explicit null, strings, and
strict decimal integers. Null is present and reaches the checker as null; it is
not an omission. A required declaration need not be non-null if the consumer's
checker accepts null. Unsupported scalar/container kinds remain present but
fail the existing kind of version-type validation before invoking the checker.
Do not coerce malformed values into strings or treat them as missing. Preserve
exact string/integer text and the existing absence of a machine-integer range
requirement. No new string-versus-integer discriminator is required by this scope.
This checks declared compatibility. Final decoding and domain validation still
validate the assembled application; fragments are not independently decoded as
complete applications, and declarations do not prove schema conformance.
### Header consumption and evidence
After a non-main declaration passes its check, consume that declaration as
structural source metadata before its application content participates in
composition. Preserve the main declaration and existing main value semantics.
Do not consume unrelated nested versions, imported payload fields at other
paths, or declarations in the default current-behavior mode.
For example, with a source-local header path `version`, a non-main document's
`version: "1"` is checked and consumed even when that document will be mounted
under `collections/weather`. A local release label at that same path cannot
simultaneously remain ordinary payload in an enabled mode. Applications must
choose a distinct location for release metadata or for their specification
header. A prompt's `version` below a separate prompt key is unaffected unless
that is the configured header path.
Retain each checked declaration, its source-local path, and its physical source
location as inspectable evidence. Keep this evidence separate from effective
domain values and their composition history; consumed headers should not appear
as surviving fields or create effective-value diff changes. Source destination
and resource context remain available through existing inspection. Missing
declarations need an explicit absence state and an appropriate source-location
fallback for diagnostics, not an invented authored header coordinate.
Preserve normal safe diagnostics, warning behavior, callback lifetime, and
disclosure rules. The public evidence representation and treatment of classified
declarations need the design decision below; retaining evidence does not authorize
unredacted exposure of sensitive values or arbitrary malformed input.
### Participation and unchanged boundaries
Document participation must be explicit, independent of filesystem naming or
collection identity. The recommended scope is:
| Contribution | Recommended treatment |
| --- | --- |
| Selected main document | Shared checker; declaration retained; no duplicate main callback |
| Direct underlay documents | Check under the selected mode; accepted headers consumed |
| Imported documents, including transitive imports and glob matches | Same policy and mode; accepted headers consumed |
| Selected overlay document | Same policy and mode; accepted header consumed before overlay application |
| Host files, native filesystem files, bytes, readers, and mapping sources | Equivalent document participation under their existing source contracts |
| CLI field overrides | Synthetic field operations; no document header requirement |
| Typed defaults, resolution operations, inheritance copies, and placement wrappers | Not newly acquired documents; no additional header checks |
| `NoSource`, unmatched optional imports, and empty groups | No acquired document and no new header requirement |
A present empty document mapping is still a document. Reuse of a source in
allowed separate tiers retains existing acquisition semantics; do not introduce
a global once-only check or a new uniqueness policy. The participation boundary
and mode API remain recommendations to settle in stage 1.
Existing source acquisition, containment, import destinations, strict composition,
precedence, inheritance, and complete-on-return loading continue to apply.
Consumption permits the checked header to serve as metadata; it must not allow
remaining payload to bypass main-version ownership. Synthetic overrides remain
subject to their existing protected-path restrictions.
This feature does not add independently versioned collections or subtrees,
per-input compatibility policies, automatic schema migration, YAML-selected
callbacks, or general source-validation hooks. These are outside this delivery,
not new permanent prohibitions on future work.
## Current baseline
Reviewed worktree baseline: `b3fc6a5`. Input destinations and relative import
destinations are implemented; the
[relative import roadmap](relative-import-destinations.md) records completion.
- [VersionRule and Version](../../lifecycle.go) provide a path, a checker, and
absent/null/text states. There is no expected-version string or source-check
mode. Root facade conversions preserve the public/internal boundary.
- The [version contract](../config.md#application-schema-versions) checks the
main version before defaults and typed decoding. The current path is an
effective application path; changing its interpretation in enabled modes
needs explicit handling for mounted main documents.
- [Loading](../../internal/engine/load.go) invokes the main check after main
acquisition/composition. [Version validation](../../internal/engine/validation.go)
already handles scalar vocabulary, checker diagnostics, and safe callback
errors. It is a useful shared mechanism, not a ready-made source-check API.
- [Imports](../../internal/engine/imports.go),
[underlays](../../internal/engine/load_request.go), and
[selected overlays](../../internal/engine/selected_overlay.go) currently reject
contributions to the protected version path. Enabled source checks need to
extract and consume headers before payload protection and composition, without
weakening the default behavior or checking only the eventual winning values.
- Existing source records carry identity, locations, and destination, but no
dedicated checked-version evidence. Existing sensitivity and inspection are
primarily organized around application paths and retained application trees.
## Ordered implementation plan
### 1. Finalize the contract and integration design
Resolve the Open Decisions below. Select public mode names and placement, source
participation, main-path handling, and evidence types. Define header lookup for
nested paths and distinguish a missing member from an obstructing non-container
ancestor. Specify consumption without broadening it into a generic transformation
API. Recommend removing only the selected header node and preserving its authored
ancestor containers, including newly empty mappings; decide handling of overlaps
with structural directives before implementation.
Record the change to main-only version ownership in an ADR and review public API
compatibility under repository policy. Do not rewrite accepted ADR decisions or
describe the feature as implemented before it lands. Existing default behavior
must remain covered; tests must not turn incidental acquisition ordering or
allocation choices into new guarantees.
Completion: the three modes and every participating source role have defined
lookup, callback, consumption, failure, and inspection semantics.
### 2. Share declaration checking and integrate source acquisition
Separate reusable version-value interpretation and checker reporting from the
current main-only invocation. Carry the selected mode through ordinary and
layered root APIs into the engine. Resolve source-local locations before mounting
and perform non-main checks before composition or precedence can mask them.
Consume only accepted non-main headers, retain their evidence, and apply existing
placement to the remaining body. Integrate direct inputs, recursive imports, and
selected overlays according to the selected participation contract. Continue
checking remaining payload for protected-path claims. Preserve source identity,
resource contexts, and applicable load accounting; consumed input still counts
as acquired input rather than bypassing source limits.
Avoid duplicate main checks or a second load of any document. Preserve the main
declaration's final value and typed conversion behavior. Ensure warnings do not
become failures merely because acquisition helpers previously treated any returned
diagnostic as fatal. Keep returned hook errors safe and callback contexts closed.
Completion: optional and required checks work across participating sources,
incompatible superseded input fails, and disabled-mode loads remain unchanged.
### 3. Expose evidence and verify lifecycle agreement
Add the selected root-owned inspection records and explicit conversions. Retain
source-local declarations and physical locations without fabricating effective
payload paths. Apply the agreed disclosure treatment to retained evidence,
including declarations consumed before final sensitivity evaluation. Return
detached records and preserve evidence when source files subsequently disappear.
Check agreement among configuration, provenance, history, rendering, and diff:
main remains represented normally; consumed headers remain source evidence;
ordinary version payload remains ordinary. Verify diagnostics identify the
actual source even when several documents use the same local header path.
Follow existing load-error reporting rather than promising exhaustive evidence
for documents never acquired after an earlier failure.
Completion: consumers can inspect checked declarations without reopening files,
and evidence does not leak sensitive values or alter effective-value semantics.
### 4. Add public acceptance coverage and update documentation
Use focused engine tests for vocabulary, extraction, and consumption; root tests
for public options and detached evidence; and executable consumer examples for
multi-file loading. Cover these distinct behaviors without duplicating all
existing import and composition tests:
- all modes, with absent, null, string, decimal-integer, and malformed values;
- multiple versions accepted by one checker without equality enforcement;
- required missing declarations and incompatible inputs hidden by later layers;
- main preservation and no duplicate callback, including the chosen mounted-main
interpretation and `NoSource` behavior;
- direct underlays, transitive/glob imports, overlays, and every document source
kind; no header requirement for synthetic field overrides;
- source-local lookup before input and relative-import destination placement;
- nested-header consumption, empty containers, structural-path interactions,
and remaining payload protection;
- ordinary release/prompt version fields outside the header path;
- warning propagation, safe checker failures, real source locations, and absent
header location fallbacks;
- inspectable detached evidence and disclosure using disposable canaries;
- consumed-header exclusion from effective rendering/diffs, with retained main
history and unchanged resource-relative behavior;
- default-mode compatibility, including existing fragment version rejection.
Demonstrate gradual adoption with optional checks and full declaration coverage
with required checks, using one policy. Do not include independent collection
checkers in examples. Update Go declaration documentation, the configuration
contract, consumer guidance, and internal overview as behavior lands; reconcile
the source-header request's earlier remaining-design notes then.
Follow the [development guide](../development.md) and
[testing policy](../policy/testing.md): focused checks during implementation,
the everyday check before handoff, ownership race checks where applicable, and
documentation-link validation. No independent performance benchmarks are part of
this roadmap. Release preparation and publication are separate work.
Completion: the agreed source coverage and three modes work through supported
public APIs, inspection and disclosure agree with the contract, and validation
and documentation are complete. Independent specification policies are not
required to complete this feature.
## Open Decisions
### Mode API and document participation
Recommendation: expose a load-wide enum on ordinary and layered options, with
zero retaining current behavior, and use the participation table above. This
allows one reusable `Definition.Version` policy to be used with different adoption
strictness on different loads. Include mapping sources and selected overlays:
they contribute documents even though they are not necessarily imported files.
Exclude synthetic field operations and absent sources as described above.
Alternative: place the mode on `VersionRule` so policy and enforcement travel
together. This reduces option plumbing but requires changing/copying the rule
to vary enforcement. A narrower first implementation checking imports and
underlays but exempting overlays is also viable; it reduces integration work but
offers weaker assurance about all contributing documents and needs an explicit
documented exception. Neither choice should imply per-source independent policies.
### Mounted main documents and the protected application path
Recommendation: in enabled modes, locate main's header source-locally as requested,
retain it, and derive its effective protected path by prefixing main's destination.
Use that derived path consistently for protection, existing typed version
conversion, and inspection. Invoke the shared checker once for main, preserving
its existing acceptance policy for absence; required mode additionally rejects a
missing declaration when an actual main document participates. `NoSource` retains
the existing absent-main checker behavior, since it is not a document.
Default mode keeps today's effective-path interpretation unchanged. This supports
mounted main input without contradicting source-local lookup or losing protection.
Alternative: separate the public source-header path from the existing effective
main-version path while still using one checker. It makes both coordinate systems
explicit and can ease migration for mounted main callers, but adds configuration
and two paths that must be kept coherent. Root-only main input would simplify the
problem but would restrict an existing capability; do not impose that restriction
as an implementation shortcut.
### Evidence representation and sensitivity
Recommendation: attach a detached declaration record to existing source
inspection, with local path, presence/null/text state, physical location, and
explicit redaction state. Reuse existing sensitivity machinery where applicable
to the source's mounted header path, retaining classification evidence before
consumption so removing a node does not erase its sensitivity. Main evidence and
source evidence should use consistent disclosure. Retain actual declaration text
privately when needed; inspectable sensitive evidence can be redacted without
denying that a declaration was checked. Specify handling for historical tags and
dependency classification before choosing storage, rather than exposing raw
`Version.Text` unconditionally.
Alternative: expose declarations through a dedicated source query returning safe
snapshots instead of extending source records. This better accommodates on-demand
redaction and richer evidence, but adds an API and lookup semantics. Both choices
must preserve physical attribution and the requested declaration evidence; neither
should expose arbitrary malformed input or require reopening the source.
### Nested consumption and structural directives
Recommendation: remove only the checked non-main declaration and retain authored
ancestor containers, even when empty. Validate source structure and interpret
existing directives so header consumption cannot conceal an invalid directive or
silently cancel an import. Preserve the existing path vocabulary where its
semantics apply; explicitly work through sequence-index and structural-path
interactions before implementation instead of adding a blanket top-level-only
header restriction. Retaining ancestors best preserves input presence outside
the requested consumed value.
Alternative: prune ancestor containers emptied by consumption. This can make a
nested metadata-only header disappear completely from application payload, but
also removes authored empty-container presence and can affect defaults and
composition. A dedicated top-level header location would simplify these cases,
but narrows the existing configurable-path model and would need an explicit
consumer requirement or design decision rather than an implementation shortcut.