From e66c37161c8cbc2edb3f0490cbad90296d28d8b2 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 16 Oct 2026 11:36:21 +0000 Subject: [PATCH] Retire completed roadmaps before the next release --- docs/adr/0011-input-destinations.md | 13 +- docs/adr/0012-relative-import-destinations.md | 8 +- docs/adr/0013-shared-source-version-checks.md | 9 +- docs/config.md | 4 +- docs/development.md | 16 + docs/roadmap/deferred.md | 12 + docs/roadmap/implementation.md | 720 ------------------ docs/roadmap/input-destinations.md | 174 ----- docs/roadmap/promptkit-feature-request.md | 500 ------------ docs/roadmap/relative-import-destinations.md | 133 ---- docs/roadmap/remediation.md | 392 ---------- docs/roadmap/source-version-checks.md | 269 ------- 12 files changed, 48 insertions(+), 2202 deletions(-) delete mode 100644 docs/roadmap/implementation.md delete mode 100644 docs/roadmap/input-destinations.md delete mode 100644 docs/roadmap/promptkit-feature-request.md delete mode 100644 docs/roadmap/relative-import-destinations.md delete mode 100644 docs/roadmap/remediation.md delete mode 100644 docs/roadmap/source-version-checks.md diff --git a/docs/adr/0011-input-destinations.md b/docs/adr/0011-input-destinations.md index 06860f4..c56d786 100644 --- a/docs/adr/0011-input-destinations.md +++ b/docs/adr/0011-input-destinations.md @@ -2,12 +2,13 @@ ## Status -Accepted. The [input-destinations roadmap](../roadmap/input-destinations.md) -owns implementation status for input placement. The accepted relative-import -extension is recorded in [ADR 0012](0012-relative-import-destinations.md), with -delivery status in its implementation plan. The completed implementation plan -and evidence are preserved at `315f464:docs/roadmap/implementation.md` in Git -history. +Accepted and implemented. Current placement behavior is owned by the +[configuration contract](../config.md#input-destinations). The accepted +relative-import extension is recorded in +[ADR 0012](0012-relative-import-destinations.md). The completed roadmap, consumer +request, and review records are preserved at `e4270c9` under `docs/roadmap/` in +Git history; the original input-destination implementation plan remains at +`315f464:docs/roadmap/implementation.md`. ## Date diff --git a/docs/adr/0012-relative-import-destinations.md b/docs/adr/0012-relative-import-destinations.md index d1f4fe8..c0979b9 100644 --- a/docs/adr/0012-relative-import-destinations.md +++ b/docs/adr/0012-relative-import-destinations.md @@ -2,9 +2,11 @@ ## Status -Accepted. The [relative-import destinations roadmap](../roadmap/relative-import-destinations.md) -owns the approved public behavior; the implementation plan records delivery -status and evidence. +Accepted and implemented. The +[configuration contract](../config.md#input-destinations) owns current behavior. +The completed roadmap and review records are preserved at `e4270c9` under +`docs/roadmap/` in Git history; the relative-import implementation plan remains +at `d9f08ff:docs/roadmap/implementation.md`. ## Date diff --git a/docs/adr/0013-shared-source-version-checks.md b/docs/adr/0013-shared-source-version-checks.md index 8b9b8fa..f845c95 100644 --- a/docs/adr/0013-shared-source-version-checks.md +++ b/docs/adr/0013-shared-source-version-checks.md @@ -2,9 +2,12 @@ ## Status -Accepted. The [shared source-version roadmap](../roadmap/source-version-checks.md) -owns target behavior, and its [implementation plan](../roadmap/implementation.md) -owns delivery status and sequencing. +Accepted and implemented. The +[configuration contract](../config.md#shared-source-version-checks) owns current +behavior. The completed roadmap, implementation plan, consumer request, and +resolved review findings are preserved at `e4270c9` under `docs/roadmap/` in +Git history; the [development guide](../development.md#baseline-validation) +provides retrieval examples. ## Date diff --git a/docs/config.md b/docs/config.md index e1c954b..87dcc8d 100644 --- a/docs/config.md +++ b/docs/config.md @@ -364,8 +364,8 @@ classification at the declaration's mounted path and ancestors, including historical classification. Redaction empties `Text` while retaining presence and null state. Returned copies are detached; evidence does not reopen inputs. Consult [ADR 0013](adr/0013-shared-source-version-checks.md) for the accepted -decision and the [source-version roadmap](roadmap/source-version-checks.md) for -its implementation record. +decision and the [development guide](development.md#baseline-validation) for +the completed implementation and remediation records in Git history. ### Layered metadata diff --git a/docs/development.md b/docs/development.md index cddd010..4aaee15 100644 --- a/docs/development.md +++ b/docs/development.md @@ -117,6 +117,22 @@ resolved review findings are preserved in Git history at `1c2610c` under comparison and disclosure behavior, and [ADR 0010](adr/0010-effective-configuration-diff.md) records the decision. +The completed input-destination, relative-import-destination, and shared +source-version roadmaps, associated consumer request, implementation plan, and +resolved review findings are preserved in Git history at `e4270c9` under +`docs/roadmap/`. For example: + +```sh +git show e4270c9:docs/roadmap/implementation.md +git show e4270c9:docs/roadmap/remediation.md +``` + +Current behavior belongs to the configuration contract's +[input destinations](config.md#input-destinations), +[imports](config.md#explicit-imports), and +[shared source-version checks](config.md#shared-source-version-checks) sections. +Possible follow-ups remain in [deferred scope](roadmap/deferred.md). + Install the Go version required by `go.mod` or a compatible newer toolchain. Acquire declared dependencies explicitly before entering an offline development or validation environment: diff --git a/docs/roadmap/deferred.md b/docs/roadmap/deferred.md index e2c1b81..305c3e3 100644 --- a/docs/roadmap/deferred.md +++ b/docs/roadmap/deferred.md @@ -39,6 +39,18 @@ Built-in sibling inheritance using `extends` is implemented as described in the [configuration contract](../config.md#sibling-inheritance). It is distinct from inheritance between selected overlay files, which remains excluded above. +## Possible source-version follow-ups + +Independent collection or subtree specification policies were outside the +completed shared source-version feature and remain possible future work, with +no committed delivery plan. Current [source-version checks](../config.md#shared-source-version-checks) +use one application-owned policy across participating documents. Reconsider +independent policies if concrete consumer needs justify a separate design +decision; this is not a permanent prohibition. The original request and its +selected scope remain at `e4270c9:docs/roadmap/promptkit-feature-request.md` in +Git history. Existing sequence-composition and cross-scope inheritance deferrals +above also remain unchanged. + ## Deferred performance experiments The completed performance audit left these internal optimization candidates for diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index 22b2bac..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,720 +0,0 @@ -# Implement shared source version checks - -Status: complete. All 14 implementation stages are validated. - -Audience: a Luna coding agent implementing one numbered stage per prompt, in -order. The [feature roadmap](source-version-checks.md) owns target behavior and -accepted policy; this plan owns concrete implementation decisions and execution. -Planning baseline: `d9f08ff7198ad5a8fb5c49a43e7ccdad2ad0e707`. - -The completed relative-import plan and evidence replaced here remain at -`d9f08ff:docs/roadmap/implementation.md`. Its roadmap and remediation are -complete, not instructions for this sprint. - -## Working instructions - -- Read `AGENTS.md`, [development.md](../development.md), and applicable policies. - This work involves [architecture](../policy/architecture.md), - [testing](../policy/testing.md), and [documentation](../policy/documentation.md). - Use graph-first discovery, verify live sources, and preserve unrelated work. -- Implement one stage at a time. Record completion, actual commands, toolchain, - results, limitations, and implementation revisions here. Commit each validated - nonempty unit with a short plain-English message. -- Keep public types root-owned with explicit engine conversions. Reuse semantic - Paths, documents, composition, overlay, version validation, sensitivity, and - inspection. No second loader, public parser nodes, generic transformation - hooks, or consumer-specific policy. -- Default mode preserves current behavior, including effective-path lookup, - main checker timing, and protected-path failures. Enabled modes deliberately - select source-local declarations and non-main consumption. Do not broaden - these changes into unrelated default-mode fixes. -- Do not add equality/SemVer rules, non-null requirements, numeric coercion, - path-form restrictions, source uniqueness policies, limits, dependencies, - toolchain changes, or release gates. Declaration acceptance is not fragment - schema conformance. Exclusions describe this delivery, not new permanent bans. -- Internal stages may test modes before exposing them publicly. Keep Go/current - documentation accurate to each delivered boundary; the roadmap owns the target. -- Use focused offline tests while iterating and `./scripts/check.sh` for each - production-code unit. Documentation-only units need local-link and whitespace - checks. Reuse existing tests; avoid a full mode/source/tier/path cross-product, - helper-call assertions, allocation identity, or exact message snapshots. - No independent benchmarks or performance thresholds are required. -- Do not rewrite accepted ADR decisions, publish, select a release version, - upgrade consumers, or retire completed roadmap records. Public struct additions - and enabled-mode path interpretation need minor-release compatibility guidance - during separately authorized release preparation. - -## Concrete implementation decisions - -### Public options and evidence - -1. Add root-owned `type SourceVersionMode uint8` with consecutive constants - `SourceVersionModeDefault` (zero), `SourceVersionModeIfPresent`, and - `SourceVersionModeRequired`. Add `SourceVersionMode SourceVersionMode` - to `Options[T]` and `LayeredOptions[T]`, not Definition, Input, or YAML. - Mirror privately and convert explicitly. Unknown modes, or enabled modes - without a usable VersionRule and checker, produce `CodeOptionsInvalid` - before source access or callbacks. Keep existing rule/path validation. -2. Add root-owned `SourceVersionCheck` with `Path Path`, `Version Version`, - `Location *Location`, and `Redacted bool`. Add - `VersionCheck *SourceVersionCheck` to SourceInfo. Nil means no enabled - declaration processing occurred. A non-nil record with Present false records - observed absence, including optional omissions whose checker was skipped; it - does not claim a successful callback or acceptance. On a malformed selected - value or overlap failure, retain only known presence/null, path, and location - with conservative redaction; never expose arbitrary scalar/container content. - Read/parse failures before this phase leave the record nil. - Reuse Version's absent/null/text model. - Redaction preserves presence/null, empties classified Text, and sets Redacted. -3. Evidence Path is source-local, with semantic segment intent resolved against - original source shape where possible. Present declarations retain their actual - authored location; absence uses the nearest available source location, not an - invented header coordinate. Destination and source identity already belong - to SourceInfo. Add no new query API, raw-node field, expected-version value, - or string/integer discriminator. -4. Clone the record, Path, and Location at all query/conversion/error boundaries. - Callback-time source queries and failed-load source records expose conservative - text-redacted evidence until final classification is available. Successful - final inspection may expose unclassified text; previous copies stay detached. - Keep raw evidence separate from public-safe source records. Do not extend a - failed load to finish classification or introduce a failed-load evidence API. - -### Lookup, checking, and diagnostics - -5. Carry the configured local rule and an effective main-owned rule separately. - Enabled modes derive the latter once by prefixing main's Destination using - semantic segments. Use it consistently for remaining-payload protection, - inheritance/CLI checks, decoding's exact-path integer exception, and application - metadata. Default mode uses the configured effective path unchanged. Never - mutate caller rules or repeatedly prefix a path. -6. Read each declaration from its original source document before mounting or - consumption. Keep contextual numeric-key/index resolution, empty/escaped keys, - parsed tokens, and explicit indices. No mapping-only/root-only restriction; - selecting a root mapping simply fails ordinary version-type validation. -7. Missing members, unavailable indices, and obstructing non-containers mean the - requested declaration is absent, as with current document lookup. Distinguish - causes privately for accurate nearest locations, not to invent rejection of - an ancestor's domain value. If-present non-main absence skips the checker and - records absence; required absence is `CodeVersionError` before callback. - A present unsupported selected value is `CodeVersionType`, not absence. - Actual main absence in if-present mode still reaches its checker once; - required mode rejects it first. NoSource is not a document and preserves the - existing single absent-main callback in both enabled modes. -8. Reuse strict decimal recognition and exact Version.Text. No machine-integer - conversion or comparison with main. Required null reaches the checker and can - be accepted. Warnings alone succeed and accepted non-main headers are consumed. -9. Enabled declaration checkers and their Validation paths use the current - source's local coordinates for every role, including mounted main. Resolve - primary/related locations against that original source. Generated header - diagnostics also use source-local paths; do not rebase them to phantom payload - fields. Ordinary payload diagnostics remain effective-path diagnostics. - Default version diagnostics are unchanged. Reuse `CodeVersionError` for - missing-required/structural-overlap failures, `CodeVersionType` for malformed - values, and existing checker error/warning/hook codes. No new code is needed. -10. Keep callback lifetime, safe generic returned-error handling, and synchronous - execution. Check main once from its own declaration, never a composed value. - Check other documents as acquired, before composition can hide rejection. - Do not promise a new inter-source callback order or exhaustive acquisition - after failure. Build schema successfully before callbacks; enabled mode may - prepare it before acquisition, without changing the default pipeline. - Defaults, decoding, resolution, secret lookup, and validation still run once - on the assembled application. - -### Consumption and structural boundaries - -11. Validate source-local controls before consumption can erase malformed - directives. Reuse import/overlay validation and inheritance-control - recognition, not per-fragment inheritance expansion. Reject actual overlap - with configkit directive structure or an extends control, including headers - inside import declarations or invalid control subtrees. Unrelated siblings - and ordinary keys containing those spellings remain allowed. Missing headers - follow their mode; they do not suppress independently invalid directives. - Add no error-order guarantee. -12. Retain main's node. Remove exactly an accepted non-main mapping member or - sequence element in an owned payload tree. Preserve authored ancestors, - newly empty mappings/sequences, sibling order, and physical node locations. - Sequence deletion shifts later payload indices; evidence keeps original - coordinates. Removing a member inside a sequence element keeps that element, - even when empty. No null placeholders, ancestor pruning, or Source mutation. -13. Extract source-root imports from original content; consume headers in its - application body before placement/composition. Underlay roots and all imports - are non-main; selected overlays are also non-main. Check remaining placed - payload against effective main ownership. Consumption does not permit - forbidden overlay controls, sequence alignment, or CLI ownership bypass. -14. Charge original acquired bytes/nodes once, including consumed headers. Apply - existing placed-body bounds and full-destination wrapper charges to remaining - payload. Keep source/depth/glob accounting, records, resource contexts, and - physical repetition unchanged. Acquired header-only mappings retain their - empty body/ancestors and placement; NoSource/optional absent sources add no - document or evidence. No new cap or full-source clone is required. - -### Evidence and sensitivity - -15. Carry private evidence per acquired SourceID in per-load acquisition/assembly - state, separate from application history. Retain an owned scalar/absence, - original local and mounted semantic paths, location, and the ancestor-shape - context needed for classification. Capture key/index intent before deletion. - Retain only needed context, not entire source-tree clones or loading handles. -16. At publication classify evidence at its original mounted path and ancestors - with existing schema tags, predicate, historical projected sensitivity, - inheritance, and recorded-dependency propagation. Include evidence-only - candidates in classification without adding them to Metadata paths, payload - applications, rendering, or Diff. Do not infer original sequence shape from - shifted payload or round-trip through pointer strings. Reuse dependency - closure rather than building a weaker independent redactor. Retained header - text never triggers credential lookup. -17. Sanitize source records before public conversion; early records remain - conservatively redacted. Final successful evidence uses completed sensitivity, - retaining historical classification despite replacement. Main's normal - payload/history remains intact. Consumed non-main headers are only source - evidence: equal payloads with different headers must not acquire effective - differences merely from that evidence. - -## Implementation owner map - -| Responsibility | Existing owners to inspect and extend | -| --- | --- | -| Public options, callbacks, records, conversions | `lifecycle.go`, `layered.go`, `records.go`, `conversion.go`, `diagnostic.go` | -| Options and per-load coordination | `internal/engine/options.go`, `load.go`, `load_request.go`, `layered.go` | -| Acquisition, directives, placement, overlay | `internal/engine/load_input.go`, `imports.go`, `directives.go`, `destination.go`, `selected_overlay.go` | -| Checking and semantic paths | `internal/engine/validation.go`, `document_walk.go`, `version_path_test.go` | -| Evidence copies, errors, publication | `internal/engine/records.go`, `diagnostic.go`, `load_result.go` | -| Classification and propagation | `internal/engine/secret.go`, schema secret-path helpers, `secret_propagation_test.go`, `render_test.go` | -| Compatibility tests | Engine `load_test.go`, `imports_test.go`, `selected_overlay_test.go`, `load_request_test.go`, `destination_*_test.go`; root `facade_version_test.go`, `facade_layered_test.go`, `facade_destination_test.go`, `records_test.go` | - -Private helper names/file splits may follow the implementation while preserving -these decisions. A new internal package solely for headers is unnecessary. - -## Validation commands - -Use the applicable subset during each stage; add new tests under the -SourceVersion name family so focused checks include them. Existing test owners -remain authoritative. Commands run from the repository root with dependencies -already available; these are development checks, not new release gates. - -```sh -GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Version|Import|Underlay|Overlay|Destination' -GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Secret|Sensitivity|History|Inheritance|Resolution' -GOTOOLCHAIN=local GOPROXY=off go test . -run 'SourceVersion|Version|Layered|SourceInfo|Destination|PublicFacadeBoundary|DependencyImportBoundary' -GOTOOLCHAIN=local GOPROXY=off go test ./examples/consumers -GOTOOLCHAIN=local GOPROXY=off go test . -run '^TestLocalDocumentationLinks$' -GOTOOLCHAIN=local ./scripts/check.sh -GOTOOLCHAIN=local GOPROXY=off go test -race ./... -git diff --check -``` - -## Stage 1 — Capture the baseline and record the extension - -1. Confirm the worktree/baseline; read the roadmap, plan, applicable policies, - contract, and relevant ADRs. Run the everyday check and record revision, - toolchain, results, and environmental limitations. -2. Write Accepted ADR `0013-shared-source-version-checks.md` after checking the - number is unused. Record one policy, modes, source-local checking, consumption, - evidence, and retained main protection. Focus on ConfigKit's selected choice; - rejected choices belong only in Alternatives Considered. Keep sequencing here. -3. Preserve prior accepted decision text. Add status/navigation links where - useful. Note struct additions and enabled mounted-main interpretation as - minor-release considerations; do not select a version or write release notes. - -Validate: local-link test and whitespace. Done: baseline and accepted decision -recorded, with no runtime changes. - -Completion evidence: reviewed baseline `d9f08ff7198ad5a8fb5c49a43e7ccdad2ad0e707` -and the clean implementation worktree at `70c95bf660794809e58182d736c67c9085238705`; -confirmed ADR 0013 was unused. The everyday check passed with Go -`go1.26.8 linux/amd64` using -`GOTOOLCHAIN=local GOPROXY=off ./scripts/check.sh`. Documentation links and -whitespace are validated for this unit. No environmental limitation affected -these checks; validation was performed on Linux/amd64 only. No runtime changes -were made. - -## Stage 2 — Share version interpretation without changing defaults - -1. Separate reusable value interpretation and callback reporting from checkVersion, - retaining its default entry point, absence/null/text, exact integers, warnings, - safe errors, primary/related locations, and context closure. -2. Reuse existing scalar/path/hook/lifetime tests; add only missing protection for - the refactor. Introduce a small private result separating checked value and - diagnostics. Failure decisions must use severity, not slice length. -3. Do not globally rewrite unrelated diagnostic plumbing or public behavior. - -Validate: engine version/validation and facade version tests; everyday check. -Done: shared mechanisms exist with unchanged default behavior. - -Completion evidence: `checkVersion` remains the existing diagnostic-only entry -point and now delegates to private scalar interpretation and callback-reporting -helpers through a result that keeps the interpreted `Version` separate from -diagnostics. Checker invocation and returned-error fallback use diagnostic -severity, preserving warnings without treating them as failures. Added a -regression for a warning followed by a returned checker error. These checks -passed with Go `go1.26.8 linux/amd64`: - -- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'Version|Validation'` -- `GOTOOLCHAIN=local GOPROXY=off go test . -run 'Version'` -- `GOTOOLCHAIN=local ./scripts/check.sh` - -No environmental limitation affected validation; checks ran on Linux/amd64. - -## Stage 3 — Implement source-local extraction and exact consumption - -1. Add a private processor receiving original document, local rule, source role, - destination, and mode. Implement decisions 6–12, including missing/obstructed - lookup and pre-consumption structural overlap. -2. Return owned remaining payload, original import declarations, and private - evidence with original segment/ancestor context. No second parser, fragment - decoding, or mutation of reusable input. -3. Add focused tests for vocabulary, missing/obstructed locations, malformed - selected nodes, literal/parsed numeric keys and indices, empty ancestors, - sequence deletion, and actual versus unrelated structural overlap. - -Validate: helper/version-path/directive/inheritance-control tests and everyday -check. Done: the processor is tested independently of public loading. - -Completion evidence: added a private source-local processor that resolves the -configured path against the original document, separates imports, records -private path/ancestor/location evidence, and consumes accepted contribution -headers only from an owned payload copy. Missing and obstructed paths retain -their cause and nearest available location; required absence fails before the -checker, while optional main absence still invokes it. Structural directive -overlap is rejected before payload consumption. Focused coverage includes the -version vocabulary, malformed nodes, contextual numeric keys and indices, -empty ancestors, sequence deletion, and actual versus unrelated structural -overlap. These checks passed with Go `go1.26.8 linux/amd64`: - -- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Version|Import|Underlay|Overlay|Destination|Inheritance'` -- `GOTOOLCHAIN=local ./scripts/check.sh` -- `git diff --check` - -No environmental limitation affected validation; checks ran on Linux/amd64. - -## Stage 4 — Add private mode state and main-document integration - -1. Add the engine mode to ordinary/layered options and loadRequest. Validate it at - both entry points before source access. Underlay helper options must not be - treated as standalone loads missing the shared rule. -2. Carry local/effective rules, source roles, warnings, and evidence in per-load - state. Let acquired inputs return evidence and nonfatal diagnostics. -3. In enabled mode, check actual main before imports/composition, retain its node, - and prepare schema before callbacks. Skip the old later main check only in - enabled mode. Preserve NoSource's absent callback and default timing. -4. Thread effective protection through typed conversion, inheritance, CLI, and - application inspection while keeping declaration callbacks local. Test mounted - main, absence/NoSource, null acceptance, invalid options before reading, and - exactly one main callback. - -Validate: load/grouped/version/destination/hook tests and everyday check. Done: -private enabled main behavior works without changing old callers. - -Completion evidence: added the private mode to ordinary and layered engine -options and the shared load request. Both entry points validate enabled modes -and their shared checker before source acquisition; enabled loads build schema -before invoking the main checker. The parsed main document is checked at its -source-local path before import traversal or layered composition, and its node -remains in the application payload. A single semantic destination-prefixed rule -now protects mounted main ownership during decoding, inheritance, acquired-input -placement, overlays, and CLI overrides; declaration callbacks and diagnostics -retain local paths. Default mode keeps its prior post-composition check timing. -The per-load assembled input retains main evidence and successful warning -diagnostics. NoSource invokes the existing absent-main callback once in either -enabled mode. These checks passed with Go `go1.26.8 linux/amd64`: - -- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Version|Load|Grouped|Destination|Hook'` -- `GOTOOLCHAIN=local ./scripts/check.sh` -- `git diff --check` - -No environmental limitation affected validation; checks ran on Linux/amd64. -The engine mode remains private; public options are scheduled separately. - -## Stage 5 — Integrate recursive imports, warnings, and accounting - -1. Thread shared state through import entry/traversal. Check each acquired child - once against its own local header, then place and compose remaining payload. - Keep nested/glob effective destinations and original directive discovery. -2. Apply effective main protection afterward. Preserve physical identity, - cycle/repetition, optional absence, source records, resource contexts, and raw - node/byte charges. Header-only acquired mappings still contribute empty bodies. -3. Propagate warnings to load diagnostics without aborting traversal. Update only - relevant callers currently treating any diagnostic as failure. -4. Cover nested/glob imports, mounted relative destinations, accepted unequal - declarations, rejected hidden declarations, required omissions, safe locations, - optional missing sources, raw accounting, and default import.version failures. - Reuse existing containment/glob/source-kind tests. - -Validate: import/glob/destination/accounting/version tests and everyday check. -Done: imports use one policy without lost warnings or changed physical traversal. - -Completion evidence: threaded one per-load source-version state through the -main import entry and recursive traversal. Each parsed child is checked against -its source-local rule before placement; accepted headers are removed from an -owned payload while the original root still supplies raw byte/node charges and -the processor's original import declarations drive nested traversal. Effective -main ownership is checked against the remaining placed payload. Evidence is -retained with the acquired SourceID, and warning diagnostics accumulate without -stopping traversal. Existing identity, cycle, optional-source, source-record, -resource-context, and accounting paths remain in use. Added coverage for nested -and glob imports at mounted relative destinations, accepted unequal declarations, -rejected hidden declarations, required omissions with source-local locations, -optional missing imports, evidence identity, and raw header byte/node charges. The -existing default import-version failure coverage also passes. These checks -passed with Go `go1.26.8 linux/amd64`: - -- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Version|Import|Glob|Destination' -count=1` -- `GOTOOLCHAIN=local ./scripts/check.sh` -- `git diff --check` - -No environmental limitation affected validation; checks ran on Linux/amd64. - -## Stage 6 — Integrate direct underlays and their strict groups - -1. Thread state through acquireUnderlayGroup/grouped assembly. Direct underlays - are non-main; imports reuse Stage 5. Preserve layers, indices, contexts, - evidence, and warnings across groups. -2. Consume local headers before strict composition, then protect remaining placed - payload. Preserve tier precedence, empty containers, raw budgets, and reusable - sources in allowed separate tiers. -3. Cover accepted distinct peer declarations, a rejected underlay hidden by main, - required absence, empty/NoSource groups, mounted nested headers, retained - payload conflicts, and unchanged default underlay.version failures. - -Validate: grouped/underlay/destination/version tests and everyday check. Done: -direct and imported underlays follow the same checking policy. - -Completion evidence: threaded the per-load version state through direct underlay -group acquisition and invoked the shared processor with contribution semantics -for each acquired underlay root. Its own processed payload/import declarations -now drive placement and recursive imports, while remaining placed payload is -still checked against the effective main-owned path before strict composition. -Warnings are accumulated through underlay groups; source records retain their -layer and group index, and the existing accounting, context, and reusable-source -paths remain in place. Added coverage for distinct declarations within one -strict group, warning retention across groups, rejected declarations beneath a -later main value, required omissions, empty groups and NoSource, mounted nested -headers/imports, and protected remaining payload. Existing default underlay -version-protection coverage remains passing. These checks passed with Go -`go1.26.8 linux/amd64`: - -- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Version|Import|Underlay|Overlay|Destination' -count=1` -- `GOTOOLCHAIN=local ./scripts/check.sh` -- `git diff --check` - -No environmental limitation affected validation; checks ran on Linux/amd64. - -## Stage 7 — Integrate selected overlays and final ownership checks - -1. Process overlay headers after parse/structural validation, before remaining - payload protection and application. Reuse shared state in ordinary and layered - overlay loading; preserve source identity, namespaces, close behavior, and raw - node/byte charges. -2. Carry accepted warnings through assembly. Keep overlay-control restrictions, - and effective main protection through CLI and inheritance. -3. Cover accepted/required/rejected overlay headers, warnings, unchanged default - behavior, ordinary CLI fields without header requirements, mounted-main - protection, and protected payload after nested/sequence consumption. Sequence - evidence stays original while remaining indices shift. - -Validate: overlay/override/inheritance/version tests and everyday check. Done: -all document roles are integrated privately; synthetic operations are exempt -from header checks, not from existing protection. - -Completion evidence: threaded the shared version state through ordinary and -layered selected-overlay loading. After parse and overlay-directive validation, -the selected overlay is checked as a non-main source; accepted headers are -removed from its owned payload before effective-path protection and application. -The original parsed node still supplies raw byte/node charges, and source -identity, filesystem namespace, closure, overlay-control checks, and override -protection remain on their existing paths. Overlay warnings are accumulated with -the other per-load diagnostics. Added coverage for accepted, required-missing, -and rejected overlay declarations; structural validation before callbacks; -warning retention; CLI overrides without document-header checks; mounted-main -protection after header consumption; and sequence deletion retaining original -evidence coordinates while later payload indices shift. Existing default overlay -version/directive/accounting coverage remains passing. These checks passed with -Go `go1.26.8 linux/amd64`: - -- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Version|Import|Underlay|Overlay|Override|Inheritance|Destination' -count=1` -- `GOTOOLCHAIN=local ./scripts/check.sh` -- `git diff --check` - -No environmental limitation affected validation; checks ran on Linux/amd64. - -## Stage 8 — Expose the public load-wide mode - -1. Add root mode/constants/option fields and both conversions from decision 1. - Preserve zero-value behavior and root-owned boundaries. -2. Add external-package tests for ordinary/layered enabled loads, zero-mode - compatibility, invalid mode/rule before reading, and reuse of one Definition - with different modes. Use representative source-kind coverage. -3. Document implemented mode behavior in Go declarations, including local versus - effective interpretation. Evidence remains private until Stage 11; do not - claim it is publicly available yet. - -Validate: facade version/layered/source and boundary tests; everyday check. -Done: both supported public load APIs expose the implemented modes. - -Completion evidence: added the root-owned `SourceVersionMode` enum and constants -to ordinary and layered options, with Go documentation for zero-value behavior, -source-local declaration paths, destination-prefixed main protection, and the -two enabled policies. The facade converts explicitly through the internal -engine boundary, which keeps its mode type private and preserves unknown values -for option validation before acquisition. Added external-package coverage for -enabled ordinary and layered loads using byte, file, and mapping sources; zero -mode compatibility; invalid modes and missing rules before reader access; and -reusing one Definition with different modes. Source evidence remains private. -These checks passed with Go `go1.26.8 linux/amd64`: - -- `GOTOOLCHAIN=local GOPROXY=off go test . -run 'SourceVersionMode|Version|Layered|SourceInfo|Destination|PublicFacadeBoundary|DependencyImportBoundary' -count=1` -- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Version|Layered|Source|Destination|Import' -count=1` -- `GOTOOLCHAIN=local ./scripts/check.sh` -- `git diff --check` - -No environmental limitation affected validation; checks ran on Linux/amd64. - -## Stage 9 — Classify evidence at original paths and ancestors - -1. Include declaration paths/ancestors in final classification candidates, - independently of effective/historical Metadata indexes. Do not add consumed - nodes to applications or explanation replay. -2. Apply schema-tag and predicate rules using original mounted semantic context, - including tags beneath dynamic containers and ancestor classification. - Preserve numeric-key/index intent before sequence shifts; consumed paths need - not survive final decoding. Reuse existing redaction mechanisms. -3. Test private evidence with disposable canaries: unclassified text, direct/ - ancestor tags and predicates, original sequence shape versus shifted payload, - and null/absence. Keep raw evidence private throughout this stage. - -Validate: source-version/sensitivity/rendering tests and everyday check. Done: -direct/ancestor classification is correct without making headers payload history. - -Completion evidence: carried private source-version evidence from per-load state -into the resolved publication input, without adding consumed declarations to -applications or metadata. Final classification now evaluates each declaration's -original mounted semantic path and captured ancestors with the existing schema -tag and predicate rules, then uses the existing sensitivity result for rendering. -This covers dynamic map/slice paths, typed original sequence indices, and -null/absent evidence while leaving unclassified evidence and shifted neighboring -payload visible. Evidence remains private. These checks passed with Go -`go1.26.8 linux/amd64`: - -- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Secret|Sensitivity|History|Inheritance|Resolution' -count=1` -- `GOTOOLCHAIN=local ./scripts/check.sh` -- `git diff --check` - -No environmental limitation affected validation; checks ran on Linux/amd64. - -## Stage 10 — Preserve historical and dependency sensitivity - -1. Connect evidence candidates to existing historical, inheritance, and recorded - dependency propagation. Later replacement or consumption must not erase - applicable classification; do not invent new propagation policy. -2. Consider evidence-only paths even when absent from final metadata. Keep - Resolution.Record's existing schema-path validity; do not invent dependencies - on headers solely to create tests. -3. Cover displaced historical classification and legitimate recorded dependencies, - with inheritance where applicable. Ensure neighboring unclassified evidence - remains visible and no synthetic effective/history/diff paths are introduced. - -Validate: source-version/history/inheritance/resolution/sensitivity tests, -`go test -race ./internal/engine`, and everyday check. Done: established -sensitivity closure applies without polluting payload history. - -Completion evidence: added original mounted evidence paths and ancestors to the -existing classification path set before schema/predicate evaluation and the -shared dependency closure. This lets existing recorded dependencies and -inheritance links carry evidence sensitivity into dependent and displaced -historical values, including evidence-only paths absent from final metadata; -Resolution.Record validation and dependency policy are unchanged. Added a -load-level regression using a valid recorded dependency on the credentials -container (not a synthetic dependency on the header), plus inherited displaced -history coverage. Neighboring values remain visible, and consumed headers create -no metadata, explanation, or Diff paths. These checks passed with Go -`go1.26.8 linux/amd64`: - -- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Secret|Sensitivity|History|Inheritance|Resolution' -count=1` -- `GOTOOLCHAIN=local GOPROXY=off go test -race ./internal/engine` -- `GOTOOLCHAIN=local ./scripts/check.sh` -- `git diff --check` - -No environmental limitation affected validation; checks ran on Linux/amd64. - -## Stage 11 — Publish detached sanitized source evidence — Done - -1. Add private/public SourceVersionCheck and SourceInfo.VersionCheck records. - Extend both conversions and all clone paths, including callback source queries, - successful inspection, and existing LoadError source access. -2. Before final classification, expose conservative sanitized records; after - successful classification, freeze final sanitized evidence. Keep local paths, - physical locations, absence/null, and unchecked distinction. Raw evidence - stays in private state; no loading handle is retained. -3. Add facade/copy tests mutating nested paths/locations and records. Repeated - queries and independent loads remain unaffected. Delete sources after load - and verify evidence/resource metadata without reopening files. -4. Cover early acquisition and later resolver/validator failures plus callback - source queries with canaries. Default and synthetic CLI records stay nil. - -Validate: record/conversion/source/version/ownership and public boundary tests, -root/engine race checks, and everyday check. Done: existing public source -inspection surfaces safely expose detached evidence. - -Completion evidence: exposed root-owned and engine-private `SourceVersionCheck` -records through `SourceInfo`, added explicit conversions and deep copies for -paths and locations, and published conservative evidence to resolver queries -and failed loads while retaining final classified evidence in successful -inspection. Covered malformed and early import-acquisition failures, resolver -and validator errors, callback-time canaries, optional absence/null, default -and synthetic CLI records, source removal after load, repeated queries, and -independent mutation of callback, error, conversion, and inspection copies. -These checks passed with Go `go1.26.8 linux/amd64`: - -- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|SourceInfo|LoadError|Resolution|Validation' -count=1` -- `GOTOOLCHAIN=local GOPROXY=off go test . -run 'SourceVersion|SourceInfo|LoadError|PublicFacadeBoundary|DependencyImportBoundary' -count=1` -- `GOTOOLCHAIN=local GOPROXY=off go test -race ./internal/engine .` -- `GOTOOLCHAIN=local ./scripts/check.sh` -- `git diff --check` - -No environmental limitation affected validation; checks ran on Linux/amd64. - -## Stage 12 — Audit lifecycle agreement and compatibility — Done - -1. Map tests to every roadmap participation row/edge decision; add only missing - cross-boundary regressions with one primary owner per behavior. -2. Verify final values, zero/false/null presence, defaults, exact-path integer - conversion, resolution, secret lookup, and validation after consumption. - Hooks see one assembled configuration. Accepted declarations need not match. -3. Compare equal payloads with differing consumed headers: evidence changes but - effective rendering/history/Diff gain no consumed fields. Main history stays - normal. Cover empty ancestors and ordinary versions outside the header path. -4. Check physical contexts after placement/inheritance/precedence, existing - accounting boundaries, all supported document source kinds, and default - import/underlay/overlay/CLI/version compatibility. -5. Run `./scripts/check.sh` and - `GOTOOLCHAIN=local GOPROXY=off go test -race ./...`. Record actual evidence; - do not claim untested platforms or consumer adoption. Consumer checks are not - new release gates. - -Done: all target behavior has public-boundary evidence and deterministic checks -pass, without extra contract restrictions or redundant coverage. - -Completion evidence: mapped source-local lookup, malformed/overlapping headers, -consumption, roles, mounted protection, directives, and source participation to -the focused `source_version_test.go` suite; mapped classification and historical -propagation to `secret_classification_test.go`, `inheritance_secret_test.go`, -and the recorded-dependency regression; and retained existing destination, -accounting, resource-context, and default-mode suites as the owners for those -boundaries. Added a public-facade lifecycle regression that accepts distinct -main/underlay declarations while verifying exact large-integer decoding, -zero/false/null, defaults, one resolved configuration, secret lookup, validation, -evidence retention, and unchanged effective explanation/rendering/Diff. No -source-order promise or additional policy was introduced. These checks passed -with Go `go1.26.8 linux/amd64`: - -- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Version|Import|Underlay|Overlay|Destination|Secret|Sensitivity|History|Inheritance|Resolution' -count=1` -- `GOTOOLCHAIN=local GOPROXY=off go test . -run 'SourceVersion|Version|Layered|SourceInfo|Destination|Diff|PublicFacadeBoundary|DependencyImportBoundary' -count=1` -- `GOTOOLCHAIN=local ./scripts/check.sh` -- `GOTOOLCHAIN=local GOPROXY=off go test -race ./...` -- `git diff --check` - -No environmental limitation affected validation; checks ran on Linux/amd64. - -## Stage 13 — Add maintained adoption examples — Done - -1. Add a small executable example showing if-present and required adoption with - one Definition.Version policy, accepted distinct declarations, main/import/ - underlay or overlay sources, and safe evidence inspection. -2. Include a mounted input illustrating local lookup and effective main - protection. Keep ordinary release metadata at another path; no consumer-side - second load/merge, collection checker, or implicit schema migration. -3. Keep copyable assets in examples with tests for final payload and safe output. - Reuse existing examples for remaining source plumbing; do not print canaries - or promise callback ordering. - -Validate: example/consumer tests, links, and everyday check. Done: tested examples -demonstrate the coherent shared policy with reduced consumer plumbing. - -Completion evidence: added an executable layered consumer example and embedded -main/underlay fixtures. One source-local `Definition.Version` policy accepts the -distinct declarations in if-present and required modes; the mounted main maps -its local `version` to `/service/version`, while ordinary application release -metadata remains at `service.metadata.release`. The example inspects only -sanitized presence/path/redaction fields, and its test verifies final payloads -and effective-path override protection without printing declaration contents or -promising callback order. These checks passed with Go `go1.26.8 linux/amd64`: - -- `GOTOOLCHAIN=local GOPROXY=off go test ./examples/consumers` -- `GOTOOLCHAIN=local GOPROXY=off go test . -run '^TestLocalDocumentationLinks$' -count=1` -- `GOTOOLCHAIN=local ./scripts/check.sh` -- `git diff --check` - -No environmental limitation affected validation; checks ran on Linux/amd64. - -## Stage 14 — Reconcile documentation and finish the audit - -1. Update Go docs and docs/config.md as canonical owners for modes, participation, - local diagnostics, consumption, effective protection, mounted main, evidence - states/disclosure, and default behavior. Explain conservative callback-time - redaction and declared compatibility versus final validation. Link from - consumer guidance, development, and internal overview. -2. Update the architecture ownership summary where needed, linking to contract - and ADR rather than duplicating mechanics. Preserve previous accepted ADR - decisions. Reconcile obsolete design/consumption notes in the source request. - Repair completed relative-import roadmap/remediation links that meant the old - overwritten plan, using the Git-history pointer above. -3. Audit for accidental default checks, equality/non-null rules, path grammar, - fragment decoding, ordering promises, extra uniqueness/caps, unsafe evidence - queries, and new release gates. Match every roadmap outcome to test evidence. -4. Mark plan/roadmap complete only after all stages pass. Run everyday checks, - full race suite after code corrections, and final link/whitespace checks. - Commit the validated unit; leave a clean workspace. Do not publish or retire - completed records in this stage. - -Done: implementation, coverage, documentation, and approved scope agree. No -product questions remain open under the roadmap and concrete decisions above. - -Completion evidence: updated the public Go comments, configuration contract, -consumer guidance, development guide, and component ownership summary for the -implemented behavior. Reconciled the original source-version request with later -accepted decisions, marked the source-version roadmap implemented, and repaired -relative-import/remediation references to the overwritten plan using the -confirmed `d9f08ff:docs/roadmap/implementation.md` history pointer. The scope -audit found no default-mode expansion, equality/non-null rule, extra path -restriction, fragment decoding, callback-order promise, new uniqueness/cap, -unsafe evidence query, or release gate. Reviewed behavioral coverage in -`internal/engine/source_version_test.go`, -`facade_source_version_mode_test.go`, -`internal/engine/secret_classification_test.go`, -`internal/engine/inheritance_secret_test.go`, and -`examples/consumers/layered_test.go`. These checks passed with Go -`go1.26.8 linux/amd64`: - -- `GOTOOLCHAIN=local ./scripts/check.sh` -- `GOTOOLCHAIN=local GOPROXY=off go test . -run '^TestLocalDocumentationLinks$' -count=1` -- `git diff --check` - -No runtime code changed in this documentation unit, so the race suite was not -rerun. No environmental limitation affected validation; checks ran on -Linux/amd64 only. Publication and release preparation remain outside this work. - -## Post-implementation remediation evidence - -The subsequent review found coverage gaps beyond the original stage validation; -the [remediation record](remediation.md#shared-source-version-review-2026-10-15) -owns their status. Stage 9–12 evidence now also includes -`TestSourceVersionEvidenceClassificationKeepsCollidingShapes` and -`TestFacadeSourceVersionEvidenceRedactsCollidingShapes`. Both failed before the -shape-collision fix and pass afterward. Classification preserves original -mapping-key and sequence-index candidates independently without erasing payload -classification. Focused source-version/sensitivity/history tests, -`GOTOOLCHAIN=local ./scripts/check.sh`, and -`GOTOOLCHAIN=local GOPROXY=off go test -race ./...` passed on -`go1.26.9 linux/amd64`. Existing historical/dependency propagation and neighboring -public-value coverage remain green; no new sensitivity policy was introduced. - -Additional remediation regressions protect import opt-in -(`TestFacadeSourceVersionModesPreserveImportOptIn`), complete absence paths -(`TestSourceVersionAbsentPathKeepsUnresolvedSegmentIntent` and -`TestFacadeSourceVersionAbsenceKeepsCompleteEvidencePath`), and optional missing -headers below controls -(`TestFacadeSourceVersionMissingHeaderDoesNotOverlapControl`). Each failed -before its production correction and passes afterward. All four independent -review probes, the final everyday check, and the full race suite now pass on -`go1.26.9 linux/amd64`. The remediation record contains completion evidence; -the agreed source-version policy and release gates are unchanged. diff --git a/docs/roadmap/input-destinations.md b/docs/roadmap/input-destinations.md deleted file mode 100644 index 2750560..0000000 --- a/docs/roadmap/input-destinations.md +++ /dev/null @@ -1,174 +0,0 @@ -# Configuration Input Destinations Roadmap - -Status: implemented. This roadmap records the approved behavior and rationale -for the [PromptKit feature request](promptkit-feature-request.md). The -completed implementation plan is preserved in Git history at -`315f464:docs/roadmap/implementation.md`, including its audit and remediation -evidence; the [configuration contract](../config.md#input-destinations) -owns current behavior. - -## Targeted end state - -A consumer can assign a mapping destination to a configuration input and perform -one ordinary `LoadLayered` call. The input's application content contributes -beneath that destination while retaining normal composition, precedence, -inheritance, typed loading, diagnostics, and inspection. PromptKit can assemble -collections without parsing and rewriting sources, merging values, or rebuilding -provenance itself. - -For example, a fragment defining `summarize.alerts` can contribute beneath -`collections`, `weatherreporter`, `prompts`. An independent source with the same -local key can contribute beneath another collection without conflict. Collection -and prompt names remain ordinary application keys, not ConfigKit namespaces. - -The public surface adds an optional `Destination Path` field on `Input`, not a -new loading entry point or an operation on completed results. Content-bearing -main inputs and underlay inputs support -the same destination model; root placement remains the default. - -The following behavior defines the target: - -- Omitted and empty destinations retain root placement. Destinations consist of - literal mapping keys, including dots, slashes, and numeric-looking text. - Explicit sequence-index destinations are outside the requested capability. -- A valid destination on `NoSource` contributes nothing. An actual empty - application mapping materializes its destination mappings as explicit input, - matching authored wrappers, without clearing lower-tier children. -- Existing structural reservations apply to introduced mappings. Destinations - that materialize `configkit` or a mapping-valued `extends` contrary to existing - contracts fail through the structured loading-error boundary. Keys merely - containing those words remain ordinary keys. -- Host files, native filesystem files, bytes, readers, and mapping sources all - support placement under their existing acquisition and ownership contracts. - Filesystem context, import opt-in, and precedence remain independent choices. -- Mounted contributions have the same application values and conflict behavior - as explicitly wrapped contributions, with accurate original source locations. - Disjoint mappings combine; overlapping non-mapping values within a group - conflict even when equal. A later tier cannot repair a strict-group conflict. -- Different destinations can coexist in one group, and the same destination can - occur in different tiers. Ancestor/descendant destinations follow ordinary - composition and overlay rules without namespace uniqueness restrictions. - Later tiers recursively overlay mappings and replace other values. Empty - mappings do not clear lower values; sequences remain atomic. -- A prompt's ordinary `version` override changes that field and retains omitted - lower fields. It neither identifies a merge record nor replaces a whole prompt. -- Source-root `configkit.imports` remains structural. Imported application data - inherits the input destination transitively when its import omits - `destination` or supplies `[]`; the accepted [relative-import extension](../adr/0012-relative-import-destinations.md) - appends any declared segments. Import paths still resolve from their declaring - source; containment, optionality, cycles, and repeated-source rules remain - applicable. Placement does not change physical source identity. -- Sibling inheritance runs after assembly in the effective destination scope. - Peer fragments can supply a child and its sibling parent. Qualified PromptKit - references do not acquire cross-collection inheritance meaning. -- Version ownership is enforced at effective application paths. A collection's - mounted `version` is ordinary data, while underlay/import contributions to the - actual protected path, or replacements of its ancestors, remain invalid. - There is one main-owned version check and no blanket nested numeric conversion. -- Defaults, decoding, resolution, secret lookup, and validation run once on the - assembled application, not independently on fragments. Callbacks and value - diagnostics use effective paths. Failures use structured load errors and yield - no usable partial result. -- Presence, provenance, history, explanations, effective rendering, and diffs - agree with the final values. Sensitivity is evaluated at effective application - paths under the existing historical and dependency rules. -- Existing source records expose placement, including inherited import - placement. Introduced ancestor mappings retain the contributing source's - identity; any root-location fallback identifies that document, not an authored - wrapper. Actual content retains its real source coordinates. -- Original source locations and passive resource contexts survive placement, - overrides, and inheritance. `Result.SourceContext` and - `Resolution.SourceContext` still identify the declaring filesystem and base - directory for a field such as `content_file: ./summary.md`. -- Destinations and the structure they introduce participate in existing load - limits; placement cannot bypass depth or node accounting. Source ownership, - complete-on-return loading, and detached result behavior remain intact. - -Import-destination syntax, independent collection specification headers, and -cross-collection inheritance are separate possible follow-ups. This work does -not add directory discovery, collection installation, remote acquisition, -namespace ownership enforcement, dependency solving, live reload, filename-based -identity, arbitrary transformations, or new merge strategies. PromptKit owns its -collection inventory, qualified IDs, and execution semantics. - -## Implementation baseline (v0.7.1) - -The request reviewed v0.7.1. At that baseline, the relevant boundaries were: - -- [Input](../../layered.go) holds a source, import opt-in, and filesystem context, - but no destination. Root types delegate to the engine through the existing - [facade boundary](../internal/overview.md). -- [Layered loading](../config.md#layered-loading) already provides strict groups, - ordered precedence, shared acquisition budgets, and one final lifecycle. - [Underlay acquisition](../../internal/engine/load_request.go) gathers source - applications and recomposes them into their strict group. -- [Input acquisition](../../internal/engine/load_input.go) and - [import traversal](../../internal/engine/imports.go) process source-root - directives and source identity before final assembly. Imported version claims - are checked during traversal; underlay claims are checked before group - composition. These currently operate without a destination. -- [Version claim checks](../../internal/engine/directives.go) inspect a document - using the configured application path. Wrapping only an already acquired - result would leave early fragment checks and diagnostic paths incorrect. -- [Sibling inheritance](../adr/0009-sibling-inheritance.md) already runs after - source precedence and preserves source-to-destination derivation. Placement - must feed that existing mechanism rather than resolve fragments independently. -- Existing source records identify acquisition and precedence but do not expose - input placement. Existing history retains source applications, so moving only - the final tree would be insufficient for coherent inspection. - -The architectural fit is an extension of input assembly, not a parser extension -or a second configuration model. The -[architecture policy](../policy/architecture.md) supplies the ownership and -consistency constraints; [deferred scope](deferred.md) continues to govern the -unrequested features above. - -## Selected Design Decisions - -### Main-input destinations and absent inputs - -Allow the same destination model for content-bearing main inputs and underlays, -with `VersionRule.Path` remaining an effective application path and any -specification declaration still owned by main. The checker retains its existing -handling of an absent declaration and its existing lifecycle phase. `NoSource` -contributes nothing even with a valid destination. This keeps `Input` coherent -across its roles and preserves absence without per-fragment version checks. - -### Public placement inspection and synthetic ancestors - -Expose effective placement as `SourceInfo.Destination Path` on existing source -records, including inherited import placement. -Use original source identity for introduced ancestor mappings, documenting that -any root-location fallback identifies the contributing document rather than a -physically authored wrapper. Actual content keeps its real coordinates. This -allows consumers to connect placement to sources without a new query system. - -Preserve that relationship through inherited fields, composite origins, and the -same source selected in separate tiers. Placement is per source application; -consumers can inspect it without inferring it from final paths. - -### Reserved structural keys in destinations - -Use existing `Path` conventions for literal keys and parsed -numeric tokens, reject explicit index segments, and preserve current structural -reservations in the effective tree. Reject destination segments that would -materialize `configkit` or a mapping-valued `extends` contrary to those contracts, -with a clear structured destination error. Ordinary keys containing those words, -dots, or slashes remain ordinary keys. Source-root directives are processed before -placement and are not rejected simply because their application body is mounted. - -Mounted and authored wrappers therefore retain the same structural meaning. -Destination diagnostics identify placement as the cause of invalid introduced -structure, using existing error categories where suitable. - -### Empty mappings and presence of introduced containers - -Mounting an actual empty application mapping materializes its -destination mappings as explicit contributions, matching authored wrappers. -This preserves the request's equivalence principle and lets consumers inspect -an intentionally empty collection. Such mappings still do not clear lower-tier -children. An absent source contributes nothing. - -This applies equally to empty imported application bodies and inputs containing -only import directives. Presence follows the contribution's meaning rather than -whether an implementation happened to allocate wrapper nodes. diff --git a/docs/roadmap/promptkit-feature-request.md b/docs/roadmap/promptkit-feature-request.md deleted file mode 100644 index 0936542..0000000 --- a/docs/roadmap/promptkit-feature-request.md +++ /dev/null @@ -1,500 +0,0 @@ -# PromptKit Feature Request: Mounted Collections And Specification Headers - -Status: - -- Input destinations are implemented on the ConfigKit development branch - (placement baseline reviewed at `315f464`). -- Relative import destinations have been accepted and implemented; the - [roadmap](relative-import-destinations.md) records their completed delivery. -- Source specification headers have been partially accepted in the shared - application-version-policy form described below. This does not accept - PromptKit's original independent collection-header semantics. Design and - implementation of this addition remain separate from relative import work. - -## Requested Capability - -Extend configuration loading with relative import destinations and opt-in -checks that contributing sources declare a version accepted by the consumer's -single configuration specification policy. Preserve the implemented capability -to select a destination mapping path for each configuration input. The input's -application content contributes -beneath that path while retaining ConfigKit's normal composition, precedence, -inheritance, validation, diagnostics, and inspection behavior. - -PromptKit needs this capability for its proposal 3, **Collections With Mounted -Configuration Fragments**, recorded in PromptKit's -`docs/roadmap/configuration-collections-mounted-fragments.md`. This request -describes the public API needs and observable acceptance criteria. It does not -prescribe ConfigKit's internal design or an implementation plan. The disposition -and revised source-version scope below supersede conflicting parts of PromptKit's -original proposal. - -The original request targeted ConfigKit v0.7.1. The development branch now -supplies `Input.Destination` and `SourceInfo.Destination`, as documented in the -[input destination contract](../config.md#input-destinations). The placement -requirements below remain the baseline for the two additions. Current released -behavior and development-branch behavior must not be confused with the newly -proposed header and import syntax. - -## Consumer Problem - -PromptKit applications assemble independently maintained collections of prompts, -inference profiles, and backends. Collections can contain identical local names, -such as `summarize.alerts`, without identifying the same resource. PromptKit wants -an effective configuration organized by collection, while letting authors omit -repeated collection and resource-container wrappers from individual files. - -A file selected from a collection's `prompts` directory would contain: - -```yaml -summarize.alerts: - version: "1" - default_profile: provider/brief - messages: - - role: user - content: Summarize the weather alerts. - output: - format: text - validation_mode: none -``` - -Given destination `[collections, weatherreporter, prompts]`, its contribution -would appear in the final application configuration as: - -```yaml -collections: - weatherreporter: - prompts: - summarize.alerts: - version: "1" - default_profile: provider/brief - messages: - - role: user - content: Summarize the weather alerts. - output: - format: text - validation_mode: none -``` - -Another collection can contribute `summarize.alerts` beneath -`[collections, homewatch, prompts]` without conflicting. After final handoff, -PromptKit interprets these as `weatherreporter/summarize.alerts` and -`homewatch/summarize.alerts`. The profile reference is illustrative and must be -supplied separately by the application configuration. - -PromptKit will support one installed definition per prompt. Its optional flat -`version` is ordinary domain metadata, not a merge identity. This request does -not need identified-record sequences, sequence-root documents, or a custom -merge policy. - -Authors could write the complete wrappers today, but that repeats assembly -details in every fragment. Consumer-side parsing and wrapping would put generic -source processing back into PromptKit and complicate source attribution. -PromptKit instead wants to select sources and destinations, then receive one -final configuration through ConfigKit's supported loading lifecycle. - -## Implemented Placement API And Required Baseline - -### Per-Input Destination - -The development branch provides `Destination Path` on each `Input` passed to -`LoadLayered`, matching the original request. The following partial consumer -example uses that implemented field, which was not available in v0.7.1: - -```go -promptInput := configkit.Input{ - Source: configkit.File("/usr/local/share/collections/weatherreporter/prompts/alerts.yml"), - Destination: configkit.Keys("collections", "weatherreporter", "prompts"), -} - -peers := configkit.Underlay{ - Inputs: []configkit.Input{promptInput}, -} -``` - -The consumer would include `peers` in its existing `LayeredOptions` and invoke -`LoadLayered` with its normal root main input and application `Definition`. -This is a partial API sketch, not an executable example or a proposed new -loading entry point. A single public layered load is sufficient for PromptKit; -it does not require a separate mounting operation on completed results. - -The API must let the consumer: - -- Express a destination relative to the application configuration root using - literal mapping-key segments. Dots, slashes, and numeric-looking text inside - a key are not implicit path separators or array indexes. PromptKit's namespace - grammar remains its own domain rule. -- Omit the destination to retain existing root placement. An explicitly empty - destination should have the same meaning. Mapping destinations are sufficient; - sequence-index destinations are not needed by this request. -- Assign different destinations to inputs in the same strict composition group, - and assign the same destination to inputs in separate precedence groups. -- Use the destination with existing content-bearing source kinds: host files, - native filesystem files, bytes, readers, and mapping sources, subject to their - existing acquisition and ownership contracts. -- Retain the existing import opt-in and filesystem context independently of the - destination. A configuration destination is not a filesystem root or base - directory. - -PromptKit can continue to enumerate an explicitly selected directory and attach -the same destination to its returned inputs. ConfigKit does not need a new -directory scanner, namespace registry, filename convention, or PromptKit schema. -The namespace comes from the application's assembly declaration, not from an -implicit interpretation of a directory basename. - -### Representative Assembly - -These are application choices, not ConfigKit-specific reserved paths: - -| Selected inputs | Destination key segments | -| --- | --- | -| Application main document | Root | -| `weatherreporter/collection.yml` | `collections`, `weatherreporter` | -| YAML files under `weatherreporter/prompts` | `collections`, `weatherreporter`, `prompts` | -| YAML files under `homewatch/prompts` | `collections`, `homewatch`, `prompts` | -| YAML files under `provider/profiles` | `collections`, `provider`, `profiles` | -| YAML files under `provider/backends` | `collections`, `provider`, `backends` | - -Peer collection contributions belong to a single strict underlay group when the -application wants conflict detection among them. Intentional operator overrides -use later groups or main. Destination selection must not establish precedence -or make source traversal order authorize replacement. - -## Required Observable Behavior - -### Composition And Precedence - -Mounted and explicitly wrapped contributions must have equivalent application -values and conflict behavior at their effective destinations. Actual source -locations naturally differ and must remain accurate. - -Retain the [strict composition](../config.md#strict-document-composition) and -[layering](../config.md#layered-loading) rules: - -- Disjoint mapping entries at the same destination combine. -- Overlapping non-mapping values within a strict group conflict, including equal - scalars, nulls, and sequences. Different destinations do not collide merely - because the source documents share a local key. -- Later precedence layers recursively override mappings and replace other - values. A strict conflict cannot be repaired by an upper layer. -- Empty mappings do not clear lower definitions; ordinary arrays remain atomic. - In particular, `messages` does not acquire sequence-merging behavior. -- Overlapping destination paths and contributions at their ancestors follow - those same rules. No namespace-level uniqueness restriction is requested. - -Changing only a prompt's `version` in a later layer changes that field while -retaining omitted lower-layer fields. ConfigKit must not treat a domain version -label as a request to replace an entire definition. - -### Imports And Existing Structural Directives - -An input with imports enabled must be able to use its existing source-root -`configkit.imports` directive while contributing application data at a nested -destination. The directive remains structural configuration, not an application -field below that destination. - -When an import omits `destination` or supplies an empty list, it inherits its -importing document's effective destination, including transitively. An explicit -sequence appends to that destination. Import paths continue to resolve relative -to their declaring source's filesystem context. A mounted document must not send -its imported application fields back to the configuration root accidentally. - -Existing opt-in, containment, missing-input, cycle, and repeated-source behavior -must remain applicable. A different destination is not a request to bypass the -current rejection of a repeated physical source within a strict group. - -Sibling `extends` must resolve against mappings at their effective destinations. -For example, two independently supplied profile fragments mounted beneath the -same collection's `profiles` mapping can establish `extends: brief` once they -compose. Inheritance remains ConfigKit-owned; PromptKit does not perform another -relationship-resolution pass. - -### Protected Specification Paths - -The distinction between source-local field names and effective application -paths is essential. PromptKit's main document will declare `version: 1` at the -configuration root. A separate collection metadata file can contain: - -```yaml -version: "2.4.0" -``` - -Mounted at `[collections, weatherreporter]`, that field is an ordinary collection -release label. It must not be rejected as an attempt to contribute the protected -root specification field. The same applies to a prompt's nested optional -version. This describes the existing placement baseline, with no new source -checks enabled. The partially accepted extension will reuse the application's -[VersionRule](../config.md#application-schema-versions) for source declarations; -header lookup and its interaction with ordinary version payload remain design -questions, not a guarantee that every source-local `version` stays unchecked. - -Conversely, placement must not permit an underlay or imported fragment to claim -or replace the actual protected specification path or an ancestor that would -replace it. The public contract must define protection in terms of effective -application paths, rather than reserving the spelling `version` in every source. -Protection must remain valid with imports, inheritance, overlays, and explicit -CLI overrides. - -Retain existing behavior when the new source checks are not enabled. The -accepted direction extends the same application version policy across source -files, rather than assigning separate policies to collections. Its interaction -with protected-path ownership and declaration retention must be designed -explicitly; it does not implicitly authorize version-field contributions. -PromptKit does not need main mounted away from the root or integer-to-string -conversion for every nested ordinary `version` field. - -### Typed Loading, Diagnostics, And Inspection - -The destination must be visible consistently through the ordinary public -loading and inspection surface: - -- Final typed configuration and application callbacks use effective destination - paths. Defaults, resolution, and validation retain the existing single-load - lifecycle; fragments are not independently validated as whole applications. -- Metadata and presence queries distinguish omission, explicit null, and other - explicit values at the mounted paths. -- Diagnostics identify the effective conflicting or invalid path and the real - contributing source locations. Consumers can determine which source and - placement produced a value without recreating that relationship themselves. -- Effective rendering, explanations, history, and configuration comparison agree - with the final configuration. Existing sensitivity and disclosure rules apply - to mounted fields, including application path-based secret classification. -- Unsupported destination forms and destination-related loading failures use - the normal structured error boundary. Exact new diagnostic codes, if needed, - are for ConfigKit to define. - -PromptKit does not require a separate mount registry or new inspection facade. -Existing inspection APIs are sufficient if they expose accurate effective paths, -source attribution, and placement information. ConfigKit should document how -source-local locations relate to effective paths. - -### Source-Relative Resources And Lifecycle - -For a mounted field such as `content_file: ./summary.md`, obtaining its effective -origin and calling `Result.SourceContext` or `Resolution.SourceContext` must -still identify the declaring source's filesystem and base directory. The same -holds when that field survives an override or is inherited into another mapping. -Its configuration destination must not change the meaning of the relative file -reference. - -Retain existing source ownership, load completion, result lifetime, containment, -and resource-access behavior. Destinations must participate in documented load -limits without creating a way to bypass them; this request does not prescribe -new limit values or their accounting machinery. - -## Partially Accepted Addition: Shared Source Specification Checks - -Status: the shared-policy portion below is implemented. Current behavior is -defined by the [configuration contract](../config.md#shared-source-version-checks), -and the accepted decision and implementation evidence are in -[ADR 0013](../adr/0013-shared-source-version-checks.md) and the -[source-version roadmap](source-version-checks.md). - -### Accepted Scope - -A ConfigKit consumer defines one configuration specification, which may evolve -through versions. `Definition.Version` identifies the declaration path and the -consumer's checker determines which versions are acceptable. The accepted -extension lets the consumer apply that same policy across the source files -contributing to a configuration, rather than checking only the main declaration. -It does not introduce independent collection or subtree specification policies. - -The consumer can opt in with increasing strictness: - -1. **Current behavior:** retain existing loading and version-checking behavior, - including existing protected-path restrictions. No new source-header check. -2. **Check declarations when present:** a source that declares a version must - satisfy `Definition.Version`; omission is allowed. -3. **Require declarations:** every participating source must declare a version - that satisfies `Definition.Version`; omission is an error. - -Acceptance is determined by the existing consumer-supplied checker, not by a -new ConfigKit-owned expected string or automatic equality with the main file's -declaration. A consumer may accept several compatible versions. A declaration -rejected under the selected mode fails the attempted load through structured -errors, with no usable partial result; a higher-precedence payload cannot turn -an incompatible source declaration into an acceptable one. - -This establishes that checked sources declare an accepted specification version. -It does not prove that every fragment conforms to its declaration. Existing -final decoding and application validation still determine the validity of the -assembled configuration; fragments are not independently loaded as complete -applications. - -### Difference From PromptKit's Original Proposal - -PromptKit originally requested explicitly selected collection-entry rules, -separate from `Definition.Version`, with ordinary imports exempt from independent -checks. It also recommended consuming accepted entry headers and exposing them -as source evidence while leaving imported release metadata untouched. - -The independent collection-entry policies and exemption for ordinary imports -were not accepted. The accepted direction instead uses one shared -`Definition.Version` policy and optional or required declarations across -participating sources. Separately versioned subsets or collections remain -outside this addition's scope. Header consumption and sanitized source evidence, -which were suggestions in the original proposal, were subsequently accepted for -this shared-policy design and are implemented as described by the linked -contract. - -### Design Questions Resolved by the Accepted Roadmap - -The following questions were open in the original request and are resolved by -the accepted roadmap and ADR 0013: - -- How the configured declaration path is interpreted for individual sources, - particularly before destination placement. -- Which source kinds and roles participate, including direct underlay inputs, - transitive imports, and other contributions. -- Whether checked declarations are consumed or retained, how that interacts - with composition and main-owned version protection, and what inspection - evidence is exposed. -- How null and malformed declarations behave in each mode, preserving the - distinction between absence and a present declaration and considering the - existing string/decimal-integer version vocabulary. -- The opt-in API, callback lifecycle, and source-local diagnostic attribution. - -In particular, an imported `version: "2.4.0"` at the configured source-local -header path is checked and consumed in an enabled mode. Consumers that need an -ordinary release label retain it at a different path. No automatic field -consumption applies in the default mode, and no new coercion or per-collection -checker is part of the implemented design. - -## Requested Addition: Relative Import Destinations - -Disposition: accepted and implemented under the completed -[relative import destinations roadmap](relative-import-destinations.md). -The behavior below records the accepted capability; exact syntax and completion -status follow that work, rather than implying a released feature. - -Extend the public YAML import directive so a `path` or `glob` import can specify -a relative destination as literal mapping-key segments. The roadmap selects -`destination` as a sequence of strings, for example `destination: [prompts]`. - -The public behavior should be: - -- The importing document has an effective destination established by its input - or its own import. An explicit import destination appends key segments to that - destination. Omission or an empty list retains the importing destination. -- Every file matched by a glob receives the same declared relative destination. - Transitive imports are relative to their immediate importing document's - resulting destination, not always to the original entry destination. -- Filesystem paths still resolve relative to the declaring file, with existing - containment rules. Logical placement and physical file resolution are - independent; there is no root-reset or parent-traversal meaning in destination - key segments. Literal keys follow the same rules as `Input.Destination`. -- Imports retain their existing strict-composition tier. Import order and - destination declarations do not create precedence or permit replacement. -- Existing optionality, repeated-source, cycle, source-kind, and ownership rules - remain intact. A destination does not authorize importing one physical file - twice in a group. Existing source-root directive and reserved-key rules apply. -- `SourceInfo.Destination` reports each import's resulting effective destination. - Effective paths, original locations, resource contexts, protected version - paths, sensitivity, and error behavior retain the placement baseline above. -- Invalid directive shapes fail through the normal structured error boundary; - destination fields must not be silently ignored. - -This is declarative access to the existing placement capability. It does not -request recursive glob syntax, implicit directory scanning, namespace inference, -or a new composition strategy. - -## Relative Import Collection Entry Example - -An entry document selected by the application with destination -`[collections, weatherreporter]` could describe its layout as follows. This -example illustrates relative imports only and assumes no new source checks: - -```yaml -configkit: - imports: - - path: metadata.yml - - glob: prompts/*.yml - destination: [prompts] - - glob: profiles/*.yml - destination: [profiles] - - glob: backends/*.yml - destination: [backends] -``` - -This is proposed syntax, not currently executable configuration. Each required -glob assumes matching files; collections can omit unused categories or use -existing optional-import behavior. The application still selects and provisions -the collection and assigns its namespace. ConfigKit does not discover it. - -With current source-version behavior, `metadata.yml` can contain -`version: "2.4.0"` as an ordinary mounted collection release label. Prompt -fragments retain local mapping keys and optional prompt versions, and contribute -beneath the collection's `prompts` mapping. The main application can retain its -own version declaration. Enabling the proposed shared source checks may require -changes to these files once declaration lookup and retention are defined. - -Publishers can describe collection layout while applications select collections -and destinations. The later shared-version feature will add opt-in declaration -checking; it does not establish package authenticity or release integrity. - -## Deferred And Excluded Work - -**Identity-based list composition.** Keep this deferred. Proposal 3's local -mapping keys already identify each installed resource; the two requested -additions do not require a different representation or sequence merge policy. - -**Cross-collection inheritance.** Nested collection profiles are not immediate -siblings across collections. Current sibling inheritance remains sufficient for -the initial proposal, with authored definitions flattened where necessary. -Generic cross-path inheritance could be considered separately if consumer needs -justify revisiting the [deferred scope](deferred.md). Mounting must not silently -reinterpret `extends: provider/brief` as a cross-collection reference. - -PromptKit also does not request collection installation, automatic directory -discovery, remote fetching, namespace ownership enforcement, dependency solving, -live reload, filename-derived identity, or arbitrary document transformations. -Its qualified resource IDs, optional prompt version semantics, collection -inventory, and execution APIs remain PromptKit responsibilities. - -## Public Acceptance Scenarios - -Evaluate the implemented placement baseline and accepted relative-import -extension through supported public loading and inspection APIs. Source-version -rows below express the partially accepted direction; their unresolved details -must be settled before they become implementation acceptance tests. - -| Scenario | Expected consumer-visible result | -| --- | --- | -| Destination omitted or empty | Existing root-placement behavior | -| Same fragment shape mounted under two distinct collection keys from independent sources | Two separate effective resources | -| Disjoint inputs at the same destination in one group | Mapping contributions combine | -| Equal or unequal overlapping scalar fields in one group | Structured composition conflict with both source locations | -| Same destination across deliberate precedence tiers | Existing recursive overlay behavior | -| Version-only prompt override | Label changes; omitted prompt body fields survive | -| Nested mounted import without a destination directive | Imported fields contribute at the inherited destination | -| Local profile inheritance across mounted peer fragments | Parent resolves at the effective sibling scope | -| Collection metadata containing `version` mounted below root, with new checks disabled | Existing ordinary nested-field behavior | -| Mounted/imported contribution actually targeting the protected version path | Rejected under version ownership rules | -| Mounted field using source-relative content, including inheritance | Original declaring resource context remains available | -| Mounted explicit null and omitted field | Distinct effective presence information | -| Mounted validation error or secret field | Effective paths, physical locations, and existing disclosure rules agree | -| File, native filesystem, byte, reader, or mapping input with equivalent content | Equivalent application values, subject to existing source contracts | -| Unsupported destination or failed load | Normal structured failure, no usable partial result | -| Source-version checks disabled | Existing version-checking and protected-path behavior | -| Check-when-present mode, declaration absent | Omission is allowed | -| Check-when-present mode, declaration present | Must satisfy the shared `Definition.Version` policy | -| Required mode, declaration absent | Structured load failure | -| Required mode, declaration present | Must satisfy the shared `Definition.Version` policy | -| Different declarations both accepted by the consumer checker | No additional exact-equality requirement | -| Rejected source declaration with payload superseded by a higher layer | Rejection still fails the load | -| Relative import destination omitted or empty | Existing inherited-destination behavior | -| Entry at `[collections, weather]`, import at `[prompts]`, nested import at `[extras]` | Nested contribution appears beneath `[collections, weather, prompts, extras]` | -| Glob import with a relative destination | Every matched contribution uses that destination and composes strictly | -| Relative import destination with source-relative resource | Effective configuration path changes; resource base remains the declaring source | -| Load without new source checks or relative import directives | Existing loading and main VersionRule semantics remain unchanged | - -The relative-import outcome is a consumer that selects entry sources and -destinations, performs one ordinary layered load, and receives final typed -configuration with reliable diagnostics and inspection. Entry documents describe -their own relative import placements without consumer-side source rewriting, -generic merging, or provenance reconstruction. - -The separately scoped, partially accepted source-version extension adds optional -or required declaration checking through the consumer's single -`Definition.Version` policy. It does not require or authorize independently -versioned collections. diff --git a/docs/roadmap/relative-import-destinations.md b/docs/roadmap/relative-import-destinations.md deleted file mode 100644 index d7afb7f..0000000 --- a/docs/roadmap/relative-import-destinations.md +++ /dev/null @@ -1,133 +0,0 @@ -# Relative Import Destinations Roadmap - -Status: complete. The implementation plan and its evidence were replaced by the -shared source-version plan and remain in Git history at -`d9f08ff:docs/roadmap/implementation.md`. -This roadmap addresses only the relative-import-destination addition in -the [PromptKit request](promptkit-feature-request.md#requested-addition-relative-import-destinations). -Source-version checks were outside this roadmap's scope and are now implemented -separately in the [shared source-version roadmap](source-version-checks.md). - -## Targeted end state - -Authors can assign a relative mapping destination to a `path` or `glob` import. -ConfigKit appends its literal key segments to the immediate importing document's -effective destination, then processes the imported application content through -the existing placement and loading lifecycle. Applications continue to select -entry sources, input destinations, filesystem contexts, and precedence tiers. - -Selected YAML syntax: - -```yaml -configkit: - imports: - - path: metadata.yml - - glob: prompts/*.yml - destination: [prompts] - - glob: profiles/*.yml - destination: [profiles] -``` - -If the application places this entry at `[collections, weather]`, its metadata -import contributes at that same path, and each prompt file contributes at -`[collections, weather, prompts]`. An import declared inside a prompt file with -`destination: [extras]` contributes at -`[collections, weather, prompts, extras]`. A subsequent sibling import from the -entry is still relative to `[collections, weather]`. - -This example does not declare or consume a source specification header. A -source-local `version` remains ordinary payload unless governed by the existing -main-owned version contract. The combined header example in the broader request -is not an acceptance requirement for this roadmap. - -The feature must preserve these requested and existing behaviors: - -- Omission and an empty destination list retain the importing document's - destination. Each glob match receives the same relative placement. Transitive - imports accumulate destinations from their immediate parents. -- An explicit `destination` is a sequence of strings. Null, scalar values, and - non-string members are invalid directive shapes; values are not coerced. - For a numeric-looking key, use a string such as `destination: ["0"]`. -- Segments are mapping keys under the existing `Input.Destination` rules. Dots, - slashes, and numeric-looking text within a key are literal. There is no - filesystem-style normalization, parent traversal, or application-root reset. - Do not add a collection-name grammar or reject otherwise supported literal - keys merely because their spelling resembles a filesystem path. -- Imports retain their strict-composition tier. Placement and traversal order - do not confer precedence. Later tiers retain existing recursive overlay - behavior; sequences remain atomic and overlapping destinations use existing - composition rules. -- Filesystem paths resolve from their declaring source independently of logical - placement. Import opt-in, optionality, glob behavior, containment, cycle and - repeated-source rules, source kinds, and ownership remain unchanged. A new - destination does not allow a physical source to repeat within a strict group. -- Source-root directives are interpreted before application placement. Existing - reserved-key rules continue to apply to destinations. Invalid new directive - shapes produce normal structured failures rather than being ignored. -- `SourceInfo.Destination` reports each acquired source's resulting effective - destination, including nested imports and sources recorded on failed loads - to the extent supported by existing error reporting. -- Composition and value diagnostics use effective application paths and real - source locations. Directive errors identify their source-local declaration. - Placement does not manufacture authored coordinates or alter resource bases. -- Main-owned version protection applies at the resulting effective paths. - Ordinary nested version fields stay ordinary payload. Inheritance resolves - in the assembled sibling scope, then defaults, decoding, resolution, secret - lookup, and validation follow the existing single-load lifecycle. -- Presence, provenance, history, explanations, rendering, diffs, and sensitivity - agree with the placed contributions. Source-relative resource contexts survive - inheritance and precedence under their existing contracts. -- Accumulated placement and introduced wrappers participate in existing depth - and node accounting. This work does not select new limit values or promise - new performance bounds. An actually imported empty mapping retains existing - placement behavior; an optional import that finds no source creates no payload. - -The syntax is available wherever existing explicit imports are supported, -including ordinary root loading and layered main/underlay inputs. It does not -require a new Go loading entry point, mount registry, or consumer-side rewriting. - -Source header policies and consumption, cross-collection inheritance, sequence -merging, root-reset syntax, automatic discovery, recursive glob syntax, and -collection management are outside this delivery. This scope statement does not -establish new permanent exclusions beyond [existing deferred scope](deferred.md). - -## Pre-implementation baseline - -Reviewed baseline: `315f464`, with input destinations implemented. At that -revision, the -[configuration contract](../config.md#input-destinations) and -[ADR 0011](../adr/0011-input-destinations.md) described that feature. - -- [Directive parsing](../../internal/engine/directives.go) accepted `path` or - `glob` plus `optional`; it rejects unrecognized fields. Import declarations - carried no relative destination. -- [Import traversal](../../internal/engine/imports.go) held one destination for - an input's entire import tree. Both document placement and imported source - records use it. Per-document effective placement is therefore required. -- [Input acquisition](../../internal/engine/load_input.go) assigned - the input destination to all returned imported source records. That uniform - assignment must no longer overwrite distinct import destinations. -- [Destination helpers](../../internal/engine/destination.go) already validate - literal mapping paths, place document bodies, and check introduced depth and - node costs. Composition, version protection, and retained source applications - already operate on placed content. -- The root facade already exposes `SourceInfo.Destination`; the new syntax can - use that inspection surface without a separate placement API. - -The change extends traversal and directive parsing. It should reuse existing -placement and assembly rather than introduce a second merge or validation model. - -## Selected Design Decision - -### YAML destination representation - -Use `destination` as a sequence of strings, for example -`destination: [prompts, "0"]`. Omission and `[]` inherit placement. Null, scalar -destinations, and non-string members are invalid shapes rather than values to -coerce. Every string key allowed by existing destination rules remains valid, -including the empty string. - -This representation directly expresses literal segments and keeps numeric key -text distinct from YAML numbers. It provides one explicit notation consistent -with the existing mapping-destination model. These shape rules define the new -directive syntax; other configuration values retain their existing semantics. diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md deleted file mode 100644 index 99258bb..0000000 --- a/docs/roadmap/remediation.md +++ /dev/null @@ -1,392 +0,0 @@ -# ConfigKit review findings - -Reviewed `d3d468a..945e111` against the approved -[input-destinations roadmap](input-destinations.md), -input-destination implementation plan (preserved in Git history at -`315f464:docs/roadmap/implementation.md`), and -[ADR 0011](../adr/0011-input-destinations.md). - -The root-owned destination fields, placement before strict composition, -transitive import placement, effective-path version ownership, final lifecycle, -source records, resource contexts, retained history, and existing disclosure -mechanisms broadly match the selected architecture. The required correction -below concerns an early diagnostic path, not loading or composition semantics. -No additional unintended strengthening of contracts was identified. - -## 1. Rebase known Mapping-source failure paths to the input destination (P2) - -Status: complete. Known Mapping value-builder paths are now prefixed with the -normalized destination at acquisition, preserving semantic key/index segments. -Public regressions cover mounted main and underlay failures, escaped keys, -sequence indices, cyclic values, unchanged root paths, and source-constructor -errors retaining source coordinates. The mounted value cases failed before the -fix and pass afterward. Existing directive/syntax checks remain unchanged. -Focused destination/source/diagnostic/import tests, `./scripts/check.sh`, and -`GOTOOLCHAIN=local GOPROXY=off go test -race ./...` passed on -`go1.26.8 linux/amd64`. - -Affected: `assembleBaseInputTracked` in -[load_input.go](../../internal/engine/load_input.go), lines 27–32 at the reviewed -commit; the Mapping failure originates in `readSource` in -[source.go](../../internal/engine/source.go), lines 206–208. - -`Mapping` snapshots its input through the existing value builder. When a nested -value is unsupported, cyclic, or exceeds a value-builder limit, the retained -issue already identifies a semantic path inside that application mapping. -`readSource` turns it into a structured diagnostic. The early return from -`assembleBaseInputTracked` annotates the source record with its destination but -returns those diagnostic paths unchanged. - -For example, select this input as either main or an underlay: - -```go -configkit.Input{ - Source: configkit.Mapping("bad mapping", map[string]any{ - "item": map[string]any{"bad": func() {}}, - }), - Destination: configkit.Keys("catalog", "entries"), -} -``` - -The load correctly fails with `source.invalid`, but `Diagnostic.Path` is -`/item/bad` rather than `/catalog/entries/item/bad`. The source record correctly -reports `/catalog/entries`, so the two inspection records use inconsistent -application coordinates. A consumer displaying or associating the failure with -the final application shape would point to the wrong configuration field. - -This diverges from implementation decision 10 and Stage 6: existing paths that -genuinely identify application content must be translated to effective paths -exactly once. A Mapping value-builder issue has that information; it is not a -source-read or YAML syntax failure for which a logical child path is unavailable. - -Required remediation: - -- Prefix known application-relative Mapping value-builder diagnostic paths with - the normalized input destination at the acquisition boundary. Preserve key - escaping and sequence-index intent in the existing semantic Path model. -- Keep source IDs, original primary/related locations, codes, value-free - messages, source destinations, and nil-result behavior unchanged. Root - placement must retain its current paths. -- Do not blanket-prefix parse/read failures, source-constructor errors, or - structural import-directive diagnostics. Do not invent logical child paths - for failures that do not identify application content. -- Add bounded regression coverage through the public API for mounted main and - underlay Mapping failures, alongside unchanged root placement. Reuse existing - syntax/directive tests to protect their different coordinate convention. -- Run the focused destination/source/diagnostic checks and the everyday check. - Reconcile the completed plan's diagnostic evidence with this correction; - do not redefine the agreed effective-path behavior to accept the gap. - -## Review validation - -- `GOTOOLCHAIN=local ./scripts/check.sh` passed at `945e111` on - `go1.26.8 linux/amd64` (formatting, whitespace, tests, vet, and build). -- `GOTOOLCHAIN=local GOPROXY=off go test -race ./...` passed. -- `GOTOOLCHAIN=local GOPROXY=off go test -count=1 ./...` passed. -- An additional external-package probe in a disposable checkout outside the - repository failed for both main and underlay placement, observing `/item/bad` - instead of `/catalog/entries/item/bad` with `source.invalid`. -- No production fixes or regression tests were added to this repository during - the review. No performance measurements or new guarantees were inferred. - -## Relative import destination review (2026-10-14) - -Reviewed `7248a8b..10edd95` against the -[relative-import roadmap](relative-import-destinations.md), completed -implementation plan (preserved in Git history at -`d9f08ff:docs/roadmap/implementation.md`), and -[ADR 0012](../adr/0012-relative-import-destinations.md). - -The implementation matches the selected architecture: literal string segments -append to each immediate parent's destination; placement uses the existing -strict-composition and final-loading pipeline; source records retain distinct -effective destinations without changing physical resource contexts. No runtime -defect, missing feature behavior, or unintended strengthening of consumer -contracts was identified. The completed input-destination finding above remains -resolved. The documentation correction identified below is now resolved. - -### 2. Restore ADR 0011's accepted decision text (P3) - -Status: complete. Restored the accepted Decision paragraph verbatim from -`7248a8b`, leaving the extension link in ADR 0011's Status section and ADR 0012 -unchanged. A comparison of everything from `## Date` onward is identical to the -baseline; the full ADR diff against `7248a8b` changes only Status. The uncached -local documentation-link check and `git diff --check` passed. The existing -`TestFacadeDestinationMappingFailurePaths` regression also passed, confirming -the earlier completed finding remains addressed. No runtime code, current -configuration/API documentation, or tests were changed. - -Affected: [ADR 0011](../adr/0011-input-destinations.md), lines 34–42 at -`10edd95`. That commit rewrote its accepted Decision paragraph to incorporate -relative import destinations. Stage 1 of the implementation plan explicitly -required preserving that text, and the -[documentation policy](../policy/documentation.md#architecture-decision-records) -treats accepted decision content as immutable. Rewriting it obscures the original -decision and its subsequent extension, despite the accurate current behavior. - -Required remediation: - -- Restore that Decision paragraph verbatim from `7248a8b` (the same paragraph - remains unchanged at `a8beb3e`). Keep the extension link in ADR 0011's Status - section and retain ADR 0012 as the accepted extension record. -- Leave the implemented relative-import behavior and current configuration/API - documentation unchanged. No additional ADR or runtime tests are needed. -- Confirm the ADR 0011 diff against `7248a8b` changes only its Status section; - run the local documentation-link and whitespace checks. - -### Relative import review validation - -At `10edd95d4d9060bd46db51e0d6782c12dbda3120`, using -`go1.26.8 linux/amd64`, all of these passed: - -- `GOTOOLCHAIN=local ./scripts/check.sh` (formatting, whitespace, tests, - examples, links, package boundaries, vet, and build). -- `GOTOOLCHAIN=local GOPROXY=off go test -race ./...`. -- `GOTOOLCHAIN=local GOPROXY=off go test -count=1 ./...`. - -The review changed only this findings document; it did not implement fixes. - -## Shared source-version review (2026-10-15) - -Reviewed `70c95bf..020d720` against the -[source-version roadmap](source-version-checks.md), completed -[implementation plan](implementation.md), and -[ADR 0013](../adr/0013-shared-source-version-checks.md). - -The root-owned mode and evidence records delegate to private per-load checking -state. Main's local header is retained, accepted non-main headers are removed -before placement/composition, and the destination-prefixed main rule reaches -payload protection and typed conversion. Warnings, raw accounting, source -contexts, and final loading reuse the existing pipeline. Four must-address -issues were identified and are now resolved below. Findings 1 and 2 above remain -completed historical findings. - -### 3. Preserve distinct semantic shapes during evidence classification (P1) - -Status: complete. Original evidence shapes are classified independently before -their results enter the existing pointer-based sensitivity/dependency closure. -Evidence no longer overwrites payload candidates. New engine and public -regressions failed before the fix and pass afterward, covering both acquisition -orders, schema tags, predicate observation of both shapes, and unchanged final -payload. Focused source-version/sensitivity/history tests, the everyday check, -and the full race suite passed on `go1.26.9 linux/amd64`. The implementation -plan records the additional Stage 9–12 regression evidence. - -Affected: `classifySecretPathsWithEvidence` and `addSecretEvidencePath` in -[secret.go](../../internal/engine/secret.go), lines 78–104 at `020d720`. - -Evidence candidates are stored in `map[string]Path`, keyed only by pointer text. -Adding a later candidate overwrites an earlier semantic Path with the same -pointer. A numeric mapping key and sequence index therefore compete for one -classification candidate. If the last shape is incompatible with a schema map, -`schemaSecretRoot` misses the secret field tag on the earlier mapping-shaped -header. Final source inspection can disclose its accepted text. - -Reproduction through `LoadLayered`: - -- Select required mode, `ParsePath("/meta/0/version")`, and a checker accepting - every declaration. -- The final Go type has ordinary `Meta map[string]Header` and - `Data.Meta map[string]SecretHeader`, with - `SecretHeader.Version string` tagged `yaml:"version" configkit:"secret"`. -- Main is `meta: {"0": {version: main}}` plus `data: {meta: null}`. -- Mount two separate underlay tiers at `Keys("data")`. The first document is - `meta: {"0": {version: KEY-HEADER-CANARY}}`; the second is - `meta: [{version: INDEX-HEADER-CANARY}]`. - -Both headers are accepted and consumed. Main's null replaces their remaining -payload, so final decoding succeeds. The mapping header's source record exposes -`Text: "KEY-HEADER-CANARY"` with `Redacted: false`, despite its schema tag. -Reversing the underlay order redacts that same header; the classification result -depends on which shape overwrites the candidate, not the established sensitivity -policy. Three repeated runs reproduced this difference. The canaries are -disposable fixture text, not real credentials. - -Required remediation: - -- Retain/evaluate all distinct original semantic candidate shapes before - combining their classification results under the existing sensitivity policy. - Preserve both evidence and payload candidates; replacing one shape with - another must not erase applicable schema or predicate classification. -- Reuse historical/dependency sensitivity closure and conservative failure - disclosure. Do not fix this by exposing less evidence, forbidding numeric - keys/sequence headers, or requiring fragment conformance to the final Go type. -- Add a public regression covering both acquisition orders, the tagged consumed - header, and unchanged final payload. Include a focused shape-collision test at - the classification boundary and reuse existing neighboring-public-value tests. -- Run source-version/sensitivity/history tests, the everyday check, and the full - race suite. Reconcile the plan's Stage 9–12 completion evidence. - -### 4. Preserve import opt-in before stripping source directives (P2) - -Status: complete. Directive opt-in is now checked on the original document -before source-version processing. The shared acquisition boundary covers main -and direct underlay sources. The public regression failed for both enabled -modes and roles before the fix and now verifies `import.disabled`, source-local -attribution, and no partial result. Focused source-version/directive/import -tests and the everyday check passed on `go1.26.9 linux/amd64`; existing enabled -import and selected-overlay tests remain green. - -Affected: `assembleBaseInputTrackedForRole` in -[load_input.go](../../internal/engine/load_input.go), lines 54–87, and -`processSourceVersion` in -[source_version.go](../../internal/engine/source_version.go), at `020d720`. - -Enabled source checking calls `splitImportDirectives` regardless of Imports. -For an input with imports disabled, it then checks `disabledDirectiveDiagnostics` -on the already stripped application payload. A valid root directive disappears -without either traversal or the required `import.disabled` error. - -For example, load bytes containing: - -```yaml -version: main -configkit: - imports: - - path: missing.yml -value: ok -``` - -Use a version rule at `Keys("version")`, an accepting checker, and leave Imports -false. Default mode correctly returns `import.disabled`. Both enabled modes -incorrectly return a successful configuration, silently omitting the requested -import. This changes import semantics even though the new mode selects only -version checking. - -Required remediation: - -- Enforce the existing directive opt-in against original source content before - stripping/consumption can hide it. Cover both ordinary main and direct underlay - acquisition through their shared boundary. Keep genuine enabled imports and - selected-overlay restrictions unchanged. -- Add a focused public regression for enabled modes with Imports false and - source-root imports. Keep default-mode behavior and safe source-local failure - attribution; do not add a second implicit import-enabling mechanism. -- Run source-version/directive/import tests and the everyday check. - -### 5. Retain the complete requested path for absent declaration evidence (P2) - -Status: complete. Lookup now clones the full requested path and refines only -segments whose container shape was actually observed. Unresolved suffixes remain -intact, and nearest source locations remain independent of the header path. -Engine regressions protect segment intent and caller ownership; public -regressions cover missing early members, scalar obstruction, and unavailable -intermediate indices through optional final/callback evidence and required-load -failure evidence. Both regressions failed before the fix and pass afterward. -Focused source-version/path/source-record tests and the everyday check passed -on `go1.26.9 linux/amd64`. The configuration contract clarifies absence evidence. - -Affected: `resolveSourceVersionPath` and `sourceVersionCheckForEvidence` in -[source_version.go](../../internal/engine/source_version.go), especially lines -74–90 at `020d720`; evidence classification uses the same partial-path fallback. - -Lookup retains only the prefix reached before absence or obstruction, and that -prefix is still a valid Path. Public evidence therefore selects it instead of -the complete configured header path. For a rule at `Keys("meta", "version")`, -an optional underlay containing `meta: false` produces `Present: false` but -`VersionCheck.Path == "/meta"`, not `/meta/version`. Missing an early member or -an intermediate sequence index likewise discards the remaining suffix. The -record identifies an existing ancestor or partial path rather than the header -whose absence was checked. - -Required remediation: - -- Preserve the full requested local path when lookup stops early. Resolve the - traversed prefix's key/index intent using actual source shape and retain - unresolved suffix intent without guessing nonexistent container kinds. -- Keep nearest physical Location separate from the declaration Path. Reuse the - complete path consistently in successful/callback/failure evidence and its - classification candidates; do not change omission policy or invent authored - coordinates below the obstruction. -- Add bounded regressions for a missing early member, scalar obstruction, and - unavailable intermediate index, including successful optional and failed - required loads. Reuse existing present-path and copy-ownership coverage. -- Run source-version/path/source-record tests and the everyday check. - -### 6. Apply structural-overlap rejection only to an actual declaration (P2) - -Status: complete. Overlap rejection now requires an actually present selected -declaration. Optional absence proceeds without a non-main checker invocation; -required absence and malformed controls still fail under their existing rules. -The public regression failed before the fix and now covers valid inheritance -in separate mounted scopes, required omission with NoSource main, and invalid -inheritance controls. Existing present-overlap tests remain green. Focused -source-version/directive/inheritance tests, the everyday check, and the full -race suite passed on `go1.26.9 linux/amd64`. The configuration contract clarifies -the omission boundary. This removes the unintended strengthened restriction. - -Affected: `processSourceVersion` in -[source_version.go](../../internal/engine/source_version.go), lines 197–219 at -`020d720`. - -The structural-overlap test runs before handling absence and examines the -partially resolved prefix. A valid scalar `extends` control obstructing a deeper -header lookup is treated as a present overlapping declaration, even though -lookup correctly classified the requested header as absent. The roadmap's -accepted overlap restriction concerns an actual header/control overlap; missing -headers remain governed by the selected mode. - -Reproduction: choose if-present mode and -`Keys("entries", "child", "extends", "version")`, with a checker accepting -main absence. Mount main `{}` at `Keys("main")`. Mount a separate underlay at -`Keys("under")` containing: - -```yaml -entries: - base: {value: good} - child: {extends: base} -``` - -The underlay has no declaration, so its checker should be skipped and its valid -inheritance should remain available. Its inheritance paths do not intersect -main's destination-prefixed protected path. Instead, the load stops with -`version.error` reporting a structural overlap. This rejects an otherwise valid -optional omission, not a present value serving two structural roles. -The same fixture succeeds in default mode. - -Required remediation: - -- Base declaration-overlap rejection on actual header presence. Continue - validating malformed source controls independently, and preserve remaining - payload/inheritance ownership checks after omission handling. -- Keep optional non-main absence permissible and required absence a missing - declaration failure. Do not widen the accepted restriction into a ban on all - configured paths beneath directive spellings. -- Add a focused load-level regression with separate mounted scopes as above, - plus missing-required behavior. Reuse present-overlap and malformed-control - tests to ensure genuine conflicts remain rejected. -- Run source-version/directive/inheritance tests and the everyday check. - -### Shared source-version review validation - -At `020d72084d5988e958c46381724e5dad78cada3d`, using -`go1.26.9 linux/amd64`, these existing checks passed: - -- `GOTOOLCHAIN=local ./scripts/check.sh`. -- `GOTOOLCHAIN=local GOPROXY=off go test -race ./...`. -- `GOTOOLCHAIN=local GOPROXY=off go test -count=1 ./...`. - -Four additional external-package regression probes reproduced findings 3–6 in -an isolated temporary module outside this repository, with a local replace to -the reviewed checkout. The full probe command was -`GOTOOLCHAIN=local GOPROXY=off GOWORK=off go test -mod=mod -count=1 -v ./...`; -the classification and absent-control cases also reproduced with `-count=3`. -The module was located at `/tmp/configkit-source-version-review.L7YGaM` during -this review; the descriptions above preserve the cases independently of that -temporary directory. - -No production fixes, repository regression tests, dependency updates, or release -changes were made. Only this findings document was revised. - -### Remediation completion validation - -All four external review probes now pass against the corrected checkout, -including both shape-collision acquisition orders. The final everyday check and -`GOTOOLCHAIN=local GOPROXY=off go test -race ./...` passed on -`go1.26.9 linux/amd64`. Each new regression was observed failing before its -corresponding production fix. - -The earlier completed Mapping diagnostic regression was rerun successfully, and -ADR 0011 from `## Date` onward still matches its accepted text at `7248a8b`. -No findings remain open. No dependencies, source-version policies, or release -gates changed, and no release was published. diff --git a/docs/roadmap/source-version-checks.md b/docs/roadmap/source-version-checks.md deleted file mode 100644 index 862e38f..0000000 --- a/docs/roadmap/source-version-checks.md +++ /dev/null @@ -1,269 +0,0 @@ -# Shared Source Version Checks Roadmap - -Status: implemented and validated. Current behavior is owned by the -[configuration contract](../config.md#shared-source-version-checks), and the -accepted decision is recorded in [ADR 0013](../adr/0013-shared-source-version-checks.md). -The completed [implementation plan](implementation.md) records execution and -validation evidence. This roadmap develops the -partially accepted source-version request in -[PromptKit's proposal](promptkit-feature-request.md#partially-accepted-addition-shared-source-specification-checks) -using the consumer's single `Definition.Version` policy. The design direction -below incorporates the maintainer's subsequent decisions on lookup, consumption, -inspection, and declaration handling; it supersedes the proposal's earlier -open-ended discussion of those subjects for this work. - -## Targeted end state - -A consumer can require that source documents contributing to a configuration -declare a version accepted by its existing `Definition.Version` checker. There -is one consumer-owned configuration specification and one acceptance policy. -Different accepted declarations may coexist: ConfigKit does not require exact -equality with the main declaration, impose SemVer, or invent an expected value. - -The consumer selects `SourceVersionMode` on ordinary and layered options. Its -zero value retains current behavior. The modes are -`SourceVersionModeDefault`, `SourceVersionModeIfPresent`, and -`SourceVersionModeRequired`. -The same `Definition.Version` can be reused across loads with different -strictness. The three modes are: - -| Mode | Missing non-main declaration | Present non-main declaration | -| --- | --- | --- | -| Current behavior, the default | Existing behavior | Existing behavior, including protected-path restrictions | -| Check when present | Proceed without a source-version callback for that omission | Apply the shared version vocabulary and checker | -| Require declarations | Structured load error | Apply the shared version vocabulary and checker | - -The main declaration remains part of the configuration and retains its existing -checker policy. The treatment of missing main input and mounted main paths is -addressed explicitly below rather than silently replacing existing behavior. -An enabled mode requires a configured version rule and checker; select a normal -options error for an unusable rule, rather than silently ignoring the mode. - -### Source-local lookup and shared policy - -In enabled modes, interpret the configured header path relative to each source -document before mounting. Read that document's own declaration, not a composed -or overlaid value. An import cannot supply its parent's missing declaration, -and another source or later tier cannot hide a rejected declaration. Check -acquired participating documents even if their application payload is later -superseded. A rejection fails the load with no usable partial result. - -Reuse the existing version vocabulary: absence, explicit null, strings, and -strict decimal integers. Null is present and reaches the checker as null; it is -not an omission. A required declaration need not be non-null if the consumer's -checker accepts null. Unsupported scalar/container kinds remain present but -fail the existing kind of version-type validation before invoking the checker. -Do not coerce malformed values into strings or treat them as missing. Preserve -exact string/integer text and the existing absence of a machine-integer range -requirement. No new string-versus-integer discriminator is required by this scope. - -This checks declared compatibility. Final decoding and domain validation still -validate the assembled application; fragments are not independently decoded as -complete applications, and declarations do not prove schema conformance. - -### Header consumption and evidence - -After a non-main declaration passes its check, consume that declaration as -structural source metadata before its application content participates in -composition. Preserve the main declaration and existing main value semantics. -Do not consume unrelated nested versions, imported payload fields at other -paths, or declarations in the default current-behavior mode. - -For example, with a source-local header path `version`, a non-main document's -`version: "1"` is checked and consumed even when that document will be mounted -under `collections/weather`. A local release label at that same path cannot -simultaneously remain ordinary payload in an enabled mode. Applications must -choose a distinct location for release metadata or for their specification -header. A prompt's `version` below a separate prompt key is unaffected unless -that is the configured header path. - -Retain each checked declaration, its source-local path, and its physical source -location as inspectable evidence. Keep this evidence separate from effective -domain values and their composition history; consumed headers should not appear -as surviving fields or create effective-value diff changes. Source destination -and resource context remain available through existing inspection. Missing -declarations need an explicit absence state and an appropriate source-location -fallback for diagnostics, not an invented authored header coordinate. - -Preserve normal safe diagnostics, warning behavior, callback lifetime, and -disclosure rules. Evidence will be attached to existing source inspection as a -detached declaration record, distinguishing unchecked sources, checked absence, -present null/text, and redaction, with local path and physical location. The -agreed sensitivity rules below govern disclosure; retaining evidence does not -authorize unredacted exposure of sensitive values or arbitrary malformed input. - -### Participation and unchanged boundaries - -Document participation must be explicit, independent of filesystem naming or -collection identity. The agreed scope is: - -| Contribution | Treatment | -| --- | --- | -| Selected main document | Shared checker; declaration retained; no duplicate main callback | -| Direct underlay documents | Check under the selected mode; accepted headers consumed | -| Imported documents, including transitive imports and glob matches | Same policy and mode; accepted headers consumed | -| Selected overlay document | Same policy and mode; accepted header consumed before overlay application | -| Host files, native filesystem files, bytes, readers, and mapping sources | Equivalent document participation under their existing source contracts | -| CLI field overrides | Synthetic field operations; no document header requirement | -| Typed defaults, resolution operations, inheritance copies, and placement wrappers | Not newly acquired documents; no additional header checks | -| `NoSource`, unmatched optional imports, and empty groups | No acquired document and no new header requirement | - -A present empty document mapping is still a document. Reuse of a source in -allowed separate tiers retains existing acquisition semantics; do not introduce -a global once-only check or a new uniqueness policy. - -Existing source acquisition, containment, import destinations, strict composition, -precedence, inheritance, and complete-on-return loading continue to apply. -Consumption permits the checked header to serve as metadata; it must not allow -remaining payload to bypass main-version ownership. Synthetic overrides remain -subject to their existing protected-path restrictions. - -This feature does not add independently versioned collections or subtrees, -per-input compatibility policies, automatic schema migration, YAML-selected -callbacks, or general source-validation hooks. These are outside this delivery, -not new permanent prohibitions on future work. - -### Agreed main-path and consumption behavior - -In enabled modes, locate main's header source-locally, retain it, and derive its -effective protected path by prefixing main's destination. Use that effective -path consistently for protection, existing typed version conversion, and -inspection. Invoke the shared checker once for main. Preserve its existing -acceptance policy for absence; required mode additionally rejects a missing -header when an actual main document participates. `NoSource` retains the existing -absent-main checker behavior because it is not a document. - -Default mode keeps the existing effective-path interpretation. Root-mounted main -inputs need no path adjustment; mounted-main callers opting in must account for -the enabled mode's source-local interpretation. No separate main-version path or -root-only main restriction is introduced. - -For nested mapping headers, consume only the selected non-main declaration and -retain authored ancestor containers, including newly empty mappings. This avoids -removing additional input presence or changing unrelated default/composition -behavior. Sequence-element and structural-directive behavior is settled below. - -## Current baseline - -Reviewed worktree baseline: `b3fc6a5`. Input destinations and relative import -destinations are implemented; the -[relative import roadmap](relative-import-destinations.md) records completion. - -- [VersionRule and Version](../../lifecycle.go) provide a path, a checker, and - absent/null/text states. There is no expected-version string or source-check - mode. Root facade conversions preserve the public/internal boundary. -- The [version contract](../config.md#application-schema-versions) checks the - main version before defaults and typed decoding. The current path is an - effective application path; changing its interpretation in enabled modes - needs explicit handling for mounted main documents. -- [Loading](../../internal/engine/load.go) invokes the main check after main - acquisition/composition. [Version validation](../../internal/engine/validation.go) - already handles scalar vocabulary, checker diagnostics, and safe callback - errors. It is a useful shared mechanism, not a ready-made source-check API. -- [Imports](../../internal/engine/imports.go), - [underlays](../../internal/engine/load_request.go), and - [selected overlays](../../internal/engine/selected_overlay.go) currently reject - contributions to the protected version path. Enabled source checks need to - extract and consume headers before payload protection and composition, without - weakening the default behavior or checking only the eventual winning values. -- Existing source records carry identity, locations, and destination, but no - dedicated checked-version evidence. Existing sensitivity and inspection are - primarily organized around application paths and retained application trees. - -## Acceptance outcomes - -The completed feature supports all three modes through ordinary and layered -public loading, with these observable outcomes: - -- One checker can accept several distinct source declarations. Rejected acquired - declarations fail the load even when later tiers would replace their payload. -- Absence, explicit null, strings, exact decimal integers, and malformed selected - values follow the shared version vocabulary and selected mode. -- Main retains its own declaration and is checked once; mounted main uses the - enabled mode's source-local interpretation, while NoSource and default mode - retain their documented behavior. -- Every participating document role and supported acquisition kind follows the - same selected policy. Synthetic field operations and absent sources add no - document-header requirement. -- Accepted non-main headers are consumed exactly, preserving authored empty - containers and ordinary sequence-removal semantics. Remaining payload retains - strict composition, precedence, protected-path rules, and ordinary versions - outside the selected header path. -- Source evidence is detached, physically attributed, available without reopening - files, and subject to the agreed sensitivity rules. Warnings survive successful - checks; generated failure messages remain safe. -- Effective values, presence, provenance, history, inheritance, rendering, diffs, - and resource contexts agree with remaining application payload. Consumed - declarations remain separate source evidence; main history stays ordinary. -- Maintained public examples demonstrate gradual and required adoption with one - policy. Current API/configuration documentation describes the delivered - contract, and existing default-mode behavior remains covered. - -Independent specification policies, release preparation, and publication are -outside this feature's completion criteria. - -## Agreed Edge-Case Decisions - -The maintainer has accepted all three decisions below. Together with the mode -API location, source participation, main-path handling, source-record evidence, -and nested mapping ancestor preservation above, these settle the discussed -product decisions. Public names and integration mechanics are recorded in the -completed [implementation plan](implementation.md); current behavior belongs -to the [configuration contract](../config.md#shared-source-version-checks). - -### Sensitivity of consumed declaration evidence - -Classify declaration evidence using its original source shape -at its mounted application path, including classified ancestors. Preserve the -header as private evidence long enough to apply applicable existing field-tag, -predicate, historical, and recorded-dependency sensitivity rules; consumption -must not make a classified declaration public. Do not require the consumed path -to survive in final effective metadata. Use explicit redaction state and omit the -classified text from public evidence while retaining presence, source attribution, -and the fact that it was checked. The checker still receives the declaration as -trusted application code, as the existing version checker does. - -For example, a header `meta.version` mounted at `collections.weather` is masked -if either that header path or an ancestor such as `collections.weather.meta` is -classified. An unclassified header remains inspectable. A later override must -not erase applicable historical classification. Sequence evidence uses original -source coordinates and segment kinds, not indexes shifted by consumption. - -On failed loads, retain the existing safe diagnostic boundary: do not include raw -header text in generated failure messages. If checked evidence is exposed through -load errors before full sensitivity is available, mask its text unless applicable -classification can already be established safely; this does not require a new -failed-load evidence API or continuing a failed load to finish classification. - -### Headers that are sequence elements - -Support consumption of a selected sequence element by deleting -that element, retaining the sequence and its order among remaining elements. -Later indexes shift in the remaining payload; an emptied sequence remains `[]`. -This is ordinary removal, not a merge strategy. Retain the header's original -index and physical location in source evidence; downstream payload paths reflect -the post-consumption sequence. Main's retained declaration is not deleted. -Removing a mapping member inside a sequence element instead retains that element, -including an empty mapping, under the agreed ancestor-preservation rule. - -This gives exact-node consumption a concrete meaning without restricting existing -path forms. Verify protected-path and typed behavior where main retains an element -but non-main payloads have shifted indexes; do not invent alignment or bypass -normal sequence replacement, decoding, or ownership rules to make such inputs fit. - -### Overlap with structural directives - -Reject an actual overlap between a checked header and an existing -structural directive node with a structured error in enabled modes. A value must -not simultaneously serve as a consumed specification declaration and as import -or inheritance control. Include a header selected inside an import declaration -or at an `extends` control; preserve ordinary structural validation so malformed -controls cannot disappear through consumption. Use source-local paths and real -locations for these errors. This is an explicitly accepted, narrow restriction -for enabled source checks, not a general prohibition on nested header paths. - -Limit the rule to structural overlap, not substrings in ordinary keys. A header -at `metadata.version` is not blocked merely because its source also contains -`configkit.imports`, and a key such as `extends_version` is not an `extends` -control. Missing headers remain governed by the selected mode; existing invalid -directives still fail independently.