Files
ConfigKit/docs/roadmap/implementation.md

616 lines
38 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Implement shared source version checks
Status: Stages 1–10 complete; public source evidence remains private.
Audience: a Luna coding agent implementing one numbered stage per prompt, in
order. The [feature roadmap](source-version-checks.md) owns target behavior and
accepted policy; this plan owns concrete implementation decisions and execution.
Planning baseline: `d9f08ff7198ad5a8fb5c49a43e7ccdad2ad0e707`.
The completed relative-import plan and evidence replaced here remain at
`d9f08ff:docs/roadmap/implementation.md`. Its roadmap and remediation are
complete, not instructions for this sprint.
## Working instructions
- Read `AGENTS.md`, [development.md](../development.md), and applicable policies.
This work involves [architecture](../policy/architecture.md),
[testing](../policy/testing.md), and [documentation](../policy/documentation.md).
Use graph-first discovery, verify live sources, and preserve unrelated work.
- Implement one stage at a time. Record completion, actual commands, toolchain,
results, limitations, and implementation revisions here. Commit each validated
nonempty unit with a short plain-English message.
- Keep public types root-owned with explicit engine conversions. Reuse semantic
Paths, documents, composition, overlay, version validation, sensitivity, and
inspection. No second loader, public parser nodes, generic transformation
hooks, or consumer-specific policy.
- Default mode preserves current behavior, including effective-path lookup,
main checker timing, and protected-path failures. Enabled modes deliberately
select source-local declarations and non-main consumption. Do not broaden
these changes into unrelated default-mode fixes.
- Do not add equality/SemVer rules, non-null requirements, numeric coercion,
path-form restrictions, source uniqueness policies, limits, dependencies,
toolchain changes, or release gates. Declaration acceptance is not fragment
schema conformance. Exclusions describe this delivery, not new permanent bans.
- Internal stages may test modes before exposing them publicly. Keep Go/current
documentation accurate to each delivered boundary; the roadmap owns the target.
- Use focused offline tests while iterating and `./scripts/check.sh` for each
production-code unit. Documentation-only units need local-link and whitespace
checks. Reuse existing tests; avoid a full mode/source/tier/path cross-product,
helper-call assertions, allocation identity, or exact message snapshots.
No independent benchmarks or performance thresholds are required.
- Do not rewrite accepted ADR decisions, publish, select a release version,
upgrade consumers, or retire completed roadmap records. Public struct additions
and enabled-mode path interpretation need minor-release compatibility guidance
during separately authorized release preparation.
## Concrete implementation decisions
### Public options and evidence
1. Add root-owned `type SourceVersionMode uint8` with consecutive constants
`SourceVersionModeDefault` (zero), `SourceVersionModeIfPresent`, and
`SourceVersionModeRequired`. Add `SourceVersionMode SourceVersionMode`
to `Options[T]` and `LayeredOptions[T]`, not Definition, Input, or YAML.
Mirror privately and convert explicitly. Unknown modes, or enabled modes
without a usable VersionRule and checker, produce `CodeOptionsInvalid`
before source access or callbacks. Keep existing rule/path validation.
2. Add root-owned `SourceVersionCheck` with `Path Path`, `Version Version`,
`Location *Location`, and `Redacted bool`. Add
`VersionCheck *SourceVersionCheck` to SourceInfo. Nil means no enabled
declaration processing occurred. A non-nil record with Present false records
observed absence, including optional omissions whose checker was skipped; it
does not claim a successful callback or acceptance. On a malformed selected
value or overlap failure, retain only known presence/null, path, and location
with conservative redaction; never expose arbitrary scalar/container content.
Read/parse failures before this phase leave the record nil.
Reuse Version's absent/null/text model.
Redaction preserves presence/null, empties classified Text, and sets Redacted.
3. Evidence Path is source-local, with semantic segment intent resolved against
original source shape where possible. Present declarations retain their actual
authored location; absence uses the nearest available source location, not an
invented header coordinate. Destination and source identity already belong
to SourceInfo. Add no new query API, raw-node field, expected-version value,
or string/integer discriminator.
4. Clone the record, Path, and Location at all query/conversion/error boundaries.
Callback-time source queries and failed-load source records expose conservative
text-redacted evidence until final classification is available. Successful
final inspection may expose unclassified text; previous copies stay detached.
Keep raw evidence separate from public-safe source records. Do not extend a
failed load to finish classification or introduce a failed-load evidence API.
### Lookup, checking, and diagnostics
5. Carry the configured local rule and an effective main-owned rule separately.
Enabled modes derive the latter once by prefixing main's Destination using
semantic segments. Use it consistently for remaining-payload protection,
inheritance/CLI checks, decoding's exact-path integer exception, and application
metadata. Default mode uses the configured effective path unchanged. Never
mutate caller rules or repeatedly prefix a path.
6. Read each declaration from its original source document before mounting or
consumption. Keep contextual numeric-key/index resolution, empty/escaped keys,
parsed tokens, and explicit indices. No mapping-only/root-only restriction;
selecting a root mapping simply fails ordinary version-type validation.
7. Missing members, unavailable indices, and obstructing non-containers mean the
requested declaration is absent, as with current document lookup. Distinguish
causes privately for accurate nearest locations, not to invent rejection of
an ancestor's domain value. If-present non-main absence skips the checker and
records absence; required absence is `CodeVersionError` before callback.
A present unsupported selected value is `CodeVersionType`, not absence.
Actual main absence in if-present mode still reaches its checker once;
required mode rejects it first. NoSource is not a document and preserves the
existing single absent-main callback in both enabled modes.
8. Reuse strict decimal recognition and exact Version.Text. No machine-integer
conversion or comparison with main. Required null reaches the checker and can
be accepted. Warnings alone succeed and accepted non-main headers are consumed.
9. Enabled declaration checkers and their Validation paths use the current
source's local coordinates for every role, including mounted main. Resolve
primary/related locations against that original source. Generated header
diagnostics also use source-local paths; do not rebase them to phantom payload
fields. Ordinary payload diagnostics remain effective-path diagnostics.
Default version diagnostics are unchanged. Reuse `CodeVersionError` for
missing-required/structural-overlap failures, `CodeVersionType` for malformed
values, and existing checker error/warning/hook codes. No new code is needed.
10. Keep callback lifetime, safe generic returned-error handling, and synchronous
execution. Check main once from its own declaration, never a composed value.
Check other documents as acquired, before composition can hide rejection.
Do not promise a new inter-source callback order or exhaustive acquisition
after failure. Build schema successfully before callbacks; enabled mode may
prepare it before acquisition, without changing the default pipeline.
Defaults, decoding, resolution, secret lookup, and validation still run once
on the assembled application.
### Consumption and structural boundaries
11. Validate source-local controls before consumption can erase malformed
directives. Reuse import/overlay validation and inheritance-control
recognition, not per-fragment inheritance expansion. Reject actual overlap
with configkit directive structure or an extends control, including headers
inside import declarations or invalid control subtrees. Unrelated siblings
and ordinary keys containing those spellings remain allowed. Missing headers
follow their mode; they do not suppress independently invalid directives.
Add no error-order guarantee.
12. Retain main's node. Remove exactly an accepted non-main mapping member or
sequence element in an owned payload tree. Preserve authored ancestors,
newly empty mappings/sequences, sibling order, and physical node locations.
Sequence deletion shifts later payload indices; evidence keeps original
coordinates. Removing a member inside a sequence element keeps that element,
even when empty. No null placeholders, ancestor pruning, or Source mutation.
13. Extract source-root imports from original content; consume headers in its
application body before placement/composition. Underlay roots and all imports
are non-main; selected overlays are also non-main. Check remaining placed
payload against effective main ownership. Consumption does not permit
forbidden overlay controls, sequence alignment, or CLI ownership bypass.
14. Charge original acquired bytes/nodes once, including consumed headers. Apply
existing placed-body bounds and full-destination wrapper charges to remaining
payload. Keep source/depth/glob accounting, records, resource contexts, and
physical repetition unchanged. Acquired header-only mappings retain their
empty body/ancestors and placement; NoSource/optional absent sources add no
document or evidence. No new cap or full-source clone is required.
### Evidence and sensitivity
15. Carry private evidence per acquired SourceID in per-load acquisition/assembly
state, separate from application history. Retain an owned scalar/absence,
original local and mounted semantic paths, location, and the ancestor-shape
context needed for classification. Capture key/index intent before deletion.
Retain only needed context, not entire source-tree clones or loading handles.
16. At publication classify evidence at its original mounted path and ancestors
with existing schema tags, predicate, historical projected sensitivity,
inheritance, and recorded-dependency propagation. Include evidence-only
candidates in classification without adding them to Metadata paths, payload
applications, rendering, or Diff. Do not infer original sequence shape from
shifted payload or round-trip through pointer strings. Reuse dependency
closure rather than building a weaker independent redactor. Retained header
text never triggers credential lookup.
17. Sanitize source records before public conversion; early records remain
conservatively redacted. Final successful evidence uses completed sensitivity,
retaining historical classification despite replacement. Main's normal
payload/history remains intact. Consumed non-main headers are only source
evidence: equal payloads with different headers must not acquire effective
differences merely from that evidence.
## Implementation owner map
| Responsibility | Existing owners to inspect and extend |
| --- | --- |
| Public options, callbacks, records, conversions | `lifecycle.go`, `layered.go`, `records.go`, `conversion.go`, `diagnostic.go` |
| Options and per-load coordination | `internal/engine/options.go`, `load.go`, `load_request.go`, `layered.go` |
| Acquisition, directives, placement, overlay | `internal/engine/load_input.go`, `imports.go`, `directives.go`, `destination.go`, `selected_overlay.go` |
| Checking and semantic paths | `internal/engine/validation.go`, `document_walk.go`, `version_path_test.go` |
| Evidence copies, errors, publication | `internal/engine/records.go`, `diagnostic.go`, `load_result.go` |
| Classification and propagation | `internal/engine/secret.go`, schema secret-path helpers, `secret_propagation_test.go`, `render_test.go` |
| Compatibility tests | Engine `load_test.go`, `imports_test.go`, `selected_overlay_test.go`, `load_request_test.go`, `destination_*_test.go`; root `facade_version_test.go`, `facade_layered_test.go`, `facade_destination_test.go`, `records_test.go` |
Private helper names/file splits may follow the implementation while preserving
these decisions. A new internal package solely for headers is unnecessary.
## Validation commands
Use the applicable subset during each stage; add new tests under the
SourceVersion name family so focused checks include them. Existing test owners
remain authoritative. Commands run from the repository root with dependencies
already available; these are development checks, not new release gates.
```sh
GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Version|Import|Underlay|Overlay|Destination'
GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Secret|Sensitivity|History|Inheritance|Resolution'
GOTOOLCHAIN=local GOPROXY=off go test . -run 'SourceVersion|Version|Layered|SourceInfo|Destination|PublicFacadeBoundary|DependencyImportBoundary'
GOTOOLCHAIN=local GOPROXY=off go test ./examples/consumers
GOTOOLCHAIN=local GOPROXY=off go test . -run '^TestLocalDocumentationLinks$'
GOTOOLCHAIN=local ./scripts/check.sh
GOTOOLCHAIN=local GOPROXY=off go test -race ./...
git diff --check
```
## Stage 1 — Capture the baseline and record the extension
1. Confirm the worktree/baseline; read the roadmap, plan, applicable policies,
contract, and relevant ADRs. Run the everyday check and record revision,
toolchain, results, and environmental limitations.
2. Write Accepted ADR `0013-shared-source-version-checks.md` after checking the
number is unused. Record one policy, modes, source-local checking, consumption,
evidence, and retained main protection. Focus on ConfigKit's selected choice;
rejected choices belong only in Alternatives Considered. Keep sequencing here.
3. Preserve prior accepted decision text. Add status/navigation links where
useful. Note struct additions and enabled mounted-main interpretation as
minor-release considerations; do not select a version or write release notes.
Validate: local-link test and whitespace. Done: baseline and accepted decision
recorded, with no runtime changes.
Completion evidence: reviewed baseline `d9f08ff7198ad5a8fb5c49a43e7ccdad2ad0e707`
and the clean implementation worktree at `70c95bf660794809e58182d736c67c9085238705`;
confirmed ADR 0013 was unused. The everyday check passed with Go
`go1.26.8 linux/amd64` using
`GOTOOLCHAIN=local GOPROXY=off ./scripts/check.sh`. Documentation links and
whitespace are validated for this unit. No environmental limitation affected
these checks; validation was performed on Linux/amd64 only. No runtime changes
were made.
## Stage 2 — Share version interpretation without changing defaults
1. Separate reusable value interpretation and callback reporting from checkVersion,
retaining its default entry point, absence/null/text, exact integers, warnings,
safe errors, primary/related locations, and context closure.
2. Reuse existing scalar/path/hook/lifetime tests; add only missing protection for
the refactor. Introduce a small private result separating checked value and
diagnostics. Failure decisions must use severity, not slice length.
3. Do not globally rewrite unrelated diagnostic plumbing or public behavior.
Validate: engine version/validation and facade version tests; everyday check.
Done: shared mechanisms exist with unchanged default behavior.
Completion evidence: `checkVersion` remains the existing diagnostic-only entry
point and now delegates to private scalar interpretation and callback-reporting
helpers through a result that keeps the interpreted `Version` separate from
diagnostics. Checker invocation and returned-error fallback use diagnostic
severity, preserving warnings without treating them as failures. Added a
regression for a warning followed by a returned checker error. These checks
passed with Go `go1.26.8 linux/amd64`:
- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'Version|Validation'`
- `GOTOOLCHAIN=local GOPROXY=off go test . -run 'Version'`
- `GOTOOLCHAIN=local ./scripts/check.sh`
No environmental limitation affected validation; checks ran on Linux/amd64.
## Stage 3 — Implement source-local extraction and exact consumption
1. Add a private processor receiving original document, local rule, source role,
destination, and mode. Implement decisions 6–12, including missing/obstructed
lookup and pre-consumption structural overlap.
2. Return owned remaining payload, original import declarations, and private
evidence with original segment/ancestor context. No second parser, fragment
decoding, or mutation of reusable input.
3. Add focused tests for vocabulary, missing/obstructed locations, malformed
selected nodes, literal/parsed numeric keys and indices, empty ancestors,
sequence deletion, and actual versus unrelated structural overlap.
Validate: helper/version-path/directive/inheritance-control tests and everyday
check. Done: the processor is tested independently of public loading.
Completion evidence: added a private source-local processor that resolves the
configured path against the original document, separates imports, records
private path/ancestor/location evidence, and consumes accepted contribution
headers only from an owned payload copy. Missing and obstructed paths retain
their cause and nearest available location; required absence fails before the
checker, while optional main absence still invokes it. Structural directive
overlap is rejected before payload consumption. Focused coverage includes the
version vocabulary, malformed nodes, contextual numeric keys and indices,
empty ancestors, sequence deletion, and actual versus unrelated structural
overlap. These checks passed with Go `go1.26.8 linux/amd64`:
- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Version|Import|Underlay|Overlay|Destination|Inheritance'`
- `GOTOOLCHAIN=local ./scripts/check.sh`
- `git diff --check`
No environmental limitation affected validation; checks ran on Linux/amd64.
## Stage 4 — Add private mode state and main-document integration
1. Add the engine mode to ordinary/layered options and loadRequest. Validate it at
both entry points before source access. Underlay helper options must not be
treated as standalone loads missing the shared rule.
2. Carry local/effective rules, source roles, warnings, and evidence in per-load
state. Let acquired inputs return evidence and nonfatal diagnostics.
3. In enabled mode, check actual main before imports/composition, retain its node,
and prepare schema before callbacks. Skip the old later main check only in
enabled mode. Preserve NoSource's absent callback and default timing.
4. Thread effective protection through typed conversion, inheritance, CLI, and
application inspection while keeping declaration callbacks local. Test mounted
main, absence/NoSource, null acceptance, invalid options before reading, and
exactly one main callback.
Validate: load/grouped/version/destination/hook tests and everyday check. Done:
private enabled main behavior works without changing old callers.
Completion evidence: added the private mode to ordinary and layered engine
options and the shared load request. Both entry points validate enabled modes
and their shared checker before source acquisition; enabled loads build schema
before invoking the main checker. The parsed main document is checked at its
source-local path before import traversal or layered composition, and its node
remains in the application payload. A single semantic destination-prefixed rule
now protects mounted main ownership during decoding, inheritance, acquired-input
placement, overlays, and CLI overrides; declaration callbacks and diagnostics
retain local paths. Default mode keeps its prior post-composition check timing.
The per-load assembled input retains main evidence and successful warning
diagnostics. NoSource invokes the existing absent-main callback once in either
enabled mode. These checks passed with Go `go1.26.8 linux/amd64`:
- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Version|Load|Grouped|Destination|Hook'`
- `GOTOOLCHAIN=local ./scripts/check.sh`
- `git diff --check`
No environmental limitation affected validation; checks ran on Linux/amd64.
The engine mode remains private; public options are scheduled separately.
## Stage 5 — Integrate recursive imports, warnings, and accounting
1. Thread shared state through import entry/traversal. Check each acquired child
once against its own local header, then place and compose remaining payload.
Keep nested/glob effective destinations and original directive discovery.
2. Apply effective main protection afterward. Preserve physical identity,
cycle/repetition, optional absence, source records, resource contexts, and raw
node/byte charges. Header-only acquired mappings still contribute empty bodies.
3. Propagate warnings to load diagnostics without aborting traversal. Update only
relevant callers currently treating any diagnostic as failure.
4. Cover nested/glob imports, mounted relative destinations, accepted unequal
declarations, rejected hidden declarations, required omissions, safe locations,
optional missing sources, raw accounting, and default import.version failures.
Reuse existing containment/glob/source-kind tests.
Validate: import/glob/destination/accounting/version tests and everyday check.
Done: imports use one policy without lost warnings or changed physical traversal.
Completion evidence: threaded one per-load source-version state through the
main import entry and recursive traversal. Each parsed child is checked against
its source-local rule before placement; accepted headers are removed from an
owned payload while the original root still supplies raw byte/node charges and
the processor's original import declarations drive nested traversal. Effective
main ownership is checked against the remaining placed payload. Evidence is
retained with the acquired SourceID, and warning diagnostics accumulate without
stopping traversal. Existing identity, cycle, optional-source, source-record,
resource-context, and accounting paths remain in use. Added coverage for nested
and glob imports at mounted relative destinations, accepted unequal declarations,
rejected hidden declarations, required omissions with source-local locations,
optional missing imports, evidence identity, and raw header byte/node charges. The
existing default import-version failure coverage also passes. These checks
passed with Go `go1.26.8 linux/amd64`:
- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Version|Import|Glob|Destination' -count=1`
- `GOTOOLCHAIN=local ./scripts/check.sh`
- `git diff --check`
No environmental limitation affected validation; checks ran on Linux/amd64.
## Stage 6 — Integrate direct underlays and their strict groups
1. Thread state through acquireUnderlayGroup/grouped assembly. Direct underlays
are non-main; imports reuse Stage 5. Preserve layers, indices, contexts,
evidence, and warnings across groups.
2. Consume local headers before strict composition, then protect remaining placed
payload. Preserve tier precedence, empty containers, raw budgets, and reusable
sources in allowed separate tiers.
3. Cover accepted distinct peer declarations, a rejected underlay hidden by main,
required absence, empty/NoSource groups, mounted nested headers, retained
payload conflicts, and unchanged default underlay.version failures.
Validate: grouped/underlay/destination/version tests and everyday check. Done:
direct and imported underlays follow the same checking policy.
Completion evidence: threaded the per-load version state through direct underlay
group acquisition and invoked the shared processor with contribution semantics
for each acquired underlay root. Its own processed payload/import declarations
now drive placement and recursive imports, while remaining placed payload is
still checked against the effective main-owned path before strict composition.
Warnings are accumulated through underlay groups; source records retain their
layer and group index, and the existing accounting, context, and reusable-source
paths remain in place. Added coverage for distinct declarations within one
strict group, warning retention across groups, rejected declarations beneath a
later main value, required omissions, empty groups and NoSource, mounted nested
headers/imports, and protected remaining payload. Existing default underlay
version-protection coverage remains passing. These checks passed with Go
`go1.26.8 linux/amd64`:
- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Version|Import|Underlay|Overlay|Destination' -count=1`
- `GOTOOLCHAIN=local ./scripts/check.sh`
- `git diff --check`
No environmental limitation affected validation; checks ran on Linux/amd64.
## Stage 7 — Integrate selected overlays and final ownership checks
1. Process overlay headers after parse/structural validation, before remaining
payload protection and application. Reuse shared state in ordinary and layered
overlay loading; preserve source identity, namespaces, close behavior, and raw
node/byte charges.
2. Carry accepted warnings through assembly. Keep overlay-control restrictions,
and effective main protection through CLI and inheritance.
3. Cover accepted/required/rejected overlay headers, warnings, unchanged default
behavior, ordinary CLI fields without header requirements, mounted-main
protection, and protected payload after nested/sequence consumption. Sequence
evidence stays original while remaining indices shift.
Validate: overlay/override/inheritance/version tests and everyday check. Done:
all document roles are integrated privately; synthetic operations are exempt
from header checks, not from existing protection.
Completion evidence: threaded the shared version state through ordinary and
layered selected-overlay loading. After parse and overlay-directive validation,
the selected overlay is checked as a non-main source; accepted headers are
removed from its owned payload before effective-path protection and application.
The original parsed node still supplies raw byte/node charges, and source
identity, filesystem namespace, closure, overlay-control checks, and override
protection remain on their existing paths. Overlay warnings are accumulated with
the other per-load diagnostics. Added coverage for accepted, required-missing,
and rejected overlay declarations; structural validation before callbacks;
warning retention; CLI overrides without document-header checks; mounted-main
protection after header consumption; and sequence deletion retaining original
evidence coordinates while later payload indices shift. Existing default overlay
version/directive/accounting coverage remains passing. These checks passed with
Go `go1.26.8 linux/amd64`:
- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Version|Import|Underlay|Overlay|Override|Inheritance|Destination' -count=1`
- `GOTOOLCHAIN=local ./scripts/check.sh`
- `git diff --check`
No environmental limitation affected validation; checks ran on Linux/amd64.
## Stage 8 — Expose the public load-wide mode
1. Add root mode/constants/option fields and both conversions from decision 1.
Preserve zero-value behavior and root-owned boundaries.
2. Add external-package tests for ordinary/layered enabled loads, zero-mode
compatibility, invalid mode/rule before reading, and reuse of one Definition
with different modes. Use representative source-kind coverage.
3. Document implemented mode behavior in Go declarations, including local versus
effective interpretation. Evidence remains private until Stage 11; do not
claim it is publicly available yet.
Validate: facade version/layered/source and boundary tests; everyday check.
Done: both supported public load APIs expose the implemented modes.
Completion evidence: added the root-owned `SourceVersionMode` enum and constants
to ordinary and layered options, with Go documentation for zero-value behavior,
source-local declaration paths, destination-prefixed main protection, and the
two enabled policies. The facade converts explicitly through the internal
engine boundary, which keeps its mode type private and preserves unknown values
for option validation before acquisition. Added external-package coverage for
enabled ordinary and layered loads using byte, file, and mapping sources; zero
mode compatibility; invalid modes and missing rules before reader access; and
reusing one Definition with different modes. Source evidence remains private.
These checks passed with Go `go1.26.8 linux/amd64`:
- `GOTOOLCHAIN=local GOPROXY=off go test . -run 'SourceVersionMode|Version|Layered|SourceInfo|Destination|PublicFacadeBoundary|DependencyImportBoundary' -count=1`
- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Version|Layered|Source|Destination|Import' -count=1`
- `GOTOOLCHAIN=local ./scripts/check.sh`
- `git diff --check`
No environmental limitation affected validation; checks ran on Linux/amd64.
## Stage 9 — Classify evidence at original paths and ancestors
1. Include declaration paths/ancestors in final classification candidates,
independently of effective/historical Metadata indexes. Do not add consumed
nodes to applications or explanation replay.
2. Apply schema-tag and predicate rules using original mounted semantic context,
including tags beneath dynamic containers and ancestor classification.
Preserve numeric-key/index intent before sequence shifts; consumed paths need
not survive final decoding. Reuse existing redaction mechanisms.
3. Test private evidence with disposable canaries: unclassified text, direct/
ancestor tags and predicates, original sequence shape versus shifted payload,
and null/absence. Keep raw evidence private throughout this stage.
Validate: source-version/sensitivity/rendering tests and everyday check. Done:
direct/ancestor classification is correct without making headers payload history.
Completion evidence: carried private source-version evidence from per-load state
into the resolved publication input, without adding consumed declarations to
applications or metadata. Final classification now evaluates each declaration's
original mounted semantic path and captured ancestors with the existing schema
tag and predicate rules, then uses the existing sensitivity result for rendering.
This covers dynamic map/slice paths, typed original sequence indices, and
null/absent evidence while leaving unclassified evidence and shifted neighboring
payload visible. Evidence remains private. These checks passed with Go
`go1.26.8 linux/amd64`:
- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Secret|Sensitivity|History|Inheritance|Resolution' -count=1`
- `GOTOOLCHAIN=local ./scripts/check.sh`
- `git diff --check`
No environmental limitation affected validation; checks ran on Linux/amd64.
## Stage 10 — Preserve historical and dependency sensitivity
1. Connect evidence candidates to existing historical, inheritance, and recorded
dependency propagation. Later replacement or consumption must not erase
applicable classification; do not invent new propagation policy.
2. Consider evidence-only paths even when absent from final metadata. Keep
Resolution.Record's existing schema-path validity; do not invent dependencies
on headers solely to create tests.
3. Cover displaced historical classification and legitimate recorded dependencies,
with inheritance where applicable. Ensure neighboring unclassified evidence
remains visible and no synthetic effective/history/diff paths are introduced.
Validate: source-version/history/inheritance/resolution/sensitivity tests,
`go test -race ./internal/engine`, and everyday check. Done: established
sensitivity closure applies without polluting payload history.
Completion evidence: added original mounted evidence paths and ancestors to the
existing classification path set before schema/predicate evaluation and the
shared dependency closure. This lets existing recorded dependencies and
inheritance links carry evidence sensitivity into dependent and displaced
historical values, including evidence-only paths absent from final metadata;
Resolution.Record validation and dependency policy are unchanged. Added a
load-level regression using a valid recorded dependency on the credentials
container (not a synthetic dependency on the header), plus inherited displaced
history coverage. Neighboring values remain visible, and consumed headers create
no metadata, explanation, or Diff paths. These checks passed with Go
`go1.26.8 linux/amd64`:
- `GOTOOLCHAIN=local GOPROXY=off go test ./internal/engine -run 'SourceVersion|Secret|Sensitivity|History|Inheritance|Resolution' -count=1`
- `GOTOOLCHAIN=local GOPROXY=off go test -race ./internal/engine`
- `GOTOOLCHAIN=local ./scripts/check.sh`
- `git diff --check`
No environmental limitation affected validation; checks ran on Linux/amd64.
## Stage 11 — Publish detached sanitized source evidence
1. Add private/public SourceVersionCheck and SourceInfo.VersionCheck records.
Extend both conversions and all clone paths, including callback source queries,
successful inspection, and existing LoadError source access.
2. Before final classification, expose conservative sanitized records; after
successful classification, freeze final sanitized evidence. Keep local paths,
physical locations, absence/null, and unchecked distinction. Raw evidence
stays in private state; no loading handle is retained.
3. Add facade/copy tests mutating nested paths/locations and records. Repeated
queries and independent loads remain unaffected. Delete sources after load
and verify evidence/resource metadata without reopening files.
4. Cover early acquisition and later resolver/validator failures plus callback
source queries with canaries. Default and synthetic CLI records stay nil.
Validate: record/conversion/source/version/ownership and public boundary tests,
root/engine race checks, and everyday check. Done: existing public source
inspection surfaces safely expose detached evidence.
## Stage 12 — Audit lifecycle agreement and compatibility
1. Map tests to every roadmap participation row/edge decision; add only missing
cross-boundary regressions with one primary owner per behavior.
2. Verify final values, zero/false/null presence, defaults, exact-path integer
conversion, resolution, secret lookup, and validation after consumption.
Hooks see one assembled configuration. Accepted declarations need not match.
3. Compare equal payloads with differing consumed headers: evidence changes but
effective rendering/history/Diff gain no consumed fields. Main history stays
normal. Cover empty ancestors and ordinary versions outside the header path.
4. Check physical contexts after placement/inheritance/precedence, existing
accounting boundaries, all supported document source kinds, and default
import/underlay/overlay/CLI/version compatibility.
5. Run `./scripts/check.sh` and
`GOTOOLCHAIN=local GOPROXY=off go test -race ./...`. Record actual evidence;
do not claim untested platforms or consumer adoption. Consumer checks are not
new release gates.
Done: all target behavior has public-boundary evidence and deterministic checks
pass, without extra contract restrictions or redundant coverage.
## Stage 13 — Add maintained adoption examples
1. Add a small executable example showing if-present and required adoption with
one Definition.Version policy, accepted distinct declarations, main/import/
underlay or overlay sources, and safe evidence inspection.
2. Include a mounted input illustrating local lookup and effective main
protection. Keep ordinary release metadata at another path; no consumer-side
second load/merge, collection checker, or implicit schema migration.
3. Keep copyable assets in examples with tests for final payload and safe output.
Reuse existing examples for remaining source plumbing; do not print canaries
or promise callback ordering.
Validate: example/consumer tests, links, and everyday check. Done: tested examples
demonstrate the coherent shared policy with reduced consumer plumbing.
## Stage 14 — Reconcile documentation and finish the audit
1. Update Go docs and docs/config.md as canonical owners for modes, participation,
local diagnostics, consumption, effective protection, mounted main, evidence
states/disclosure, and default behavior. Explain conservative callback-time
redaction and declared compatibility versus final validation. Link from
consumer guidance, development, and internal overview.
2. Update the architecture ownership summary where needed, linking to contract
and ADR rather than duplicating mechanics. Preserve previous accepted ADR
decisions. Reconcile obsolete design/consumption notes in the source request.
Repair completed relative-import roadmap/remediation links that meant the old
overwritten plan, using the Git-history pointer above.
3. Audit for accidental default checks, equality/non-null rules, path grammar,
fragment decoding, ordering promises, extra uniqueness/caps, unsafe evidence
queries, and new release gates. Match every roadmap outcome to test evidence.
4. Mark plan/roadmap complete only after all stages pass. Run everyday checks,
full race suite after code corrections, and final link/whitespace checks.
Commit the validated unit; leave a clean workspace. Do not publish or retire
completed records in this stage.
Done: implementation, coverage, documentation, and approved scope agree. No
product questions remain open under the roadmap and concrete decisions above.