Files

Eric Rakestraw 4bf940c579 Use deferred acquisition consistently

2026-10-25 22:28:35 +00:00

62 KiB

Raw Blame History

Audit Remediation Implementation Plan

Status: in progress; completion is recorded per stage below. Prepared: 2026-10-25. Planning baseline: 4a5316b7fc2da57dcced33190cb6499c9a1beb5c. Audience: a Luna coding agent implementing the stages sequentially.

Scope And Execution

Implement the 29 selected entries in the roadmap. The audit owns the evidence, accepted scope, and detailed acceptance cases. This plan owns implementation order, private design choices, validation, and execution progress. Preserve the finding IDs when recording completion. The production baseline is unchanged by the intervening audit and planning commits; nevertheless, inspect actual callers before deleting code. Follow AGENTS.md, agent policy, and the development guide.

This is one end-to-end sprint. Intermediate stages may temporarily fail to compile or have pending tests. Do not create compatibility shims, duplicate implementations, or extra public interfaces just to make each stage independently green. Run the focused checks when their affected code is buildable; otherwise record exactly what is pending and run it once the consuming stage lands. Never describe an unfinished stage or deferred check as complete. Commit focused, nonempty units under repository policy, with plain-English messages and accurate validation notes. Final completion requires the integrated end state and checks, not just completion markers.

For each stage, record its status, affected finding IDs, and actual validation here. Include already known commit identifiers in later stage updates; report the final commit in the handoff rather than making a commit self-referential. Keep execution status in this plan rather than maintaining another progress inventory in audit.md or roadmap.md. A finding split across stages closes only after all of its stages and acceptance cases are satisfied.

Boundaries

Preserve supported public types, signatures, root ownership, configuration syntax, tier semantics, collection operations, source/version conventions, callback ordering, diagnostics, confidentiality, and detached ownership.
Repair the nine reported defects and their regressions. Safe rejection of invalid values, acceptance of supported finite values, and enforcement of the existing quotas are remediation, not new restrictions.
Keep panic propagation, borrowed-resource ownership, regular-file close policy, partial bootstrap/writer behavior, and existing optional-absence rules.
Keep raw recorded resolution snapshots, active history, and required copies. Remove only the proven unused copies and evidence named below.
C03 glob readability extraction and F07 wider facade transport simplification remain deferred in future work. Do not redesign the clone visitor, explanation replay, public metadata handoff, or cross-load schema caching.
Add no dependencies, public options, global caches, traversal caps, background activity, release gates, or performance guarantees. Do not publish a release, update consumer dependencies, or edit accepted ADR decisions during this sprint.

Validation Conventions

Commands below run from the repository root with the installed compatible Go toolchain and already available dependencies. Use GOTOOLCHAIN=local, GOPROXY=off, and GOTELEMETRY=off for offline checks. Keep disposable diagnostics, coverage files, and binaries outside the repository. Do not change go.mod or go.sum to obtain audit tooling.

Focused -run expressions identify behavior owners, not required names for new tests. Include the new regression explicitly in the focused invocation if its name falls outside the expression. Extend existing test owners; do not duplicate the internal matrix through every facade. Tests for termination use a bounded child process/watchdog where a regression could crash or hang the test process; that watchdog is not a performance threshold. Reuse the subprocess pattern in internal/engine/source_fifo_test.go where appropriate.

Use small controlled budgets and bounded filesystem operation observation for resource/scaling cases. No benchmark campaign, exact syscall choreography, coverage percentage, or timing ratio is an acceptance criterion. Final development, race, consumer, and documentation validation is specified in Stage 33.

Stage 1 — Terminate Pointer-Only Descriptor Traversal

Finding: A01; also the schemaPath portion of F06. Owners: internal/engine/schema.go, decode.go, secret.go, and their tests.

Status: complete (A01; schema-path portion of F06). Validation passed: GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Schema|Decode|Resolution|Secret|PointerOnly' -count=1, the specified focused check, and GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1.

Introduce one private pointer-descriptor terminal lookup that detects a repeated descriptor, using a traversal-local visited set. Return failure for a pointer-only loop; do not reject that type during schema construction.
Use it where decoding must unwrap a non-null pointer chain, in acceptsMapping, and before schema/secret path traversal consumes the next segment. A non-null value for an uninhabitable pointer-only chain produces the existing CodeDecodeType classification at the input path/location. Keep null handling before this check so supported nil pointers still decode. For decoding a finite chain, peel the base/descriptors iteratively, decode the terminal value once, and rebuild pointer wrappers in reverse. Preserve existing typed-default behavior; do not rescan the remaining chain at every recursive pointer call and introduce quadratic chain traversal.
Remove unread schemaPath.field and schemaPath.dynamic and the terminal unwrapping used only to compute them. A path ending at a nullable pointer field may still resolve to its descriptor; reject attempts to descend through its pointer-only loop. Update consumers/tests accordingly.
Review other pointer-unwrapping helpers. Apply the shared check to analogous descriptor-only loops; retain finite value traversal and existing inline-field restrictions rather than adding a blanket recursive-type rejection.

Acceptance: reproduce type Loop *Loop non-null decoding and helper-path termination in bounded subprocesses; verify null/omitted values, direct nullable field operations, and normal recursive structs with acyclic values. Check safe codes/paths rather than raw values or exact new wording. Focused check: go test ./internal/engine -run 'Schema|Decode|Resolution|Secret'.

Stage 2 — Roll Back Failed Dynamic Descriptor Builds

Finding: A02. Owners: internal/engine/schema.go, value_operation.go, projection/resolution tests.

Status: complete (A02). Previous completed unit: 2736ddf. Validation passed: GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Dynamic|Projection|Clone|Resolution' -count=1 and GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1.

Give schemaBuilder a journal of types newly inserted during a build. Record insertion only after a cache miss; cached, completed descriptors are borrowed and not mutated. Recursive new descriptors may refer to one another through their provisional entries.
In dynamicDescriptor, treat construction plus dynamic-secret eligibility as one operation. On failure, delete every journaled cache entry, including ancestor/container placeholders, before returning the existing path-specific projection issue. Deleting an entry already removed by a local failure is safe.
On success, retain the completed additions in the existing load-local cache. Do not clone the entire cache on each request, cache failures, or clear valid preexisting entries. Initial schema construction can use the same journaling mechanics without introducing another persistent cache.

Acceptance: the two Normalize calls from A02—typed nil then non-nil pointer to an unsupported dynamic struct—report safe failures without a panic. Cover nested and mutually recursive provisional entries, subsequent valid operations, prior valid cache entries, and repeated failures at different paths in one load. Focused check: go test ./internal/engine -run 'Dynamic|Projection|Clone|Resolution'.

Stage 3 — Distinguish Finite Slice Views From Cycles

Finding: A03. Owners: internal/engine/value_operation.go, value.go, clone.go, and tests.

Status: complete (A03). Prior stage commits: 2736ddf, deb9aa5. Validation passed: GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Mapping|Value|Projection|Clone|Override|Cycle|FiniteSlice|SliceCycles' -count=1 and GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1.

Add visible slice length to valueVisit identity in visitFor. Keep type, kind, and starting pointer, existing empty/nil handling, and active-path removal on unwinding. Capacity is not part of the visible traversed graph. Maps and pointers retain their current identity rules. Follow all shared callers rather than patching only one projection entry point.

Acceptance: shared := []any{"leaf", nil}; shared[1] = shared[:1] succeeds through mapping input, typed defaults/projection, cloning, and CLI override construction. Equivalent detached values agree. Actual self/mutual cycles still fail safely; mutation of an existing returned descendant must not affect the original. Focused check: go test ./internal/engine -run 'Mapping|Value|Projection|Clone|Override|Cycle'.

Stage 4 — Check Container Lower Bounds Before Bulk Allocation

Finding: A04. Owners: internal/engine/projection.go, value_operation.go, projection_test.go.

Status: complete (A04). Prior stage commits: 2736ddf, deb9aa5, ae3408a. Validation passed: GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Projection|Inline|Resolution' -count=1 and GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1.

After charging a container node and before its bulk reservation, compare its minimum remaining cost with the current node allowance: at least one node per sequence element and one key plus one value node per map entry. Use comparisons or division against remaining allowance, not overflowing size multiplication.
Reject an impossible container with CodeSchemaProjectionLimit before make/map-key materialization. Preserve negative/unlimited internal allowances, per-child charging, depth checks, key sorting, and inline-container accounting.
Do not generalize this into a new container-length or heap-size limit. Review related reservations for the same known-lower-bound error without redesigning typed cloning or making unrelated allocation promises.

Acceptance: bounded tests cover exact/insufficient remaining allowances for maps, slices, arrays, and flattened inline containers. Use a child process for the maximum-length zero-sized slice case; prove safe rejection without a huge backing allocation. Check defaults and resolution paths using the shared projector. Focused check: go test ./internal/engine -run 'Projection|Inline|Resolution'.

Stage 5 — Close Callback Contexts On Every Exit

Finding: B01. Owners: internal/engine/resolution.go, validation.go, callback tests.

Status: complete (B01). Prior stage commits: 2736ddf, deb9aa5, ae3408a, ea27f17. Validation passed: GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Callback|Resolution|Validation|Version' -count=1 and GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1.

Wrap each application callback invocation in a small function with deferred context deactivation. The defer must run when that callback returns or panics, not at the end of subsequent projection/auditing work. Apply this independently to resolution, final validation, and the shared version checker (reportVersion). Retain the existing error/warning handling and propagate the original panic unchanged; do not recover, synchronize, or broaden concurrent callback support.

Acceptance: external recovery retains each context and confirms its operations now produce the established closed-context panic. Use a distinct panic sentinel to verify original propagation. Retain normal-return closure and lifecycle tests. Focused check: go test ./internal/engine -run 'Callback|Resolution|Validation|Version'.

Stage 6 — Close The Owned Import Root During Panic Unwinding

Finding: B02. Owners: internal/engine/import_acquisition.go, filesystem_access.go, import tests.

Status: complete (B02). Prior stage commits: 2736ddf, deb9aa5, ae3408a, ea27f17, 08a8c25. Validation passed: GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Import|SourceVersion|Filesystem' -count=1 and GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1.

Install deferred fallback closure at the entry of acquireImportedWithAccess, before operations that may invoke application code. Use a local closed flag or one local close-once function so existing explicit close sites preserve their normal timing, return diagnostics, and primary-error precedence. Mark closure attempted even if closing returns an error; do not re-close an owned handle or close the borrowed native filesystem. The deferred fallback must not replace a propagating callback panic with a cleanup failure.

Acceptance: capture a real owned os.Root when invoking this acquisition boundary with temporary host files, make an imported version checker panic, recover, and verify that operations on that same root report closed state. Also verify child file closure and normal/error exits. This observes the actual host root without requiring a production FD counter or treating native file counts as equivalent. Document genuine platform prerequisites; do not create a human validation gate. Focused check: go test ./internal/engine -run 'Import|SourceVersion|Filesystem'.

Stage 7 — Preserve Directory Failure Phase And Per-Handle Evidence

Finding: C01, including its integrated test improvements. Owners: internal/engine/filesystem_access.go, import_selection.go, glob.go, native_filesystem_test.go, and resource_context_test.go.

Status: complete (C01). Prior completed commits: 2736ddf, deb9aa5, ae3408a, ea27f17, 08a8c25, 877c497. Validation passed: env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Directory|Glob|Filesystem|Resource' -count=1, env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1, and git diff --check.

Return private directory failure information distinguishing unsuccessful initial open from enumeration/close failures. Preserve error unwrapping for classification; do not expose backing error text.
Apply optional/missing behavior only to ErrNotExist from the initial open. Batched ReadDir, no-progress, visit-stop closure, and final close failures after opening are failures. Keep read failure ahead of an accompanying close failure, and preserve existing selector diagnostics and entry accounting.
The fallback closes the initial file before using fs.ReadDir on the native filesystem. Since initial opening succeeded, any fallback error, including ErrNotExist, is a post-open failure. Preserve standard ReadDirFS dispatch instead of imposing a new native-filesystem interface requirement.
Update both directory and glob selectors and all callers. Enhance the shared fake to track individual opened-handle closure, including the fallback, rather than only aggregate counts. Do not change regular-file close-error policy.

Acceptance: directory plus glob, batched plus fallback paths, data/entries with errors, read/close ErrNotExist, real missing-on-open controls, and a successful second directory that must not hide failure. Assert nil result, structured code/source, per-handle closure, and no canary in messages or rendered diagnostics. Use a compact table/shared fixture, not duplicated matrices in every package. Focused check: go test ./internal/engine -run 'Directory|Glob|Filesystem|Resource'.

Stage 8 — Memoize Completed Native Link Dependencies

Finding: native part of C02. Owners: internal/engine/filesystem_links.go, filesystem_links_test.go, directory_links_test.go, and glob link tests.

Status: complete (native C02). Prior completed commits: 2736ddf, deb9aa5, ae3408a, ea27f17, 08a8c25, 877c497, 8aba43d. Validation passed: env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Link|Directories|Glob' -count=1, env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1, and git diff --check.

Extract the existing portable frame evaluator into one private resolution operation with active and completed maps. A small set of operation-local functions supplies Lstat, link reading, and validated target normalization. Keep this inside the engine; it is not a general filesystem/provider interface.
Wire native resolution first. Keep native prefix/root containment and target admission rules in its binding. Key maps by the complete normalized link path; the binding fixes the namespace for this one invocation.
An active back-edge remains a cycle. On a completed hit, restore the cached canonical target and process the caller's remaining segments. Record completion only when the link's own target frame finishes, not when first discovering it or after resolving the entire outer pathname. Cache successful targets only.

Acceptance: finite repeated links, nested/non-dot roots, self/mutual cycles, outside-root targets, and follow-disabled behavior. Extend real-link fixtures with a counting native wrapper for the A/B-doubling graph from C02. Use a generous linear operation ceiling on small graph sizes, with a bounded abort to keep the old failure safe; do not assert an exact call trace or elapsed time. Exercise both directory and glob selection through this binding. Focused check: go test ./internal/engine -run 'Link|Directories|Glob'.

Stage 9 — Use The Same Dependency Evaluation For Host Links

Finding: remaining host part of C02. Owners: internal/engine/filesystem_links.go and the same link/discovery tests.

Status: complete (C02). Prior completed commits: 2736ddf, deb9aa5, ae3408a, ea27f17, 08a8c25, 877c497, 8aba43d, 7bf2f0c. Validation passed: env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Link|Directory|Glob|Filesystem|Discovery' -count=1, env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1, and git diff --check.

Bind rooted host Lstat/Readlink operations to Stage 8's evaluator and retire the duplicate host frame loop. Retain portable component evaluation, rooted access, host-relative targets, and contained absolute targets translated with filepath. Preserve the final-entry follow policy and handling of links in parent components. Keep the completed map invocation-local; do not attach a cross-call cache to filesystemAccess or change behavior for concurrently changing filesystems.

Acceptance: wrap real host-root operation functions in the private binding to count bounded work for the same finite graph; this needs no production observer hook. Verify contained absolute links, escaping targets, parent links, true cycles, finite aliases, directory/glob integration, and independent namespaces. Run all native cases again after the shared evaluator is fully wired. Remove stale comments describing the old duplicated implementation. Focused check: go test ./internal/engine -run 'Link|Directory|Glob|Filesystem|Discovery'.

Stage 10 — Remove The Two Confirmed Unused History Copies

Findings: D02, D03. Owners: internal/engine/override_decode.go, override_context.go, load.go, load_result.go, and corresponding tests.

Status: complete (D02, D03). Prior completed commits: 2736ddf, deb9aa5, ae3408a, ea27f17, 08a8c25, 877c497, 8aba43d, 7bf2f0c, 0f9ecf1. Validation passed: env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Override|History|Explain|Secret|Independent' -count=1, env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1, and git diff --check.

Remove decodedOverrideResult.preCLI, its clone, and assertions only about that unused output. Keep the pre-CLI context actually built from before.final. Remove Inspection.structuralInput, the final publication clone made solely for it, and that clone's immutable-lookup inclusion. Preserve earlier structural-input uses for metadata and sensitivity; preserve Inspection.resolution and all active replay trees. Do not combine indexes, share mutable paths, or optimize other copies.

Acceptance: CLI, inherited/removed paths, default reconciliation, historical sensitivity, and independent-load/query ownership remain correct. Use existing tests rather than adding tests of the absence of private fields. D01 is not fixed by these deletions; its remaining root set is addressed next. Focused check: go test ./internal/engine -run 'Override|History|Explain|Secret|Independent'.

Stage 11 — Remove Dead Combination Evidence And Orphan Helpers

Findings: D04, F05. Owners: internal/engine/sequence_combine.go, inheritance*.go, compose.go, history.go, secret.go, destination.go, and associated tests.

Status: complete (D04, F05). Prior completed commits: 2736ddf, deb9aa5, ae3408a, ea27f17, 08a8c25, 877c497, 8aba43d, 7bf2f0c, 0f9ecf1, 119b42c. Validation passed: env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Sequence|Collection|Inheritance|History|Secret|Destination' -count=1, env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1, and git diff --check.

Remove unread discardedLower and inheritanceReplacement.path, previous, and effective, plus calculations solely producing them. Keep replacement before and after trees for accounting, projections, active origins/history, and append's use of sequenceIndexes. Keep overlay's uses of displacedOrigins.

Reconfirm and remove the six uncalled definitions listed in audit F05: mergeDisjointMappings, mergeHistoryDocument, inheritanceDirectiveLocation, addSecretEvidencePath, PlacementModeFromFacade, and PlacementModeForFacade. Preserve their active counterparts and implicit/interface invocation elsewhere. Update only private assertions about the removed evidence and stale comments/imports.

Acceptance: sequence replace/append/upsert, inheritance history, sensitivity, and retained replacement accounting still work. Inspect all remaining references; absence of graph callers alone is not the deletion criterion. Focused check: go test ./internal/engine -run 'Sequence|Collection|Inheritance|History|Secret|Destination'.

Stage 12 — Prepare The Actual Inspection Document State Once

Finding: preparation portion of D01. Owners: internal/engine/load.go, load_result.go, history.go, inheritance_load.go.

Status: complete (preparation portion of D01). Prior completed commits: 2736ddf, deb9aa5, ae3408a, ea27f17, 08a8c25, 877c497, 8aba43d, 7bf2f0c, 0f9ecf1, 119b42c, 247a12b. Validation passed: env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Load|History|Explain|Inheritance|Validation|Secret' -count=1, env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1, and git diff --check.

Extract preparation of the existing Inspection document/history fields from publication. Produce the actual owned pre-CLI copy, inherited-operation copies, application lists, raw resolution snapshots, and coalesced replay that the result will retain. Reuse the existing Inspection owner, not another universal DTO.
Prepare this state at the current history-check boundary: after resolution and before secret environment lookup and final validation. Metadata's separately cloned inheritance evidence is part of this state too. Keep required copies; this stage moves their ownership, not their semantics.
Pass that prepared object onward privately. Publication fills metadata paths, sensitivity, source evidence, contexts, diagnostics, and validated configuration using the same document objects; it must not create fresh document/history copies after the final quota check. Build the immutable lookup after admission.
Keep application callback/predicate order unchanged. In particular, do not move secret classification/predicate calls ahead of validation just to prepare the document roots. Earlier structural trees may remain temporary inputs to finalization without becoming published roots.

Acceptance: existing lifecycle and explanation behavior stays intact. Document the concrete retained document fields for Stage 13. This preparatory stage alone does not close D01; avoid temporary public state or compatibility APIs if the handoff is not yet buildable.

Prepared retained roots for Stage 13 admission are Inspection.baseline, input, final, preCLI, source/CLI application values, raw and coalesced resolution before/after values, and inspection inheritance parent/effective values. Metadata.inheritance holds its separate cloned parent/effective values; the immutable document lookup is built only during publication. Focused check when buildable: go test ./internal/engine -run 'Load|History|Explain|Inheritance|Validation|Secret'.

Stage 13 — Enforce The Quota On The Prepared Published Graph

Finding: completes D01. Owners: internal/engine/history.go, load.go, load_result.go, history/lifecycle tests.

Status: complete (D01). Prior completed commits: 2736ddf, deb9aa5, ae3408a, ea27f17, 08a8c25, 877c497, 8aba43d, 7bf2f0c, 0f9ecf1, 119b42c, 247a12b, 4fcff5f. Validation passed: env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'History|Explain|Inheritance|Override|Resolution|Load' -count=1, env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1, and git diff --check.

Use one explicit root visitor on the prepared inspection, feeding a pointer-aware retainedNodeCounter fresh for this publication, not the earlier transient expansion counter. Include baseline, input, final, pre-CLI, source/CLI application values, every raw and coalesced resolution before/after tree, inspection inheritance parents/effective trees, and metadata inheritance parents/effective trees. Count each distinct node once; different copies count separately. Lookup keys may reference only trees covered by that retained set. Keep the admission counter temporary; do not publish its visited-node map.
Replace the mismatched final checkpoint with this admission before secrets and validation. Keep existing early inheritance guards; do not cumulatively charge unrelated transient graphs against the final published-root check. Share that publication counter with preparation to charge actual retained copies as they become owned, stopping before further copies after overflow. Do not charge an unretained source merely because it is about to be cloned. Use source-size prechecks where straightforward, without a new budget-aware generic clone framework or universal allocation guarantee.
Keep production historyNodeLimit unchanged. Pass a private history-limit argument from the two engine entry paths to executeLoadRequest so engine tests can exercise the real lifecycle with a small allowance. Do not add a public option, global mutable limit, or environment switch.
Overflow returns nil result and the existing safe CodeExplainLimitExceeded. Publication consumes the admitted prepared state with no unaccounted document copies. Reuse the accounting visitor rather than maintaining two root lists for separate ordinary/grouped implementations.

Acceptance: use the real execution/publication path for a simple load and a small layered CLI+inheritance+resolution case. Independently enumerate document fields from the resulting inspection in test code, rather than using the production collector as its own oracle. Exact count succeeds; one less fails before final validation/environment lookup. Verify shared-node count-once behavior and distinct copies. Keep required raw snapshot retention and detached inspection tests. Focused check: go test ./internal/engine -run 'History|Explain|Inheritance|Override|Resolution|Load'.

Stage 14 — Cache Completed Dynamic Eligibility Within An Operation

Finding: A05; requires Stage 2. Owners: internal/engine/value_operation.go, projection/clone tests.

Status: complete (A05). Prior completed commits: 2736ddf, deb9aa5, ae3408a, ea27f17, 08a8c25, 877c497, 8aba43d, 7bf2f0c, 0f9ecf1, 119b42c, 247a12b, 4fcff5f, 12d350a. Validation passed: env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Dynamic|Projection|Clone|Resolution' -count=1, env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1, and git diff --check.

Add an operation-local set of successfully approved concrete types/descriptors. After a completed descriptor build and a full successful secret-eligibility walk, record approval. Subsequent values of that type in the same valueOperation reuse it. Do not treat cycle-walker visits as approvals, retain partial results, cache path-bearing failures, or transfer this set across loads/operations.

Acceptance: repeated typed nil pointers to a wide supported type avoid repeated eligibility walks; true unsupported/secret-bearing types still fail, including recursive descriptor graphs. Verify failed construction remains recoverable and diagnostics identify the current path. Source/algorithm review must show O(S + N) eligibility work for N repetitions of one graph of size S; no timing threshold or new production instrumentation is necessary. Focused check: go test ./internal/engine -run 'Dynamic|Projection|Clone|Resolution'.

Stage 15 — Establish Public Historical-Path And Record Ownership Coverage

Findings: F09, replacement path coverage for F12, prerequisites for F04. Owners: root external-package facade tests and existing source-snapshot tests.

Status: complete (F09; replacement path coverage for F12; public snapshot ownership review for F04). Prior completed commits: 2736ddf, deb9aa5, ae3408a, ea27f17, 08a8c25, 877c497, 8aba43d, 7bf2f0c, 0f9ecf1, 119b42c, 247a12b, 4fcff5f, 12d350a, 08f2a8f. Validation passed: env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test . -run 'Historical|Metadata|Facade|Source|WrappedPaths' -count=1 and git diff --check. Existing public source-version and import-selection snapshot tests cover repeated-query detachment of location/version pointers and nested selections; no duplicate ownership matrix was added.

Add compact public tests using only root APIs. Cover effective and removed historical paths, sorted unique pointer spelling, zero metadata, escaped keys, and a numeric-key/sequence shape change. Feed returned paths into Lookup and History to verify contextual shape behavior. Replace an element of a returned slice and query again; do not expose private segment storage to test detachment.

Before deleting clone scaffolding, verify meaningful mutable returned source records through repeated public queries: location/version pointers and nested selection slices stay detached, with existing nil/empty distinctions preserved. Reuse existing cases where they already protect the risk rather than duplicating the engine matrix. Remove TestWrappedPathsRetainEngineSegmentIntent only after its useful public behavior is covered.

Acceptance: a broken root path conversion or shallow returned record would fail these tests even while engine tests pass. Engine path-intent tests remain. Focused check: go test . -run 'Historical|Metadata|Facade|Source|WrappedPaths'.

Stage 16 — Retire Disconnected Facade Clones And Redundant Scaffolding

Findings: F04, remaining F12; requires Stage 15. Owners: root records.go, records_test.go, facade_internal_test.go, facade_compatibility_test.go, facade_candidate_test.go, facade_equivalence_test.go.

Status: complete (F04; remaining F12). Prior completed commits: 2736ddf, deb9aa5, ae3408a, ea27f17, 08a8c25, 877c497, 8aba43d, 7bf2f0c, 0f9ecf1, 119b42c, 247a12b, 4fcff5f, 12d350a, 08f2a8f, 826062a. Validation passed: env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ., env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./examples/consumers, and git diff --check.

Remove the complete private clone family named in audit F04 after reconfirming that only the tests enter it. Preserve engine clones, active Path.clone, explicit root conversions, and nil/empty conversion checks. Remove the ineffective whole-Destination assignment assertion and private clone-only assertions, retaining conversion/public query ownership protection from Stage 15.

Remove the duplicate six-type ownership loop and trivial assignment equality assertion, keeping the comprehensive root type owner and compile-time interfaces. Route source-ownership tests through public Load and root error typing instead of source.toEngine() plus direct engine loads. Consolidate reader cases with the existing equivalence owner while preserving byte mutation and copied-reader cases.

Acceptance: public construction, errors, callback lifetime, source identity, disclosure, and returned ownership remain represented. Do not delete static public or dependency-boundary tests or infer that engine coverage replaces facade tests. Focused check: go test . and go test ./examples/consumers.

Stage 17 — Remove The Remaining Unread Private Members

Finding: completes F06, following its schema-path subset in Stage 1. Owners: the exact records/files listed in audit F06.

Status: complete (F06). Prior completed commits: 2736ddf, deb9aa5, ae3408a, ea27f17, 08a8c25, 877c497, 8aba43d, 7bf2f0c, 0f9ecf1, 119b42c, 247a12b, 4fcff5f, 12d350a, 08f2a8f, 826062a, e4a5ecb. Validation passed: env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine, env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ., and git diff --check.

Remove collectionPolicyResolution's four named unread fields; inputPresence.null; directive extraction's two presence flags; importSelection.outcome; importDeclarationSpec.excludeSet; selectedOverlayRequest.filesystem; sequenceDeclarationSpec.destinationLocation; source-version evidence's absence and checked and ancestor kind; and postDecodeOverrideContext.sources. Remove computations only feeding those slots and adjust their private assertions.

Keep declaration normalization, effective policy flags, candidate/admission rules, public empty-selection outcomes, actual version callbacks, ancestor sensitivity paths, public null metadata, location reporting, and the local source list used to construct resolution input. Check references/build constraints again; do not extend this deletion list to identity tokens, map-key fields, or interface methods.

Acceptance: use public behavior for tests that formerly checked inactive annotations. No active validation or distinction is removed with its redundant transport flag. Focused check: go test ./internal/engine and go test ..

Stage 18 — Retire The Parallel Import Directive Parser

Finding: F03. Owners: internal/engine/import_spec.go, import_spec_test.go, directive/selector tests.

Status: complete (F03). Prior completed commits: 2736ddf, deb9aa5, ae3408a, ea27f17, 08a8c25, 877c497, 8aba43d, 7bf2f0c, 0f9ecf1, 119b42c, 247a12b, 4fcff5f, 12d350a, 08f2a8f, 826062a, e4a5ecb, 3122e2f. Validation passed: env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Import|Directive|Directory|Glob|SourceVersion' -count=1, env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine, and git diff --check.

Change parseImportSpecsYAML to enter extractRootDirectives with the appropriate source context and return its imports/diagnostics. Keep parsing source fixtures through the real YAML boundary. Remove parseImportSpecificationDirective, its parallel mapping/field validation, and support used solely by it. Do not remove parseImportDeclarations or the imports-only context still used after layered outer-directive extraction.

Acceptance: malformed fields/types/selectors and exact diagnostic paths continue to be checked by the actual production parser; valid import fixtures retain their meaning. Inspect the assertions before accepting wording changes from replacing the obsolete parser. Keep classifications/sites, not incidental wording, as the contractual expectations. Focused check: go test ./internal/engine -run 'Import|Directive|Directory|Glob|SourceVersion'.

Stage 19 — Strengthen Ownership Fuzzing And Observable Import Ordering

Findings: F10, F11. Owners: internal/engine/composition_fuzz_test.go, import_selection_test.go.

Status: complete (F10, F11). Prior completed commits: 2736ddf, deb9aa5, ae3408a, ea27f17, 08a8c25, 877c497, 8aba43d, 7bf2f0c, 0f9ecf1, 119b42c, 247a12b, 4fcff5f, 12d350a, 08f2a8f, 826062a, e4a5ecb, 3122e2f, 6f6130f. Validation passed: env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'FuzzCompositionAndOverlay|SelectNamedImports' -count=1 and git diff --check. The bounded composition campaign remains scheduled for Stage 33.

In FuzzCompositionAndOverlay, mutate an existing retained descendant or in-length element before comparing both original inputs with their saved values. Handle empty containers without assuming append affects shared storage. Keep existing bounded inputs, immutability checks, and commutativity properties; do not add another fuzz target or ownership matrix.

Replace the exact two-stats-per-file assertion in TestSelectNamedImportsChecksNormalizedPathsLexically with ordered candidate paths and meaningful first-failure location. Consolidate with existing lexical-selection tests if this subsumes the standalone case. Keep non-opening of excluded contents and genuine first-error behavior where those are the tested requirements.

Acceptance: a shallow existing-descendant alias would fail the fuzz assertion, while a harmless change in stat implementation would not fail ordering coverage. Focused check: go test ./internal/engine -run 'FuzzCompositionAndOverlay|SelectNamedImports'. Run the bounded composition campaign in Stage 33 after the later assembly changes have landed, rather than treating an earlier run as evidence for those changes.

Stage 20 — Use Only Deferred Acquisition In Production And Fixtures

Finding: F01. Owners: internal/engine/acquisition.go, import_acquisition.go, imports_test.go, and callers identified by request-construction tracing.

Remove deferAssembly and the eager branch. Retain pending-item placement reservation and return acquired scopes/import graph/accounting consistently. Migrate test helpers needing final values to invoke the real composition boundary explicitly after acquisition. Keep acquisition-only tests on the acquired scope instead of silently composing it. Ordinary and grouped production requests use the same handoff without a replacement mode flag.

Acceptance: imports, item-local scopes, nested destinations, dependency ordering, version protections, optional absence, resource closure, and byte/node accounting still run through the active protocol. Keep tests' intended boundaries; do not delete meaningful composition cases when changing fixture setup. Focused check: go test ./internal/engine -run 'Import|Destination|SourceVersion|LoadGrouped|Underlay'.

Status: complete (F01). Prior completed commits: 2736ddf, deb9aa5, ae3408a, ea27f17, 08a8c25, 877c497, 8aba43d, 7bf2f0c, 0f9ecf1, 119b42c, 247a12b, 4fcff5f, 12d350a, 08f2a8f, 826062a, e4a5ecb, 3122e2f, 6f6130f, and 4fd3bee. Validation passed: env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Import|Destination|SourceVersion|LoadGrouped|Underlay' -count=1, env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1, env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -count=1, and git diff --check.

Stage 21 — Isolate Parsed-Input Directive And Version Processing

Finding: first part of F02; requires Stage 20. Owners: internal/engine/acquisition.go, existing directive/source-version records.

Extract the directive/version-processing block from acquireInput into a cohesive operation returning the existing sourceVersionProcessingResult and diagnostics. Pass the parsed source, existing inputAcquisitionRequest, and load-local version state explicitly; read role, destination/placement, context, and protection from that request rather than introducing a long flattened parameter list. Compute source ID once. Keep updating mainProcessing with a clearly owned completed result, not pointers into incidental locals scattered among branches.

Retain distinct source and layered processing: ordinary sources process their root directives with their source policy; layered loading extracts its outer directives before enabled version handling and restores those declarations to the processing result. Default/enabled mode timing and warning accumulation must not become one inferred common policy. Keep the imports-only processing context where it remains necessary. Do not add a new context hierarchy or public request.

Acceptance: source-version tests for main, imports, underlays, mounted/item destinations, selected overlays, and NoSource preserve callback counts, order, warnings, safe diagnostics, and retained declaration locations. Focused check: go test ./internal/engine -run 'SourceVersion|Version|Directive|Destination'.

Stage 22 — Isolate Standalone Placement And Acquisition Completion

Finding: second part of F02. Owners: internal/engine/acquisition.go, load_input.go, destination/context helpers.

Extract standalone contribution placement/registration and passive-context completion into explicit operations on the existing assembled-input/acquisition owners. Keep discovery selection and parsing visible in acquireInput, then make its imported-versus-standalone dispatch and completion readable. Each helper returns its result/diagnostic; avoid closures mutating a collection of surrounding locals. Keep live handles confined to their acquisition owner.

Preserve local versus placed documents, unresolved item destinations, wrapper-node reservation, source identity, NoSource/discovery behavior, and the ordinary overlay case that defers filesystem completion. Review aliasing of source slices and version evidence when assembling the return value.

Acceptance: existing discovery, standalone mapping/reader/native/host input, destination accounting, item placement, and passive resource-context tests still exercise the same observable behavior. Do not require exact private helper calls. Focused check: go test ./internal/engine -run 'Discovery|Destination|Source|Resource|Filesystem'.

Stage 23 — Clarify Ordinary Context And Structural-Expansion Handoffs

Finding: third part of F02. Owners: internal/engine/load.go, load_input.go, inheritance/version helpers.

Extract the substantial ordinary-load filesystem/resource-context fallback into an operation that returns the completed assembledInput plus diagnostics. Keep existing resolved filesystem facts when available rather than reopening handles to recover the same information; retain required validation when facts are absent. Isolate structural inheritance expansion/version-path projection as one further handoff using the existing composed/assembled records.

Keep the coordinator's visible order: schema/source-version timing, acquisition, ordinary/grouped assembly and selected overlay, structural expansion, decoding, CLI, resolution, Stage 13 history admission, secrets, identity validation, final validation, and publication. Do not move callback execution into an extraction that changes when errors suppress later phases.

Acceptance: ordinary imports and non-import overlays, absent main input with explicit context, resource paths, inheritance version protection, callback order, and fail-before-later-callback cases remain intact. Re-run panic cleanup and retained-history integration cases after moving the handoffs. Focused check: go test ./internal/engine -run 'Load|Overlay|Resource|Inheritance|Version|History'.

Finding: completes F02. Owners: internal/engine/load_request.go, load_input.go, tier projection helpers.

Extract the repeated completed-tier application operation: collect that tier's declarations, initialize or apply through the assembly owner, then project only the incoming sources/version evidence using the returned sequence operations. Use explicit source/evidence ranges from the caller; keep main-version projection and its protection rules recognizable rather than hiding them in a generic loop.

Keep acquisition order separate from precedence application, distinct tier-local and load-wide accounting, absence of empty groups, main-only default version checking, and selected-overlay rules. This helper must accept the existing policy scope contract so Stages 27–28 can later change that private contract once, at named composition/application boundaries.

Acceptance: grouped source ordering, underlay precedence, strict-within-tier conflicts, mounted source/version coordinates, ordinary/grouped differences, metadata/history/diff, and warnings remain stable. Update the implementation-flow description only after the handoffs actually land, without restating YAML policy. Focused check: go test ./internal/engine -run 'Grouped|Layered|Underlay|SourceVersion|Sequence|Diff'.

Stage 25 — Separate Collection Preparation, Application, And Finalization

Finding: preparation/ownership part of E03. Owners: internal/engine/overlay.go, compose.go, collection_policy*.go.

Refactor mergeOverlayMappingsWithCollectionPolicy into three named responsibilities:

Prepare incoming claims/declaration groups and per-destination resolutions against read-only lower state, with no value/origin mutation.
Apply recursive value combination through one small overlay operation owner holding the existing result/index plus its accumulated diagnostics, sequence history, and destinations receiving values. Use its named callback methods rather than many callbacks capturing separate mutable maps/slices. Put the root-origin merge in this application phase as well.
Finalize declaration-only policy against the completed value and emit sorted sequence evidence. Build any retained-value lookup after mutation, not before.

Keep these as engine-private operations, not generic strategy interfaces. Preserve current data semantics here; Stages 26–28 replace the full-state work. A temporary large clone retained by this extraction is not completion of E01/E02.

Acceptance: policy declarations/conflicts, pending keys, mode-only changes, replace/append/upsert, nested record ownership, origin/history ordering, and failed combination diagnostics remain protected. Newly separated inputs/results make lower versus incoming versus retained ownership explicit. Focused check: go test ./internal/engine -run 'Collection|Sequence|Overlay|Inheritance|History'.

Stage 26 — Index Assembly Policy Retirement By Semantic Path

Finding: E01. Owners: internal/engine/collection_policy_state.go, compose.go, overlay.go, and a small private policy-index file if that clarifies ownership.

Add a mutable semantic-segment retirement trie alongside the assembly's policy table. Index destinations with state cleared by clearCollectionIdentity; mode-only entries stay in the table for exact lookup but need no retirement terminal. Use pathSegment keys so numeric mapping keys and sequence indices stay distinct. Build once when the assembly adopts its initial owned scope, not for every replacement or tier.
Route assembly policy writes/retirement through its owner so table and index remain synchronized. Keep immutable/local scopes outside this owner independent; do not add index rebuilding to their every lookup or snapshot.
Descendant retirement walks only the indexed subtree for the replaced prefix. Clear old YAML identity and record-owned state, preserving explicit modes and application constraints. Retire before installing policy introduced by the incoming replacement; declarations established afterward must survive. Remove cleared terminals and prune empty branches. Replacing an ancestor again must not rescan already cleared mode-only policies; later writes reindex new state.
Reuse the semantic-index conventions of inheritancePolicyIndex, but keep this mutable assembly owner distinct from that borrowed record-local index. Do not change the pointer-spelling index used for history/dependencies.

Acceptance: retain collection_retirement_test.go cases for all replacement kinds, ancestor removal with new pending identity, explicit modes, and registrations. Add escaped/numeric semantic-path cases and a bounded disjoint P-policy/Q-scalar case, plus repeated retirement above retained mode-only entries. Review the complete caller path: one index construction plus prefix/affected-subtree work must replace Q scans of all P entries. No exact syscall/timing test or public complexity guarantee is needed. Focused check: go test ./internal/engine -run 'Collection|Retirement|Sequence|Inheritance'.

Stage 27 — Borrow Lower Policy And Produce Sparse Strict-Tier State

Finding: first part of E02; coordinates E03. Owners: internal/engine/collection_retirement.go, compose.go, import_scope.go, and nested/acquired scope composition callers.

Replace the full lower-scope copy used for strict resolution with a small borrowed read view: the existing lower table plus current replacement masks. Exact lookup returns an immutable policy view; if old identity is masked at that destination, clear it in the returned header, not in the borrowed table. Keep explicit modes. Reuse collectionReplacementPaths' inclusive/exclusive prefix semantics; do not accumulate a chain of views across tiers.
Compute replacement eligibility from incoming claims/declarations and exact lower/application lookups. Do not enumerate the full lower table just to clear entries in a transient copy. Clone an affected policy only when constructing an independently owned result or changing record-owned state.
resolveStrictCollectionPolicies produces identity information and an owned sparse policy scope for destinations touched by incoming values/declarations. That scope is not another complete copy of accumulated lower policy. Preserve record ownership when a declaration has no new value.
Thread this contract through acquired/item-local strict composition. Keep complete independently owned snapshots where local record scopes/history need them; sparse incoming state must not accidentally borrow mutable record maps. Update private tests that assumed an incoming tier included unrelated lower entries to check the complete assembled result instead.

Acceptance: strict duplicate rejection still sees applicable lower identity; higher declarations cannot repair invalid lower composition. Pending declarations, replacement masks, mode persistence, and nested item-local scopes remain correct. Inspect the scalar-only-tier path: work depends on incoming claims/declarations, not P unrelated retained policies. Wiring to assembly is finished in Stage 28; do not retain an eager full-scope adapter merely to make this stage compile. Focused check when buildable: go test ./internal/engine -run 'Compose|Collection|Item|Underlay|Sequence'.

Stage 28 — Apply Sparse Incoming Policy Without Full-Table Preparation

Finding: completes E02; uses Stage 26 and Stage 27. Owners: internal/engine/overlay.go, load_request.go, import_scope.go, inheritance_prepare.go, and collection retirement callers.

In overlay preparation, discover relevant destinations by walking incoming claims and declaration paths, checking the lower table at those exact paths. Replace the loop over every retained policy and the whole-overlay lookup built solely to support that loop. Include non-sequence claims at exact retained policy destinations so invalid append/upsert inputs still fail as before.
Use the borrowed lower view for operation resolution; retain only incoming owned resolutions and sparse policy changes. Apply those changes at the named assembly owner using Stage 26's synchronized updates. Unchanged lower entries remain in that owner, not copied into every incoming tier.
Retirement caused by replaced ancestors uses the live index at the existing mutation boundary. Reconcile declaration-only updates after value application. Preserve current-tier policy established after retirement and validate a retained sequence when a declaration alone requires it.
Update tier application and any inheritance callers of the old lower-policy preparation helper. For inheritance, retain independent parent/child local scopes where ownership requires them; share the replacement-mask/read-view logic rather than copying the load-wide policy store into a tier.

Acceptance: with P retained policies and T disjoint scalar-only tiers, no path may clone/scan all P policies per tier or rebuild their index. Inspect all preparation, strict composition, application, and finalization callers for hidden full-table fallbacks. Add a small multi-tier case preserving untouched rules and nested record owners, plus retained-key-only/mode-only and ancestor-replacement cases. Required value/history copies and affected record payload remain legitimate costs. Focused check: go test ./internal/engine -run 'Collection|Retirement|Sequence|Underlay|Layered|Inheritance|History'.

Stage 29 — Consolidate Sequence-History Translation And Finalize Policy Owners

Finding: completes E03. Owners: internal/engine/overlay.go, inheritance_combine.go, inheritance_collection.go, and sequence-history helpers.

Extract one pure sequence-history descriptor constructor taking destination, operation, incoming value, and sequenceCombinationResult. It owns replace-versus- mapped translation and detached projection/replacement storage. Use it in both overlay and inheritance instead of independently rebuilding that descriptor.

Keep overlay and inheritance record-policy rebuilding as separate named operations: overlay retains incoming/lower record ownership, while inheritance uses current versus inherited scope conversions with different mode lifetimes. Share existing projection indexes but do not add a callback/strategy framework or common mutable origin owner to unify those distinct operations. This resolves the roadmap's record-selection-sharing evaluation in favor of explicit ownership-specific loops.

Acceptance: value combination still has one combineSequence owner; preparation, application, and declaration-only finalization have clear inputs/results without reintroducing full-table work. Verify sequence history and sparse nested projection, record replacement/removal, modes, origin locations, and sensitivity for both precedence and inheritance. Focused check: go test ./internal/engine -run 'Sequence|Collection|Inheritance|History|Explain|Sensitivity'.

Stage 30 — Index Original Tier-History Values Lazily

Finding: E04. Owners: internal/engine/sequence_history_projection.go, projection/history tests.

In projectTierHistoryApplications, lazily construct one documentLookup for an original application value only when an applicable mapped sequence needs its container. Reuse it for that application's sequence contributions through documentAtPathWithLookup. Keep source values immutable and authored coordinates separate from assigned output positions. Rebuild projected sidecar indexes after edits as before; do not retain the temporary lookup or cache it across loads.

Acceptance: wildcard-selected sibling collections, sparse/noncontiguous mappings, nested sequence projections, and upsert replacement evidence remain correct and detached. Source review shows O(V + S × D + I) lookup/projection work for one application, excluding existing sorting/copies, rather than scanning a V-wide mapping for each of S contributions. Use existing wide/sparse fixtures, not timing thresholds or another explanation-replay implementation. Focused check: go test ./internal/engine -run 'Tier|Projection|Sequence|History'.

Stage 31 — Index Compound Identity Restoration

Finding: E05. Owners: internal/engine/inheritance_scope.go and identified-inheritance tests.

Build an identity-field membership set and a mapping-position index for the merged record. Walk child entries once; for identity members, replace the corresponding merged entry with a detached copy preserving the child's key/value locations. Keep merged mapping order and exact identity comparison/parent selection. Do not remove the restoration step or reinterpret upsert as recursive body merging.

Acceptance: compound identity, many nonidentity siblings, escaped/literal keys, child locations, mapping order, missing-key validation, and detached retained templates remain correct. Source review establishes O(K + C + M) lookup work plus required copied values for K identity fields and C/M mapping entries. Focused check: go test ./internal/engine -run 'Identified|Identity|Inheritance'.

Stage 32 — Index Secret Failure Locations Only When Needed

Finding: F08. Owners: internal/engine/secret_reference.go, validation.go lookup helper, secret_reference_test.go.

Let the existing per-resolution secret operation own a nil documentLookup until its first diagnostic needs attribution. On that failure, build it from the fixed reference-input tree and reuse nearestDocumentLocationWithLookup for subsequent failures. Thread that operation-local state through the existing walk/diagnostic helper rather than rebuilding an index per reference or introducing global state.

Acceptance: many missing/empty/invalid references in a wide mapping retain every safe diagnostic and exact/ancestor fallback location; repeated environment names still use the existing lookup cache. Include escaped/numeric and defaults-only paths. The successful path builds no failure-only document index. Inspect for O(V + F × D) location work, apart from sorting, rather than repeated full scans; keep the environment cache and credential bytes out of diagnostics/history. Focused check: go test ./internal/engine -run 'Secret|Location|Validation'.

Stage 33 — Integrate, Document, And Verify The Complete Sprint

Findings: final acceptance for all selected IDs.

Review the traceability table below against actual changes and tests. Finish every check deferred by an intermediate unbuildable stage. Reconfirm no selected work was silently replaced by a smaller cleanup or deferred profiling.
Review boundary interactions: transactional descriptors plus eligibility caching; slice views plus detachment; callback panic plus root closure; both directory-reading implementations and link bindings; complete published-history admission after copy cleanup; policy deltas/retirement plus tier history, inheritance, source-version projection, and secret locations.
Reuse representative public/consumer cases to verify agreement among final values, presence/null, origins, historical queries, explanation, redaction, diff, safe errors, and independent loads. Add only a missing integration case with a concrete risk; do not duplicate every engine test at the root.
Update docs/internal/overview.md and docs/internal/imports.md for the actual resulting responsibilities. Correct stale implementation-stage comments in changed owners. Update canonical declaration/configuration wording only where needed to accurately describe a repaired existing behavior; do not recast private algorithms or tests as new public promises. No new ADR/policy is needed.
Perform the checks below, inspect the diff for unintended public/API, dependency, limit, or scope changes, and record completion/validation per stage. Keep the selected planning records until normal release preparation retires them.

Final checks, from the repository root:

env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off ./scripts/check.sh
env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test -race ./...
env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -shuffle=on -count=3
env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off \
  go test ./internal/engine -run '^$' -fuzz '^FuzzCompositionAndOverlay$' -fuzztime=30s -parallel=2
env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./examples/consumers -count=1
env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off \
  go test . -run 'TestLocalDocumentationLinks|TestDependencyImportBoundary|TestPublicFacadeBoundary' -count=1
git diff --check

The consumer/boundary reruns are focused confirmation of the changed cross-package surface, not new release prerequisites. Reuse unchanged successful checks when appropriate under the testing policy. Record actual platform skips and limitations; no express macOS validation, CI wait, or release publication is part of completion. All Go and documentation changes must be committed in focused nonempty units, with a clean workspace and a concise summary of fixed behavior and any remaining out-of-scope work. Do not create empty commits for validation-only stages.

Finding-To-Stage Traceability

Finding	Owning stages
A01	1
A02	2
A03	3
A04	4
A05	14
B01	5
B02	6
C01	7
C02	8–9
D01	12–13
D02	10
D03	10
D04	11
E01	26
E02	27–28
E03	25, 29, coordinated with 26–28
E04	30
E05	31
F01	20
F02	21–24
F03	18
F04	16, with replacement coverage in 15
F05	11
F06	1, 17
F08	32
F09	15
F10	19
F11	19
F12	15–16

Stage 33 verifies all 29 entries. C03 and F07 remain deferred, not missing from this table. No user/product question remains open; the implementation choices necessary for this scope are selected above. Helper names/file splits not fixed by an existing declaration can follow local idiom without changing those choices.

62 KiB Raw Blame History Unescape Escape