From b1d89810ca835f46399526515684c5076253cb45 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 26 Oct 2026 00:03:02 +0000 Subject: [PATCH] Plan discovery and facade boundary refactoring --- docs/roadmap/implementation.md | 1408 +++++---------------------- docs/roadmap/internal-boundaries.md | 27 +- 2 files changed, 277 insertions(+), 1158 deletions(-) diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index e3a3d7d..dd8778e 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,1163 +1,287 @@ -# Audit Remediation Implementation Plan +# Import Discovery And Facade Simplification Implementation Plan -Status: complete; all work and final integration are recorded below. -Prepared: 2026-10-25. -Planning baseline: `4a5316b7fc2da57dcced33190cb6499c9a1beb5c`. +Status: not started. +Baseline: `bb5540b62e382106d59678e2edb7655acfd5c960`. Audience: a Luna coding agent implementing the stages sequentially. ## Scope And Execution -Implement the 29 selected entries in the [roadmap](roadmap.md). The -[audit](audit.md#settled-scope-decisions) owns the evidence, accepted scope, and -detailed acceptance cases. This plan owns implementation order, private design -choices, validation, and execution progress. Preserve the finding IDs when -recording completion. The production baseline is unchanged by the intervening -audit and planning commits; nevertheless, inspect actual callers before deleting -code. Follow `AGENTS.md`, [agent policy](../policy/agents.md), and the -[development guide](../development.md). +Implement **C03** and **F07** from the +[internal-boundary roadmap](internal-boundaries.md). That document owns the target +state and preservation requirements; this plan owns concrete private choices, +steps, validation, and progress. The preceding 29-finding remediation sprint is +complete. Its implementation record remains at +`bb5540b:docs/roadmap/implementation.md`; this plan does not reopen it. -This is one end-to-end sprint. Intermediate stages may temporarily fail to -compile or have pending tests. Do not create compatibility shims, duplicate -implementations, or extra public interfaces just to make each stage independently -green. Run the focused checks when their affected code is buildable; otherwise -record exactly what is pending and run it once the consuming stage lands. Never -describe an unfinished stage or deferred check as complete. Commit focused, -nonempty units under repository policy, with plain-English messages and accurate -validation notes. Final completion requires the integrated end state and checks, -not just completion markers. +Read `AGENTS.md`, [agent policy](../policy/agents.md), the [development +guide](../development.md), and applicable architecture, testing, and documentation +policies. Use graph-first discovery and verify actual callers before deleting +helpers. Preserve unrelated work. -For each stage, record its status, affected finding IDs, and actual validation -here. Include already known commit identifiers in later stage updates; report the -final commit in the handoff rather than making a commit self-referential. Keep -execution status in this plan rather than maintaining -another progress inventory in `audit.md` or `roadmap.md`. A finding split across -stages closes only after all of its stages and acceptance cases are satisfied. +This is one end-to-end sprint. Intermediate stages may fail to compile, especially +the coordinated engine/facade cutover in Stages 2–3. Do not create adapters, +aliases, duplicate paths, or extra abstractions solely to make intermediate +stages green. Run focused checks once their affected surface is buildable; record +pending checks and finish them before final acceptance. Commit focused nonempty +units under repository policy and record stage status, actual validation, and +known commit identifiers here. Completion requires the integrated result, not +just checked-off stages. ### Boundaries -- Preserve supported public types, signatures, root ownership, configuration - syntax, tier semantics, collection operations, source/version conventions, - callback ordering, diagnostics, confidentiality, and detached ownership. -- Repair the nine reported defects and their regressions. Safe rejection of - invalid values, acceptance of supported finite values, and enforcement of the - existing quotas are remediation, not new restrictions. -- Keep panic propagation, borrowed-resource ownership, regular-file close policy, - partial bootstrap/writer behavior, and existing optional-absence rules. -- Keep raw recorded resolution snapshots, active history, and required copies. - Remove only the proven unused copies and evidence named below. -- C03 glob readability extraction and F07 wider facade transport simplification - remain deferred in [future work](future.md). Do not redesign the clone visitor, - explanation replay, public metadata handoff, or cross-load schema caching. -- Add no dependencies, public options, global caches, traversal caps, background - activity, release gates, or performance guarantees. Do not publish a release, - update consumer dependencies, or edit accepted ADR decisions during this sprint. +- Preserve public root-owned declarations and signatures, YAML syntax, loading + semantics, ordinary/layered differences, callback order, diagnostics, source + identity, inspection, confidentiality, and ownership. +- Preserve glob literal-versus-discovered rules and directory/glob differences. + Keep completed-link reuse, directory failure phases, containment, and existing + accounting. This is not a discovery algorithm or permission-policy change. +- Keep `resolvedImportPolicy`, registration compilation, normalized request + responsibilities, and conversions with real ownership or domain meaning. + Simplification is limited to defaults transport and placement construction. +- Add no dependencies, caller controls, caches, limits, background work, release + gates, or performance promises. Do not publish a release, update consumers, or + revise accepted ADR decisions. + +### Selected Ownership Model + +Use the existing engine `ImportDefaults` name and its exported `Required`, +`Follow`, `Hidden`, `Recursive`, and `Pattern` fields as the sole engine +input-defaults type. Root `ImportDefaults` stays separately defined. Root +conversion maps the fields without cloning them; the public loading wrappers +immediately enter engine preflight. `loadSettings` takes the per-load snapshot +before acquisition or callbacks, as it already does. The normalized request and +acquisition policy then borrow that load-owned read-only value. + +Declaration resolution reads the owned defaults and retains its existing copies +of selected pattern/exclusion slices and destinations in `resolvedImportPolicy`. +This separates presence-aware load input from independently owned effective +declaration policy. No public copy-count or thread-safety guarantee follows. + +For placement, expose the engine's existing enum as `PlacementMode uint8` with +`PlacementModeMapping` and `PlacementModeSequenceItem`, and expose placement +fields on engine input/options records for direct construction. These remain +internal-package implementation surfaces. Root types stay root-owned; direct +casts preserve unknown values for existing preflight validation. ### Validation Conventions -Commands below run from the repository root with the installed compatible Go -toolchain and already available dependencies. Use `GOTOOLCHAIN=local`, -`GOPROXY=off`, and `GOTELEMETRY=off` for offline checks. Keep disposable diagnostics, -coverage files, and binaries outside the repository. Do not change `go.mod` or -`go.sum` to obtain audit tooling. - -Focused `-run` expressions identify behavior owners, not required names for new -tests. Include the new regression explicitly in the focused invocation if its -name falls outside the expression. Extend existing test owners; do not duplicate -the internal matrix through every facade. Tests for termination use a bounded -child process/watchdog where a regression could crash or hang the test process; -that watchdog is not a performance threshold. Reuse the subprocess pattern in -`internal/engine/source_fifo_test.go` where appropriate. - -Use small controlled budgets and bounded filesystem operation observation for -resource/scaling cases. No benchmark campaign, exact syscall choreography, -coverage percentage, or timing ratio is an acceptance criterion. Final development, -race, consumer, and documentation validation is specified in Stage 33. - -## Stage 1 — Terminate Pointer-Only Descriptor Traversal - -Finding: **A01**; also the `schemaPath` portion of **F06**. -Owners: `internal/engine/schema.go`, `decode.go`, `secret.go`, and their tests. - -Status: **complete** (A01; schema-path portion of F06). Validation passed: -`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Schema|Decode|Resolution|Secret|PointerOnly' -count=1`, the specified focused check, and `GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`. - -1. Introduce one private pointer-descriptor terminal lookup that detects a - repeated descriptor, using a traversal-local visited set. Return failure for - a pointer-only loop; do not reject that type during schema construction. -2. Use it where decoding must unwrap a non-null pointer chain, in - `acceptsMapping`, and before schema/secret path traversal consumes the next - segment. A non-null value for an uninhabitable pointer-only chain produces - the existing `CodeDecodeType` classification at the input path/location. - Keep null handling before this check so supported nil pointers still decode. - For decoding a finite chain, peel the base/descriptors iteratively, decode - the terminal value once, and rebuild pointer wrappers in reverse. Preserve - existing typed-default behavior; do not rescan the remaining chain at every - recursive pointer call and introduce quadratic chain traversal. -3. Remove unread `schemaPath.field` and `schemaPath.dynamic` and the terminal - unwrapping used only to compute them. A path ending at a nullable pointer - field may still resolve to its descriptor; reject attempts to descend through - its pointer-only loop. Update consumers/tests accordingly. -4. Review other pointer-unwrapping helpers. Apply the shared check to analogous - descriptor-only loops; retain finite value traversal and existing inline-field - restrictions rather than adding a blanket recursive-type rejection. - -Acceptance: reproduce `type Loop *Loop` non-null decoding and helper-path -termination in bounded subprocesses; verify null/omitted values, direct nullable -field operations, and normal recursive structs with acyclic values. Check safe -codes/paths rather than raw values or exact new wording. -Focused check: `go test ./internal/engine -run 'Schema|Decode|Resolution|Secret'`. - -## Stage 2 — Roll Back Failed Dynamic Descriptor Builds - -Finding: **A02**. -Owners: `internal/engine/schema.go`, `value_operation.go`, projection/resolution tests. - -Status: **complete** (A02). Previous completed unit: `2736ddf`. Validation -passed: `GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Dynamic|Projection|Clone|Resolution' -count=1` and `GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`. - -1. Give `schemaBuilder` a journal of types newly inserted during a build. - Record insertion only after a cache miss; cached, completed descriptors are - borrowed and not mutated. Recursive new descriptors may refer to one another - through their provisional entries. -2. In `dynamicDescriptor`, treat construction plus dynamic-secret eligibility - as one operation. On failure, delete every journaled cache entry, including - ancestor/container placeholders, before returning the existing path-specific - projection issue. Deleting an entry already removed by a local failure is safe. -3. On success, retain the completed additions in the existing load-local cache. - Do not clone the entire cache on each request, cache failures, or clear valid - preexisting entries. Initial schema construction can use the same journaling - mechanics without introducing another persistent cache. - -Acceptance: the two `Normalize` calls from A02—typed nil then non-nil pointer to -an unsupported dynamic struct—report safe failures without a panic. Cover nested -and mutually recursive provisional entries, subsequent valid operations, prior -valid cache entries, and repeated failures at different paths in one load. -Focused check: `go test ./internal/engine -run 'Dynamic|Projection|Clone|Resolution'`. - -## Stage 3 — Distinguish Finite Slice Views From Cycles - -Finding: **A03**. -Owners: `internal/engine/value_operation.go`, `value.go`, `clone.go`, and tests. - -Status: **complete** (A03). Prior stage commits: `2736ddf`, `deb9aa5`. -Validation passed: `GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Mapping|Value|Projection|Clone|Override|Cycle|FiniteSlice|SliceCycles' -count=1` and `GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`. - -Add visible slice length to `valueVisit` identity in `visitFor`. Keep type, kind, -and starting pointer, existing empty/nil handling, and active-path removal on -unwinding. Capacity is not part of the visible traversed graph. Maps and pointers -retain their current identity rules. Follow all shared callers rather than -patching only one projection entry point. - -Acceptance: `shared := []any{"leaf", nil}; shared[1] = shared[:1]` succeeds through -mapping input, typed defaults/projection, cloning, and CLI override construction. -Equivalent detached values agree. Actual self/mutual cycles still fail safely; -mutation of an existing returned descendant must not affect the original. -Focused check: `go test ./internal/engine -run 'Mapping|Value|Projection|Clone|Override|Cycle'`. - -## Stage 4 — Check Container Lower Bounds Before Bulk Allocation - -Finding: **A04**. -Owners: `internal/engine/projection.go`, `value_operation.go`, `projection_test.go`. - -Status: **complete** (A04). Prior stage commits: `2736ddf`, `deb9aa5`, -`ae3408a`. Validation passed: `GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Projection|Inline|Resolution' -count=1` and `GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`. - -1. After charging a container node and before its bulk reservation, compare its - minimum remaining cost with the current node allowance: at least one node per - sequence element and one key plus one value node per map entry. Use comparisons - or division against remaining allowance, not overflowing size multiplication. -2. Reject an impossible container with `CodeSchemaProjectionLimit` before - `make`/map-key materialization. Preserve negative/unlimited internal allowances, - per-child charging, depth checks, key sorting, and inline-container accounting. -3. Do not generalize this into a new container-length or heap-size limit. Review - related reservations for the same known-lower-bound error without redesigning - typed cloning or making unrelated allocation promises. - -Acceptance: bounded tests cover exact/insufficient remaining allowances for maps, -slices, arrays, and flattened inline containers. Use a child process for the -maximum-length zero-sized slice case; prove safe rejection without a huge backing -allocation. Check defaults and resolution paths using the shared projector. -Focused check: `go test ./internal/engine -run 'Projection|Inline|Resolution'`. - -## Stage 5 — Close Callback Contexts On Every Exit - -Finding: **B01**. -Owners: `internal/engine/resolution.go`, `validation.go`, callback tests. - -Status: **complete** (B01). Prior stage commits: `2736ddf`, `deb9aa5`, -`ae3408a`, `ea27f17`. Validation passed: `GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Callback|Resolution|Validation|Version' -count=1` and `GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`. - -Wrap each application callback invocation in a small function with deferred -context deactivation. The defer must run when that callback returns or panics, -not at the end of subsequent projection/auditing work. Apply this independently -to resolution, final validation, and the shared version checker (`reportVersion`). -Retain the existing error/warning handling and propagate the original panic -unchanged; do not recover, synchronize, or broaden concurrent callback support. - -Acceptance: external recovery retains each context and confirms its operations -now produce the established closed-context panic. Use a distinct panic sentinel -to verify original propagation. Retain normal-return closure and lifecycle tests. -Focused check: `go test ./internal/engine -run 'Callback|Resolution|Validation|Version'`. - -## Stage 6 — Close The Owned Import Root During Panic Unwinding - -Finding: **B02**. -Owners: `internal/engine/import_acquisition.go`, `filesystem_access.go`, import tests. - -Status: **complete** (B02). Prior stage commits: `2736ddf`, `deb9aa5`, -`ae3408a`, `ea27f17`, `08a8c25`. Validation passed: `GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Import|SourceVersion|Filesystem' -count=1` and `GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`. - -Install deferred fallback closure at the entry of `acquireImportedWithAccess`, -before operations that may invoke application code. Use a local closed flag or -one local close-once function so existing explicit close sites preserve their -normal timing, return diagnostics, and primary-error precedence. Mark closure -attempted even if closing returns an error; do not re-close an owned handle or -close the borrowed native filesystem. The deferred fallback must not replace a -propagating callback panic with a cleanup failure. - -Acceptance: capture a real owned `os.Root` when invoking this acquisition boundary -with temporary host files, make an imported version checker panic, recover, and -verify that operations on that same root report closed state. Also verify child -file closure and normal/error exits. This observes the actual host root without -requiring a production FD counter or treating native file counts as equivalent. -Document genuine platform prerequisites; do not create a human validation gate. -Focused check: `go test ./internal/engine -run 'Import|SourceVersion|Filesystem'`. - -## Stage 7 — Preserve Directory Failure Phase And Per-Handle Evidence - -Finding: **C01**, including its integrated test improvements. -Owners: `internal/engine/filesystem_access.go`, `import_selection.go`, `glob.go`, -`native_filesystem_test.go`, and `resource_context_test.go`. - -Status: **complete** (C01). Prior completed commits: `2736ddf`, `deb9aa5`, -`ae3408a`, `ea27f17`, `08a8c25`, `877c497`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Directory|Glob|Filesystem|Resource' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -and `git diff --check`. - -1. Return private directory failure information distinguishing unsuccessful - initial open from enumeration/close failures. Preserve error unwrapping for - classification; do not expose backing error text. -2. Apply optional/missing behavior only to `ErrNotExist` from the initial open. - Batched `ReadDir`, no-progress, visit-stop closure, and final close failures - after opening are failures. Keep read failure ahead of an accompanying close - failure, and preserve existing selector diagnostics and entry accounting. -3. The fallback closes the initial file before using `fs.ReadDir` on the native - filesystem. Since initial opening succeeded, any fallback error, including - `ErrNotExist`, is a post-open failure. Preserve standard `ReadDirFS` dispatch - instead of imposing a new native-filesystem interface requirement. -4. Update both directory and glob selectors and all callers. Enhance the shared - fake to track individual opened-handle closure, including the fallback, rather - than only aggregate counts. Do not change regular-file close-error policy. - -Acceptance: directory plus glob, batched plus fallback paths, data/entries with -errors, read/close `ErrNotExist`, real missing-on-open controls, and a successful -second directory that must not hide failure. Assert nil result, structured -code/source, per-handle closure, and no canary in messages or rendered diagnostics. -Use a compact table/shared fixture, not duplicated matrices in every package. -Focused check: `go test ./internal/engine -run 'Directory|Glob|Filesystem|Resource'`. - -## Stage 8 — Memoize Completed Native Link Dependencies - -Finding: native part of **C02**. -Owners: `internal/engine/filesystem_links.go`, `filesystem_links_test.go`, -`directory_links_test.go`, and glob link tests. - -Status: **complete** (native C02). Prior completed commits: `2736ddf`, -`deb9aa5`, `ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`. -Validation passed: `env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Link|Directories|Glob' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -and `git diff --check`. - -1. Extract the existing portable frame evaluator into one private resolution - operation with `active` and `completed` maps. A small set of operation-local - functions supplies `Lstat`, link reading, and validated target normalization. - Keep this inside the engine; it is not a general filesystem/provider interface. -2. Wire native resolution first. Keep native prefix/root containment and target - admission rules in its binding. Key maps by the complete normalized link path; - the binding fixes the namespace for this one invocation. -3. An active back-edge remains a cycle. On a completed hit, restore the cached - canonical target and process the caller's remaining segments. Record completion - only when the link's own target frame finishes, not when first discovering it - or after resolving the entire outer pathname. Cache successful targets only. - -Acceptance: finite repeated links, nested/non-dot roots, self/mutual cycles, -outside-root targets, and follow-disabled behavior. Extend real-link fixtures -with a counting native wrapper for the A/B-doubling graph from C02. Use a generous -linear operation ceiling on small graph sizes, with a bounded abort to keep the -old failure safe; do not assert an exact call trace or elapsed time. Exercise both -directory and glob selection through this binding. -Focused check: `go test ./internal/engine -run 'Link|Directories|Glob'`. - -## Stage 9 — Use The Same Dependency Evaluation For Host Links - -Finding: remaining host part of **C02**. -Owners: `internal/engine/filesystem_links.go` and the same link/discovery tests. - -Status: **complete** (C02). Prior completed commits: `2736ddf`, `deb9aa5`, -`ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, `7bf2f0c`. -Validation passed: `env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Link|Directory|Glob|Filesystem|Discovery' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -and `git diff --check`. - -Bind rooted host `Lstat`/`Readlink` operations to Stage 8's evaluator and retire the -duplicate host frame loop. Retain portable component evaluation, rooted access, -host-relative targets, and contained absolute targets translated with `filepath`. -Preserve the final-entry follow policy and handling of links in parent components. -Keep the completed map invocation-local; do not attach a cross-call cache to -`filesystemAccess` or change behavior for concurrently changing filesystems. - -Acceptance: wrap real host-root operation functions in the private binding to -count bounded work for the same finite graph; this needs no production observer -hook. Verify contained absolute links, escaping targets, parent links, true cycles, -finite aliases, directory/glob integration, and independent namespaces. Run all -native cases again after the shared evaluator is fully wired. Remove stale -comments describing the old duplicated implementation. -Focused check: `go test ./internal/engine -run 'Link|Directory|Glob|Filesystem|Discovery'`. - -## Stage 10 — Remove The Two Confirmed Unused History Copies - -Findings: **D02, D03**. -Owners: `internal/engine/override_decode.go`, `override_context.go`, `load.go`, -`load_result.go`, and corresponding tests. - -Status: **complete** (D02, D03). Prior completed commits: `2736ddf`, -`deb9aa5`, `ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, -`7bf2f0c`, `0f9ecf1`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Override|History|Explain|Secret|Independent' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -and `git diff --check`. - -Remove `decodedOverrideResult.preCLI`, its clone, and assertions only about that -unused output. Keep the pre-CLI context actually built from `before.final`. -Remove `Inspection.structuralInput`, the final publication clone made solely for -it, and that clone's immutable-lookup inclusion. Preserve earlier structural-input -uses for metadata and sensitivity; preserve `Inspection.resolution` and all active -replay trees. Do not combine indexes, share mutable paths, or optimize other copies. - -Acceptance: CLI, inherited/removed paths, default reconciliation, historical -sensitivity, and independent-load/query ownership remain correct. Use existing -tests rather than adding tests of the absence of private fields. D01 is not fixed -by these deletions; its remaining root set is addressed next. -Focused check: `go test ./internal/engine -run 'Override|History|Explain|Secret|Independent'`. - -## Stage 11 — Remove Dead Combination Evidence And Orphan Helpers - -Findings: **D04, F05**. -Owners: `internal/engine/sequence_combine.go`, `inheritance*.go`, `compose.go`, -`history.go`, `secret.go`, `destination.go`, and associated tests. - -Status: **complete** (D04, F05). Prior completed commits: `2736ddf`, -`deb9aa5`, `ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, -`7bf2f0c`, `0f9ecf1`, `119b42c`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Sequence|Collection|Inheritance|History|Secret|Destination' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -and `git diff --check`. - -Remove unread `discardedLower` and `inheritanceReplacement.path`, `previous`, and -`effective`, plus calculations solely producing them. Keep replacement `before` -and `after` trees for accounting, projections, active origins/history, and append's -use of `sequenceIndexes`. Keep overlay's uses of `displacedOrigins`. - -Reconfirm and remove the six uncalled definitions listed in audit F05: -`mergeDisjointMappings`, `mergeHistoryDocument`, `inheritanceDirectiveLocation`, -`addSecretEvidencePath`, `PlacementModeFromFacade`, and `PlacementModeForFacade`. -Preserve their active counterparts and implicit/interface invocation elsewhere. -Update only private assertions about the removed evidence and stale comments/imports. - -Acceptance: sequence replace/append/upsert, inheritance history, sensitivity, -and retained replacement accounting still work. Inspect all remaining references; -absence of graph callers alone is not the deletion criterion. -Focused check: `go test ./internal/engine -run 'Sequence|Collection|Inheritance|History|Secret|Destination'`. - -## Stage 12 — Prepare The Actual Inspection Document State Once - -Finding: preparation portion of **D01**. -Owners: `internal/engine/load.go`, `load_result.go`, `history.go`, `inheritance_load.go`. - -Status: **complete** (preparation portion of D01). Prior completed commits: -`2736ddf`, `deb9aa5`, `ae3408a`, `ea27f17`, `08a8c25`, `877c497`, -`8aba43d`, `7bf2f0c`, `0f9ecf1`, `119b42c`, `247a12b`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Load|History|Explain|Inheritance|Validation|Secret' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -and `git diff --check`. - -1. Extract preparation of the existing `Inspection` document/history fields from - publication. Produce the actual owned pre-CLI copy, inherited-operation copies, - application lists, raw resolution snapshots, and coalesced replay that the result - will retain. Reuse the existing `Inspection` owner, not another universal DTO. -2. Prepare this state at the current history-check boundary: after resolution and - before secret environment lookup and final validation. Metadata's separately - cloned inheritance evidence is part of this state too. Keep required copies; - this stage moves their ownership, not their semantics. -3. Pass that prepared object onward privately. Publication fills metadata paths, - sensitivity, source evidence, contexts, diagnostics, and validated configuration - using the same document objects; it must not create fresh document/history - copies after the final quota check. Build the immutable lookup after admission. -4. Keep application callback/predicate order unchanged. In particular, do not - move secret classification/predicate calls ahead of validation just to prepare - the document roots. Earlier structural trees may remain temporary inputs to - finalization without becoming published roots. - -Acceptance: existing lifecycle and explanation behavior stays intact. Document -the concrete retained document fields for Stage 13. This preparatory stage alone -does not close D01; avoid temporary public state or compatibility APIs if the -handoff is not yet buildable. - -Prepared retained roots for Stage 13 admission are `Inspection.baseline`, -`input`, `final`, `preCLI`, source/CLI application values, raw and coalesced -resolution before/after values, and inspection inheritance parent/effective -values. `Metadata.inheritance` holds its separate cloned parent/effective values; -the immutable document lookup is built only during publication. -Focused check when buildable: `go test ./internal/engine -run 'Load|History|Explain|Inheritance|Validation|Secret'`. - -## Stage 13 — Enforce The Quota On The Prepared Published Graph - -Finding: completes **D01**. -Owners: `internal/engine/history.go`, `load.go`, `load_result.go`, history/lifecycle tests. - -Status: **complete** (D01). Prior completed commits: `2736ddf`, `deb9aa5`, -`ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, `7bf2f0c`, -`0f9ecf1`, `119b42c`, `247a12b`, `4fcff5f`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'History|Explain|Inheritance|Override|Resolution|Load' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -and `git diff --check`. - -1. Use one explicit root visitor on the prepared inspection, feeding a - pointer-aware `retainedNodeCounter` fresh for this publication, not the earlier - transient expansion counter. Include baseline, input, final, pre-CLI, - source/CLI application values, every raw and coalesced resolution before/after - tree, inspection inheritance parents/effective trees, and metadata inheritance - parents/effective trees. Count each distinct node once; different copies count - separately. Lookup keys may reference only trees covered by that retained set. - Keep the admission counter temporary; do not publish its visited-node map. -2. Replace the mismatched final checkpoint with this admission before secrets - and validation. Keep existing early inheritance guards; do not cumulatively - charge unrelated transient graphs against the final published-root check. - Share that publication counter with preparation to charge actual retained - copies as they become owned, stopping before further copies after overflow. - Do not charge an unretained source merely because it is about to be cloned. - Use source-size prechecks where straightforward, without a - new budget-aware generic clone framework or universal allocation guarantee. -3. Keep production `historyNodeLimit` unchanged. Pass a private history-limit - argument from the two engine entry paths to `executeLoadRequest` so engine - tests can exercise the real lifecycle with a small allowance. Do not add a - public option, global mutable limit, or environment switch. -4. Overflow returns nil result and the existing safe `CodeExplainLimitExceeded`. - Publication consumes the admitted prepared state with no unaccounted document - copies. Reuse the accounting visitor rather than maintaining two root lists - for separate ordinary/grouped implementations. - -Acceptance: use the real execution/publication path for a simple load and a small -layered CLI+inheritance+resolution case. Independently enumerate document fields -from the resulting inspection in test code, rather than using the production -collector as its own oracle. Exact count succeeds; one less fails before final -validation/environment lookup. Verify shared-node count-once behavior and distinct -copies. Keep required raw snapshot retention and detached inspection tests. -Focused check: `go test ./internal/engine -run 'History|Explain|Inheritance|Override|Resolution|Load'`. - -## Stage 14 — Cache Completed Dynamic Eligibility Within An Operation - -Finding: **A05**; requires Stage 2. -Owners: `internal/engine/value_operation.go`, projection/clone tests. - -Status: **complete** (A05). Prior completed commits: `2736ddf`, `deb9aa5`, -`ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, `7bf2f0c`, -`0f9ecf1`, `119b42c`, `247a12b`, `4fcff5f`, `12d350a`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Dynamic|Projection|Clone|Resolution' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -and `git diff --check`. - -Add an operation-local set of successfully approved concrete types/descriptors. -After a completed descriptor build and a full successful secret-eligibility walk, -record approval. Subsequent values of that type in the same `valueOperation` reuse -it. Do not treat cycle-walker visits as approvals, retain partial results, cache -path-bearing failures, or transfer this set across loads/operations. - -Acceptance: repeated typed nil pointers to a wide supported type avoid repeated -eligibility walks; true unsupported/secret-bearing types still fail, including -recursive descriptor graphs. Verify failed construction remains recoverable and -diagnostics identify the current path. Source/algorithm review must show O(S + N) -eligibility work for N repetitions of one graph of size S; no timing threshold or -new production instrumentation is necessary. -Focused check: `go test ./internal/engine -run 'Dynamic|Projection|Clone|Resolution'`. - -## Stage 15 — Establish Public Historical-Path And Record Ownership Coverage - -Findings: **F09**, replacement path coverage for **F12**, prerequisites for **F04**. -Owners: root external-package facade tests and existing source-snapshot tests. - -Status: **complete** (F09; replacement path coverage for F12; public snapshot -ownership review for F04). Prior completed commits: `2736ddf`, `deb9aa5`, -`ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, `7bf2f0c`, -`0f9ecf1`, `119b42c`, `247a12b`, `4fcff5f`, `12d350a`, `08f2a8f`. -Validation passed: `env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test . -run 'Historical|Metadata|Facade|Source|WrappedPaths' -count=1` -and `git diff --check`. Existing public source-version and import-selection -snapshot tests cover repeated-query detachment of location/version pointers and -nested selections; no duplicate ownership matrix was added. - -Add compact public tests using only root APIs. Cover effective and removed -historical paths, sorted unique pointer spelling, zero metadata, escaped keys, -and a numeric-key/sequence shape change. Feed returned paths into `Lookup` and -`History` to verify contextual shape behavior. Replace an element of a returned -slice and query again; do not expose private segment storage to test detachment. - -Before deleting clone scaffolding, verify meaningful mutable returned source -records through repeated public queries: location/version pointers and nested -selection slices stay detached, with existing nil/empty distinctions preserved. -Reuse existing cases where they already protect the risk rather than duplicating -the engine matrix. Remove `TestWrappedPathsRetainEngineSegmentIntent` only after -its useful public behavior is covered. - -Acceptance: a broken root path conversion or shallow returned record would fail -these tests even while engine tests pass. Engine path-intent tests remain. -Focused check: `go test . -run 'Historical|Metadata|Facade|Source|WrappedPaths'`. - -## Stage 16 — Retire Disconnected Facade Clones And Redundant Scaffolding - -Findings: **F04**, remaining **F12**; requires Stage 15. -Owners: root `records.go`, `records_test.go`, `facade_internal_test.go`, -`facade_compatibility_test.go`, `facade_candidate_test.go`, `facade_equivalence_test.go`. - -Status: **complete** (F04; remaining F12). Prior completed commits: `2736ddf`, -`deb9aa5`, `ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, `7bf2f0c`, -`0f9ecf1`, `119b42c`, `247a12b`, `4fcff5f`, `12d350a`, `08f2a8f`, `826062a`. -Validation passed: `env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test .`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./examples/consumers`, -and `git diff --check`. - -Remove the complete private clone family named in audit F04 after reconfirming -that only the tests enter it. Preserve engine clones, active `Path.clone`, explicit -root conversions, and nil/empty conversion checks. Remove the ineffective -whole-Destination assignment assertion and private clone-only assertions, retaining -conversion/public query ownership protection from Stage 15. - -Remove the duplicate six-type ownership loop and trivial assignment equality -assertion, keeping the comprehensive root type owner and compile-time interfaces. -Route source-ownership tests through public `Load` and root error typing instead -of `source.toEngine()` plus direct engine loads. Consolidate reader cases with the -existing equivalence owner while preserving byte mutation and copied-reader cases. - -Acceptance: public construction, errors, callback lifetime, source identity, -disclosure, and returned ownership remain represented. Do not delete static public -or dependency-boundary tests or infer that engine coverage replaces facade tests. -Focused check: `go test .` and `go test ./examples/consumers`. - -## Stage 17 — Remove The Remaining Unread Private Members - -Finding: completes **F06**, following its schema-path subset in Stage 1. -Owners: the exact records/files listed in [audit F06](audit.md#f06-trim-unread-private-result-and-transport-members). - -Status: **complete** (F06). Prior completed commits: `2736ddf`, `deb9aa5`, -`ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, `7bf2f0c`, -`0f9ecf1`, `119b42c`, `247a12b`, `4fcff5f`, `12d350a`, `08f2a8f`, -`826062a`, `e4a5ecb`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test .`, and -`git diff --check`. - -Remove `collectionPolicyResolution`'s four named unread fields; `inputPresence.null`; -directive extraction's two presence flags; `importSelection.outcome`; -`importDeclarationSpec.excludeSet`; `selectedOverlayRequest.filesystem`; -`sequenceDeclarationSpec.destinationLocation`; source-version evidence's `absence` -and `checked` and ancestor `kind`; and `postDecodeOverrideContext.sources`. -Remove computations only feeding those slots and adjust their private assertions. - -Keep declaration normalization, effective policy flags, candidate/admission rules, -public empty-selection outcomes, actual version callbacks, ancestor sensitivity -paths, public null metadata, location reporting, and the local source list used -to construct resolution input. Check references/build constraints again; do not -extend this deletion list to identity tokens, map-key fields, or interface methods. - -Acceptance: use public behavior for tests that formerly checked inactive annotations. -No active validation or distinction is removed with its redundant transport flag. -Focused check: `go test ./internal/engine` and `go test .`. - -## Stage 18 — Retire The Parallel Import Directive Parser - -Finding: **F03**. -Owners: `internal/engine/import_spec.go`, `import_spec_test.go`, directive/selector tests. - -Status: **complete** (F03). Prior completed commits: `2736ddf`, `deb9aa5`, -`ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, `7bf2f0c`, -`0f9ecf1`, `119b42c`, `247a12b`, `4fcff5f`, `12d350a`, `08f2a8f`, -`826062a`, `e4a5ecb`, `3122e2f`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Import|Directive|Directory|Glob|SourceVersion' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine`, -and `git diff --check`. - -Change `parseImportSpecsYAML` to enter `extractRootDirectives` with the appropriate -source context and return its imports/diagnostics. Keep parsing source fixtures -through the real YAML boundary. Remove `parseImportSpecificationDirective`, its -parallel mapping/field validation, and support used solely by it. Do not remove -`parseImportDeclarations` or the imports-only context still used after layered -outer-directive extraction. - -Acceptance: malformed fields/types/selectors and exact diagnostic paths continue -to be checked by the actual production parser; valid import fixtures retain their -meaning. Inspect the assertions before accepting wording changes from replacing -the obsolete parser. Keep classifications/sites, not incidental wording, as the -contractual expectations. -Focused check: `go test ./internal/engine -run 'Import|Directive|Directory|Glob|SourceVersion'`. - -## Stage 19 — Strengthen Ownership Fuzzing And Observable Import Ordering - -Findings: **F10, F11**. -Owners: `internal/engine/composition_fuzz_test.go`, `import_selection_test.go`. - -Status: **complete** (F10, F11). Prior completed commits: `2736ddf`, `deb9aa5`, -`ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, `7bf2f0c`, -`0f9ecf1`, `119b42c`, `247a12b`, `4fcff5f`, `12d350a`, `08f2a8f`, -`826062a`, `e4a5ecb`, `3122e2f`, `6f6130f`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'FuzzCompositionAndOverlay|SelectNamedImports' -count=1` -and `git diff --check`. The bounded composition campaign remains scheduled for -Stage 33. - -In `FuzzCompositionAndOverlay`, mutate an existing retained descendant or in-length -element before comparing both original inputs with their saved values. Handle -empty containers without assuming append affects shared storage. Keep existing -bounded inputs, immutability checks, and commutativity properties; do not add -another fuzz target or ownership matrix. - -Replace the exact two-stats-per-file assertion in -`TestSelectNamedImportsChecksNormalizedPathsLexically` with ordered candidate paths -and meaningful first-failure location. Consolidate with existing lexical-selection -tests if this subsumes the standalone case. Keep non-opening of excluded contents -and genuine first-error behavior where those are the tested requirements. - -Acceptance: a shallow existing-descendant alias would fail the fuzz assertion, -while a harmless change in stat implementation would not fail ordering coverage. -Focused check: `go test ./internal/engine -run 'FuzzCompositionAndOverlay|SelectNamedImports'`. -Run the bounded composition campaign in Stage 33 after the later assembly changes -have landed, rather than treating an earlier run as evidence for those changes. - -## Stage 20 — Use Only Deferred Acquisition In Production And Fixtures - -Finding: **F01**. -Owners: `internal/engine/acquisition.go`, `import_acquisition.go`, `imports_test.go`, -and callers identified by request-construction tracing. - -Remove `deferAssembly` and the eager branch. Retain pending-item placement -reservation and return acquired scopes/import graph/accounting consistently. -Migrate test helpers needing final values to invoke the real composition boundary -explicitly after acquisition. Keep acquisition-only tests on the acquired scope -instead of silently composing it. Ordinary and grouped production requests use -the same handoff without a replacement mode flag. - -Acceptance: imports, item-local scopes, nested destinations, dependency ordering, -version protections, optional absence, resource closure, and byte/node accounting -still run through the active protocol. Keep tests' intended boundaries; do not -delete meaningful composition cases when changing fixture setup. -Focused check: `go test ./internal/engine -run 'Import|Destination|SourceVersion|LoadGrouped|Underlay'`. - -Status: **complete** (F01). Prior completed commits: `2736ddf`, `deb9aa5`, -`ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, `7bf2f0c`, `0f9ecf1`, -`119b42c`, `247a12b`, `4fcff5f`, `12d350a`, `08f2a8f`, `826062a`, `e4a5ecb`, -`3122e2f`, `6f6130f`, and `4fd3bee`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Import|Destination|SourceVersion|LoadGrouped|Underlay' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -count=1`, -and `git diff --check`. - -## Stage 21 — Isolate Parsed-Input Directive And Version Processing - -Finding: first part of **F02**; requires Stage 20. -Owners: `internal/engine/acquisition.go`, existing directive/source-version records. - -Extract the directive/version-processing block from `acquireInput` into a cohesive -operation returning the existing `sourceVersionProcessingResult` and diagnostics. -Pass the parsed source, existing `inputAcquisitionRequest`, and load-local version -state explicitly; read role, destination/placement, context, and protection from -that request rather than introducing a long flattened parameter list. Compute -source ID once. Keep updating -`mainProcessing` with a clearly owned completed result, not pointers into incidental -locals scattered among branches. - -Retain distinct source and layered processing: ordinary sources process their -root directives with their source policy; layered loading extracts its outer -directives before enabled version handling and restores those declarations to -the processing result. Default/enabled mode timing and warning accumulation must -not become one inferred common policy. Keep the imports-only processing context -where it remains necessary. Do not add a new context hierarchy or public request. - -Acceptance: source-version tests for main, imports, underlays, mounted/item -destinations, selected overlays, and NoSource preserve callback counts, order, -warnings, safe diagnostics, and retained declaration locations. -Focused check: `go test ./internal/engine -run 'SourceVersion|Version|Directive|Destination'`. - -Status: **complete** (first part of F02). Prior completed commits: `2736ddf`, -`deb9aa5`, `ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, `7bf2f0c`, -`0f9ecf1`, `119b42c`, `247a12b`, `4fcff5f`, `12d350a`, `08f2a8f`, `826062a`, -`e4a5ecb`, `3122e2f`, `6f6130f`, `4fd3bee`, and `4bf940c`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'SourceVersion|Version|Directive|Destination' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -count=1`, -and `git diff --check`. - -## Stage 22 — Isolate Standalone Placement And Acquisition Completion - -Finding: second part of **F02**. -Owners: `internal/engine/acquisition.go`, `load_input.go`, destination/context helpers. - -Extract standalone contribution placement/registration and passive-context -completion into explicit operations on the existing assembled-input/acquisition -owners. Keep discovery selection and parsing visible in `acquireInput`, then make -its imported-versus-standalone dispatch and completion readable. Each helper -returns its result/diagnostic; avoid closures mutating a collection of surrounding -locals. Keep live handles confined to their acquisition owner. - -Preserve local versus placed documents, unresolved item destinations, wrapper-node -reservation, source identity, NoSource/discovery behavior, and the ordinary overlay -case that defers filesystem completion. Review aliasing of source slices and -version evidence when assembling the return value. - -Acceptance: existing discovery, standalone mapping/reader/native/host input, -destination accounting, item placement, and passive resource-context tests still -exercise the same observable behavior. Do not require exact private helper calls. -Focused check: `go test ./internal/engine -run 'Discovery|Destination|Source|Resource|Filesystem'`. - -Status: **complete** (second part of F02). Prior completed commits: `2736ddf`, -`deb9aa5`, `ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, `7bf2f0c`, -`0f9ecf1`, `119b42c`, `247a12b`, `4fcff5f`, `12d350a`, `08f2a8f`, `826062a`, -`e4a5ecb`, `3122e2f`, `6f6130f`, `4fd3bee`, `4bf940c`, and `d8bf6ab`. Validation -passed: `env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Discovery|Destination|Source|Resource|Filesystem' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -count=1`, -and `git diff --check`. - -## Stage 23 — Clarify Ordinary Context And Structural-Expansion Handoffs - -Finding: third part of **F02**. -Owners: `internal/engine/load.go`, `load_input.go`, inheritance/version helpers. - -Extract the substantial ordinary-load filesystem/resource-context fallback into -an operation that returns the completed `assembledInput` plus diagnostics. Keep -existing resolved filesystem facts when available rather than reopening handles -to recover the same information; retain required validation when facts are absent. -Isolate structural inheritance expansion/version-path -projection as one further handoff using the existing composed/assembled records. - -Keep the coordinator's visible order: schema/source-version timing, acquisition, -ordinary/grouped assembly and selected overlay, structural expansion, decoding, -CLI, resolution, Stage 13 history admission, secrets, identity validation, final -validation, and publication. Do not move callback execution into an extraction -that changes when errors suppress later phases. - -Acceptance: ordinary imports and non-import overlays, absent main input with -explicit context, resource paths, inheritance version protection, callback order, -and fail-before-later-callback cases remain intact. Re-run panic cleanup and -retained-history integration cases after moving the handoffs. -Focused check: `go test ./internal/engine -run 'Load|Overlay|Resource|Inheritance|Version|History'`. - -Status: **complete** (third part of F02). Prior completed commits: `2736ddf`, -`deb9aa5`, `ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, `7bf2f0c`, -`0f9ecf1`, `119b42c`, `247a12b`, `4fcff5f`, `12d350a`, `08f2a8f`, `826062a`, -`e4a5ecb`, `3122e2f`, `6f6130f`, `4fd3bee`, `4bf940c`, `d8bf6ab`, and `f185821`. -Validation passed: `env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Load|Overlay|Resource|Inheritance|Version|History' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'ImportedAcquisitionCloses|History' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -count=1`, -and `git diff --check`. - -## Stage 24 — Share Layered Tier Application And Evidence Projection - -Finding: completes **F02**. -Owners: `internal/engine/load_request.go`, `load_input.go`, tier projection helpers. - -Extract the repeated completed-tier application operation: collect that tier's -declarations, initialize or apply through the assembly owner, then project only -the incoming sources/version evidence using the returned sequence operations. -Use explicit source/evidence ranges from the caller; keep main-version projection -and its protection rules recognizable rather than hiding them in a generic loop. - -Keep acquisition order separate from precedence application, distinct tier-local -and load-wide accounting, absence of empty groups, main-only default version -checking, and selected-overlay rules. This helper must accept the existing policy -scope contract so Stages 27–28 can later change that private contract once, at -named composition/application boundaries. - -Acceptance: grouped source ordering, underlay precedence, strict-within-tier -conflicts, mounted source/version coordinates, ordinary/grouped differences, -metadata/history/diff, and warnings remain stable. Update the implementation-flow -description only after the handoffs actually land, without restating YAML policy. -Focused check: `go test ./internal/engine -run 'Grouped|Layered|Underlay|SourceVersion|Sequence|Diff'`. - -Status: **complete** (completes F02). Prior completed F02 commits: -`4bf940c`, `d8bf6ab`, `f185821`, and `a8ae2e9`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Grouped|Layered|Underlay|SourceVersion|Sequence|Diff' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -count=1`, -and `git diff --check`. - -## Stage 25 — Separate Collection Preparation, Application, And Finalization - -Finding: preparation/ownership part of **E03**. -Owners: `internal/engine/overlay.go`, `compose.go`, `collection_policy*.go`. - -Refactor `mergeOverlayMappingsWithCollectionPolicy` into three named responsibilities: - -- Prepare incoming claims/declaration groups and per-destination resolutions - against read-only lower state, with no value/origin mutation. -- Apply recursive value combination through one small overlay operation owner - holding the existing result/index plus its accumulated diagnostics, sequence - history, and destinations receiving values. Use its named callback methods - rather than many callbacks capturing separate mutable maps/slices. Put the - root-origin merge in this application phase as well. -- Finalize declaration-only policy against the completed value and emit sorted - sequence evidence. Build any retained-value lookup after mutation, not before. - -Keep these as engine-private operations, not generic strategy interfaces. Preserve -current data semantics here; Stages 26–28 replace the full-state work. A temporary -large clone retained by this extraction is not completion of E01/E02. - -Acceptance: policy declarations/conflicts, pending keys, mode-only changes, -replace/append/upsert, nested record ownership, origin/history ordering, and failed -combination diagnostics remain protected. Newly separated inputs/results make -lower versus incoming versus retained ownership explicit. -Focused check: `go test ./internal/engine -run 'Collection|Sequence|Overlay|Inheritance|History'`. - -Status: **complete** (preparation/ownership part of E03). Prior completed commits: -`2736ddf`, `deb9aa5`, `ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, -`7bf2f0c`, `0f9ecf1`, `119b42c`, `247a12b`, `4fcff5f`, `12d350a`, `08f2a8f`, -`826062a`, `e4a5ecb`, `3122e2f`, `6f6130f`, `4fd3bee`, `4bf940c`, `d8bf6ab`, -`f185821`, `a8ae2e9`, and `8832449`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Collection|Sequence|Overlay|Inheritance|History' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -count=1`, -and `git diff --check`. - -## Stage 26 — Index Assembly Policy Retirement By Semantic Path - -Finding: **E01**. -Owners: `internal/engine/collection_policy_state.go`, `compose.go`, `overlay.go`, -and a small private policy-index file if that clarifies ownership. - -1. Add a mutable semantic-segment retirement trie alongside the assembly's policy - table. Index destinations with state cleared by `clearCollectionIdentity`; - mode-only entries stay in the table for exact lookup but need no retirement - terminal. Use `pathSegment` keys so numeric mapping keys and sequence indices - stay distinct. Build once when the assembly adopts its initial owned scope, - not for every replacement or tier. -2. Route assembly policy writes/retirement through its owner so table and index - remain synchronized. Keep immutable/local scopes outside this owner independent; - do not add index rebuilding to their every lookup or snapshot. -3. Descendant retirement walks only the indexed subtree for the replaced prefix. - Clear old YAML identity and record-owned state, preserving explicit modes and - application constraints. Retire before installing policy introduced by the - incoming replacement; declarations established afterward must survive. Remove - cleared terminals and prune empty branches. Replacing an ancestor again must - not rescan already cleared mode-only policies; later writes reindex new state. -4. Reuse the semantic-index conventions of `inheritancePolicyIndex`, but keep - this mutable assembly owner distinct from that borrowed record-local index. - Do not change the pointer-spelling index used for history/dependencies. - -Acceptance: retain `collection_retirement_test.go` cases for all replacement kinds, -ancestor removal with new pending identity, explicit modes, and registrations. -Add escaped/numeric semantic-path cases and a bounded disjoint P-policy/Q-scalar -case, plus repeated retirement above retained mode-only entries. Review the -complete caller path: one index construction plus prefix/affected-subtree work -must replace Q scans of all P entries. No exact syscall/timing test -or public complexity guarantee is needed. -Focused check: `go test ./internal/engine -run 'Collection|Retirement|Sequence|Inheritance'`. - -Status: **complete** (E01). Prior completed commits: `2736ddf`, `deb9aa5`, -`ae3408a`, `ea27f17`, `08a8c25`, `877c497`, `8aba43d`, `7bf2f0c`, `0f9ecf1`, -`119b42c`, `247a12b`, `4fcff5f`, `12d350a`, `08f2a8f`, `826062a`, `e4a5ecb`, -`3122e2f`, `6f6130f`, `4fd3bee`, `4bf940c`, `d8bf6ab`, `f185821`, `a8ae2e9`, -`8832449`, and `549fca2`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Collection|Retirement|Sequence|Inheritance' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -count=1`, -and `git diff --check`. - -## Stage 27 — Borrow Lower Policy And Produce Sparse Strict-Tier State - -Finding: first part of **E02**; coordinates **E03**. -Owners: `internal/engine/collection_retirement.go`, `compose.go`, `import_scope.go`, -and nested/acquired scope composition callers. - -1. Replace the full lower-scope copy used for strict resolution with a small - borrowed read view: the existing lower table plus current replacement masks. - Exact lookup returns an immutable policy view; if old identity is masked at - that destination, clear it in the returned header, not in the borrowed table. - Keep explicit modes. Reuse `collectionReplacementPaths`' inclusive/exclusive - prefix semantics; do not accumulate a chain of views across tiers. -2. Compute replacement eligibility from incoming claims/declarations and exact - lower/application lookups. Do not enumerate the full lower table just to clear - entries in a transient copy. Clone an affected policy only when constructing - an independently owned result or changing record-owned state. -3. `resolveStrictCollectionPolicies` produces identity information and an owned - sparse policy scope for destinations touched by incoming values/declarations. - That scope is not another complete copy of accumulated lower policy. Preserve - record ownership when a declaration has no new value. -4. Thread this contract through acquired/item-local strict composition. Keep - complete independently owned snapshots where local record scopes/history need - them; sparse incoming state must not accidentally borrow mutable record maps. - Update private tests that assumed an incoming tier included unrelated lower - entries to check the complete assembled result instead. - -Acceptance: strict duplicate rejection still sees applicable lower identity; -higher declarations cannot repair invalid lower composition. Pending declarations, -replacement masks, mode persistence, and nested item-local scopes remain correct. -Inspect the scalar-only-tier path: work depends on incoming claims/declarations, -not P unrelated retained policies. Wiring to assembly is finished in Stage 28; -do not retain an eager full-scope adapter merely to make this stage compile. -Focused check when buildable: `go test ./internal/engine -run 'Compose|Collection|Item|Underlay|Sequence'`. - -Status: **complete** (first part of E02; coordinates E03). Prior completed -commits include `8832449`, `549fca2`, and `68383f3`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Compose|Collection|Item|Underlay|Sequence' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test -race ./...`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off ./scripts/check.sh`, -and `git diff --check`. - -## Stage 28 — Apply Sparse Incoming Policy Without Full-Table Preparation - -Finding: completes **E02**; uses Stage 26 and Stage 27. -Owners: `internal/engine/overlay.go`, `load_request.go`, `import_scope.go`, -`inheritance_prepare.go`, and collection retirement callers. - -1. In overlay preparation, discover relevant destinations by walking incoming - claims and declaration paths, checking the lower table at those exact paths. - Replace the loop over every retained policy and the whole-overlay lookup built - solely to support that loop. Include non-sequence claims at exact retained - policy destinations so invalid append/upsert inputs still fail as before. -2. Use the borrowed lower view for operation resolution; retain only incoming - owned resolutions and sparse policy changes. Apply those changes at the named - assembly owner using Stage 26's synchronized updates. Unchanged lower entries - remain in that owner, not copied into every incoming tier. -3. Retirement caused by replaced ancestors uses the live index at the existing - mutation boundary. Reconcile declaration-only updates after value application. - Preserve current-tier policy established after retirement and validate a - retained sequence when a declaration alone requires it. -4. Update tier application and any inheritance callers of the old lower-policy - preparation helper. For inheritance, retain independent parent/child local - scopes where ownership requires them; share the replacement-mask/read-view - logic rather than copying the load-wide policy store into a tier. - -Acceptance: with P retained policies and T disjoint scalar-only tiers, no path may -clone/scan all P policies per tier or rebuild their index. Inspect all preparation, -strict composition, application, and finalization callers for hidden full-table -fallbacks. Add a small multi-tier case preserving untouched rules and nested -record owners, plus retained-key-only/mode-only and ancestor-replacement cases. -Required value/history copies and affected record payload remain legitimate costs. -Focused check: `go test ./internal/engine -run 'Collection|Retirement|Sequence|Underlay|Layered|Inheritance|History'`. - -Status: **complete** (E02). Prior completed commits include `8832449`, -`549fca2`, `68383f3`, and `3a1b6ff`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Collection|Retirement|Sequence|Underlay|Layered|Inheritance|History' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test -race ./...`, -and `git diff --check`. - -## Stage 29 — Consolidate Sequence-History Translation And Finalize Policy Owners - -Finding: completes **E03**. -Owners: `internal/engine/overlay.go`, `inheritance_combine.go`, -`inheritance_collection.go`, and sequence-history helpers. - -Extract one pure sequence-history descriptor constructor taking destination, -operation, incoming value, and `sequenceCombinationResult`. It owns replace-versus- -mapped translation and detached projection/replacement storage. Use it in both -overlay and inheritance instead of independently rebuilding that descriptor. - -Keep overlay and inheritance record-policy rebuilding as separate named operations: -overlay retains incoming/lower record ownership, while inheritance uses current -versus inherited scope conversions with different mode lifetimes. Share existing -projection indexes but do not add a callback/strategy framework or common mutable -origin owner to unify those distinct operations. This resolves the roadmap's -record-selection-sharing evaluation in favor of explicit ownership-specific loops. - -Acceptance: value combination still has one `combineSequence` owner; preparation, -application, and declaration-only finalization have clear inputs/results without -reintroducing full-table work. Verify sequence history and sparse nested projection, -record replacement/removal, modes, origin locations, and sensitivity for both -precedence and inheritance. -Focused check: `go test ./internal/engine -run 'Sequence|Collection|Inheritance|History|Explain|Sensitivity'`. - -Status: **complete** (E03). Prior completed commits include `8832449`, -`549fca2`, `3a1b6ff`, and `d172a5e`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Sequence|Collection|Inheritance|History|Explain|Sensitivity' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test -race ./...`, -and `git diff --check`. - -## Stage 30 — Index Original Tier-History Values Lazily - -Finding: **E04**. -Owners: `internal/engine/sequence_history_projection.go`, projection/history tests. - -In `projectTierHistoryApplications`, lazily construct one `documentLookup` for an -original application value only when an applicable mapped sequence needs its -container. Reuse it for that application's sequence contributions through -`documentAtPathWithLookup`. Keep source values immutable and authored coordinates -separate from assigned output positions. Rebuild projected sidecar indexes after -edits as before; do not retain the temporary lookup or cache it across loads. - -Acceptance: wildcard-selected sibling collections, sparse/noncontiguous mappings, -nested sequence projections, and upsert replacement evidence remain correct and -detached. Source review shows O(V + S × D + I) lookup/projection work for one -application, excluding existing sorting/copies, rather than scanning a V-wide -mapping for each of S contributions. Use existing wide/sparse fixtures, not timing -thresholds or another explanation-replay implementation. -Focused check: `go test ./internal/engine -run 'Tier|Projection|Sequence|History'`. - -Status: **complete** (E04). Prior completed commits include `3a1b6ff`, -`d172a5e`, and `1dd94a3`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Tier|Projection|Sequence|History' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -count=1`, -and `git diff --check`. - -## Stage 31 — Index Compound Identity Restoration - -Finding: **E05**. -Owners: `internal/engine/inheritance_scope.go` and identified-inheritance tests. - -Build an identity-field membership set and a mapping-position index for the merged -record. Walk child entries once; for identity members, replace the corresponding -merged entry with a detached copy preserving the child's key/value locations. -Keep merged mapping order and exact identity comparison/parent selection. Do not -remove the restoration step or reinterpret upsert as recursive body merging. - -Acceptance: compound identity, many nonidentity siblings, escaped/literal keys, -child locations, mapping order, missing-key validation, and detached retained -templates remain correct. Source review establishes O(K + C + M) lookup work plus -required copied values for K identity fields and C/M mapping entries. -Focused check: `go test ./internal/engine -run 'Identified|Identity|Inheritance'`. - -Status: **complete** (E05). Prior completed commits include `549fca2`, -`3a1b6ff`, `d172a5e`, `1dd94a3`, and `75e209e`. Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Identified|Identity|Inheritance' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test -race ./...`, -and `git diff --check`. - -## Stage 32 — Index Secret Failure Locations Only When Needed - -Finding: **F08**. -Owners: `internal/engine/secret_reference.go`, `validation.go` lookup helper, -`secret_reference_test.go`. - -Let the existing per-resolution secret operation own a nil `documentLookup` until -its first diagnostic needs attribution. On that failure, build it from the fixed -reference-input tree and reuse `nearestDocumentLocationWithLookup` for subsequent -failures. Thread that operation-local state through the existing walk/diagnostic -helper rather than rebuilding an index per reference or introducing global state. - -Acceptance: many missing/empty/invalid references in a wide mapping retain every -safe diagnostic and exact/ancestor fallback location; repeated environment names -still use the existing lookup cache. Include escaped/numeric and defaults-only -paths. The successful path builds no failure-only document index. Inspect for -O(V + F × D) location work, apart from sorting, rather than repeated full scans; -keep the environment cache and credential bytes out of diagnostics/history. -Focused check: `go test ./internal/engine -run 'Secret|Location|Validation'`. - -Status: **complete** (F08). Prior completed commit: `f132cd6`. Validation -passed: `env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Secret|Location|Validation' -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -count=1`, -and `git diff --check`. - -## Stage 33 — Integrate, Document, And Verify The Complete Sprint - -Findings: final acceptance for **all selected IDs**. - -1. Review the traceability table below against actual changes and tests. Finish - every check deferred by an intermediate unbuildable stage. Reconfirm no - selected work was silently replaced by a smaller cleanup or deferred profiling. -2. Review boundary interactions: transactional descriptors plus eligibility - caching; slice views plus detachment; callback panic plus root closure; both - directory-reading implementations and link bindings; complete published-history - admission after copy cleanup; policy deltas/retirement plus tier history, - inheritance, source-version projection, and secret locations. -3. Reuse representative public/consumer cases to verify agreement among final - values, presence/null, origins, historical queries, explanation, redaction, - diff, safe errors, and independent loads. Add only a missing integration case - with a concrete risk; do not duplicate every engine test at the root. -4. Update `docs/internal/overview.md` and `docs/internal/imports.md` for the actual - resulting responsibilities. Correct stale implementation-stage comments in - changed owners. Update canonical declaration/configuration wording only where - needed to accurately describe a repaired existing behavior; do not recast - private algorithms or tests as new public promises. No new ADR/policy is needed. -5. Perform the checks below, inspect the diff for unintended public/API, dependency, - limit, or scope changes, and record completion/validation per stage. Keep the - selected planning records until normal release preparation retires them. - -Final checks, from the repository root: +Run from the repository root with an installed compatible Go toolchain and +dependencies already available. Use `GOTOOLCHAIN=local`, `GOPROXY=off`, and +`GOTELEMETRY=off`. Keep disposable artifacts outside the repository and do not +change dependencies for tooling. Focused expressions identify behavior owners; +include a newly named regression explicitly if needed. Reuse unchanged successful +checks. No timing, allocation, exact syscall, or helper-shape assertions are +acceptance criteria. + +## Stage 1 — Extract Patterned Glob Entry Preparation + +Finding: **C03**. Owners: `internal/engine/glob.go`, existing glob selection, +link, and directory failure tests. Status: **not started**. + +1. Add `prepareGlobEntry` beside the selector. Take the filesystem access, + base path, parent `selectedImportCandidate`, `fs.DirEntry`, pattern segment, + `last` flag, and `resolvedImportPolicy`. Return a candidate, a boolean meaning + selected versus ordinary skip, and a nullable `*Diagnostic`. Use the parent's + existing `site`, which is propagated from the selector's root candidate. + No candidate-set, accounting, or traversal-state mutation belongs here. +2. Move only the patterned branch's per-entry body into this helper. Preserve + its order: `path.Match`; match/hidden filtering; lexical/access path creation + and containment; `importEntryTarget` and existing diagnostic translation; + follow filtering; intermediate-directory eligibility; host/native physical + path selection and canonicalization fallback; candidate label/site creation. + Return existing safe codes, paths, and sites. Do not turn inspection errors + into skips or skip validation merely because exclusion will later match. +3. Replace that inline body with the helper call and candidate insertion. + Leave pattern validation, `explicitGlobCandidate`, directory enumeration, + per-segment union/sorting, final-file validation, exclusions, requiredness, + and source-admission timing in the selector. +4. Keep directory entry preparation separate. Reuse existing filesystem/link + primitives; do not introduce a generic entry strategy or unify differing + missing-target, literal-link, filtering, or recursion rules. Remove imports + only when genuinely unused after extraction. + +Acceptance: the selector now expresses traversal without embedding preparation +details. Reuse `glob_selection_test.go`, `glob_test.go`, filesystem-link tests, +and the preceding sprint's directory-phase regressions. They must still cover +literal hidden/link components versus wildcard filtering, aliases and native +spelling, escaping/cyclic links, matched-directory errors before exclusion, +selection order, and bounded accounting. Add a behavioral case only for a gap +identified during extraction, not a unit test asserting the new helper exists. + +Focused check: +`go test ./internal/engine -run 'Glob|Directory|Link|Filesystem|Resource'`. + +## Stage 2 — Consolidate The Engine Defaults Representation + +Finding: **F07**, engine representation. Owners: `internal/engine/import_spec.go`, +`load.go`, `layered.go`, `load_request.go`, `acquisition.go`, `options.go`, and +their callers/fixtures. Status: **not started**. + +1. Retain engine `ImportDefaults` with its existing five exported fields. + Remove private `importDefaults`, without a compatibility alias. Update + engine option/request/acquisition fields, helper signatures, readers, and + fixture literals to use the canonical type and exported default-field names. + Private containing fields such as `loadRequest.importDefaults` can keep their + names. Do not rename the distinct fields of `importDeclarationSpec` or + `resolvedImportPolicy` just because they have similar meanings. +2. Expose `ImportDefaults ImportDefaults` on engine `Options[T]` and + `LayeredOptions[T]`. Remove `OptionsWithImportDefaults`, + `LayeredOptionsWithImportDefaults`, and `privateImportDefaults`. Stage 3 changes + root callers; a temporary compile failure is preferable to an interim bridge. +3. Adapt `snapshotImportDefaults` to the canonical type, retaining nil boolean + pointers and nil versus non-nil empty pattern. Keep `cloneBool` and + `cloneStrings` where they still have real consumers. Keep current snapshot + call sites until Stage 3 establishes their final ownership placement. +4. Keep effective declaration policy resolution and validation semantics intact: + explicit false beats defaults, each control resolves independently, and + selector-specific applicability and depth checks stay unchanged. Do not + combine defaults with resolved policy or registration compilation. + +Acceptance: one engine input-defaults type reaches every production path and +fixture; no equivalent private record or alias remains. Inspect ordinary, +layered, transitive-import, directory/glob, and test-helper call sites. The root +cutover and removal of redundant snapshots are explicitly pending until Stage 3. + +Run the import/default focused tests after Stage 3 if the root cutover prevents +building the package here; do not report that deferred check as passed. + +## Stage 3 — Simplify Facade Defaults Construction And Snapshot Ownership + +Finding: **F07**, defaults handoff. Owners: root `conversion.go`, `lifecycle.go`, +`layered.go`; engine defaults/preflight/coordinator owners; `facade_import_defaults_test.go` +and `internal/engine/import_spec_test.go`. Status: **not started**. + +1. Set engine `ImportDefaults` fields directly in root `toEngineOptions` and + `toEngineLayeredOptions`. Retain `toEngineImportDefaults` as the explicit + root-to-engine type mapping, but map its booleans/pattern without copies. + Remove root `cloneImportDefaultBool` and `cloneImportDefaultStrings` after + confirming no other caller. Keep public root types and declaration contracts. +2. Keep `snapshotImportDefaults` in `loadSettings` as the load-entry ownership + operation for both APIs. Remove repeated snapshots in engine `LoadLayered` + request construction, `executeLoadRequest` acquisition-policy construction, + and `resolveImportPolicy`. Carry the owned defaults read-only thereafter. + Review all callers of those helpers, including test setup; direct declaration + resolution can read its input because it returns its own concrete policy. +3. Keep per-policy copies of the selected pattern, exclusions, and destination. + Do not borrow caller-owned storage beyond preflight, mutate the shared owned + defaults, or remove copying from unrelated option/record paths. Each new load + snapshots its own current inputs; add no reuse cache or ownership flag. +4. Replace the private conversion-snapshot fragment in + `TestImportDefaultsPreserveEmptyPatternAndRequiredness`, which converts options, + mutates inputs, and calls engine `Load`. It should no longer demand a snapshot + from a private conversion helper. Keep that test's public nil/empty, + requiredness, and inspection-evidence checks. +5. Protect the public snapshot contract with a compact ordinary/layered table: + use a main document with a version and imports, enable + `SourceVersionModeIfPresent`, and use a configuration type with version and + imported-value fields. In the main version-check callback, mutate the caller's + recursive default from true to false and its existing pattern element from + `*.cfg` to `*.skip`. This callback runs after preflight but before imported + discovery. Verify the original recursive/pattern selection still supplies a + value from a nested imported file. Use existing fixture helpers and no + concurrent mutation. A subsequent load with those changed defaults must omit + that imported contribution rather than reuse the prior snapshot. Keep fixtures + reusable or create fresh readers for each load. +6. Reuse or extend the small snapshot/policy tests for all four optional booleans, + explicit false, pattern mutation, nil, and non-nil empty pattern. Preserve + existing invalid-pattern-before-read/hook tests and declaration override, + transitive, underlay, and selected-overlay coverage. Do not assert exact copy + counts, pointer identities, or the private conversion boundary as public policy. + +Acceptance: the engine/facade cutover builds; public defaults remain detached +before acquisition/callbacks, reused loads remain independent, and only genuine +ownership copies remain. Confirm the absence of full defaults snapshots at the +three removed downstream sites by source review, not production instrumentation. + +Focused checks: +`go test ./internal/engine -run 'Import|Glob|Directory|SourceVersion'` and +`go test . -run 'ImportDefaults|FacadeImport|SourceVersion'`. + +## Stage 4 — Construct Placement Directly Across The Facade + +Finding: **F07**, placement handoff. Owners: engine `destination.go`, `load.go`, +`layered.go`, normalized input and placement consumers; root `lifecycle.go`, +`layered.go`, and placement fixtures. Status: **not started**. + +1. Rename the existing engine enum and its constants to `PlacementMode`, + `PlacementModeMapping`, and `PlacementModeSequenceItem`, preserving underlying + `uint8` values and zero semantics. Use the same type throughout engine + placement parsing, normalization, acquisition, source evidence, and version + projection. Do not add a second raw-mode record or type alias. +2. Expose engine `Options[T].Destination` and `.PlacementMode`, and + `Input.PlacementMode`. Normalized `loadInput` and other private owner fields + can retain their names. Populate the normalized input from those fields. + Remove `OptionsWithPlacement` and `InputWithPlacement` and migrate all callers, + including the direct engine construction in root `records_test.go`. +3. In root `toEngineOptions` and `toEngineInput`, set destination using the existing + `toEnginePath` conversion and placement using + `engine.PlacementMode(options.PlacementMode)` or its input equivalent. + Preserve invalid enum values for `normalizeInput` to diagnose before reads. + Do not add validation/defaulting to the facade or a new setter/cast function. +4. Preserve normalization of literal/parsed destination keys, root-item rejection, + wrapper accounting, mounted/item source coordinates, and version projection. + Root `Path` is immutable and `toEnginePath` carries its private engine value; + keep normalization and actual mutable ownership copies. Do not redesign path + conversion or add redundant clones solely to replace the deleted decorator. +5. Leave source-result transport such as `SourceInfoPlacementModeForFacade`, + source-version conversion, sequence registration preparation, and opaque + records alone. They are not input setters targeted by this step. Adapt tests + mechanically to new engine construction without weakening public assertions. + +Acceptance: both public loading APIs retain zero/mapping and sequence-item +placement, errors before input consumption, final values, source metadata, and +history. Reuse the public placement tests plus destination/accounting, +item-import, mounted-version, and invalid-option cases. Root signatures still +contain no engine types, and engine exports remain below the internal boundary. + +Focused checks: +`go test ./internal/engine -run 'Destination|Placement|Item|SourceVersion'` and +`go test . -run 'Placement|Destination|SourceInfo|SourceVersion|PublicFacadeBoundary|DependencyImportBoundary'`. + +## Stage 5 — Integrate, Document, And Verify The Result + +Findings: final acceptance for **C03, F07**. Status: **not started**. + +1. Review both paths together: defaults resolving wildcard discovery in ordinary, + layered, and transitive imports; placement carrying the resulting sources + through composition and inspection; and failures stopping before later hooks. + Existing integration tests own these interactions. Add only a concrete missing + case rather than a second copy of the complete matrix or a new fixture framework. +2. Inspect the resulting handoffs. Confirm the glob selector no longer embeds + per-entry preparation, the engine has one defaults input type, direct fields + replaced the four input decorators, and the load snapshot is not repeated at + each private representation. Keep domain-specific policy/registration and + root conversion boundaries, necessary ownership copies, and existing budgets. +3. Update `docs/internal/imports.md` for the selector/preparation split and + `docs/internal/overview.md` for defaults conversion, preflight snapshot ownership, + and direct placement construction. Keep algorithms private and avoid public + performance or copy-count guarantees. No ADR/policy change is needed. +4. Replace C03/F07 detail in the first two `docs/roadmap/future.md` entries with + short pointers to `internal-boundaries.md`. Retain their heading anchors for + existing audit links; leave other future candidates unchanged. Correct old + roadmap references that would otherwise mistake this replacement plan for + the previous sprint's completed record: use + `bb5540b:docs/roadmap/implementation.md` for that historical plan. Do not reopen + the prior audit or retire unrelated planning records in this sprint. +5. Format changed Go files, finish deferred checks, run the final checks below, + and inspect the complete diff for API, dependency, semantics, or scope changes. + Record actual results and platform limitations here. Commit focused nonempty + units and hand off a clean workspace; do not create empty validation commits. + +Final validation, following the existing development guide: ```sh env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off ./scripts/check.sh env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test -race ./... -env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -shuffle=on -count=3 -env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off \ - go test ./internal/engine -run '^$' -fuzz '^FuzzCompositionAndOverlay$' -fuzztime=30s -parallel=2 -env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./examples/consumers -count=1 -env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off \ - go test . -run 'TestLocalDocumentationLinks|TestDependencyImportBoundary|TestPublicFacadeBoundary' -count=1 git diff --check ``` -The consumer/boundary reruns are focused confirmation of the changed cross-package -surface, not new release prerequisites. Reuse unchanged successful checks when -appropriate under the testing policy. Record actual platform skips and limitations; -no express macOS validation, CI wait, or release publication is part of completion. -All Go and documentation changes must be committed in focused nonempty units, -with a clean workspace and a concise summary of fixed behavior and any remaining -out-of-scope work. Do not create empty commits for validation-only stages. +The everyday check includes consumers, documentation links, and public/dependency +boundary tests. Confirm those existing tests ran rather than adding redundant +final invocations. Report filesystem/platform skips accurately; express macOS +validation, CI completion, release publication, extended fuzz campaigns, and +benchmarks are not additional acceptance requirements. -## Finding-To-Stage Traceability - -| Finding | Owning stages | -| --- | --- | -| A01 | 1 | -| A02 | 2 | -| A03 | 3 | -| A04 | 4 | -| A05 | 14 | -| B01 | 5 | -| B02 | 6 | -| C01 | 7 | -| C02 | 8–9 | -| D01 | 12–13 | -| D02 | 10 | -| D03 | 10 | -| D04 | 11 | -| E01 | 26 | -| E02 | 27–28 | -| E03 | 25, 29, coordinated with 26–28 | -| E04 | 30 | -| E05 | 31 | -| F01 | 20 | -| F02 | 21–24 | -| F03 | 18 | -| F04 | 16, with replacement coverage in 15 | -| F05 | 11 | -| F06 | 1, 17 | -| F08 | 32 | -| F09 | 15 | -| F10 | 19 | -| F11 | 19 | -| F12 | 15–16 | - -Stage 33 verifies all 29 entries. C03 and F07 remain deferred, not missing from -this table. No user/product question remains open; the implementation choices -necessary for this scope are selected above. Helper names/file splits not fixed -by an existing declaration can follow local idiom without changing those choices. - -Status: **complete** (final acceptance for all 29 selected findings). The final -integration also formatted the identity-restoration test identified by the -repository check. Prior final implementation commits: `f132cd6` and `b4af900`. -Validation passed: -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off ./scripts/check.sh`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test -race ./...`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./... -shuffle=on -count=3`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run '^$' -fuzz '^FuzzCompositionAndOverlay$' -fuzztime=30s -parallel=2`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./examples/consumers -count=1`, -`env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test . -run 'TestLocalDocumentationLinks|TestDependencyImportBoundary|TestPublicFacadeBoundary' -count=1`, -and `git diff --check`. No platform skip or release action was required. +No product or architectural question remains open. The concrete choices above +complete the plan without changing the roadmap's public contracts; private +formatting and file organization can follow local idiom. diff --git a/docs/roadmap/internal-boundaries.md b/docs/roadmap/internal-boundaries.md index 149e7d1..3189552 100644 --- a/docs/roadmap/internal-boundaries.md +++ b/docs/roadmap/internal-boundaries.md @@ -1,6 +1,6 @@ # Import Discovery And Facade Boundary Simplification -Status: target defined; staged implementation planning pending. +Status: target defined; execution is tracked in the [implementation plan](implementation.md). Baseline: `80cf23e7102fca2c5d7c7621e75f930a1ef1ecad`. This roadmap brings together the two refactors deferred until after the audit @@ -13,8 +13,8 @@ entries and the [consolidated audit](audit.md). The goal is clearer responsibility and ownership with fewer redundant internal representations. Both changes preserve the public API and configuration model. They are independent work areas within one maintenance effort, not a new loading -workflow. A subsequent implementation plan will own concrete edits, stages, -validation commands, and execution progress. +workflow. The [implementation plan](implementation.md) owns concrete edits, +stages, validation commands, and execution progress. ## Baseline And Targeted End State @@ -145,20 +145,15 @@ The intended benefit is fewer concepts, field mappings, and construction paths to maintain. Reduced copying is useful but does not establish an end-to-end performance guarantee. -## Implementation And Documentation Work +## Documentation Ownership -The later plan should identify the glob extraction boundary and map the defaults -and placement ownership handoffs against current callers before changing them. -The two refactors can be implemented and reviewed independently; neither needs -the other to land. Retire replaced private representations and helpers together -with their callers rather than adding a second production path. - -Update internal documentation when the resulting boundaries land. Public -declaration wording needs changes only if required to keep its existing -ownership description accurate; this effort does not select new public behavior. -Replace the corresponding C03/F07 candidate detail in `future.md` with a pointer -to the active work when the planning records are reorganized, avoiding parallel -target or progress inventories. Leave the unrelated future candidates alone. +Internal documentation describes the resulting discovery and transport +boundaries once implemented. Public declarations continue to own the existing +snapshot and type contracts; this effort selects no new public behavior. +The corresponding C03/F07 future-work entries serve as navigation to this +roadmap rather than parallel target or progress inventories. Unrelated future +candidates remain separate. The implementation plan owns the documentation +updates and retirement of replaced private representations and helpers. ## Validation And Completion