diff --git a/docs/roadmap/consolidation.md b/docs/roadmap/consolidation.md index f74a6d9..40fd35f 100644 --- a/docs/roadmap/consolidation.md +++ b/docs/roadmap/consolidation.md @@ -1,9 +1,9 @@ # ConfigKit Consolidation Toward 1.0 -Status: proposed scope and target state for review. This document recommends -refactors; it does not record implementation progress or authorize a release. -The scope questions are settled. Prepare the staged implementation plan after -reviewing this roadmap. +Status: selected target state; implementation pending. This document defines the +refactor's scope and rationale; it does not authorize a release. The scope +questions are settled. The [implementation plan](implementation.md) owns the +concrete steps, sequencing, and implementation progress. ## Purpose @@ -281,9 +281,9 @@ the value, including retained underlay and inherited values. Use the acquired source's canonical identity where available. Preserve containment failures, source associations, deferred search bases, and terminal selected-file errors. -Implement this work alongside acquisition consolidation so the same callers are -not redesigned twice. The description is private; it does not replace the public -`FilesystemContext` or `ResourceContext` contracts. +Acquisition and context handling use the same private handoff rather than +maintaining parallel models for overlapping callers. The description does not +replace the public `FilesystemContext` or `ResourceContext` contracts. ### 3. Make Inheritance Combination Results Explicit @@ -411,27 +411,6 @@ is otherwise ready. Future performance experiments remain in [future.md](future.md#3-performance-experiments); no benchmark campaign was run or is required to claim readability improvements. -## Recommended Work Order - -This is dependency guidance for the later plan, not a commit-by-commit schedule: - -1. Confirm the implementation baseline and settled scope; map - retained behavior to existing tests and record concrete coverage gaps. -2. Remove unused reverse inspection conversions and adjust their tests. This is - small and independently reviewable. -3. Normalize acquisition inputs and separate import-default ownership. Develop - the resolved-context handoff with those callers, preserving validation and - closure timing; then separate acquisition, item assembly, and evidence work. -4. Refactor inheritance around explicit preparation and combination results, - retaining completed correctness and algorithmic regressions. -5. Refactor projection and schema field assembly in separate reviewable changes. -6. Consolidate real facade adapters, finish consumer/internal navigation, and run - the applicable integration and ownership checks on the combined result. - -Steps 4 and 5 have no necessary implementation dependency on each other. Keep -correctness fixes discovered during refactoring identifiable with their own -regression evidence rather than hiding behavior changes inside code movement. - ## Validation And Completion Use the [development guide](../development.md) and @@ -477,9 +456,9 @@ Completion requires reviewable evidence that: - Consumer guidance, contributor references, and internal documentation describe the resulting implementation. Current contracts continue to own behavior. -Private type/helper names, final file splits, and exact stage boundaries belong -in the implementation plan. No open scope questions remain, and no additional -feature semantics are needed to carry out the recommended refactors. +The [implementation plan](implementation.md) defines private records, file owners, +and stage boundaries. No open scope questions remain, and no additional feature +semantics are needed to carry out the selected refactors. ## Settled Decisions diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..3288283 --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,633 @@ +# Consolidation Implementation Plan + +Status: ready for implementation. Follow Stage 1 through Stage 20 in order. +The target executor is a Luna coding agent working through one end-to-end sprint. + +## Authority, Baseline, And Execution + +The [consolidation roadmap](consolidation.md) owns the target state, rationale, +and settled scope. Start from `8a99e6107001c69117a41dc37bc6ea09d2910af9` plus the +commit introducing this plan. Production code is unchanged from the audited +`5959537d1adc2516e7f7e8881df72b707ec48c3f`; intervening commits consolidated the +planning documents and settled the public-API and clone decisions. + +Read [development.md](../development.md), [agents.md](../policy/agents.md), +[architecture.md](../policy/architecture.md), [testing.md](../policy/testing.md), +and [documentation.md](../policy/documentation.md) as applicable. Use the code +graph for discovery and confirm source at the implementation baseline. Current +public comments and [config.md](../config.md) define behavior; this plan does not +add public obligations. + +Implement only the selected consolidation. Retain the public `Load`, +`LoadLayered`, options, inputs, callbacks, source/search descriptions, inspection, +and sequence-registration APIs. Keep the root-to-engine-to-document/YAML +dependency direction. Clone traversal review/refactoring remains deferred under +[future.md](future.md#4-clone-traversal-review). Changes to shared descriptors must +preserve existing cloning behavior. This sprint does not publish a release, +update dependencies/toolchain requirements, change configuration semantics, or +introduce complexity/performance gates. + +Stages are bounded work assignments, not independently shippable commits. +Intermediate stages may leave callers, tests, or compilation unfinished until a +later stage. Proceed directly through those dependencies; do not build temporary +compatibility layers solely to keep each stage compiling. Update affected tests +with their callers, and run the focused checks below when that portion builds. +Record any deferred verification accurately. The completed sprint must compile, +pass its checks, and leave no superseded plumbing. Commit completed, reviewable +units under the agent policy; stage boundaries do not require separate commits. + +Preserve existing meaningful validation/processing order, diagnostics, ownership, +and limits. Private helper order and representation are not new contracts. Do +not add rejection rules, error recovery, caches, new merge strategies, another +pipeline framework, or tests for incidental helper choreography. A correctness +issue discovered during the refactor needs identifiable regression evidence and +an explicit explanation, rather than an unexplained expectation change. + +## Chosen Private Design + +Use the existing `loadRequest[T]` as the normalized complete-load model and +`loadInput` for each selected input. Keep generic application definitions and +the final lifecycle in `executeLoadRequest[T]`; acquisition is non-generic. +The following records are ordinary engine-private data, not extension points. +Use these names to make migrations concrete; their fields describe the selected +ownership model, not a requirement to duplicate data already available there. + +| Record | Fields and ownership | +| --- | --- | +| `acquisitionPolicy` | Read-only operation snapshot of `importDefaults`, `sequencePolicy`, and `importLimits`. Snapshot caller-owned defaults at entry, preserving nil versus empty. | +| `acquisitionState` | Borrowed pointers to the load's `importAccounting` and `sourceVersionState`. One instance per load; version state no longer transports import defaults. | +| `tierAcquisitionState` | The tier's visited-resource map and shared host namespace token. Underlay groups each own a fresh instance. Existing group graph aggregation and namespace anchors retain their current owners. | +| `inputAcquisitionRequest` | `input loadInput`, source-version role, root-directive context, applicable protection rule, deferred-assembly choice, and `ordinaryOverlay string`. The last field carries only the ordinary main input's overlay-dependent validation/context timing; it is empty for underlays and layered main. | +| `importAcquisitionRequest` | The input request, parsed main source, and already computed source-version/directive processing result. It preserves the distinction between source-local processing and application-path protection. | +| `resolvedFilesystemContext` | `native *nativeNamespace` (nil means host), `hostToken *namespaceToken` where supplied, and `rootPath`, `basePath`, `mainPath` strings. Native facts use slash-relative paths; host facts use the existing canonical host paths. The record contains no open root/file or acquisition access object. | + +The main private entry is +`acquireInput(request inputAcquisitionRequest, policy acquisitionPolicy, state *acquisitionState, tier *tierAcquisitionState) (assembledInput, []Diagnostic)`. +Import acquisition uses the corresponding named request with the same borrowed +policy/state/tier owners. An access-based worker receives a resolved filesystem +description and an explicitly owned `*filesystemAccess`, rather than more +positional paths and independent optional policy parameters. Retain +`assembledInput`, `acquisitionScope`, and `acquiredContribution` as the structural +handoffs; add an optional resolved-context field to `assembledInput` for reuse. + +`importTraversal` borrows the policy and state references. Active routes, stack, +current item scope/prefix/fallback, and the acquired inclusion graph remain +traversal-local. Nested item traversal shares the active graph and tier repetition +state; it does not create another tier. Reusing a file in a later tier remains +valid. Do not collapse namespace identity, lexical candidate spelling, and stable +scheduling keys into one key. + +Context facts are obtained at the existing points where those facts are needed. +There is no global eager context-validation pass or persistent live-handle owner. +Directory checks, canonicalization, handle opening/closing, and passive view +creation remain distinguishable. Reusing a description does not authorize +earlier checks that bypass an existing earlier error or implicit loading activity +after publication. Consumer-initiated passive filesystem operations retain their +existing contract. + +For inheritance, retain both scope adapters and one executor. Use private +prepared-child/prepared-parent records for combination inputs, then an +`inheritanceTemplateResult` containing the resolved value, origins, local policy, +policy templates, and the controls/transfers/replacements/operations produced by +that template. Sequence position evidence lives in the existing operation +records. Merge callbacks mutate that local result, not the enclosing executor's +accumulator. The executor appends the result and applies existing accounting. + +For reflection, keep `valueOperation` and `schemaBuilder` as their existing state +owners. Extract substantial value-kind and inline-field operations as methods; +do not introduce a generic reflection visitor or merge the two schema views. + +## Validation Groups + +Use the installed compatible Go toolchain and already acquired dependencies as +described in the development guide. Commands run from the repository root with +`GOTOOLCHAIN=local` and `GOPROXY=off`. These are development checks for this +refactor, not additions to the release procedure. + +| Group | Commands and purpose | +| --- | --- | +| F: facade cleanup | `go test . -run 'SourceInfo|SourceVersionCheck|ImportSelection|Facade|PublicFacadeBoundary|DependencyImportBoundary'` checks useful forward evidence and the existing public boundary. | +| A: acquisition integration | `go test ./internal/engine -run 'Load|Grouped|Source|Search|Discovery|Import|Glob|Filesystem|Native|Destination|Overlay|Resource|Version|Sequence|Collection'`; `go test . -run 'Facade|EquivalentSourceForms|Search|SourceContext|ImportDefaults|SourceVersion'`; `go test ./examples/consumers`. Covers normalization, ordinary/layered acquisition, selection, contexts, scopes, and consumer workflows. | +| I: inheritance | `go test ./internal/engine -run 'Inheritance|Inherited|Sequence|Collection|Version|History|Explain|Sensitivity|Secret'`; `go test . -run 'Inheritance|Inherited|SequenceIdentity|FinalSequence'`. Covers shared adapters and their evidence. | +| R: reflection | `go test ./internal/engine -run 'Schema|Decode|Projection|Clone|Inline|CompareDocument|Callback|Ownership|Independent|Secret'`. Covers descriptors, projection, shared cloning behavior, and ownership. | +| Final | `./scripts/check.sh` followed by `go test -race ./...`. The first owns formatting, whitespace, ordinary tests/examples/doc links, vet, and build; race validation is appropriate because state and ownership handoffs change throughout this sprint. | + +Run F after Stage 2, A after Stages 9 and 13, I after Stage 16, and R after +Stage 18 when buildable. If earlier migrations still prevent compilation, repair +them at the next integration point and run the deferred group there. Run the +final checks once on the finished code/documentation; repeat only for subsequent +changes affecting their evidence. No benchmark campaign, timing threshold, +traversal-count test, extended fuzz campaign, CI wait, or manual platform check is +required by this plan. Add narrow behavior tests only for consequential gaps +identified below that existing tests do not already cover. + +## Stage 1 — Establish The Starting Map + +**Work:** + +1. Confirm the revision/worktree and preserve unrelated work. Read the roadmap's + settled decisions and the selected owners above. +2. Map the current callers of `selectOptions`, `assembleBaseInput...`, both + `loadImported...` ladders, `contextsForAcquiredSources`, the selected-overlay + loaders, and `applyInheritanceScopeTemplates`. Inspect test callers as well + as production callers; some old entry variants exist mainly for tests. +3. Identify the existing behavioral owners for repeated resources within/across + tiers, enabled/disabled version checking, selected/absent searches, ordinary + versus independent layered overlays, item-local helper scopes, inherited + identity/history, inline fields, and detached records. +4. Keep brief working notes or implementation progress in this plan. Reuse + existing validation evidence where applicable; establish an implicated + baseline only when needed to distinguish a pre-existing failure. Do not + generate another broad audit or blanket characterization-test suite. + +**Finish:** a concrete caller/coverage map and any actual coverage gaps. No +production change or empty commit is needed for orientation alone. + +## Stage 2 — Remove Unused Reverse Bridges And Share Real Facade Conversion + +**Owners:** root `conversion.go`, `records_test.go`, `lifecycle.go`, `layered.go`, +and engine `records.go`. + +1. Reconfirm callers, then delete `toEngineSourceInfo`, `toEngineOrigin`, + `toEngineSourceVersionCheck`, and `toEngineImportSelections`, plus + `SourceInfoWithPlacement`, `SourceInfoWithImportSelections`, + `importSelectorFromString`, and `importSelectionOutcomeFromString` when their + only remaining purpose is the deleted chain. +2. Rewrite the three reverse-round-trip record tests using genuine engine output + and forward conversion/public inspection. Retain placement, version-evidence, + selection-evidence, nil/empty, and detached-copy protection. Engine test + fixtures can use existing loading APIs; do not add production fixture setters. +3. Extract one root `toEngineOverrides` helper and use it from both option + converters. Reuse `toEngineInput`, filesystem, path, definition, and default + conversion where already shared. Preserve slice ownership and nil/empty + behavior; do not make conversion interpret/validate input values. +4. Retain reverse diagnostic conversions used by `WriteDiagnostics`, forward + source/origin inspection conversion, and all public named types. + +**Finish:** the unnecessary cross-package inverse interface is gone and real +conversion is shared without a new transport layer. Run F. + +## Stage 3 — Normalize Preflight Without Fabricated Options + +**Owners:** engine `options.go`, ordinary entry in `load.go`, grouped entry in +`load_request.go`, and their option tests. + +1. Extract non-generic input normalization/validation over `loadInput`, returning + the normalized input plus diagnostics. Pass the ordinary overlay name only + where its context requirements apply. Keep discovery-description validation, + placement checks, normalized destinations, and filesystem option checks at + their existing pre-acquisition boundary. +2. Separate load-wide import-default validation, version-rule/mode validation, + and sequence-registration compilation from per-input checks. Share their + mechanisms between entries without adding a second interpretation of options. +3. Keep the public/engine entry arity handling and each entry's existing error + classification. Ordinary loading still translates its one selected options + value; layered loading validates main, all underlay inputs, and its independent + overlay context in the existing meaningful order. Preserve canonical diagnostic + accumulation and early returns, including invalid options before reads. +4. Replace the grouped entry's constructed `Options[T]` values with direct + normalization of `loadInput`; set the effective protection rule from the + normalized main placement. Leave typed application behavior in the definition. +5. Remove `importDefaultsValidated` and preflight-only forwarding machinery when + callers migrate. Update tests through the entry/helper that owns the behavior, + rather than keeping `selectOptions` solely as a test compatibility surface. + +**Finish:** both entries produce validated `loadRequest[T]` values without using +the ordinary options type to validate grouped inputs. Acquisition may still use +old options until Stage 6; do not broaden this stage into that migration. + +## Stage 4 — Express Input Acquisition With Non-Generic Requests + +**Owners:** new engine `acquisition.go` and existing `load_input.go`. + +1. Define `acquisitionPolicy`, `acquisitionState`, `tierAcquisitionState`, + `inputAcquisitionRequest`, and `importAcquisitionRequest` as selected above. + Reuse existing enums, policy records, and accounting; introduce no new public + controls or duplicate budget counter. +2. Replace the generic `assembleBaseInputTrackedForRole` implementation with + `acquireInput`. Translate its references to source, imports, filesystem, + placement, role, directive context, version rule, policy, and deferred + assembly into explicit request/policy/state fields. +3. Keep its phases visible: search selection; parse and failed-source evidence; + disabled-directive checks; source-local version/directive processing; import + acquisition or single-input placement; accounting/repetition registration; + context handoff. Small helpers may isolate version/directive preparation. +4. Preserve discovery resolution once per input consumption and the acquired + concrete source on both success and failure. Keep mapping diagnostic + destination projection and the ordinary-overlay context deferral unchanged. +5. Retain `assembledInput`'s existing structural/evidence fields. Do not merge + parsed/local payloads with placed documents or eagerly compose deferred input. + +**Finish:** the shared input-acquisition implementation no longer depends on +`Options[T]` or an application type. Call-site and import-worker migration is +completed by Stages 5–6; temporary non-compilation is acceptable. + +## Stage 5 — Consolidate Import Requests And Default Ownership + +**Owners:** engine import entry points, `importTraversal`, `source_version.go`, +and tests that directly invoke the old import variants. + +1. Replace the `loadImportedDocumentsTracked...` ladder with + `acquireImportedInput` using `importAcquisitionRequest` and the shared + policy/state/tier owners. Replace the access-based ladder with one worker + receiving the same request plus explicit access/resolved facts. Keep only + genuinely distinct convenience entries, not one wrapper per optional field. +2. Move `importDefaults` out of `sourceVersionState`. Resolve declarations from + `acquisitionPolicy.importDefaults`, including nested item scopes. Import + defaults must work independently of whether a version rule/state is active. + Preserve enabled version processing and its separate local/effective rules. +3. Make traversal references to aggregate accounting, version evidence, tier + repetition, and immutable policies point to their existing operation owners. + Keep active-route tracking, scope switching, and inclusion edges local to + the traversal. Do not allocate a fresh visited map inside a nested item scope. +4. Preserve nil version-state behavior used by lower-level operations and the + existing default processing branch. Replace optional positional/variadic + policy passing with explicit inputs for meaningful operation distinctions. + Private test entry spellings do not need compatibility wrappers; preserve + their relevant behavioral cases through the resulting operation. +5. Migrate direct import tests to named inputs. Small test-only constructors may + supply default state/policy values when this reduces repetitive fixtures. + Keep their behavior assertions intact. + +**Finish:** the import request surface has named cohesive inputs; defaults no +longer travel through version processing; no forwarding ladder remains merely +to supply another nil/default positional parameter. + +## Stage 6 — Wire Ordinary And Grouped Lifecycle To Acquisition + +**Owners:** engine `load.go`, `load_request.go`, `load_input.go`, related tests. + +1. Construct the load's acquisition policy and mutable state once in + `executeLoadRequest`. Keep the definition and schema/callback lifecycle there. +2. Call `acquireInput` for main using its ordinary/layered directive context, + main version role, current protection rule, and deferred assembly. Preserve + the existing schema-build and version-check timing, including the different + default/enabled source-version paths. +3. Have `acquireUnderlayGroupScopes` use the same load policy/accounting/version + state and one fresh tier state per group. Remove `Options[struct{}]` + construction. Preserve explicit namespace anchors and graph reanchoring. +4. Main acquisition still precedes underlays. Strict assembly still follows + the existing dependency-ready scopes and lower-tier identity information. + Keep group-local repetition separate from load-wide limits, and retain the + existing selected-overlay duplicate-source checks. +5. Replace the ordinary `applyInputLayers` options argument with the precise + main/overlay/protection/policy inputs it uses. A named overlay request will + complete that migration in Stage 9. Remove remaining production + `assembleBaseInput...` variants and the acquisition-only `sequencePolicy`, + `deferAssembly`, and `directiveContext` fields from `Options`. Retain the + placement, destination, and import-default fields used by facade translation. +6. Retain real lower-level test/benchmark capabilities with direct request + construction or test-only adapters. Do not delete meaningful source/limit + cases to make removed helpers disappear. + +**Finish:** no executor or underlay caller reconstructs broad/generic options +for acquisition. The shared path is wired end to end; context and import-flow +extraction continue in later stages. + +## Stage 7 — Separate Resolved Filesystem Facts From Access + +**Owners:** new engine `filesystem_context.go`, `filesystem.go`, +`filesystem_access.go`. + +1. Add `resolvedFilesystemContext` with the fields selected above. Reuse native + namespace/token identity; do not compare arbitrary `fs.FS` values or equate + equal labels with equal namespaces. +2. Extract host and native fact derivation from `importFilesystem` and + `nativeFilesystemAccess`. Prefer a selected `SourceInfo.Path` when it already + supplies canonical source identity. Preserve source-derived file bases, + explicit in-memory roots/bases, native slash paths, and the root-only absent + host default. +3. Keep separate functions for the directory checks and opening access that + currently occur at each operation. A returned description does not mean + every possible check has run. Do not add validation-state flags or an + all-purpose eager resolver with stronger acceptance rules. +4. Adapt the import worker to consume resolved facts and explicitly own its + access until the existing acquisition handoff closes it. Keep close-error + reporting and primary-error precedence unchanged. Native access borrows its + caller filesystem and must not close it. +5. Keep any old derivation adapters only while genuine callers migrate in + Stages 8–9; remove them at the end of that migration. + +**Finish:** common context facts have one private representation while validation +timing and live resource ownership remain explicit. + +## Stage 8 — Carry Context Facts Through Input And Passive Views + +**Owners:** engine `load_input.go`, `source_context.go`, `load_request.go`, and +the ordinary context backfill in `load.go`. + +1. Add the optional resolved-context field to `assembledInput`. Populate it + from import acquisition or the existing non-import context validation point. + When context work is deferred for an ordinary overlay, leave it deferred. +2. Replace the source-kind derivation branches of `contextsForAcquiredSources` + with `contextsForResolvedSources`: consume resolved facts and source records, + construct the existing rooted host/native passive view, and compute each + source's relative base using the appropriate path rules. +3. Share relative-base/containment calculation where applicable; retain each + acquired source's own path and source-ID association. Do not substitute the + first source record for the selected main file or attach all values to main. +4. Migrate normal input acquisition, underlay contexts, and ordinary context + backfill. Resolve only still-missing facts at the same lifecycle point as + before. Retained descriptions/views never capture `filesystemAccess` or + `*os.Root`; the existing passive `ResourceContext` contract stays unchanged. +5. Retain failure evidence, root-only absent host context, discovered native + outcomes, and deferred search-base behavior. Update focused context tests + only where private function signatures changed. + +**Finish:** ordinary/underlay acquisition and passive views use the common facts; +selected-overlay callers are the remaining migration in Stage 9. + +## Stage 9 — Migrate Selected Overlays Without Merging Their Semantics + +**Owners:** engine `selected_overlay.go`, ordinary `applyInputLayers`, and +`assembleLayeredInput`. + +1. Define a private `selectedOverlayRequest` containing the overlay name, + resolved filesystem description, protection rule, sequence policy, and + existing limits/version state. Derive context at the caller's existing + lifecycle point. Pass assembly and source evidence explicitly to the worker. +2. Route ordinary overlays through the main source's context; route layered + overlays through their independent `OverlayFilesystem`. Use named operation + inputs instead of fabricating a `sourceBytes`/empty parsed-main pair merely + to select a context branch. Preserve the actual absent-host root behavior. +3. Return or retain the resolved description used by overlay acquisition and + build overlay passive context from it. Reuse main facts only when that + ordinary operation previously used the same root. Context facts do not share + a live handle between the main import traversal and the later overlay read. +4. Keep the production collection-aware overlay path. Migrate meaningful + lower-level tests to that path with explicit policy/limits, retaining their + budget, version, and failure assertions. Delete the test-only legacy branch + selected by missing variadic policy and its exclusive adapters, rather than + treating optional argument count as a contract. The baseline production entry + points all supply the policy argument. +5. Keep overlay source IDs, duplicate detection, version protection, directory + validation, read/close errors, aggregate accounting, and post-read application + order. Remove superseded context derivation and overlay-forwarding adapters. + +**Finish:** all acquisition/context callers share facts and named inputs while +ordinary and independent layered overlay semantics remain distinct. Run A when +buildable; carry any genuine remaining migration dependencies to the next +integration opportunity instead of adding scaffolding or stopping the sprint. + +## Stage 10 — Give Import Responsibilities Focused Files + +**Owners:** engine `imports.go` and new `import_acquisition.go`, +`import_scope.go`, `import_projection.go`. + +1. Put the request entry/access worker and traversal admission/orchestration in + `import_acquisition.go`. Keep `imports.go` as the small common owner for + limits, accounting/route/traversal data, and shared import diagnostics. +2. Move `composeAcquiredInputWithPolicies`, `composeAcquiredScopeWithPolicy`, + pending-item composition, placement reservation, and their item-scope + assembly helpers into `import_scope.go`. +3. Move nested-origin/policy/evidence projection, acquired source destinations, + version-evidence projection, sequence-prefix helpers, and diagnostic-path + projection into `import_projection.go`. Scheduling and graph identity remain + with their existing graph/scheduler owners. +4. Make the acquisition-to-closed-resources-to-assembly handoff visible. Keep + evidence projection based on existing composition applications/position maps; + file movement does not authorize another merge, full-tree replay, or scan of + every unrelated policy for each record. + +**Finish:** maintainers can locate acquisition, local strict assembly, and evidence +work separately. Keep this stage structural; parse/admission extraction follows. + +## Stage 11 — Extract Import Declaration Parsing Operations + +**Owner:** engine `import_spec.go`. + +1. Retain the outer mapping check, exactly-one-selector check, source-order field + walk, applicability checks, and final selector parsing in + `parseImportDeclarationSpec`. +2. Extract boolean assignment, wildcard-set parsing/validation (shared by + `pattern` and `exclude`), placement parsing, and selector-value parsing into + ordinary private helpers. Reuse existing `parseStringSet`, authored-member + sites, and destination/depth parsers. +3. Preserve unspecified pointers and set flags, especially nil versus empty + pattern lists. Keep error paths/locations tied to the authored selector or + list member. Do not reorder consequential checks or replace the selector + matrix with a configurable parser registry. + +**Finish:** field decoding is readable without changing declaration syntax, +selector rules, default resolution, or diagnostics. Keep behavior tests as the +owner; no helper-by-helper assertion inventory is needed. + +## Stage 12 — Extract Directory Entry Admission And Descent + +**Owner:** engine `import_selection.go`, using existing directory/link helpers. + +1. Keep root normalization/deduplication, root ordering, the explicit traversal + stack, candidate union, sorting, exclusion, and required-empty decision visible + in `selectDirectoryImportCandidates`. +2. Extract coherent entry inspection and descent preparation. Return a small + result identifying skip, admitted file, or directory frame to enter, plus a + diagnostic when appropriate. Keep stack/active-directory mutation with the + traversal so its lifetime is visible; do not invent a general visitor API. +3. Preserve hidden/follow/depth behavior, distinct lexical candidates versus + canonical access identity, resource-entry accounting, root-error handling, + and active-path cycle detection. A later non-active alias remains eligible; + exclusions still do not suppress discovery errors. +4. Keep globs and first-match main-file discovery on their existing mechanisms. + Share a helper only where its existing admission semantics actually coincide. + +**Finish:** substantial entry/descent policy is separated from stack orchestration +without changing the selection algorithm or adding repeated root walks. + +## Stage 13 — Extract Imported-Source Admission From Recursion + +**Owner:** engine `import_acquisition.go` after Stage 10. + +1. Keep `importTraversal.visitPath` as the visible coordinator of containment, + canonical identity, cycle/repetition checks, read/parse, directive/version + processing, placement, and recursive declaration traversal. +2. Extract source admission/read-and-process results into a private + `admittedImport` carrying canonical resource/node identity, source record, + processed payload/declarations, and processing evidence. Return an explicit + optional-missing outcome; do not represent a read failure as absence. +3. Preserve when source records are appended, rolled back for optional missing + input, and retained for failures; preserve source IDs, accounting increments, + version-callback ordering, and inclusion-edge creation on repetition/cycles. +4. Pass the admitted record to mapping/item placement helpers rather than + repeating its payload, declarations, and processing evidence as parallel + arguments to `visitItemSource`. +5. Factor route push/pop and item-scope save/restore only where it makes their + shared lifetime clearer. Keep active graph state across item scopes and + reserve placement nodes once. Item helpers complete before outer placement; + they are strict local peers, not a precedence layer. + +**Finish:** admission decisions and recursion have clear handoffs, with one +read/process path and the original scheduling/evidence algorithms. Run A again +because Stages 10–13 changed the import implementation after the first checkpoint. + +## Stage 14 — Extract Inheritance Child And Parent Preparation + +**Owners:** engine `inheritance_schedule.go`, new `inheritance_prepare.go`, and +existing scope/policy helpers. + +1. Extract preparation of the child's directive-free payload, direct origins, + local policy templates, and optional control/control-origin evidence. +2. Extract parent preparation: clone/project policies, lower them against the + child's claims, exclude parent identity from the inherited payload, and + prepare the separate parent history snapshot where currently required. + Keep effective policy templates and transfer construction available to the + combination result rather than reconstructing them afterward. +3. Use focused prepared-child/prepared-parent records of these related values. + Borrow authored/resolved parent inputs read-only; retain existing clone points + for mutable payloads and history. Keep the current preliminary lower-bound + budget decisions at the executor's existing point before materialization. +4. Keep mapping and identified-sequence adapters, the stable Kahn scheduler, and + the parentless/cancelled path semantics intact. Do not flatten nested directives + before their destination-local evaluation. + +**Finish:** the executor no longer embeds child/control and parent/policy +preparation. Actual recursive combination/evidence moves in Stage 15. + +## Stage 15 — Return One Inheritance Combination Result + +**Owners:** new engine `inheritance_combine.go`, `inheritance_collection.go`, +and the shared executor. + +1. Add `inheritanceTemplateResult` as described above and extract the existing + parent/child combination into a helper receiving prepared inputs and the + application policy. Parentless templates return the same result shape. +2. Keep `mergeRecursiveMappings` and `combineSequence` as the combination + mechanisms. Their callbacks update local origins/policy and local replacement + evidence. Have `applyInheritanceSequenceOrigins` return its replacement slice + instead of taking the whole enclosing scope result; append that slice to the + local template result while retaining its updates to the supplied origin maps. +3. Derive sequence history, transferred origins, retired/replaced policies, + identity preservation, and the transfer operation from that same combination. + Preserve `historyParent` and all policy/identity fields on the existing + `inheritanceOperation`; do not rerun a merge or project values to recover them. +4. Keep sequence validation, diagnostic canonicalization, location assignment, + replacement order, and no-parent cloning behavior at their established + semantic points. Keep child identity absent from inherited-parent history but + present with its own origin in the effective child. + +**Finish:** merge callbacks no longer mutate the executor accumulator. The +resolved template and all evidence travel together for Stage 16 integration. + +## Stage 16 — Finish The Inheritance Executor And Accounting + +**Owner:** engine `inheritance_schedule.go` and focused finalization helpers. + +1. Reduce the executor to preparation, budget precheck, combination, accumulation, + and final rebuild. Append each template's control before its transfer/history + in the same order as before, and update the indexed resolved value/origin/ + policy/template state once per template. +2. Extract retained-history charging for one template's returned operations and + replacements, reusing the same `retainedNodeCounter` across the scope as + before. Preserve pointer-based shared-node accounting, lower-bound checks, + scope-slot overhead, final expansion check, limits, and diagnostics. +3. Extract final rebuilding and scope policy/origin projection where it clarifies + the coordinator. Reuse `newInheritancePolicyIndex` once for scope localization; + do not return to scanning all policy entries for every slot. +4. Confirm both adapters and nested walking consume the same output. Retain + source-version movement, sensitivity, independent snapshots, assembled order, + cancellation, and policy retirement on actual replacement. + +**Finish:** one readable executor coordinates explicit value/policy/evidence +handoffs, with no parallel inheritance implementation. Run I. + +## Stage 17 — Extract Substantial Projection Handlers + +**Owners:** engine `projection.go`, optionally new `projection_collection.go`. + +1. Keep `projectConfiguration`'s schema/type guard and one `valueOperation`. + Keep `project`'s depth check, pointer/dynamic dispatch, and scalar conversion + choices visible; delegate structs, maps, and sequences to named methods. +2. Move struct member/inline flattening, collision checks, mapping-key charging, + and flattened-field/inline-map secret classification into `projectStruct`. + Preserve declaration order and the current decrement for a flattened + container that is not retained in the output. +3. Move deterministic sorted-map traversal into `projectMap`; keep its cycle + entry active until all recursive children complete. Move slice/array + construction into `projectSequence`, preserving nil-slice handling and the + slice-only cycle guard. +4. All recursive children call the same operation's dispatcher. Avoid duplicate + node charging/depth increments or early cycle release introduced by extraction. + Preserve float/text/duration/secret scalar handling and error paths. + +**Finish:** dispatch and substantial handlers are separately readable, using the +original operation state. Do not refactor `clone`, `decodeValue`, or +`valueBuilder.build` as an additional cleanup project. + +## Stage 18 — Separate Schema Field Discovery And Assembly + +**Owner:** engine `schema.go`, optionally new `schema_fields.go`. + +1. Keep `describeFields` as declaration-order enumeration, exported/ignored/tag + decisions, and dispatch to ordinary or inline handling. +2. Extract inline pointer-chain inspection and inline-struct/map assembly, + retaining recursion/building-state checks and the existing codec-versus- + structural distinction. Reuse the one builder/cache. +3. Make ordinary registration and inline flattening visibly maintain both + `members` and flattened `fields`/`fieldByName`/`inlineMap`. Preserve index + paths, nested secret propagation, ambiguity paths, and the single-inline-map + rule. Share registration helpers only for genuinely identical updates. +4. Preserve descriptor reuse and the physical-member view used by typed cloning. + Do not unify these views or add alternate field/tag policy. Existing schema, + projection, decoding, and cloning expectations remain the behavior baseline. + +**Finish:** field discovery and descriptor assembly have explicit responsibilities +without a new registration framework. Run R for both Stages 17–18. + +## Stage 19 — Update Consumer And Internal Navigation + +**Owners:** `docs/consumers/README.md`, `docs/internal/overview.md`, +`docs/internal/imports.md`, and affected links/test-owner entries in +`docs/development.md`. + +1. Add a concise consumer choice guide: ordinary `Load` for one selected source, + explicit discovery as a source usable by either entry, and `LoadLayered` for + underlay groups. Distinguish input-owned placement/import/filesystem choices + from load-wide definition/default/version/override choices and independent + layered overlay context. Link current API/config owners for exact rules. +2. Link the existing simple README example, bundled-file/layered examples, + import-focused prompt catalog, and discovered sequence catalog. Keep the + README's simple path simple; do not duplicate complete examples or rewrite + consumer domain rules to showcase new internal types. +3. Update implementation ownership and flow for normalized requests, acquisition + policy/state/tier lifetimes, resolved context versus live handles, import file + owners, and explicit inheritance/projection/schema handoffs. Fix moved-symbol + and behavioral-test navigation and stale comments affected by the refactor. +4. Preserve public contract definitions and accepted ADR decisions. Keep deferred + clone work in `future.md`. No policy change or new ADR is needed for the + selected private refactor. + +**Finish:** current documentation describes implemented structure, with the +roadmap/plan retaining future work and progress until release cleanup. + +## Stage 20 — Verify And Close The Combined Sprint + +1. Search for obsolete import/input forwarding ladders, synthetic `Options` + acquisition construction, `sourceVersionState.importDefaults`, discarded + context derivation branches, unused inverse bridges, and temporary scaffolds. + Remove leftovers along with exclusive test scaffolding. Preserve distinct + operations and real diagnostic conversions; do not delete private code solely + because it resembles another helper. +2. Review the combined call flow: one normalization model, non-generic acquisition, + load versus tier versus traversal state, library-owned acquisition handles + closed before composition/publication, one combination outcome for inheritance + evidence, and one state owner per reflective operation. Review for added whole-tree + traversals, policy scans, or ownership copies without requiring measurements. +3. Account for every selected finding using the table below. Confirm any new + handoff coverage addresses a concrete gap. Existing correctness/algorithmic + regressions, public-boundary tests, and meaningful assertions stay effective. +4. Run the Final validation group, resolve failures, inspect the staged diff and + workspace, and commit the completed remaining work with plain-English + messages. Report changes, checks, limitations, and any unrelated work retained. + Do not tag, push, publish, or update consumer repositories in this sprint. + +| Audit finding | Completion evidence | +| --- | --- | +| 1: imports | Stages 3–6 and 10–13 remove options reconstruction/default coupling and forwarding ladders; acquisition/scope/evidence owners and admission operations are clear. A and final checks pass. | +| 2: inheritance | Stages 14–16 return explicit template results, keep one executor, and preserve accounting and value/history/identity agreement. I and final checks pass. | +| 3: projection | Stage 17 extracts substantial handlers sharing operation state. R and final checks pass. Clone review remains deferred, not implicitly completed. | +| 4: schema | Stage 18 clarifies ordinary/inline assembly while preserving both descriptor views. R and final checks pass. | +| 5: contexts | Stages 7–9 share resolved facts across input/import/overlay/passive contexts without retaining handles or changing validation timing. A and final checks pass. | +| 6: facade | Stage 2 removes unused inverse conversions/adapters and shares real conversion. F and final boundary/consumer checks pass. | + +**Finish:** the selected roadmap target is implemented, validated, documented, +and reviewable as one complete sprint. No unresolved product/scope decisions are +needed to begin these stages.