Files
ConfigKit/docs/roadmap/implementation.md

670 lines
42 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Consolidation Implementation Plan
Status: in progress. Stages 1 and 2 are complete; follow Stage 3 through Stage 20 in order.
The target executor is a Luna coding agent working through one end-to-end sprint.
## Authority, Baseline, And Execution
The [consolidation roadmap](consolidation.md) owns the target state, rationale,
and settled scope. Start from `8a99e6107001c69117a41dc37bc6ea09d2910af9` plus the
commit introducing this plan. Production code is unchanged from the audited
`5959537d1adc2516e7f7e8881df72b707ec48c3f`; intervening commits consolidated the
planning documents and settled the public-API and clone decisions.
Read [development.md](../development.md), [agents.md](../policy/agents.md),
[architecture.md](../policy/architecture.md), [testing.md](../policy/testing.md),
and [documentation.md](../policy/documentation.md) as applicable. Use the code
graph for discovery and confirm source at the implementation baseline. Current
public comments and [config.md](../config.md) define behavior; this plan does not
add public obligations.
Implement only the selected consolidation. Retain the public `Load`,
`LoadLayered`, options, inputs, callbacks, source/search descriptions, inspection,
and sequence-registration APIs. Keep the root-to-engine-to-document/YAML
dependency direction. Clone traversal review/refactoring remains deferred under
[future.md](future.md#4-clone-traversal-review). Changes to shared descriptors must
preserve existing cloning behavior. This sprint does not publish a release,
update dependencies/toolchain requirements, change configuration semantics, or
introduce complexity/performance gates.
Stages are bounded work assignments, not independently shippable commits.
Intermediate stages may leave callers, tests, or compilation unfinished until a
later stage. Proceed directly through those dependencies; do not build temporary
compatibility layers solely to keep each stage compiling. Update affected tests
with their callers, and run the focused checks below when that portion builds.
Record any deferred verification accurately. The completed sprint must compile,
pass its checks, and leave no superseded plumbing. Commit completed, reviewable
units under the agent policy; stage boundaries do not require separate commits.
Preserve existing meaningful validation/processing order, diagnostics, ownership,
and limits. Private helper order and representation are not new contracts. Do
not add rejection rules, error recovery, caches, new merge strategies, another
pipeline framework, or tests for incidental helper choreography. A correctness
issue discovered during the refactor needs identifiable regression evidence and
an explicit explanation, rather than an unexplained expectation change.
## Chosen Private Design
Use the existing `loadRequest[T]` as the normalized complete-load model and
`loadInput` for each selected input. Keep generic application definitions and
the final lifecycle in `executeLoadRequest[T]`; acquisition is non-generic.
The following records are ordinary engine-private data, not extension points.
Use these names to make migrations concrete; their fields describe the selected
ownership model, not a requirement to duplicate data already available there.
| Record | Fields and ownership |
| --- | --- |
| `acquisitionPolicy` | Read-only operation snapshot of `importDefaults`, `sequencePolicy`, and `importLimits`. Snapshot caller-owned defaults at entry, preserving nil versus empty. |
| `acquisitionState` | Borrowed pointers to the load's `importAccounting` and `sourceVersionState`. One instance per load; version state no longer transports import defaults. |
| `tierAcquisitionState` | The tier's visited-resource map and shared host namespace token. Underlay groups each own a fresh instance. Existing group graph aggregation and namespace anchors retain their current owners. |
| `inputAcquisitionRequest` | `input loadInput`, source-version role, root-directive context, applicable protection rule, deferred-assembly choice, and `ordinaryOverlay string`. The last field carries only the ordinary main input's overlay-dependent validation/context timing; it is empty for underlays and layered main. |
| `importAcquisitionRequest` | The input request, parsed main source, and already computed source-version/directive processing result. It preserves the distinction between source-local processing and application-path protection. |
| `resolvedFilesystemContext` | `native *nativeNamespace` (nil means host), `hostToken *namespaceToken` where supplied, and `rootPath`, `basePath`, `mainPath` strings. Native facts use slash-relative paths; host facts use the existing canonical host paths. The record contains no open root/file or acquisition access object. |
The main private entry is
`acquireInput(request inputAcquisitionRequest, policy acquisitionPolicy, state *acquisitionState, tier *tierAcquisitionState) (assembledInput, []Diagnostic)`.
Import acquisition uses the corresponding named request with the same borrowed
policy/state/tier owners. An access-based worker receives a resolved filesystem
description and an explicitly owned `*filesystemAccess`, rather than more
positional paths and independent optional policy parameters. Retain
`assembledInput`, `acquisitionScope`, and `acquiredContribution` as the structural
handoffs; add an optional resolved-context field to `assembledInput` for reuse.
`importTraversal` borrows the policy and state references. Active routes, stack,
current item scope/prefix/fallback, and the acquired inclusion graph remain
traversal-local. Nested item traversal shares the active graph and tier repetition
state; it does not create another tier. Reusing a file in a later tier remains
valid. Do not collapse namespace identity, lexical candidate spelling, and stable
scheduling keys into one key.
Context facts are obtained at the existing points where those facts are needed.
There is no global eager context-validation pass or persistent live-handle owner.
Directory checks, canonicalization, handle opening/closing, and passive view
creation remain distinguishable. Reusing a description does not authorize
earlier checks that bypass an existing earlier error or implicit loading activity
after publication. Consumer-initiated passive filesystem operations retain their
existing contract.
For inheritance, retain both scope adapters and one executor. Use private
prepared-child/prepared-parent records for combination inputs, then an
`inheritanceTemplateResult` containing the resolved value, origins, local policy,
policy templates, and the controls/transfers/replacements/operations produced by
that template. Sequence position evidence lives in the existing operation
records. Merge callbacks mutate that local result, not the enclosing executor's
accumulator. The executor appends the result and applies existing accounting.
For reflection, keep `valueOperation` and `schemaBuilder` as their existing state
owners. Extract substantial value-kind and inline-field operations as methods;
do not introduce a generic reflection visitor or merge the two schema views.
## Validation Groups
Use the installed compatible Go toolchain and already acquired dependencies as
described in the development guide. Commands run from the repository root with
`GOTOOLCHAIN=local` and `GOPROXY=off`. These are development checks for this
refactor, not additions to the release procedure.
| Group | Commands and purpose |
| --- | --- |
| F: facade cleanup | `go test . -run 'SourceInfo|SourceVersionCheck|ImportSelection|Facade|PublicFacadeBoundary|DependencyImportBoundary'` checks useful forward evidence and the existing public boundary. |
| A: acquisition integration | `go test ./internal/engine -run 'Load|Grouped|Source|Search|Discovery|Import|Glob|Filesystem|Native|Destination|Overlay|Resource|Version|Sequence|Collection'`; `go test . -run 'Facade|EquivalentSourceForms|Search|SourceContext|ImportDefaults|SourceVersion'`; `go test ./examples/consumers`. Covers normalization, ordinary/layered acquisition, selection, contexts, scopes, and consumer workflows. |
| I: inheritance | `go test ./internal/engine -run 'Inheritance|Inherited|Sequence|Collection|Version|History|Explain|Sensitivity|Secret'`; `go test . -run 'Inheritance|Inherited|SequenceIdentity|FinalSequence'`. Covers shared adapters and their evidence. |
| R: reflection | `go test ./internal/engine -run 'Schema|Decode|Projection|Clone|Inline|CompareDocument|Callback|Ownership|Independent|Secret'`. Covers descriptors, projection, shared cloning behavior, and ownership. |
| Final | `./scripts/check.sh` followed by `go test -race ./...`. The first owns formatting, whitespace, ordinary tests/examples/doc links, vet, and build; race validation is appropriate because state and ownership handoffs change throughout this sprint. |
Run F after Stage 2, A after Stages 9 and 13, I after Stage 16, and R after
Stage 18 when buildable. If earlier migrations still prevent compilation, repair
them at the next integration point and run the deferred group there. Run the
final checks once on the finished code/documentation; repeat only for subsequent
changes affecting their evidence. No benchmark campaign, timing threshold,
traversal-count test, extended fuzz campaign, CI wait, or manual platform check is
required by this plan. Add narrow behavior tests only for consequential gaps
identified below that existing tests do not already cover.
## Stage 1 — Establish The Starting Map (Complete)
**Work:**
1. Confirm the revision/worktree and preserve unrelated work. Read the roadmap's
settled decisions and the selected owners above.
2. Map the current callers of `selectOptions`, `assembleBaseInput...`, both
`loadImported...` ladders, `contextsForAcquiredSources`, the selected-overlay
loaders, and `applyInheritanceScopeTemplates`. Inspect test callers as well
as production callers; some old entry variants exist mainly for tests.
3. Identify the existing behavioral owners for repeated resources within/across
tiers, enabled/disabled version checking, selected/absent searches, ordinary
versus independent layered overlays, item-local helper scopes, inherited
identity/history, inline fields, and detached records.
4. Keep brief working notes or implementation progress in this plan. Reuse
existing validation evidence where applicable; establish an implicated
baseline only when needed to distinguish a pre-existing failure. Do not
generate another broad audit or blanket characterization-test suite.
**Finish:** a concrete caller/coverage map and any actual coverage gaps. No
production change or empty commit is needed for orientation alone.
Progress notes (2026-10-25): the worktree was clean at `8ab1568`, immediately
after the planning baseline `8a99e61`; the audited production baseline
`5959537` is present. `selectOptions` is called by the ordinary and grouped
load entry points. The `assembleBaseInput...` wrappers converge on
`assembleBaseInputTrackedForRole`, reached from ordinary loading, layered main
loading, and underlay-group acquisition. The tracked import wrappers converge
on `loadImportedDocumentsTrackedWithProcessing`; production callers are input
acquisition and underlay-group acquisition, while older convenience variants
also have direct test callers. `contextsForAcquiredSources` is shared by ordinary
loading, layered assembly, input acquisition, and overlay application. Selected
overlay loading is reached from ordinary overlay application and layered input
assembly. Both inheritance adapters, for ordinary and identified sequences,
feed `applyInheritanceScopeTemplates`.
Existing behavioral owners cover repeated/cyclic resources and cross-tier reuse
(`internal/engine/imports_test.go`, `facade_equivalence_test.go`); default and
enabled source-version modes (`internal/engine/source_version_test.go`,
`facade_source_version_mode_test.go`); selected/absent discovery
(`internal/engine/discovery_load_test.go`, `facade_discovery_test.go`); ordinary
and layered overlays (`internal/engine/selected_overlay_test.go`,
`facade_layered_test.go`); item-local helper scopes (`internal/engine/imports_test.go`,
`facade_import_lifecycle_test.go`); inheritance identity/history
(`internal/engine/inheritance_identity_history_test.go`,
`internal/engine/inheritance_history_test.go`); inline fields
(`internal/engine/schema_test.go`, `internal/engine/projection_test.go`); and
detached results and callback ownership (`internal/engine/metadata_test.go`,
`internal/engine/callback_ownership_test.go`, `facade_diff_test.go`). No specific
coverage gap surfaced in this map; later focused checks remain assigned in the
validation groups above.
## Stage 2 — Remove Unused Reverse Bridges And Share Real Facade Conversion (Complete)
**Owners:** root `conversion.go`, `records_test.go`, `lifecycle.go`, `layered.go`,
and engine `records.go`.
1. Reconfirm callers, then delete `toEngineSourceInfo`, `toEngineOrigin`,
`toEngineSourceVersionCheck`, and `toEngineImportSelections`, plus
`SourceInfoWithPlacement`, `SourceInfoWithImportSelections`,
`importSelectorFromString`, and `importSelectionOutcomeFromString` when their
only remaining purpose is the deleted chain.
2. Rewrite the three reverse-round-trip record tests using genuine engine output
and forward conversion/public inspection. Retain placement, version-evidence,
selection-evidence, nil/empty, and detached-copy protection. Engine test
fixtures can use existing loading APIs; do not add production fixture setters.
3. Extract one root `toEngineOverrides` helper and use it from both option
converters. Reuse `toEngineInput`, filesystem, path, definition, and default
conversion where already shared. Preserve slice ownership and nil/empty
behavior; do not make conversion interpret/validate input values.
4. Retain reverse diagnostic conversions used by `WriteDiagnostics`, forward
source/origin inspection conversion, and all public named types.
**Finish:** the unnecessary cross-package inverse interface is gone and real
conversion is shared without a new transport layer. Run F.
Progress notes (2026-10-25): removed the unused reverse source-record bridge
and engine setters, replaced synthetic round-trip fixtures with engine loads,
and shared override conversion across ordinary and layered options. Validation
group F passed with `GOTOOLCHAIN=local GOPROXY=off go test . -run
'SourceInfo|SourceVersionCheck|ImportSelection|Facade|PublicFacadeBoundary|DependencyImportBoundary' -count=1`.
## Stage 3 — Normalize Preflight Without Fabricated Options
**Owners:** engine `options.go`, ordinary entry in `load.go`, grouped entry in
`load_request.go`, and their option tests.
1. Extract non-generic input normalization/validation over `loadInput`, returning
the normalized input plus diagnostics. Pass the ordinary overlay name only
where its context requirements apply. Keep discovery-description validation,
placement checks, normalized destinations, and filesystem option checks at
their existing pre-acquisition boundary.
2. Separate load-wide import-default validation, version-rule/mode validation,
and sequence-registration compilation from per-input checks. Share their
mechanisms between entries without adding a second interpretation of options.
3. Keep the public/engine entry arity handling and each entry's existing error
classification. Ordinary loading still translates its one selected options
value; layered loading validates main, all underlay inputs, and its independent
overlay context in the existing meaningful order. Preserve canonical diagnostic
accumulation and early returns, including invalid options before reads.
4. Replace the grouped entry's constructed `Options[T]` values with direct
normalization of `loadInput`; set the effective protection rule from the
normalized main placement. Leave typed application behavior in the definition.
5. Remove `importDefaultsValidated` and preflight-only forwarding machinery when
callers migrate. Update tests through the entry/helper that owns the behavior,
rather than keeping `selectOptions` solely as a test compatibility surface.
**Finish:** both entries produce validated `loadRequest[T]` values without using
the ordinary options type to validate grouped inputs. Acquisition may still use
old options until Stage 6; do not broaden this stage into that migration.
## Stage 4 — Express Input Acquisition With Non-Generic Requests
**Owners:** new engine `acquisition.go` and existing `load_input.go`.
1. Define `acquisitionPolicy`, `acquisitionState`, `tierAcquisitionState`,
`inputAcquisitionRequest`, and `importAcquisitionRequest` as selected above.
Reuse existing enums, policy records, and accounting; introduce no new public
controls or duplicate budget counter.
2. Replace the generic `assembleBaseInputTrackedForRole` implementation with
`acquireInput`. Translate its references to source, imports, filesystem,
placement, role, directive context, version rule, policy, and deferred
assembly into explicit request/policy/state fields.
3. Keep its phases visible: search selection; parse and failed-source evidence;
disabled-directive checks; source-local version/directive processing; import
acquisition or single-input placement; accounting/repetition registration;
context handoff. Small helpers may isolate version/directive preparation.
4. Preserve discovery resolution once per input consumption and the acquired
concrete source on both success and failure. Keep mapping diagnostic
destination projection and the ordinary-overlay context deferral unchanged.
5. Retain `assembledInput`'s existing structural/evidence fields. Do not merge
parsed/local payloads with placed documents or eagerly compose deferred input.
**Finish:** the shared input-acquisition implementation no longer depends on
`Options[T]` or an application type. Call-site and import-worker migration is
completed by Stages 5–6; temporary non-compilation is acceptable.
## Stage 5 — Consolidate Import Requests And Default Ownership
**Owners:** engine import entry points, `importTraversal`, `source_version.go`,
and tests that directly invoke the old import variants.
1. Replace the `loadImportedDocumentsTracked...` ladder with
`acquireImportedInput` using `importAcquisitionRequest` and the shared
policy/state/tier owners. Replace the access-based ladder with one worker
receiving the same request plus explicit access/resolved facts. Keep only
genuinely distinct convenience entries, not one wrapper per optional field.
2. Move `importDefaults` out of `sourceVersionState`. Resolve declarations from
`acquisitionPolicy.importDefaults`, including nested item scopes. Import
defaults must work independently of whether a version rule/state is active.
Preserve enabled version processing and its separate local/effective rules.
3. Make traversal references to aggregate accounting, version evidence, tier
repetition, and immutable policies point to their existing operation owners.
Keep active-route tracking, scope switching, and inclusion edges local to
the traversal. Do not allocate a fresh visited map inside a nested item scope.
4. Preserve nil version-state behavior used by lower-level operations and the
existing default processing branch. Replace optional positional/variadic
policy passing with explicit inputs for meaningful operation distinctions.
Private test entry spellings do not need compatibility wrappers; preserve
their relevant behavioral cases through the resulting operation.
5. Migrate direct import tests to named inputs. Small test-only constructors may
supply default state/policy values when this reduces repetitive fixtures.
Keep their behavior assertions intact.
**Finish:** the import request surface has named cohesive inputs; defaults no
longer travel through version processing; no forwarding ladder remains merely
to supply another nil/default positional parameter.
## Stage 6 — Wire Ordinary And Grouped Lifecycle To Acquisition
**Owners:** engine `load.go`, `load_request.go`, `load_input.go`, related tests.
1. Construct the load's acquisition policy and mutable state once in
`executeLoadRequest`. Keep the definition and schema/callback lifecycle there.
2. Call `acquireInput` for main using its ordinary/layered directive context,
main version role, current protection rule, and deferred assembly. Preserve
the existing schema-build and version-check timing, including the different
default/enabled source-version paths.
3. Have `acquireUnderlayGroupScopes` use the same load policy/accounting/version
state and one fresh tier state per group. Remove `Options[struct{}]`
construction. Preserve explicit namespace anchors and graph reanchoring.
4. Main acquisition still precedes underlays. Strict assembly still follows
the existing dependency-ready scopes and lower-tier identity information.
Keep group-local repetition separate from load-wide limits, and retain the
existing selected-overlay duplicate-source checks.
5. Replace the ordinary `applyInputLayers` options argument with the precise
main/overlay/protection/policy inputs it uses. A named overlay request will
complete that migration in Stage 9. Remove remaining production
`assembleBaseInput...` variants and the acquisition-only `sequencePolicy`,
`deferAssembly`, and `directiveContext` fields from `Options`. Retain the
placement, destination, and import-default fields used by facade translation.
6. Retain real lower-level test/benchmark capabilities with direct request
construction or test-only adapters. Do not delete meaningful source/limit
cases to make removed helpers disappear.
**Finish:** no executor or underlay caller reconstructs broad/generic options
for acquisition. The shared path is wired end to end; context and import-flow
extraction continue in later stages.
## Stage 7 — Separate Resolved Filesystem Facts From Access
**Owners:** new engine `filesystem_context.go`, `filesystem.go`,
`filesystem_access.go`.
1. Add `resolvedFilesystemContext` with the fields selected above. Reuse native
namespace/token identity; do not compare arbitrary `fs.FS` values or equate
equal labels with equal namespaces.
2. Extract host and native fact derivation from `importFilesystem` and
`nativeFilesystemAccess`. Prefer a selected `SourceInfo.Path` when it already
supplies canonical source identity. Preserve source-derived file bases,
explicit in-memory roots/bases, native slash paths, and the root-only absent
host default.
3. Keep separate functions for the directory checks and opening access that
currently occur at each operation. A returned description does not mean
every possible check has run. Do not add validation-state flags or an
all-purpose eager resolver with stronger acceptance rules.
4. Adapt the import worker to consume resolved facts and explicitly own its
access until the existing acquisition handoff closes it. Keep close-error
reporting and primary-error precedence unchanged. Native access borrows its
caller filesystem and must not close it.
5. Keep any old derivation adapters only while genuine callers migrate in
Stages 8–9; remove them at the end of that migration.
**Finish:** common context facts have one private representation while validation
timing and live resource ownership remain explicit.
## Stage 8 — Carry Context Facts Through Input And Passive Views
**Owners:** engine `load_input.go`, `source_context.go`, `load_request.go`, and
the ordinary context backfill in `load.go`.
1. Add the optional resolved-context field to `assembledInput`. Populate it
from import acquisition or the existing non-import context validation point.
When context work is deferred for an ordinary overlay, leave it deferred.
2. Replace the source-kind derivation branches of `contextsForAcquiredSources`
with `contextsForResolvedSources`: consume resolved facts and source records,
construct the existing rooted host/native passive view, and compute each
source's relative base using the appropriate path rules.
3. Share relative-base/containment calculation where applicable; retain each
acquired source's own path and source-ID association. Do not substitute the
first source record for the selected main file or attach all values to main.
4. Migrate normal input acquisition, underlay contexts, and ordinary context
backfill. Resolve only still-missing facts at the same lifecycle point as
before. Retained descriptions/views never capture `filesystemAccess` or
`*os.Root`; the existing passive `ResourceContext` contract stays unchanged.
5. Retain failure evidence, root-only absent host context, discovered native
outcomes, and deferred search-base behavior. Update focused context tests
only where private function signatures changed.
**Finish:** ordinary/underlay acquisition and passive views use the common facts;
selected-overlay callers are the remaining migration in Stage 9.
## Stage 9 — Migrate Selected Overlays Without Merging Their Semantics
**Owners:** engine `selected_overlay.go`, ordinary `applyInputLayers`, and
`assembleLayeredInput`.
1. Define a private `selectedOverlayRequest` containing the overlay name,
resolved filesystem description, protection rule, sequence policy, and
existing limits/version state. Derive context at the caller's existing
lifecycle point. Pass assembly and source evidence explicitly to the worker.
2. Route ordinary overlays through the main source's context; route layered
overlays through their independent `OverlayFilesystem`. Use named operation
inputs instead of fabricating a `sourceBytes`/empty parsed-main pair merely
to select a context branch. Preserve the actual absent-host root behavior.
3. Return or retain the resolved description used by overlay acquisition and
build overlay passive context from it. Reuse main facts only when that
ordinary operation previously used the same root. Context facts do not share
a live handle between the main import traversal and the later overlay read.
4. Keep the production collection-aware overlay path. Migrate meaningful
lower-level tests to that path with explicit policy/limits, retaining their
budget, version, and failure assertions. Delete the test-only legacy branch
selected by missing variadic policy and its exclusive adapters, rather than
treating optional argument count as a contract. The baseline production entry
points all supply the policy argument.
5. Keep overlay source IDs, duplicate detection, version protection, directory
validation, read/close errors, aggregate accounting, and post-read application
order. Remove superseded context derivation and overlay-forwarding adapters.
**Finish:** all acquisition/context callers share facts and named inputs while
ordinary and independent layered overlay semantics remain distinct. Run A when
buildable; carry any genuine remaining migration dependencies to the next
integration opportunity instead of adding scaffolding or stopping the sprint.
## Stage 10 — Give Import Responsibilities Focused Files
**Owners:** engine `imports.go` and new `import_acquisition.go`,
`import_scope.go`, `import_projection.go`.
1. Put the request entry/access worker and traversal admission/orchestration in
`import_acquisition.go`. Keep `imports.go` as the small common owner for
limits, accounting/route/traversal data, and shared import diagnostics.
2. Move `composeAcquiredInputWithPolicies`, `composeAcquiredScopeWithPolicy`,
pending-item composition, placement reservation, and their item-scope
assembly helpers into `import_scope.go`.
3. Move nested-origin/policy/evidence projection, acquired source destinations,
version-evidence projection, sequence-prefix helpers, and diagnostic-path
projection into `import_projection.go`. Scheduling and graph identity remain
with their existing graph/scheduler owners.
4. Make the acquisition-to-closed-resources-to-assembly handoff visible. Keep
evidence projection based on existing composition applications/position maps;
file movement does not authorize another merge, full-tree replay, or scan of
every unrelated policy for each record.
**Finish:** maintainers can locate acquisition, local strict assembly, and evidence
work separately. Keep this stage structural; parse/admission extraction follows.
## Stage 11 — Extract Import Declaration Parsing Operations
**Owner:** engine `import_spec.go`.
1. Retain the outer mapping check, exactly-one-selector check, source-order field
walk, applicability checks, and final selector parsing in
`parseImportDeclarationSpec`.
2. Extract boolean assignment, wildcard-set parsing/validation (shared by
`pattern` and `exclude`), placement parsing, and selector-value parsing into
ordinary private helpers. Reuse existing `parseStringSet`, authored-member
sites, and destination/depth parsers.
3. Preserve unspecified pointers and set flags, especially nil versus empty
pattern lists. Keep error paths/locations tied to the authored selector or
list member. Do not reorder consequential checks or replace the selector
matrix with a configurable parser registry.
**Finish:** field decoding is readable without changing declaration syntax,
selector rules, default resolution, or diagnostics. Keep behavior tests as the
owner; no helper-by-helper assertion inventory is needed.
## Stage 12 — Extract Directory Entry Admission And Descent
**Owner:** engine `import_selection.go`, using existing directory/link helpers.
1. Keep root normalization/deduplication, root ordering, the explicit traversal
stack, candidate union, sorting, exclusion, and required-empty decision visible
in `selectDirectoryImportCandidates`.
2. Extract coherent entry inspection and descent preparation. Return a small
result identifying skip, admitted file, or directory frame to enter, plus a
diagnostic when appropriate. Keep stack/active-directory mutation with the
traversal so its lifetime is visible; do not invent a general visitor API.
3. Preserve hidden/follow/depth behavior, distinct lexical candidates versus
canonical access identity, resource-entry accounting, root-error handling,
and active-path cycle detection. A later non-active alias remains eligible;
exclusions still do not suppress discovery errors.
4. Keep globs and first-match main-file discovery on their existing mechanisms.
Share a helper only where its existing admission semantics actually coincide.
**Finish:** substantial entry/descent policy is separated from stack orchestration
without changing the selection algorithm or adding repeated root walks.
## Stage 13 — Extract Imported-Source Admission From Recursion
**Owner:** engine `import_acquisition.go` after Stage 10.
1. Keep `importTraversal.visitPath` as the visible coordinator of containment,
canonical identity, cycle/repetition checks, read/parse, directive/version
processing, placement, and recursive declaration traversal.
2. Extract source admission/read-and-process results into a private
`admittedImport` carrying canonical resource/node identity, source record,
processed payload/declarations, and processing evidence. Return an explicit
optional-missing outcome; do not represent a read failure as absence.
3. Preserve when source records are appended, rolled back for optional missing
input, and retained for failures; preserve source IDs, accounting increments,
version-callback ordering, and inclusion-edge creation on repetition/cycles.
4. Pass the admitted record to mapping/item placement helpers rather than
repeating its payload, declarations, and processing evidence as parallel
arguments to `visitItemSource`.
5. Factor route push/pop and item-scope save/restore only where it makes their
shared lifetime clearer. Keep active graph state across item scopes and
reserve placement nodes once. Item helpers complete before outer placement;
they are strict local peers, not a precedence layer.
**Finish:** admission decisions and recursion have clear handoffs, with one
read/process path and the original scheduling/evidence algorithms. Run A again
because Stages 10–13 changed the import implementation after the first checkpoint.
## Stage 14 — Extract Inheritance Child And Parent Preparation
**Owners:** engine `inheritance_schedule.go`, new `inheritance_prepare.go`, and
existing scope/policy helpers.
1. Extract preparation of the child's directive-free payload, direct origins,
local policy templates, and optional control/control-origin evidence.
2. Extract parent preparation: clone/project policies, lower them against the
child's claims, exclude parent identity from the inherited payload, and
prepare the separate parent history snapshot where currently required.
Keep effective policy templates and transfer construction available to the
combination result rather than reconstructing them afterward.
3. Use focused prepared-child/prepared-parent records of these related values.
Borrow authored/resolved parent inputs read-only; retain existing clone points
for mutable payloads and history. Keep the current preliminary lower-bound
budget decisions at the executor's existing point before materialization.
4. Keep mapping and identified-sequence adapters, the stable Kahn scheduler, and
the parentless/cancelled path semantics intact. Do not flatten nested directives
before their destination-local evaluation.
**Finish:** the executor no longer embeds child/control and parent/policy
preparation. Actual recursive combination/evidence moves in Stage 15.
## Stage 15 — Return One Inheritance Combination Result
**Owners:** new engine `inheritance_combine.go`, `inheritance_collection.go`,
and the shared executor.
1. Add `inheritanceTemplateResult` as described above and extract the existing
parent/child combination into a helper receiving prepared inputs and the
application policy. Parentless templates return the same result shape.
2. Keep `mergeRecursiveMappings` and `combineSequence` as the combination
mechanisms. Their callbacks update local origins/policy and local replacement
evidence. Have `applyInheritanceSequenceOrigins` return its replacement slice
instead of taking the whole enclosing scope result; append that slice to the
local template result while retaining its updates to the supplied origin maps.
3. Derive sequence history, transferred origins, retired/replaced policies,
identity preservation, and the transfer operation from that same combination.
Preserve `historyParent` and all policy/identity fields on the existing
`inheritanceOperation`; do not rerun a merge or project values to recover them.
4. Keep sequence validation, diagnostic canonicalization, location assignment,
replacement order, and no-parent cloning behavior at their established
semantic points. Keep child identity absent from inherited-parent history but
present with its own origin in the effective child.
**Finish:** merge callbacks no longer mutate the executor accumulator. The
resolved template and all evidence travel together for Stage 16 integration.
## Stage 16 — Finish The Inheritance Executor And Accounting
**Owner:** engine `inheritance_schedule.go` and focused finalization helpers.
1. Reduce the executor to preparation, budget precheck, combination, accumulation,
and final rebuild. Append each template's control before its transfer/history
in the same order as before, and update the indexed resolved value/origin/
policy/template state once per template.
2. Extract retained-history charging for one template's returned operations and
replacements, reusing the same `retainedNodeCounter` across the scope as
before. Preserve pointer-based shared-node accounting, lower-bound checks,
scope-slot overhead, final expansion check, limits, and diagnostics.
3. Extract final rebuilding and scope policy/origin projection where it clarifies
the coordinator. Reuse `newInheritancePolicyIndex` once for scope localization;
do not return to scanning all policy entries for every slot.
4. Confirm both adapters and nested walking consume the same output. Retain
source-version movement, sensitivity, independent snapshots, assembled order,
cancellation, and policy retirement on actual replacement.
**Finish:** one readable executor coordinates explicit value/policy/evidence
handoffs, with no parallel inheritance implementation. Run I.
## Stage 17 — Extract Substantial Projection Handlers
**Owners:** engine `projection.go`, optionally new `projection_collection.go`.
1. Keep `projectConfiguration`'s schema/type guard and one `valueOperation`.
Keep `project`'s depth check, pointer/dynamic dispatch, and scalar conversion
choices visible; delegate structs, maps, and sequences to named methods.
2. Move struct member/inline flattening, collision checks, mapping-key charging,
and flattened-field/inline-map secret classification into `projectStruct`.
Preserve declaration order and the current decrement for a flattened
container that is not retained in the output.
3. Move deterministic sorted-map traversal into `projectMap`; keep its cycle
entry active until all recursive children complete. Move slice/array
construction into `projectSequence`, preserving nil-slice handling and the
slice-only cycle guard.
4. All recursive children call the same operation's dispatcher. Avoid duplicate
node charging/depth increments or early cycle release introduced by extraction.
Preserve float/text/duration/secret scalar handling and error paths.
**Finish:** dispatch and substantial handlers are separately readable, using the
original operation state. Do not refactor `clone`, `decodeValue`, or
`valueBuilder.build` as an additional cleanup project.
## Stage 18 — Separate Schema Field Discovery And Assembly
**Owner:** engine `schema.go`, optionally new `schema_fields.go`.
1. Keep `describeFields` as declaration-order enumeration, exported/ignored/tag
decisions, and dispatch to ordinary or inline handling.
2. Extract inline pointer-chain inspection and inline-struct/map assembly,
retaining recursion/building-state checks and the existing codec-versus-
structural distinction. Reuse the one builder/cache.
3. Make ordinary registration and inline flattening visibly maintain both
`members` and flattened `fields`/`fieldByName`/`inlineMap`. Preserve index
paths, nested secret propagation, ambiguity paths, and the single-inline-map
rule. Share registration helpers only for genuinely identical updates.
4. Preserve descriptor reuse and the physical-member view used by typed cloning.
Do not unify these views or add alternate field/tag policy. Existing schema,
projection, decoding, and cloning expectations remain the behavior baseline.
**Finish:** field discovery and descriptor assembly have explicit responsibilities
without a new registration framework. Run R for both Stages 17–18.
## Stage 19 — Update Consumer And Internal Navigation
**Owners:** `docs/consumers/README.md`, `docs/internal/overview.md`,
`docs/internal/imports.md`, and affected links/test-owner entries in
`docs/development.md`.
1. Add a concise consumer choice guide: ordinary `Load` for one selected source,
explicit discovery as a source usable by either entry, and `LoadLayered` for
underlay groups. Distinguish input-owned placement/import/filesystem choices
from load-wide definition/default/version/override choices and independent
layered overlay context. Link current API/config owners for exact rules.
2. Link the existing simple README example, bundled-file/layered examples,
import-focused prompt catalog, and discovered sequence catalog. Keep the
README's simple path simple; do not duplicate complete examples or rewrite
consumer domain rules to showcase new internal types.
3. Update implementation ownership and flow for normalized requests, acquisition
policy/state/tier lifetimes, resolved context versus live handles, import file
owners, and explicit inheritance/projection/schema handoffs. Fix moved-symbol
and behavioral-test navigation and stale comments affected by the refactor.
4. Preserve public contract definitions and accepted ADR decisions. Keep deferred
clone work in `future.md`. No policy change or new ADR is needed for the
selected private refactor.
**Finish:** current documentation describes implemented structure, with the
roadmap/plan retaining future work and progress until release cleanup.
## Stage 20 — Verify And Close The Combined Sprint
1. Search for obsolete import/input forwarding ladders, synthetic `Options`
acquisition construction, `sourceVersionState.importDefaults`, discarded
context derivation branches, unused inverse bridges, and temporary scaffolds.
Remove leftovers along with exclusive test scaffolding. Preserve distinct
operations and real diagnostic conversions; do not delete private code solely
because it resembles another helper.
2. Review the combined call flow: one normalization model, non-generic acquisition,
load versus tier versus traversal state, library-owned acquisition handles
closed before composition/publication, one combination outcome for inheritance
evidence, and one state owner per reflective operation. Review for added whole-tree
traversals, policy scans, or ownership copies without requiring measurements.
3. Account for every selected finding using the table below. Confirm any new
handoff coverage addresses a concrete gap. Existing correctness/algorithmic
regressions, public-boundary tests, and meaningful assertions stay effective.
4. Run the Final validation group, resolve failures, inspect the staged diff and
workspace, and commit the completed remaining work with plain-English
messages. Report changes, checks, limitations, and any unrelated work retained.
Do not tag, push, publish, or update consumer repositories in this sprint.
| Audit finding | Completion evidence |
| --- | --- |
| 1: imports | Stages 3–6 and 10–13 remove options reconstruction/default coupling and forwarding ladders; acquisition/scope/evidence owners and admission operations are clear. A and final checks pass. |
| 2: inheritance | Stages 14–16 return explicit template results, keep one executor, and preserve accounting and value/history/identity agreement. I and final checks pass. |
| 3: projection | Stage 17 extracts substantial handlers sharing operation state. R and final checks pass. Clone review remains deferred, not implicitly completed. |
| 4: schema | Stage 18 clarifies ordinary/inline assembly while preserving both descriptor views. R and final checks pass. |
| 5: contexts | Stages 7–9 share resolved facts across input/import/overlay/passive contexts without retaining handles or changing validation timing. A and final checks pass. |
| 6: facade | Stage 2 removes unused inverse conversions/adapters and shares real conversion. F and final boundary/consumer checks pass. |
**Finish:** the selected roadmap target is implemented, validated, documented,
and reviewable as one complete sprint. No unresolved product/scope decisions are
needed to begin these stages.