From 50199eb0af99292bfa5df43b141cad7f661e07d8 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 25 Oct 2026 15:09:34 +0000 Subject: [PATCH] Retire completed consolidation roadmap documents --- docs/development.md | 10 +- docs/roadmap/consolidation.md | 490 -------------------- docs/roadmap/implementation.md | 785 --------------------------------- docs/roadmap/remediation.md | 206 --------- 4 files changed, 8 insertions(+), 1483 deletions(-) delete mode 100644 docs/roadmap/consolidation.md delete mode 100644 docs/roadmap/implementation.md delete mode 100644 docs/roadmap/remediation.md diff --git a/docs/development.md b/docs/development.md index 277ac8a..2909376 100644 --- a/docs/development.md +++ b/docs/development.md @@ -189,8 +189,14 @@ remediation record are preserved in Git history at `0369111`, for example [configuration discovery](config.md#configuration-discovery); the public [discovery declarations](../discovery.go) own the Go API. -The [consolidation roadmap](roadmap/consolidation.md) owns the current refactoring -proposal and the associated complexity-audit findings. +The completed consolidation proposal, implementation plan, and resolved audit +findings are preserved in Git history at `34cbc0d` under `docs/roadmap/`. For +example: + +```sh +git show 34cbc0d:docs/roadmap/implementation.md +git show 34cbc0d:docs/roadmap/remediation.md +``` Potential enhancements are tracked in [future features](roadmap/future.md). diff --git a/docs/roadmap/consolidation.md b/docs/roadmap/consolidation.md deleted file mode 100644 index 070402a..0000000 --- a/docs/roadmap/consolidation.md +++ /dev/null @@ -1,490 +0,0 @@ -# ConfigKit Consolidation Toward 1.0 - -Status: selected target state; implementation pending. This document defines the -refactor's scope and rationale; it does not authorize a release. The scope -questions are settled. The [implementation plan](implementation.md) owns the -concrete steps, sequencing, and implementation progress. - -## Purpose - -Consolidate the machinery introduced by recent import, collection, inheritance, -and discovery work so ConfigKit is easier to understand, maintain, and adopt -before its public contracts become stable. Success means fewer independently -maintained paths through the same behavior, clearer state ownership, and concise -consumer workflows. Reducing a function's line count is useful only when it -serves those outcomes. - -The review considered public API consolidation as well as internal extraction. -The settled public direction retains the existing simple and layered loading APIs, -consolidates their private translation and acquisition machinery, and removes -unused reverse inspection conversions. No public signature or configuration -semantic change is planned in this effort. This is an assessment of the reviewed -consumer workflows, not a requirement to preserve every beta API in perpetuity. - -## Evidence And Baseline - -Reviewed worktree: `c9574ef`, immediately after the `v0.10.0` release at `5959537`. -This document replaces the earlier `complexity.md` roadmap and `remediation.md` -audit record. Their recommendations and relevant evidence are incorporated here; -the original documents remain available in Git history: - -```sh -git show fbaaaf1:docs/roadmap/complexity.md -git show fbaaaf1:docs/roadmap/remediation.md -``` - -The earlier roadmap reviewed `7a83704f43ef9870db33661231cdff5d6ee502f7`, before -discovery. The audit reviewed `5959537d1adc2516e7f7e8881df72b707ec48c3f`, including -implemented discovery. The consolidation review checked their recommendations -against that newer baseline and also considered the public API. - -This review inspected the current public load/input/discovery declarations and -maintained consumer examples, the shared request executor and acquisition flow, -import parsing and selection, inheritance execution, reflective projection and -cloning, schema field construction, and filesystem-context derivation. Graph -caller traces and text-reference checks confirmed the unused reverse conversion -chain. This was source review, not a new runtime correctness or performance audit. -Neither this review nor the audit established a runtime correctness defect. - -### Audit Measurements - -The audit refreshed the codebase-memory graph and used function size, cognitive -complexity, branch counts, and parameter counts to rank candidates for source -inspection. It indexed 901 non-test functions/methods and 928 test-file -declarations. Its fast index excludes examples, documentation, and scripts; a -tracked-file inventory covered those separately. Standalone `gocyclo`, `gocognit`, -`scc`, and `cloc` tools were unavailable. - -These are the graph tool's reported measurements at the audited revision. -Function spans include comments and blank lines. The scores support relative -triage; they are not standardized acceptance thresholds or evidence of a defect. - -| Function | Source | Lines | Cognitive score | -| --- | --- | ---: | ---: | -| `valueOperation.project` | [projection.go](../../internal/engine/projection.go) | 195 | 174 | -| `parseImportDeclarationSpec` | [import_spec.go](../../internal/engine/import_spec.go) | 127 | 129 | -| `valueOperation.clone` | [clone.go](../../internal/engine/clone.go) | 130 | 104 | -| `schemaBuilder.describeFields` | [schema.go](../../internal/engine/schema.go) | 99 | 100 | -| `assembleBaseInputTrackedForRole` | [load_input.go](../../internal/engine/load_input.go) | 188 | 96 | -| `selectDirectoryImportCandidates` | [import_selection.go](../../internal/engine/import_selection.go) | 154 | 96 | -| `valueBuilder.build` | [value.go](../../internal/engine/value.go) | 104 | 93 | -| `decodeValue` | [decode.go](../../internal/engine/decode.go) | 115 | 85 | -| `applyInheritanceScopeTemplates` | [inheritance_schedule.go](../../internal/engine/inheritance_schedule.go) | 219 | 75 | -| `contextsForAcquiredSources` | [source_context.go](../../internal/engine/source_context.go) | 102 | 72 | - -At that revision, the engine contained 88 production Go files and approximately -20,144 physical lines; the root facade contained 16 files and 1,861 lines. -`imports.go` was the largest production file at 1,195 lines. These measurements -locate concentrated responsibilities. They do not establish a need for more -packages or make the smaller document/YAML visitors and explicit facade -conversions maintenance defects. - -### Findings And Recommended Treatment - -The finding numbers below preserve the audit's identifiers. The implementation -plan records progress on the selected work. The cloning portion of finding 3 -was reviewed separately and closed without a refactor; see -[OQ-2](#oq-2-how-much-clone-cleanup-should-this-effort-commit-to). - -| Audit finding | Maintenance priority | Assessment and recommended treatment | -| --- | --- | --- | -| 1. Import entry points and responsibilities | Higher | Main refactor. Include shared acquisition normalization, import-default ownership, declaration parsing, and directory admission. | -| 2. Shared inheritance executor | Higher | Main refactor. Expose meaningful phases and explicit combination results while retaining one executor. | -| 3. Projection and cloning | Higher for projection; clone review complete | Main refactor for projection. The separate clone review concluded that the existing traversal should be retained; see [OQ-2](#oq-2-how-much-clone-cleanup-should-this-effort-commit-to). | -| 4. Schema field discovery and inline assembly | Medium | Include a focused refactor coordinated with projection. Keep one schema builder and its two descriptor views. | -| 5. Resolved filesystem context | Medium | Include alongside acquisition work. Reuse resolved facts without merging resource lifetimes or changing validation timing. | -| 6. Unused reverse inspection conversions | Small, independently reviewable cleanup | Include in the root facade and engine. Retain useful forward-conversion and detached-result coverage. | - -Findings 1–3 confirm the earlier roadmap's main opportunities; findings 4–6 add -focused work. The following sections own their rationale and intended changes. -The implementation plan records validation of the selected work; the completed -clone review retains its separate disposition. - -Completed correctness and algorithmic remediations remain part of the baseline, -including sequence records preserved at `ea46001:docs/roadmap/remediation.md` -and discovery records preserved at `0369111:docs/roadmap/remediation.md`. -Item-scope application matching and inheritance policy localization are examples -of completed corrections that overlap this work and whose regressions should -remain protected. - -The [architecture policy](../policy/architecture.md), -[configuration contract](../config.md), and exported declaration comments own -existing constraints and behavior. The -[component overview](../internal/overview.md) and -[import internals](../internal/imports.md) locate implementation responsibilities. - -## Consumer-Facing Target - -### Keep Simple Loading Simple And Advanced Inputs Explicit - -The current APIs already provide progressive setup: - -- A simple consumer calls `Load[Config](File(path))`. -- A consumer with defaults, imports, or resolution supplies `Options[Config]`. -- Explicit discovery produces an ordinary `Source`, so it uses the same loading - call and lifecycle. -- A layered consumer supplies an `Input` and `LayeredOptions[Config]`, with - `Underlay.Inputs` distinguishing strict peers from precedence tiers. - -The two loading functions already converge on `loadRequest` and -`executeLoadRequest`. Their public differences have meaning: ordinary options -include the one input's controls; layered loading has per-input controls and a -separate overlay filesystem. Ordinary overlay context follows its selected -source, whereas layered overlay context is independently specified. These are -observable conventions, not interchangeable forwarding spellings. - -**Decision (OQ-1, settled):** retain `Load`, `LoadLayered`, `Options`, `LayeredOptions`, -`Input`, and `Underlay`. Remove repeated translation and option reconstruction -behind them. Keep common application rules in `Definition`; do not require -applications to build a runtime loader or manually coordinate phases. - -A single public request type could place main input, underlays, and common -options together. It would reduce duplicated option fields and make promotion -from ordinary to layered loading more uniform. A convenience entry could preserve -the simplest call. Such a design would still need to resolve the existing -overlay-context differences and demonstrate a clearer consumer model. The -maintained examples do not show enough consumer benefit to justify that redesign -in this effort. Retaining the current APIs accepts some duplicated public option -fields and setup changes when adopting underlays; internal consolidation and -clearer guidance address the observed maintenance needs. The decision is based on -the resulting API, rather than the amount of refactoring needed to achieve it. - -### Preserve Useful Convenience And Ownership Boundaries - -| Surface reviewed | Recommended public end state | -| --- | --- | -| `Source`, `Filesystem`, `Search`, filename groups and presets | Retain typed, reusable source/search descriptions. Discovery selects a concrete source during loading; applications do not loop over complete loads. | -| `FilesystemContext` | Retain the public description and current host/native rules. Consolidate its resolved private representation; do not expose acquisition handles or engine context records. | -| `SequenceIdentityAt` and `SequenceIdentityMatching` | Retain the exact-path convenience and dynamic selector form over the common registration model. Both serve distinct consumer shapes. | -| `Definition`, resolution and validation callbacks | Retain application-owned domain rules inside one recorded lifecycle. Preserve the distinction between mutable recorded resolution and read-only validation. | -| Results, metadata, explanations, source context and diff | Retain detached, safe inspection with consumer-owned presentation. Remove unused inverse conversions without removing supported queries. | -| Root wrappers and private conversions | Retain root-owned public types and explicit ownership conversion under ADR 0006. Consolidate repeated real conversions and delete unused ones. | - -The consumer-facing work is to make the ordinary path easier to find and show -how additional features compose, using the existing maintained examples. Update -the adoption guide with a concise choice between simple loading, discovery, and -layered loading, and show which options belong to an input versus a whole load. -Keep exact behavior in its canonical owners. Examples should make application -rules recognizable and avoid requiring consumers to recreate generic merging, -presence, or provenance machinery. - -Use the current simple README call, bundled-file and layered catalog examples, -the import-focused prompt catalog, and the discovered sequence catalog to assess -the end state. The latter exercises imports, underlays, independent overlay -context, CLI assignment, and recorded resolution in one workflow. A high number -of options in that example does not by itself justify a new abstraction. - -## Internal Target - -Keep the root facade, cohesive `internal/engine`, ordered document model, and -private YAML adapter. Use ordinary private records, functions, and focused files -to express responsibilities. Existing mapping combination, sequence combination, -dependency scheduling, schema descriptors, and traversal accounting remain shared. - -`executeLoadRequest` stays the visible lifecycle coordinator. It should show -acquisition, assembly, inheritance, decoding/CLI assignment, resolution, secret -processing, validation, and publication in their established order. Extract a -phase only when it gives that work a clear input, output, or ownership boundary. -Do not replace the visible lifecycle with a registry of configurable stages. - -### 1. Normalize Loading Inputs And Consolidate Import Acquisition - -The current executor constructs an engine `Options[T]` from `loadRequest`, and -underlay acquisition constructs `Options[struct{}]` to call the same input -assembler. `assembleBaseInputTrackedForRole` then combines discovery, directive -processing, version rules, imports, placement, accounting, and contexts. Import -entry points carry long positional lists through several forwarding variants. -This repeats the representation of information already normalized for a load. - -In [imports.go](../../internal/engine/imports.go), the -`loadImportedDocumentsTrackedWithVersion`, -`loadImportedDocumentsTrackedWithProcessing`, and -`loadImportedDocumentsTrackedWithProcessingDeferred` variants have corresponding -`loadImportedWithAccess...` variants. Both final signatures carry 13 parameters. -Placement, processing results, sequence policy, accounting, repetition tracking, -namespace identity, and deferred assembly must stay aligned through the chain. -The same file's `importTraversal.visitPath` combines source-admission decisions -in 174 lines, alongside acquisition, pending-item assembly, and evidence work. - -Introduce cohesive private acquisition inputs and explicit state ownership, -reusing `loadRequest`, `loadInput`, `acquisitionScope`, and `importTraversal` -where they fit. Translate ordinary and layered public declarations into the -shared loading model once. Acquisition should receive the source, input role, -placement, context, and policies it needs without a fabricated application type -or a reconstructed broad options object. Small adapters may preserve meaningful -entry operations while callers migrate; remove obsolete forwarding variants -within the completed refactor. - -Separate immutable policy, mutable processing state, and results. Move import -defaults out of `sourceVersionState` and into the load/acquisition policy owner. -Version state continues to own version processing, projected effective version -location, evidence, and diagnostics. Local and effective version rules have -different meanings and must remain distinguishable. - -The state boundaries are especially important: - -| Lifetime or scope | State and responsibility | -| --- | --- | -| Whole load | Aggregate resource accounting, application definition/registrations, import defaults, shared version evidence and diagnostics. | -| One strict tier | Repetition tracking shared by that tier's inputs, namespace scheduling anchors, and its inclusion/composition graph relationship. A later tier may reuse a file. | -| Active import traversal | Active routes and inclusion edges, filesystem access ownership, and cycle detection, including edges crossing item-scope boundaries. | -| One composition scope | Peer contributions, pending collection declarations, item-local helpers, and deferred item placement. Nested scopes do not become separate precedence tiers. | -| Retained result | Source records, effective destinations, origins, history, and passive resource-context descriptions. | - -Keep repetition tracking tier-scoped. In the current code, -`acquireUnderlayGroupScopes` creates a fresh visited map per group. Preserve valid -reuse across tiers while keeping shared aggregate limits. Namespace equality, -candidate spelling, and stable scheduling keys also serve different purposes. - -Give acquisition, already-acquired scope assembly, and evidence projection -focused homes. Keep discovery at the acquisition boundary: resolve a search once -per input consumption, then process the concrete file or absence. Preserve -source IDs and source-local version processing in acquisition order, even when -composition uses a different dependency-ready order. The layered main is acquired -before underlays; lower-tier identity can still inform its later strict assembly. - -Within `parseImportDeclarationSpec`, extract boolean/collection/placement field -handling and selector-value parsing where it clarifies responsibilities. Keep -field presence, authored error locations, selector applicability, and the order -of consequential validation visible. Within `selectDirectoryImportCandidates` and -`importTraversal.visitPath`, separate admission and descent decisions from -traversal orchestration where they represent distinct work. Retain the existing -root deduplication, candidate union, exclusions, sorting, and active-path cycle -rules. No shared abstraction should conflate literal first-match main discovery -with multi-file import selection. - -### 2. Share Resolved Filesystem Facts Without Sharing Live Handles - -`importFilesystem`, `nativeFilesystemAccess`, and `contextsForAcquiredSources` -derive overlapping root/base/namespace facts. The latter reconstructs context -from combinations of source kind and optional fields, including absent input. -Discovery and independent layered overlays make these combinations consequential. -Context construction is used by ordinary acquisition and both selected-overlay -paths, so those callers need to use the same applicable derivation rules. - -Use a small engine-owned resolved-context description for the facts actually -needed by acquisition and passive views: namespace identity/kind, resolved root, -source base, and selected source path when present. Preserve the distinction -between host paths and native slash paths. Reuse already resolved facts where -available instead of repeatedly deriving them from the original options. - -Resolution and validation can occur at different existing lifecycle points. -The record must not force every source to perform all filesystem checks early. -Context descriptions, library-owned acquisition handles, and returned passive -resource views have different lifetimes. Closing acquisition must remain -explicit on success and failure; retaining context must not retain an open root -handle or assume ownership of a borrowed `fs.FS`. - -Cover selected and absent discovery outcomes, root-only absent host context, -native sources, in-memory inputs with explicit context, and independently rooted -overlays. Passive context remains associated with the source that contributed -the value, including retained underlay and inherited values. Use the acquired -source's canonical identity where available. Preserve containment failures, -source associations, deferred search bases, and terminal selected-file errors. - -Acquisition and context handling use the same private handoff rather than -maintaining parallel models for overlapping callers. The description does not -replace the public `FilesystemContext` or `ResourceContext` contracts. - -### 3. Make Inheritance Combination Results Explicit - -Keep the mapping and identified-record adapters and one shared executor. -`applyInheritanceScopeTemplates` currently prepares templates, mutates values -through merge callbacks, projects policy and origins, records replacements and -replay evidence, and enforces budgets in one body. The callbacks mutate several -surrounding result collections, making a combination difficult to review alone. - -Give the executor named operations for child/control preparation, parent payload -and policy preparation, combination, evidence construction, and retained-node -accounting. Use a small private result for outputs of one combination where it -replaces unrelated surrounding mutations. Value, policy, origin, replacement, -and sequence-position evidence should come from the same combination outcome; -do not rerun a merge to reconstruct its history. - -Keep borrowed authored templates separate from owned resolved payloads and from -retained history snapshots. Parent identity exclusion and child identity/origin -preservation must apply consistently to values and history. Preserve nested -unresolved instructions for destination-local evaluation, policy retirement on -actual replacement, cancellation, and source-version movement through sequences. - -The shared Kahn scheduler with stable keys remains unchanged. Resolve enclosing -relationships before nested relationships and retain assembled record order. -Preserve expansion limits and retained-history accounting for shared nodes; -extraction must not charge shared evidence twice or make an unbounded snapshot. - -### 4. Separate Projection Handlers And Clarify Schema Assembly - -Keep `projectConfiguration` as the schema-checked entry and `valueOperation` as -the operation-state owner. Move substantial struct, map, and sequence projection -into focused handlers. Keep pointer/dynamic dispatch and scalar representation -choices visible. All handlers share the same cycle, depth, and node state. - -Struct projection needs particular care: physical members drive traversal, -flattened fields drive name lookup and sensitivity, and an inline container is -not retained as an extra output node. Preserve declaration order, deterministic -map order, nil/empty distinctions, collision diagnostics, codec failures, and -secret classification. Scalar projection retains the existing number, duration, -text-codec, and secret-reference representations. Keep cycle entries active -through child traversal and balance their release on every exit. - -In `schemaBuilder.describeFields`, separate field discovery and tag parsing from -inline struct assembly, inline map assembly, and ordinary field registration. -Retain one builder/cache and its recursion detection. Make the paired updates to -`members`, flattened `fields`, `fieldByName`, and `inlineMap` easy to follow. -Keep index-path construction and inherited secret flags intact. These descriptor -views are both needed; combining them into one collection would shift complexity -into decoding and projection. -Keep pointer-chain and recursive-inline decisions explicit, preserve supported -field/tag forms and ambiguity paths, and retain cache/building-state behavior. - -Clone cleanup remains outside this effort; the separate -[review](#oq-2-how-much-clone-cleanup-should-this-effort-commit-to) concluded that -no refactor was warranted. This effort preserves cloning's existing behavior -while changing the shared schema and projection machinery. -Keep projection's document-building semantics distinct from typed copying. -The mostly explicit dispatch in `decodeValue` and `valueBuilder.build` does not -warrant a generic reflection visitor solely because it has a high complexity -score. - -### 5. Remove Unused Inspection Conversions And Consolidate Real Adapters - -Remove `toEngineSourceInfo`, `toEngineOrigin`, `toEngineSourceVersionCheck`, and -`toEngineImportSelections` from [conversion.go](../../conversion.go), along with -the engine adapters used only by that chain. -At the reviewed revision, the three callers of `toEngineSourceInfo` are tests; -`toEngineOrigin` has none. `SourceInfoWithPlacement`, -`SourceInfoWithImportSelections`, `importSelectorFromString`, and -`importSelectionOutcomeFromString` in [records.go](../../internal/engine/records.go) -have no production role beyond that reverse chain. Consumers receive inspection -records; production facade operations do not send those records back to the -engine. The unused inverse path adds another ownership mapping to maintain as -inspection evidence grows. Recheck callers at the implementation baseline before -removal. - -Rewrite the affected [record tests](../../records_test.go) around real -engine-to-public conversion and detached public inspection. Preserve useful -placement, version evidence, import-selection, and copy-isolation assertions. -Avoid manufacturing replacement production setters solely to keep an old -round-trip fixture working. Existing facade loads already provide genuine sources -and inspection records. - -Retain reverse conversions that serve real inputs: `toEngineDiagnostics`, for -example, supports `WriteDiagnostics`. Consolidate repeated override/input -conversion in `toEngineOptions` and `toEngineLayeredOptions` where semantics -match. Keep explicit root-to-engine ownership conversion and the public-boundary -checks. A reflective transport framework or public engine aliases would obscure -the boundary rather than simplify it. - -## Scope, Compatibility, And Policy Consequences - -The proposed work spans private engine code, private root-facade conversions, -affected tests, consumer guidance, and implementation documentation. Under the -settled answer to [OQ-1](#oq-1-should-the-public-loading-apis-change-in-this-effort), -preserve public types/signatures and the behavior owned by the current contracts, -including: - -- Acquisition versus composition ordering, strict scopes versus precedence - tiers, identity ownership, placement, and source-version protection. -- Discovery exhaustion and error behavior, source lifetime, passive resource - contexts, and host/native namespace distinctions. -- Defaults, post-decode CLI assignment, recorded resolution, secret handling, - final registered-identity enforcement, and semantic validation. -- Explicit presence, provenance/history, sensitivity, safe diagnostics and diff, - frozen inspection, and detached publication/query results. -- Contracted diagnostic codes, paths, coordinates, ordering, and resource limits. - -Do not turn incidental helper order, exact private representations, numerical -complexity scores, or allocation counts into new guarantees. Keep the existing -algorithmic remediations effective; watch for repeated whole-tree traversals, -policy scans, or copies introduced by helper extraction. - -The recommendation fits the existing architecture policy and ADRs 0006, -0015, 0017, 0018, and 0019. It needs no new architectural ownership decision or -policy revision merely to introduce private helpers. Update the component -overview and import internals when code moves. Accepted ADR decision content -remains unchanged. A public loading-model redesign is outside this effort's -settled scope. - -Pre-1.0 public changes are possible under the [release policy](../policy/release.md), -but they must have a consumer benefit and documented impact. This draft does not -propose a parallel compatibility API or temporary migration scaffold. It also -does not make refactoring completion a new release gate or establish that 1.0 -is otherwise ready. Future performance experiments remain in -[future.md](future.md#3-performance-experiments); no benchmark campaign was run -or is required to claim readability improvements. - -## Validation And Completion - -Use the [development guide](../development.md) and -[testing policy](../policy/testing.md) for focused checks and the ordinary full -development check. Preserve existing behavior tests; rewrite tests coupled only -to removed plumbing. Add a test when a consequential handoff risk lacks coverage, -not for every new helper. - -| Work area | Existing protection and review focus | -| --- | --- | -| Acquisition and imports | Selection, defaults, scheduling, relative/item placement, source-version evidence, aggregate limits, repetition within versus across tiers, and ordinary/layered equivalence. | -| Resolved contexts | Host/native access, discovery selected/absent outcomes, terminal context failures, source-relative resources, independent overlay context, and handle closure. | -| Inheritance | Mapping and record inheritance, nested destination-local evaluation, child identity, policy lifetime, version movement, history ordering, sensitivity, and retained-node budgets. | -| Projection/schema | Inline fields, recursive schemas, dynamic values, nil/empty values, cycles and limits, text codecs, secret states, callback ownership, and independent loads; preserve existing cloning behavior where shared descriptors/state are affected. | -| Facade and consumers | Forward conversion, detached records, supported API boundaries, source-form equivalence, maintained simple/catalog/discovery examples, and safe explanation/diff. | - -Examples of established boundaries include -[source-form equivalence](../../facade_equivalence_test.go), -[discovery integration](../../facade_discovery_test.go), -[final identity validation](../../facade_final_sequence_identity_test.go), -[inheritance history](../../internal/engine/inheritance_history_test.go), and -[projection and cloning](../../internal/engine/projection_test.go). -Use applicable race checks when changing ownership or shared state. Reuse checks -for unchanged content and keep benchmark work separate unless new evidence -justifies it and it is authorized. - -Completion requires reviewable evidence that: - -- Simple and advanced consumer examples remain concise and valid, with no new - public concepts required for existing workflows. -- Both load entry points reach one normalized internal model; acquisition no - longer reconstructs broad options or uses version state to transport defaults. -- Import, context, inheritance, and reflection responsibilities have identifiable - owners and explicit handoffs without duplicate semantic implementations. -- Unused reverse conversions and their exclusive adapters are removed, with - useful inspection ownership coverage retained. -- Contracted values, presence, provenance/history, diagnostics, sensitivity, - resource lifetime, and algorithmic corrections remain protected by appropriate - tests and source review. -- The reviewed scope is accounted for against all six audit findings, with clone - traversal review closed separately without a refactor. Completed portions have - validation evidence rather than being closed because code moved. -- Consumer guidance, contributor references, and internal documentation describe - the resulting implementation. Current contracts continue to own behavior. - -The [implementation plan](implementation.md) defines private records, file owners, -and stage boundaries. No open scope questions remain, and no additional feature -semantics are needed to carry out the selected refactors. - -## Settled Decisions - -### OQ-1: Should The Public Loading APIs Change In This Effort? - -**Settled:** retain `Load`/`Options`, `LoadLayered`/`LayeredOptions`, `Input`, and -`Underlay`, including their existing overlay-context behavior. Consolidate their -private inputs, conversions, and acquisition machinery, and improve consumer -guidance. The [consumer-facing target](#consumer-facing-target) records the -rationale and tradeoffs. This decision governs the consolidation effort; it does -not establish a perpetual restriction on future API design. - -### OQ-2: How Much Clone Cleanup Should This Effort Commit To? - -**Settled:** clone review was excluded from the consolidation sprint and completed -separately on 2026-10-25 against `8dfaf44`. Retain the existing clone traversal; -no cleanup is planned. - -The review examined the traversal, shared helpers, schema-member construction, -lifecycle callers, and relevant tests. Its branches already keep allocation, -recursive copying, and cycle cleanup together. Shared depth, cycle, dynamic-type, -and text-marshaling operations already have common owners. Further extraction, -including the similar slice/array loops, did not offer enough maintenance benefit -to justify the additional indirection. Focused engine tests for projection, -cloning, inline fields, callback ownership, and secret-copy behavior passed. -No code changes or benchmarks were needed. This is a maintenance assessment, -not a restriction on future changes justified by new needs. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index f966090..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,785 +0,0 @@ -# Consolidation Implementation Plan - -Status: complete. Stages 1 through 20 are complete. -The target executor is a Luna coding agent working through one end-to-end sprint. - -## Authority, Baseline, And Execution - -The [consolidation roadmap](consolidation.md) owns the target state, rationale, -and settled scope. Start from `8a99e6107001c69117a41dc37bc6ea09d2910af9` plus the -commit introducing this plan. Production code is unchanged from the audited -`5959537d1adc2516e7f7e8881df72b707ec48c3f`; intervening commits consolidated the -planning documents and settled the public-API and clone decisions. - -Read [development.md](../development.md), [agents.md](../policy/agents.md), -[architecture.md](../policy/architecture.md), [testing.md](../policy/testing.md), -and [documentation.md](../policy/documentation.md) as applicable. Use the code -graph for discovery and confirm source at the implementation baseline. Current -public comments and [config.md](../config.md) define behavior; this plan does not -add public obligations. - -Implement only the selected consolidation. Retain the public `Load`, -`LoadLayered`, options, inputs, callbacks, source/search descriptions, inspection, -and sequence-registration APIs. Keep the root-to-engine-to-document/YAML -dependency direction. Clone traversal review/refactoring was excluded from this -sprint; the subsequent [review outcome](consolidation.md#oq-2-how-much-clone-cleanup-should-this-effort-commit-to) -retains the existing traversal. Changes to shared descriptors must -preserve existing cloning behavior. This sprint does not publish a release, -update dependencies/toolchain requirements, change configuration semantics, or -introduce complexity/performance gates. - -Stages are bounded work assignments, not independently shippable commits. -Intermediate stages may leave callers, tests, or compilation unfinished until a -later stage. Proceed directly through those dependencies; do not build temporary -compatibility layers solely to keep each stage compiling. Update affected tests -with their callers, and run the focused checks below when that portion builds. -Record any deferred verification accurately. The completed sprint must compile, -pass its checks, and leave no superseded plumbing. Commit completed, reviewable -units under the agent policy; stage boundaries do not require separate commits. - -Preserve existing meaningful validation/processing order, diagnostics, ownership, -and limits. Private helper order and representation are not new contracts. Do -not add rejection rules, error recovery, caches, new merge strategies, another -pipeline framework, or tests for incidental helper choreography. A correctness -issue discovered during the refactor needs identifiable regression evidence and -an explicit explanation, rather than an unexplained expectation change. - -## Chosen Private Design - -Use the existing `loadRequest[T]` as the normalized complete-load model and -`loadInput` for each selected input. Keep generic application definitions and -the final lifecycle in `executeLoadRequest[T]`; acquisition is non-generic. -The following records are ordinary engine-private data, not extension points. -Use these names to make migrations concrete; their fields describe the selected -ownership model, not a requirement to duplicate data already available there. - -| Record | Fields and ownership | -| --- | --- | -| `acquisitionPolicy` | Read-only operation snapshot of `importDefaults`, `sequencePolicy`, and `importLimits`. Snapshot caller-owned defaults at entry, preserving nil versus empty. | -| `acquisitionState` | Borrowed pointers to the load's `importAccounting` and `sourceVersionState`. One instance per load; version state no longer transports import defaults. | -| `tierAcquisitionState` | The tier's visited-resource map and shared host namespace token. Underlay groups each own a fresh instance. Existing group graph aggregation and namespace anchors retain their current owners. | -| `inputAcquisitionRequest` | `input loadInput`, source-version role, root-directive context, applicable protection rule, deferred-assembly choice, and `ordinaryOverlay string`. The last field carries only the ordinary main input's overlay-dependent validation/context timing; it is empty for underlays and layered main. | -| `importAcquisitionRequest` | The input request, parsed main source, and already computed source-version/directive processing result. It preserves the distinction between source-local processing and application-path protection. | -| `resolvedFilesystemContext` | `native *nativeNamespace` (nil means host), `hostToken *namespaceToken` where supplied, and `rootPath`, `basePath`, `mainPath` strings. Native facts use slash-relative paths; host facts use the existing canonical host paths. The record contains no open root/file or acquisition access object. | - -The main private entry is -`acquireInput(request inputAcquisitionRequest, policy acquisitionPolicy, state *acquisitionState, tier *tierAcquisitionState) (assembledInput, []Diagnostic)`. -Import acquisition uses the corresponding named request with the same borrowed -policy/state/tier owners. An access-based worker receives a resolved filesystem -description and an explicitly owned `*filesystemAccess`, rather than more -positional paths and independent optional policy parameters. Retain -`assembledInput`, `acquisitionScope`, and `acquiredContribution` as the structural -handoffs; add an optional resolved-context field to `assembledInput` for reuse. - -`importTraversal` borrows the policy and state references. Active routes, stack, -current item scope/prefix/fallback, and the acquired inclusion graph remain -traversal-local. Nested item traversal shares the active graph and tier repetition -state; it does not create another tier. Reusing a file in a later tier remains -valid. Do not collapse namespace identity, lexical candidate spelling, and stable -scheduling keys into one key. - -Context facts are obtained at the existing points where those facts are needed. -There is no global eager context-validation pass or persistent live-handle owner. -Directory checks, canonicalization, handle opening/closing, and passive view -creation remain distinguishable. Reusing a description does not authorize -earlier checks that bypass an existing earlier error or implicit loading activity -after publication. Consumer-initiated passive filesystem operations retain their -existing contract. - -For inheritance, retain both scope adapters and one executor. Use private -prepared-child/prepared-parent records for combination inputs, then an -`inheritanceTemplateResult` containing the resolved value, origins, local policy, -policy templates, and the controls/transfers/replacements/operations produced by -that template. Sequence position evidence lives in the existing operation -records. Merge callbacks mutate that local result, not the enclosing executor's -accumulator. The executor appends the result and applies existing accounting. - -For reflection, keep `valueOperation` and `schemaBuilder` as their existing state -owners. Extract substantial value-kind and inline-field operations as methods; -do not introduce a generic reflection visitor or merge the two schema views. - -## Validation Groups - -Use the installed compatible Go toolchain and already acquired dependencies as -described in the development guide. Commands run from the repository root with -`GOTOOLCHAIN=local` and `GOPROXY=off`. These are development checks for this -refactor, not additions to the release procedure. - -| Group | Commands and purpose | -| --- | --- | -| F: facade cleanup | `go test . -run 'SourceInfo|SourceVersionCheck|ImportSelection|Facade|PublicFacadeBoundary|DependencyImportBoundary'` checks useful forward evidence and the existing public boundary. | -| A: acquisition integration | `go test ./internal/engine -run 'Load|Grouped|Source|Search|Discovery|Import|Glob|Filesystem|Native|Destination|Overlay|Resource|Version|Sequence|Collection'`; `go test . -run 'Facade|EquivalentSourceForms|Search|SourceContext|ImportDefaults|SourceVersion'`; `go test ./examples/consumers`. Covers normalization, ordinary/layered acquisition, selection, contexts, scopes, and consumer workflows. | -| I: inheritance | `go test ./internal/engine -run 'Inheritance|Inherited|Sequence|Collection|Version|History|Explain|Sensitivity|Secret'`; `go test . -run 'Inheritance|Inherited|SequenceIdentity|FinalSequence'`. Covers shared adapters and their evidence. | -| R: reflection | `go test ./internal/engine -run 'Schema|Decode|Projection|Clone|Inline|CompareDocument|Callback|Ownership|Independent|Secret'`. Covers descriptors, projection, shared cloning behavior, and ownership. | -| Final | `./scripts/check.sh` followed by `go test -race ./...`. The first owns formatting, whitespace, ordinary tests/examples/doc links, vet, and build; race validation is appropriate because state and ownership handoffs change throughout this sprint. | - -Run F after Stage 2, A after Stages 9 and 13, I after Stage 16, and R after -Stage 18 when buildable. If earlier migrations still prevent compilation, repair -them at the next integration point and run the deferred group there. Run the -final checks once on the finished code/documentation; repeat only for subsequent -changes affecting their evidence. No benchmark campaign, timing threshold, -traversal-count test, extended fuzz campaign, CI wait, or manual platform check is -required by this plan. Add narrow behavior tests only for consequential gaps -identified below that existing tests do not already cover. - -## Stage 1 — Establish The Starting Map (Complete) - -**Work:** - -1. Confirm the revision/worktree and preserve unrelated work. Read the roadmap's - settled decisions and the selected owners above. -2. Map the current callers of `selectOptions`, `assembleBaseInput...`, both - `loadImported...` ladders, `contextsForAcquiredSources`, the selected-overlay - loaders, and `applyInheritanceScopeTemplates`. Inspect test callers as well - as production callers; some old entry variants exist mainly for tests. -3. Identify the existing behavioral owners for repeated resources within/across - tiers, enabled/disabled version checking, selected/absent searches, ordinary - versus independent layered overlays, item-local helper scopes, inherited - identity/history, inline fields, and detached records. -4. Keep brief working notes or implementation progress in this plan. Reuse - existing validation evidence where applicable; establish an implicated - baseline only when needed to distinguish a pre-existing failure. Do not - generate another broad audit or blanket characterization-test suite. - -**Finish:** a concrete caller/coverage map and any actual coverage gaps. No -production change or empty commit is needed for orientation alone. - -Progress notes (2026-10-25): the worktree was clean at `8ab1568`, immediately -after the planning baseline `8a99e61`; the audited production baseline -`5959537` is present. `selectOptions` is called by the ordinary and grouped -load entry points. The `assembleBaseInput...` wrappers converge on -`assembleBaseInputTrackedForRole`, reached from ordinary loading, layered main -loading, and underlay-group acquisition. The tracked import wrappers converge -on `loadImportedDocumentsTrackedWithProcessing`; production callers are input -acquisition and underlay-group acquisition, while older convenience variants -also have direct test callers. `contextsForAcquiredSources` is shared by ordinary -loading, layered assembly, input acquisition, and overlay application. Selected -overlay loading is reached from ordinary overlay application and layered input -assembly. Both inheritance adapters, for ordinary and identified sequences, -feed `applyInheritanceScopeTemplates`. - -Existing behavioral owners cover repeated/cyclic resources and cross-tier reuse -(`internal/engine/imports_test.go`, `facade_equivalence_test.go`); default and -enabled source-version modes (`internal/engine/source_version_test.go`, -`facade_source_version_mode_test.go`); selected/absent discovery -(`internal/engine/discovery_load_test.go`, `facade_discovery_test.go`); ordinary -and layered overlays (`internal/engine/selected_overlay_test.go`, -`facade_layered_test.go`); item-local helper scopes (`internal/engine/imports_test.go`, -`facade_import_lifecycle_test.go`); inheritance identity/history -(`internal/engine/inheritance_identity_history_test.go`, -`internal/engine/inheritance_history_test.go`); inline fields -(`internal/engine/schema_test.go`, `internal/engine/projection_test.go`); and -detached results and callback ownership (`internal/engine/metadata_test.go`, -`internal/engine/callback_ownership_test.go`, `facade_diff_test.go`). No specific -coverage gap surfaced in this map; later focused checks remain assigned in the -validation groups above. - -## Stage 2 — Remove Unused Reverse Bridges And Share Real Facade Conversion (Complete) - -**Owners:** root `conversion.go`, `records_test.go`, `lifecycle.go`, `layered.go`, -and engine `records.go`. - -1. Reconfirm callers, then delete `toEngineSourceInfo`, `toEngineOrigin`, - `toEngineSourceVersionCheck`, and `toEngineImportSelections`, plus - `SourceInfoWithPlacement`, `SourceInfoWithImportSelections`, - `importSelectorFromString`, and `importSelectionOutcomeFromString` when their - only remaining purpose is the deleted chain. -2. Rewrite the three reverse-round-trip record tests using genuine engine output - and forward conversion/public inspection. Retain placement, version-evidence, - selection-evidence, nil/empty, and detached-copy protection. Engine test - fixtures can use existing loading APIs; do not add production fixture setters. -3. Extract one root `toEngineOverrides` helper and use it from both option - converters. Reuse `toEngineInput`, filesystem, path, definition, and default - conversion where already shared. Preserve slice ownership and nil/empty - behavior; do not make conversion interpret/validate input values. -4. Retain reverse diagnostic conversions used by `WriteDiagnostics`, forward - source/origin inspection conversion, and all public named types. - -**Finish:** the unnecessary cross-package inverse interface is gone and real -conversion is shared without a new transport layer. Run F. - -Progress notes (2026-10-25): removed the unused reverse source-record bridge -and engine setters, replaced synthetic round-trip fixtures with engine loads, -and shared override conversion across ordinary and layered options. Validation -group F passed with `GOTOOLCHAIN=local GOPROXY=off go test . -run -'SourceInfo|SourceVersionCheck|ImportSelection|Facade|PublicFacadeBoundary|DependencyImportBoundary' -count=1`. - -## Stage 3 — Normalize Preflight Without Fabricated Options (Complete) - -**Owners:** engine `options.go`, ordinary entry in `load.go`, grouped entry in -`load_request.go`, and their option tests. - -1. Extract non-generic input normalization/validation over `loadInput`, returning - the normalized input plus diagnostics. Pass the ordinary overlay name only - where its context requirements apply. Keep discovery-description validation, - placement checks, normalized destinations, and filesystem option checks at - their existing pre-acquisition boundary. -2. Separate load-wide import-default validation, version-rule/mode validation, - and sequence-registration compilation from per-input checks. Share their - mechanisms between entries without adding a second interpretation of options. -3. Keep the public/engine entry arity handling and each entry's existing error - classification. Ordinary loading still translates its one selected options - value; layered loading validates main, all underlay inputs, and its independent - overlay context in the existing meaningful order. Preserve canonical diagnostic - accumulation and early returns, including invalid options before reads. -4. Replace the grouped entry's constructed `Options[T]` values with direct - normalization of `loadInput`; set the effective protection rule from the - normalized main placement. Leave typed application behavior in the definition. -5. Remove `importDefaultsValidated` and preflight-only forwarding machinery when - callers migrate. Update tests through the entry/helper that owns the behavior, - rather than keeping `selectOptions` solely as a test compatibility surface. - -**Finish:** both entries produce validated `loadRequest[T]` values without using -the ordinary options type to validate grouped inputs. Acquisition may still use -old options until Stage 6; do not broaden this stage into that migration. - -Progress notes (2026-10-25): added shared normalization over `loadInput`, moved -load-wide defaults, source-version, and sequence-registration validation into -shared settings preparation, and removed `selectOptions` plus its validation -flag from grouped preflight. `GOTOOLCHAIN=local GOPROXY=off go test ./...` passed. - -## Stage 4 — Express Input Acquisition With Non-Generic Requests (Complete) - -**Owners:** new engine `acquisition.go` and existing `load_input.go`. - -1. Define `acquisitionPolicy`, `acquisitionState`, `tierAcquisitionState`, - `inputAcquisitionRequest`, and `importAcquisitionRequest` as selected above. - Reuse existing enums, policy records, and accounting; introduce no new public - controls or duplicate budget counter. -2. Replace the generic `assembleBaseInputTrackedForRole` implementation with - `acquireInput`. Translate its references to source, imports, filesystem, - placement, role, directive context, version rule, policy, and deferred - assembly into explicit request/policy/state fields. -3. Keep its phases visible: search selection; parse and failed-source evidence; - disabled-directive checks; source-local version/directive processing; import - acquisition or single-input placement; accounting/repetition registration; - context handoff. Small helpers may isolate version/directive preparation. -4. Preserve discovery resolution once per input consumption and the acquired - concrete source on both success and failure. Keep mapping diagnostic - destination projection and the ordinary-overlay context deferral unchanged. -5. Retain `assembledInput`'s existing structural/evidence fields. Do not merge - parsed/local payloads with placed documents or eagerly compose deferred input. - -**Finish:** the shared input-acquisition implementation no longer depends on -`Options[T]` or an application type. Call-site and import-worker migration is -completed by Stages 5–6; temporary non-compilation is acceptable. - -Progress notes (2026-10-25): introduced the non-generic acquisition request and -policy/state/tier records, moved the input-acquisition implementation into -`acquireInput`, and added the resolved-context handoff field. The focused engine -test command currently cannot compile because the old assembly callers in -`load_input.go` and `load_request.go` are scheduled to migrate in Stage 6; test -verification is deferred until that integration. - -## Stage 5 — Consolidate Import Requests And Default Ownership (Complete) - -**Owners:** engine import entry points, `importTraversal`, `source_version.go`, -and tests that directly invoke the old import variants. - -1. Replace the `loadImportedDocumentsTracked...` ladder with - `acquireImportedInput` using `importAcquisitionRequest` and the shared - policy/state/tier owners. Replace the access-based ladder with one worker - receiving the same request plus explicit access/resolved facts. Keep only - genuinely distinct convenience entries, not one wrapper per optional field. -2. Move `importDefaults` out of `sourceVersionState`. Resolve declarations from - `acquisitionPolicy.importDefaults`, including nested item scopes. Import - defaults must work independently of whether a version rule/state is active. - Preserve enabled version processing and its separate local/effective rules. -3. Make traversal references to aggregate accounting, version evidence, tier - repetition, and immutable policies point to their existing operation owners. - Keep active-route tracking, scope switching, and inclusion edges local to - the traversal. Do not allocate a fresh visited map inside a nested item scope. -4. Preserve nil version-state behavior used by lower-level operations and the - existing default processing branch. Replace optional positional/variadic - policy passing with explicit inputs for meaningful operation distinctions. - Private test entry spellings do not need compatibility wrappers; preserve - their relevant behavioral cases through the resulting operation. -5. Migrate direct import tests to named inputs. Small test-only constructors may - supply default state/policy values when this reduces repetitive fixtures. - Keep their behavior assertions intact. - -**Finish:** the import request surface has named cohesive inputs; defaults no -longer travel through version processing; no forwarding ladder remains merely -to supply another nil/default positional parameter. - -Progress notes (2026-10-25): replaced the import entry ladders with named -requests and one access worker, moved defaults to acquisition policy, and -retargeted direct import tests. The focused engine test command cannot compile -yet because the remaining ordinary and grouped assembly callers are scheduled -for Stage 6; verification is deferred until that integration. `git diff --check` -passed. - -## Stage 6 — Wire Ordinary And Grouped Lifecycle To Acquisition (Complete) - -**Owners:** engine `load.go`, `load_request.go`, `load_input.go`, related tests. - -1. Construct the load's acquisition policy and mutable state once in - `executeLoadRequest`. Keep the definition and schema/callback lifecycle there. -2. Call `acquireInput` for main using its ordinary/layered directive context, - main version role, current protection rule, and deferred assembly. Preserve - the existing schema-build and version-check timing, including the different - default/enabled source-version paths. -3. Have `acquireUnderlayGroupScopes` use the same load policy/accounting/version - state and one fresh tier state per group. Remove `Options[struct{}]` - construction. Preserve explicit namespace anchors and graph reanchoring. -4. Main acquisition still precedes underlays. Strict assembly still follows - the existing dependency-ready scopes and lower-tier identity information. - Keep group-local repetition separate from load-wide limits, and retain the - existing selected-overlay duplicate-source checks. -5. Replace the ordinary `applyInputLayers` options argument with the precise - main/overlay/protection/policy inputs it uses. A named overlay request will - complete that migration in Stage 9. Remove remaining production - `assembleBaseInput...` variants and the acquisition-only `sequencePolicy`, - `deferAssembly`, and `directiveContext` fields from `Options`. Retain the - placement, destination, and import-default fields used by facade translation. -6. Retain real lower-level test/benchmark capabilities with direct request - construction or test-only adapters. Do not delete meaningful source/limit - cases to make removed helpers disappear. - -**Finish:** no executor or underlay caller reconstructs broad/generic options -for acquisition. The shared path is wired end to end; context and import-flow -extraction continue in later stages. - -Progress notes (2026-10-25): `executeLoadRequest` now creates one acquisition -policy/state and acquires the main input through `acquireInput`; underlay groups -reuse those load owners with a fresh tier state. Removed the generic base-input -assembly variants and the acquisition-only fields from `Options`, and adapted -direct tests to construct named requests. A search-source regression was caught -and fixed during verification. The focused engine selection, facade selection, -consumer example tests, and `GOTOOLCHAIN=local GOPROXY=off go test ./...` passed. - -## Stage 7 — Separate Resolved Filesystem Facts From Access (Complete) - -**Owners:** new engine `filesystem_context.go`, `filesystem.go`, -`filesystem_access.go`. - -1. Add `resolvedFilesystemContext` with the fields selected above. Reuse native - namespace/token identity; do not compare arbitrary `fs.FS` values or equate - equal labels with equal namespaces. -2. Extract host and native fact derivation from `importFilesystem` and - `nativeFilesystemAccess`. Prefer a selected `SourceInfo.Path` when it already - supplies canonical source identity. Preserve source-derived file bases, - explicit in-memory roots/bases, native slash paths, and the root-only absent - host default. -3. Keep separate functions for the directory checks and opening access that - currently occur at each operation. A returned description does not mean - every possible check has run. Do not add validation-state flags or an - all-purpose eager resolver with stronger acceptance rules. -4. Adapt the import worker to consume resolved facts and explicitly own its - access until the existing acquisition handoff closes it. Keep close-error - reporting and primary-error precedence unchanged. Native access borrows its - caller filesystem and must not close it. -5. Keep any old derivation adapters only while genuine callers migrate in - Stages 8–9; remove them at the end of that migration. - -**Finish:** common context facts have one private representation while validation -timing and live resource ownership remain explicit. - -Progress notes (2026-10-25): added shared host/native filesystem descriptions, -separated directory validation and access creation, and migrated import -acquisition to pass resolved facts alongside its owned access. The prior -derivation adapters remain for the validator and overlay migration. Focused -acquisition tests, `GOTOOLCHAIN=local GOPROXY=off go test ./...`, and -`git diff --check` passed. - -## Stage 8 — Carry Context Facts Through Input And Passive Views (Complete) - -**Owners:** engine `load_input.go`, `source_context.go`, `load_request.go`, and -the ordinary context backfill in `load.go`. - -1. Add the optional resolved-context field to `assembledInput`. Populate it - from import acquisition or the existing non-import context validation point. - When context work is deferred for an ordinary overlay, leave it deferred. -2. Replace the source-kind derivation branches of `contextsForAcquiredSources` - with `contextsForResolvedSources`: consume resolved facts and source records, - construct the existing rooted host/native passive view, and compute each - source's relative base using the appropriate path rules. -3. Share relative-base/containment calculation where applicable; retain each - acquired source's own path and source-ID association. Do not substitute the - first source record for the selected main file or attach all values to main. -4. Migrate normal input acquisition, underlay contexts, and ordinary context - backfill. Resolve only still-missing facts at the same lifecycle point as - before. Retained descriptions/views never capture `filesystemAccess` or - `*os.Root`; the existing passive `ResourceContext` contract stays unchanged. -5. Retain failure evidence, root-only absent host context, discovered native - outcomes, and deferred search-base behavior. Update focused context tests - only where private function signatures changed. - -**Finish:** ordinary/underlay acquisition and passive views use the common facts; -selected-overlay callers are the remaining migration in Stage 9. - -Progress notes (2026-10-25): input acquisition now retains resolved filesystem -facts and builds passive contexts from those facts for ordinary and underlay -sources. Migrated the ordinary backfill and reordered-source context test. The -selected-overlay adapters remain for Stage 9. Focused context/acquisition tests, -`GOTOOLCHAIN=local GOPROXY=off go test ./...`, and `git diff --check` passed. - -## Stage 9 — Migrate Selected Overlays Without Merging Their Semantics (Complete) - -**Owners:** engine `selected_overlay.go`, ordinary `applyInputLayers`, and -`assembleLayeredInput`. - -1. Define a private `selectedOverlayRequest` containing the overlay name, - resolved filesystem description, protection rule, sequence policy, and - existing limits/version state. Derive context at the caller's existing - lifecycle point. Pass assembly and source evidence explicitly to the worker. -2. Route ordinary overlays through the main source's context; route layered - overlays through their independent `OverlayFilesystem`. Use named operation - inputs instead of fabricating a `sourceBytes`/empty parsed-main pair merely - to select a context branch. Preserve the actual absent-host root behavior. -3. Return or retain the resolved description used by overlay acquisition and - build overlay passive context from it. Reuse main facts only when that - ordinary operation previously used the same root. Context facts do not share - a live handle between the main import traversal and the later overlay read. -4. Keep the production collection-aware overlay path. Migrate meaningful - lower-level tests to that path with explicit policy/limits, retaining their - budget, version, and failure assertions. Delete the test-only legacy branch - selected by missing variadic policy and its exclusive adapters, rather than - treating optional argument count as a contract. The baseline production entry - points all supply the policy argument. -5. Keep overlay source IDs, duplicate detection, version protection, directory - validation, read/close errors, aggregate accounting, and post-read application - order. Remove superseded context derivation and overlay-forwarding adapters. - -**Finish:** all acquisition/context callers share facts and named inputs while -ordinary and independent layered overlay semantics remain distinct. Run A when -buildable; carry any genuine remaining migration dependencies to the next -integration opportunity instead of adding scaffolding or stopping the sprint. - -Progress notes (2026-10-25): introduced the named overlay request and migrated -ordinary and independent layered overlay acquisition to resolved facts plus -explicit access. Replaced the variadic compatibility branch with the existing -collection-aware path, removed old context adapters, and retained overlay -validation timing, source evidence, budgets, version protection, and close -reporting. Validation group A, consumer examples, the full Go suite, and -`git diff --check` passed. - -## Stage 10 — Give Import Responsibilities Focused Files (Complete) - -**Owners:** engine `imports.go` and new `import_acquisition.go`, -`import_scope.go`, `import_projection.go`. - -1. Put the request entry/access worker and traversal admission/orchestration in - `import_acquisition.go`. Keep `imports.go` as the small common owner for - limits, accounting/route/traversal data, and shared import diagnostics. -2. Move `composeAcquiredInputWithPolicies`, `composeAcquiredScopeWithPolicy`, - pending-item composition, placement reservation, and their item-scope - assembly helpers into `import_scope.go`. -3. Move nested-origin/policy/evidence projection, acquired source destinations, - version-evidence projection, sequence-prefix helpers, and diagnostic-path - projection into `import_projection.go`. Scheduling and graph identity remain - with their existing graph/scheduler owners. -4. Make the acquisition-to-closed-resources-to-assembly handoff visible. Keep - evidence projection based on existing composition applications/position maps; - file movement does not authorize another merge, full-tree replay, or scan of - every unrelated policy for each record. - -**Finish:** maintainers can locate acquisition, local strict assembly, and evidence -work separately. Keep this stage structural; parse/admission extraction follows. - -Progress notes (2026-10-25): moved request acquisition and traversal admission to -`import_acquisition.go`, local strict composition and item-scope assembly to -`import_scope.go`, and origin, destination, version-evidence, and diagnostic-path -projection to `import_projection.go`. Shared limits, traversal data, and import -diagnostics remain in `imports.go`. `./scripts/check.sh` passed. - -## Stage 11 — Extract Import Declaration Parsing Operations (Complete) - -**Owner:** engine `import_spec.go`. - -1. Retain the outer mapping check, exactly-one-selector check, source-order field - walk, applicability checks, and final selector parsing in - `parseImportDeclarationSpec`. -2. Extract boolean assignment, wildcard-set parsing/validation (shared by - `pattern` and `exclude`), placement parsing, and selector-value parsing into - ordinary private helpers. Reuse existing `parseStringSet`, authored-member - sites, and destination/depth parsers. -3. Preserve unspecified pointers and set flags, especially nil versus empty - pattern lists. Keep error paths/locations tied to the authored selector or - list member. Do not reorder consequential checks or replace the selector - matrix with a configurable parser registry. - -**Finish:** field decoding is readable without changing declaration syntax, -selector rules, default resolution, or diagnostics. Keep behavior tests as the -owner; no helper-by-helper assertion inventory is needed. - -Progress notes (2026-10-25): extracted boolean assignment, shared wildcard set -assignment, placement parsing, and selector-value parsing into private helpers. -The declaration parser retains its validation ordering, authored member sites, -and nil-versus-empty presence behavior. Import-focused tests and -`./scripts/check.sh` passed. - -## Stage 12 — Extract Directory Entry Admission And Descent (Complete) - -**Owner:** engine `import_selection.go`, using existing directory/link helpers. - -1. Keep root normalization/deduplication, root ordering, the explicit traversal - stack, candidate union, sorting, exclusion, and required-empty decision visible - in `selectDirectoryImportCandidates`. -2. Extract coherent entry inspection and descent preparation. Return a small - result identifying skip, admitted file, or directory frame to enter, plus a - diagnostic when appropriate. Keep stack/active-directory mutation with the - traversal so its lifetime is visible; do not invent a general visitor API. -3. Preserve hidden/follow/depth behavior, distinct lexical candidates versus - canonical access identity, resource-entry accounting, root-error handling, - and active-path cycle detection. A later non-active alias remains eligible; - exclusions still do not suppress discovery errors. -4. Keep globs and first-match main-file discovery on their existing mechanisms. - Share a helper only where its existing admission semantics actually coincide. - -**Finish:** substantial entry/descent policy is separated from stack orchestration -without changing the selection algorithm or adding repeated root walks. - -Progress notes (2026-10-25): extracted entry inspection and file/directory -admission into a result-returning helper. The selector retains root handling, -stack and active-path mutation, candidate union, sorting, exclusions, and the -required-empty decision. Directory-focused tests and `./scripts/check.sh` passed. - -## Stage 13 — Extract Imported-Source Admission From Recursion (Complete) - -**Owner:** engine `import_acquisition.go` after Stage 10. - -1. Keep `importTraversal.visitPath` as the visible coordinator of containment, - canonical identity, cycle/repetition checks, read/parse, directive/version - processing, placement, and recursive declaration traversal. -2. Extract source admission/read-and-process results into a private - `admittedImport` carrying canonical resource/node identity, source record, - processed payload/declarations, and processing evidence. Return an explicit - optional-missing outcome; do not represent a read failure as absence. -3. Preserve when source records are appended, rolled back for optional missing - input, and retained for failures; preserve source IDs, accounting increments, - version-callback ordering, and inclusion-edge creation on repetition/cycles. -4. Pass the admitted record to mapping/item placement helpers rather than - repeating its payload, declarations, and processing evidence as parallel - arguments to `visitItemSource`. -5. Factor route push/pop and item-scope save/restore only where it makes their - shared lifetime clearer. Keep active graph state across item scopes and - reserve placement nodes once. Item helpers complete before outer placement; - they are strict local peers, not a precedence layer. - -**Finish:** admission decisions and recursion have clear handoffs, with one -read/process path and the original scheduling/evidence algorithms. Run A again -because Stages 10–13 changed the import implementation after the first checkpoint. - -Progress notes (2026-10-25): introduced an explicit admitted-import record and -optional-missing outcome for source read/parse/version processing. `visitPath` -retains containment, identity/cycle checks, graph updates, and recursive -placement coordination; item placement receives the admitted record directly. -Validation group A and `./scripts/check.sh` passed. - -## Stage 14 — Extract Inheritance Child And Parent Preparation (Complete) - -**Owners:** engine `inheritance_schedule.go`, new `inheritance_prepare.go`, and -existing scope/policy helpers. - -1. Extract preparation of the child's directive-free payload, direct origins, - local policy templates, and optional control/control-origin evidence. -2. Extract parent preparation: clone/project policies, lower them against the - child's claims, exclude parent identity from the inherited payload, and - prepare the separate parent history snapshot where currently required. - Keep effective policy templates and transfer construction available to the - combination result rather than reconstructing them afterward. -3. Use focused prepared-child/prepared-parent records of these related values. - Borrow authored/resolved parent inputs read-only; retain existing clone points - for mutable payloads and history. Keep the current preliminary lower-bound - budget decisions at the executor's existing point before materialization. -4. Keep mapping and identified-sequence adapters, the stable Kahn scheduler, and - the parentless/cancelled path semantics intact. Do not flatten nested directives - before their destination-local evaluation. - -**Finish:** the executor no longer embeds child/control and parent/policy -preparation. Actual recursive combination/evidence moves in Stage 15. - -Progress notes (2026-10-25): extracted child/control and parent/policy preparation -into focused records in `inheritance_prepare.go`. The executor retains its -preliminary budget checks, scheduler, and combination path, while history and -transfer inputs stay available to combination. Engine inheritance coverage and -facade inheritance tests passed. - -## Stage 15 — Return One Inheritance Combination Result (Complete) - -**Owners:** new engine `inheritance_combine.go`, `inheritance_collection.go`, -and the shared executor. - -1. Add `inheritanceTemplateResult` as described above and extract the existing - parent/child combination into a helper receiving prepared inputs and the - application policy. Parentless templates return the same result shape. -2. Keep `mergeRecursiveMappings` and `combineSequence` as the combination - mechanisms. Their callbacks update local origins/policy and local replacement - evidence. Have `applyInheritanceSequenceOrigins` return its replacement slice - instead of taking the whole enclosing scope result; append that slice to the - local template result while retaining its updates to the supplied origin maps. -3. Derive sequence history, transferred origins, retired/replaced policies, - identity preservation, and the transfer operation from that same combination. - Preserve `historyParent` and all policy/identity fields on the existing - `inheritanceOperation`; do not rerun a merge or project values to recover them. -4. Keep sequence validation, diagnostic canonicalization, location assignment, - replacement order, and no-parent cloning behavior at their established - semantic points. Keep child identity absent from inherited-parent history but - present with its own origin in the effective child. - -**Finish:** merge callbacks no longer mutate the executor accumulator. The -resolved template and all evidence travel together for Stage 16 integration. - -Progress notes (2026-10-25): added `inheritanceTemplateResult` and moved the -parent/child merge and evidence construction into `combineInheritanceTemplate`. -Callbacks now update the local origins, policies, and replacement evidence; -sequence-origin handling returns replacement records to the template result. -Parentless templates use the same result shape. Engine and facade inheritance -tests passed. - -## Stage 16 — Finish The Inheritance Executor And Accounting (Complete) - -**Owner:** engine `inheritance_schedule.go` and focused finalization helpers. - -1. Reduce the executor to preparation, budget precheck, combination, accumulation, - and final rebuild. Append each template's control before its transfer/history - in the same order as before, and update the indexed resolved value/origin/ - policy/template state once per template. -2. Extract retained-history charging for one template's returned operations and - replacements, reusing the same `retainedNodeCounter` across the scope as - before. Preserve pointer-based shared-node accounting, lower-bound checks, - scope-slot overhead, final expansion check, limits, and diagnostics. -3. Extract final rebuilding and scope policy/origin projection where it clarifies - the coordinator. Reuse `newInheritancePolicyIndex` once for scope localization; - do not return to scanning all policy entries for every slot. -4. Confirm both adapters and nested walking consume the same output. Retain - source-version movement, sensitivity, independent snapshots, assembled order, - cancellation, and policy retirement on actual replacement. - -**Finish:** one readable executor coordinates explicit value/policy/evidence -handoffs, with no parallel inheritance implementation. Run I. - -Progress notes (2026-10-25): extracted per-template retained-history charging and -scope rebuild/origin/policy projection into `inheritance_finalize.go`. The shared -counter, budget checks, final expansion check, and one policy index remain intact. -Both mapping and identified-sequence adapters still use the shared executor. -Validation group I passed. - -## Stage 17 — Extract Substantial Projection Handlers (Complete) - -**Owners:** engine `projection.go`, optionally new `projection_collection.go`. - -1. Keep `projectConfiguration`'s schema/type guard and one `valueOperation`. - Keep `project`'s depth check, pointer/dynamic dispatch, and scalar conversion - choices visible; delegate structs, maps, and sequences to named methods. -2. Move struct member/inline flattening, collision checks, mapping-key charging, - and flattened-field/inline-map secret classification into `projectStruct`. - Preserve declaration order and the current decrement for a flattened - container that is not retained in the output. -3. Move deterministic sorted-map traversal into `projectMap`; keep its cycle - entry active until all recursive children complete. Move slice/array - construction into `projectSequence`, preserving nil-slice handling and the - slice-only cycle guard. -4. All recursive children call the same operation's dispatcher. Avoid duplicate - node charging/depth increments or early cycle release introduced by extraction. - Preserve float/text/duration/secret scalar handling and error paths. - -**Finish:** dispatch and substantial handlers are separately readable, using the -original operation state. Do not refactor `clone`, `decodeValue`, or -`valueBuilder.build` as an additional cleanup project. - -Progress notes (2026-10-25): delegated struct, map, and sequence projection to -methods on the existing `valueOperation`. The dispatcher retains depth checks, -pointer/dynamic handling, and scalar conversion; struct order, map sorting, -flattened-node charging, nil slices, and cycle guards remain in their original -paths. Validation group R passed. - -## Stage 18 — Separate Schema Field Discovery And Assembly (Complete) - -**Owner:** engine `schema.go`, optionally new `schema_fields.go`. - -1. Keep `describeFields` as declaration-order enumeration, exported/ignored/tag - decisions, and dispatch to ordinary or inline handling. -2. Extract inline pointer-chain inspection and inline-struct/map assembly, - retaining recursion/building-state checks and the existing codec-versus- - structural distinction. Reuse the one builder/cache. -3. Make ordinary registration and inline flattening visibly maintain both - `members` and flattened `fields`/`fieldByName`/`inlineMap`. Preserve index - paths, nested secret propagation, ambiguity paths, and the single-inline-map - rule. Share registration helpers only for genuinely identical updates. -4. Preserve descriptor reuse and the physical-member view used by typed cloning. - Do not unify these views or add alternate field/tag policy. Existing schema, - projection, decoding, and cloning expectations remain the behavior baseline. - -Progress notes (2026-10-25): kept declaration-order enumeration, exported and -ignored-field decisions, and tag parsing in `describeFields`; moved ordinary -registration, inline pointer inspection, and inline struct/map assembly into -focused helpers in `schema_fields.go`. Physical members remain distinct from -flattened fields and inline-map metadata. Validation group R passed. - -**Finish:** field discovery and descriptor assembly have explicit responsibilities -without a new registration framework. Run R for both Stages 17–18. - -## Stage 19 — Update Consumer And Internal Navigation (Complete) - -**Owners:** `docs/consumers/README.md`, `docs/internal/overview.md`, -`docs/internal/imports.md`, and affected links/test-owner entries in -`docs/development.md`. - -1. Add a concise consumer choice guide: ordinary `Load` for one selected source, - explicit discovery as a source usable by either entry, and `LoadLayered` for - underlay groups. Distinguish input-owned placement/import/filesystem choices - from load-wide definition/default/version/override choices and independent - layered overlay context. Link current API/config owners for exact rules. -2. Link the existing simple README example, bundled-file/layered examples, - import-focused prompt catalog, and discovered sequence catalog. Keep the - README's simple path simple; do not duplicate complete examples or rewrite - consumer domain rules to showcase new internal types. -3. Update implementation ownership and flow for normalized requests, acquisition - policy/state/tier lifetimes, resolved context versus live handles, import file - owners, and explicit inheritance/projection/schema handoffs. Fix moved-symbol - and behavioral-test navigation and stale comments affected by the refactor. -4. Preserve public contract definitions and accepted ADR decisions. Keep clone - work outside this sprint; its subsequent disposition is recorded under - [OQ-2](consolidation.md#oq-2-how-much-clone-cleanup-should-this-effort-commit-to). - No policy change or new ADR is needed for the selected private refactor. - -Progress notes (2026-10-25): added a consumer entry-point and ownership guide, -linked the existing adoption examples, and documented normalized requests, -acquisition/context lifetimes, import owners, and inheritance/projection/schema -handoffs. Updated test navigation and retained clone review as deferred at this -stage. Documentation links and repository boundaries passed. - -**Finish:** current documentation describes implemented structure, with the -roadmap/plan retaining future work and progress until release cleanup. - -## Stage 20 — Verify And Close The Combined Sprint (Complete) - -1. Search for obsolete import/input forwarding ladders, synthetic `Options` - acquisition construction, `sourceVersionState.importDefaults`, discarded - context derivation branches, unused inverse bridges, and temporary scaffolds. - Remove leftovers along with exclusive test scaffolding. Preserve distinct - operations and real diagnostic conversions; do not delete private code solely - because it resembles another helper. -2. Review the combined call flow: one normalization model, non-generic acquisition, - load versus tier versus traversal state, library-owned acquisition handles - closed before composition/publication, one combination outcome for inheritance - evidence, and one state owner per reflective operation. Review for added whole-tree - traversals, policy scans, or ownership copies without requiring measurements. -3. Account for every selected finding using the table below. Confirm any new - handoff coverage addresses a concrete gap. Existing correctness/algorithmic - regressions, public-boundary tests, and meaningful assertions stay effective. -4. Run the Final validation group, resolve failures, inspect the staged diff and - workspace, and commit the completed remaining work with plain-English - messages. Report changes, checks, limitations, and any unrelated work retained. - Do not tag, push, publish, or update consumer repositories in this sprint. - -Progress notes (2026-10-25): reviewed the combined request/acquisition/tier and -traversal flow, the inheritance result handoff, and the shared reflection state -owners. The named obsolete source-version field, synthetic acquisition-options -construction, and inverse conversion symbols are absent; no additional cleanup -was needed. The final development check and race-enabled full test suite passed. - -| Audit finding | Completion evidence | -| --- | --- | -| 1: imports | Stages 3–6 and 10–13 remove options reconstruction/default coupling and forwarding ladders; acquisition/scope/evidence owners and admission operations are clear. A and final checks pass. | -| 2: inheritance | Stages 14–16 return explicit template results, keep one executor, and preserve accounting and value/history/identity agreement. I and final checks pass. | -| 3: projection | Stage 17 extracts substantial handlers sharing operation state. R and final checks pass. Clone review was excluded from the sprint and subsequently closed without a refactor; see [OQ-2](consolidation.md#oq-2-how-much-clone-cleanup-should-this-effort-commit-to). | -| 4: schema | Stage 18 clarifies ordinary/inline assembly while preserving both descriptor views. R and final checks pass. | -| 5: contexts | Stages 7–9 share resolved facts across input/import/overlay/passive contexts without retaining handles or changing validation timing. A and final checks pass. | -| 6: facade | Stage 2 removes unused inverse conversions/adapters and shares real conversion. F and final boundary/consumer checks pass. | - -**Finish:** the selected roadmap target is implemented, validated, documented, -and reviewable as one complete sprint. No unresolved product/scope decisions are -needed to begin these stages. diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md deleted file mode 100644 index 91da07f..0000000 --- a/docs/roadmap/remediation.md +++ /dev/null @@ -1,206 +0,0 @@ -# Consolidation Review Findings - -Reviewed revision: `557e006a4ab98b666735a71b3845418c702f7cec`. -Comparison baseline: `8ab1568` (the committed implementation plan). - -The review compared the [consolidation target](consolidation.md) and -[implementation plan](implementation.md) with the completed sprint, including -public/private conversions, preflight and acquisition, import selection and -evidence, filesystem contexts, inheritance, projection, schema construction, -changed tests, and documentation. The development check (`./scripts/check.sh`) -and full race suite (`go test -race ./...`) passed with Go 1.26.10 on Linux. - -## 1. Complete The Non-Import Resolved-Context Handoff - -Status: complete (2026-10-25). Priority: medium. Category: implementation gap and maintainability. -No user-facing correctness regression has been established for this finding. - -**Target.** Roadmap internal target 2 and implementation Stages 7–9 select one -resolved filesystem description shared by validation/acquisition and passive -context construction. Stage 8 explicitly requires populating the description -at the existing non-import validation point and resolving only facts that are -still missing later. - -**Evidence.** In [acquisition.go](../../internal/engine/acquisition.go), -`acquireInput` calls `validateFilesystemContext` at line 181 for inputs without -import traversal and with an applicable explicit context. That validator returns -only a diagnostic. Its host/native branches derive a resolved description but -discard it. At lines 220–226, acquisition sees `input.resolvedFilesystem == nil` -and calls `resolveFilesystemContext` again before building passive views. - -For example, an ordinary `Load` of a byte or mapping source with explicit host -`FilesystemContext` and imports disabled resolves/canonicalizes the base and -root during validation, then derives those same facts again for its source -context. The retained description therefore does not complete the intended -validation-to-context handoff. - -Additionally, [filesystem.go](../../internal/engine/filesystem.go) still contains -`validateAbsentHostFilesystemContext` (line 40), with its own root/base -canonicalization, root-defaulting, and containment logic. That derivation -duplicates `resolveAbsentFilesystemContext` in -[filesystem_context.go](../../internal/engine/filesystem_context.go) (line 29). -The two paths already maintain separate containment-diagnostic messages. Graph -caller inspection confirms that the old validator is reachable from -`acquireInput`; it is not unused migration scaffolding. - -**Why address it.** The new model exists, but one important path still discards -and reconstructs its results, while absent-host context derivation has two -production implementations. This leaves part of audit finding 5 unresolved and -preserves the drift risk the consolidation was intended to remove. This is a -structural completion issue, not a claim of measured performance loss or a new -single-lookup guarantee. - -**Required revision.** - -- Have non-import context resolution/validation return or populate the resolved - description, and retain it on `assembledInput` at the existing validation - point. Later passive-view construction and ordinary-overlay handling should - reuse those applicable facts. -- Route absent-host derivation through the common resolved-context owner and - remove the duplicate derivation from the old validator. Keep the existing - directory/access checks and close-error handling at their appropriate points. -- Preserve deferred discovery/ordinary-overlay checks, root-only absent host - context, host/native distinctions, diagnostics, and borrowed filesystem - ownership. Independently required validation must not disappear merely to - reduce filesystem calls. -- Keep internal documentation aligned with the completed handoff. Describe - aggregate accounting as load-wide and repetition/namespace state as - tier-scoped; the new internal overview/import prose currently calls the latter - per-tier accounting despite its distinct role. - -**Verification.** Review the non-import call path to confirm that applicable -resolved facts are carried forward and absent-host derivation has one owner. -Reuse the existing host/native, discovery selected/absent, ordinary/layered -overlay, and passive-resource-context tests. Add a focused behavioral regression -only for a consequential gap exposed by the change; do not assert incidental -canonicalization/stat call counts. Run the development check and applicable -ownership race checks after remediation. - -**Completion.** Non-import validation now returns its resolved filesystem -description and acquisition retains it for passive contexts and ordinary -overlays, including absent inputs. Absent-host derivation uses the common -resolver; the duplicate validator was removed. Validation timing, directory and -root-open/close checks, and the existing absent-input containment diagnostic -are retained. Internal documentation now distinguishes load-wide accounting -from tier-scoped repetition/namespace state. Existing filesystem, discovery, -overlay, and resource tests, the development check, and `go test -race ./...` -passed with Go 1.26.10 on Linux. No new incidental call-count assertions were -added. - -## Algorithmic Audit Scope - -Audited worktree: `d898cb1`, with production code at the reviewed implementation -revision above. This follow-up used source inspection and graph caller traces, -not benchmarks or new runtime measurements. It covered acquisition and import -bookkeeping, scope scheduling and projection, filesystem context handoffs, -inheritance preparation/combination/finalization, and reflective projection and -schema construction. - -Findings 2–3 are retained inefficiencies in paths carried through the refactor, -not algorithmic regressions introduced by the consolidation. They warrant -focused follow-up without changing the selected configuration model. The audit -did not identify an additional correctness defect. Finding 1 was open at the -time of that audit; completion is recorded with the finding above. - -The selected stable Kahn scheduler, deterministic sorting, composition rules, -scope boundaries, and required ownership copies are not reconsidered here. -Previously deferred experiments remain in [future.md](future.md#3-performance-experiments). -The separately completed [clone review](consolidation.md#oq-2-how-much-clone-cleanup-should-this-effort-commit-to) -concluded that no refactor was warranted. These findings do not establish new -public complexity guarantees, input restrictions, performance thresholds, or -release gates. - -## 2. Avoid Searching All Sources For Each Empty Import Selection - -Status: complete (2026-10-25). Priority: low. Category: algorithmic efficiency. - -**Evidence.** `walkDeclarations` calls `recordEmptyImportSelection` for each -optional declaration that admits no source. In -[import_acquisition.go](../../internal/engine/import_acquisition.go), lines -289–300, the latter finds the declaring source by scanning `s.sources` from the -beginning, even though traversal already knows which source owns the declaration. - -**Complexity.** With S acquired sources and D empty declarations, source lookup -takes O(S × D) comparisons in the worst case, excluding path-copying and source -selection itself. A broad main import of N constant-size fragments, each with -one empty optional declaration, visits source positions 1 through N: the sum of -lookup work is Θ(N²), despite linear source and selection-evidence volume and -constant import depth. Existing input budgets bound the workload but do not -remove this avoidable scaling within those budgets. - -**Potential refactoring.** Carry the declaring source's integer index through -the declaration traversal, or maintain a traversal-local source-ID-to-index map. -Either approach makes each evidence-owner lookup O(1), with O(S + D) total -bookkeeping for the map alternative. Keep source order and IDs unchanged. Use -indices rather than retained pointers into the appendable source slice, and -preserve optional-missing rollback and failed-source retention if maintaining -an index across admission. - -**Verification.** Reuse the source-local/detached empty-selection and partial -failure coverage in [import_evidence_test.go](../../internal/engine/import_evidence_test.go). -A broad-fragment behavioral case is useful if existing coverage does not protect -the chosen index handoff. Verify the removed scan by source review rather than -adding timing thresholds or assertions about private lookup counts. - -**Completion.** Declaration traversal now receives the declaring source's stable -integer position, captured at admission and carried through mapping and item -imports. Empty selections append evidence directly to that record without -searching the source slice or retaining pointers into it. The broad-fragment -test covers source-slice growth, missing-source rollback, and parent evidence -after child traversal; existing detached evidence, nested item, underlay, and -partial-failure tests remain effective. Focused tests, the development check, -and race-enabled import/destination tests passed with Go 1.26.10 on Linux. - -## 3. Restrict Import-Scope Edge Preparation To Relevant Nodes - -Status: complete (2026-10-25). Priority: medium. Category: algorithmic efficiency and allocation -scaling. - -**Evidence.** `scheduleAcquisitionScopes` in -[imports.go](../../internal/engine/imports.go), lines 57–64, recursively schedules -the outer scope and every pending item-local scope using the same complete -inclusion graph. `scheduleContributionSlots` in -[import_graph.go](../../internal/engine/import_graph.go), lines 185–213, constructs -a local identity-to-slot map but then scans every graph edge and allocates an -edge slice with capacity `len(graph.edges)` for each scope with at least two -eligible contributions. Most edges need not touch that scope. - -**Complexity.** For Q such scopes and E inclusion edges, preparing the scheduler -inputs takes O(Q × E) edge inspections and requests O(Q × E) edge-slice capacity -over the operation. This is cumulative allocation demand, not a claim that all -those slices remain live together. For N sibling item imports, each with one -distinct mapping helper, Q and E are both Θ(N), making this preparation Θ(N²) -even at constant nesting depth. The selected scheduler itself is not the -problem: each small item-local graph has constant size, and the outer graph -requires its normal deterministic scheduling work. - -**Potential refactoring.** Build a traversal-local adjacency index once after -acquisition, then gather edges for a scope by its participating source identities -and retain only edges whose endpoints both have local slots. Alternatively, -partition the existing edges by their applicable scopes once. Preserve the -current endpoint-membership rule, item-boundary representation, slot placement, -group scheduling, and stable Kahn tie-breaking. For the broad sibling family, -this reduces edge preparation to O(E + P), where P is the total number of local -slots, followed by the unchanged scheduler cost. Keep this an indexing change -around the agreed scheduler, not a different ordering algorithm or a cross-load -cache. - -**Verification.** Reuse item/helper interleaving and grouped import scheduling -coverage in [imports_test.go](../../internal/engine/imports_test.go) and -[load_request_test.go](../../internal/engine/load_request_test.go), together with -existing destination/provenance evidence tests. Cover broad independent item -scopes if a consequential gap remains; verify index construction and edge -visitation bounds by source review without new performance gates. - -**Completion.** Scheduling now builds an importer-to-target adjacency index once -for an acquisition and shares it across outer and pending item scopes. Grouped -scheduling builds its own index once. Local preparation visits only adjacency -lists for participating identities, includes only edges with both endpoints in -the local slot map, and allocates edge storage as needed rather than reserving -the whole graph per scope. For the broad sibling family, preparation is O(E + P) -before the unchanged stable Kahn scheduling. The inclusion graph and cycle -diagnostics remain unchanged. Tests cover excluded endpoints/unselected slots -and broad main/underlay item scopes with retained helper origins and projected -destinations. Focused scheduling tests, the development check, documentation -links, and `go test -race ./...` passed with Go 1.26.10 on Linux. No timing or -allocation thresholds were introduced.