ConfigKit/docs/roadmap/implementation.md

Implement configuration input destinations

Status: planned. Execute Stage 1 through Stage 12 in order, one focused stage per Luna implementation prompt. The feature roadmap owns the approved behavior and rationale; the consumer request supplies the public acceptance scenarios. This document owns execution and completion evidence. Planning baseline: 6353598.

Working instructions

• Start each stage with AGENTS.md, development.md, the applicable policies, and working-tree inspection. Use graph-first code discovery. Preserve unrelated changes and previously completed stages.
• Implement only that stage and its supporting tests/documentation. Mark it complete here with actual commands and evidence, then commit the nonempty validated unit with a short plain-English message. Subsequent stages remain planned. A stage is not complete solely because its narrow tests pass.
• Keep early mechanics private until the complete engine behavior is ready. Root destination fields land in Stage 10. Ordinary root loading must remain working throughout. Do not publish a partially supported public placement API.
• Follow the existing engine/document/YAML boundaries. Reuse strict composition, recursive overlay, sibling inheritance, paths, history, and sensitivity. No dependencies, toolchain changes, alternate mergers, mount registry, per-fragment lifecycle, import-destination syntax, resource discovery, or new release gates are needed. These are scope boundaries, not new prohibitions on separately approved future work.
• Update current-behavior documentation when public behavior lands; keep future claims here or in the roadmap until then. Add the ADR in Stage 1 under the documentation policy's accepted-decision exception.
• Use focused offline tests while iterating; run ./scripts/check.sh before committing production changes. Documentation-only stages need whitespace and local-link checks. Stage 12 adds full ownership/race validation. Tests protect observable behavior, not wrapper allocation or helper-call counts. Reuse coverage rather than repeat an entire loading matrix at every layer.
• No benchmark campaign or public complexity guarantee is required. Prefer straightforward bounded walks and wrapping each acquired contribution once; avoid rescanning all preceding contributions for every new input.
• Do not push, tag, publish, update downstream dependencies, or retire unfinished records. Release compatibility review is separate from publication.

Resolved implementation decisions

1. Add root-owned Input.Destination Path and SourceInfo.Destination Path, mirrored privately in engine input/source records. Extend loadInput and explicit root/engine conversion and cloning. No new loading entry point, public wrapper type, option, or origin field is needed. Ordinary Load continues to supply root placement.
2. The zero path means root. Validate Path.valid() and reject every explicit index segment, including a negative index. Normalize parsed tokens to literal mapping-key segments: /0 means the key 0, not sequence element zero. Empty, dot, dot-dot, slash-containing, tilde-containing, and numeric keys keep existing semantic-path meaning. Do not split dots, interpret filesystem traversal, enforce identifier grammar, or add namespace uniqueness.
3. Reject destination segments equal to configkit or extends: placement would introduce a reserved mapping key with a mapping value. Substrings such as myextends are ordinary keys. This preserves the approved structural rules rather than adding an application naming policy. Source-root import directives are still processed before placement.
4. Validate destination shape/reservations for all declared inputs during grouped option preflight, before reading any source. Reuse CodeOptionsInvalid for invalid destination declarations; include the offending semantic path when representable, with no fabricated physical location. An invalid source still follows existing source-option validation. A valid destination on NoSource contributes no nodes, source record, or budget charges. No extra error type or exact message contract is introduced.
5. Separate acquired source syntax from application placement. Retain original parsed source metadata for filesystem/directive handling. Strip source-root directives, then wrap each application body at the normalized destination before fragment version checks and strict composition. Imports inherit the same destination transitively. Do not wrap both a source application and its recomposed result; every retained application must already use effective coordinates. Content roots remain mappings under existing source contracts.
6. Wrapping a body at Keys("a", "b") creates {a: {b: body}} as private document mappings. Actual body nodes and entries keep their physical source coordinates. Introduced mappings/entries use the contributing document's root location as a documented source fallback, not purported authored coordinates. They have the same contributing source ID and create no synthetic source. Retained history trees remain immutable and separate from assembly mutation.
7. A present empty body creates explicit destination mappings, including an import-only source whose application body is empty. It still cannot clear lower mappings. A missing optional import and NoSource remain absent. Empty source text remains the existing invalid document, not an empty mapping.
8. VersionRule.Path stays an effective application path. Check main once after its own imports compose and before underlays/overlay/CLI, at the existing phase. Imported and underlay claims are checked against placed application content, including scalar/null/sequence ancestors replacing a protected path. A mapping ancestor with no protected descendant is not a version claim. Mounted domain version fields are ordinary data. Existing inheritance, selected-overlay, CLI protection and exact-path numeric conversion remain; no nested blanket conversion or additional fragment checker is introduced.
9. Placement does not alter physical source identity, filesystem namespace, source IDs, acquisition order, imported-from locations, or resource contexts. Repeated physical sources in one strict group still fail even at different destinations. The same physical source in separate tiers retains the existing independent source applications/records and each record's own destination. Overlay and CLI source records have root Destination; they are not mounted merely because main is mounted. Active layered overlays retain their explicit independent filesystem selection and application-root meaning.
10. Value-level paths use effective coordinates. Keep genuine source-local import-directive diagnostics and parse/read failures at their original locations; do not invent a logical child path when parsing cannot identify one. Prefix existing application-relative diagnostic paths exactly once where available. Preserve structured codes, related physical locations, deterministic ordering, and value-free messages.
11. Use current depth/node limits, not a new destination cap. With root depth zero, each destination segment adds one level and, under countInputNodes, one mapping node plus one key. Retain current raw input accounting (including directives); charge only the additional 2 * segmentCount synthetic nodes per acquired body, once, to the shared input budget. Placement adds no bytes, acquired sources, import nesting, or glob visits. Validate placed application depth and per-document node bounds before composition; share accounting across inputs and imports. Precheck wrapper depth/count before allocating wrappers. Use existing source/import limit diagnostics in their acquisition contexts; retain projection, expansion, and history bounds independently. Root placement adds no charge or new rejection. NoSource adds none either.
12. Cloned source records include independently owned destination paths at every existing publication boundary: results, load errors, metadata/inspection, callback input information, and root conversion. Passive resource contexts remain keyed by the real source identity; inspection must work without an open configuration handle after load. This does not extend caller-owned filesystem or callback resource lifetimes.

Stage 1 — Record the decision and capture the baseline

Status: planned. Scope: decision record and evidence, no feature code.

1. Read the roadmap, request, relevant policies, and ADRs 0001, 0003, 0006, 0008, and 0009. Inspect the owners identified in the roadmap and the function map below. Confirm the baseline and worktree before starting.
2. Run ./scripts/check.sh; record the commit, Go version, and results here. Locate existing coverage for root loading, strict underlays, version ownership, repeated sources, history, resource contexts, and detached records.
3. Write docs/adr/0011-input-destinations.md after confirming the number is unused. Record the approved input-placement decision as Accepted, with the roadmap owning implementation status. Follow the repository ADR format. Focus on ConfigKit, put rejected options only in Alternatives Considered, and avoid negative parallelism elsewhere. No PromptKit-specific semantics.
4. Record the compatibility assessment under the release policy: adding fields preserves zero-value/keyed-literal use, but callers using unkeyed literals for Input or SourceInfo must migrate to keyed literals. Treat the feature as minor-release work; select a version and write its note only during separately authorized release preparation.

Validate: local-link test and whitespace checks. Done: baseline evidence and the decision are recorded; implemented contracts remain unchanged.

Stage 2 — Add private destination normalization and wrapping

Status: planned. Scope: pure private mechanics with bounded unit tests.

1. Add a small engine helper file for normalized mapping-only destinations and constructing placed document bodies. Follow decisions 2–7; keep helpers private and avoid introducing a second path abstraction.
2. Validate invalid paths, explicit indices, and exact reserved tokens. Normalize parsed numeric tokens into keys, preserving literal text and pointer escaping.
3. Build wrappers without mutating the supplied tree. Preserve body locations; give introduced entries/mappings the source-root fallback. Implement depth and node measurement/prechecks usable with existing limits, including the extra synthetic-node count; aggregate charging is wired in Stage 7.
4. Cover root identity/equivalence, nested empty mapping, literal/escaped/numeric keys, invalid index/path, exact versus substring reserved keys, owned paths, unchanged body coordinates, and bounded wrapper checks. Do not count helper invocations or fix an allocation strategy in tests.

Validate: focused destination/path tests and the everyday check. Done: the mechanism is usable privately; no loader accepts a new public field yet.

Stage 3 — Integrate private non-import input placement

Status: planned. Scope: engine request plumbing and ordinary acquired bodies.

1. Add destination fields to engine Input, loadInput, and SourceInfo. Preserve ownership through engine LoadLayered request construction, cloneSourceInfo, and source-record copies. Keep the root facade unchanged.
2. Extend grouped preflight to validate/normalize destinations before source reads. Use the existing diagnostic accumulator and options-invalid category. Ordinary Load supplies root placement through its existing request adapter.
3. Thread placement into assembleBaseInputTracked without confusing raw parsedSource with effective assembledInput.document. Preserve the root path through existing wrappers/callers. For imports-disabled acquisition, validate disabled directives on raw content, place the body, and retain the destination on the acquired source record. NoSource still short-circuits.
4. Use engine loads to cover mounted main and underlay bodies, omitted versus explicit root destinations, empty mapping versus NoSource, and preflight rejection without consuming a test-owned reader. Reuse current source-kind acquisition tests; full source equivalence is verified publicly in Stage 10.

Validate: destination, grouped-load, source, and existing ordinary-load tests; the everyday check. Done: private non-import placement reaches assembly, with root behavior unchanged. Imports are the next private integration step.

Stage 4 — Carry placement through import acquisition

Status: planned. Scope: source-local directives and placed imported applications.

1. Thread a normalized destination through loadImportedDocumentsTracked, loadImportedWithAccessTracked, and importTraversal, retaining root defaults for current private callers. Place the selected body's application content after splitImportDirectives; do the same for every imported body.
2. Set placement on selected/imported source records, including error records once a source has been acquired. Keep ImportedFrom at the physical directive. Run imported fragmentVersionClaim against placed bodies and retain placed bodies in state.documents before composeDocuments builds applications.
3. Leave filesystem joins, containment, glob traversal, optional missing inputs, active cycles, visited identities, and acquisition IDs/order unchanged. Destination must not become part of the repeated-source identity key.
4. Test native and host transitive imports, nested source-root directives, import-only bodies, and mounted imported protected versus ordinary version paths. Verify a repeated file at two destinations in one group still fails, including an import/direct-input alias. Reuse cycle/containment regressions.

Validate: focused destination/import/glob/grouped tests and everyday check. Done: every retained imported contribution has effective placement and physical source identity, with no double wrapping or application-level directive leakage.

Stage 5 — Complete strict tiers and main version ownership

Status: planned. Scope: assembly and version interactions, not a new merger.

1. Review acquireUnderlayGroup: version checks must use placed content, and documents taken from acquired.composition.applications must already be placed. Recompose strict peers with ordinary composeDocuments; preserve per-source applications, origins, UnderlayIndex, and input accounting.
2. In executeLoadRequest, main's version check uses its placed main/import composition at the existing point. assembleLayeredInput recursively overlays completed underlay groups beneath main, selected overlay, and CLI as before. Do not mount the final assembly or prepend main's destination to upper tiers.
3. Test disjoint same-destination peers, different destinations, equal and unequal conflicts, ancestor/descendant destinations, and a strict conflict despite an upper override. Verify recursive mapping overlays, atomic sequences, null, explicit false/zero, and version-only domain overrides retaining lower fields.
4. Exercise mounted main with a nested VersionRule, absent root declarations, imported/underlay direct and ancestor claims, and nonclaiming mapping ancestors. Check the hook runs once and does not inspect each fragment independently. Preserve exact configured-path integer conversion and ordinary nested typing.

Validate: destination/composition/overlay/version/grouped tests and everyday check. Done: placed and authored equivalents agree on values and conflicts; main ownership and lifecycle timing are preserved.

Stage 6 — Reconcile diagnostic coordinate systems

Status: planned. Scope: useful paths without fictional source locations.

1. Audit early acquisition, import, fragment-version, and composition failures. Use one narrow path-prefix translation where diagnostics genuinely identify application content; do not blanket-prefix every diagnostic or double-prefix diagnostics already produced by placed trees.
2. Preserve syntax/read/directive locations and related import sites. Value conflicts and later typed/resolver/validation failures use effective paths but original content coordinates. Invalid introduced structure identifies the destination declaration rather than a nonexistent source wrapper line.
3. Add focused assertions for a mounted conflict with two real sources, a mounted decode or validation error, an imported version error, invalid destination, and malformed source-local directives. Test stable codes/paths/locations and nil results, not whole messages or callback choreography.

Validate: destination/diagnostic/import/version/decode tests and everyday check. Done: diagnostics consistently distinguish application paths from source syntax.

Stage 7 — Complete shared placement accounting

Status: planned. Scope: resource bounds with no new public limit system.

1. Wire Stage 2's prechecks and synthetic-node charges into both tracked ordinary acquisition and import traversal, using importAccounting and current limit structures. Charge each source body once, including empty import bodies.
2. Preserve existing raw byte/source/node accounting and imported directive inspection. Additional wrappers count in per-body depth/node checks and shared aggregate nodes; recomposition must not charge them again. Ensure selected overlays and CLI start from totals including placement charges.
3. Test below/at/above relevant bounds using small private limit fixtures where existing seams permit; add a narrow private seam if necessary. Cover deep body plus destination, many shallow mounted inputs, imported wrappers, and empty mappings. Verify root placement and NoSource do not add charges. Preserve history/expansion limits; do not turn fixture values into defaults.

Validate: destination/limit/import/override tests and everyday check. Done: placement cannot bypass current budgets and introduces no unrelated caps.

Stage 8 — Preserve provenance, history, and source resource contexts

Status: planned. Scope: engine inspection and retained evidence.

1. Audit compositionOrigins, completeCompositionOrigins, application replay, metadata construction/finalization, and source-record cloning against placed applications. Prefer feeding placed trees into existing processing rather than adding another path-offset index or rewriting final metadata afterward.
2. Verify explicit destination containers, actual leaves, composite origins, superseded/removed descendants, and root/source fallbacks. Historical and effective indexes keep independently owned paths and existing shape semantics.
3. Preserve source-ID resource contexts through imports, surviving lower-tier values, and inheritance. Test completed-result and resolution-time context queries against real relative files, without making ConfigKit open the referenced application resource or retain loading handles.
4. Exercise the same physical source in separate tiers with different placement, inspect each source record, and verify independent destination paths. Repeat inspection after comparison and independent loads to detect retained mutation.

Validate: destination/metadata/provenance/history/resource tests, go test -race ./internal/engine, and everyday check. Done: inspection explains effective placement and physical origin without consumer reconstruction.

Stage 9 — Verify final inheritance, sensitivity, and lifecycle integration

Status: planned. Scope: complete private engine behavior before facade exposure.

1. Resolve sibling inheritance only after full source precedence using the existing expansion path. Cover a child and parent supplied by separate peers, nested destination-local siblings, cancellation, and an inherited relative resource retaining its declaring context. Cross-collection search is unchanged.
2. Reuse existing inheritance-version, overlay, and CLI guards; add targeted mounted cases for actual protected paths and their ancestors. Do not reject unrelated inheritance merely because it shares a mapping ancestor.
3. Test effective-path secret predicates, field tags and recorded dependencies, superseded classified values, safe effective output/history/explanation, and bilateral diff redaction using disposable canaries. Compare mounted versus explicitly wrapped loads for equal effective values and no differences.
4. Verify defaults, decoding, resolution, secret lookup, and validation execute once on the final shape; failures return no usable result. Reuse existing ordinary-load lifetime/ownership coverage and add mounted protection where placement creates a distinct risk.

Validate: destination/inheritance/secret/sensitivity/explain/diff/lifecycle tests and everyday check. Done: all approved engine semantics work together without per-fragment loading or a post-return processing pass.

Stage 10 — Expose the root-owned facade and public acceptance tests

Status: planned. Scope: the two public fields and complete consumer behavior.

1. Add Input.Destination Path and SourceInfo.Destination Path to root types. Extend toEngineInput, fromEngineSourceInfo, toEngineSourceInfo, and all source publication paths with owned conversion. Run existing public/import boundary tests; expose no engine nodes, aliases, or bridge methods.
2. Document zero/root behavior, mapping-only paths, main/underlay parity, inherited import placement, source-coordinate fallback, and source ownership beside declarations. Update the configuration contract for implemented input destinations, preserving canonical ownership of shared semantics.
3. Add external-package public-load coverage for equivalent host file, native fs.FS file, byte, reader, and Mapping sources. Cover default/root destinations, literal escaped/numeric keys, empty mapping, NoSource, and invalid declarations. Include one public mounted main case, not just underlays.
4. Add representative public tests tying final configuration to metadata, source placement, history, resources, callbacks, and safe diffs. Mutate returned record copies and repeat queries/loads to confirm detachment. Reuse engine matrices; public tests demonstrate supported integration rather than every private branch. Confirm failed loads expose detached source destinations.

Validate: go test . -run 'Destination|Facade|PublicFacadeBoundary|DependencyImportBoundary', go test -race ./..., and everyday check. Done: the requested workflow is usable entirely through ConfigKit's supported import path and documented surface.

Stage 11 — Add maintained simple and collection-style consumer examples

Status: planned. Scope: executable adoption examples with consumer-owned types.

1. Add a minimal mounted-fragment example using Input.Destination in an ordinary layered load. Show literal keys and root-default compatibility without introducing consumer parsing or generic merge helpers.
2. Add one representative collection assembly under examples/consumers: peer prompt/profile fragments, collection metadata with ordinary version, a source-root import, local sibling inheritance, and an intentional higher-tier version-only override retaining the body. Keep the application VersionRule main-owned. Use a controlled native filesystem or temporary files.
3. Demonstrate effective-path inspection plus source-relative resource lookup during resolution and from the result, including an inherited/surviving field. Application code owns opening/interpreting the resource and any qualified IDs.
4. Add tests for final values, presence, actual source placement/coordinates, resource context, and expected conflict behavior. Link maintained examples from the consumer guide; add a small executable root example if useful for discovery without duplicating the complete scenario.

Validate: go test ./examples/consumers, applicable Example tests, local links, and everyday check. Done: simple and representative consumers substantiate the same configuration model and need no compensating structural pass.

Stage 12 — Reconcile documentation and audit the complete implementation

Status: planned. Scope: final integration audit and evidence, not release work.

1. Update README navigation, consumer guidance, the component overview, and development reading/test maps. Add ADR 0011 to navigation. Update architecture policy only as needed to recognize placement within existing assembly and source ownership, linking precise behavior to the configuration contract. Do not change unrelated policies, deferred boundaries, or published notes.
2. Audit every resolved decision, roadmap behavior, and request acceptance row against implementation and tests. Record the corresponding test names here; for reused coverage explain the behavior it proves. Do not mark complete based on green tests that do not exercise mounting.
3. Review the complete baseline-to-candidate diff for accidental strengthening: extra naming rules, namespace uniqueness, main-only restrictions, changed merge/version semantics, invented source-read ordering guarantees, new limits, or expanded secret policies. Remove temporary helpers/unsupported branches left by staged integration. Confirm original source coordinates remain real.
4. Run ./scripts/check.sh and GOTOOLCHAIN=local GOPROXY=off go test -race ./.... Review source-kind and placement ownership coverage, error disclosure, complete-on-return handling, and root/private boundaries. Record commit, Go version, results, and any limitations. No release command, CI waiting, benchmark, or downstream upgrade.
5. Mark this plan and the roadmap implemented only when code, public examples, and owning contracts agree. Retain completed planning evidence until separately authorized release preparation.

Done: one complete layered load supports input destinations with coherent values, diagnostics, inspection, limits, and source ownership across the approved surface.

Implementation owner map

Use graph tools to locate these symbols rather than assume line numbers:

• Public input forwarding: layered.go, records.go, conversion.go (source conversions), and existing facade tests.
• Private request and lifecycle: internal/engine/layered.go, load_request.go (loadInput, acquireUnderlayGroup, assembleLayeredInput), load.go (executeLoadRequest), and load_input.go (assembleBaseInputTracked).
• Directives and imports: directives.go (splitImportDirectives, fragmentVersionClaim), and imports.go (visitPath).
• Shared document semantics: compose.go, document_walk.go (countInputNodes), path.go, parse.go, and value_operation.go limits.
• Inspection: metadata.go, history.go, explain.go, resource_context.go, inheritance_origin.go, render.go, and diff.go; source copies live in engine records.go. Confirm current filenames through the component overview.

Open Questions

None required for implementation. The four policy choices are approved; this plan selects API spelling, validation timing/error category, coordinate handling, and accounting within them. Surface a newly discovered incompatibility or scope change with evidence rather than resolving it by silently narrowing the feature or strengthening an unrelated contract.