ConfigKit/docs/roadmap/implementation.md

# Implement relative import destinations

Status: in progress. Execute Stage 1 through Stage 9 in order, one focused stage
per Luna implementation prompt. The
[feature roadmap](relative-import-destinations.md) owns the approved target and
syntax; this document owns implementation decisions, sequencing, and evidence.

Planning baseline: `b2db323`, including input-destination remediation at
`315f464`. The previous completed input-destination implementation record is
preserved at `315f464:docs/roadmap/implementation.md` in Git history. The broader
[consumer request](promptkit-feature-request.md#requested-addition-relative-import-destinations)
also proposes independent specification headers, which are outside this plan.

## Working instructions

- Start with `AGENTS.md`, [development.md](../development.md), applicable policies,
  and working-tree inspection. Use graph-first code discovery and preserve
  unrelated changes. Read existing import and placement contracts before edits.
- Implement one stage at a time with its relevant tests/documentation. Record
  actual commands, revisions, results, and limitations here; mark only completed
  stages and commit each validated nonempty unit with a plain-English message.
- Follow the root-owned facade and private engine/document/YAML boundaries.
  Reuse paths, placement, composition, inheritance, version ownership, history,
  and resource contexts. No public Go entry point, source-header API, namespace
  registry, or second merger is needed.
- Specification-header checking and consumption remain deferred. Ordinary
  `version` data, the main-owned checker, exact-path numeric conversion, and
  callback timing keep their existing behavior. Examples must work without the
  broader request's combined header example.
- Roadmap exclusions define this delivery, not new permanent prohibitions. Do
  not add directory discovery, caps, key-name grammars, root-reset syntax,
  arbitrary callbacks, or stronger security/release guarantees.
- Keep Stage 2 behavior-preserving before Stage 3 accepts new syntax. Once
  accepted, syntax must work through existing import entry points, not only path
  imports or only root loads.
- Use focused offline tests while iterating and `./scripts/check.sh` before
  committing production changes. Documentation-only work needs whitespace and
  local-link checks. Ownership race checks belong to Stages 6 and 9. Reuse
  unchanged coverage; do not assert helper-call counts, allocation identity,
  exact diagnostic prose, or a cross-product of already protected cases.
- No independent benchmarks, performance thresholds, dependency changes,
  toolchain upgrades, or new release gates are required. Prefer copying path
  segments once per declaration and existing bounded tree processing; introduce
  no public complexity guarantee.
- Do not push, tag, publish, upgrade downstream dependencies, rewrite accepted
  ADR decisions, or remove unfinished work. Completed records retire during
  separately authorized release preparation.

## Resolved implementation decisions

### Syntax and validation

1. Add one optional import-declaration field, `destination`, beside existing
   `path`/`glob` and `optional`. Its value is a sequence of strings. Omission
   and `[]` mean zero additional segments. Null, a scalar (including a pointer
   string), and any non-string member produce `CodeImportInvalid`. No coercion:
   quote numeric-looking keys when YAML would otherwise parse them as numbers.
2. Construct a relative `Path` of literal mapping keys and reuse
   `normalizeDestination`. Empty strings, dots, dot-dot, slash/tilde-containing,
   and numeric-looking strings keep their literal meaning. Existing exact
   `configkit` and `extends` reservations apply. Do not trim keys, split
   dots/slashes, clean filesystem paths, infer sequence indices, impose a
   collection grammar, or add syntax aliases.
3. Validate shapes/reservations in `parseImportDeclaration`, before traversal
   can suppress an optional missing target or unmatched glob. Invalid shapes
   identify source-local `/configkit/imports/<index>/destination`; invalid
   members identify its indexed child and actual member location. Follow
   `nodeImportDiagnostic` conventions and keep messages value-free. Preserve
   unknown-field, duplicate-key, path/glob, and optional rules. No new public
   error code/type, all-errors aggregation, or error-order guarantee is needed.

### Traversal and source records

4. Add a private relative-destination field to `importDeclaration`. Pass the
   immediate parent's effective destination explicitly through
   `walkDeclarations`. Append each declaration's relative path once and pass
   that already computed child destination to `visitPath`. Recursive walks use
   the child's effective destination. Remove the single
   `importTraversal.destination` context rather than mutating/restoring it.
5. Add a small private destination-append helper in `destination.go`, unless a
   semantically suitable general helper already exists. Copy semantic segments
   into owned storage without pointer-string round trips, filesystem operations,
   or repeatedly extending a growing prefix with `Path.Key`. Its inputs are
   validated mapping paths. Place each body once, not once per ancestor.
6. Every glob match receives the same computed child destination.
   `expandImportGlob` remains about physical matching. Turning a match into a
   path declaration must not append its suffix again. Nested context must not
   leak into siblings, later matches, or later inputs.
7. Use the child destination for source records, placement, fragment-version
   checks, wrapper accounting, and recursion. Keep source-record creation at its
   existing point so imports that later fail retain their effective destination.
   `ImportedFrom` still identifies the physical path/glob declaration, not the
   destination list. Source IDs, namespaces, acquisition order, active cycles,
   physical identity, and strict-group repetition rules remain unchanged.
8. Remove the post-traversal assignment in `assembleBaseInputTracked` that
   overwrites every imported source's destination. Set the selected entry's
   per-load `parsedSource.source.Destination` before import-context setup so
   early `sourceInfos` fallback copies retain entry placement. Leave correctly
   assigned children untouched. Preserve `315f464`'s Mapping value-error
   rebasing and root behavior; do not rebase syntax/read/directive failures.

### Assembly, limits, and compatibility

9. Strip source-root directives before placing each application body and
   checking imported version ownership. Existing composition/history retain the
   placed source applications. Strict peers conflict, tiers recursively overlay,
   sequences stay atomic, and sibling inheritance follows precedence. Do not add
   post-load merging or per-fragment decoding, defaults, callbacks, or headers.
10. Reuse current depth, node, source, byte, import-depth, glob-entry, projection,
    and history limits. Charge each actual body's wrappers once using its full
    accumulated destination, not only the suffix. Preserve raw source accounting.
    Placement adds no source bytes, source records, import nesting, or glob
    visits beyond existing accounting. A glob declaration/ancestor walk itself
    adds no payload. Omitted/empty suffixes reproduce baseline charges.
11. Malformed syntax fails even when optional input is absent. Resource charging
    remains tied to actual contributions: a valid optional no-match creates no
    source or wrappers; an actual empty body materializes its full destination.
    Use existing placed-body bounds and source/import diagnostics. Add no
    destination-length limit or rejection solely because an absent optional
    contribution's unused path would be deep.
12. Preserve authored locations and source-root fallback for synthetic mappings.
    Resource contexts remain physical and survive inheritance/overlays.
    Metadata, history, sensitivity, rendering, diffs, detached records, and
    complete-on-return behavior reuse their existing mechanisms; no new
    inspection API or retained loading handle is needed.
13. This is additive YAML functionality. Existing valid imports and Go
    signatures keep their contracts. Treat it as minor-release work; version
    selection, notes, retirement, and publication remain separate tasks. Do not
    edit published release notes.

## Stage 1 — Capture the baseline and record the extension

Status: complete. Scope: baseline evidence and one ADR; no production code.

1. Read the feature roadmap, relative-import request/acceptance rows, applicable
   policies, ADR 0011, and the owner map below. Confirm the worktree and baseline.
   Run `./scripts/check.sh`; record revision, local Go version, results, and
   environmental limitations here.
2. Locate existing parser, host/native import, glob, repetition, version,
   placement-accounting, resource, history, and facade coverage. Root and
   inherited placement are the compatibility baseline; header rows in the
   broader request are not acceptance requirements for this plan.
3. Write `docs/adr/0012-relative-import-destinations.md` after confirming the
   next unused number. Record the selected syntax and immediate-parent-relative
   semantics as an Accepted extension to ADR 0011. Use the repository format.
   Keep implementation status in the roadmap and rejected choices only under
   Alternatives Considered. Focus on ConfigKit, without downstream semantics.
4. Note the extension in ADR 0011's status/navigation if needed; preserve its
   accepted decision text. Record the additive minor-release assessment without
   selecting a version or adding release gates.

Validate: whitespace and local-documentation-link checks. Done: baseline and
decision are recorded; implemented behavior is unchanged.

Baseline evidence (2026-10-14): revision `934c1022e038115475c1d33dd85216855b8be522`
on `main`; the worktree was clean and four commits ahead of `origin/main`.
Local toolchain: `go1.26.8 linux/amd64`. `./scripts/check.sh` passed, including
format/whitespace checks, all Go tests, vet, build, examples, local links, and
package-boundary checks. No environmental limitations. Existing coverage is
centered in `internal/engine/parse_test.go`, `imports_test.go`, `glob_test.go`,
`native_filesystem_test.go`, `version_path_test.go`, and the `destination_*_test.go`
files; public/lifecycle coverage includes `facade_destination_test.go`,
`facade_layered_test.go`, `facade_version_test.go`,
`facade_resource_context_test.go`, `facade_inheritance_test.go`, and
`facade_diff_test.go`, with engine resource and history coverage in
`resource_context_test.go` and `inheritance_history_test.go`.

Done: baseline and decision are recorded; implemented behavior is unchanged.

## Stage 2 — Refactor traversal to explicit per-document placement

Status: complete. Scope: private context plumbing; old syntax remains unchanged.

1. Add the relative-destination slot to `importDeclaration`, initially zero for
   all parsed declarations. Add the destination-append helper and bounded
   literal-path/ownership tests using existing Path conventions.
2. Give `walkDeclarations` the parent's destination, compute each child once,
   pass it into `visitPath`, and pass that child path to recursive walks. Remove
   the traversal's uniform destination field and update private callers/tests.
   Initialize the first walk from the entry Input destination.
3. Use the per-call destination in records, `placeDocument`, version checks,
   and wrapper accounting. Leave glob matching unchanged; each match receives
   the computed child path without re-appending relative segments.
4. Narrow source-record assignment per decision 8. Check import-context/open
   failures, imported parse failures, successful imports, and the completed
   Mapping error regression. Exported fields/signatures remain unchanged.
5. Add one small engine fixture with private nonzero declaration destinations
   proving accumulation and sibling/glob isolation before parsing is enabled.
   Public syntax and complete acceptance follow Stage 3.

Validate: `go test ./internal/engine -run 'Import|Glob|Destination|Grouped'`,
`go test . -run 'Destination|FacadeLayered|Mapping'`, and `./scripts/check.sh`.
Done: baseline loads remain equivalent; all placement consumers use explicit
document context and no uniform setter can erase per-import placement.

Evidence (2026-10-14): based on `a8beb3e62014cb960f3c922c79cd98bd04a657e7`,
using `go1.26.8 linux/amd64`. The focused engine command, the root destination,
layered, and Mapping regression tests, and `./scripts/check.sh` all passed. The
private traversal fixture verifies nested accumulation, sibling isolation, and
one shared destination across glob matches; failure fixtures verify entry
placement survives import-context and imported-parse errors. No limitations.

## Stage 3 — Accept and document the selected YAML syntax

Status: complete. Scope: directive parsing and first complete public behavior.

1. Parse sequence-of-strings `destination` in `parseImportDeclaration`.
   Construct literal keys, normalize with existing destination validation, and
   retain the relative path. Use `CodeImportInvalid` and actual declaration or
   member nodes for failures; retain path/glob locations independently.
2. Add table-driven parser coverage: omission, `[]`, valid nonempty lists,
   empty/slash/tilde/dot/numeric strings, null/scalar/non-string failures, and
   exact reserved versus substring keys. Preserve mutual exclusion of path/glob,
   unknown-field failures, and optional's boolean type. One invalid optional
   missing path and one unmatched glob prove validation precedes suppression;
   no full cross-product is needed.
3. Add public ordinary-root `Load` and mounted layered entry cases. Include
   path/glob imports, a child with its own import, a subsequent sibling, and
   multiple matches. Assert final values and individual source destinations;
   context leakage must fail the test.
4. Update canonical import/input-destination sections in `docs/config.md` when
   syntax lands. Qualify uniform inherited-placement claims with omission;
   explicit suffixes accumulate from the immediate parent. No Go API change.

Validate: focused engine import/glob/destination tests, public import-destination
tests, local links, and everyday check. Done: syntax works across existing
import-capable entry points and acquisition forms, not a silently ignored or
root/path-only subset.

Evidence (2026-10-14): based on `49c0c6d908c0746a0ae49a332691c383ac3ff941`,
using `go1.26.8 linux/amd64`. Engine import/glob/destination tests, root public
destination and mounted-load tests, `TestLocalDocumentationLinks`, and
`./scripts/check.sh` all passed. Coverage includes literal keys, malformed
shapes and members with source-local coordinates, validation before optional
missing/unmatched suppression, and ordinary-root plus mounted layered loads.
No limitations.

## Stage 4 — Verify composition and effective version ownership

Status: complete. Scope: assembly interactions; reuse existing algorithms.

1. Cover disjoint same-destination contributions, different destinations,
   ancestor overlaps, and collisions with explicitly wrapped content. Equal and
   unequal non-mapping overlaps remain strict conflicts. Verify an upper tier
   cannot repair a lower strict-group conflict.
2. Exercise deliberate upper-tier overrides: recursive maps, retained omitted
   fields, atomic sequences, and an ordinary version-only override retaining
   the rest of the prompt. Overlay/CLI paths stay application-rooted.
3. Verify accumulated placement governs imported protected-path/ancestor claims
   and ordinary nested `version` fields remain payload. Main's hook runs once
   at its existing point; exact-path numeric conversion does not spread. Reuse
   current inheritance/overlay/CLI version guards where they prove unchanged
   behavior.
4. Test repeated physical sources at distinct logical destinations, including
   direct-input/import aliases and cycles reached through changed placement.
   Keep original inclusion routes/locations; add no namespace uniqueness.

Validate: `go test ./internal/engine -run 'Import|Destination|Compose|Overlay|Override|Version|Grouped'`
and everyday check. Done: relative placement changes neither tier semantics nor
the main-owned version contract.

Evidence (2026-10-14): based on `4635ef08d5cbeec0b1a1f316cff59cf99e70eba0`,
using `go1.26.8 linux/amd64`. The engine import/destination/composition/overlay/
override/version/grouped suite, root destination/layered/version suite, and
`./scripts/check.sh` all passed. New coverage exercises strict peer behavior,
wrapped-content conflicts, lower-tier conflict preservation, nested protected
version paths, main-checker timing, and repeated/cyclic physical sources under
changed destinations. Existing destination overlay tests continue to verify
root-scoped upper tiers. No limitations.

## Stage 5 — Verify accumulated resource accounting

Status: complete. Scope: limit regressions and any necessary narrow correction.

1. Audit `placeDocument`, `destinationWrapperNodeCount`, `importAccounting`,
   and composition totals. Each body uses the full effective destination for
   depth/wrapper nodes, charged once. Keep raw directives in acquired accounting.
   Synthetic wrappers add no bytes, source count, glob visits, or import depth.
2. Add below/at/above cases with small private limit fixtures where existing
   seams permit. Cover a short suffix whose accumulated path exceeds placed
   depth, empty bodies at nested destinations, and several glob matches sharing
   aggregate nodes. Derive counts from fixture helpers, not copied defaults.
3. Verify valid optional missing/unmatched sources create no source record or
   wrappers, including when their unused destination would be deep. An actual
   empty mapping is explicit. Reuse Stage 3 for malformed optional declarations.
4. Confirm later overlays/CLI and history/expansion checks see correct totals.
   Reuse baseline tests; introduce no cap, performance assertion, or new failure
   precedence merely for this feature.

Validate: focused import/destination/limit/override tests and everyday check.
Done: accumulation cannot bypass current bounds or falsely charge absent input.

Evidence (2026-10-14): based on `430985c6293ca8f9151149f1af34481ac325eed3`,
using `go1.26.8 linux/amd64`. The focused engine import/glob/destination/grouped/
overlay/override/history/inheritance suite and `./scripts/check.sh` passed.
Exact, below, and above node-budget fixtures cover nested empty imports and
multiple glob matches using their accumulated destinations. Additional cases
verify accumulated depth bounds and that absent optional path/glob contributions
at unused depth add no source or wrapper charge. Existing
`TestSelectedOverlayAndCLIUsePlacementNodeCharges` continues to cover later
overlay/CLI accounting. No limitations.

## Stage 6 — Preserve diagnostics, source inspection, and resource contexts

Status: complete. Scope: coordinates, detached records, and resource ownership.

1. Check entry/child/grandchild source records' individual destinations, kinds,
   IDs, physical paths, and ImportedFrom locations. Include public failure
   records after imported parse and effective-path version/composition errors;
   preserve existing reporting completeness.
2. Directive failures identify source-local paths and actual locations.
   Composition/typed-validation failures use effective paths with authored
   content coordinates. Keep syntax/read failures and Mapping rebasing at their
   established boundaries; never prefix all diagnostic paths.
3. Use temporary host files and a native-filesystem fixture to prove physical
   joins stay relative to the declaring file as destinations change. Exercise
   byte/reader/mapping entries with existing explicit filesystem contexts where
   applicable; imported files keep existing acquisition contracts. Reuse source
   equivalence tests rather than duplicate a whole matrix.
4. Through resolution/result APIs, follow a direct and inherited resource field
   to its original filesystem/base, including a lower value surviving override.
   Application/test code owns opening the resource.
5. Mutate returned copies, repeat queries, and compare independent loads to
   verify detached destination storage. Confirm library acquisition handles
   close before return and existing failure cleanup remains covered.

Validate: focused public import/destination and engine resource/provenance/
diagnostic/import tests, `go test -race ./internal/engine`, and everyday check.
Done: logical placement is inspectable without changing physical identity,
filesystem lifetime, or the public facade boundary.

Evidence (2026-10-14): based on `43e55dfa85963d050d5899aec9d96f4ef637ebf6`,
using `go1.26.8 linux/amd64`. Focused public facade import/destination/resource
tests, the engine import/destination/provenance/diagnostic/resource suite,
`go test -race ./internal/engine`, and `./scripts/check.sh` all passed. New
coverage verifies exact host and native physical source paths, IDs, kinds,
effective destinations, declaration locations, native resource opens from
their declaring directories, detached destination copies across repeated and
independent loads, and public source/diagnostic records for imported parse,
source-local directive, effective-path composition/version, and typed validation
failures. Existing layered inheritance/context and source-acquisition tests
cover inherited lower values, explicit filesystem contexts, and cleanup. No
limitations.

## Stage 7 — Verify inheritance, history, sensitivity, and diffs

Status: planned. Scope: retained placed data and one final application lifecycle.

1. Resolve a sibling parent/child supplied by separate relative imports after
   all tiers. Reuse nested inheritance/cancellation tests; add no cross-collection
   lookup semantics.
2. Compare a relative-import load with equivalent wrapped/per-input-placed
   inputs. Values, presence, and safe diff equality agree; physical source
   locations may naturally differ.
3. Cover historical contributions, replaced/removed descendants, effective-path
   and field-tag sensitivity, inherited fields, and a recorded derived value
   using existing dependencies. Check effective rendering, explanations/history,
   and bilateral diffs with disposable canaries. Preserve the existing
   Redacted/HiddenChanged distinction.
4. Verify defaults, decode, resolution, secret lookup, and semantic validation
   use the final assembled shape without per-fragment callbacks. No source header
   is checked, consumed, coerced, or retained as a new evidence type. Failures
   still return no usable partial result.

Validate: engine `Import|Destination|Inheritance|Secret|Sensitivity|Explain|Diff`
tests, representative facade tests, and everyday check. Done: the feature feeds
existing inspection/completion mechanisms without reconstructing them.

## Stage 8 — Demonstrate ordinary and mounted collection entries

Status: planned. Scope: maintained public examples and their behavior tests.

1. Add a compact ordinary-root example: its entry imports metadata and a fragment
   with `destination: [prompts]`, using existing `Load` import opt-in and no
   Input destination or header policy.
2. Add/adapt a maintained layered example under `examples/consumers`. Mount an
   entry beneath collection keys and let its YAML declare metadata/prompt/profile
   placement, including a nested import. Keep the application schema declaration
   main-owned. The entry has no consumed specification header; collection/prompt
   version labels are ordinary payload.
3. Include peer composition, local sibling inheritance, an upper-tier version-only
   domain override, and a source-relative resource query where representative.
   Reuse current example types/fixtures; keep imports compact and necessary.
4. Test final values, source placement, presence, and resource context through
   supported APIs. Applications select sources/tiers and interpret resources;
   add no source rewriting, generic merging, or provenance reconstruction.
   Link maintained examples from the consumer guide.

Validate: `go test ./examples/consumers`, applicable executable `Example` tests,
local links, and everyday check. Done: ordinary and layered workflows operate
without the deferred source-header proposal.

## Stage 9 — Reconcile documentation and audit the complete feature

Status: planned. Scope: final consistency and evidence; no publication work.

1. Reconcile `docs/config.md`, declaration comments, consumer guide, component
   overview, and development reading/test maps. Change only affected material.
   Distinguish input entry placement and per-import effective placement; remove
   unqualified uniform-inheritance claims. Add ADR navigation. Update architecture
   policy only as needed to recognize the extension and link exact behavior to
   its canonical contract.
2. Map each relative-import request row and roadmap behavior to specific tests
   or examples here. Include root/layered loads, host/native acquisition, literal
   keys, parser failures, peer/glob isolation, composition/version ownership,
   failure records, resources, disclosure/history/diffs, accumulated limits, and
   empty/absent contributions. Explain reused evidence; header rows stay deferred.
3. Review the baseline-to-candidate diff for accidental strengthening: extra
   naming rules/caps, weakened containment/repetition, root resets, shifted
   callbacks/version phases, error-order promises, header processing, or expanded
   merge/secret semantics. Check explicit per-document traversal and the absence
   of a uniform post-traversal source overwrite.
4. Run `./scripts/check.sh` and
   `GOTOOLCHAIN=local GOPROXY=off go test -race ./...`. Record revision, Go version,
   commands, results, and limitations. Check links and root/private boundary
   coverage; no performance claims or new release gates.
5. Mark this plan and the relative-import roadmap complete only when code, tests,
   maintained examples, and canonical docs agree. Preserve the broader request's
   unfinished header proposal. Retain completed records until separately
   authorized release preparation; no release/downstream upgrade is implied.

Done: authors can declare immediate-parent-relative path/glob destinations in
existing import-capable loads with coherent values, provenance, bounds, and
ownership. Independent specification headers remain a separate future decision.

## Implementation owner map

Use graph discovery to confirm current symbols and locations before editing:

- `internal/engine/directives.go`: `importDeclaration`,
  `parseImportDeclaration`, `parseImportDirective`, `splitImportDirectives`,
  and source-local diagnostic construction.
- `internal/engine/imports.go`: `importTraversal`,
  `loadImportedWithAccessTracked`, `walkDeclarations`, `visitPath`, and shared
  accounting. `glob.go` owns physical matching and its ordering.
- `internal/engine/destination.go`: `normalizeDestination`, `placeDocument`,
  `destinationWrapperNodeCount`, and source-destination assignment.
- `internal/engine/load_input.go`: `assembleBaseInputTracked`, early failures,
  context handoff, and completed Mapping diagnostic remediation.
- Existing assembly/lifecycle: `load_request.go`, `load.go`, `compose.go`,
  `inheritance_load.go`, and metadata/history/rendering/diff owners.
- Root inspection: `records.go`, `conversion.go`, `layered.go`, and existing
  public destination/import tests. No new exported Go surface.
- Behavioral foundations: engine `imports_test.go`, `glob_test.go`,
  `destination_import_test.go`, `destination_accounting_test.go`,
  `destination_diagnostic_test.go`, `destination_provenance_test.go`,
  `destination_integration_test.go`; root `facade_destination_test.go`; and
  maintained consumer fixtures/tests. Reuse owners rather than duplicate suites.

## Open Questions

None required for this feature. Selected list syntax and existing placement
contracts resolve the public choices; this plan settles context, diagnostic,
and accounting details within them. Report a newly discovered incompatibility
with evidence and alternatives rather than adding a restriction or implementing
the separate source-header proposal.