Plan relative import destinations

docs/roadmap/relative-import-destinations.md

@@ -0,0 +1,231 @@
+# Relative Import Destinations Roadmap
+
+Status: planned; implementation has not started. Execute the stages below in
+order. This roadmap addresses only the relative-import-destination addition in
+the [PromptKit request](promptkit-feature-request.md#requested-addition-relative-import-destinations).
+Independent source specification headers are deferred for reconsideration after
+this work is implemented; they are not a prerequisite or part of these stages.
+
+## Targeted end state
+
+Authors can assign a relative mapping destination to a `path` or `glob` import.
+ConfigKit appends its literal key segments to the immediate importing document's
+effective destination, then processes the imported application content through
+the existing placement and loading lifecycle. Applications continue to select
+entry sources, input destinations, filesystem contexts, and precedence tiers.
+
+Proposed syntax, pending the syntax decision below:
+
+```yaml
+configkit:
+  imports:
+    - path: metadata.yml
+    - glob: prompts/*.yml
+      destination: [prompts]
+    - glob: profiles/*.yml
+      destination: [profiles]
+```
+
+If the application places this entry at `[collections, weather]`, its metadata
+import contributes at that same path, and each prompt file contributes at
+`[collections, weather, prompts]`. An import declared inside a prompt file with
+`destination: [extras]` contributes at
+`[collections, weather, prompts, extras]`. A subsequent sibling import from the
+entry is still relative to `[collections, weather]`.
+
+This example does not declare or consume a source specification header. A
+source-local `version` remains ordinary payload unless governed by the existing
+main-owned version contract. The combined header example in the broader request
+is not an acceptance requirement for this roadmap.
+
+The feature must preserve these requested and existing behaviors:
+
+- Omission and an empty destination list retain the importing document's
+  destination. Each glob match receives the same relative placement. Transitive
+  imports accumulate destinations from their immediate parents.
+- Segments are mapping keys under the existing `Input.Destination` rules. Dots,
+  slashes, and numeric-looking text within a key are literal. There is no
+  filesystem-style normalization, parent traversal, or application-root reset.
+  Do not add a collection-name grammar or reject otherwise supported literal
+  keys merely because their spelling resembles a filesystem path.
+- Imports retain their strict-composition tier. Placement and traversal order
+  do not confer precedence. Later tiers retain existing recursive overlay
+  behavior; sequences remain atomic and overlapping destinations use existing
+  composition rules.
+- Filesystem paths resolve from their declaring source independently of logical
+  placement. Import opt-in, optionality, glob behavior, containment, cycle and
+  repeated-source rules, source kinds, and ownership remain unchanged. A new
+  destination does not allow a physical source to repeat within a strict group.
+- Source-root directives are interpreted before application placement. Existing
+  reserved-key rules continue to apply to destinations. Invalid new directive
+  shapes produce normal structured failures rather than being ignored.
+- `SourceInfo.Destination` reports each acquired source's resulting effective
+  destination, including nested imports and sources recorded on failed loads
+  to the extent supported by existing error reporting.
+- Composition and value diagnostics use effective application paths and real
+  source locations. Directive errors identify their source-local declaration.
+  Placement does not manufacture authored coordinates or alter resource bases.
+- Main-owned version protection applies at the resulting effective paths.
+  Ordinary nested version fields stay ordinary payload. Inheritance resolves
+  in the assembled sibling scope, then defaults, decoding, resolution, secret
+  lookup, and validation follow the existing single-load lifecycle.
+- Presence, provenance, history, explanations, rendering, diffs, and sensitivity
+  agree with the placed contributions. Source-relative resource contexts survive
+  inheritance and precedence under their existing contracts.
+- Accumulated placement and introduced wrappers participate in existing depth
+  and node accounting. This work does not select new limit values or promise
+  new performance bounds. An actually imported empty mapping retains existing
+  placement behavior; an optional import that finds no source creates no payload.
+
+The syntax is available wherever existing explicit imports are supported,
+including ordinary root loading and layered main/underlay inputs. It does not
+require a new Go loading entry point, mount registry, or consumer-side rewriting.
+
+Source header policies and consumption, cross-collection inheritance, sequence
+merging, root-reset syntax, automatic discovery, recursive glob syntax, and
+collection management are outside this delivery. This scope statement does not
+establish new permanent exclusions beyond [existing deferred scope](deferred.md).
+
+## Current baseline
+
+Reviewed baseline: `315f464`, with input destinations implemented. The
+[configuration contract](../config.md#input-destinations) and
+[ADR 0011](../adr/0011-input-destinations.md) describe that feature.
+
+- [Directive parsing](../../internal/engine/directives.go) accepts `path` or
+  `glob` plus `optional`; it rejects unrecognized fields. Import declarations
+  currently carry no relative destination.
+- [Import traversal](../../internal/engine/imports.go) holds one destination for
+  an input's entire import tree. Both document placement and imported source
+  records use it. Per-document effective placement is therefore required.
+- [Input acquisition](../../internal/engine/load_input.go) currently assigns
+  the input destination to all returned imported source records. That uniform
+  assignment must no longer overwrite distinct import destinations.
+- [Destination helpers](../../internal/engine/destination.go) already validate
+  literal mapping paths, place document bodies, and check introduced depth and
+  node costs. Composition, version protection, and retained source applications
+  already operate on placed content.
+- The root facade already exposes `SourceInfo.Destination`; the new syntax can
+  use that inspection surface without a separate placement API.
+
+The change extends traversal and directive parsing. It should reuse existing
+placement and assembly rather than introduce a second merge or validation model.
+
+## Ordered implementation plan
+
+### 1. Select syntax and document the semantic extension
+
+Resolve the Open Decision below and specify valid shapes, literal key handling,
+and structured failures. Preserve existing `path`/`glob` and `optional` behavior.
+Validate declared destination syntax even when an optional import has no match;
+optionality suppresses missing input, not malformed directives.
+
+Record the extension to ADR 0011's inherited-destination model in a new ADR
+without rewriting the accepted decision. Distinguish the new declarative syntax
+from unchanged input placement and version ownership. Review compatibility
+under the release policy without choosing or publishing a release in this work.
+
+Completion: the syntax and parent-relative semantics are explicit, with source
+specification headers still deferred and no additional merge or namespace rules.
+
+### 2. Parse destinations and carry effective placement through traversal
+
+Extend private import declarations and parser validation with the selected
+relative destination representation and useful declaration locations. Reuse
+existing key validation rather than maintaining a competing path convention.
+
+Compute each child's effective destination from the immediate parent's placement.
+Pass that context through path imports, glob expansion, and recursive traversal.
+Explicit per-call state is a straightforward implementation option; the public
+contract need not prescribe a parameter, stack, or allocation layout. Ensure
+nested placement cannot leak into siblings, subsequent glob matches, or inputs.
+
+Use the resulting destination when placing the body, checking effective version
+claims, accounting for limits, and creating source records. Remove or narrow
+uniform post-traversal source-destination assignment. Retain original physical
+identity and resource contexts; do not include destination in repetition keys.
+
+Completion: nested and sibling imports, glob matches, and source records all use
+the right effective destination, with structured errors and existing bounds.
+
+### 3. Verify integration with assembly and inspection
+
+Check that placed source applications feed existing strict composition, tier
+precedence, inheritance, and final typed loading. Confirm conflicts report the
+effective path and contributing physical locations, including imports that
+collide with explicitly wrapped input or a destination ancestor.
+
+Verify protected version claims after destination accumulation, and ordinary
+version payload away from the protected path. Confirm history, sensitivity,
+rendering, and diffs use placed data without reconstructing placement in callers.
+Exercise inherited and surviving lower-tier resource references through existing
+source-context APIs. Preserve detached source records and existing failure
+behavior; do not add promises about collecting every error or exact failure
+ordering merely to simplify tests.
+
+Completion: imports with explicit destinations agree with equivalent existing
+placement for application behavior, while actual source attribution stays intact.
+
+### 4. Add acceptance coverage and consumer guidance
+
+Use parser tests for declaration shape, engine tests for traversal and placement,
+and public tests/examples for complete consumer workflows. Reuse existing tests
+where they already protect unchanged behavior. Cover the following distinct risks:
+
+| Scenario | Evidence required |
+| --- | --- |
+| Omitted destination and `[]` | Existing inherited placement, including root imports |
+| Entry, child, and grandchild destinations | Placement accumulates from the immediate importer |
+| Nested import followed by a sibling; multiple glob matches | Child state does not leak to peers |
+| Literal slash, dot, numeric-looking, and empty keys | Existing mapping-key meaning is preserved |
+| Invalid shapes and reserved keys, including optional unmatched imports | Structured directive failure with a real declaration location |
+| Disjoint, overlapping, and ancestor-related destinations | Existing composition conflicts and tier precedence |
+| Same physical source selected at different destinations in a group | Existing repetition/cycle policy is retained |
+| Mounted import targeting a protected version path | Existing ownership checks use the accumulated path |
+| Ordinary version payload elsewhere | No new checking, coercion, or consumption |
+| Inheritance, overrides, and resource-relative fields | Effective paths and original resource bases agree |
+| History, classified values, and diffs | Existing provenance and disclosure rules remain effective |
+| Accumulated depth, wrapper nodes, empty bodies, and absent optional sources | Existing accounting and contribution behavior |
+| Host/native filesystem imports and supported entry source forms | Placement is independent of acquisition kind |
+| Public source inspection and load failures | Accurate effective destinations and detached records |
+
+Add an executable collection-entry example that imports metadata and mounts
+prompt/profile fragments. It must use no source header checker or consumer-side
+parsing, merging, or provenance reconstruction. Demonstrate ordinary root loading
+as well as a representative layered mounted entry without duplicating all cases.
+
+Update the owning configuration contract and consumer guide as behavior lands;
+update source-record declaration documentation and internal overview only where
+their descriptions change. Keep the header proposal deferred for later review.
+
+Run focused tests, then the everyday check and appropriate ownership race checks
+from the [development guide](../development.md), following the
+[testing policy](../policy/testing.md). Validate documentation links. No independent
+performance benchmarks are part of this roadmap.
+
+Completion: the relative-import acceptance scenarios in the PromptKit request
+are covered, the combined workflow works without source headers, validation
+passes, and implemented documentation matches the behavior. Source specification
+headers can then be reconsidered as a separate feature; their implementation is
+not required to complete this roadmap.
+
+## Open Decisions
+
+### YAML destination representation
+
+Recommendation: adopt `destination` as a sequence of strings, for example
+`destination: [prompts, "0"]`. Omission and `[]` inherit placement. Treat null,
+scalar destinations, and non-string members as invalid shapes rather than
+coercing them. Permit every string key allowed by existing destination rules,
+including the empty string. This directly expresses literal segments, matches
+the request's example, and keeps numeric key text distinct from YAML numbers.
+These shape choices are proposed syntax, not pre-existing restrictions.
+
+Alternative: also accept a scalar JSON-pointer representation using existing
+`ParsePath` conventions, while retaining the sequence form. This is viable and
+may be convenient for users already familiar with public paths, but introduces
+two spellings and escaping rules for one operation. A leading slash would still
+describe relative segments, not reset to the application root, which could be
+surprising. Prefer the sequence-only form unless consumer evidence justifies
+the additional notation. A null-as-inherit shorthand is also viable but adds a
+third spelling for behavior already covered by omission and an empty list.