diff --git a/docs/roadmap/relative-import-destinations.md b/docs/roadmap/relative-import-destinations.md new file mode 100644 index 0000000..41f0b33 --- /dev/null +++ b/docs/roadmap/relative-import-destinations.md @@ -0,0 +1,231 @@ +# Relative Import Destinations Roadmap + +Status: planned; implementation has not started. Execute the stages below in +order. This roadmap addresses only the relative-import-destination addition in +the [PromptKit request](promptkit-feature-request.md#requested-addition-relative-import-destinations). +Independent source specification headers are deferred for reconsideration after +this work is implemented; they are not a prerequisite or part of these stages. + +## Targeted end state + +Authors can assign a relative mapping destination to a `path` or `glob` import. +ConfigKit appends its literal key segments to the immediate importing document's +effective destination, then processes the imported application content through +the existing placement and loading lifecycle. Applications continue to select +entry sources, input destinations, filesystem contexts, and precedence tiers. + +Proposed syntax, pending the syntax decision below: + +```yaml +configkit: + imports: + - path: metadata.yml + - glob: prompts/*.yml + destination: [prompts] + - glob: profiles/*.yml + destination: [profiles] +``` + +If the application places this entry at `[collections, weather]`, its metadata +import contributes at that same path, and each prompt file contributes at +`[collections, weather, prompts]`. An import declared inside a prompt file with +`destination: [extras]` contributes at +`[collections, weather, prompts, extras]`. A subsequent sibling import from the +entry is still relative to `[collections, weather]`. + +This example does not declare or consume a source specification header. A +source-local `version` remains ordinary payload unless governed by the existing +main-owned version contract. The combined header example in the broader request +is not an acceptance requirement for this roadmap. + +The feature must preserve these requested and existing behaviors: + +- Omission and an empty destination list retain the importing document's + destination. Each glob match receives the same relative placement. Transitive + imports accumulate destinations from their immediate parents. +- Segments are mapping keys under the existing `Input.Destination` rules. Dots, + slashes, and numeric-looking text within a key are literal. There is no + filesystem-style normalization, parent traversal, or application-root reset. + Do not add a collection-name grammar or reject otherwise supported literal + keys merely because their spelling resembles a filesystem path. +- Imports retain their strict-composition tier. Placement and traversal order + do not confer precedence. Later tiers retain existing recursive overlay + behavior; sequences remain atomic and overlapping destinations use existing + composition rules. +- Filesystem paths resolve from their declaring source independently of logical + placement. Import opt-in, optionality, glob behavior, containment, cycle and + repeated-source rules, source kinds, and ownership remain unchanged. A new + destination does not allow a physical source to repeat within a strict group. +- Source-root directives are interpreted before application placement. Existing + reserved-key rules continue to apply to destinations. Invalid new directive + shapes produce normal structured failures rather than being ignored. +- `SourceInfo.Destination` reports each acquired source's resulting effective + destination, including nested imports and sources recorded on failed loads + to the extent supported by existing error reporting. +- Composition and value diagnostics use effective application paths and real + source locations. Directive errors identify their source-local declaration. + Placement does not manufacture authored coordinates or alter resource bases. +- Main-owned version protection applies at the resulting effective paths. + Ordinary nested version fields stay ordinary payload. Inheritance resolves + in the assembled sibling scope, then defaults, decoding, resolution, secret + lookup, and validation follow the existing single-load lifecycle. +- Presence, provenance, history, explanations, rendering, diffs, and sensitivity + agree with the placed contributions. Source-relative resource contexts survive + inheritance and precedence under their existing contracts. +- Accumulated placement and introduced wrappers participate in existing depth + and node accounting. This work does not select new limit values or promise + new performance bounds. An actually imported empty mapping retains existing + placement behavior; an optional import that finds no source creates no payload. + +The syntax is available wherever existing explicit imports are supported, +including ordinary root loading and layered main/underlay inputs. It does not +require a new Go loading entry point, mount registry, or consumer-side rewriting. + +Source header policies and consumption, cross-collection inheritance, sequence +merging, root-reset syntax, automatic discovery, recursive glob syntax, and +collection management are outside this delivery. This scope statement does not +establish new permanent exclusions beyond [existing deferred scope](deferred.md). + +## Current baseline + +Reviewed baseline: `315f464`, with input destinations implemented. The +[configuration contract](../config.md#input-destinations) and +[ADR 0011](../adr/0011-input-destinations.md) describe that feature. + +- [Directive parsing](../../internal/engine/directives.go) accepts `path` or + `glob` plus `optional`; it rejects unrecognized fields. Import declarations + currently carry no relative destination. +- [Import traversal](../../internal/engine/imports.go) holds one destination for + an input's entire import tree. Both document placement and imported source + records use it. Per-document effective placement is therefore required. +- [Input acquisition](../../internal/engine/load_input.go) currently assigns + the input destination to all returned imported source records. That uniform + assignment must no longer overwrite distinct import destinations. +- [Destination helpers](../../internal/engine/destination.go) already validate + literal mapping paths, place document bodies, and check introduced depth and + node costs. Composition, version protection, and retained source applications + already operate on placed content. +- The root facade already exposes `SourceInfo.Destination`; the new syntax can + use that inspection surface without a separate placement API. + +The change extends traversal and directive parsing. It should reuse existing +placement and assembly rather than introduce a second merge or validation model. + +## Ordered implementation plan + +### 1. Select syntax and document the semantic extension + +Resolve the Open Decision below and specify valid shapes, literal key handling, +and structured failures. Preserve existing `path`/`glob` and `optional` behavior. +Validate declared destination syntax even when an optional import has no match; +optionality suppresses missing input, not malformed directives. + +Record the extension to ADR 0011's inherited-destination model in a new ADR +without rewriting the accepted decision. Distinguish the new declarative syntax +from unchanged input placement and version ownership. Review compatibility +under the release policy without choosing or publishing a release in this work. + +Completion: the syntax and parent-relative semantics are explicit, with source +specification headers still deferred and no additional merge or namespace rules. + +### 2. Parse destinations and carry effective placement through traversal + +Extend private import declarations and parser validation with the selected +relative destination representation and useful declaration locations. Reuse +existing key validation rather than maintaining a competing path convention. + +Compute each child's effective destination from the immediate parent's placement. +Pass that context through path imports, glob expansion, and recursive traversal. +Explicit per-call state is a straightforward implementation option; the public +contract need not prescribe a parameter, stack, or allocation layout. Ensure +nested placement cannot leak into siblings, subsequent glob matches, or inputs. + +Use the resulting destination when placing the body, checking effective version +claims, accounting for limits, and creating source records. Remove or narrow +uniform post-traversal source-destination assignment. Retain original physical +identity and resource contexts; do not include destination in repetition keys. + +Completion: nested and sibling imports, glob matches, and source records all use +the right effective destination, with structured errors and existing bounds. + +### 3. Verify integration with assembly and inspection + +Check that placed source applications feed existing strict composition, tier +precedence, inheritance, and final typed loading. Confirm conflicts report the +effective path and contributing physical locations, including imports that +collide with explicitly wrapped input or a destination ancestor. + +Verify protected version claims after destination accumulation, and ordinary +version payload away from the protected path. Confirm history, sensitivity, +rendering, and diffs use placed data without reconstructing placement in callers. +Exercise inherited and surviving lower-tier resource references through existing +source-context APIs. Preserve detached source records and existing failure +behavior; do not add promises about collecting every error or exact failure +ordering merely to simplify tests. + +Completion: imports with explicit destinations agree with equivalent existing +placement for application behavior, while actual source attribution stays intact. + +### 4. Add acceptance coverage and consumer guidance + +Use parser tests for declaration shape, engine tests for traversal and placement, +and public tests/examples for complete consumer workflows. Reuse existing tests +where they already protect unchanged behavior. Cover the following distinct risks: + +| Scenario | Evidence required | +| --- | --- | +| Omitted destination and `[]` | Existing inherited placement, including root imports | +| Entry, child, and grandchild destinations | Placement accumulates from the immediate importer | +| Nested import followed by a sibling; multiple glob matches | Child state does not leak to peers | +| Literal slash, dot, numeric-looking, and empty keys | Existing mapping-key meaning is preserved | +| Invalid shapes and reserved keys, including optional unmatched imports | Structured directive failure with a real declaration location | +| Disjoint, overlapping, and ancestor-related destinations | Existing composition conflicts and tier precedence | +| Same physical source selected at different destinations in a group | Existing repetition/cycle policy is retained | +| Mounted import targeting a protected version path | Existing ownership checks use the accumulated path | +| Ordinary version payload elsewhere | No new checking, coercion, or consumption | +| Inheritance, overrides, and resource-relative fields | Effective paths and original resource bases agree | +| History, classified values, and diffs | Existing provenance and disclosure rules remain effective | +| Accumulated depth, wrapper nodes, empty bodies, and absent optional sources | Existing accounting and contribution behavior | +| Host/native filesystem imports and supported entry source forms | Placement is independent of acquisition kind | +| Public source inspection and load failures | Accurate effective destinations and detached records | + +Add an executable collection-entry example that imports metadata and mounts +prompt/profile fragments. It must use no source header checker or consumer-side +parsing, merging, or provenance reconstruction. Demonstrate ordinary root loading +as well as a representative layered mounted entry without duplicating all cases. + +Update the owning configuration contract and consumer guide as behavior lands; +update source-record declaration documentation and internal overview only where +their descriptions change. Keep the header proposal deferred for later review. + +Run focused tests, then the everyday check and appropriate ownership race checks +from the [development guide](../development.md), following the +[testing policy](../policy/testing.md). Validate documentation links. No independent +performance benchmarks are part of this roadmap. + +Completion: the relative-import acceptance scenarios in the PromptKit request +are covered, the combined workflow works without source headers, validation +passes, and implemented documentation matches the behavior. Source specification +headers can then be reconsidered as a separate feature; their implementation is +not required to complete this roadmap. + +## Open Decisions + +### YAML destination representation + +Recommendation: adopt `destination` as a sequence of strings, for example +`destination: [prompts, "0"]`. Omission and `[]` inherit placement. Treat null, +scalar destinations, and non-string members as invalid shapes rather than +coercing them. Permit every string key allowed by existing destination rules, +including the empty string. This directly expresses literal segments, matches +the request's example, and keeps numeric key text distinct from YAML numbers. +These shape choices are proposed syntax, not pre-existing restrictions. + +Alternative: also accept a scalar JSON-pointer representation using existing +`ParsePath` conventions, while retaining the sequence form. This is viable and +may be convenient for users already familiar with public paths, but introduces +two spellings and escaping rules for one operation. A leading slash would still +describe relative segments, not reset to the application root, which could be +surprising. Prefer the sequence-only form unless consumer evidence justifies +the additional notation. A null-as-inherit shorthand is also viable but adds a +third spelling for behavior already covered by omission and an empty list.