From 3c2d7facb39c145bd34cb50b8998a32389541479 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Wed, 14 Oct 2026 03:27:34 +0000 Subject: [PATCH] Plan configuration input destinations --- docs/roadmap/input-destinations.md | 292 +++++++++++++++++++++++++++++ 1 file changed, 292 insertions(+) create mode 100644 docs/roadmap/input-destinations.md diff --git a/docs/roadmap/input-destinations.md b/docs/roadmap/input-destinations.md new file mode 100644 index 0000000..8509cab --- /dev/null +++ b/docs/roadmap/input-destinations.md @@ -0,0 +1,292 @@ +# Configuration Input Destinations Roadmap + +Status: proposed; implementation has not started. Execute the stages below in +order. This roadmap responds to the +[PromptKit feature request](promptkit-feature-request.md) and owns the target +state, delivery sequence, and unresolved design choices. Public spelling and +recommendations below are proposals until selected explicitly. + +## Targeted end state + +A consumer can assign a mapping destination to a configuration input and perform +one ordinary `LoadLayered` call. The input's application content contributes +beneath that destination while retaining normal composition, precedence, +inheritance, typed loading, diagnostics, and inspection. PromptKit can assemble +collections without parsing and rewriting sources, merging values, or rebuilding +provenance itself. + +For example, a fragment defining `summarize.alerts` can contribute beneath +`collections`, `weatherreporter`, `prompts`. An independent source with the same +local key can contribute beneath another collection without conflict. Collection +and prompt names remain ordinary application keys, not ConfigKit namespaces. + +The intended public surface is per-input placement, illustratively an optional +`Destination Path` field on `Input`, not a new loading entry point or an operation +on completed results. The minimum supported workflow has a root main input and +mounted underlay inputs; broader main placement is an open decision below. + +The following behavior defines the target: + +- Omitted and empty destinations retain root placement. Destinations consist of + literal mapping keys, including dots, slashes, and numeric-looking text. + Explicit sequence-index destinations are outside the requested capability. +- Host files, native filesystem files, bytes, readers, and mapping sources all + support placement under their existing acquisition and ownership contracts. + Filesystem context, import opt-in, and precedence remain independent choices. +- Mounted contributions have the same application values and conflict behavior + as explicitly wrapped contributions, with accurate original source locations. + Disjoint mappings combine; overlapping non-mapping values within a group + conflict even when equal. A later tier cannot repair a strict-group conflict. +- Different destinations can coexist in one group, and the same destination can + occur in different tiers. Ancestor/descendant destinations follow ordinary + composition and overlay rules without namespace uniqueness restrictions. + Later tiers recursively overlay mappings and replace other values. Empty + mappings do not clear lower values; sequences remain atomic. +- A prompt's ordinary `version` override changes that field and retains omitted + lower fields. It neither identifies a merge record nor replaces a whole prompt. +- Source-root `configkit.imports` remains structural. Imported application data + inherits the input destination transitively. Import paths still resolve from + their declaring source; containment, optionality, cycles, and repeated-source + rules remain applicable. Placement does not change physical source identity. +- Sibling inheritance runs after assembly in the effective destination scope. + Peer fragments can supply a child and its sibling parent. Qualified PromptKit + references do not acquire cross-collection inheritance meaning. +- Version ownership is enforced at effective application paths. A collection's + mounted `version` is ordinary data, while underlay/import contributions to the + actual protected path, or replacements of its ancestors, remain invalid. + There is one main-owned version check and no blanket nested numeric conversion. +- Defaults, decoding, resolution, secret lookup, and validation run once on the + assembled application, not independently on fragments. Callbacks and value + diagnostics use effective paths. Failures use structured load errors and yield + no usable partial result. +- Presence, provenance, history, explanations, effective rendering, and diffs + agree with the final values. Sensitivity is evaluated at effective application + paths under the existing historical and dependency rules. +- Original source locations and passive resource contexts survive placement, + overrides, and inheritance. `Result.SourceContext` and + `Resolution.SourceContext` still identify the declaring filesystem and base + directory for a field such as `content_file: ./summary.md`. +- Destinations and the structure they introduce participate in existing load + limits; placement cannot bypass depth or node accounting. Source ownership, + complete-on-return loading, and detached result behavior remain intact. + +Import-destination syntax, independent collection specification headers, and +cross-collection inheritance are separate possible follow-ups. This work does +not add directory discovery, collection installation, remote acquisition, +namespace ownership enforcement, dependency solving, live reload, filename-based +identity, arbitrary transformations, or new merge strategies. PromptKit owns its +collection inventory, qualified IDs, and execution semantics. + +## Current baseline + +The request reviewed v0.7.1. The current worktree has these relevant boundaries: + +- [Input](../../layered.go) holds a source, import opt-in, and filesystem context, + but no destination. Root types delegate to the engine through the existing + [facade boundary](../internal/overview.md). +- [Layered loading](../config.md#layered-loading) already provides strict groups, + ordered precedence, shared acquisition budgets, and one final lifecycle. + [Underlay acquisition](../../internal/engine/load_request.go) gathers source + applications and recomposes them into their strict group. +- [Input acquisition](../../internal/engine/load_input.go) and + [import traversal](../../internal/engine/imports.go) process source-root + directives and source identity before final assembly. Imported version claims + are checked during traversal; underlay claims are checked before group + composition. These currently operate without a destination. +- [Version claim checks](../../internal/engine/directives.go) inspect a document + using the configured application path. Wrapping only an already acquired + result would leave early fragment checks and diagnostic paths incorrect. +- [Sibling inheritance](../adr/0009-sibling-inheritance.md) already runs after + source precedence and preserves source-to-destination derivation. Placement + must feed that existing mechanism rather than resolve fragments independently. +- Existing source records identify acquisition and precedence but do not expose + input placement. Existing history retains source applications, so moving only + the final tree would be insufficient for coherent inspection. + +The architectural fit is an extension of input assembly, not a parser extension +or a second configuration model. The +[architecture policy](../policy/architecture.md) supplies the ownership and +consistency constraints; [deferred scope](deferred.md) continues to govern the +unrequested features above. + +## Ordered implementation plan + +### 1. Settle placement semantics and the public contract + +Resolve the Open Decisions before their dependent implementation. Select the +public API spelling, main/NoSource behavior, placement inspection, reserved-key +handling, and empty-contribution semantics. Define destination validation and +structured errors, including when validation occurs relative to source access. +Use existing error categories where suitable rather than inventing a parallel +error system. + +Record the significant assembly decision in an ADR. Review compatibility under +the release policy, including exported struct changes and unchanged zero-value +behavior. Keep unimplemented claims in the roadmap; update canonical contracts +when implementation lands. Do not turn unrelated deferred features into work. + +Completion: selected semantics cover the request's public acceptance scenarios +and a simple consumer as well as PromptKit's multi-collection workflow. + +### 2. Carry destinations through acquisition and imports + +Thread the selected destination through root-to-engine conversion and private +load requests. Preserve its ownership and semantic key representation. Separate +source-local directive processing from application placement: extract directives +at their original root, inherit placement through traversal, and place application +content before checks and composition that require effective paths. + +Make version protection destination-aware for underlay roots and imported +fragments, preserving the main check's lifecycle position. Ensure diagnostics +produced before final composition identify effective application paths where +available while preserving actual source coordinates. Structural directive +errors still need their genuine source-local declaration locations. + +Retain repeated-source detection across the whole strict group, independent of +destination. Preserve filesystem namespace, resource contexts, and traversal +ordering. Account for destination depth and introduced nodes using existing +budgets; distinguish them from acquired source bytes and physical source count. + +Completion: all content-bearing source kinds and transitive imports contribute +at the intended destination, with correct version protection and source identity. + +### 3. Integrate composition, provenance, and the final lifecycle + +Apply ordinary composition to placed contributions and ordinary overlay between +tiers. Retain each source application at its effective placement for history; +carry real source attribution and the selected synthetic-container convention +through origin construction. Avoid a separate merge algorithm or a mount registry. + +Exercise overlapping destinations, empty mappings, and a version-only domain +override. Resolve sibling inheritance after the full precedence assembly, then +use existing defaults, typed decoding, resolution, secrets, and validation. +Check protection of the specification path through inheritance, selected +overlays, and CLI overrides as well as direct contributions. + +Completion: mounted and explicitly wrapped equivalents agree on values, +conflicts, presence, and lifecycle results; physical locations remain accurate. + +### 4. Complete inspection and public consumer workflows + +Expose placement through the selected existing inspection surface, with detached +records. Verify effective paths throughout metadata, explanation replay, history, +rendering, and diffs. Apply the existing sensitivity rules to mounted and inherited +fields, including superseded values. Do not fabricate coordinates for wrappers +that were never written in a source file. + +Demonstrate source-relative resource lookup from both resolution and completed +results, including inherited fields and lower-tier fields surviving an override. +Add executable public examples for one mounted fragment and a representative +multi-collection assembly with peer composition and an intentional override. +Applications select sources and destinations; the examples should require no +consumer-side parsing, generic merging, or provenance reconstruction. + +Completion: the supported root API is sufficient for the requested end-to-end +workflow, including accurate placement and source-relative resource inspection. + +### 5. Validate the full feature and update owning documentation + +Use engine tests for placement mechanics and public tests for observable loading +and inspection. Map every scenario in the request's acceptance table to a test +or executable example, reusing existing coverage where it proves the same +behavior. Include focused cases for: + +- omitted/empty destinations, literal escaped and numeric keys, invalid index + destinations, and the selected reserved-key and main-input rules; +- different destinations, shared destinations, ancestor overlaps, equal and + unequal conflicts, explicit null, omitted fields, and empty contributions; +- recursive precedence, atomic sequences, and domain-version-only overrides; +- inherited import destinations, source-root directives, repeated physical + sources at different destinations, and unchanged cycle/containment behavior; +- nested ordinary version fields versus actual protected-path and ancestor + claims, including mounted imported content; +- local inheritance across peers and source contexts after inheritance/overlay; +- effective diagnostic paths with real locations, callback paths, history, + rendering, diffs, and classified values protected with disposable canaries; +- equivalent content across every supported source kind, invalid loads with no + usable result, introduced depth/node limits, and detached inspection ownership. + +Follow the [testing policy](../policy/testing.md): protect observable contracts +without fixing incidental wrapper allocation or helper-call sequences. Run +focused tests, then the everyday check and ownership race validation in the +[development guide](../development.md). Validate local documentation links. +No independent performance benchmark runs are required for this roadmap. + +Update declaration documentation, the configuration contract, consumer guidance, +and the component overview as their behavior lands. Document how source-local +coordinates relate to effective paths and how placement is inspected. Review +release compatibility separately from publication; feature completion does not +authorize a release. + +Completion: all requested public scenarios pass, owning documentation describes +implemented behavior, and consumers can perform one complete layered load with +no compensating source-rewriting or post-load structural pass. + +## Open Decisions + +### Main-input destinations and absent inputs + +Recommendation: allow the same destination model for content-bearing main inputs +as for underlays, with `VersionRule.Path` remaining an effective application path +and any specification declaration still owned by main. The checker retains its +existing handling of an absent declaration. Treat `NoSource` as no contribution, +even with a valid destination. This keeps `Input` coherent and avoids a role-only +restriction. PromptKit can continue using root main; broader main placement must +not add per-fragment checks or move the existing version-check phase. + +Alternative: initially support non-root destinations only for underlay inputs +and reject non-root main placement explicitly. This meets PromptKit's minimum +need and reduces the first implementation's version-check interactions, but adds +a role-dependent rule to a shared public type. Adopt that restriction only as an +explicit scope choice, not because threading main placement is inconvenient. +For `NoSource`, rejecting a nonempty destination could catch mistaken assembly +declarations, but makes an otherwise absent contribution less composable. + +### Public placement inspection and synthetic ancestors + +Recommendation: expose effective placement on existing source records, +illustratively `SourceInfo.Destination`, including inherited import placement. +Use original source identity for introduced ancestor mappings, documenting that +any root-location fallback identifies the contributing document rather than a +physically authored wrapper. Actual content keeps its real coordinates. This +allows consumers to connect placement to sources without a new query system. + +Alternative: add placement to origin records or a dedicated lookup, and represent +synthetic ancestors without a physical location. This can distinguish authored +and synthetic structure more precisely, but expands per-value metadata or adds +an API when placement is fundamentally per source application. Evaluate the +choice against inherited fields, composite origins, and the same source selected +in separate tiers; do not infer placement from a final path by guesswork. + +### Reserved structural keys in destinations + +Recommendation: use existing `Path` conventions for literal keys and parsed +numeric tokens, reject explicit index segments, and preserve current structural +reservations in the effective tree. Reject destination segments that would +materialize `configkit` or a mapping-valued `extends` contrary to those contracts, +with a clear structured destination error. Ordinary keys containing those words, +dots, or slashes remain ordinary keys. Source-root directives are processed before +placement and are not rejected simply because their application body is mounted. + +Alternative: permit structural spellings as literal synthetic wrapper keys by +distinguishing them from authored directives. This enables additional application +paths but introduces an exception where mounted and explicitly wrapped inputs +can have different meanings. It would require an explicit change to reserved-key +semantics and is not needed for the requested collection paths. Running existing +directive validation on the placed tree is another viable error mechanism, but +must still identify that the invalid structure came from destination selection. + +### Empty mappings and presence of introduced containers + +Recommendation: mounting an actual empty application mapping materializes its +destination mappings as explicit contributions, matching authored wrappers. +This preserves the request's equivalence principle and lets consumers inspect +an intentionally empty collection. Such mappings still do not clear lower-tier +children. An absent source contributes nothing. + +Alternative: treat an empty mounted body as no contribution. This avoids empty +containers for fragments containing only imports or no application fields, but +loses equivalence with explicitly wrapped empty mappings and erases an observable +presence distinction. If selected, document that exception and its effect on +history and empty imported bodies rather than deriving it accidentally from +whether the implementation allocated wrapper nodes.