ConfigKit/docs/roadmap/input-destinations.md

Configuration Input Destinations Roadmap

Status: proposed; implementation has not started. Execute the stages below in order. This roadmap responds to the PromptKit feature request and owns the target state, delivery sequence, and unresolved design choices. Public spelling and recommendations below are proposals until selected explicitly.

Targeted end state

A consumer can assign a mapping destination to a configuration input and perform one ordinary LoadLayered call. The input's application content contributes beneath that destination while retaining normal composition, precedence, inheritance, typed loading, diagnostics, and inspection. PromptKit can assemble collections without parsing and rewriting sources, merging values, or rebuilding provenance itself.

For example, a fragment defining summarize.alerts can contribute beneath collections, weatherreporter, prompts. An independent source with the same local key can contribute beneath another collection without conflict. Collection and prompt names remain ordinary application keys, not ConfigKit namespaces.

The intended public surface is per-input placement, illustratively an optional Destination Path field on Input, not a new loading entry point or an operation on completed results. The minimum supported workflow has a root main input and mounted underlay inputs; broader main placement is an open decision below.

The following behavior defines the target:

• Omitted and empty destinations retain root placement. Destinations consist of literal mapping keys, including dots, slashes, and numeric-looking text. Explicit sequence-index destinations are outside the requested capability.
• Host files, native filesystem files, bytes, readers, and mapping sources all support placement under their existing acquisition and ownership contracts. Filesystem context, import opt-in, and precedence remain independent choices.
• Mounted contributions have the same application values and conflict behavior as explicitly wrapped contributions, with accurate original source locations. Disjoint mappings combine; overlapping non-mapping values within a group conflict even when equal. A later tier cannot repair a strict-group conflict.
• Different destinations can coexist in one group, and the same destination can occur in different tiers. Ancestor/descendant destinations follow ordinary composition and overlay rules without namespace uniqueness restrictions. Later tiers recursively overlay mappings and replace other values. Empty mappings do not clear lower values; sequences remain atomic.
• A prompt's ordinary version override changes that field and retains omitted lower fields. It neither identifies a merge record nor replaces a whole prompt.
• Source-root configkit.imports remains structural. Imported application data inherits the input destination transitively. Import paths still resolve from their declaring source; containment, optionality, cycles, and repeated-source rules remain applicable. Placement does not change physical source identity.
• Sibling inheritance runs after assembly in the effective destination scope. Peer fragments can supply a child and its sibling parent. Qualified PromptKit references do not acquire cross-collection inheritance meaning.
• Version ownership is enforced at effective application paths. A collection's mounted version is ordinary data, while underlay/import contributions to the actual protected path, or replacements of its ancestors, remain invalid. There is one main-owned version check and no blanket nested numeric conversion.
• Defaults, decoding, resolution, secret lookup, and validation run once on the assembled application, not independently on fragments. Callbacks and value diagnostics use effective paths. Failures use structured load errors and yield no usable partial result.
• Presence, provenance, history, explanations, effective rendering, and diffs agree with the final values. Sensitivity is evaluated at effective application paths under the existing historical and dependency rules.
• Original source locations and passive resource contexts survive placement, overrides, and inheritance. Result.SourceContext and Resolution.SourceContext still identify the declaring filesystem and base directory for a field such as content_file: ./summary.md.
• Destinations and the structure they introduce participate in existing load limits; placement cannot bypass depth or node accounting. Source ownership, complete-on-return loading, and detached result behavior remain intact.

Import-destination syntax, independent collection specification headers, and cross-collection inheritance are separate possible follow-ups. This work does not add directory discovery, collection installation, remote acquisition, namespace ownership enforcement, dependency solving, live reload, filename-based identity, arbitrary transformations, or new merge strategies. PromptKit owns its collection inventory, qualified IDs, and execution semantics.

Current baseline

The request reviewed v0.7.1. The current worktree has these relevant boundaries:

• Input holds a source, import opt-in, and filesystem context, but no destination. Root types delegate to the engine through the existing facade boundary.
• Layered loading already provides strict groups, ordered precedence, shared acquisition budgets, and one final lifecycle. Underlay acquisition gathers source applications and recomposes them into their strict group.
• Input acquisition and import traversal process source-root directives and source identity before final assembly. Imported version claims are checked during traversal; underlay claims are checked before group composition. These currently operate without a destination.
• Version claim checks inspect a document using the configured application path. Wrapping only an already acquired result would leave early fragment checks and diagnostic paths incorrect.
• Sibling inheritance already runs after source precedence and preserves source-to-destination derivation. Placement must feed that existing mechanism rather than resolve fragments independently.
• Existing source records identify acquisition and precedence but do not expose input placement. Existing history retains source applications, so moving only the final tree would be insufficient for coherent inspection.

The architectural fit is an extension of input assembly, not a parser extension or a second configuration model. The architecture policy supplies the ownership and consistency constraints; deferred scope continues to govern the unrequested features above.

Ordered implementation plan

1. Settle placement semantics and the public contract

Resolve the Open Decisions before their dependent implementation. Select the public API spelling, main/NoSource behavior, placement inspection, reserved-key handling, and empty-contribution semantics. Define destination validation and structured errors, including when validation occurs relative to source access. Use existing error categories where suitable rather than inventing a parallel error system.

Record the significant assembly decision in an ADR. Review compatibility under the release policy, including exported struct changes and unchanged zero-value behavior. Keep unimplemented claims in the roadmap; update canonical contracts when implementation lands. Do not turn unrelated deferred features into work.

Completion: selected semantics cover the request's public acceptance scenarios and a simple consumer as well as PromptKit's multi-collection workflow.

2. Carry destinations through acquisition and imports

Thread the selected destination through root-to-engine conversion and private load requests. Preserve its ownership and semantic key representation. Separate source-local directive processing from application placement: extract directives at their original root, inherit placement through traversal, and place application content before checks and composition that require effective paths.

Make version protection destination-aware for underlay roots and imported fragments, preserving the main check's lifecycle position. Ensure diagnostics produced before final composition identify effective application paths where available while preserving actual source coordinates. Structural directive errors still need their genuine source-local declaration locations.

Retain repeated-source detection across the whole strict group, independent of destination. Preserve filesystem namespace, resource contexts, and traversal ordering. Account for destination depth and introduced nodes using existing budgets; distinguish them from acquired source bytes and physical source count.

Completion: all content-bearing source kinds and transitive imports contribute at the intended destination, with correct version protection and source identity.

3. Integrate composition, provenance, and the final lifecycle

Apply ordinary composition to placed contributions and ordinary overlay between tiers. Retain each source application at its effective placement for history; carry real source attribution and the selected synthetic-container convention through origin construction. Avoid a separate merge algorithm or a mount registry.

Exercise overlapping destinations, empty mappings, and a version-only domain override. Resolve sibling inheritance after the full precedence assembly, then use existing defaults, typed decoding, resolution, secrets, and validation. Check protection of the specification path through inheritance, selected overlays, and CLI overrides as well as direct contributions.

Completion: mounted and explicitly wrapped equivalents agree on values, conflicts, presence, and lifecycle results; physical locations remain accurate.

4. Complete inspection and public consumer workflows

Expose placement through the selected existing inspection surface, with detached records. Verify effective paths throughout metadata, explanation replay, history, rendering, and diffs. Apply the existing sensitivity rules to mounted and inherited fields, including superseded values. Do not fabricate coordinates for wrappers that were never written in a source file.

Demonstrate source-relative resource lookup from both resolution and completed results, including inherited fields and lower-tier fields surviving an override. Add executable public examples for one mounted fragment and a representative multi-collection assembly with peer composition and an intentional override. Applications select sources and destinations; the examples should require no consumer-side parsing, generic merging, or provenance reconstruction.

Completion: the supported root API is sufficient for the requested end-to-end workflow, including accurate placement and source-relative resource inspection.

5. Validate the full feature and update owning documentation

Use engine tests for placement mechanics and public tests for observable loading and inspection. Map every scenario in the request's acceptance table to a test or executable example, reusing existing coverage where it proves the same behavior. Include focused cases for:

• omitted/empty destinations, literal escaped and numeric keys, invalid index destinations, and the selected reserved-key and main-input rules;
• different destinations, shared destinations, ancestor overlaps, equal and unequal conflicts, explicit null, omitted fields, and empty contributions;
• recursive precedence, atomic sequences, and domain-version-only overrides;
• inherited import destinations, source-root directives, repeated physical sources at different destinations, and unchanged cycle/containment behavior;
• nested ordinary version fields versus actual protected-path and ancestor claims, including mounted imported content;
• local inheritance across peers and source contexts after inheritance/overlay;
• effective diagnostic paths with real locations, callback paths, history, rendering, diffs, and classified values protected with disposable canaries;
• equivalent content across every supported source kind, invalid loads with no usable result, introduced depth/node limits, and detached inspection ownership.

Follow the testing policy: protect observable contracts without fixing incidental wrapper allocation or helper-call sequences. Run focused tests, then the everyday check and ownership race validation in the development guide. Validate local documentation links. No independent performance benchmark runs are required for this roadmap.

Update declaration documentation, the configuration contract, consumer guidance, and the component overview as their behavior lands. Document how source-local coordinates relate to effective paths and how placement is inspected. Review release compatibility separately from publication; feature completion does not authorize a release.

Completion: all requested public scenarios pass, owning documentation describes implemented behavior, and consumers can perform one complete layered load with no compensating source-rewriting or post-load structural pass.

Open Decisions

Main-input destinations and absent inputs

Recommendation: allow the same destination model for content-bearing main inputs as for underlays, with VersionRule.Path remaining an effective application path and any specification declaration still owned by main. The checker retains its existing handling of an absent declaration. Treat NoSource as no contribution, even with a valid destination. This keeps Input coherent and avoids a role-only restriction. PromptKit can continue using root main; broader main placement must not add per-fragment checks or move the existing version-check phase.

Alternative: initially support non-root destinations only for underlay inputs and reject non-root main placement explicitly. This meets PromptKit's minimum need and reduces the first implementation's version-check interactions, but adds a role-dependent rule to a shared public type. Adopt that restriction only as an explicit scope choice, not because threading main placement is inconvenient. For NoSource, rejecting a nonempty destination could catch mistaken assembly declarations, but makes an otherwise absent contribution less composable.

Public placement inspection and synthetic ancestors

Recommendation: expose effective placement on existing source records, illustratively SourceInfo.Destination, including inherited import placement. Use original source identity for introduced ancestor mappings, documenting that any root-location fallback identifies the contributing document rather than a physically authored wrapper. Actual content keeps its real coordinates. This allows consumers to connect placement to sources without a new query system.

Alternative: add placement to origin records or a dedicated lookup, and represent synthetic ancestors without a physical location. This can distinguish authored and synthetic structure more precisely, but expands per-value metadata or adds an API when placement is fundamentally per source application. Evaluate the choice against inherited fields, composite origins, and the same source selected in separate tiers; do not infer placement from a final path by guesswork.

Reserved structural keys in destinations

Recommendation: use existing Path conventions for literal keys and parsed numeric tokens, reject explicit index segments, and preserve current structural reservations in the effective tree. Reject destination segments that would materialize configkit or a mapping-valued extends contrary to those contracts, with a clear structured destination error. Ordinary keys containing those words, dots, or slashes remain ordinary keys. Source-root directives are processed before placement and are not rejected simply because their application body is mounted.

Alternative: permit structural spellings as literal synthetic wrapper keys by distinguishing them from authored directives. This enables additional application paths but introduces an exception where mounted and explicitly wrapped inputs can have different meanings. It would require an explicit change to reserved-key semantics and is not needed for the requested collection paths. Running existing directive validation on the placed tree is another viable error mechanism, but must still identify that the invalid structure came from destination selection.

Empty mappings and presence of introduced containers

Recommendation: mounting an actual empty application mapping materializes its destination mappings as explicit contributions, matching authored wrappers. This preserves the request's equivalence principle and lets consumers inspect an intentionally empty collection. Such mappings still do not clear lower-tier children. An absent source contributes nothing.

Alternative: treat an empty mounted body as no contribution. This avoids empty containers for fragments containing only imports or no application fields, but loses equivalence with explicitly wrapped empty mappings and erases an observable presence distinction. If selected, document that exception and its effect on history and empty imported bodies rather than deriving it accidentally from whether the implementation allocated wrapper nodes.