ConfigKit/docs/roadmap/future.md

# Potential Future Features

This document lists potential enhancements for consideration in future ConfigKit
releases. Entries are not accepted contracts, implementation plans, or release
commitments. Adopting a candidate requires an explicit design decision and a
separate plan. Significant architectural exclusions and their rationale belong
in ADRs; this document does not maintain a list of rejected proposals.

## 1. Identified record sequences

Status: deferred. PromptKit proposed this capability from its retired proposal 2,
**Namespaced Collections As Identified Records**. The original proposal is in the
PromptKit repository at
`b5866dd:docs/roadmap/configuration-collections-identified-records.md`.
PromptKit selected mounted mappings instead; this feature is not a dependency
of that design or a request to restore multiple installed prompt versions.

A narrower [sequence placement and composition roadmap](sequence-composition.md)
now covers placing source mappings as sequence items, single-field identities,
duplicate rejection during strict composition, and whole-sequence replacement
between precedence layers. The additional capabilities below remain deferred
and are not requirements of that roadmap.

### Motivation and proposed behavior

Allow applications to declare that selected sequences contain records identified
by explicit fields, while retaining sequence-shaped output for slice decoding
and effective YAML. This could make composite identities and independently
authored catalogs more convenient than encoding identity in nested mapping keys.
Current [composition](../config.md#strict-document-composition) and
[overlay](../config.md#selected-overlays) treat sequences atomically.

The proposed opt-in registration belongs in the reusable application definition:
an exact effective sequence path after input/import placement, plus an ordered,
nonempty set of distinct direct string-field names. For example, records at
`profiles` could be identified by `(namespace, id)`. Records `(provider, brief)`
and `(application, brief)` would coexist, and a later layer could update the
provider record's temperature while retaining its backend and model.

The requested semantics, all still subject to design review, are:

- Match exact string tuples, preserving component boundaries and spelling,
  including empty strings. Every contribution supplies its complete identity;
  defaults, inheritance, filenames, and normalization do not supply it.
- Retain distinct identities. Matching records from strict peers recursively
  compose disjoint body fields and conflict on overlapping non-mapping body
  values, even equal values. Equal identity fields act as structural keys rather
  than conflicts and retain their contributing source evidence.
- Higher layers recursively overlay matching bodies. A changed identity denotes
  a different record, not a rename. Proposed ordering is first contribution in
  precedence-assembly order; matching records keep position and new ones append.
- Reject malformed identities, non-mapping records, non-null non-sequence values
  at registered paths, and duplicate tuples within one source sequence. Later
  layers cannot repair malformed structural identity in an acquired contribution.
- Treat an empty registered sequence as contributing no records; whole-path null
  retains ordinary conflict/replacement semantics. Unregistered sequences,
  including nested message lists, retain existing atomic behavior.
- Support record inheritance within the same registered sequence using an
  `extends` mapping containing the complete parent tuple. Resolve only final
  active relationships after precedence, preserve the child's identity, overlay
  its body on the parent's body, and consume the directive before decoding.
  Preserve cancellation, missing-parent/cycle errors, and declaring resource
  contexts. Existing string-based mapping-sibling inheritance stays unchanged.

Integrate ordinary and layered loads, imports, mounted paths, selected overlays,
and whole-sequence CLI overrides. Keep origins, presence, history, sensitivity,
and resource contexts attached to the correct record despite different source
and final indexes. Diagnostics must retain source locations without exposing
sensitive identity values. Final inspection may use snapshot-specific sequence
indexes; existing atomic sequence diffs need not become identity-aware.

Successful results would have valid, unique identities, including after defaults
and application resolution, without a second merge or inheritance pass. Existing
source-version checks remain independent; consumed headers cannot supply missing
record identities. No identity-selector path syntax, arbitrary merge callbacks,
wildcard registration, per-record deletion/rename operator, multiple inheritance,
or cross-path parent references are requested.

### Questions to resolve before implementation

- **Need and delivery scope:** does an active consumer benefit enough to justify
  the added semantics over keyed mappings? Are composite identities required?
  Staging composition before inheritance is possible but would leave part of
  the request incomplete.
- **Defaults and callbacks:** do default records merge by identity or retain
  existing sequence-replacement behavior? How are callback-created duplicates,
  identity edits, and reordering validated and reflected in provenance?
- **Null, empty, and order:** what happens when null clears a collection and a
  later layer reintroduces records? How do absent and explicitly empty sequences
  interact with defaults and first-contribution order?
- **Inheritance references:** how do peers compose relationship declarations,
  and how are complete references enforced without assembling partial tuples
  accidentally? What happens if identity fields overlap structural controls?
- **Registration paths:** how do overlapping registrations and paths beneath
  sequence indexes behave when assembly changes those indexes?
- **Evidence and lifecycle:** how are record identities mapped between source
  trees, history, inheritance, callback snapshots, and final indexes without
  mixing records' provenance or sensitivity? Where do identity checks run relative
  to header consumption and other structural processing?

Keep the feature deferred until its consumer value and full lifecycle design
justify reconsidering the sequence-merging exclusion. A correct final sequence
alone would not establish correct inspection or disclosure behavior.

## 2. Required version declarations for selected sources

Status: deferred. PromptKit requested a way to require version declarations on
a selected subset of source files while permitting other fragments to omit them.

### Motivation and proposed behavior

PromptKit could require each collection's root file (for example,
`collection.yml` or `meta.yml`, with naming still undecided) to declare an
accepted specification version. Other files would use the existing
[check-when-present behavior](../config.md#shared-source-version-checks).
This would enforce a rule that otherwise remains a documentation convention.

All declarations would use the same application-owned `Definition.Version`
path and acceptance policy. The proposal concerns where declarations are
required; it does not introduce independently versioned collections or subtrees.
A root declaration would not prove that headerless descendants declare a
compatible version or replace final configuration validation.

### Questions to resolve before implementation

- **Selection API:** should callers mark explicit inputs, select sources by
  path or predicate, or annotate imports? An input-level requirement is simpler,
  but cannot by itself select a root reached only through an import.
- **Scope:** does a requirement apply only to the selected document or also to
  its imports? Applying it only to that document best fits PromptKit's request;
  imported fragments could still declare versions checked by the shared policy.
- **Mode interaction:** how should selection interact with disabled source
  checks? Require an enabled mode, or activate checking for selected sources?
  Selection would strengthen check-when-present behavior and add no requirement
  beyond the load-wide required mode.
- **Source handling and evidence:** how should selection work across file,
  reader, byte, and mapping inputs, glob matches, and repeated imports? Define
  behavior for absent optional inputs and preserve source-local diagnostics,
  inspection evidence, and existing sensitivity handling.

## 3. Performance experiments

Status: deferred pending evidence. The completed performance audit identified
four internal optimization candidates for separate investigation:

- Branch-local explanation replay instead of the current whole-tree replay.
- Redesigning the private metadata-copy handoff while preserving detached public
  records.
- Caching schema descriptors across loads.
- Indexing complex contributor-trace membership while preserving its equality
  rules and compact linked traces.

These are experiments, not committed implementation work or promised speedups.
Reconsider them with evidence that the benefit justifies the ownership,
retained-memory, or equivalence risks. The completed audit and implementation
evidence are preserved in Git history at `8e91a2b` under `docs/roadmap/`.

### Questions to resolve before implementation

- Which representative workloads show enough remaining cost to justify each
  optimization, and what measurements would establish a useful improvement?
- Can narrower replay and membership indexes preserve explanation results,
  ordering, provenance, and sensitivity behavior?
- Can metadata handoff changes preserve detached public records and ownership
  across loads without introducing aliasing?
- What cache lifetime and concurrency design would make schema reuse worthwhile
  without excessive retained memory?