promptkit/docs/roadmap/configuration-collections-identified-records.md

Proposal 2: Namespaced Collections As Identified Records

Status: proposed alternative; not implemented or accepted. This is a middle ground between proposal 1, which uses current ConfigKit, and proposal 3, which changes source placement and document organization. Proposal 3 also adopts the later requirement for one installed definition per prompt and an optional version; this proposal retains versioned record identities for comparison. The parent namespace and collection roadmap retains the accepted product decisions and the pending configuration decision.

Recommendation And Scope

Represent prompts, profiles, and backends as sequences of records with explicit identity fields. Prefer separate namespace and local id fields. Ask ConfigKit to support opt-in, application-declared identity rules at selected configuration paths, so those records compose and overlay by identity rather than position.

The extension is one cohesive capability: identified record sequences, including their strict composition, overlay, inheritance, and provenance behavior. It is not a general user-defined merge framework. ConfigKit continues to own the complete assembly process and hands PromptKit one final resolved configuration.

Keep the root prompts:, profiles:, and backends: keys. Keep collection metadata as the ordinary namespace-keyed mapping from proposal 1. Keep the main input's specification declaration and the existing source/layer model. This proposal does not add bare single-record files, source mounting, implicit namespace injection, or per-import specification checking. Those are separate questions for proposal 3.

Current Baseline And Required Extension

The reviewed ConfigKit baseline remains v0.7.1, local checkout commit 6fffde4af8f31f99d3ee295c426eb3234cb0fb77. Its configuration contract and the implementation/tests reviewed for proposal 1 establish that:

• Sequences are atomic: overlapping sequences conflict during strict composition and replace one another across precedence layers.
• ConfigKit can already decode a single sequence of mappings into application records. It does not currently use their fields as merge identities.
• extends currently identifies immediate mapping siblings. A sequence-element mapping cannot establish its own sibling relationship, and mapping-valued extends is not a supported reference form.
• Final decoding, application resolution, and semantic validation happen after configuration assembly and inheritance.

Consequently, changing PromptKit's document structs alone would not implement this proposal. Concatenating sequences in PromptKit or converting each source to keyed mappings before calling ConfigKit would violate the chosen boundary.

Proposed Authoring Format

Each prompt record describes one prompt version, without a nested version-key mapping. Use explicit strings for identity fields:

collections:
  weatherreporter:
    version: "2.4.0"
prompts:
  - namespace: weatherreporter
    id: summarize.alerts
    version: "1"
    default_profile: openrouter/gemini-flash-latest
    messages:
      - role: user
        content: Summarize the weather alerts.
    output:
      format: text
      validation_mode: none

A second collection can use the same local ID without colliding:

collections:
  homewatch:
    version: "3.1.0"
prompts:
  - namespace: homewatch
    id: summarize.alerts
    version: "1"
    default_profile: openrouter/gemini-flash-latest
    messages:
      - role: user
        content: Summarize the household alerts.
    output:
      format: text
      validation_mode: none

The profile name is an illustrative configured reference, not a statement about current provider inventory. A synthetic provider contribution shows the same record shape for backends and profiles:

collections:
  provider:
    version: "2026.10.14"
backends:
  - namespace: provider
    id: default
    endpoint: https://inference.example.invalid/v1
profiles:
  - namespace: provider
    id: brief
    backend: provider/default
    model: vendor/example-model
    temperature: 0.2

These are independent format examples; the synthetic provider does not satisfy the earlier examples' OpenRouter reference. An assembled configuration must supply its selected references through ordinary configuration and existing PromptKit validation/selection rules.

Retain the existing default_profile field instead of also renaming it to profile. Reducing nesting and making identity explicit are the changes under evaluation; unrelated field renames are unnecessary.

Identity And Public References

Register these record identities with ConfigKit:

Configuration path	Identity fields, in order
backends	namespace, id
profiles	namespace, id
prompts	namespace, id, version

PromptKit turns each final (namespace, id) pair into its public namespace/id identifier after handoff; the prompt version remains a separate selection dimension. The conversion creates runtime indexes, not another merge or source-processing pass. It must reject invalid final domain identities rather than silently re-keying them.

Require identity fields to be explicitly present strings in every record, including partial override records. A prompt's nested version is a prompt identity component, not the protected top-level specification declaration. Do not derive any identity component from a typed default, parent, filename, collection metadata entry, or caller execution scope: matching happens before those application-level interpretations.

For unqualified application definitions, use namespace: "" explicitly; their public identifier is the local id alone, without a leading slash. Nonempty namespaces and local IDs follow the accepted namespace grammar; local IDs cannot contain /. Preserve exact spelling. PromptKit owns these domain checks on the final values; ConfigKit only requires usable structural identity fields and compares exact string tuples. Distinct tuples must never be conflated by concatenating fields with an unescaped delimiter internally.

Stored default_profile and backend references remain fully qualified or exact unqualified strings, as already selected in the namespace roadmap. Provider model values retain their existing syntax, including slashes.

Proposed ConfigKit Contract

Explicit Registration

The application's ConfigKit definition declares record-sequence rules at exact semantic paths, with ordered identity field names. The table above is a conceptual registration, not a proposed Go API declaration. ConfigKit does not guess from field names such as id, and configuration files cannot silently switch their own merge policy. No arbitrary merge callbacks are required.

An initial implementation only needs direct string identity fields at explicitly registered paths. Wildcard registration, computed keys, key normalization, implicit identity defaults, and nested identity expressions are not required by this proposal.

At a registered path, a non-null contribution must be a sequence of mappings. Every record must supply all structural identity fields with the declared string type. Missing/null/non-string identity fields or non-mapping records are ConfigKit structural errors: the loader cannot assign them a record identity. These errors cannot be hidden by a later layer. Nonidentity body fields retain the existing final-value decoding/validation lifecycle.

For consistency with duplicate mapping-key handling, recommend rejecting two records with the same tuple within a single source sequence. Across separate sources, matching tuples participate in normal composition as described below. This is a proposed explicit upstream rule, not a claim about current ConfigKit or a PromptKit source-level duplicate scan.

Strict Composition And Overlay

Treat identity fields as the record's structural key. Their repeated equal values are not overlapping body fields that cause composition conflicts. Retain the fields in the final record and preserve their origin information. All remaining fields form the body to which existing mapping rules apply.

Contributions at a registered path	Proposed result
Different identity tuples in one group or different layers	Retain both records
Same tuple across peer sources, disjoint body fields	Recursively compose one body
Same tuple across peer sources, overlapping non-mapping body values	Composition conflict, even if equal
Same tuple in a higher-priority layer	Recursively overlay the higher body on the lower body
Same prompt namespace/local ID but different prompt version	Distinct records
Identity field changed in a later contribution	A different record; never an implicit rename of the old record

Once a tuple is selected for matching, identity fields cannot themselves be overlaid into a different tuple. This avoids record identity changing during composition. A higher layer need only repeat the tuple and changed body fields:

version: 1
profiles:
  - namespace: provider
    id: brief
    temperature: 0.1

The provider/backend/model fields inherited through layering remain. A new record introduced by an override still undergoes ordinary final validation; matching by identity does not guarantee it is a complete runnable profile.

For registered sequences, recommend that [] contributes no records and does not clear lower records, like an empty keyed mapping. This differs deliberately from ordinary sequence replacement. Explicit null at the whole registered path uses ordinary cross-layer replacement and can clear that whole collection when the application's final schema permits null. Within strict composition, null conflicts with another present value as usual. No per-record deletion marker, rename operation, or general patch language is added.

Process tiers in precedence-assembly order, lowest first, with deterministic source order within a tier. Preserve first-contribution order for distinct records, retain a matched record's position, and append newly encountered tuples. This makes output deterministic without giving position precedence or identity meaning; acquisition order need not equal assembly order. PromptKit can still sort its public inventory independently.

Unregistered sequences keep all current semantics. In particular, messages remains atomic: it conflicts across strict peer definitions and is replaced as a whole by a later override. Identified record support must not turn every sequence into a union or keyed merge.

Inheritance Within A Registered Sequence

The extension must give identified records a sibling identity space. For composite identities, recommend a structural extends mapping containing the complete parent tuple. A profile example is:

profiles:
  - namespace: weatherreporter
    id: brief
    extends:
      namespace: provider
      id: brief
    temperature: 0.4

ConfigKit resolves this relationship within the same registered profiles sequence after all composition and precedence. It overlays the child's body over the parent's resolved body, retains the child's identity fields, and consumes extends before typed decoding. Parent identity fields are never inherited into the child. References for a three-field identity, such as a prompt record, must include all three fields; no version guessing or namespace fallback is introduced. References cannot cross resource paths.

Preserve existing behavior for omitted relationships, extends: null, missing parents, cycles, and overridden relationships. A partial child contribution can retain a lower-layer relationship, and only final active relationships are resolved. Inherited file-backed content keeps the declaring field's source context, not the child's source directory.

This structural reference form is only enabled for records at registered paths. Existing mapping-sibling string inheritance remains unchanged elsewhere. A future single-field record rule could use that same field in the reference mapping; no PromptKit-specific interpretation of slash-separated parent strings is needed inside ConfigKit.

Provenance, Inspection, And Typed Handoff

ConfigKit owns record matching and provenance throughout assembly. A conflict must identify the record and conflicting source locations, not merely report that two sequences occupy prompts. Preserve ConfigKit's safe diagnostic contract when representing that identity; do not indiscriminately echo input field values in error messages. Identity-field declarations retain their contributing locations even when those equal structural fields do not conflict.

Keep the final application-facing shape as a sequence of records, suitable for ordinary slice decoding and effective-config rendering. ConfigKit can use a private identity index, but should not expose private normalized nodes to PromptKit. Final body presence/null state, field origins, inheritance history, redaction, and passive source resource contexts must survive matching.

No new public identity-selector path language is required for this proposal. Inspection may address final records by sequence index, with ConfigKit mapping effective field paths to the appropriate historical source locations internally. Indexes are snapshot-specific; consumers locate a record by its identity before constructing an indexed path. Histories must not become attached to another record merely because its source position differed. Existing positional diff behavior need not become an identity-aware cross-snapshot diff in this work.

Rules must apply consistently to all enabled source kinds and composition entry points, including imports and layered input. Existing explicit CLI overrides must not leave duplicate or malformed structural identities in the final registered sequences; ConfigKit owns their integration and consistency checks. Ordinary application resolution must not trigger another merge or inheritance pass. Detailed public option declarations and implementation mechanics belong in the upstream design if this proposal is selected.

Collection Metadata, Versions, And Assembly

Retain proposal 1's ordinary collections.<namespace>.version metadata and effective-descriptor inventory. It is distinct from the generic ConfigKit term "record sequence." Collection declarations still follow mapping composition; they are not a source registry or exclusive ownership claim. Repeated metadata release strings conflict in a strict peer group and can be overridden in a later layer. PromptKit does not add an independent duplicate-namespace scan.

Also retain proposal 1's main-owned version: 1 declaration and its proposed explicit-version requirement. The value denotes the specification for the assembled configuration, not each imported package. Prompt record versions and collection release strings do not change that ownership rule. These proposals are alternative candidates for a future standard; they are not two simultaneously supported meanings for a released specification version.

Combine peer collection inputs in one strict underlay group; use later groups or main for intentional overrides. Callers continue to choose grouping and precedence. Directory inputs and explicit imports use existing ConfigKit selection and resource-context rules. There is no PromptKit pre-parser that reformats records, removes version fields, or replays the original sources.

Independent collection specification checks remain unresolved. This bounded extension does not permit imported or underlay roots to declare their own protected top-level version. A publisher must still supply configuration compatible with the selected main standard. Proposal 3 may consider a broader source/version contract; proposal 2 must not imply that record identity solves that separate issue.

Ergonomic Alternatives And Tradeoffs

The first syntax discussed is also compatible with the generic extension:

prompts:
  - id: weatherreporter/summarize.alerts
    version: "1"
    default_profile: openrouter/gemini-flash-latest
    messages:
      - role: user
        content: Summarize the weather alerts.
    output:
      format: text
      validation_mode: none

That alternative registers prompt identity as (id, version) and profile/backend identity as (id). It is simpler to align with public reference strings and requires fewer identity fields in partial overrides. It repeats the namespace inside each full ID and gives authors no separate namespace field.

Prefer the composite form for explicit author-facing fields and easier editing of local names. Do not support both forms opportunistically in the same standard: that would require aliasing or normalization before identity matching. Select one representation for the adopted specification. The first form remains a reasonable alternative if the extra namespace line proves more cumbersome than the qualified string.

Retaining keyed mappings only for profiles could avoid extending inheritance to records, but would give authors different identity conventions by resource type. The recommendation favors a consistent record model and includes the necessary generic inheritance work explicitly.

Concern	Proposal 1	Proposal 2
Author identity	Qualified YAML mapping key	Explicit namespace/local-ID fields
Prompt version	Nested mapping key	Field in each prompt record
Cross-source composition	Current ConfigKit	Opt-in record identity extension
Body conflicts and precedence	Mapping rules	Same mapping rules after tuple matching
Root prompts: wrapper	Required	Required
Provider model strings and execution references	Unchanged	Unchanged
Independent imported specification checks	Unavailable	Still unavailable
Main maintenance cost	PromptKit schema/API work	PromptKit work plus ConfigKit record lifecycle support

This is more than a syntax-only change, but remains bounded: it introduces one generic representation for identified entries while preserving configuration source ownership and ordinary merge policy. It does not introduce collection installation, dependency resolution, namespace ownership, mounted sources, arbitrary transforms, or a general-purpose merge strategy system.

Delivery And Validation

If selected, agree the upstream ConfigKit record contract first, implement and test it there, then adopt that capability in PromptKit's prospective schema. PromptKit would decode final records, validate domain identities/settings, build its ordinary execution indexes, and implement the already accepted scope and resource-discovery behavior plus the proposed collection inventory. Bundled catalog/profile removal remains the accepted direction.

Migration must convert each old keyed resource into its chosen record identity, flatten prompt-version entries, update any inheritance reference syntax, and preserve provider model strings. Choose specification and software release versions deliberately; do not silently reinterpret deployed configuration.

The upstream acceptance cases should include:

• Different tuples coexist; same local prompt names in different namespaces never collide; different versions of one prompt remain separate.
• Matching tuples compose disjoint bodies but reject overlapping scalar/null/ ordinary-sequence body fields, including equal values.
• Matching tuples overlay across tiers without deleting unrelated records. Changing a tuple adds another record; [], whole-path null, and ordinary message sequences follow their distinct documented semantics.
• Missing or malformed identity fields and repeated tuples in one source fail structurally; identity comparisons cannot collide through delimiter encoding.
• Composite inheritance resolves final relationships, preserves child identity, rejects missing/cyclic parents, and retains declaring resource contexts.
• Matching, effective indexes, source positions, sensitivity, and provenance remain consistent through imports, overrides, inheritance, and rendering.
• Unregistered configuration loads and ordinary sequence behavior are unchanged.
• Existing version-path protections remain intact; record version fields are not mistaken for the protected root field.

PromptKit acceptance should exercise the authoring examples, final identity conversion, scoped selection, metadata inventory, and migrated references with offline collaborators. The record composition and inheritance examples require the proposed upstream extension; they are not executable supported workflows on ConfigKit v0.7.1 today. Syntax checks and current-behavior probes can establish the starting point, but cannot validate an unimplemented extension.

Review verification on 2026-10-14: a temporary offline consumer of the local ConfigKit checkout decoded the five examples without the new inheritance form, confirmed that the composite extends example is rejected today, and confirmed current sequence conflict/replacement behavior across peer and precedence layers. Local documentation links and whitespace checks passed. The temporary consumer and its module replacement are outside both repositories.

This proposal remains for evaluation alongside proposal 1. The parent roadmap's configuration decision remains open. Proposal 3 develops the alternative based on source placement; independent collection specification checking remains open.