eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Propose identity-based collection records for ConfigKit

Browse Source
This commit is contained in:
Eric Rakestraw
2026-10-14 01:32:10 +00:00
parent 9e5ed08635
commit c7d1b0dca7
2 changed files with 436 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

431
docs/roadmap/configuration-collections-identified-records.md Normal file
View File

@@ -0,0 +1,431 @@
# Proposal 2: Namespaced Collections As Identified Records
Status: proposed alternative; not implemented or accepted. This is a middle
ground between [proposal 1](configuration-collections-current-configkit.md),
which uses current ConfigKit, and a future proposal 3 that may change source
and document organization more substantially. The parent
[namespace and collection roadmap](namespaces.md) retains the accepted product
decisions and the pending configuration decision.
## Recommendation And Scope
Represent prompts, profiles, and backends as sequences of records with explicit
identity fields. Prefer separate `namespace` and local `id` fields. Ask ConfigKit
to support opt-in, application-declared identity rules at selected configuration
paths, so those records compose and overlay by identity rather than position.
The extension is one cohesive capability: identified record sequences, including
their strict composition, overlay, inheritance, and provenance behavior. It is
not a general user-defined merge framework. ConfigKit continues to own the
complete assembly process and hands PromptKit one final resolved configuration.
Keep the root `prompts:`, `profiles:`, and `backends:` keys. Keep collection
metadata as the ordinary namespace-keyed mapping from proposal 1. Keep the main
input's specification declaration and the existing source/layer model. This
proposal does not add bare single-record files, source mounting, implicit
namespace injection, or per-import specification checking. Those are separate
questions for proposal 3.
## Current Baseline And Required Extension
The reviewed ConfigKit baseline remains v0.7.1, local checkout commit
`6fffde4af8f31f99d3ee295c426eb3234cb0fb77`. Its
[configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.7.1/docs/config.md)
and the implementation/tests reviewed for proposal 1 establish that:
- Sequences are atomic: overlapping sequences conflict during strict composition
and replace one another across precedence layers.
- ConfigKit can already decode a single sequence of mappings into application
records. It does not currently use their fields as merge identities.
- `extends` currently identifies immediate mapping siblings. A sequence-element
mapping cannot establish its own sibling relationship, and mapping-valued
`extends` is not a supported reference form.
- Final decoding, application resolution, and semantic validation happen after
configuration assembly and inheritance.
Consequently, changing PromptKit's document structs alone would not implement
this proposal. Concatenating sequences in PromptKit or converting each source
to keyed mappings before calling ConfigKit would violate the chosen boundary.
## Proposed Authoring Format
Each prompt record describes one prompt version, without a nested version-key
mapping. Use explicit strings for identity fields:
```yaml
collections:
weatherreporter:
version: "2.4.0"
prompts:
- namespace: weatherreporter
id: summarize.alerts
version: "1"
default_profile: openrouter/gemini-flash-latest
messages:
- role: user
content: Summarize the weather alerts.
output:
format: text
validation_mode: none
```
A second collection can use the same local ID without colliding:
```yaml
collections:
homewatch:
version: "3.1.0"
prompts:
- namespace: homewatch
id: summarize.alerts
version: "1"
default_profile: openrouter/gemini-flash-latest
messages:
- role: user
content: Summarize the household alerts.
output:
format: text
validation_mode: none
```
The profile name is an illustrative configured reference, not a statement about
current provider inventory. A synthetic provider contribution shows the same
record shape for backends and profiles:
```yaml
collections:
provider:
version: "2026.10.14"
backends:
- namespace: provider
id: default
endpoint: https://inference.example.invalid/v1
profiles:
- namespace: provider
id: brief
backend: provider/default
model: vendor/example-model
temperature: 0.2
```
These are independent format examples; the synthetic provider does not satisfy
the earlier examples' OpenRouter reference. An assembled configuration must
supply its selected references through ordinary configuration and existing
PromptKit validation/selection rules.
Retain the existing `default_profile` field instead of also renaming it to
`profile`. Reducing nesting and making identity explicit are the changes under
evaluation; unrelated field renames are unnecessary.
### Identity And Public References
Register these record identities with ConfigKit:
| Configuration path | Identity fields, in order |
| --- | --- |
| `backends` | `namespace`, `id` |
| `profiles` | `namespace`, `id` |
| `prompts` | `namespace`, `id`, `version` |
PromptKit turns each final `(namespace, id)` pair into its public
`namespace/id` identifier after handoff; the prompt version remains a separate
selection dimension. The conversion creates runtime indexes, not another merge
or source-processing pass. It must reject invalid final domain identities rather
than silently re-keying them.
Require identity fields to be explicitly present strings in every record,
including partial override records. A prompt's nested `version` is a prompt
identity component, not the protected top-level specification declaration.
Do not derive any identity component from a typed default, parent, filename,
collection metadata entry, or caller execution scope: matching happens before
those application-level interpretations.
For unqualified application definitions, use `namespace: ""` explicitly; their
public identifier is the local `id` alone, without a leading slash.
Nonempty namespaces and local IDs follow the accepted namespace grammar;
local IDs cannot contain `/`. Preserve exact spelling. PromptKit owns these
domain checks on the final values; ConfigKit only requires usable structural
identity fields and compares exact string tuples. Distinct tuples must never
be conflated by concatenating fields with an unescaped delimiter internally.
Stored `default_profile` and `backend` references remain fully qualified or
exact unqualified strings, as already selected in the namespace roadmap.
Provider model values retain their existing syntax, including slashes.
## Proposed ConfigKit Contract
### Explicit Registration
The application's ConfigKit definition declares record-sequence rules at exact
semantic paths, with ordered identity field names. The table above is a
conceptual registration, not a proposed Go API declaration. ConfigKit does not
guess from field names such as `id`, and configuration files cannot silently
switch their own merge policy. No arbitrary merge callbacks are required.
An initial implementation only needs direct string identity fields at explicitly
registered paths. Wildcard registration, computed keys, key normalization,
implicit identity defaults, and nested identity expressions are not required
by this proposal.
At a registered path, a non-null contribution must be a sequence of mappings.
Every record must supply all structural identity fields with the declared
string type. Missing/null/non-string identity fields or non-mapping records
are ConfigKit structural errors: the loader cannot assign them a record identity.
These errors cannot be hidden by a later layer. Nonidentity body fields retain
the existing final-value decoding/validation lifecycle.
For consistency with duplicate mapping-key handling, recommend rejecting two
records with the same tuple within a single source sequence. Across separate
sources, matching tuples participate in normal composition as described below.
This is a proposed explicit upstream rule, not a claim about current ConfigKit
or a PromptKit source-level duplicate scan.
### Strict Composition And Overlay
Treat identity fields as the record's structural key. Their repeated equal
values are not overlapping body fields that cause composition conflicts.
Retain the fields in the final record and preserve their origin information.
All remaining fields form the body to which existing mapping rules apply.
| Contributions at a registered path | Proposed result |
| --- | --- |
| Different identity tuples in one group or different layers | Retain both records |
| Same tuple across peer sources, disjoint body fields | Recursively compose one body |
| Same tuple across peer sources, overlapping non-mapping body values | Composition conflict, even if equal |
| Same tuple in a higher-priority layer | Recursively overlay the higher body on the lower body |
| Same prompt namespace/local ID but different prompt version | Distinct records |
| Identity field changed in a later contribution | A different record; never an implicit rename of the old record |
Once a tuple is selected for matching, identity fields cannot themselves be
overlaid into a different tuple. This avoids record identity changing during
composition. A higher layer need only repeat the tuple and changed body fields:
```yaml
version: 1
profiles:
- namespace: provider
id: brief
temperature: 0.1
```
The provider/backend/model fields inherited through layering remain. A new
record introduced by an override still undergoes ordinary final validation;
matching by identity does not guarantee it is a complete runnable profile.
For registered sequences, recommend that `[]` contributes no records and does
not clear lower records, like an empty keyed mapping. This differs deliberately
from ordinary sequence replacement. Explicit null at the whole registered path
uses ordinary cross-layer replacement and can clear that whole collection when
the application's final schema permits null. Within strict composition, null
conflicts with another present value as usual. No per-record deletion marker,
rename operation, or general patch language is added.
Process tiers in precedence-assembly order, lowest first, with deterministic
source order within a tier. Preserve first-contribution order for distinct
records, retain a matched record's position, and append newly encountered
tuples. This makes output deterministic without giving position precedence or
identity meaning; acquisition order need not equal assembly order. PromptKit
can still sort its public inventory independently.
**Unregistered sequences keep all current semantics.** In particular, `messages`
remains atomic: it conflicts across strict peer definitions and is replaced
as a whole by a later override. Identified record support must not turn every
sequence into a union or keyed merge.
### Inheritance Within A Registered Sequence
The extension must give identified records a sibling identity space. For
composite identities, recommend a structural `extends` mapping containing the
complete parent tuple. A profile example is:
```yaml
profiles:
- namespace: weatherreporter
id: brief
extends:
namespace: provider
id: brief
temperature: 0.4
```
ConfigKit resolves this relationship within the same registered `profiles`
sequence after all composition and precedence. It overlays the child's body
over the parent's resolved body, retains the child's identity fields, and
consumes `extends` before typed decoding. Parent identity fields are never
inherited into the child. References for a three-field identity, such as a
prompt record, must include all three fields; no version guessing or namespace
fallback is introduced. References cannot cross resource paths.
Preserve existing behavior for omitted relationships, `extends: null`, missing
parents, cycles, and overridden relationships. A partial child contribution
can retain a lower-layer relationship, and only final active relationships are
resolved. Inherited file-backed content keeps the declaring field's source
context, not the child's source directory.
This structural reference form is only enabled for records at registered paths.
Existing mapping-sibling string inheritance remains unchanged elsewhere. A
future single-field record rule could use that same field in the reference
mapping; no PromptKit-specific interpretation of slash-separated parent strings
is needed inside ConfigKit.
### Provenance, Inspection, And Typed Handoff
ConfigKit owns record matching and provenance throughout assembly. A conflict
must identify the record and conflicting source locations, not merely report
that two sequences occupy `prompts`. Preserve ConfigKit's safe diagnostic
contract when representing that identity; do not indiscriminately echo input
field values in error messages. Identity-field declarations retain their
contributing locations even when those equal structural fields do not conflict.
Keep the final application-facing shape as a sequence of records, suitable for
ordinary slice decoding and effective-config rendering. ConfigKit can use a
private identity index, but should not expose private normalized nodes to
PromptKit. Final body presence/null state, field origins, inheritance history,
redaction, and passive source resource contexts must survive matching.
No new public identity-selector path language is required for this proposal.
Inspection may address final records by sequence index, with ConfigKit mapping
effective field paths to the appropriate historical source locations internally.
Indexes are snapshot-specific; consumers locate a record by its identity before
constructing an indexed path. Histories must not become attached to another
record merely because its source position differed. Existing positional diff
behavior need not become an identity-aware cross-snapshot diff in this work.
Rules must apply consistently to all enabled source kinds and composition
entry points, including imports and layered input. Existing explicit CLI
overrides must not leave duplicate or malformed structural identities in the
final registered sequences; ConfigKit owns their integration and consistency
checks. Ordinary application resolution must not trigger another merge or
inheritance pass. Detailed public option declarations and implementation
mechanics belong in the upstream design if this proposal is selected.
## Collection Metadata, Versions, And Assembly
Retain proposal 1's ordinary `collections.<namespace>.version` metadata and
effective-descriptor inventory. It is distinct from the generic ConfigKit term
"record sequence." Collection declarations still follow mapping composition;
they are not a source registry or exclusive ownership claim. Repeated metadata
release strings conflict in a strict peer group and can be overridden in a
later layer. PromptKit does not add an independent duplicate-namespace scan.
Also retain proposal 1's main-owned `version: 1` declaration and its proposed
explicit-version requirement. The value denotes the specification for the
assembled configuration, not each imported package. Prompt record versions and
collection release strings do not change that ownership rule. These proposals
are alternative candidates for a future standard; they are not two simultaneously
supported meanings for a released specification version.
Combine peer collection inputs in one strict underlay group; use later groups
or main for intentional overrides. Callers continue to choose grouping and
precedence. Directory inputs and explicit imports use existing ConfigKit
selection and resource-context rules. There is no PromptKit pre-parser that
reformats records, removes version fields, or replays the original sources.
**Independent collection specification checks remain unresolved.** This bounded
extension does not permit imported or underlay roots to declare their own
protected top-level `version`. A publisher must still supply configuration
compatible with the selected main standard. Proposal 3 may consider a broader
source/version contract; proposal 2 must not imply that record identity solves
that separate issue.
## Ergonomic Alternatives And Tradeoffs
The first syntax discussed is also compatible with the generic extension:
```yaml
prompts:
- id: weatherreporter/summarize.alerts
version: "1"
default_profile: openrouter/gemini-flash-latest
messages:
- role: user
content: Summarize the weather alerts.
output:
format: text
validation_mode: none
```
That alternative registers prompt identity as `(id, version)` and profile/backend
identity as `(id)`. It is simpler to align with public reference strings and
requires fewer identity fields in partial overrides. It repeats the namespace
inside each full ID and gives authors no separate namespace field.
Prefer the composite form for explicit author-facing fields and easier editing
of local names. Do not support both forms opportunistically in the same standard:
that would require aliasing or normalization before identity matching. Select
one representation for the adopted specification. The first form remains a
reasonable alternative if the extra namespace line proves more cumbersome than
the qualified string.
Retaining keyed mappings only for profiles could avoid extending inheritance
to records, but would give authors different identity conventions by resource
type. The recommendation favors a consistent record model and includes the
necessary generic inheritance work explicitly.
| Concern | Proposal 1 | Proposal 2 |
| --- | --- | --- |
| Author identity | Qualified YAML mapping key | Explicit namespace/local-ID fields |
| Prompt version | Nested mapping key | Field in each prompt record |
| Cross-source composition | Current ConfigKit | Opt-in record identity extension |
| Body conflicts and precedence | Mapping rules | Same mapping rules after tuple matching |
| Root `prompts:` wrapper | Required | Required |
| Provider model strings and execution references | Unchanged | Unchanged |
| Independent imported specification checks | Unavailable | Still unavailable |
| Main maintenance cost | PromptKit schema/API work | PromptKit work plus ConfigKit record lifecycle support |
This is more than a syntax-only change, but remains bounded: it introduces one
generic representation for identified entries while preserving configuration
source ownership and ordinary merge policy. It does not introduce collection
installation, dependency resolution, namespace ownership, mounted sources,
arbitrary transforms, or a general-purpose merge strategy system.
## Delivery And Validation
If selected, agree the upstream ConfigKit record contract first, implement and
test it there, then adopt that capability in PromptKit's prospective schema.
PromptKit would decode final records, validate domain identities/settings, build
its ordinary execution indexes, and implement the already accepted scope and
resource-discovery behavior plus the proposed collection inventory. Bundled
catalog/profile removal remains the accepted direction.
Migration must convert each old keyed resource into its chosen record identity,
flatten prompt-version entries, update any inheritance reference syntax, and
preserve provider model strings. Choose specification and software release
versions deliberately; do not silently reinterpret deployed configuration.
The upstream acceptance cases should include:
- Different tuples coexist; same local prompt names in different namespaces
never collide; different versions of one prompt remain separate.
- Matching tuples compose disjoint bodies but reject overlapping scalar/null/
ordinary-sequence body fields, including equal values.
- Matching tuples overlay across tiers without deleting unrelated records.
Changing a tuple adds another record; `[]`, whole-path null, and ordinary
message sequences follow their distinct documented semantics.
- Missing or malformed identity fields and repeated tuples in one source fail
structurally; identity comparisons cannot collide through delimiter encoding.
- Composite inheritance resolves final relationships, preserves child identity,
rejects missing/cyclic parents, and retains declaring resource contexts.
- Matching, effective indexes, source positions, sensitivity, and provenance
remain consistent through imports, overrides, inheritance, and rendering.
- Unregistered configuration loads and ordinary sequence behavior are unchanged.
- Existing version-path protections remain intact; record `version` fields are
not mistaken for the protected root field.
PromptKit acceptance should exercise the authoring examples, final identity
conversion, scoped selection, metadata inventory, and migrated references with
offline collaborators. The record composition and inheritance examples require
the proposed upstream extension; they are not executable supported workflows
on ConfigKit v0.7.1 today. Syntax checks and current-behavior probes can establish
the starting point, but cannot validate an unimplemented extension.
Review verification on 2026-10-14: a temporary offline consumer of the local
ConfigKit checkout decoded the five examples without the new inheritance form,
confirmed that the composite `extends` example is rejected today, and confirmed
current sequence conflict/replacement behavior across peer and precedence layers.
Local documentation links and whitespace checks passed. The temporary consumer
and its module replacement are outside both repositories.
This proposal remains for evaluation alongside proposal 1. The parent roadmap's
configuration decision remains open, and broader source-placement/versioning
changes are reserved for the later proposal 3 discussion.

6
docs/roadmap/namespaces.md
View File

@@ -487,7 +487,11 @@ dependencies instead.
[Proposal 1: current ConfigKit](configuration-collections-current-configkit.md)
develops a concrete configuration standard for the existing composition rules.
It is a proposal for comparison; this decision remains open.
[Proposal 2: identified records](configuration-collections-identified-records.md)
uses an opt-in upstream extension for identity-based record composition and
inheritance. Both retain main-owned specification versioning; broader source
and version changes can be evaluated in a later proposal 3. These are proposals
for comparison; this decision remains open.
**Desired capability:** describe collection namespace and release version in
configuration and expose an API through which downstream applications can list