eric/ConfigKit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Record whole-record sequence replacement and application identity constraints

Browse Source
This commit is contained in:
Eric Rakestraw
2026-10-20 13:57:19 +00:00
parent b58cd75c98
commit cba3a0efb0
1 changed files with 152 additions and 77 deletions
Download Patch File Download Diff File Expand all files Collapse all files

229
docs/roadmap/sequences.md
View File

@@ -1,7 +1,8 @@
# Configuration-Declared Sequence Identity And Layered Append
# Sequence Identity And Cross-Tier Combination
Status: draft proposal for discussion. This document captures the requested
capabilities and initial recommendations; the open decisions are not settled.
capabilities, settled design choices, and initial recommendations. Choices under
Open Decisions remain unsettled.
It is not an implementation plan or an implemented contract. All new syntax
shown below is illustrative.
@@ -10,14 +11,14 @@ shown below is illustrative.
Allow configuration authors to assemble collections from independently authored
records, declare their identity in configuration, and progressively extend a
baseline collection through higher layers. Authors should also be able to
replace the accumulated collection explicitly.
replace the accumulated collection or replace individual records by identity.
The motivating example is a prompt collection whose defaults supply some
records and whose application or installation layers add others. A repeated
identity should produce an error when records are combined. This proposal
concerns adding distinct records and replacing whole collections; matching
records would not acquire field-by-field update semantics merely by adding
this feature.
records and whose application or installation layers add or supersede records.
Across tiers, authors need three operations: whole-sequence replacement,
duplicate-rejecting append, and whole-record replacement by identity. The latter
retains unmatched lower-tier records and adds higher-tier records with new
identities. Matching records are replaced in full.
## Current Baseline
@@ -43,9 +44,10 @@ later. Extending one does not automatically imply extending the other.
The proposed import syntax introduces an identity declaration, tentatively
named `key`, that accepts either one field or a list of fields for a compound
identity. The original proposal is to move the application's sequence
registration into configuration. Whether to remove, retain, or adapt the Go
registration remains open.
identity. Applications retain an optional interface for registering a collection
and imposing its identity definition. Configuration declarations must conform
to an application-owned constraint when one exists. The exact Go interface and
configuration declaration site remain to be designed.
```yaml
configkit:
@@ -55,7 +57,7 @@ configkit:
destination: [prompts]
placement: sequence-item
key: [namespace, id]
append: true
mode: append
```
Each selected file would contribute a record such as:
@@ -66,26 +68,71 @@ id: welcome
text: Welcome
```
Here `append: true` illustrates retaining a lower-layer collection and adding
distinct records from the higher layer. The user also proposed `append: false`
or an inverse `replace: true` choice for complete replacement. An explicit
operation such as `mode: append` or `mode: replace` is another candidate;
field spelling and default behavior remain open.
Here `mode: append` illustrates retaining a lower-layer collection and adding
distinct records from the higher layer. A named operation is the current
recommendation for expressing all three choices; field spelling, operation
names, and default behavior remain open. The earlier `append` or inverse
`replace` boolean proposals would need additional syntax for the third operation.
The intended results can be expressed without settling the syntax. Letters
below denote record identities:
below denote record identities; `B*` is a higher-tier record with the same
identity as `B` and a different payload. The outcomes are settled; the ordering
shown is the recommendation discussed below.
| Lower-tier sequence | Higher-tier sequence | Operation | Proposed result |
| --- | --- | --- | --- |
| `[A, B]` | `[C]` | Append | `[A, B, C]` |
| `[A, B]` | `[C]` | Replace | `[C]` |
| `[A, B]` | `[B]` | Append | Duplicate-identity error |
| `[A, B]` | `[B]` | Replace | `[B]` |
| `[A, B]` | `[C]` | Whole-sequence replacement | `[C]` |
| `[A, B]` | `[B*]` | Append | Duplicate-identity error |
| `[A, B]` | `[B*]` | Whole-sequence replacement | `[B*]` |
| `[A, B]` | `[B*]` | Replacement by identity | `[A, B*]` |
| `[A, B]` | `[B*, C]` | Replacement by identity | `[A, B*, C]` |
Replacement discards the lower sequence as a whole. A repeated identity in the
Whole-sequence replacement discards the lower sequence. A repeated identity in the
retained combined sequence is an error under the proposed append semantics,
including when the records have identical or disjoint payloads.
## Settled Design Choices
### Whole-Record Replacement By Identity
The third cross-tier operation replaces each matching lower record with the
complete higher record, retains unmatched lower records, and adds new identities.
For example, given these lower and higher records:
```yaml
# Lower record
id: B
text: Original
temperature: 0.5
```
```yaml
# Higher record
id: B
text: Revised
```
The resulting record at this precedence boundary contains `id` and `text`.
The lower record's `temperature` is discarded. Subsequent defaulting or other
lifecycle operations retain their separately defined semantics. Recursive
field-by-field overlay of matching records is outside this selected operation.
### Optional Application-Owned Identity Constraints
Retain an interface through which applications may register a collection and
impose its unique-key definition. If the application requires `key: id`, a
configuration declaration of `key: [id, name]` is invalid: the compound key
would permit repeated `id` values with different names. The application identity
constraint applies whichever cross-tier operation configuration selects.
This is an authoritative constraint, distinct from an application default that
only fills an omission. Configuration can define identity for collections
without an application-owned constraint. The recommendation is for omitted
configuration identity to inherit a registered definition and for an explicit
matching declaration to be accepted. Enforcement timing, including whether the
constraint guarantees uniqueness in the final published result, remains OD-1.
## Initial Architectural Direction
Treat identity as a property of the destination collection, even if an import
@@ -101,11 +148,13 @@ contribution at that destination. It would not sequentially erase contributions
from peer imports. Multiple agreeing declarations could coalesce; conflicting
explicit identity or operation declarations would produce errors.
For append, retain lower-tier item order and then the higher tier's assembled
item order. Existing dependency-ready scheduling would continue to determine
assembly order within a tier. Replacement would retain only the higher tier's
assembled sequence. These are recommendations awaiting agreement, not changes
to the current ordering contract.
For append, the ordering recommendation is to retain lower-tier item order and
then the higher tier's assembled item order. For replacement by identity, retain
the position of each replaced record and append new identities in higher-tier
order. Existing dependency-ready scheduling would continue to determine assembly
order within a tier. Whole-sequence replacement would retain only the higher
tier's assembled sequence. Ordering and the treatment of peer declarations remain
recommendations awaiting agreement, not changes to the current ordering contract.
This requires collection policy to survive import expansion through precedence
assembly. Correctness includes final values, projected item paths, source
@@ -118,11 +167,12 @@ The implementation must account for both through the existing loading lifecycle.
This is narrower than the deferred
[identified-record sequences proposal](future.md#1-identified-record-sequences),
which also considers recursive merging of matching records and record-specific
inheritance. Those capabilities remain separate candidates. Compound keys and
duplicate-rejecting append do not require adopting them.
inheritance. Those capabilities remain separate candidates. Compound keys,
duplicate-rejecting append, and whole-record replacement by identity do not
require adopting them.
The proposal would revise the application-owned registration and cross-tier
replacement decisions in
The proposal would extend identity declaration ownership and revise the
cross-tier replacement decision in
[ADR 0014](../adr/0014-sequence-placement-and-composition.md). If adopted, record
the architectural change in a superseding decision and reconcile affected
architecture policy before implementation. The current ADR and contracts remain
@@ -136,23 +186,28 @@ design and test sequencing belong to that later plan.
## Open Decisions
### OD-1. Who Owns Identity Policy?
### OD-1. When Is An Application Identity Constraint Enforced?
Should configuration replace `Definition.SequenceCompositions`, supplement it,
or merely provide defaults for an application-owned policy?
Application authority over a registered collection's identity definition is
settled above. Does that registration govern assembly alone, or also guarantee
valid, unique identities in the final published collection? The existing
registration checks strict-composition contributions; later defaults,
inheritance, overrides, and application callbacks have separate lifecycle roles.
**Initial recommendation:** Let configuration own identity for collections
assembled through the new mechanism, with applications retaining responsibility
for their domain requirements. Resolve the treatment of existing Go registration
before removing it: applications may depend on a fixed identity definition even
when configuration authors control source layout. Avoid two independent policy
systems whose precedence depends on acquisition order.
**Initial recommendation:** Enforce an application-owned identity constraint
during participating collection assembly and validate it again on the final
effective collection before publication. This makes the application's constraint
meaningful even when a later operation creates or changes records. It does not
require applying append or replacement-by-identity semantics during those later
operations, or validating every intermediate callback state. Whether YAML-only
identity declarations also imply final validation remains to be decided.
**Alternative:** Keep application registration authoritative and allow YAML to
declare or confirm compatible policies. This preserves schema-owned identity
constraints and supports programmatic use, but adds a reconciliation rule and
retains some application setup. YAML-only ownership is simpler for independently
authored collections but moves enforcement of fixed domain identities elsewhere.
**Alternative:** Limit registration to structural assembly and leave final
uniqueness checks to application validation. This preserves the current phase
boundary and reduces built-in validation work, but the registration alone would
not establish a final-result invariant. Its public contract would need to state
that limitation clearly. Decide enforcement separately from the combination
scope in OD-7.
### OD-2. Where Does Configuration Declare Collection Policy?
@@ -179,66 +234,80 @@ relationship to that declaration.
Settle compound-key field selection, value types, missing/null components,
empty and repeated field lists, empty identity strings, and agreement between
declarations. Decide whether replacement can introduce a different identity
definition after discarding the lower collection.
declarations. Decide whether whole-sequence replacement can introduce a different identity
definition after discarding the lower collection when the application has imposed
no identity constraint. An application-owned constraint remains authoritative.
**Initial recommendation:** Interpret `key: id` as `[id]` and a compound key as
an ordered tuple of direct, literal string fields. Retain exact string comparison
and require every component to be present. Preserve tuple boundaries rather than
concatenating values. Identity declarations participating in an append must
agree; equal definitions within a scope coalesce and conflicting definitions
fail. Detailed list validation and identity changes at replacement boundaries
remain to be settled.
concatenating values. Identity declarations participating in append or replacement
by identity must agree; equal definitions within a scope coalesce and conflicting
definitions fail. Detailed list validation and identity changes at whole-sequence
replacement boundaries remain to be settled.
**Alternative:** Permit additional scalar types or nested field paths. These
support more schemas but add type-equality and addressing rules. A load-wide fixed
identity definition simplifies agreement across tiers; allowing a replacement to
establish a new definition gives independently authored layers more flexibility.
### OD-4. What Does An Import's Append Or Replace Choice Control?
### OD-4. What Does An Import's Combination Choice Control?
What happens when peer imports into the same destination request different
operations, or when some contributions omit the operation?
operations, or when some contributions omit the operation? Settle ordering for
replacement by identity, including where replaced records and new identities
appear.
**Initial recommendation:** Assemble one strict sequence per destination in
each tier, then apply its agreed operation at the precedence boundary. Reject
conflicting explicit choices within a tier. This preserves peer composition and
prevents declaration or scheduling order from granting replacement authority.
Define how omitted choices inherit the effective collection operation once its
declaration site is selected.
declaration site is selected. Duplicate identities among peers would remain
strict-composition errors under all three cross-tier operations. For replacement
by identity, keep matched records in their lower-tier positions and append new
identities in higher-tier order.
**Alternative:** Treat each import as a sequential append-or-replace operation.
**Alternative:** Treat each import as a sequential collection operation.
This permits deliberate resets among imports, but a replacement can discard peer
contributions and would require an explicitly selected operation-order model.
Another alternative is to allow the choice only on a collection declaration,
which reduces conflicts between import annotations but changes the proposed
authoring surface.
For ordering, moving replaced records to their higher-tier contribution positions
is another viable choice. It gives the higher layer more influence over ordering,
but changing a record's payload could also move it relative to retained records.
### OD-5. What Is The Default Operation And Its Syntax?
Choose between `append`, an inverse `replace` option, or a named operation, and
decide what omission means.
Choose syntax for all three operations and decide what omission means.
**Initial recommendation:** Keep replacement as the cross-tier default and
make append explicit. Lists can represent complete ordered instructions as well
as extensible catalogs; an explicit append communicates the intent to retain
lower-layer records. Prefer a positive named operation if both modes need equal
prominence, while considering the simplicity of `append: true`.
**Initial recommendation:** Keep whole-sequence replacement as the cross-tier
default and make append and replacement by identity explicit. Lists can represent
complete ordered instructions as well as extensible catalogs; an explicit append
communicates the intent to retain
lower-layer records. Prefer one named operation with three values to interacting
booleans; choose names that distinguish whole-sequence from whole-record
replacement.
**Alternative:** Default identified collections to append and require explicit
replacement. This makes progressive collections concise, but retains lower-layer
records unless an author deliberately clears them. An inverse `replace` boolean
expresses that model compactly; a named operation makes both choices explicit.
**Alternative:** Default identified collections to append or replacement by
identity and require explicit whole-sequence replacement. These defaults make
progressive collections concise, but retain lower-layer records unless an author
deliberately replaces the collection. Extending the boolean proposal with a
separate duplicate-handling option is possible, but introduces interacting
settings whose combinations need explanation and validation.
### OD-6. What Do Absence, Empty Sequences, And Null Mean?
Can a declaration selecting nothing affect a lower collection? How do explicit
empty sequences, null, and non-sequence values interact with append or replace?
empty sequences, null, and non-sequence values interact with all three operations?
**Initial recommendation:** An optional import selecting nothing contributes
nothing and does not clear lower values. An explicitly supplied empty sequence
with replacement can clear a collection; append of an empty sequence adds no
records. Preserve the distinction between absence and supplied values. Decide
with whole-sequence replacement can clear a collection; append or replacement
by identity with an empty sequence contributes no records. Preserve the
distinction between absence and supplied values. Decide
null and kind-mismatch handling explicitly rather than silently treating them
as empty sequences.
@@ -249,22 +318,28 @@ as an explicit value avoids that coupling.
### OD-7. Which Lifecycle Stages Participate?
Does append extend only source-backed tiers, or also `Definition.Defaults`,
inheritance, CLI overrides, and application-created records? At which boundaries
Do the new cross-tier operations extend only source-backed tiers, or also
`Definition.Defaults`, inheritance, CLI overrides, and application-created records?
At which boundaries
must identity be validated, including helper-supplied fields and consumed source
version headers?
**Initial recommendation:** Start with source-backed strict composition and
precedence assembly. Validate records at the relevant collection assembly
boundaries and uniqueness across each retained append result. Preserve existing
boundaries and uniqueness across each combined result. Preserve existing
typed-default and application-resolution semantics unless their inclusion is
explicitly chosen. Decide selected-overlay coverage with OD-2; it is part of
source precedence but currently has no import declaration site. Account for
item-local helpers and source-version consumption before deciding identity is
available, consistent with the current lifecycle.
**Alternative:** Apply collection policy throughout defaulting, inheritance,
overrides, and application resolution, including final identity checks. This
would provide a broader final-collection guarantee, but requires additional
combination, callback, and inspection rules. Supporting typed default lists is
a separate scope choice from extending source-backed baseline collections.
OD-1 separately addresses final validation of application constraints. Validating
the final collection does not itself change how defaults, inheritance, CLI
overrides, or callbacks combine values.
**Alternative:** Extend the combination operations through defaulting,
inheritance, overrides, and application resolution. This would support
progressively extending collections across more of the lifecycle, but requires
additional combination, callback, and inspection rules. Supporting typed default
lists is a separate scope choice from extending source-backed baseline
collections; final identity enforcement remains the separate OD-1 decision.