eric/ConfigKit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Retire the completed sequence roadmap and implementation plan

Browse Source
This commit is contained in:
Eric Rakestraw
2026-10-24 13:56:19 +00:00
parent d0a605afbb
commit 571d063675
4 changed files with 18 additions and 2073 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
docs/development.md
View File

@@ -167,6 +167,21 @@ The subsequent import-integration audit and completed remediation record are
preserved at `6903c9e:docs/roadmap/remediation.md`; retrieve them with
`git show 6903c9e:docs/roadmap/remediation.md`.
The completed sequence-identity/collection-policy roadmap and implementation
plan are preserved in Git history at `d0a605a`:
```sh
git show d0a605a:docs/roadmap/sequences.md
git show d0a605a:docs/roadmap/implementation.md
```
Current behavior belongs to the configuration contract's
[strict composition](config.md#strict-document-composition),
[sibling inheritance](config.md#sibling-inheritance), and
[CLI overrides](config.md#explicit-cli-overrides) sections.
The full [remediation record](roadmap/remediation.md) remains available, including
the open algorithm-audit findings.
Potential enhancements are tracked in [future features](roadmap/future.md).
Install the Go version required by `go.mod` or a compatible newer toolchain.

1073
docs/roadmap/implementation.md
View File

File diff suppressed because it is too large Load Diff

5
docs/roadmap/remediation.md
View File

@@ -6,8 +6,9 @@ gate.
Reviewed commit: `0be7cdb00eff576fda6b30c0600b2134d4e2e086`.
Implementation range: `bc6b977..0be7cdb`.
Authority: the [sequence roadmap](sequences.md),
[implementation plan](implementation.md),
Authority: the sequence roadmap and implementation plan preserved at
`d0a605a:docs/roadmap/sequences.md` and
`d0a605a:docs/roadmap/implementation.md`,
[ADR 0017](../adr/0017-collection-sequence-policy.md), and
[ADR 0018](../adr/0018-cli-after-structural-resolution.md).

998
docs/roadmap/sequences.md
View File

@@ -1,998 +0,0 @@
# Sequence Identity And Combination Roadmap
Status: settled feature roadmap; implementation is specified in the
[staged implementation plan](implementation.md). This document defines the
targeted end state, not an implemented contract. The implementation plan owns
execution order, exact Go names and signatures, bookkeeping, and validation.
## Purpose And Scope
Allow configuration authors to assemble collections from independently authored
records, declare identity in configuration, and progressively extend a baseline
collection through higher layers. Authors can replace the entire collection,
append distinct records, or replace complete records by identity while retaining
unmatched records. Sibling inheritance uses the same sequence operations and
identity-based defaults. Within an identified sequence, a record can explicitly
inherit settings from another record in that sequence while keeping its own
identity.
The motivating use case is a prompt catalog whose source-backed baseline supplies
records and whose application or installation layers add or supersede records.
Applications can optionally impose identity constraints that configuration must
respect and that hold in the final loaded configuration. These constraints can
cover exact destinations or collections beneath dynamically named mapping keys,
without requiring the application to discover those names before loading.
The feature covers ordinary and layered loading, imported items, inline
sequences, selected overlays, and sibling inheritance. Typed defaults, CLI
assignments, and application resolution retain their own combination semantics;
CLI application moves later in the loading lifecycle.
Automatic recursive merging of matching records remains deferred
(see [future work](future.md#1-recursive-body-merging-for-identified-records));
explicit inheritance between distinct records is included. The initial
declaration surface is `configkit.sequences`; `key` and `mode` shorthand on
imports is deferred. Application lists remain
ordinary YAML sequences without control wrappers. Unkeyed sequence concatenation
is outside this feature.
## Current Baseline And Required Changes
The [import contract](../config.md#explicit-imports) owns existing selectors,
destinations, placement, and ordering. `placement: sequence-item` already places
each selected source mapping as one item at its effective destination. Item-local
helper imports can contribute fields before placement.
The [composition contract](../config.md#strict-document-composition),
[layered-loading contract](../config.md#layered-loading), and
[inheritance contract](../config.md#sibling-inheritance) describe the implemented
baseline:
| Area | Current behavior | Target behavior |
| --- | --- | --- |
| Identity declaration | `Definition.SequenceCompositions` registers one literal string field at each registered mapping path. | YAML collection declarations and optional authoritative application registrations support field sets and scalar identity components. |
| Application registration destinations | Registrations select exact literal mapping paths. | Exact and structural selectors can impose identity beneath arbitrary mapping keys, independently at each matched destination. |
| Strict composition | Registered sequences append distinct string identities and reject duplicates; repeated unregistered sequences conflict. | Collections with effective identity append distinct records and reject duplicates. Applicable lower-tier identity is available before composing higher-tier peers. Unidentified sequences retain strict-conflict behavior. |
| Source precedence and selected overlays | A present higher sequence replaces the lower sequence wholesale. | Collection policy selects `replace`, `append`, or `upsert`; overlays can declare policy for inline sequences. |
| Sibling inheritance | An inheriting mapping recursively overlays its resolved parent; supplied sequences replace wholesale. | Mappings retain recursive combination, while sequences use destination policy and the same three operations. |
| Sequence-member inheritance | A sequence element has no sibling scope for its own inheritance relationship. | An identified sequence provides a sibling scope: a record can inherit from one record in the same sequence, selected by complete identity. |
| CLI overrides | Apply before inheritance and typed defaults/decoding. | Apply after inheritance and typed defaults/decoding, before application resolution. |
| Identity enforcement | Registration governs strict sequence composition. | Structural identity checks cover identified combinations; application registrations additionally constrain final effective sequences. |
Source-backed underlays participate in source assembly. `Definition.Defaults`
provides typed defaults later; extending source combination does not turn typed
default lists into source underlays.
## Collection Declarations
Use a source-root `configkit.sequences` block containing collection declarations.
Each entry identifies a `destination` and may declare `key`, `mode`, or both.
Identity and mode can be supplied independently. This form serves imported items,
inline records, ordinary mapping imports, and inline sequences in selected
overlays.
```yaml
configkit:
sequences:
- destination: [prompts]
key: [namespace, id]
mode: append
imports:
- dir: prompts
pattern: '*.yaml'
destination: [prompts]
placement: sequence-item
```
Each selected file contributes a record such as:
```yaml
namespace: support
id: welcome
text: Welcome
```
Here append retains a lower-layer collection and adds distinct higher-layer
records. Identity and combination controls belong to the collection declaration,
so imported and inline contributions share one policy surface.
### Destinations And Scope
`destination` uses the existing sequence of literal mapping keys. Resolve it
relative to the declaring source's placement: a fragment mounted under `catalog`
can declare `[prompts]` locally. A sequence-item source can declare a nested
collection within its own item without referring to an eventual numeric index.
Preserve source and item-local scope boundaries.
A collection's scope is its resolved destination within its composition scope.
Resolve relative destinations before reconciling declarations. Different imported
items retain separate scopes for their nested collections. A declaration governs
all contributions to its collection in the tier, including imported and inline
records; it does not govern only records in the declaring file.
Gather declarations before composing the relevant scope so their effect does not
depend on which contribution is assembled first. Retain applicable controls
through source precedence and inheritance, then keep them out of application
decoding.
Sequence declarations are independent of filesystem import enablement. Selected
overlays may contain `configkit.sequences`, extending the current blanket
prohibition on overlay `configkit` declarations. This does not enable imports
in overlays.
### Overlay Authoring
For an existing sequence `[foo, bar]`, an overlay can replace `foo` in full:
```yaml
configkit:
sequences:
- destination: [prompts]
key: id
mode: upsert
prompts:
- id: foo
text: Revised definition
```
The result is `[foo*, bar]`. The explicit upsert mode can be omitted when no
different explicit mode applies from a lower tier: effective identity then
selects upsert. To add a distinct record instead:
```yaml
configkit:
sequences:
- destination: [prompts]
key: id
mode: append
prompts:
- id: baz
text: Another prompt
```
The result is `[foo, bar, baz]`. To replace the whole sequence:
```yaml
configkit:
sequences:
- destination: [prompts]
mode: replace
prompts:
- id: bar
text: Bar definition
- id: baz
text: Baz definition
```
The result is `[bar, baz]`. Explicitly selecting the default operation is valid.
With no applicable explicit mode, omitting this declaration selects replace for
an unidentified collection and upsert for an identified one. A retained explicit
mode continues to govern.
## Identity Definitions And Constraints
### Field Sets
`key` accepts a string or a nonempty list of strings naming direct, literal
fields in each record. A name such as `metadata.id` denotes that literal field,
not a nested lookup. Treat field names as a set: remove repeated names and
ignore authored order when comparing definitions.
Thus `id`, `[id]`, and `[id, id]` are equivalent; `[namespace, id]`,
`[id, namespace]`, and `[namespace, id, id]` are also equivalent. An empty list
is invalid because it supplies no identity fields. These rules apply to both
configuration declarations and application registrations.
Use a consistent field order internally to construct identity tuples. Preserve
component types and tuple boundaries so unstructured concatenation cannot cause
otherwise distinct identities to collide.
### Scalar Components And Equality
Each identified record must be a mapping with every designated identity field
present. Components may be strings, numbers, booleans, or explicit null. Missing
fields and mapping or sequence components are invalid. Present null differs
from absence; two single-field null identities are equal. Compound identities
are equal when all corresponding components are equal.
Strings compare exactly, including empty strings, without trimming, case folding,
or normalization. Scalar categories remain distinct: `"1"`, `1`, `true`, and
null are different identities.
Numbers compare by mathematical value across integer, fractional, and exponent
spellings: `1`, `1.0`, and `1e0` are equal, as are `0` and `-0`. Use exact numeric
normalization for identity comparison without rounding through a floating-point
conversion. This does not rewrite authored values or change existing typed
decoding and projection semantics. The parser's rejection of non-finite numeric
forms remains unchanged.
### Application-Owned Constraints
Retain an optional application interface for registering collections and imposing
their identity definitions. Configuration may define identity where no application
constraint exists. When a declaration omits `key`, an applicable registration
supplies identity. An explicit equivalent definition is accepted, and a different
definition fails under every mode, including replace.
For example, if the application requires `id`, `key: [id, name]` is invalid:
that compound identity would allow repeated `id` values with different names.
A registration is an authoritative constraint, not an omission-filling default.
Copying a collection through inheritance does not create another application
registration; registrations constrain their registered destinations.
Identity constraints validate actual sequences. They do not independently require
a path to exist, be non-null, or have sequence shape. The application schema and
validation own those requirements. Final-result enforcement for application
registrations is defined under [Validation And Diagnostics](#validation-and-diagnostics).
### Application Registration API
Use one public sequence-identity registration type, collected in one field on
`Definition`. Each registration contains an explicit destination selector and
its identity fields. Exact and dynamic registrations share matching, overlap
validation, identity comparison, and lifecycle behavior.
Provide two constructors returning that same registration type:
- An exact-path convenience constructor accepting a literal `Path` and identity
field names. It converts the path into a selector containing literal segments.
- An explicit-selector constructor accepting a structural selector and identity
field names, for destinations that include arbitrary mapping keys.
Both constructors accept identity fields as variadic strings, so one-field and
compound identities use the same construction form. Apply the settled field-set
rules, including deduplication. Invalid registrations, including an empty identity
field set, fail during application-definition validation.
Existing `Path` values retain their literal meaning throughout ConfigKit. The
exact-path constructor supplies convenience at the registration boundary; dynamic
matching uses the explicit selector type. Both forms can appear together in the
same definition field.
Replace the current registration shape with this model. Preserving the existing
`SequenceComposition` fields and `Definition.SequenceCompositions` spelling is not
a requirement, and no compatibility facade is needed solely to retain that syntax.
Choose names centered on sequence identity, reflecting its role in composition,
inheritance, and final validation. Exact Go type, field, and constructor names
and signatures belong in the implementation plan.
### Dynamic Application Registration Destinations
Allow applications to register identity for collections whose containing mapping
keys become known only during loading. For example:
```yaml
collections:
weatherreporter:
prompts:
- id: foo.bar
text: Report the weather.
homewatch:
prompts:
- id: foo.bar
text: Report the household status.
```
One application registration can require `id` for every namespace's `prompts`
list. The equal IDs above are valid because each list has its own uniqueness
scope. An optional root `prompts` list can have a separate exact registration.
Use an explicit application-side structural selector composed of literal mapping
keys and an arbitrary-mapping-key segment. Conceptually, the namespace rule is:
```text
Literal("collections") -> AnyMappingKey -> Literal("prompts")
```
This notation explains the selector; it is not a Go signature or YAML syntax.
Each segment consumes exactly one mapping level. Match the complete destination,
not an arbitrary prefix or suffix. Literal keys remain exact, including empty
strings and keys containing punctuation such as `*`, `/`, or `.`. An arbitrary-key
segment is distinct from any literal spelling and does not match a sequence
index. Existing `Path` values retain their literal meaning.
This capability belongs to the reusable application definition and applies to
ordinary and layered loads. YAML collection destinations and import destinations
retain their existing literal syntax. Recursive search, regular expressions,
filesystem glob rules, sequence-index selectors, and custom matching callbacks
are outside this addition.
At each concrete matched destination, apply the same authoritative identity
constraint as an exact registration. Use the settled field-set and scalar-equality
rules. Do not pool identities across destinations or require records to repeat
the containing mapping key as an identity component. Applications retain ownership
of namespace interpretation, qualified runtime IDs, and domain validation.
Matching supplies effective identity before relevant strict composition and
precedence operations. Without an applicable explicit mode, this identity selects
upsert under the existing default rule. It also supplies identity for inheritance
within the matched sequence. YAML declarations must agree with it under every
mode, including replace. These are ordinary application constraints with a broader
destination selector, not a separate final-validation facility.
### Registration Overlap
Equivalent application constraints coalesce at any shared destination. Equivalence
uses the identity field-set rules: for example, `id` and `[id, id]` agree.
Incompatible constraints fail; exact registrations do not override broader
selectors, and registration order grants no precedence.
Detect incompatible overlapping registrations while validating the application
definition, before source acquisition. For these fixed-depth mapping selectors,
overlap exists when the selectors have the same number of segments and every
corresponding pair can match a common key. Unequal literal keys at any position
make the selectors disjoint; an arbitrary-key segment can overlap a literal or
another arbitrary-key segment. Compare segment structure rather than rendered
path strings. No configuration pre-read is needed.
Thus an `id` rule for all namespace prompt lists conflicts with a `[id, name]`
rule for one named namespace, even if that namespace is absent from a particular
load. This is an invalid application definition. A valid selector matching no
collections is harmless: registration does not require a path to exist, create
collections, or impose shape and nullability requirements.
### Matching Across The Loading Lifecycle
Match application selectors against effective configuration destinations after
accounting for input placement. Physical filenames and source-local paths do not
determine applicability. Make constraints available before composing contributions
at those destinations, including namespaces first introduced by imports, mounted
inputs, or higher tiers. Apply matching constraints to key-only YAML declarations
as well, even before their sequence exists. Consumers need neither a preliminary
load nor source inspection to enumerate namespaces.
During inheritance, apply registrations at the inheriting destination, including
newly created matching destinations. Copying a collection does not transfer its
application registrations to a different path. The existing YAML-policy ownership
and transfer rules remain separate from application selector matching.
During final validation, find matching actual sequences in the final effective
configuration. This includes namespaces introduced or replaced by typed defaults,
CLI overrides, and application resolution. Reusing only destinations found during
source assembly would miss these values. Use the same selector meaning and
identity validator in both phases; later assignments do not trigger composition
or inheritance again. YAML-only identities retain their existing enforcement
boundary.
## Strict Composition And Precedence Operations
### Compose Peers Before Applying Precedence
Resolve applicable identity and mode policy, strictly compose the tier's peer
contributions, then apply one selected operation per collection against the
accumulated lower value. The tier operation does not sequentially replace peer
source contributions. Selected overlay contributions use the same combination
semantics.
Within a strict tier, equivalent identity definitions coalesce and different
field sets fail. Agreeing explicit modes coalesce; conflicting explicit modes
fail. Omission expresses no preference. A root declaration or a later declaration
has no precedence over conflicting peers. Explicitly selecting a default mode
is still a real instruction and can conflict with another explicit mode.
Identified peer sequences append distinct records. Duplicate identities among
peers or within a contributed list fail under all three modes, including replace
and upsert. Equal or disjoint record payloads do not excuse a duplicate identity.
Unidentified sequences retain their current strict-composition conflict behavior.
For example, higher-tier fragments `[C]` and `[D]` with an agreed identity assemble
as `[C, D]`. Against lower `[A, B]`, append or upsert produces `[A, B, C, D]`,
while replace produces `[C, D]`. Peers `[B]` and `[B*]` conflict even under
upsert because neither has higher precedence.
### Available Operations
Use one spelling for each mode: `replace`, `append`, and `upsert`.
| Mode | Behavior at source-precedence and inheritance boundaries |
| --- | --- |
| `replace` | Replace the complete lower sequence with the assembled higher sequence. |
| `append` | Retain lower records and append higher records; duplicate identities fail. |
| `upsert` | Replace matching records in full, retain unmatched lower records, and append new identities. |
Append and upsert require effective identity. ConfigKit does not infer identity
from an `id` field or record position. Whole-sequence replacement does not require
identity agreement with lower YAML policy, but application constraints still apply.
Letters denote record identities; `B*` has the same identity as `B` with a different
payload:
| Lower | Higher | Mode | Result |
| --- | --- | --- | --- |
| `[A, B]` | `[C]` | `append` | `[A, B, C]` |
| `[A, B]` | `[C]` | `replace` | `[C]` |
| `[A, B]` | `[B*]` | `append` | Duplicate-identity error |
| `[A, B]` | `[B*]` | `replace` | `[B*]` |
| `[A, B]` | `[B*]` | `upsert` | `[A, B*]` |
| `[A, B]` | `[B*, C]` | `upsert` | `[A, B*, C]` |
Upsert replaces the whole matched record. If lower `B` has `id`, `text`, and
`temperature`, while higher `B` has only `id` and `text`, the result at this
boundary has only `id` and `text`. Subsequent defaulting or other lifecycle
operations retain their own semantics. Mappings continue to combine recursively;
matching sequence records do not recursively merge fields.
### Mode Selection And Persistence
Select the operation in this order:
1. Reconcile explicit modes within the current tier; conflicting peers fail.
2. Use the current tier's explicit mode if supplied.
3. Otherwise retain an applicable explicit mode from a lower tier.
4. With no explicit mode, infer upsert when effective identity is available.
5. Otherwise infer replace.
Effective identity can come from an application registration, current declarations,
or applicable lower policy, subject to the identity agreement and lifetime rules
below. Determine the inferred mode from available identity before applying the
operation. A plain higher-tier sequence can therefore use upsert without repeating
a key declaration.
An explicit mode persists at its applicable destination until a higher-priority
explicit declaration changes it. New sequence values alone do not reset it, and
an unrelated overlay leaves it intact. A mode-only declaration can change policy
without supplying records; it neither clears the collection nor retroactively
recombines earlier tiers. Later source contributions and inheritance use the
resulting policy.
| Lower explicit mode | Higher declaration | Effective mode |
| --- | --- | --- |
| `append` | Mode omitted | `append` |
| `append` | `upsert` | `upsert` |
| `upsert` | `replace` | `replace` |
| None | Mode omitted, effective identity available | Inferred `upsert` |
| None | Mode omitted, no effective identity | Inferred `replace` |
Only explicit modes persist. Inferred defaults are recomputed from effective
identity when needed. Explicit replace continues to apply in subsequent tiers
until another mode is declared; introducing identity alone does not override it.
Mode persistence does not preserve YAML identity discarded by replacement, nor
policies owned by discarded records. Source precedence and inheritance use the
same destination policy, without separate per-phase settings. CLI assignments
remain outside mode selection.
### Presence, Empty Sequences, Null, And Kind Mismatches
At a collection's precedence boundary:
| Higher contribution | `replace` | `append` / `upsert` |
| --- | --- | --- |
| Absent | Leave the lower value unchanged. | Leave the lower value unchanged. |
| Explicit `[]` | Clear the sequence. | Contribute no records. |
| Explicit null or another non-sequence value | Ordinary replacement, subject to application schema. | Error: the operation requires a sequence. |
A declaration alone does not clear a collection. An optional import selecting
nothing contributes nothing and does not erase existing records. An absent lower
collection can be initialized by an incoming sequence under any mode, subject to
identity requirements. Explicit lower null or another non-sequence value is not
absence: append and upsert fail against it, while replacement remains available.
An empty incoming sequence bypasses neither kind checks nor identity requirements.
With effective identity and no explicit mode, `prompts: []` uses upsert and does
not clear the collection. Clearing requires replace, selected in the current tier
or retained from lower policy. Replacing an identified collection with null also
requires replace and a schema permitting null. Collection null is distinct from
a present null identity component inside a record.
### Ordering
Retain existing dependency-ready source scheduling within each tier and authored
item order within lists. Gathering declarations does not change assembly order;
identity values do not sort records.
- Append keeps lower records in order, followed by higher records in assembled order.
- Replace keeps only higher records in their assembled order.
- Upsert keeps a matched record's lower position and appends new identities in
higher-tier order.
For example, lower `[A, B, C]` and higher `[C*, A*, D]` produce `[A*, B, C*, D]`
under upsert. Whole-sequence replacement gives the higher tier control of the
complete sequence order.
## Policy Agreement, Lifetime, And Ownership
### Identity Agreement And Replacement
Append and upsert use one agreed identity for all participating records:
| Lower identity | Higher declaration | Result when combining records |
| --- | --- | --- |
| `id` | Omitted | Use `id`. |
| `id` | `id`, `[id]`, or `[id, id]` | Accept the equivalent definition. |
| `id` | `[namespace, id]` | Error: different definitions. |
| None | `id` | Establish `id` and validate all participating records. |
Introducing identity for an unkeyed lower sequence requires checking both lower
and incoming records, including uniqueness within each contribution. With no
identity from the application or either participating collection, append and
upsert fail.
YAML identity follows the collection that survives an operation:
| Operation | Effective YAML identity |
| --- | --- |
| Append or upsert | Retain or establish the agreed identity. |
| Whole-sequence replacement | Discard lower YAML identity and use the replacing tier's identity declaration, if any. |
| Replacement by null or another non-sequence value | Discard lower YAML identity and retain any identity explicitly declared by the replacing tier. |
Application constraints remain authoritative under every operation. Without an
application constraint, replacement can introduce a different identity, such as
changing from `id` to `name`. Replacing a sequence without a new key declaration
or application registration produces an unkeyed sequence. No identity-removal
directive is needed.
The reset occurs only when the collection value is actually replaced. Declaring
replace without supplying a replacement does not authorize changing the identity
of retained records. Explicit mode lifetime is separate from YAML identity
lifetime.
### Ancestor Replacement
Replacing an enclosing mapping can remove a collection without replacing that
collection directly. Apply the same identity and ownership rules to this removal:
discard the removed collection's lower YAML identity, and discard policies owned
by any removed records. Reintroducing the same path later does not restore those
discarded policies.
Independently declared explicit modes retain their existing persistence rules.
Removing a descendant collection does not remove a mode whose declaring scope
survives. Application registrations remain applicable at matching destinations,
including destinations reintroduced later. Identity newly declared by the
replacing tier follows the existing declaration and ownership rules.
For example, a root-level declaration establishes `key: id` and `mode: append`
for `[catalog, prompts]`. A higher tier replaces `catalog` with null, removing
the prompts collection and its YAML identity. The independently declared append
mode survives. If a later tier recreates `catalog.prompts`, append requires
identity supplied by a new declaration or an application registration. Without
that identity, loading fails; the later tier can instead explicitly choose
replace. The replacement of `catalog` itself follows ordinary ancestor-value
precedence and does not attempt to append null to the removed prompts list.
### Declarations Without Sequence Values
A key declaration without accompanying records applies immediately to an existing
retained sequence: check agreement and validate its records, without recombining
earlier tiers. For example, a higher tier that only declares
`destination: [prompts], key: id` identifies and validates the retained lower
`prompts` sequence.
If no collection exists, retain the declaration in its owning scope for later
contributions. An otherwise empty underlay can establish identity for main-tier
fragments. An explicit identity declaration can also remain pending at a
non-sequence destination; it does not independently impose a shape requirement.
The presence and kind rules still determine whether an attempted operation fails.
Item-local declarations retain their ownership boundaries.
### Lower-Tier Identity During Strict Composition
Resolve applicable identity before composing a higher tier's peers, using
application registrations, available lower-tier policy, and declarations gathered
within the higher tier's scope. Authors need not repeat a baseline key solely
because the higher tier has multiple fragments.
For example, an underlay establishes `key: id` for `prompts`, and two main-tier
fragments contribute `[foo]` and `[bar]` without a key declaration. They strictly
compose as `[foo, bar]` before the main tier combines with the accumulated lower
collection. If both contribute `foo`, composition fails; upsert does not give
precedence to peers.
Lower YAML identity remains subject to agreement, replacement, and ownership.
A replacing collection must not be constrained by obsolete lower identity just
because policy resolution precedes composition. Likewise, a replacement record
cannot borrow nested identity owned by the discarded lower record, even when
outer identity or numeric position matches. Application constraints still apply.
Higher-tier declarations do not retroactively repair invalid lower-tier composition.
### Nested Record Ownership
Attach item-local policies to their owning contributions so they follow surviving
records independently of numeric indexes. Whole-record upsert discards the old
record's local policies along with its fields. Whole-sequence replacement discards
policies owned by removed records. A replacement record supplies its own local
declarations; equal outer identity or final index does not transfer discarded
policies.
For example, a source imported as one record into `prompts` can declare:
```yaml
configkit:
sequences:
- destination: [steps]
key: id
mode: append
id: foo
steps:
- id: prepare
```
The declaration governs `steps` inside that `foo` record. Retaining `foo` retains
its local policy, including the explicit mode. Upserting a new `foo` discards the
old `steps` and its declaration even though the outer record keeps its position.
Policies independently applicable at a destination are not discarded merely
because an individual record is replaced. Ownership determines which declarations
are removed. Application registrations remain constraints at their registered
destinations. Inheritance copies transfer identity and choose operations as
specified next.
## Sibling Inheritance
Complete source-tier assembly before resolving inheritance. Retain existing
mapping-sibling scopes and add identified-sequence scopes as described below.
Both use the existing dependency-ready algorithm. At each inheritance combination,
the resolved parent's collection is the lower value and the inheriting mapping's
local collection is the higher value. Mappings combine recursively; upsert within
an identified sequence remains whole-record replacement. Explicit record
inheritance is a separate construction step after those replacements.
Use the inheriting destination collection's policy: an applicable explicit mode,
including one retained through source tiers, wins. Otherwise infer upsert with
effective identity and replace without it. There is no separate default-append
rule for inheritance. The same destination policy governs source-precedence and
inheritance combinations.
YAML identity follows a collection copied through inheritance, subject to agreement
and replacement rules. A source collection's mode does not automatically become an
instruction at a different inheriting destination. For example, source `steps`
has `key: id` and `mode: append`; an inheriting mapping supplies local `steps`
without an applicable explicit mode. The copied identity selects upsert there.
An explicit destination append instead selects append. A matching destination key
agrees; a different key conflicts when combining records, while explicit replace
can establish new YAML identity subject to application constraints.
For example:
```yaml
configkit:
sequences:
- destination: [parent, items]
key: id
child:
items:
- id: foo
value: A
- id: bar
value: B
parent:
extends: child
items:
- id: baz
value: C
```
Despite the names, `parent` inherits from `child`. Identity at `[parent, items]`
selects upsert, producing `[foo, bar, baz]`. If the local sequence instead supplies
a revised `foo`, the result is `[foo*, bar]`. Append rejects that repeated identity;
replace retains only the local sequence.
The same ordering, presence, and kind rules apply at inheritance boundaries.
An omitted local sequence retains the inherited sequence. Explicit `[]` clears it
under replace and contributes no records under append or upsert. Retain controls
until inheritance uses them, then exclude them from application decoding.
### Inheritance Between Identified Sequence Records
An identified sequence supplies a sibling scope for its records. Identity can
come from an application registration or an applicable YAML declaration. A record
may inherit from one other record in that same sequence using an `extends`
mapping that selects the parent's complete identity:
```yaml
configkit:
sequences:
- destination: [prompts]
key: id
prompts:
- id: base
model: standard
temperature: 0.5
- id: concise
extends: {id: base}
temperature: 0.2
```
The final `concise` record is:
```yaml
id: concise
model: standard
temperature: 0.2
```
The `base` record remains in the collection. Parents are ordinary records subject
to the same decoding and validation as other records; there is no abstract
template category.
Use mapping-form references for both single and compound keys. With
`key: [namespace, id]`, a reference can be
`extends: {namespace: shared, id: base}`. Match components using the collection's
field-set and scalar-equality rules, including mathematical numeric equality.
`extends: {id: null}` selects a null-valued identity; `extends: null` cancels a
relationship.
Parent-reference mappings use ordinary strict mapping composition. Item-local
helper imports may supply disjoint fields of a compound reference; overlapping
fields conflict, even when equal. For example, a helper's
`extends: {namespace: shared}` and the importing record's `extends: {id: base}`
compose into `extends: {namespace: shared, id: base}`. This assembles one record;
it does not permit strict peers to contribute duplicate record identities.
Before parent lookup, require the surviving effective reference mapping to contain
exactly the collection's identity fields, with supported scalar values. Missing
or extra fields fail. Validate the completed reference during inheritance, rather
than requiring each helper contribution to be independently complete. Superseded
references remain historical evidence; earlier strict-composition conflicts
remain errors.
Every record must supply its complete own identity before inheritance. Preserve
the child's identity fields and inherit only the parent's nonidentity payload;
inheritance cannot supply a missing child identity. Child payload fields take
precedence, mappings combine recursively, and nested sequences use the inheriting
destination's collection policy. Consume the record's directive before decoding.
Support one parent within the same identified sequence, including forward
references and chains. Missing parents and cycles, including self-reference,
fail the load. Cross-collection references, position-based references, multiple
parents, and inheritance between unidentified sequence members are outside this
scope. Existing string-form references between mapping siblings retain their
meaning; they do not become shorthand for sequence identity references.
Resolve relationships against records surviving source-tier assembly and selected
overlays. A higher layer can replace a parent before inheritance runs; removing
that parent makes a surviving reference fail. Whole-record upsert discards the
old record's directive along with its payload. Omitting `extends` in the
replacement record does not retain the displaced record's relationship. An
explicit reference to the record's own identity is a cycle, not a way to access
its lower-tier predecessor.
Reuse Kahn's algorithm with the existing stable canonical semantic-pointer key
for dependency-ready ordering. Build parent lookup from the sequence's effective
identity, and preserve the assembled list order independently of resolution
order. Enclosing relationships resolve before nested relationships. Parent
templates retain nested instructions for evaluation at each inheriting
destination, following the existing destination-local inheritance rule. Copied
collection identity, destination operation selection, and record-owned policy
transfer follow the rules above.
Record inheritance runs in the existing structural phase, before typed defaults,
decoding, CLI overrides, and application resolution. Those later stages do not
trigger another inheritance pass. Preserve inherited presence, origin and
history evidence, sensitivity, declaring resource contexts, and existing
expansion budgets just as for mapping-sibling inheritance. Identity lookup must
not expose identity values in diagnostics. Existing index-based inspection paths
remain sufficient; no new identity-selector path API is needed.
## Loading Lifecycle
Use the following order:
1. Assemble source tiers and selected overlays, resolving policy before each
tier's strict composition and applying collection operations at precedence
boundaries.
2. Resolve sibling inheritance in mappings and identified sequences, then apply
typed defaults and decode the assembled configuration.
3. Apply explicit CLI overrides to that configuration and check supplied values
against the application schema.
4. Run application resolution against the overridden configuration.
5. Resolve effective secret references, validate the final configuration including
application-registered identities, and publish it.
### Typed Defaults And Application Resolution
Source-backed underlays, including application-supplied mapping sources,
participate in sequence combination. Typed `Definition.Defaults` remains the
initial decoded state: supplied sequences replace default sequences. Source
append does not append to a typed default list.
Application resolution retains assignment and recorded-transformation semantics.
ConfigKit does not automatically combine a callback's assigned sequence with its
previous value. Application-registered identities receive final validation
independently of how defaults, CLI overrides, or callbacks produce their values.
### CLI Precedence
CLI overrides are the highest-precedence input to application resolution.
Supplied sequences replace whole sequences, including sequences inside supplied
mappings. Mappings retain recursive override behavior; empty sequences and
permitted nulls retain explicit replacement meaning. Configuration append or
upsert does not reinterpret CLI assignments. Inheritance and defaulting do not
run again afterward: overriding assembled `[A, B]` with `[C]` presents `[C]` to
application resolution without inheritance reintroducing prior records.
Application resolution sees CLI values and their explicit-presence and source
metadata. `Resolution.Default` preserves explicit CLI values; normalization and
recorded application transformations may still change them. For example, a derived
URL can use a CLI hostname, while an explicit CLI URL can be preserved by an
absence-based default. Retain recording and provenance requirements for those
transformations.
Reject structural directives such as `extends` and `configkit` in CLI override
paths or supplied values. CLI input supplies application values after structural
processing; it cannot supply or cancel inheritance directives, change completed
relationships, or prevent earlier inheritance failures.
Typed decoding also precedes CLI application, so a CLI replacement cannot repair
an earlier decoding failure. A source string in an integer field fails before a
valid CLI integer can apply. This changes current override-before-decoding
behavior. Semantic validation remains after overrides and application resolution.
Secret lookup uses effective references after overrides and resolution. Neither
stage bypasses final application identity constraints or semantic validation.
## Validation And Diagnostics
### Structural Identity Checks
Allow item-local helper imports to complete and consume applicable non-main
source-version headers before checking payload identity for collection assembly.
A helper can supply an identity component; a consumed header cannot supply one
after removal.
Structural controls are likewise unavailable as source identity components.
Check identity against application payload fields, excluding fields acting as
structural directives even when the engine retains them for later processing.
For example, `key: extends` with an item containing `extends: null` does not
supply a null identity: that field cancels inheritance and is consumed. Report
the ordinary missing-identity-component error at the collection/item path using
available origins, without exposing values. Do not defer this failure until
directive consumption or final application validation.
This rule follows a field's structural role; it does not introduce a broader
restriction on unrelated literal field names. Final application identity checks
continue to examine the effective typed payload under their existing rules.
Validate each identified contribution before combination and check uniqueness in
the resulting identified collection, including inheritance operations. Upsert
matches across precedence are replacements; duplicates within a contribution
remain errors. Later inheritance, typed defaults, or callbacks cannot repair an
identity missing at an earlier assembly boundary.
YAML-only identities are enforced during structural composition, precedence
assembly, and identified inheritance operations. They have no final-result
guarantee after typed defaulting, CLI overrides, or application resolution. Those
stages can produce duplicate identities or otherwise violate a YAML-only identity.
Applications needing a final guarantee register the constraint through the
application-owned interface.
Adding record inheritance does not expand this guarantee: a YAML declaration
can enable parent lookup without imposing final validation on values later
produced by defaults, CLI overrides, or application resolution.
### Final Application Constraints
On successful loading, final effective collections covered by application
registrations satisfy their declared identity constraints. Retain assembly-time
checks and validate final effective sequences before publication, after defaults,
inheritance, overrides, application resolution, and secret lookup.
This covers both exact and dynamic application registrations. Evaluate dynamic
selectors against the final effective configuration, with independent validation
for each concrete matched sequence as described under
[Matching Across The Loading Lifecycle](#matching-across-the-loading-lifecycle).
ConfigKit performs these identity checks; consumers register the constraint
without needing to repeat duplicate detection in `Definition.Validate`.
Application-owned semantic validation continues to cover domain-specific rules.
This guarantee covers actual sequences, not path existence or allowed shape.
It does not require every intermediate callback state to satisfy the constraint
and does not cover caller mutations after loading returns. Final checks examine
effective typed values; assembly checks validate authored contributions. A final
scalar alone does not prove that a source explicitly supplied that field.
Use the existing canonical final-value representation and shared publication
path as the foundation. The guarantee requires neither a new mutation-history
system nor exact causal reconstruction for records supplied by defaults or
callbacks.
### Diagnostics And Inspection
Identity diagnostics identify collection and item paths using available origin
information without exposing identity values. Conflicting declarations identify
the collection destination and declaration locations.
For dynamic registrations, use concrete matched collection and item paths for
value diagnostics, with available origins and existing sensitivity rules.
Definition-time overlap errors identify the conflicting registrations; they do
not require or invent a concrete destination from a loaded document.
Correctness includes final values, projected item paths, source destinations,
provenance, history, sensitivity, and passive resource contexts. Appending can
change final indexes; replacing changes which contributions survive. Preserve
the existing loading and inspection contracts through both operations, including
policy ownership and inheritance transfer.
## Work Needed From The Baseline
The baseline needs the following connected capabilities to reach the target
state. Execution order and test sequencing belong to the
[implementation plan](implementation.md):
- **Declarations and public API:** extend application registration to the settled
identity model with one public registration type and definition field, an
exact-path convenience constructor, and an explicit-selector constructor.
Preserve literal `Path` semantics. Parse and consume `configkit.sequences` across
supported source forms and selected overlays; preserve placement-relative and
item-local scopes. Keep sequence declarations independent of filesystem import
enablement.
- **Identity and policy representation:** implement field-set comparison, typed
scalar identity tuples with exact numeric equality, declaration agreement,
pending declarations, explicit-mode persistence, replacement lifetime, and
record-owned policy transfer. Choose bookkeeping proportional to these semantics.
Handle descendant collection removal through ancestor replacement, preserving
independently declared modes and application constraints while discarding
removed collection identity and record-owned policies.
- **Registration matching:** support literal and arbitrary-mapping-key segments,
detect incompatible selector overlap during definition validation, and resolve
applicable constraints at concrete placed destinations. Reuse the identity
rules and validator for exact and dynamic registrations. Keep matching separate
from source discovery and from YAML declaration ownership.
- **Assembly handoffs:** make accumulated lower policy available before higher-tier
strict composition. Separate source acquisition from policy-aware composition
or otherwise revise assembly sequencing; current main-before-underlay composition
is not a design constraint. Preserve distinctions among acquisition order,
dependency-ready scheduling within tiers, and precedence order. Account for
source records, callbacks, main-version checks, and failure diagnostics. Retain
version ownership and validation semantics; higher replacement cannot hide
lower-tier structural failures.
- **Combination and inheritance:** implement the three operations with strict peer
handling, whole-record replacement, presence rules, stable ordering, and the
destination policy rules for inheritance. Extend sibling-scope discovery and
parent lookup to identified sequences, validate mapping-form references, and
preserve child identities while copying parent payloads. Reuse the shared
dependency-ready scheduler and destination-local nested resolution. Preserve
provenance, history, sensitivity, paths, and resource contexts as records survive,
move, or disappear.
- **Lifecycle and validation:** move CLI application after inheritance and typed
defaults/decoding, preserve explicit-presence and source metadata for application
resolution, reject CLI structural directives, and add final application identity
validation through the shared publication path. Keep YAML-only enforcement
within structural operations. Match dynamic registrations during assembly and
inheritance, and against the final effective configuration rather than a saved
list of source-backed destinations.
Exclude structural controls from source identity checks even while retaining
those controls for later inheritance processing.
- **Documentation and verification:** exercise ordinary and layered loading,
imported and inline collections, item-local scopes, version-header consumption,
selected overlays, inheritance, typed defaults, CLI, callbacks, and inspection.
Cover single and compound parent references, scalar equality, forward references
and chains, missing parents and cycles, parent replacement/removal, child identity
preservation, nested destination policy, cancellation, and stable output order.
Verify helper-composed references, overlapping reference fields, missing or extra
fields, and replacement of references together with their containing records.
Verify independent namespace uniqueness, root and namespaced registrations,
literal punctuation keys, equivalent and incompatible overlaps, and conflicts
detected even without a matching input destination. Cover namespaces introduced
by imports, placement, higher tiers, inheritance, defaults, CLI, and callbacks;
preserve ordinary strict-peer and upsert behavior. Check YAML disagreement under
replace, harmless absent or empty collections, schema-owned shape requirements,
unrelated sequences remaining unselected, and concrete-path diagnostics.
Verify both constructors feed the same registration model, including mixed
exact and dynamic registrations, compound identities, and definition-time
rejection of empty identity field sets.
Cover ancestor replacement followed by collection reintroduction, including a
surviving append mode with and without renewed identity, explicit replacement,
and application-registered identity. Verify that `extends: null` cannot supply
an identity component and that structural-control exclusion uses ordinary
value-free missing-identity diagnostics.
Update the public contract and consumer examples when behavior lands.
The collection-policy and CLI-ordering decisions are accepted in
[ADR 0017](../adr/0017-collection-sequence-policy.md) for collection policy and
[ADR 0018](../adr/0018-cli-after-structural-resolution.md) for CLI ordering.
Their scoped supersession is recorded in the affected earlier ADRs, and the
[architecture policy](../policy/architecture.md) reflects the accepted decisions.
These govern the target architecture; the configuration contract continues to
describe implemented behavior until the feature lands. This roadmap does not
select migration machinery.
ADR 0017 now includes inheritance within identified sequences;
[ADR 0009](../adr/0009-sibling-inheritance.md) records that scope extension in its
status while preserving its decision content. The architecture policy is aligned.
Dynamic selectors extend the destinations covered by application registration;
they retain the accepted application authority and final-validation boundary.
No open feature decisions remain. The implementation plan translates this end
state into reviewable stages without adding guarantees, restrictions, or
combination semantics.