Consolidate the sequence roadmap around the settled design
This commit is contained in:
669
docs/roadmap/roadmap.md
Normal file
669
docs/roadmap/roadmap.md
Normal file
@@ -0,0 +1,669 @@
|
|||||||
|
# Sequence Identity And Combination Roadmap
|
||||||
|
|
||||||
|
Status: settled feature roadmap, ready for a staged implementation plan. This
|
||||||
|
is the targeted end state, not an implemented contract. The subsequent plan will
|
||||||
|
specify execution order, the exact Go API, internal bookkeeping, and validation
|
||||||
|
stages.
|
||||||
|
|
||||||
|
## Purpose And Scope
|
||||||
|
|
||||||
|
Allow configuration authors to assemble collections from independently authored
|
||||||
|
records, declare identity in configuration, and progressively extend a baseline
|
||||||
|
collection through higher layers. Authors can replace the entire collection,
|
||||||
|
append distinct records, or replace complete records by identity while retaining
|
||||||
|
unmatched records. Sibling inheritance uses the same sequence operations and
|
||||||
|
identity-based defaults.
|
||||||
|
|
||||||
|
The motivating use case is a prompt catalog whose source-backed baseline supplies
|
||||||
|
records and whose application or installation layers add or supersede records.
|
||||||
|
Applications can optionally impose identity constraints that configuration must
|
||||||
|
respect and that hold in the final loaded configuration.
|
||||||
|
|
||||||
|
The feature covers ordinary and layered loading, imported items, inline
|
||||||
|
sequences, selected overlays, and sibling inheritance. Typed defaults, CLI
|
||||||
|
assignments, and application resolution retain their own combination semantics;
|
||||||
|
CLI application moves later in the loading lifecycle.
|
||||||
|
|
||||||
|
This scope is narrower than the deferred
|
||||||
|
[identified-record sequences proposal](future.md#1-identified-record-sequences).
|
||||||
|
Recursive merging of matching records and record-specific inheritance remain
|
||||||
|
separate candidates. The initial declaration surface is `configkit.sequences`;
|
||||||
|
`key` and `mode` shorthand on imports is deferred. Application lists remain
|
||||||
|
ordinary YAML sequences without control wrappers. Unkeyed sequence concatenation
|
||||||
|
is outside this feature.
|
||||||
|
|
||||||
|
## Current Baseline And Required Changes
|
||||||
|
|
||||||
|
The [import contract](../config.md#explicit-imports) owns existing selectors,
|
||||||
|
destinations, placement, and ordering. `placement: sequence-item` already places
|
||||||
|
each selected source mapping as one item at its effective destination. Item-local
|
||||||
|
helper imports can contribute fields before placement.
|
||||||
|
|
||||||
|
The [composition contract](../config.md#strict-document-composition),
|
||||||
|
[layered-loading contract](../config.md#layered-loading), and
|
||||||
|
[inheritance contract](../config.md#sibling-inheritance) describe the implemented
|
||||||
|
baseline:
|
||||||
|
|
||||||
|
| Area | Current behavior | Target behavior |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Identity declaration | `Definition.SequenceCompositions` registers one literal string field at each registered mapping path. | YAML collection declarations and optional authoritative application registrations support field sets and scalar identity components. |
|
||||||
|
| Strict composition | Registered sequences append distinct string identities and reject duplicates; repeated unregistered sequences conflict. | Collections with effective identity append distinct records and reject duplicates. Applicable lower-tier identity is available before composing higher-tier peers. Unidentified sequences retain strict-conflict behavior. |
|
||||||
|
| Source precedence and selected overlays | A present higher sequence replaces the lower sequence wholesale. | Collection policy selects `replace`, `append`, or `upsert`; overlays can declare policy for inline sequences. |
|
||||||
|
| Sibling inheritance | An inheriting mapping recursively overlays its resolved parent; supplied sequences replace wholesale. | Mappings retain recursive combination, while sequences use destination policy and the same three operations. |
|
||||||
|
| CLI overrides | Apply before inheritance and typed defaults/decoding. | Apply after inheritance and typed defaults/decoding, before application resolution. |
|
||||||
|
| Identity enforcement | Registration governs strict sequence composition. | Structural identity checks cover identified combinations; application registrations additionally constrain final effective sequences. |
|
||||||
|
|
||||||
|
Source-backed underlays participate in source assembly. `Definition.Defaults`
|
||||||
|
provides typed defaults later; extending source combination does not turn typed
|
||||||
|
default lists into source underlays.
|
||||||
|
|
||||||
|
## Collection Declarations
|
||||||
|
|
||||||
|
Use a source-root `configkit.sequences` block containing collection declarations.
|
||||||
|
Each entry identifies a `destination` and may declare `key`, `mode`, or both.
|
||||||
|
Identity and mode can be supplied independently. This form serves imported items,
|
||||||
|
inline records, ordinary mapping imports, and inline sequences in selected
|
||||||
|
overlays.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
configkit:
|
||||||
|
sequences:
|
||||||
|
- destination: [prompts]
|
||||||
|
key: [namespace, id]
|
||||||
|
mode: append
|
||||||
|
imports:
|
||||||
|
- dir: prompts
|
||||||
|
pattern: '*.yaml'
|
||||||
|
destination: [prompts]
|
||||||
|
placement: sequence-item
|
||||||
|
```
|
||||||
|
|
||||||
|
Each selected file contributes a record such as:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
namespace: support
|
||||||
|
id: welcome
|
||||||
|
text: Welcome
|
||||||
|
```
|
||||||
|
|
||||||
|
Here append retains a lower-layer collection and adds distinct higher-layer
|
||||||
|
records. Identity and combination controls belong to the collection declaration,
|
||||||
|
so imported and inline contributions share one policy surface.
|
||||||
|
|
||||||
|
### Destinations And Scope
|
||||||
|
|
||||||
|
`destination` uses the existing sequence of literal mapping keys. Resolve it
|
||||||
|
relative to the declaring source's placement: a fragment mounted under `catalog`
|
||||||
|
can declare `[prompts]` locally. A sequence-item source can declare a nested
|
||||||
|
collection within its own item without referring to an eventual numeric index.
|
||||||
|
Preserve source and item-local scope boundaries.
|
||||||
|
|
||||||
|
A collection's scope is its resolved destination within its composition scope.
|
||||||
|
Resolve relative destinations before reconciling declarations. Different imported
|
||||||
|
items retain separate scopes for their nested collections. A declaration governs
|
||||||
|
all contributions to its collection in the tier, including imported and inline
|
||||||
|
records; it does not govern only records in the declaring file.
|
||||||
|
|
||||||
|
Gather declarations before composing the relevant scope so their effect does not
|
||||||
|
depend on which contribution is assembled first. Retain applicable controls
|
||||||
|
through source precedence and inheritance, then keep them out of application
|
||||||
|
decoding.
|
||||||
|
|
||||||
|
Sequence declarations are independent of filesystem import enablement. Selected
|
||||||
|
overlays may contain `configkit.sequences`, extending the current blanket
|
||||||
|
prohibition on overlay `configkit` declarations. This does not enable imports
|
||||||
|
in overlays.
|
||||||
|
|
||||||
|
### Overlay Authoring
|
||||||
|
|
||||||
|
For an existing sequence `[foo, bar]`, an overlay can replace `foo` in full:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
configkit:
|
||||||
|
sequences:
|
||||||
|
- destination: [prompts]
|
||||||
|
key: id
|
||||||
|
mode: upsert
|
||||||
|
prompts:
|
||||||
|
- id: foo
|
||||||
|
text: Revised definition
|
||||||
|
```
|
||||||
|
|
||||||
|
The result is `[foo*, bar]`. The explicit upsert mode can be omitted when no
|
||||||
|
different explicit mode applies from a lower tier: effective identity then
|
||||||
|
selects upsert. To add a distinct record instead:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
configkit:
|
||||||
|
sequences:
|
||||||
|
- destination: [prompts]
|
||||||
|
key: id
|
||||||
|
mode: append
|
||||||
|
prompts:
|
||||||
|
- id: baz
|
||||||
|
text: Another prompt
|
||||||
|
```
|
||||||
|
|
||||||
|
The result is `[foo, bar, baz]`. To replace the whole sequence:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
configkit:
|
||||||
|
sequences:
|
||||||
|
- destination: [prompts]
|
||||||
|
mode: replace
|
||||||
|
prompts:
|
||||||
|
- id: bar
|
||||||
|
text: Bar definition
|
||||||
|
- id: baz
|
||||||
|
text: Baz definition
|
||||||
|
```
|
||||||
|
|
||||||
|
The result is `[bar, baz]`. Explicitly selecting the default operation is valid.
|
||||||
|
With no applicable explicit mode, omitting this declaration selects replace for
|
||||||
|
an unidentified collection and upsert for an identified one. A retained explicit
|
||||||
|
mode continues to govern.
|
||||||
|
|
||||||
|
## Identity Definitions And Constraints
|
||||||
|
|
||||||
|
### Field Sets
|
||||||
|
|
||||||
|
`key` accepts a string or a nonempty list of strings naming direct, literal
|
||||||
|
fields in each record. A name such as `metadata.id` denotes that literal field,
|
||||||
|
not a nested lookup. Treat field names as a set: remove repeated names and
|
||||||
|
ignore authored order when comparing definitions.
|
||||||
|
|
||||||
|
Thus `id`, `[id]`, and `[id, id]` are equivalent; `[namespace, id]`,
|
||||||
|
`[id, namespace]`, and `[namespace, id, id]` are also equivalent. An empty list
|
||||||
|
is invalid because it supplies no identity fields. These rules apply to both
|
||||||
|
configuration declarations and application registrations.
|
||||||
|
|
||||||
|
Use a consistent field order internally to construct identity tuples. Preserve
|
||||||
|
component types and tuple boundaries so unstructured concatenation cannot cause
|
||||||
|
otherwise distinct identities to collide.
|
||||||
|
|
||||||
|
### Scalar Components And Equality
|
||||||
|
|
||||||
|
Each identified record must be a mapping with every designated identity field
|
||||||
|
present. Components may be strings, numbers, booleans, or explicit null. Missing
|
||||||
|
fields and mapping or sequence components are invalid. Present null differs
|
||||||
|
from absence; two single-field null identities are equal. Compound identities
|
||||||
|
are equal when all corresponding components are equal.
|
||||||
|
|
||||||
|
Strings compare exactly, including empty strings, without trimming, case folding,
|
||||||
|
or normalization. Scalar categories remain distinct: `"1"`, `1`, `true`, and
|
||||||
|
null are different identities.
|
||||||
|
|
||||||
|
Numbers compare by mathematical value across integer, fractional, and exponent
|
||||||
|
spellings: `1`, `1.0`, and `1e0` are equal, as are `0` and `-0`. Use exact numeric
|
||||||
|
normalization for identity comparison without rounding through a floating-point
|
||||||
|
conversion. This does not rewrite authored values or change existing typed
|
||||||
|
decoding and projection semantics. The parser's rejection of non-finite numeric
|
||||||
|
forms remains unchanged.
|
||||||
|
|
||||||
|
### Application-Owned Constraints
|
||||||
|
|
||||||
|
Retain an optional application interface for registering collections and imposing
|
||||||
|
their identity definitions. Configuration may define identity where no application
|
||||||
|
constraint exists. When a declaration omits `key`, an applicable registration
|
||||||
|
supplies identity. An explicit equivalent definition is accepted, and a different
|
||||||
|
definition fails under every mode, including replace.
|
||||||
|
|
||||||
|
For example, if the application requires `id`, `key: [id, name]` is invalid:
|
||||||
|
that compound identity would allow repeated `id` values with different names.
|
||||||
|
A registration is an authoritative constraint, not an omission-filling default.
|
||||||
|
Copying a collection through inheritance does not create another application
|
||||||
|
registration; registrations constrain their registered destinations.
|
||||||
|
|
||||||
|
Identity constraints validate actual sequences. They do not independently require
|
||||||
|
a path to exist, be non-null, or have sequence shape. The application schema and
|
||||||
|
validation own those requirements. Final-result enforcement for application
|
||||||
|
registrations is defined under [Validation And Diagnostics](#validation-and-diagnostics).
|
||||||
|
|
||||||
|
## Strict Composition And Precedence Operations
|
||||||
|
|
||||||
|
### Compose Peers Before Applying Precedence
|
||||||
|
|
||||||
|
Resolve applicable identity and mode policy, strictly compose the tier's peer
|
||||||
|
contributions, then apply one selected operation per collection against the
|
||||||
|
accumulated lower value. The tier operation does not sequentially replace peer
|
||||||
|
source contributions. Selected overlay contributions use the same combination
|
||||||
|
semantics.
|
||||||
|
|
||||||
|
Within a strict tier, equivalent identity definitions coalesce and different
|
||||||
|
field sets fail. Agreeing explicit modes coalesce; conflicting explicit modes
|
||||||
|
fail. Omission expresses no preference. A root declaration or a later declaration
|
||||||
|
has no precedence over conflicting peers. Explicitly selecting a default mode
|
||||||
|
is still a real instruction and can conflict with another explicit mode.
|
||||||
|
|
||||||
|
Identified peer sequences append distinct records. Duplicate identities among
|
||||||
|
peers or within a contributed list fail under all three modes, including replace
|
||||||
|
and upsert. Equal or disjoint record payloads do not excuse a duplicate identity.
|
||||||
|
Unidentified sequences retain their current strict-composition conflict behavior.
|
||||||
|
|
||||||
|
For example, higher-tier fragments `[C]` and `[D]` with an agreed identity assemble
|
||||||
|
as `[C, D]`. Against lower `[A, B]`, append or upsert produces `[A, B, C, D]`,
|
||||||
|
while replace produces `[C, D]`. Peers `[B]` and `[B*]` conflict even under
|
||||||
|
upsert because neither has higher precedence.
|
||||||
|
|
||||||
|
### Available Operations
|
||||||
|
|
||||||
|
Use one spelling for each mode: `replace`, `append`, and `upsert`.
|
||||||
|
|
||||||
|
| Mode | Behavior at source-precedence and inheritance boundaries |
|
||||||
|
| --- | --- |
|
||||||
|
| `replace` | Replace the complete lower sequence with the assembled higher sequence. |
|
||||||
|
| `append` | Retain lower records and append higher records; duplicate identities fail. |
|
||||||
|
| `upsert` | Replace matching records in full, retain unmatched lower records, and append new identities. |
|
||||||
|
|
||||||
|
Append and upsert require effective identity. ConfigKit does not infer identity
|
||||||
|
from an `id` field or record position. Whole-sequence replacement does not require
|
||||||
|
identity agreement with lower YAML policy, but application constraints still apply.
|
||||||
|
|
||||||
|
Letters denote record identities; `B*` has the same identity as `B` with a different
|
||||||
|
payload:
|
||||||
|
|
||||||
|
| Lower | Higher | Mode | Result |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `[A, B]` | `[C]` | `append` | `[A, B, C]` |
|
||||||
|
| `[A, B]` | `[C]` | `replace` | `[C]` |
|
||||||
|
| `[A, B]` | `[B*]` | `append` | Duplicate-identity error |
|
||||||
|
| `[A, B]` | `[B*]` | `replace` | `[B*]` |
|
||||||
|
| `[A, B]` | `[B*]` | `upsert` | `[A, B*]` |
|
||||||
|
| `[A, B]` | `[B*, C]` | `upsert` | `[A, B*, C]` |
|
||||||
|
|
||||||
|
Upsert replaces the whole matched record. If lower `B` has `id`, `text`, and
|
||||||
|
`temperature`, while higher `B` has only `id` and `text`, the result at this
|
||||||
|
boundary has only `id` and `text`. Subsequent defaulting or other lifecycle
|
||||||
|
operations retain their own semantics. Mappings continue to combine recursively;
|
||||||
|
matching sequence records do not recursively merge fields.
|
||||||
|
|
||||||
|
### Mode Selection And Persistence
|
||||||
|
|
||||||
|
Select the operation in this order:
|
||||||
|
|
||||||
|
1. Reconcile explicit modes within the current tier; conflicting peers fail.
|
||||||
|
2. Use the current tier's explicit mode if supplied.
|
||||||
|
3. Otherwise retain an applicable explicit mode from a lower tier.
|
||||||
|
4. With no explicit mode, infer upsert when effective identity is available.
|
||||||
|
5. Otherwise infer replace.
|
||||||
|
|
||||||
|
Effective identity can come from an application registration, current declarations,
|
||||||
|
or applicable lower policy, subject to the identity agreement and lifetime rules
|
||||||
|
below. Determine the inferred mode from available identity before applying the
|
||||||
|
operation. A plain higher-tier sequence can therefore use upsert without repeating
|
||||||
|
a key declaration.
|
||||||
|
|
||||||
|
An explicit mode persists at its applicable destination until a higher-priority
|
||||||
|
explicit declaration changes it. New sequence values alone do not reset it, and
|
||||||
|
an unrelated overlay leaves it intact. A mode-only declaration can change policy
|
||||||
|
without supplying records; it neither clears the collection nor retroactively
|
||||||
|
recombines earlier tiers. Later source contributions and inheritance use the
|
||||||
|
resulting policy.
|
||||||
|
|
||||||
|
| Lower explicit mode | Higher declaration | Effective mode |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `append` | Mode omitted | `append` |
|
||||||
|
| `append` | `upsert` | `upsert` |
|
||||||
|
| `upsert` | `replace` | `replace` |
|
||||||
|
| None | Mode omitted, effective identity available | Inferred `upsert` |
|
||||||
|
| None | Mode omitted, no effective identity | Inferred `replace` |
|
||||||
|
|
||||||
|
Only explicit modes persist. Inferred defaults are recomputed from effective
|
||||||
|
identity when needed. Explicit replace continues to apply in subsequent tiers
|
||||||
|
until another mode is declared; introducing identity alone does not override it.
|
||||||
|
Mode persistence does not preserve YAML identity discarded by replacement, nor
|
||||||
|
policies owned by discarded records. Source precedence and inheritance use the
|
||||||
|
same destination policy, without separate per-phase settings. CLI assignments
|
||||||
|
remain outside mode selection.
|
||||||
|
|
||||||
|
### Presence, Empty Sequences, Null, And Kind Mismatches
|
||||||
|
|
||||||
|
At a collection's precedence boundary:
|
||||||
|
|
||||||
|
| Higher contribution | `replace` | `append` / `upsert` |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Absent | Leave the lower value unchanged. | Leave the lower value unchanged. |
|
||||||
|
| Explicit `[]` | Clear the sequence. | Contribute no records. |
|
||||||
|
| Explicit null or another non-sequence value | Ordinary replacement, subject to application schema. | Error: the operation requires a sequence. |
|
||||||
|
|
||||||
|
A declaration alone does not clear a collection. An optional import selecting
|
||||||
|
nothing contributes nothing and does not erase existing records. An absent lower
|
||||||
|
collection can be initialized by an incoming sequence under any mode, subject to
|
||||||
|
identity requirements. Explicit lower null or another non-sequence value is not
|
||||||
|
absence: append and upsert fail against it, while replacement remains available.
|
||||||
|
An empty incoming sequence bypasses neither kind checks nor identity requirements.
|
||||||
|
|
||||||
|
With effective identity and no explicit mode, `prompts: []` uses upsert and does
|
||||||
|
not clear the collection. Clearing requires replace, selected in the current tier
|
||||||
|
or retained from lower policy. Replacing an identified collection with null also
|
||||||
|
requires replace and a schema permitting null. Collection null is distinct from
|
||||||
|
a present null identity component inside a record.
|
||||||
|
|
||||||
|
### Ordering
|
||||||
|
|
||||||
|
Retain existing dependency-ready source scheduling within each tier and authored
|
||||||
|
item order within lists. Gathering declarations does not change assembly order;
|
||||||
|
identity values do not sort records.
|
||||||
|
|
||||||
|
- Append keeps lower records in order, followed by higher records in assembled order.
|
||||||
|
- Replace keeps only higher records in their assembled order.
|
||||||
|
- Upsert keeps a matched record's lower position and appends new identities in
|
||||||
|
higher-tier order.
|
||||||
|
|
||||||
|
For example, lower `[A, B, C]` and higher `[C*, A*, D]` produce `[A*, B, C*, D]`
|
||||||
|
under upsert. Whole-sequence replacement gives the higher tier control of the
|
||||||
|
complete sequence order.
|
||||||
|
|
||||||
|
## Policy Agreement, Lifetime, And Ownership
|
||||||
|
|
||||||
|
### Identity Agreement And Replacement
|
||||||
|
|
||||||
|
Append and upsert use one agreed identity for all participating records:
|
||||||
|
|
||||||
|
| Lower identity | Higher declaration | Result when combining records |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `id` | Omitted | Use `id`. |
|
||||||
|
| `id` | `id`, `[id]`, or `[id, id]` | Accept the equivalent definition. |
|
||||||
|
| `id` | `[namespace, id]` | Error: different definitions. |
|
||||||
|
| None | `id` | Establish `id` and validate all participating records. |
|
||||||
|
|
||||||
|
Introducing identity for an unkeyed lower sequence requires checking both lower
|
||||||
|
and incoming records, including uniqueness within each contribution. With no
|
||||||
|
identity from the application or either participating collection, append and
|
||||||
|
upsert fail.
|
||||||
|
|
||||||
|
YAML identity follows the collection that survives an operation:
|
||||||
|
|
||||||
|
| Operation | Effective YAML identity |
|
||||||
|
| --- | --- |
|
||||||
|
| Append or upsert | Retain or establish the agreed identity. |
|
||||||
|
| Whole-sequence replacement | Discard lower YAML identity and use the replacing tier's identity declaration, if any. |
|
||||||
|
| Replacement by null or another non-sequence value | Discard lower YAML identity and retain any identity explicitly declared by the replacing tier. |
|
||||||
|
|
||||||
|
Application constraints remain authoritative under every operation. Without an
|
||||||
|
application constraint, replacement can introduce a different identity, such as
|
||||||
|
changing from `id` to `name`. Replacing a sequence without a new key declaration
|
||||||
|
or application registration produces an unkeyed sequence. No identity-removal
|
||||||
|
directive is needed.
|
||||||
|
|
||||||
|
The reset occurs only when the collection value is actually replaced. Declaring
|
||||||
|
replace without supplying a replacement does not authorize changing the identity
|
||||||
|
of retained records. Explicit mode lifetime is separate from YAML identity
|
||||||
|
lifetime.
|
||||||
|
|
||||||
|
### Declarations Without Sequence Values
|
||||||
|
|
||||||
|
A key declaration without accompanying records applies immediately to an existing
|
||||||
|
retained sequence: check agreement and validate its records, without recombining
|
||||||
|
earlier tiers. For example, a higher tier that only declares
|
||||||
|
`destination: [prompts], key: id` identifies and validates the retained lower
|
||||||
|
`prompts` sequence.
|
||||||
|
|
||||||
|
If no collection exists, retain the declaration in its owning scope for later
|
||||||
|
contributions. An otherwise empty underlay can establish identity for main-tier
|
||||||
|
fragments. An explicit identity declaration can also remain pending at a
|
||||||
|
non-sequence destination; it does not independently impose a shape requirement.
|
||||||
|
The presence and kind rules still determine whether an attempted operation fails.
|
||||||
|
Item-local declarations retain their ownership boundaries.
|
||||||
|
|
||||||
|
### Lower-Tier Identity During Strict Composition
|
||||||
|
|
||||||
|
Resolve applicable identity before composing a higher tier's peers, using
|
||||||
|
application registrations, available lower-tier policy, and declarations gathered
|
||||||
|
within the higher tier's scope. Authors need not repeat a baseline key solely
|
||||||
|
because the higher tier has multiple fragments.
|
||||||
|
|
||||||
|
For example, an underlay establishes `key: id` for `prompts`, and two main-tier
|
||||||
|
fragments contribute `[foo]` and `[bar]` without a key declaration. They strictly
|
||||||
|
compose as `[foo, bar]` before the main tier combines with the accumulated lower
|
||||||
|
collection. If both contribute `foo`, composition fails; upsert does not give
|
||||||
|
precedence to peers.
|
||||||
|
|
||||||
|
Lower YAML identity remains subject to agreement, replacement, and ownership.
|
||||||
|
A replacing collection must not be constrained by obsolete lower identity just
|
||||||
|
because policy resolution precedes composition. Likewise, a replacement record
|
||||||
|
cannot borrow nested identity owned by the discarded lower record, even when
|
||||||
|
outer identity or numeric position matches. Application constraints still apply.
|
||||||
|
Higher-tier declarations do not retroactively repair invalid lower-tier composition.
|
||||||
|
|
||||||
|
### Nested Record Ownership
|
||||||
|
|
||||||
|
Attach item-local policies to their owning contributions so they follow surviving
|
||||||
|
records independently of numeric indexes. Whole-record upsert discards the old
|
||||||
|
record's local policies along with its fields. Whole-sequence replacement discards
|
||||||
|
policies owned by removed records. A replacement record supplies its own local
|
||||||
|
declarations; equal outer identity or final index does not transfer discarded
|
||||||
|
policies.
|
||||||
|
|
||||||
|
For example, a source imported as one record into `prompts` can declare:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
configkit:
|
||||||
|
sequences:
|
||||||
|
- destination: [steps]
|
||||||
|
key: id
|
||||||
|
mode: append
|
||||||
|
id: foo
|
||||||
|
steps:
|
||||||
|
- id: prepare
|
||||||
|
```
|
||||||
|
|
||||||
|
The declaration governs `steps` inside that `foo` record. Retaining `foo` retains
|
||||||
|
its local policy, including the explicit mode. Upserting a new `foo` discards the
|
||||||
|
old `steps` and its declaration even though the outer record keeps its position.
|
||||||
|
|
||||||
|
Policies independently applicable at a destination are not discarded merely
|
||||||
|
because an individual record is replaced. Ownership determines which declarations
|
||||||
|
are removed. Application registrations remain constraints at their registered
|
||||||
|
destinations. Inheritance copies transfer identity and choose operations as
|
||||||
|
specified next.
|
||||||
|
|
||||||
|
## Sibling Inheritance
|
||||||
|
|
||||||
|
Complete source-tier assembly before resolving inheritance. Retain the existing
|
||||||
|
dependency-ready algorithm and sibling scope rules. At each inheritance combination,
|
||||||
|
the resolved parent's collection is the lower value and the inheriting mapping's
|
||||||
|
local collection is the higher value. Mappings combine recursively; identified
|
||||||
|
sequence records remain whole-record replacements.
|
||||||
|
|
||||||
|
Use the inheriting destination collection's policy: an applicable explicit mode,
|
||||||
|
including one retained through source tiers, wins. Otherwise infer upsert with
|
||||||
|
effective identity and replace without it. There is no separate default-append
|
||||||
|
rule for inheritance. The same destination policy governs source-precedence and
|
||||||
|
inheritance combinations.
|
||||||
|
|
||||||
|
YAML identity follows a collection copied through inheritance, subject to agreement
|
||||||
|
and replacement rules. A source collection's mode does not automatically become an
|
||||||
|
instruction at a different inheriting destination. For example, source `steps`
|
||||||
|
has `key: id` and `mode: append`; an inheriting mapping supplies local `steps`
|
||||||
|
without an applicable explicit mode. The copied identity selects upsert there.
|
||||||
|
An explicit destination append instead selects append. A matching destination key
|
||||||
|
agrees; a different key conflicts when combining records, while explicit replace
|
||||||
|
can establish new YAML identity subject to application constraints.
|
||||||
|
|
||||||
|
For example:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
configkit:
|
||||||
|
sequences:
|
||||||
|
- destination: [parent, items]
|
||||||
|
key: id
|
||||||
|
child:
|
||||||
|
items:
|
||||||
|
- id: foo
|
||||||
|
value: A
|
||||||
|
- id: bar
|
||||||
|
value: B
|
||||||
|
parent:
|
||||||
|
extends: child
|
||||||
|
items:
|
||||||
|
- id: baz
|
||||||
|
value: C
|
||||||
|
```
|
||||||
|
|
||||||
|
Despite the names, `parent` inherits from `child`. Identity at `[parent, items]`
|
||||||
|
selects upsert, producing `[foo, bar, baz]`. If the local sequence instead supplies
|
||||||
|
a revised `foo`, the result is `[foo*, bar]`. Append rejects that repeated identity;
|
||||||
|
replace retains only the local sequence.
|
||||||
|
|
||||||
|
The same ordering, presence, and kind rules apply at inheritance boundaries.
|
||||||
|
An omitted local sequence retains the inherited sequence. Explicit `[]` clears it
|
||||||
|
under replace and contributes no records under append or upsert. Retain controls
|
||||||
|
until inheritance uses them, then exclude them from application decoding.
|
||||||
|
|
||||||
|
## Loading Lifecycle
|
||||||
|
|
||||||
|
Use the following order:
|
||||||
|
|
||||||
|
1. Assemble source tiers and selected overlays, resolving policy before each
|
||||||
|
tier's strict composition and applying collection operations at precedence
|
||||||
|
boundaries.
|
||||||
|
2. Resolve sibling inheritance, then apply typed defaults and decode the assembled
|
||||||
|
configuration.
|
||||||
|
3. Apply explicit CLI overrides to that configuration and check supplied values
|
||||||
|
against the application schema.
|
||||||
|
4. Run application resolution against the overridden configuration.
|
||||||
|
5. Resolve effective secret references, validate the final configuration including
|
||||||
|
application-registered identities, and publish it.
|
||||||
|
|
||||||
|
### Typed Defaults And Application Resolution
|
||||||
|
|
||||||
|
Source-backed underlays, including application-supplied mapping sources,
|
||||||
|
participate in sequence combination. Typed `Definition.Defaults` remains the
|
||||||
|
initial decoded state: supplied sequences replace default sequences. Source
|
||||||
|
append does not append to a typed default list.
|
||||||
|
|
||||||
|
Application resolution retains assignment and recorded-transformation semantics.
|
||||||
|
ConfigKit does not automatically combine a callback's assigned sequence with its
|
||||||
|
previous value. Application-registered identities receive final validation
|
||||||
|
independently of how defaults, CLI overrides, or callbacks produce their values.
|
||||||
|
|
||||||
|
### CLI Precedence
|
||||||
|
|
||||||
|
CLI overrides are the highest-precedence input to application resolution.
|
||||||
|
Supplied sequences replace whole sequences, including sequences inside supplied
|
||||||
|
mappings. Mappings retain recursive override behavior; empty sequences and
|
||||||
|
permitted nulls retain explicit replacement meaning. Configuration append or
|
||||||
|
upsert does not reinterpret CLI assignments. Inheritance and defaulting do not
|
||||||
|
run again afterward: overriding assembled `[A, B]` with `[C]` presents `[C]` to
|
||||||
|
application resolution without inheritance reintroducing prior records.
|
||||||
|
|
||||||
|
Application resolution sees CLI values and their explicit-presence and source
|
||||||
|
metadata. `Resolution.Default` preserves explicit CLI values; normalization and
|
||||||
|
recorded application transformations may still change them. For example, a derived
|
||||||
|
URL can use a CLI hostname, while an explicit CLI URL can be preserved by an
|
||||||
|
absence-based default. Retain recording and provenance requirements for those
|
||||||
|
transformations.
|
||||||
|
|
||||||
|
Reject structural directives such as `extends` and `configkit` in CLI override
|
||||||
|
paths or supplied values. CLI input supplies application values after structural
|
||||||
|
processing; it cannot supply or cancel inheritance directives, change completed
|
||||||
|
relationships, or prevent earlier inheritance failures.
|
||||||
|
|
||||||
|
Typed decoding also precedes CLI application, so a CLI replacement cannot repair
|
||||||
|
an earlier decoding failure. A source string in an integer field fails before a
|
||||||
|
valid CLI integer can apply. This changes current override-before-decoding
|
||||||
|
behavior. Semantic validation remains after overrides and application resolution.
|
||||||
|
|
||||||
|
Secret lookup uses effective references after overrides and resolution. Neither
|
||||||
|
stage bypasses final application identity constraints or semantic validation.
|
||||||
|
|
||||||
|
## Validation And Diagnostics
|
||||||
|
|
||||||
|
### Structural Identity Checks
|
||||||
|
|
||||||
|
Allow item-local helper imports to complete and consume applicable non-main
|
||||||
|
source-version headers before checking payload identity for collection assembly.
|
||||||
|
A helper can supply an identity component; a consumed header cannot supply one
|
||||||
|
after removal.
|
||||||
|
|
||||||
|
Validate each identified contribution before combination and check uniqueness in
|
||||||
|
the resulting identified collection, including inheritance operations. Upsert
|
||||||
|
matches across precedence are replacements; duplicates within a contribution
|
||||||
|
remain errors. Later inheritance, typed defaults, or callbacks cannot repair an
|
||||||
|
identity missing at an earlier assembly boundary.
|
||||||
|
|
||||||
|
YAML-only identities are enforced during structural composition, precedence
|
||||||
|
assembly, and identified inheritance operations. They have no final-result
|
||||||
|
guarantee after typed defaulting, CLI overrides, or application resolution. Those
|
||||||
|
stages can produce duplicate identities or otherwise violate a YAML-only identity.
|
||||||
|
Applications needing a final guarantee register the constraint through the
|
||||||
|
application-owned interface.
|
||||||
|
|
||||||
|
### Final Application Constraints
|
||||||
|
|
||||||
|
On successful loading, final effective collections covered by application
|
||||||
|
registrations satisfy their declared identity constraints. Retain assembly-time
|
||||||
|
checks and validate final effective sequences before publication, after defaults,
|
||||||
|
inheritance, overrides, application resolution, and secret lookup.
|
||||||
|
|
||||||
|
This guarantee covers actual sequences, not path existence or allowed shape.
|
||||||
|
It does not require every intermediate callback state to satisfy the constraint
|
||||||
|
and does not cover caller mutations after loading returns. Final checks examine
|
||||||
|
effective typed values; assembly checks validate authored contributions. A final
|
||||||
|
scalar alone does not prove that a source explicitly supplied that field.
|
||||||
|
|
||||||
|
Use the existing canonical final-value representation and shared publication
|
||||||
|
path as the foundation. The guarantee requires neither a new mutation-history
|
||||||
|
system nor exact causal reconstruction for records supplied by defaults or
|
||||||
|
callbacks.
|
||||||
|
|
||||||
|
### Diagnostics And Inspection
|
||||||
|
|
||||||
|
Identity diagnostics identify collection and item paths using available origin
|
||||||
|
information without exposing identity values. Conflicting declarations identify
|
||||||
|
the collection destination and declaration locations.
|
||||||
|
|
||||||
|
Correctness includes final values, projected item paths, source destinations,
|
||||||
|
provenance, history, sensitivity, and passive resource contexts. Appending can
|
||||||
|
change final indexes; replacing changes which contributions survive. Preserve
|
||||||
|
the existing loading and inspection contracts through both operations, including
|
||||||
|
policy ownership and inheritance transfer.
|
||||||
|
|
||||||
|
## Work Needed From The Baseline
|
||||||
|
|
||||||
|
The implementation plan should cover these connected changes, with execution
|
||||||
|
order and test sequencing defined there:
|
||||||
|
|
||||||
|
- **Declarations and public API:** extend application registration to the settled
|
||||||
|
identity model; parse and consume `configkit.sequences` across supported source
|
||||||
|
forms and selected overlays; preserve placement-relative and item-local scopes.
|
||||||
|
Keep sequence declarations independent of filesystem import enablement.
|
||||||
|
- **Identity and policy representation:** implement field-set comparison, typed
|
||||||
|
scalar identity tuples with exact numeric equality, declaration agreement,
|
||||||
|
pending declarations, explicit-mode persistence, replacement lifetime, and
|
||||||
|
record-owned policy transfer. Choose bookkeeping proportional to these semantics.
|
||||||
|
- **Assembly handoffs:** make accumulated lower policy available before higher-tier
|
||||||
|
strict composition. Separate source acquisition from policy-aware composition
|
||||||
|
or otherwise revise assembly sequencing; current main-before-underlay composition
|
||||||
|
is not a design constraint. Preserve distinctions among acquisition order,
|
||||||
|
dependency-ready scheduling within tiers, and precedence order. Account for
|
||||||
|
source records, callbacks, main-version checks, and failure diagnostics. Retain
|
||||||
|
version ownership and validation semantics; higher replacement cannot hide
|
||||||
|
lower-tier structural failures.
|
||||||
|
- **Combination and inheritance:** implement the three operations with strict peer
|
||||||
|
handling, whole-record replacement, presence rules, stable ordering, and the
|
||||||
|
destination policy rules for inheritance. Preserve provenance, history,
|
||||||
|
sensitivity, paths, and resource contexts as records survive, move, or disappear.
|
||||||
|
- **Lifecycle and validation:** move CLI application after inheritance and typed
|
||||||
|
defaults/decoding, preserve explicit-presence and source metadata for application
|
||||||
|
resolution, reject CLI structural directives, and add final application identity
|
||||||
|
validation through the shared publication path. Keep YAML-only enforcement
|
||||||
|
within structural operations.
|
||||||
|
- **Documentation and verification:** exercise ordinary and layered loading,
|
||||||
|
imported and inline collections, item-local scopes, version-header consumption,
|
||||||
|
selected overlays, inheritance, typed defaults, CLI, callbacks, and inspection.
|
||||||
|
Update the public contract and consumer examples when behavior lands.
|
||||||
|
|
||||||
|
Before implementation, record the durable architectural changes in a superseding
|
||||||
|
decision and reconcile affected architecture policy. The feature changes identity
|
||||||
|
ownership and precedence replacement in
|
||||||
|
[ADR 0014](../adr/0014-sequence-placement-and-composition.md), and sequence
|
||||||
|
combination and CLI-before-inheritance ordering in
|
||||||
|
[ADR 0009](../adr/0009-sibling-inheritance.md). Existing ADRs and contracts remain
|
||||||
|
authoritative until that architectural and implementation work lands. This roadmap
|
||||||
|
does not select migration machinery.
|
||||||
|
|
||||||
|
No open feature decisions remain. The implementation plan should translate this
|
||||||
|
end state into reviewable stages without adding new guarantees, restrictions, or
|
||||||
|
combination semantics.
|
||||||
@@ -1,904 +0,0 @@
|
|||||||
# Sequence Identity And Combination
|
|
||||||
|
|
||||||
Status: feature roadmap with OD-1 through OD-12 settled and ready for a staged
|
|
||||||
implementation plan. No open design decisions remain in this roadmap. This is
|
|
||||||
not an implementation plan or an implemented contract.
|
|
||||||
|
|
||||||
## Motivation
|
|
||||||
|
|
||||||
Allow configuration authors to assemble collections from independently authored
|
|
||||||
records, declare their identity in configuration, and progressively extend a
|
|
||||||
baseline collection through higher layers. Authors should also be able to
|
|
||||||
replace the accumulated collection or replace individual records by identity.
|
|
||||||
|
|
||||||
The motivating example is a prompt collection whose defaults supply some
|
|
||||||
records and whose application or installation layers add or supersede records.
|
|
||||||
Across tiers, authors need three operations: whole-sequence replacement,
|
|
||||||
duplicate-rejecting append, and whole-record replacement by identity. The latter
|
|
||||||
retains unmatched lower-tier records and adds higher-tier records with new
|
|
||||||
identities. Matching records are replaced in full. Sibling inheritance through
|
|
||||||
`extends` uses the same sequence operations and effective-identity defaults.
|
|
||||||
|
|
||||||
## Current Baseline
|
|
||||||
|
|
||||||
The [import contract](../config.md#explicit-imports) owns current selector,
|
|
||||||
destination, placement, and ordering behavior. An import with
|
|
||||||
`placement: sequence-item` places each selected source mapping as one item at
|
|
||||||
its effective destination. Item-local helper imports can contribute fields
|
|
||||||
before placement.
|
|
||||||
|
|
||||||
The application currently registers sequence identity through
|
|
||||||
`Definition.SequenceCompositions`, using one literal string field at each
|
|
||||||
registered mapping path. Within a strict tier, distinct identities append and
|
|
||||||
duplicate identities fail. Repeated unregistered sequences conflict. Across
|
|
||||||
precedence tiers, a higher sequence replaces the lower sequence wholesale.
|
|
||||||
The [composition contract](../config.md#strict-document-composition) owns these
|
|
||||||
rules; [layered loading](../config.md#layered-loading) owns tier precedence.
|
|
||||||
|
|
||||||
Current [sibling inheritance](../config.md#sibling-inheritance) recursively
|
|
||||||
overlays an inheriting mapping onto its resolved parent and replaces supplied
|
|
||||||
sequences wholesale. The proposed inheritance extension is settled in OD-7.
|
|
||||||
|
|
||||||
Source-backed underlays and `Definition.Defaults` have different lifecycle
|
|
||||||
roles. Underlays participate in source assembly; typed defaults are applied
|
|
||||||
later. Extending one does not automatically imply extending the other.
|
|
||||||
|
|
||||||
## Requested Capabilities
|
|
||||||
|
|
||||||
The proposed `configkit.sequences` block introduces a `key` declaration that
|
|
||||||
accepts either one field or a list of fields for a compound identity.
|
|
||||||
Fields are treated as a set; identity components may be strings, numbers,
|
|
||||||
booleans, or explicitly supplied null values.
|
|
||||||
Applications retain an optional interface for registering a collection
|
|
||||||
and imposing its identity definition. Configuration declarations must conform
|
|
||||||
to an application-owned constraint when one exists. The exact Go interface
|
|
||||||
remains to be designed.
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
configkit:
|
|
||||||
sequences:
|
|
||||||
- destination: [prompts]
|
|
||||||
key: [namespace, id]
|
|
||||||
mode: append
|
|
||||||
imports:
|
|
||||||
- dir: prompts
|
|
||||||
pattern: '*.yaml'
|
|
||||||
destination: [prompts]
|
|
||||||
placement: sequence-item
|
|
||||||
```
|
|
||||||
|
|
||||||
Each selected file would contribute a record such as:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
namespace: support
|
|
||||||
id: welcome
|
|
||||||
text: Welcome
|
|
||||||
```
|
|
||||||
|
|
||||||
Here `mode: append` illustrates retaining a lower-layer collection and adding
|
|
||||||
distinct records from the higher layer. The `mode` field expresses the operation;
|
|
||||||
OD-5 defines its names and effective-identity default. Identity and combination
|
|
||||||
controls belong to the collection declaration; import shorthand is deferred.
|
|
||||||
|
|
||||||
Letters below denote record identities; `B*` is a higher-tier record with the same
|
|
||||||
identity as `B` and a different payload. The outcomes and ordering shown are
|
|
||||||
settled below.
|
|
||||||
|
|
||||||
| Lower-tier sequence | Higher-tier sequence | Operation | Proposed result |
|
|
||||||
| --- | --- | --- | --- |
|
|
||||||
| `[A, B]` | `[C]` | `append` | `[A, B, C]` |
|
|
||||||
| `[A, B]` | `[C]` | `replace` | `[C]` |
|
|
||||||
| `[A, B]` | `[B*]` | `append` | Duplicate-identity error |
|
|
||||||
| `[A, B]` | `[B*]` | `replace` | `[B*]` |
|
|
||||||
| `[A, B]` | `[B*]` | `upsert` | `[A, B*]` |
|
|
||||||
| `[A, B]` | `[B*, C]` | `upsert` | `[A, B*, C]` |
|
|
||||||
|
|
||||||
Whole-sequence replacement discards the lower sequence. A repeated identity in the
|
|
||||||
retained combined sequence is an error under the proposed append semantics,
|
|
||||||
including when the records have identical or disjoint payloads.
|
|
||||||
|
|
||||||
## Settled Design Choices
|
|
||||||
|
|
||||||
### Collection Declarations
|
|
||||||
|
|
||||||
**OD-2: settled.**
|
|
||||||
|
|
||||||
Use a root `configkit.sequences` block containing a sequence of collection
|
|
||||||
declarations. Each entry identifies a `destination` and may declare `key`,
|
|
||||||
`mode`, or both. Application sequences remain ordinary YAML data. This declaration
|
|
||||||
form serves imported items, inline records, and sequences supplied by ordinary
|
|
||||||
mapping imports, including inline sequences in selected overlays.
|
|
||||||
|
|
||||||
`destination` uses the existing sequence of literal mapping keys. Resolve it
|
|
||||||
relative to the declaring source's placement: a fragment mounted under
|
|
||||||
`catalog` can declare `[prompts]` locally. A sequence-item source can declare a
|
|
||||||
nested collection within its own item without referring to an eventual numeric
|
|
||||||
index. Preserve those source and item-local scope boundaries.
|
|
||||||
OD-11 settles policy ownership when enclosing records are replaced or inherited.
|
|
||||||
|
|
||||||
Identity and mode can be supplied independently. Omitted identity can use an
|
|
||||||
application registration; an explicit matching declaration is accepted, and a
|
|
||||||
conflicting declaration is invalid. Whole-sequence replacement does not itself
|
|
||||||
require identity matching, but application-owned constraints still apply.
|
|
||||||
Identity-policy lifetime across tiers is settled in OD-3 below.
|
|
||||||
|
|
||||||
Mode describes the collection's combination instruction at source-precedence
|
|
||||||
and inheritance boundaries, as detailed in OD-7. An explicit mode persists at
|
|
||||||
the same applicable destination until a higher-tier declaration changes it.
|
|
||||||
Gather collection declarations before composing their relevant scope so their
|
|
||||||
effect does not depend on which contribution is assembled first. Identity agreement is settled
|
|
||||||
in OD-3; operation agreement and omission among peers are settled in OD-4.
|
|
||||||
OD-9 settles how operation controls survive source precedence for later inheritance.
|
|
||||||
|
|
||||||
Permit sequence declarations in selected overlays and make their use independent
|
|
||||||
of filesystem import enablement. This deliberately changes the current blanket
|
|
||||||
prohibition on overlay `configkit` declarations. Enabling imports in overlays
|
|
||||||
is not part of this decision. Consume sequence controls before application
|
|
||||||
decoding, as with other structural directives.
|
|
||||||
|
|
||||||
For example, an overlay can replace a complete record by identity:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
configkit:
|
|
||||||
sequences:
|
|
||||||
- destination: [prompts]
|
|
||||||
key: id
|
|
||||||
mode: upsert
|
|
||||||
|
|
||||||
prompts:
|
|
||||||
- id: foo
|
|
||||||
text: Revised definition
|
|
||||||
```
|
|
||||||
|
|
||||||
The explicit `mode: upsert` line can be omitted here when no different explicit
|
|
||||||
mode applies from a lower tier: `key: id` then supplies the identity for inferred
|
|
||||||
upsert. An overlay can instead select duplicate-rejecting append:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
configkit:
|
|
||||||
sequences:
|
|
||||||
- destination: [prompts]
|
|
||||||
key: id
|
|
||||||
mode: append
|
|
||||||
|
|
||||||
prompts:
|
|
||||||
- id: baz
|
|
||||||
text: Another prompt
|
|
||||||
```
|
|
||||||
|
|
||||||
Or it can explicitly select whole-sequence replacement:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
configkit:
|
|
||||||
sequences:
|
|
||||||
- destination: [prompts]
|
|
||||||
mode: replace
|
|
||||||
|
|
||||||
prompts:
|
|
||||||
- id: bar
|
|
||||||
text: Bar definition
|
|
||||||
- id: baz
|
|
||||||
text: Baz definition
|
|
||||||
```
|
|
||||||
|
|
||||||
Explicitly selecting the default operation is valid. With no applicable explicit
|
|
||||||
mode, removing the last example's `configkit` block selects replace only when no
|
|
||||||
effective identity is available; an identified collection instead selects upsert.
|
|
||||||
A retained explicit mode also affects omission. Explicit `mode: replace` overrides
|
|
||||||
either another retained mode or inferred upsert. OD-5 defines the selection rule
|
|
||||||
for `replace`, `append`, and `upsert`.
|
|
||||||
|
|
||||||
Start with this common collection declaration and defer `key` and `mode`
|
|
||||||
shorthand on imports. A single declaration form covers inline overlays and mixed
|
|
||||||
inline/imported collections without duplicate authoring surfaces. Local wrappers
|
|
||||||
around sequence values would require distinguishing application data from
|
|
||||||
control mappings; the existing reserved namespace keeps that distinction clear.
|
|
||||||
|
|
||||||
### Identity Definitions And Lifetime
|
|
||||||
|
|
||||||
**OD-3: settled.**
|
|
||||||
|
|
||||||
#### Field Sets
|
|
||||||
|
|
||||||
`key` accepts a string or a nonempty list of strings naming direct, literal
|
|
||||||
fields in each record. `key: id` and `key: [id]` are equivalent. A name such as
|
|
||||||
`metadata.id` denotes that literal field, not a nested lookup.
|
|
||||||
|
|
||||||
Treat the selected fields as a set: remove repeated names and disregard authored
|
|
||||||
field order when comparing definitions. For example, `[namespace, id]`,
|
|
||||||
`[id, namespace]`, and `[namespace, id, id]` define the same identity. An empty
|
|
||||||
list is invalid because it supplies no identity fields. These equivalence rules
|
|
||||||
also apply when comparing configuration declarations with application
|
|
||||||
registrations.
|
|
||||||
|
|
||||||
Use a consistent field order internally to construct compound identity tuples.
|
|
||||||
Preserve component types and tuple boundaries; concatenating unstructured values
|
|
||||||
must not cause otherwise distinct identities to collide.
|
|
||||||
|
|
||||||
#### Scalar Components And Equality
|
|
||||||
|
|
||||||
Each identified record must be a mapping, and every designated identity field
|
|
||||||
must be present. Components may be strings, numbers, booleans, or explicit null.
|
|
||||||
Missing fields and mapping or sequence components are invalid. A present null is
|
|
||||||
a valid component and is distinct from an absent field. Two single-field null
|
|
||||||
identities are equal; in a compound identity, all components must compare equal.
|
|
||||||
|
|
||||||
Strings compare exactly, with no trimming, case folding, or normalization.
|
|
||||||
Empty strings remain valid. Scalar categories remain distinct: the string
|
|
||||||
`"1"`, the number `1`, the boolean `true`, and null are different identities.
|
|
||||||
|
|
||||||
Numbers compare by mathematical value across integer, fractional, and exponent
|
|
||||||
spellings: `1`, `1.0`, and `1e0` are equal, as are `0` and `-0`. Use exact numeric
|
|
||||||
normalization for identity comparison without rounding through a floating-point
|
|
||||||
conversion. This comparison does not rewrite authored values or alter existing
|
|
||||||
typed decoding and projection semantics. The existing parser's rejection of
|
|
||||||
non-finite numeric forms remains unchanged.
|
|
||||||
|
|
||||||
#### Agreement And Introduction
|
|
||||||
|
|
||||||
Within a strict tier, identity declarations for the same collection must agree.
|
|
||||||
Equivalent field sets coalesce; different sets produce an error. Across tiers,
|
|
||||||
append and replacement by identity use one agreed definition for the records
|
|
||||||
being combined:
|
|
||||||
|
|
||||||
| Lower collection identity | Higher declaration | Result when combining records |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `id` | Omitted | Use `id` |
|
|
||||||
| `id` | `id`, `[id]`, or `[id, id]` | Accept the equivalent definition |
|
|
||||||
| `id` | `[namespace, id]` | Error: different identity definitions |
|
|
||||||
| None | `id` | Establish `id` and validate all participating records under it |
|
|
||||||
|
|
||||||
OD-10 makes applicable lower-tier YAML identity available during the higher
|
|
||||||
tier's strict composition, before this precedence combination occurs.
|
|
||||||
|
|
||||||
An application registration supplies identity when configuration omits it and
|
|
||||||
remains authoritative over explicit declarations. If neither the application nor
|
|
||||||
either participating collection supplies an identity, append and replacement by
|
|
||||||
identity fail; ConfigKit does not infer an `id` convention. Introducing identity
|
|
||||||
for a previously unkeyed lower sequence requires checking both lower and incoming
|
|
||||||
records, including uniqueness within each contribution.
|
|
||||||
|
|
||||||
#### Replacement And Policy Lifetime
|
|
||||||
|
|
||||||
YAML-owned identity follows the collection that survives an operation. OD-12
|
|
||||||
also permits declarations pending at absent or non-sequence destinations.
|
|
||||||
|
|
||||||
| Operation | Effective YAML-owned identity |
|
|
||||||
| --- | --- |
|
|
||||||
| Append | Retain or establish the agreed identity |
|
|
||||||
| Replacement by identity | Retain or establish the agreed identity |
|
|
||||||
| Whole-sequence replacement | Use the replacing tier's declaration and discard the lower collection's YAML-only identity |
|
|
||||||
| Replacement by null or another non-sequence value | Discard the lower YAML-only identity; retain any identity explicitly declared by the replacing tier under OD-12 |
|
|
||||||
|
|
||||||
Without an application constraint, whole-sequence replacement may introduce a
|
|
||||||
different identity, such as replacing an `id`-keyed collection with a `name`-keyed
|
|
||||||
collection. A whole-sequence replacement with no new identity declaration and no
|
|
||||||
application registration produces an unkeyed sequence. No separate identity
|
|
||||||
removal directive is needed.
|
|
||||||
|
|
||||||
Application registrations continue to constrain the replacement collection.
|
|
||||||
If the application requires `id`, declaring `name` is invalid under every mode.
|
|
||||||
The policy reset above applies when the collection value is actually replaced.
|
|
||||||
OD-6 defines the value effects of absence, null, and kind mismatches; OD-12
|
|
||||||
defines the corresponding declaration-only and identity-lifetime rules.
|
|
||||||
YAML-only identity declarations receive assembly-time enforcement, as settled
|
|
||||||
in OD-1 below.
|
|
||||||
|
|
||||||
### Combination Scope, Declarations, And Ordering
|
|
||||||
|
|
||||||
**OD-4: settled.**
|
|
||||||
|
|
||||||
#### One Operation Per Collection Per Tier
|
|
||||||
|
|
||||||
Resolve applicable collection policy, including available lower-tier identity
|
|
||||||
under OD-10, before strictly composing a tier's contributions. Then apply one
|
|
||||||
selected operation per collection against the accumulated lower sequence. A collection declaration
|
|
||||||
governs all contributions to that collection in its tier, including imported and
|
|
||||||
inline records; it does not govern only records in the declaring file.
|
|
||||||
|
|
||||||
Collection scope is the resolved destination within its composition scope.
|
|
||||||
Resolve relative destinations first and preserve separate item-local scopes for
|
|
||||||
collections inside different imported items. Apply the same combination
|
|
||||||
semantics to selected overlay contributions.
|
|
||||||
|
|
||||||
For example, with an agreed identity, higher-tier fragments contributing `[C]`
|
|
||||||
and `[D]` assemble `[C, D]`. Against lower `[A, B]`, append or replacement by
|
|
||||||
identity produces `[A, B, C, D]`; whole-sequence replacement produces `[C, D]`.
|
|
||||||
The tier operation does not sequentially replace peer source contributions.
|
|
||||||
|
|
||||||
#### Operation Agreement And Omission
|
|
||||||
|
|
||||||
Coalesce agreeing explicit modes for the same collection within a tier and
|
|
||||||
reject conflicting explicit modes. Omission expresses no preference; apply the
|
|
||||||
effective-identity default only after reconciling declarations and checking for
|
|
||||||
an applicable explicit mode retained from a lower tier.
|
|
||||||
|
|
||||||
| Modes declared within one tier | Effective operation |
|
|
||||||
| --- | --- |
|
|
||||||
| `append` and `append` | `append` |
|
|
||||||
| `append` and omitted | `append` |
|
|
||||||
| `replace` and omitted | `replace` |
|
|
||||||
| `append` and `replace` | Error |
|
|
||||||
| All omitted | Retain an applicable lower explicit mode; otherwise infer from effective identity |
|
|
||||||
|
|
||||||
Explicitly selecting the default is a real instruction and can conflict with
|
|
||||||
another explicit selection. Different modes in different tiers are valid; the
|
|
||||||
higher tier's explicit mode replaces the lower mode, while omission retains it.
|
|
||||||
Neither a root declaration nor a later declaration wins over conflicting peers.
|
|
||||||
Diagnostics identify the collection
|
|
||||||
destination and conflicting declaration locations. OD-5 defines mode names and
|
|
||||||
the default-selection rule.
|
|
||||||
|
|
||||||
#### Strict Duplicate Handling
|
|
||||||
|
|
||||||
Duplicate identities among peer contributions or within one contributed list
|
|
||||||
remain strict-composition errors under all three modes. For example, peer
|
|
||||||
fragments contributing `[B]` and `[B*]` conflict even under replacement by
|
|
||||||
identity: neither has higher precedence. Lower `[B]` and higher `[B*]` can
|
|
||||||
instead produce `[B*]` at a precedence boundary. Whole-sequence replacement
|
|
||||||
does not permit duplicate identities within an identified higher-tier sequence.
|
|
||||||
|
|
||||||
#### Ordering
|
|
||||||
|
|
||||||
Retain existing dependency-ready source scheduling within each tier and authored
|
|
||||||
item order within each list. Gathering collection declarations does not change
|
|
||||||
assembly order, and identity values do not sort records.
|
|
||||||
|
|
||||||
Across tiers:
|
|
||||||
|
|
||||||
- Append retains lower records in order, followed by higher records in assembled
|
|
||||||
order.
|
|
||||||
- Whole-sequence replacement retains only the higher sequence, in its assembled
|
|
||||||
order.
|
|
||||||
- Replacement by identity replaces matched records in their existing lower
|
|
||||||
positions and appends new identities in higher-tier order.
|
|
||||||
|
|
||||||
For example, lower `[A, B, C]` and higher `[C*, A*, D]` produce
|
|
||||||
`[A*, B, C*, D]` under replacement by identity. Updating a record's contents
|
|
||||||
preserves its position; whole-sequence replacement gives the higher tier control
|
|
||||||
of the complete sequence order.
|
|
||||||
|
|
||||||
OD-5 settles default selection and operation naming. OD-6 settles absent
|
|
||||||
contributions, empty sequences, null, and kind mismatches.
|
|
||||||
|
|
||||||
### Mode Names And Effective-Identity Defaults
|
|
||||||
|
|
||||||
**OD-5: settled.**
|
|
||||||
|
|
||||||
Use the mode names `replace`, `append`, and `upsert`, with one spelling per
|
|
||||||
operation:
|
|
||||||
|
|
||||||
| Mode | Behavior at source-precedence and inheritance boundaries |
|
|
||||||
| --- | --- |
|
|
||||||
| `replace` | Replace the complete lower sequence with the assembled higher sequence |
|
|
||||||
| `append` | Retain lower records and append higher records; duplicate identities fail |
|
|
||||||
| `upsert` | Replace matching records in full, retain unmatched lower records, and append new identities |
|
|
||||||
|
|
||||||
`upsert` preserves the existing positions of matching lower records and adds new
|
|
||||||
identities in higher-tier order, as settled in OD-4. It does not recursively merge
|
|
||||||
fields within matching records.
|
|
||||||
|
|
||||||
Select the operation at each collection's precedence boundary in this order:
|
|
||||||
|
|
||||||
1. Reconcile explicit modes within the current tier under OD-4; conflicting
|
|
||||||
explicit modes fail.
|
|
||||||
2. If the current tier supplies an explicit mode, use it in place of a lower mode.
|
|
||||||
3. Otherwise, retain an applicable explicit mode from a lower tier.
|
|
||||||
4. If no explicit mode applies, select `upsert` when effective identity is available.
|
|
||||||
5. Otherwise, select `replace`.
|
|
||||||
|
|
||||||
Effective identity may come from an application registration, a current-tier
|
|
||||||
declaration, or the lower collection, subject to OD-3's agreement rules.
|
|
||||||
Selection depends on identity availability, not on whether the current tier
|
|
||||||
explicitly repeats a `key` declaration. With no applicable explicit mode, a plain
|
|
||||||
overlay sequence can therefore default to `upsert` when identity was established
|
|
||||||
elsewhere. New sequence values do not reset an existing explicit mode. Inferred
|
|
||||||
defaults do not become persistent explicit instructions: when no explicit mode
|
|
||||||
applies, determine the default from the identity available at that boundary.
|
|
||||||
|
|
||||||
Determine the inferred mode using the available identity before applying the
|
|
||||||
operation. `replace`, including a retained explicit mode, follows OD-3: discard
|
|
||||||
the lower collection's YAML-only identity with the lower sequence, use the
|
|
||||||
replacing tier's identity declaration if supplied, and preserve application-owned
|
|
||||||
constraints. Selecting
|
|
||||||
`append` or `upsert` explicitly still requires an effective identity.
|
|
||||||
|
|
||||||
This default governs combination between tiers and during sibling inheritance
|
|
||||||
under OD-7. Within a tier, identified
|
|
||||||
sequences append distinct records and reject duplicate identities regardless of
|
|
||||||
mode. Unidentified sequences retain their existing strict-composition conflict
|
|
||||||
behavior; this decision does not introduce unkeyed concatenation.
|
|
||||||
|
|
||||||
### Whole-Record Replacement By Identity
|
|
||||||
|
|
||||||
The `upsert` operation replaces each matching lower record with the
|
|
||||||
complete higher record, retains unmatched lower records, and adds new identities.
|
|
||||||
For example, given these lower and higher records:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
# Lower record
|
|
||||||
id: B
|
|
||||||
text: Original
|
|
||||||
temperature: 0.5
|
|
||||||
```
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
# Higher record
|
|
||||||
id: B
|
|
||||||
text: Revised
|
|
||||||
```
|
|
||||||
|
|
||||||
The resulting record at this precedence boundary contains `id` and `text`.
|
|
||||||
The lower record's `temperature` is discarded. Subsequent defaulting or other
|
|
||||||
lifecycle operations retain their separately defined semantics. Recursive
|
|
||||||
field-by-field overlay of matching records is outside this selected operation.
|
|
||||||
|
|
||||||
### Selected Overlay Control
|
|
||||||
|
|
||||||
Selected overlay authors must be able to choose any of the three operations
|
|
||||||
for a sequence supplied by the overlay, including records authored inline.
|
|
||||||
The operation applies against the existing effective sequence at that path.
|
|
||||||
For an existing sequence `[foo, bar]`:
|
|
||||||
|
|
||||||
| Overlay contribution | Selected operation | Result |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `[foo*]` | Replacement by identity | `[foo*, bar]` |
|
|
||||||
| `[baz]` | Append | `[foo, bar, baz]` |
|
|
||||||
| `[bar, baz]` | Whole-sequence replacement | `[bar, baz]` |
|
|
||||||
|
|
||||||
Here `foo*` has the same identity as `foo` and supplies its complete replacement
|
|
||||||
record. Append retains its duplicate-identity error semantics. The retained
|
|
||||||
records, replacement behavior, and ordering shown are settled, including the
|
|
||||||
OD-4 rule that matched records retain their lower positions.
|
|
||||||
|
|
||||||
The settled OD-2 collection declaration supplies the overlay authoring surface.
|
|
||||||
OD-5 supplies operation names and effective-identity default selection. The current
|
|
||||||
[selected-overlay contract](../config.md#selected-overlays) remains the baseline
|
|
||||||
until the new controls are implemented.
|
|
||||||
|
|
||||||
### Optional Application-Owned Identity Constraints
|
|
||||||
|
|
||||||
Retain an interface through which applications may register a collection and
|
|
||||||
impose its unique-key definition. If the application requires `key: id`, a
|
|
||||||
configuration declaration of `key: [id, name]` is invalid: the compound key
|
|
||||||
would permit repeated `id` values with different names. The application identity
|
|
||||||
constraint applies whichever cross-tier operation configuration selects.
|
|
||||||
|
|
||||||
This is an authoritative constraint, distinct from an application default that
|
|
||||||
only fills an omission. Configuration can define identity for collections
|
|
||||||
without an application-owned constraint. Omitted configuration identity uses a
|
|
||||||
registered definition when present, and an explicit matching declaration is
|
|
||||||
accepted, as settled in OD-2. OD-1 below distinguishes the final-result guarantee
|
|
||||||
for application-owned constraints from assembly-time enforcement for YAML-only
|
|
||||||
declarations.
|
|
||||||
|
|
||||||
### Final-Result Guarantee For Application Registrations
|
|
||||||
|
|
||||||
**OD-1: settled, including the YAML-only follow-up below.**
|
|
||||||
|
|
||||||
> On successful loading, the final effective collections covered by application
|
|
||||||
> registrations satisfy their declared identity constraints.
|
|
||||||
|
|
||||||
Retain assembly-time identity checks and validate the final effective
|
|
||||||
collections before publication. This covers the loaded state after defaults,
|
|
||||||
inheritance, overrides, and application resolution have taken effect. Those
|
|
||||||
operations retain their separately defined combination semantics. The guarantee
|
|
||||||
does not require every intermediate callback state to satisfy the constraint
|
|
||||||
and does not extend to caller mutations after loading returns.
|
|
||||||
|
|
||||||
Keep the promise limited to the declared identity constraints. It does not by
|
|
||||||
itself require registered paths to exist or be non-null. Identity validation checks
|
|
||||||
actual sequences; the application schema and validation determine whether absence,
|
|
||||||
null, or another shape is acceptable, as settled in OD-6. Final validation examines
|
|
||||||
effective typed values, while assembly-time checks retain their role in
|
|
||||||
validating authored contributions. A final scalar value alone does not prove
|
|
||||||
that the field was explicitly supplied in a source.
|
|
||||||
|
|
||||||
The existing canonical final-value representation and shared publication path
|
|
||||||
provide the implementation foundation. Diagnostics should identify the affected
|
|
||||||
collection and item paths, using available origin information without exposing
|
|
||||||
identity values. The guarantee requires neither a new mutation-history system
|
|
||||||
nor exact causal reconstruction for records supplied by defaults or callbacks.
|
|
||||||
|
|
||||||
### Assembly-Time Enforcement For YAML-Only Identities
|
|
||||||
|
|
||||||
Enforce identities declared only in YAML during structural composition,
|
|
||||||
precedence assembly, and identified collection operations during sibling
|
|
||||||
inheritance. They do not carry a final-result guarantee after defaulting, CLI
|
|
||||||
overrides, or application resolution. Those operations may produce a collection
|
|
||||||
that no longer satisfies a YAML-only identity, including duplicate identities.
|
|
||||||
OD-7 defines the combination and validation boundaries.
|
|
||||||
|
|
||||||
Applications requiring a final identity constraint register it through the
|
|
||||||
application-owned interface. This keeps YAML declarations aligned with the
|
|
||||||
structural operations they control and avoids extending their policy lifetime
|
|
||||||
through later replacements, callbacks, and nested-record changes solely for
|
|
||||||
final validation.
|
|
||||||
|
|
||||||
### Absence, Empty Sequences, Null, And Kind Mismatches
|
|
||||||
|
|
||||||
**OD-6: settled.**
|
|
||||||
|
|
||||||
At a collection's precedence boundary:
|
|
||||||
|
|
||||||
| Higher-tier contribution | `replace` | `append` / `upsert` |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| Absent | Leave the lower value unchanged | Leave the lower value unchanged |
|
|
||||||
| Explicit `[]` | Clear the sequence | Contribute no records |
|
|
||||||
| Explicit null or another non-sequence value | Ordinary replacement, subject to application schema | Error: the operation requires a sequence |
|
|
||||||
|
|
||||||
A declaration alone does not clear a collection. OD-12 defines how a key-only
|
|
||||||
declaration validates a retained sequence or remains pending when no sequence
|
|
||||||
exists. An optional import selecting nothing contributes nothing; it does not
|
|
||||||
erase records supplied elsewhere.
|
|
||||||
An absent lower collection can be initialized by an incoming sequence under
|
|
||||||
any mode, subject to identity requirements. An explicitly null or other
|
|
||||||
non-sequence lower value is distinct from absence: append and upsert against it
|
|
||||||
fail, while replacement remains available. An empty incoming sequence does not
|
|
||||||
bypass these kind checks or the operation's identity requirements.
|
|
||||||
|
|
||||||
With effective identity available and no applicable explicit mode, omission
|
|
||||||
selects upsert. Consequently, `prompts: []` alone does not clear such a collection.
|
|
||||||
Clearing it requires replace, selected explicitly in the current tier or retained
|
|
||||||
from a lower declaration. Replacing an identified collection with null likewise
|
|
||||||
requires replacement and an application schema that permits null.
|
|
||||||
|
|
||||||
An application identity registration does not itself require the collection path
|
|
||||||
to exist or be non-null, and does not independently reject non-sequence values.
|
|
||||||
Identity validation checks actual sequences; application schema and validation
|
|
||||||
own the permitted shape. Null as a collection is distinct from a present null
|
|
||||||
identity component inside a record, which is valid under OD-3.
|
|
||||||
|
|
||||||
### Lifecycle Coverage And Sibling Inheritance
|
|
||||||
|
|
||||||
**OD-7: settled.**
|
|
||||||
|
|
||||||
Apply the sequence operations during source composition and precedence assembly,
|
|
||||||
selected overlays, and sibling inheritance through `extends`. Preserve the
|
|
||||||
existing combination semantics of `Definition.Defaults`, CLI overrides, and
|
|
||||||
application resolution callbacks.
|
|
||||||
|
|
||||||
#### Inheritance Operations And Controls
|
|
||||||
|
|
||||||
Complete source-tier assembly before resolving inheritance, retaining the
|
|
||||||
existing dependency-ready inheritance algorithm and scope rules. At each
|
|
||||||
inheritance combination, the resolved parent's collection is the lower value
|
|
||||||
and the inheriting mapping's local collection is the higher value. Use the same
|
|
||||||
operation selection as source precedence: the effective explicit mode wins,
|
|
||||||
including one retained under OD-9; otherwise use `upsert` with effective identity
|
|
||||||
and `replace` without it. There is no separate
|
|
||||||
default-append rule for inheritance.
|
|
||||||
|
|
||||||
Use the inheriting destination collection's controls. Its selected mode governs
|
|
||||||
both source-precedence and inheritance combinations at that destination, without
|
|
||||||
separate per-phase mode settings. A mode governing the parent's source collection
|
|
||||||
does not automatically become an instruction at the inheriting destination merely
|
|
||||||
because records are inherited. Retain the relevant controls until inheritance
|
|
||||||
uses them, then keep them out of application decoding. Identity agreement,
|
|
||||||
application constraints, and replacement lifetime follow OD-3.
|
|
||||||
|
|
||||||
For example:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
configkit:
|
|
||||||
sequences:
|
|
||||||
- destination: [parent, items]
|
|
||||||
key: id
|
|
||||||
|
|
||||||
child:
|
|
||||||
items:
|
|
||||||
- id: foo
|
|
||||||
value: A
|
|
||||||
- id: bar
|
|
||||||
value: B
|
|
||||||
parent:
|
|
||||||
extends: child
|
|
||||||
items:
|
|
||||||
- id: baz
|
|
||||||
value: C
|
|
||||||
```
|
|
||||||
|
|
||||||
Despite the mapping names, `parent` inherits from `child` in this example.
|
|
||||||
The effective identity at `[parent, items]` selects `upsert`, yielding records
|
|
||||||
`[foo, bar, baz]`. If the local contribution instead supplies a revised `foo`,
|
|
||||||
the result is `[foo*, bar]`: replace the complete record in its existing position.
|
|
||||||
Adding `mode: append` would reject that repeated identity; `mode: replace` would
|
|
||||||
retain only the local sequence.
|
|
||||||
|
|
||||||
Mappings continue to combine recursively. Identified sequence records remain
|
|
||||||
whole-record replacements, not recursive field merges. Identity supplies the
|
|
||||||
matching key; ConfigKit does not infer identity from a field named `id` or from
|
|
||||||
record position. Append and upsert continue to require effective identity, and
|
|
||||||
this extension does not introduce unkeyed sequence concatenation.
|
|
||||||
|
|
||||||
Apply OD-4 ordering and OD-6 presence and kind rules at inheritance boundaries.
|
|
||||||
An omitted local sequence retains the inherited sequence. An explicit empty
|
|
||||||
sequence clears it under replace, while append and upsert contribute no records.
|
|
||||||
|
|
||||||
#### Other Lifecycle Stages
|
|
||||||
|
|
||||||
Source-backed underlays participate in the new combination model, including
|
|
||||||
application-supplied mapping sources. Typed `Definition.Defaults` remain the
|
|
||||||
initial decoded state: supplied sequences replace default sequences. Source
|
|
||||||
`mode: append` does not append to a typed default list.
|
|
||||||
|
|
||||||
CLI overrides retain explicit replacement for supplied sequences. OD-8 moves
|
|
||||||
their application after inheritance and typed defaulting, before application
|
|
||||||
resolution, secret lookup, and final validation.
|
|
||||||
Application resolution retains its existing assignment and recorded-transformation
|
|
||||||
semantics; ConfigKit does not automatically combine a callback's assigned sequence
|
|
||||||
with its previous value. Application-registered identities still receive final
|
|
||||||
validation under OD-1, independently of how these stages produce their values.
|
|
||||||
|
|
||||||
#### Validation Timing
|
|
||||||
|
|
||||||
Allow item-local helper imports to complete, and consume applicable non-main
|
|
||||||
source-version headers, before checking the application payload's identity for
|
|
||||||
collection assembly. A helper may supply an identity field; a consumed header
|
|
||||||
cannot supply a component after removal. Validate each identified contribution
|
|
||||||
before combination and uniqueness in the resulting identified collection,
|
|
||||||
including inheritance operations. An upsert match across a precedence boundary
|
|
||||||
is a replacement, not a duplicate error; duplicates within a contribution remain
|
|
||||||
errors.
|
|
||||||
|
|
||||||
Later inheritance, typed defaults, or callbacks do not repair an identity missing
|
|
||||||
at an earlier assembly boundary. Final application-registered validation examines
|
|
||||||
the final effective typed collection, while YAML-only enforcement remains limited
|
|
||||||
to the structural operations described above.
|
|
||||||
|
|
||||||
### CLI Precedence And Application Resolution
|
|
||||||
|
|
||||||
**OD-8: settled.**
|
|
||||||
|
|
||||||
CLI overrides are the highest-precedence input to application resolution. Use
|
|
||||||
this lifecycle order:
|
|
||||||
|
|
||||||
1. Assemble source tiers and selected overlays.
|
|
||||||
2. Resolve sibling inheritance, then apply typed defaults and decode the assembled
|
|
||||||
configuration.
|
|
||||||
3. Apply explicit CLI overrides to that configuration and check supplied values
|
|
||||||
against the application schema.
|
|
||||||
4. Run application resolution against the overridden configuration.
|
|
||||||
5. Resolve effective secret references, validate the final configuration including
|
|
||||||
application-registered identities, and publish it.
|
|
||||||
|
|
||||||
Sequence values supplied by the CLI replace whole sequences, including sequences
|
|
||||||
inside supplied mappings. Mappings retain ordinary recursive override behavior;
|
|
||||||
empty sequences and permitted nulls retain their explicit replacement meaning.
|
|
||||||
Configuration-selected append or upsert does not reinterpret CLI assignments.
|
|
||||||
Inheritance and defaulting do not run again after CLI application. For inherited
|
|
||||||
and assembled `[A, B]` overridden by CLI `[C]`, application resolution receives
|
|
||||||
`[C]`; inheritance cannot reintroduce `[A, B]` afterward.
|
|
||||||
|
|
||||||
Application resolution sees CLI values and their explicit-presence and source
|
|
||||||
metadata. Existing `Resolution.Default` behavior preserves explicit CLI values,
|
|
||||||
while normalization and recorded application transformations may still change
|
|
||||||
them. CLI input is authoritative over sources, inheritance, and typed defaults;
|
|
||||||
it is not immutable during application-owned resolution. For example, a derived
|
|
||||||
URL can use a CLI-supplied hostname, while an explicitly supplied CLI URL can be
|
|
||||||
preserved by an absence-based default. Retain existing recording and provenance
|
|
||||||
requirements for those transformations.
|
|
||||||
|
|
||||||
Secret lookup uses the effective references after both overrides and resolution.
|
|
||||||
Final validation remains after these stages; neither CLI replacement nor
|
|
||||||
application resolution bypasses identity constraints or semantic validation.
|
|
||||||
|
|
||||||
Reject structural directives such as `extends` and `configkit` in CLI override
|
|
||||||
paths or supplied values. CLI overrides supply application values after structural
|
|
||||||
processing; they do not initiate a second inheritance pass. This deliberately
|
|
||||||
changes the current ability to supply or cancel `extends` through CLI input.
|
|
||||||
CLI changes cannot prevent an earlier inheritance failure or change which
|
|
||||||
relationships that completed pass resolved. Likewise, because typed decoding
|
|
||||||
precedes CLI application, an invalid source value that fails decoding cannot be
|
|
||||||
repaired by a CLI replacement. For example, a string supplied for an integer
|
|
||||||
field fails before a valid integer override can apply. This differs from the
|
|
||||||
current override-before-decoding behavior. Semantic validation remains after
|
|
||||||
overrides and application resolution, as specified above.
|
|
||||||
|
|
||||||
This ordering supersedes the current CLI-before-inheritance lifecycle described
|
|
||||||
in [ADR 0009](../adr/0009-sibling-inheritance.md). It avoids special replacement
|
|
||||||
markers carried through inheritance. Current contracts remain the implemented
|
|
||||||
baseline until the architectural and implementation work lands.
|
|
||||||
|
|
||||||
### Persistent Operation Policy
|
|
||||||
|
|
||||||
**OD-9: settled; this revises the earlier tier-local mode rule in OD-4 and OD-5.**
|
|
||||||
|
|
||||||
An explicit mode is collection policy that persists at its applicable destination
|
|
||||||
until a higher-priority declaration changes it. Reconcile peers strictly within
|
|
||||||
each tier, then let a higher explicit mode replace a lower one. Omission retains
|
|
||||||
an applicable explicit mode regardless of whether the higher tier supplies new
|
|
||||||
sequence values. Source precedence and subsequent inheritance use the same
|
|
||||||
effective operation policy.
|
|
||||||
|
|
||||||
| Lower explicit mode | Higher declaration | Effective mode |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `append` | Mode omitted | `append` |
|
|
||||||
| `append` | `upsert` | `upsert` |
|
|
||||||
| `upsert` | `replace` | `replace` |
|
|
||||||
| None | Mode omitted, effective identity available | Inferred `upsert` |
|
|
||||||
| None | Mode omitted, no effective identity | Inferred `replace` |
|
|
||||||
|
|
||||||
An unrelated overlay leaves the policy intact. A mode-only declaration may change
|
|
||||||
the policy without supplying records; it neither clears the collection nor
|
|
||||||
retroactively recombines earlier tiers. A later source contribution or inheritance
|
|
||||||
operation uses the resulting policy. Supplying values alone does not reset it.
|
|
||||||
|
|
||||||
Only explicit modes persist. Inferred defaults are recomputed from effective
|
|
||||||
identity when needed. Thus explicitly selecting replace for an identified
|
|
||||||
collection continues to select replace in subsequent tiers until another mode is
|
|
||||||
declared; an author wanting upsert afterward must select it explicitly. Identity
|
|
||||||
availability and constraints still follow OD-3, including YAML-only identity
|
|
||||||
removal on whole-sequence replacement. Persisting a mode does not preserve an
|
|
||||||
identity that OD-3 discards.
|
|
||||||
|
|
||||||
This persistence is scoped to the applicable collection. It does not transfer a
|
|
||||||
parent mapping's mode automatically to another destination through `extends` or
|
|
||||||
resurrect policies owned by discarded records. OD-11 addresses nested policy
|
|
||||||
ownership and transfer, including the distinction between surviving records and
|
|
||||||
inheritance copies. CLI overrides remain outside sequence mode selection
|
|
||||||
under OD-8. Strict peer composition and all ordering rules in OD-4 are unchanged.
|
|
||||||
|
|
||||||
### Lower-Tier Identity During Strict Composition
|
|
||||||
|
|
||||||
**OD-10: settled.**
|
|
||||||
|
|
||||||
Allow identity established in accumulated lower tiers to inform strict composition
|
|
||||||
of a higher tier. Resolve applicable collection identity before combining that
|
|
||||||
tier's peer contributions, using application registrations, available lower-tier
|
|
||||||
identity, and declarations gathered within the higher tier's relevant scope.
|
|
||||||
Authors need not repeat a baseline identity declaration solely because the higher
|
|
||||||
tier contributes the collection through multiple fragments.
|
|
||||||
|
|
||||||
For example, an underlay declares `key: id` for `prompts`; two main-tier fragments
|
|
||||||
contribute records `[foo]` and `[bar]` without redeclaring a key. The inherited
|
|
||||||
identity enables those fragments to compose as `[foo, bar]` before the main tier
|
|
||||||
combines with the accumulated lower collection. If both fragments instead
|
|
||||||
contribute `foo`, strict composition fails. A selected upsert mode never grants
|
|
||||||
precedence to peers within the tier.
|
|
||||||
|
|
||||||
This does not turn lower YAML identity into an application-owned constraint.
|
|
||||||
Determine applicability using OD-3's agreement and replacement rules: append and
|
|
||||||
upsert require agreement, while whole-sequence replacement may change or discard
|
|
||||||
YAML-only identity. A replacing collection must not be constrained by an obsolete
|
|
||||||
lower identity merely because policy resolution precedes strict composition.
|
|
||||||
Application registrations remain authoritative. Under OD-11, a replacement
|
|
||||||
record also cannot borrow item-local identity owned by the discarded lower
|
|
||||||
record, even if both records have the same outer identity or numeric position.
|
|
||||||
Higher-tier declarations do not retroactively repair invalid lower-tier
|
|
||||||
composition.
|
|
||||||
|
|
||||||
Separate source acquisition from policy-aware composition, or otherwise revise
|
|
||||||
assembly sequencing, so accumulated lower policy is available before higher-tier
|
|
||||||
strict composition. The current main-before-underlay composition order is not a
|
|
||||||
constraint on this design. Preserve the distinction between source acquisition
|
|
||||||
order, strict within-tier dependency-ready scheduling, and precedence order.
|
|
||||||
The implementation plan must account explicitly for source records, callbacks,
|
|
||||||
main-version checks, and failure diagnostics when changing the handoffs. Retain
|
|
||||||
version ownership and validation semantics; do not use higher-tier replacement
|
|
||||||
to hide lower-tier structural failures.
|
|
||||||
|
|
||||||
### Nested Policy Ownership And Transfer
|
|
||||||
|
|
||||||
**OD-11: settled.**
|
|
||||||
|
|
||||||
Attach item-local collection policies to their owning contributions. Policies
|
|
||||||
follow surviving records independently of numeric indexes. Whole-record upsert
|
|
||||||
replacement discards the old record's local policies along with its fields;
|
|
||||||
whole-sequence replacement likewise discards policies owned by removed records.
|
|
||||||
A replacement record supplies its own local declarations. Sharing an outer
|
|
||||||
identity or occupying the same final index does not transfer the discarded
|
|
||||||
record's policies to its replacement.
|
|
||||||
|
|
||||||
For example, a source imported as one record into `prompts` can declare:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
configkit:
|
|
||||||
sequences:
|
|
||||||
- destination: [steps]
|
|
||||||
key: id
|
|
||||||
mode: append
|
|
||||||
id: foo
|
|
||||||
steps:
|
|
||||||
- id: prepare
|
|
||||||
```
|
|
||||||
|
|
||||||
The declaration governs `steps` inside that particular `foo` record. Retaining
|
|
||||||
`foo` retains its local policy, including its explicit mode under OD-9. Upserting
|
|
||||||
a new `foo` discards the old `steps` and its declaration; the replacement does not
|
|
||||||
inherit either merely because upsert retains the outer record's position.
|
|
||||||
|
|
||||||
Distinguish inherited collection identity from operation instructions. YAML
|
|
||||||
identity follows a collection copied through sibling inheritance, subject to
|
|
||||||
OD-3's agreement and replacement rules. The inheriting destination selects its
|
|
||||||
operation under OD-7 and OD-9; the source collection's mode does not automatically
|
|
||||||
become an instruction at the new destination.
|
|
||||||
|
|
||||||
For example, a source mapping's `steps` collection has `key: id` and
|
|
||||||
`mode: append`. An inheriting mapping supplies local `steps` but no applicable
|
|
||||||
explicit mode. The copied identity selects upsert at the inheriting destination;
|
|
||||||
the source's append mode remains applicable at its own destination. An explicit
|
|
||||||
append declaration at the inheriting destination instead selects append there.
|
|
||||||
A destination declaration of the same key agrees; a different key conflicts when
|
|
||||||
combining records, while explicit replacement can establish a new YAML identity
|
|
||||||
under OD-3.
|
|
||||||
|
|
||||||
Application registrations remain constraints at their registered destinations;
|
|
||||||
copying a collection does not create another application registration. Policies
|
|
||||||
that independently apply at the destination are not discarded merely because
|
|
||||||
an individual record is replaced. Ownership determines which declarations are
|
|
||||||
removed. This preserves OD-9's persistence rule for applicable explicit modes
|
|
||||||
without retaining declarations owned by discarded records. The implementation
|
|
||||||
plan should choose the bookkeeping needed to preserve these distinctions.
|
|
||||||
|
|
||||||
### Identity Declarations Without Sequence Values
|
|
||||||
|
|
||||||
**OD-12: settled.**
|
|
||||||
|
|
||||||
Allow identity declarations without accompanying records. Apply a declaration to
|
|
||||||
an existing retained sequence immediately, checking identity agreement and
|
|
||||||
validating its records. Do not recombine earlier tiers. For example, if a lower
|
|
||||||
tier supplies an unkeyed `prompts` sequence and a higher tier only declares
|
|
||||||
`destination: [prompts], key: id`, the retained records must satisfy that identity
|
|
||||||
at this boundary.
|
|
||||||
|
|
||||||
If the collection is absent, retain the declaration in its owning scope for later
|
|
||||||
contributions without requiring the collection to exist. An otherwise empty
|
|
||||||
underlay can therefore establish identity for main-tier fragments under OD-10.
|
|
||||||
Item-local declarations remain subject to OD-11's ownership rules.
|
|
||||||
|
|
||||||
Merely declaring replace without supplying a replacement value does not authorize
|
|
||||||
changing a retained collection's identity. Actual replacement of the collection
|
|
||||||
by null or another non-sequence value discards the lower YAML-only identity,
|
|
||||||
retains any identity explicitly declared by the replacing tier, and preserves
|
|
||||||
application constraints. This extends OD-3's replacement rule to non-sequence
|
|
||||||
values.
|
|
||||||
|
|
||||||
An explicit identity declaration can remain pending at an absent or non-sequence
|
|
||||||
destination without independently imposing a shape requirement. OD-6 still
|
|
||||||
governs whether an attempted operation on a non-sequence value fails. These rules
|
|
||||||
allow baseline policy declarations while keeping identity removal tied to actual
|
|
||||||
replacement. Explicit mode persistence remains governed by OD-9.
|
|
||||||
|
|
||||||
## Architectural Direction
|
|
||||||
|
|
||||||
The settled collection declaration gives imported and inline records at the same
|
|
||||||
destination a shared identity policy. Declarations are gathered before composing
|
|
||||||
the relevant scope.
|
|
||||||
|
|
||||||
OD-3 settles identity agreement and lifetime; OD-4 settles tier-wide operation
|
|
||||||
selection, strict peer composition, and ordering at precedence boundaries.
|
|
||||||
OD-10 requires resolving applicable lower-tier policy before higher-tier strict
|
|
||||||
composition, independently of source acquisition order.
|
|
||||||
|
|
||||||
This requires collection policy to survive import expansion through precedence
|
|
||||||
assembly and the applicable inheritance operations. Correctness includes final
|
|
||||||
values, projected item paths, source destinations, provenance, history,
|
|
||||||
sensitivity, and passive resource contexts.
|
|
||||||
Appending changes final indexes; replacing changes which contributions survive.
|
|
||||||
The implementation must account for both through the existing loading lifecycle.
|
|
||||||
|
|
||||||
## Scope And Decision Ownership
|
|
||||||
|
|
||||||
This is narrower than the deferred
|
|
||||||
[identified-record sequences proposal](future.md#1-identified-record-sequences),
|
|
||||||
which also considers recursive merging of matching records and record-specific
|
|
||||||
inheritance. Those capabilities remain separate candidates. Compound keys,
|
|
||||||
duplicate-rejecting append, and whole-record replacement by identity do not
|
|
||||||
require adopting them.
|
|
||||||
|
|
||||||
The proposal would extend identity declaration ownership and revise the
|
|
||||||
cross-tier replacement decision in
|
|
||||||
[ADR 0014](../adr/0014-sequence-placement-and-composition.md), as well as sequence
|
|
||||||
replacement during inheritance in
|
|
||||||
[ADR 0009](../adr/0009-sibling-inheritance.md), including its CLI-before-inheritance
|
|
||||||
lifecycle ordering. Record
|
|
||||||
the architectural change in a superseding decision and reconcile affected
|
|
||||||
architecture policy before implementation. The current ADR and contracts remain
|
|
||||||
authoritative until that decision is made. This draft neither changes them nor
|
|
||||||
selects migration machinery.
|
|
||||||
|
|
||||||
The settled roadmap is ready for a staged implementation plan.
|
|
||||||
The plan should cover ordinary and layered loading, item-local scopes, source-version
|
|
||||||
processing, identity validation, precedence, inheritance, and inspection together.
|
|
||||||
Public API design and test sequencing belong to that later plan.
|
|
||||||
Reference in New Issue
Block a user