Files

Eric Rakestraw a5f0b92ee9 Add dynamic identity constraints to the sequence roadmap

2026-10-23 01:56:00 +00:00

48 KiB

Raw Blame History

Sequence Identity And Combination Roadmap

Status: feature roadmap with dynamic application identity constraints included. The remaining registration API decision is recorded under Open Decisions. This is the targeted end state, not an implemented contract. The subsequent plan will specify execution order, exact Go names and signatures, internal bookkeeping, and validation stages.

Purpose And Scope

Allow configuration authors to assemble collections from independently authored records, declare identity in configuration, and progressively extend a baseline collection through higher layers. Authors can replace the entire collection, append distinct records, or replace complete records by identity while retaining unmatched records. Sibling inheritance uses the same sequence operations and identity-based defaults. Within an identified sequence, a record can explicitly inherit settings from another record in that sequence while keeping its own identity.

The motivating use case is a prompt catalog whose source-backed baseline supplies records and whose application or installation layers add or supersede records. Applications can optionally impose identity constraints that configuration must respect and that hold in the final loaded configuration. These constraints can cover exact destinations or collections beneath dynamically named mapping keys, without requiring the application to discover those names before loading.

The feature covers ordinary and layered loading, imported items, inline sequences, selected overlays, and sibling inheritance. Typed defaults, CLI assignments, and application resolution retain their own combination semantics; CLI application moves later in the loading lifecycle.

This scope is narrower than the deferred identified-record sequences proposal. Automatic recursive merging of matching records remains outside this scope; explicit inheritance between distinct records is included. The initial declaration surface is configkit.sequences; key and mode shorthand on imports is deferred. Application lists remain ordinary YAML sequences without control wrappers. Unkeyed sequence concatenation is outside this feature.

Current Baseline And Required Changes

The import contract owns existing selectors, destinations, placement, and ordering. placement: sequence-item already places each selected source mapping as one item at its effective destination. Item-local helper imports can contribute fields before placement.

The composition contract, layered-loading contract, and inheritance contract describe the implemented baseline:

Area	Current behavior	Target behavior
Identity declaration	`Definition.SequenceCompositions` registers one literal string field at each registered mapping path.	YAML collection declarations and optional authoritative application registrations support field sets and scalar identity components.
Application registration destinations	Registrations select exact literal mapping paths.	Exact and structural selectors can impose identity beneath arbitrary mapping keys, independently at each matched destination.
Strict composition	Registered sequences append distinct string identities and reject duplicates; repeated unregistered sequences conflict.	Collections with effective identity append distinct records and reject duplicates. Applicable lower-tier identity is available before composing higher-tier peers. Unidentified sequences retain strict-conflict behavior.
Source precedence and selected overlays	A present higher sequence replaces the lower sequence wholesale.	Collection policy selects `replace`, `append`, or `upsert`; overlays can declare policy for inline sequences.
Sibling inheritance	An inheriting mapping recursively overlays its resolved parent; supplied sequences replace wholesale.	Mappings retain recursive combination, while sequences use destination policy and the same three operations.
Sequence-member inheritance	A sequence element has no sibling scope for its own inheritance relationship.	An identified sequence provides a sibling scope: a record can inherit from one record in the same sequence, selected by complete identity.
CLI overrides	Apply before inheritance and typed defaults/decoding.	Apply after inheritance and typed defaults/decoding, before application resolution.
Identity enforcement	Registration governs strict sequence composition.	Structural identity checks cover identified combinations; application registrations additionally constrain final effective sequences.

Source-backed underlays participate in source assembly. Definition.Defaults provides typed defaults later; extending source combination does not turn typed default lists into source underlays.

Collection Declarations

Use a source-root configkit.sequences block containing collection declarations. Each entry identifies a destination and may declare key, mode, or both. Identity and mode can be supplied independently. This form serves imported items, inline records, ordinary mapping imports, and inline sequences in selected overlays.

configkit:
  sequences:
    - destination: [prompts]
      key: [namespace, id]
      mode: append
  imports:
    - dir: prompts
      pattern: '*.yaml'
      destination: [prompts]
      placement: sequence-item

Each selected file contributes a record such as:

namespace: support
id: welcome
text: Welcome

Here append retains a lower-layer collection and adds distinct higher-layer records. Identity and combination controls belong to the collection declaration, so imported and inline contributions share one policy surface.

Destinations And Scope

destination uses the existing sequence of literal mapping keys. Resolve it relative to the declaring source's placement: a fragment mounted under catalog can declare [prompts] locally. A sequence-item source can declare a nested collection within its own item without referring to an eventual numeric index. Preserve source and item-local scope boundaries.

A collection's scope is its resolved destination within its composition scope. Resolve relative destinations before reconciling declarations. Different imported items retain separate scopes for their nested collections. A declaration governs all contributions to its collection in the tier, including imported and inline records; it does not govern only records in the declaring file.

Gather declarations before composing the relevant scope so their effect does not depend on which contribution is assembled first. Retain applicable controls through source precedence and inheritance, then keep them out of application decoding.

Sequence declarations are independent of filesystem import enablement. Selected overlays may contain configkit.sequences, extending the current blanket prohibition on overlay configkit declarations. This does not enable imports in overlays.

Overlay Authoring

For an existing sequence [foo, bar], an overlay can replace foo in full:

configkit:
  sequences:
    - destination: [prompts]
      key: id
      mode: upsert
prompts:
  - id: foo
    text: Revised definition

The result is [foo*, bar]. The explicit upsert mode can be omitted when no different explicit mode applies from a lower tier: effective identity then selects upsert. To add a distinct record instead:

configkit:
  sequences:
    - destination: [prompts]
      key: id
      mode: append
prompts:
  - id: baz
    text: Another prompt

The result is [foo, bar, baz]. To replace the whole sequence:

configkit:
  sequences:
    - destination: [prompts]
      mode: replace
prompts:
  - id: bar
    text: Bar definition
  - id: baz
    text: Baz definition

The result is [bar, baz]. Explicitly selecting the default operation is valid. With no applicable explicit mode, omitting this declaration selects replace for an unidentified collection and upsert for an identified one. A retained explicit mode continues to govern.

Identity Definitions And Constraints

Field Sets

key accepts a string or a nonempty list of strings naming direct, literal fields in each record. A name such as metadata.id denotes that literal field, not a nested lookup. Treat field names as a set: remove repeated names and ignore authored order when comparing definitions.

Thus id, [id], and [id, id] are equivalent; [namespace, id], [id, namespace], and [namespace, id, id] are also equivalent. An empty list is invalid because it supplies no identity fields. These rules apply to both configuration declarations and application registrations.

Use a consistent field order internally to construct identity tuples. Preserve component types and tuple boundaries so unstructured concatenation cannot cause otherwise distinct identities to collide.

Scalar Components And Equality

Each identified record must be a mapping with every designated identity field present. Components may be strings, numbers, booleans, or explicit null. Missing fields and mapping or sequence components are invalid. Present null differs from absence; two single-field null identities are equal. Compound identities are equal when all corresponding components are equal.

Strings compare exactly, including empty strings, without trimming, case folding, or normalization. Scalar categories remain distinct: "1", 1, true, and null are different identities.

Numbers compare by mathematical value across integer, fractional, and exponent spellings: 1, 1.0, and 1e0 are equal, as are 0 and -0. Use exact numeric normalization for identity comparison without rounding through a floating-point conversion. This does not rewrite authored values or change existing typed decoding and projection semantics. The parser's rejection of non-finite numeric forms remains unchanged.

Application-Owned Constraints

Retain an optional application interface for registering collections and imposing their identity definitions. Configuration may define identity where no application constraint exists. When a declaration omits key, an applicable registration supplies identity. An explicit equivalent definition is accepted, and a different definition fails under every mode, including replace.

For example, if the application requires id, key: [id, name] is invalid: that compound identity would allow repeated id values with different names. A registration is an authoritative constraint, not an omission-filling default. Copying a collection through inheritance does not create another application registration; registrations constrain their registered destinations.

Identity constraints validate actual sequences. They do not independently require a path to exist, be non-null, or have sequence shape. The application schema and validation own those requirements. Final-result enforcement for application registrations is defined under Validation And Diagnostics.

Dynamic Application Registration Destinations

Allow applications to register identity for collections whose containing mapping keys become known only during loading. For example:

collections:
  weatherreporter:
    prompts:
      - id: foo.bar
        text: Report the weather.
  homewatch:
    prompts:
      - id: foo.bar
        text: Report the household status.

One application registration can require id for every namespace's prompts list. The equal IDs above are valid because each list has its own uniqueness scope. An optional root prompts list can have a separate exact registration.

Use an explicit application-side structural selector composed of literal mapping keys and an arbitrary-mapping-key segment. Conceptually, the namespace rule is:

Literal("collections") -> AnyMappingKey -> Literal("prompts")

This notation explains the selector; it is not a Go signature or YAML syntax. Each segment consumes exactly one mapping level. Match the complete destination, not an arbitrary prefix or suffix. Literal keys remain exact, including empty strings and keys containing punctuation such as *, /, or .. An arbitrary-key segment is distinct from any literal spelling and does not match a sequence index. Existing Path values retain their literal meaning.

This capability belongs to the reusable application definition and applies to ordinary and layered loads. YAML collection destinations and import destinations retain their existing literal syntax. Recursive search, regular expressions, filesystem glob rules, sequence-index selectors, and custom matching callbacks are outside this addition.

At each concrete matched destination, apply the same authoritative identity constraint as an exact registration. Use the settled field-set and scalar-equality rules. Do not pool identities across destinations or require records to repeat the containing mapping key as an identity component. Applications retain ownership of namespace interpretation, qualified runtime IDs, and domain validation.

Matching supplies effective identity before relevant strict composition and precedence operations. Without an applicable explicit mode, this identity selects upsert under the existing default rule. It also supplies identity for inheritance within the matched sequence. YAML declarations must agree with it under every mode, including replace. These are ordinary application constraints with a broader destination selector, not a separate final-validation facility.

Registration Overlap

Equivalent application constraints coalesce at any shared destination. Equivalence uses the identity field-set rules: for example, id and [id, id] agree. Incompatible constraints fail; exact registrations do not override broader selectors, and registration order grants no precedence.

Detect incompatible overlapping registrations while validating the application definition, before source acquisition. For these fixed-depth mapping selectors, overlap exists when the selectors have the same number of segments and every corresponding pair can match a common key. Unequal literal keys at any position make the selectors disjoint; an arbitrary-key segment can overlap a literal or another arbitrary-key segment. Compare segment structure rather than rendered path strings. No configuration pre-read is needed.

Thus an id rule for all namespace prompt lists conflicts with a [id, name] rule for one named namespace, even if that namespace is absent from a particular load. This is an invalid application definition. A valid selector matching no collections is harmless: registration does not require a path to exist, create collections, or impose shape and nullability requirements.

Matching Across The Loading Lifecycle

Match application selectors against effective configuration destinations after accounting for input placement. Physical filenames and source-local paths do not determine applicability. Make constraints available before composing contributions at those destinations, including namespaces first introduced by imports, mounted inputs, or higher tiers. Apply matching constraints to key-only YAML declarations as well, even before their sequence exists. Consumers need neither a preliminary load nor source inspection to enumerate namespaces.

During inheritance, apply registrations at the inheriting destination, including newly created matching destinations. Copying a collection does not transfer its application registrations to a different path. The existing YAML-policy ownership and transfer rules remain separate from application selector matching.

During final validation, find matching actual sequences in the final effective configuration. This includes namespaces introduced or replaced by typed defaults, CLI overrides, and application resolution. Reusing only destinations found during source assembly would miss these values. Use the same selector meaning and identity validator in both phases; later assignments do not trigger composition or inheritance again. YAML-only identities retain their existing enforcement boundary.

Strict Composition And Precedence Operations

Compose Peers Before Applying Precedence

Resolve applicable identity and mode policy, strictly compose the tier's peer contributions, then apply one selected operation per collection against the accumulated lower value. The tier operation does not sequentially replace peer source contributions. Selected overlay contributions use the same combination semantics.

Within a strict tier, equivalent identity definitions coalesce and different field sets fail. Agreeing explicit modes coalesce; conflicting explicit modes fail. Omission expresses no preference. A root declaration or a later declaration has no precedence over conflicting peers. Explicitly selecting a default mode is still a real instruction and can conflict with another explicit mode.

Identified peer sequences append distinct records. Duplicate identities among peers or within a contributed list fail under all three modes, including replace and upsert. Equal or disjoint record payloads do not excuse a duplicate identity. Unidentified sequences retain their current strict-composition conflict behavior.

For example, higher-tier fragments [C] and [D] with an agreed identity assemble as [C, D]. Against lower [A, B], append or upsert produces [A, B, C, D], while replace produces [C, D]. Peers [B] and [B*] conflict even under upsert because neither has higher precedence.

Available Operations

Use one spelling for each mode: replace, append, and upsert.

Mode	Behavior at source-precedence and inheritance boundaries
`replace`	Replace the complete lower sequence with the assembled higher sequence.
`append`	Retain lower records and append higher records; duplicate identities fail.
`upsert`	Replace matching records in full, retain unmatched lower records, and append new identities.

Append and upsert require effective identity. ConfigKit does not infer identity from an id field or record position. Whole-sequence replacement does not require identity agreement with lower YAML policy, but application constraints still apply.

Letters denote record identities; B* has the same identity as B with a different payload:

Lower	Higher	Mode	Result
`[A, B]`	`[C]`	`append`	`[A, B, C]`
`[A, B]`	`[C]`	`replace`	`[C]`
`[A, B]`	`[B*]`	`append`	Duplicate-identity error
`[A, B]`	`[B*]`	`replace`	`[B*]`
`[A, B]`	`[B*]`	`upsert`	`[A, B*]`
`[A, B]`	`[B*, C]`	`upsert`	`[A, B*, C]`

Upsert replaces the whole matched record. If lower B has id, text, and temperature, while higher B has only id and text, the result at this boundary has only id and text. Subsequent defaulting or other lifecycle operations retain their own semantics. Mappings continue to combine recursively; matching sequence records do not recursively merge fields.

Mode Selection And Persistence

Select the operation in this order:

Reconcile explicit modes within the current tier; conflicting peers fail.
Use the current tier's explicit mode if supplied.
Otherwise retain an applicable explicit mode from a lower tier.
With no explicit mode, infer upsert when effective identity is available.
Otherwise infer replace.

Effective identity can come from an application registration, current declarations, or applicable lower policy, subject to the identity agreement and lifetime rules below. Determine the inferred mode from available identity before applying the operation. A plain higher-tier sequence can therefore use upsert without repeating a key declaration.

An explicit mode persists at its applicable destination until a higher-priority explicit declaration changes it. New sequence values alone do not reset it, and an unrelated overlay leaves it intact. A mode-only declaration can change policy without supplying records; it neither clears the collection nor retroactively recombines earlier tiers. Later source contributions and inheritance use the resulting policy.

Lower explicit mode	Higher declaration	Effective mode
`append`	Mode omitted	`append`
`append`	`upsert`	`upsert`
`upsert`	`replace`	`replace`
None	Mode omitted, effective identity available	Inferred `upsert`
None	Mode omitted, no effective identity	Inferred `replace`

Only explicit modes persist. Inferred defaults are recomputed from effective identity when needed. Explicit replace continues to apply in subsequent tiers until another mode is declared; introducing identity alone does not override it. Mode persistence does not preserve YAML identity discarded by replacement, nor policies owned by discarded records. Source precedence and inheritance use the same destination policy, without separate per-phase settings. CLI assignments remain outside mode selection.

Presence, Empty Sequences, Null, And Kind Mismatches

At a collection's precedence boundary:

Higher contribution	`replace`	`append` / `upsert`
Absent	Leave the lower value unchanged.	Leave the lower value unchanged.
Explicit `[]`	Clear the sequence.	Contribute no records.
Explicit null or another non-sequence value	Ordinary replacement, subject to application schema.	Error: the operation requires a sequence.

A declaration alone does not clear a collection. An optional import selecting nothing contributes nothing and does not erase existing records. An absent lower collection can be initialized by an incoming sequence under any mode, subject to identity requirements. Explicit lower null or another non-sequence value is not absence: append and upsert fail against it, while replacement remains available. An empty incoming sequence bypasses neither kind checks nor identity requirements.

With effective identity and no explicit mode, prompts: [] uses upsert and does not clear the collection. Clearing requires replace, selected in the current tier or retained from lower policy. Replacing an identified collection with null also requires replace and a schema permitting null. Collection null is distinct from a present null identity component inside a record.

Ordering

Retain existing dependency-ready source scheduling within each tier and authored item order within lists. Gathering declarations does not change assembly order; identity values do not sort records.

Append keeps lower records in order, followed by higher records in assembled order.
Replace keeps only higher records in their assembled order.
Upsert keeps a matched record's lower position and appends new identities in higher-tier order.

For example, lower [A, B, C] and higher [C*, A*, D] produce [A*, B, C*, D] under upsert. Whole-sequence replacement gives the higher tier control of the complete sequence order.

Policy Agreement, Lifetime, And Ownership

Identity Agreement And Replacement

Append and upsert use one agreed identity for all participating records:

Lower identity	Higher declaration	Result when combining records
`id`	Omitted	Use `id`.
`id`	`id`, `[id]`, or `[id, id]`	Accept the equivalent definition.
`id`	`[namespace, id]`	Error: different definitions.
None	`id`	Establish `id` and validate all participating records.

Introducing identity for an unkeyed lower sequence requires checking both lower and incoming records, including uniqueness within each contribution. With no identity from the application or either participating collection, append and upsert fail.

YAML identity follows the collection that survives an operation:

Operation	Effective YAML identity
Append or upsert	Retain or establish the agreed identity.
Whole-sequence replacement	Discard lower YAML identity and use the replacing tier's identity declaration, if any.
Replacement by null or another non-sequence value	Discard lower YAML identity and retain any identity explicitly declared by the replacing tier.

Application constraints remain authoritative under every operation. Without an application constraint, replacement can introduce a different identity, such as changing from id to name. Replacing a sequence without a new key declaration or application registration produces an unkeyed sequence. No identity-removal directive is needed.

The reset occurs only when the collection value is actually replaced. Declaring replace without supplying a replacement does not authorize changing the identity of retained records. Explicit mode lifetime is separate from YAML identity lifetime.

Declarations Without Sequence Values

A key declaration without accompanying records applies immediately to an existing retained sequence: check agreement and validate its records, without recombining earlier tiers. For example, a higher tier that only declares destination: [prompts], key: id identifies and validates the retained lower prompts sequence.

If no collection exists, retain the declaration in its owning scope for later contributions. An otherwise empty underlay can establish identity for main-tier fragments. An explicit identity declaration can also remain pending at a non-sequence destination; it does not independently impose a shape requirement. The presence and kind rules still determine whether an attempted operation fails. Item-local declarations retain their ownership boundaries.

Lower-Tier Identity During Strict Composition

Resolve applicable identity before composing a higher tier's peers, using application registrations, available lower-tier policy, and declarations gathered within the higher tier's scope. Authors need not repeat a baseline key solely because the higher tier has multiple fragments.

For example, an underlay establishes key: id for prompts, and two main-tier fragments contribute [foo] and [bar] without a key declaration. They strictly compose as [foo, bar] before the main tier combines with the accumulated lower collection. If both contribute foo, composition fails; upsert does not give precedence to peers.

Lower YAML identity remains subject to agreement, replacement, and ownership. A replacing collection must not be constrained by obsolete lower identity just because policy resolution precedes composition. Likewise, a replacement record cannot borrow nested identity owned by the discarded lower record, even when outer identity or numeric position matches. Application constraints still apply. Higher-tier declarations do not retroactively repair invalid lower-tier composition.

Nested Record Ownership

Attach item-local policies to their owning contributions so they follow surviving records independently of numeric indexes. Whole-record upsert discards the old record's local policies along with its fields. Whole-sequence replacement discards policies owned by removed records. A replacement record supplies its own local declarations; equal outer identity or final index does not transfer discarded policies.

For example, a source imported as one record into prompts can declare:

configkit:
  sequences:
    - destination: [steps]
      key: id
      mode: append
id: foo
steps:
  - id: prepare

The declaration governs steps inside that foo record. Retaining foo retains its local policy, including the explicit mode. Upserting a new foo discards the old steps and its declaration even though the outer record keeps its position.

Policies independently applicable at a destination are not discarded merely because an individual record is replaced. Ownership determines which declarations are removed. Application registrations remain constraints at their registered destinations. Inheritance copies transfer identity and choose operations as specified next.

Sibling Inheritance

Complete source-tier assembly before resolving inheritance. Retain existing mapping-sibling scopes and add identified-sequence scopes as described below. Both use the existing dependency-ready algorithm. At each inheritance combination, the resolved parent's collection is the lower value and the inheriting mapping's local collection is the higher value. Mappings combine recursively; upsert within an identified sequence remains whole-record replacement. Explicit record inheritance is a separate construction step after those replacements.

Use the inheriting destination collection's policy: an applicable explicit mode, including one retained through source tiers, wins. Otherwise infer upsert with effective identity and replace without it. There is no separate default-append rule for inheritance. The same destination policy governs source-precedence and inheritance combinations.

YAML identity follows a collection copied through inheritance, subject to agreement and replacement rules. A source collection's mode does not automatically become an instruction at a different inheriting destination. For example, source steps has key: id and mode: append; an inheriting mapping supplies local steps without an applicable explicit mode. The copied identity selects upsert there. An explicit destination append instead selects append. A matching destination key agrees; a different key conflicts when combining records, while explicit replace can establish new YAML identity subject to application constraints.

For example:

configkit:
  sequences:
    - destination: [parent, items]
      key: id
child:
  items:
    - id: foo
      value: A
    - id: bar
      value: B
parent:
  extends: child
  items:
    - id: baz
      value: C

Despite the names, parent inherits from child. Identity at [parent, items] selects upsert, producing [foo, bar, baz]. If the local sequence instead supplies a revised foo, the result is [foo*, bar]. Append rejects that repeated identity; replace retains only the local sequence.

The same ordering, presence, and kind rules apply at inheritance boundaries. An omitted local sequence retains the inherited sequence. Explicit [] clears it under replace and contributes no records under append or upsert. Retain controls until inheritance uses them, then exclude them from application decoding.

Inheritance Between Identified Sequence Records

An identified sequence supplies a sibling scope for its records. Identity can come from an application registration or an applicable YAML declaration. A record may inherit from one other record in that same sequence using an extends mapping that selects the parent's complete identity:

configkit:
  sequences:
    - destination: [prompts]
      key: id
prompts:
  - id: base
    model: standard
    temperature: 0.5
  - id: concise
    extends: {id: base}
    temperature: 0.2

The final concise record is:

id: concise
model: standard
temperature: 0.2

The base record remains in the collection. Parents are ordinary records subject to the same decoding and validation as other records; there is no abstract template category.

Use mapping-form references for both single and compound keys. With key: [namespace, id], a reference can be extends: {namespace: shared, id: base}. Match components using the collection's field-set and scalar-equality rules, including mathematical numeric equality. extends: {id: null} selects a null-valued identity; extends: null cancels a relationship.

Parent-reference mappings use ordinary strict mapping composition. Item-local helper imports may supply disjoint fields of a compound reference; overlapping fields conflict, even when equal. For example, a helper's extends: {namespace: shared} and the importing record's extends: {id: base} compose into extends: {namespace: shared, id: base}. This assembles one record; it does not permit strict peers to contribute duplicate record identities.

Before parent lookup, require the surviving effective reference mapping to contain exactly the collection's identity fields, with supported scalar values. Missing or extra fields fail. Validate the completed reference during inheritance, rather than requiring each helper contribution to be independently complete. Superseded references remain historical evidence; earlier strict-composition conflicts remain errors.

Every record must supply its complete own identity before inheritance. Preserve the child's identity fields and inherit only the parent's nonidentity payload; inheritance cannot supply a missing child identity. Child payload fields take precedence, mappings combine recursively, and nested sequences use the inheriting destination's collection policy. Consume the record's directive before decoding.

Support one parent within the same identified sequence, including forward references and chains. Missing parents and cycles, including self-reference, fail the load. Cross-collection references, position-based references, multiple parents, and inheritance between unidentified sequence members are outside this scope. Existing string-form references between mapping siblings retain their meaning; they do not become shorthand for sequence identity references.

Resolve relationships against records surviving source-tier assembly and selected overlays. A higher layer can replace a parent before inheritance runs; removing that parent makes a surviving reference fail. Whole-record upsert discards the old record's directive along with its payload. Omitting extends in the replacement record does not retain the displaced record's relationship. An explicit reference to the record's own identity is a cycle, not a way to access its lower-tier predecessor.

Reuse Kahn's algorithm with the existing stable canonical semantic-pointer key for dependency-ready ordering. Build parent lookup from the sequence's effective identity, and preserve the assembled list order independently of resolution order. Enclosing relationships resolve before nested relationships. Parent templates retain nested instructions for evaluation at each inheriting destination, following the existing destination-local inheritance rule. Copied collection identity, destination operation selection, and record-owned policy transfer follow the rules above.

Record inheritance runs in the existing structural phase, before typed defaults, decoding, CLI overrides, and application resolution. Those later stages do not trigger another inheritance pass. Preserve inherited presence, origin and history evidence, sensitivity, declaring resource contexts, and existing expansion budgets just as for mapping-sibling inheritance. Identity lookup must not expose identity values in diagnostics. Existing index-based inspection paths remain sufficient; no new identity-selector path API is needed.

Loading Lifecycle

Use the following order:

Assemble source tiers and selected overlays, resolving policy before each tier's strict composition and applying collection operations at precedence boundaries.
Resolve sibling inheritance in mappings and identified sequences, then apply typed defaults and decode the assembled configuration.
Apply explicit CLI overrides to that configuration and check supplied values against the application schema.
Run application resolution against the overridden configuration.
Resolve effective secret references, validate the final configuration including application-registered identities, and publish it.

Typed Defaults And Application Resolution

Source-backed underlays, including application-supplied mapping sources, participate in sequence combination. Typed Definition.Defaults remains the initial decoded state: supplied sequences replace default sequences. Source append does not append to a typed default list.

Application resolution retains assignment and recorded-transformation semantics. ConfigKit does not automatically combine a callback's assigned sequence with its previous value. Application-registered identities receive final validation independently of how defaults, CLI overrides, or callbacks produce their values.

CLI Precedence

CLI overrides are the highest-precedence input to application resolution. Supplied sequences replace whole sequences, including sequences inside supplied mappings. Mappings retain recursive override behavior; empty sequences and permitted nulls retain explicit replacement meaning. Configuration append or upsert does not reinterpret CLI assignments. Inheritance and defaulting do not run again afterward: overriding assembled [A, B] with [C] presents [C] to application resolution without inheritance reintroducing prior records.

Application resolution sees CLI values and their explicit-presence and source metadata. Resolution.Default preserves explicit CLI values; normalization and recorded application transformations may still change them. For example, a derived URL can use a CLI hostname, while an explicit CLI URL can be preserved by an absence-based default. Retain recording and provenance requirements for those transformations.

Reject structural directives such as extends and configkit in CLI override paths or supplied values. CLI input supplies application values after structural processing; it cannot supply or cancel inheritance directives, change completed relationships, or prevent earlier inheritance failures.

Typed decoding also precedes CLI application, so a CLI replacement cannot repair an earlier decoding failure. A source string in an integer field fails before a valid CLI integer can apply. This changes current override-before-decoding behavior. Semantic validation remains after overrides and application resolution.

Secret lookup uses effective references after overrides and resolution. Neither stage bypasses final application identity constraints or semantic validation.

Validation And Diagnostics

Structural Identity Checks

Allow item-local helper imports to complete and consume applicable non-main source-version headers before checking payload identity for collection assembly. A helper can supply an identity component; a consumed header cannot supply one after removal.

Validate each identified contribution before combination and check uniqueness in the resulting identified collection, including inheritance operations. Upsert matches across precedence are replacements; duplicates within a contribution remain errors. Later inheritance, typed defaults, or callbacks cannot repair an identity missing at an earlier assembly boundary.

YAML-only identities are enforced during structural composition, precedence assembly, and identified inheritance operations. They have no final-result guarantee after typed defaulting, CLI overrides, or application resolution. Those stages can produce duplicate identities or otherwise violate a YAML-only identity. Applications needing a final guarantee register the constraint through the application-owned interface.

Adding record inheritance does not expand this guarantee: a YAML declaration can enable parent lookup without imposing final validation on values later produced by defaults, CLI overrides, or application resolution.

Final Application Constraints

On successful loading, final effective collections covered by application registrations satisfy their declared identity constraints. Retain assembly-time checks and validate final effective sequences before publication, after defaults, inheritance, overrides, application resolution, and secret lookup.

This covers both exact and dynamic application registrations. Evaluate dynamic selectors against the final effective configuration, with independent validation for each concrete matched sequence as described under Matching Across The Loading Lifecycle.

ConfigKit performs these identity checks; consumers register the constraint without needing to repeat duplicate detection in Definition.Validate. Application-owned semantic validation continues to cover domain-specific rules.

This guarantee covers actual sequences, not path existence or allowed shape. It does not require every intermediate callback state to satisfy the constraint and does not cover caller mutations after loading returns. Final checks examine effective typed values; assembly checks validate authored contributions. A final scalar alone does not prove that a source explicitly supplied that field.

Use the existing canonical final-value representation and shared publication path as the foundation. The guarantee requires neither a new mutation-history system nor exact causal reconstruction for records supplied by defaults or callbacks.

Diagnostics And Inspection

Identity diagnostics identify collection and item paths using available origin information without exposing identity values. Conflicting declarations identify the collection destination and declaration locations.

For dynamic registrations, use concrete matched collection and item paths for value diagnostics, with available origins and existing sensitivity rules. Definition-time overlap errors identify the conflicting registrations; they do not require or invent a concrete destination from a loaded document.

Correctness includes final values, projected item paths, source destinations, provenance, history, sensitivity, and passive resource contexts. Appending can change final indexes; replacing changes which contributions survive. Preserve the existing loading and inspection contracts through both operations, including policy ownership and inheritance transfer.

Work Needed From The Baseline

The implementation plan should cover these connected changes, with execution order and test sequencing defined there:

Declarations and public API: extend application registration to the settled identity model and explicit structural selectors while preserving literal Path semantics. Parse and consume configkit.sequences across supported source forms and selected overlays; preserve placement-relative and item-local scopes. Keep sequence declarations independent of filesystem import enablement.
Identity and policy representation: implement field-set comparison, typed scalar identity tuples with exact numeric equality, declaration agreement, pending declarations, explicit-mode persistence, replacement lifetime, and record-owned policy transfer. Choose bookkeeping proportional to these semantics.
Registration matching: support literal and arbitrary-mapping-key segments, detect incompatible selector overlap during definition validation, and resolve applicable constraints at concrete placed destinations. Reuse the identity rules and validator for exact and dynamic registrations. Keep matching separate from source discovery and from YAML declaration ownership.
Assembly handoffs: make accumulated lower policy available before higher-tier strict composition. Separate source acquisition from policy-aware composition or otherwise revise assembly sequencing; current main-before-underlay composition is not a design constraint. Preserve distinctions among acquisition order, dependency-ready scheduling within tiers, and precedence order. Account for source records, callbacks, main-version checks, and failure diagnostics. Retain version ownership and validation semantics; higher replacement cannot hide lower-tier structural failures.
Combination and inheritance: implement the three operations with strict peer handling, whole-record replacement, presence rules, stable ordering, and the destination policy rules for inheritance. Extend sibling-scope discovery and parent lookup to identified sequences, validate mapping-form references, and preserve child identities while copying parent payloads. Reuse the shared dependency-ready scheduler and destination-local nested resolution. Preserve provenance, history, sensitivity, paths, and resource contexts as records survive, move, or disappear.
Lifecycle and validation: move CLI application after inheritance and typed defaults/decoding, preserve explicit-presence and source metadata for application resolution, reject CLI structural directives, and add final application identity validation through the shared publication path. Keep YAML-only enforcement within structural operations. Match dynamic registrations during assembly and inheritance, and against the final effective configuration rather than a saved list of source-backed destinations.
Documentation and verification: exercise ordinary and layered loading, imported and inline collections, item-local scopes, version-header consumption, selected overlays, inheritance, typed defaults, CLI, callbacks, and inspection. Cover single and compound parent references, scalar equality, forward references and chains, missing parents and cycles, parent replacement/removal, child identity preservation, nested destination policy, cancellation, and stable output order. Verify helper-composed references, overlapping reference fields, missing or extra fields, and replacement of references together with their containing records. Verify independent namespace uniqueness, root and namespaced registrations, literal punctuation keys, equivalent and incompatible overlaps, and conflicts detected even without a matching input destination. Cover namespaces introduced by imports, placement, higher tiers, inheritance, defaults, CLI, and callbacks; preserve ordinary strict-peer and upsert behavior. Check YAML disagreement under replace, harmless absent or empty collections, schema-owned shape requirements, unrelated sequences remaining unselected, and concrete-path diagnostics. Update the public contract and consumer examples when behavior lands.

The collection-policy and CLI-ordering decisions are accepted in ADR 0017 for collection policy and ADR 0018 for CLI ordering. Their scoped supersession is recorded in the affected earlier ADRs, and the architecture policy reflects the accepted decisions. These govern the target architecture; the configuration contract continues to describe implemented behavior until the feature lands. This roadmap does not select migration machinery.

ADR 0017 now includes inheritance within identified sequences; ADR 0009 records that scope extension in its status while preserving its decision content. The architecture policy is aligned. Dynamic selectors extend the destinations covered by application registration; they retain the accepted application authority and final-validation boundary.

After settling the API decision below, the implementation plan should translate this end state into reviewable stages without adding guarantees, restrictions, or combination semantics.

Open Decisions

Registration API Shape

The selector's behavior and definition-time overlap validation are settled. How should the public registration represent exact destinations alongside selectors containing arbitrary-mapping-key segments?

Recommendation: use one registration model with an explicit selector type whose segments distinguish literal keys from arbitrary mapping keys. Provide a straightforward way to construct an exact selector from a literal Path. Both forms then share identity fields, validation, and lifecycle behavior, while ordinary Path remains unchanged. This requires a small new public type, but keeps one registration surface and avoids mutually exclusive destination fields. Exact Go names and constructor signatures belong in the implementation plan.

Alternative: retain exact-path registrations and add a separate dynamic registration form, normalizing both into the same internal selector model. This keeps exact registration familiar for simple consumers, but introduces two public forms with the same identity rules and requires overlap validation across both. Both choices must preserve the settled matching and enforcement behavior; neither introduces selector syntax into ordinary paths or YAML destinations.

48 KiB Raw Blame History