# Sequence Identity And Cross-Tier Combination Status: draft proposal for discussion. This document captures the requested capabilities, settled design choices, and initial recommendations. Choices under Open Decisions remain unsettled. It is not an implementation plan or an implemented contract. All new syntax shown below is illustrative. ## Motivation Allow configuration authors to assemble collections from independently authored records, declare their identity in configuration, and progressively extend a baseline collection through higher layers. Authors should also be able to replace the accumulated collection or replace individual records by identity. The motivating example is a prompt collection whose defaults supply some records and whose application or installation layers add or supersede records. Across tiers, authors need three operations: whole-sequence replacement, duplicate-rejecting append, and whole-record replacement by identity. The latter retains unmatched lower-tier records and adds higher-tier records with new identities. Matching records are replaced in full. ## Current Baseline The [import contract](../config.md#explicit-imports) owns current selector, destination, placement, and ordering behavior. An import with `placement: sequence-item` places each selected source mapping as one item at its effective destination. Item-local helper imports can contribute fields before placement. The application currently registers sequence identity through `Definition.SequenceCompositions`, using one literal string field at each registered mapping path. Within a strict tier, distinct identities append and duplicate identities fail. Repeated unregistered sequences conflict. Across precedence tiers, a higher sequence replaces the lower sequence wholesale. The [composition contract](../config.md#strict-document-composition) owns these rules; [layered loading](../config.md#layered-loading) owns tier precedence. Source-backed underlays and `Definition.Defaults` have different lifecycle roles. Underlays participate in source assembly; typed defaults are applied later. Extending one does not automatically imply extending the other. ## Requested Capabilities The proposed import syntax introduces an identity declaration, tentatively named `key`, that accepts either one field or a list of fields for a compound identity. Applications retain an optional interface for registering a collection and imposing its identity definition. Configuration declarations must conform to an application-owned constraint when one exists. The exact Go interface and configuration declaration site remain to be designed. ```yaml configkit: imports: - dir: prompts pattern: '*.yaml' destination: [prompts] placement: sequence-item key: [namespace, id] mode: append ``` Each selected file would contribute a record such as: ```yaml namespace: support id: welcome text: Welcome ``` Here `mode: append` illustrates retaining a lower-layer collection and adding distinct records from the higher layer. A named operation is the current recommendation for expressing all three choices; field spelling, operation names, and default behavior remain open. The earlier `append` or inverse `replace` boolean proposals would need additional syntax for the third operation. The intended results can be expressed without settling the syntax. Letters below denote record identities; `B*` is a higher-tier record with the same identity as `B` and a different payload. The outcomes are settled; the ordering shown is the recommendation discussed below. | Lower-tier sequence | Higher-tier sequence | Operation | Proposed result | | --- | --- | --- | --- | | `[A, B]` | `[C]` | Append | `[A, B, C]` | | `[A, B]` | `[C]` | Whole-sequence replacement | `[C]` | | `[A, B]` | `[B*]` | Append | Duplicate-identity error | | `[A, B]` | `[B*]` | Whole-sequence replacement | `[B*]` | | `[A, B]` | `[B*]` | Replacement by identity | `[A, B*]` | | `[A, B]` | `[B*, C]` | Replacement by identity | `[A, B*, C]` | Whole-sequence replacement discards the lower sequence. A repeated identity in the retained combined sequence is an error under the proposed append semantics, including when the records have identical or disjoint payloads. ## Settled Design Choices ### Whole-Record Replacement By Identity The third cross-tier operation replaces each matching lower record with the complete higher record, retains unmatched lower records, and adds new identities. For example, given these lower and higher records: ```yaml # Lower record id: B text: Original temperature: 0.5 ``` ```yaml # Higher record id: B text: Revised ``` The resulting record at this precedence boundary contains `id` and `text`. The lower record's `temperature` is discarded. Subsequent defaulting or other lifecycle operations retain their separately defined semantics. Recursive field-by-field overlay of matching records is outside this selected operation. ### Optional Application-Owned Identity Constraints Retain an interface through which applications may register a collection and impose its unique-key definition. If the application requires `key: id`, a configuration declaration of `key: [id, name]` is invalid: the compound key would permit repeated `id` values with different names. The application identity constraint applies whichever cross-tier operation configuration selects. This is an authoritative constraint, distinct from an application default that only fills an omission. Configuration can define identity for collections without an application-owned constraint. The recommendation is for omitted configuration identity to inherit a registered definition and for an explicit matching declaration to be accepted. The final-result guarantee for application-owned constraints is settled below; YAML-only enforcement remains an open follow-up to OD-1. ### Final-Result Guarantee For Application Registrations **OD-1: settled for application-owned constraints.** > On successful loading, the final effective collections covered by application > registrations satisfy their declared identity constraints. Retain assembly-time identity checks and validate the final effective collections before publication. This covers the loaded state after defaults, inheritance, overrides, and application resolution have taken effect. Those operations retain their separately defined combination semantics. The guarantee does not require every intermediate callback state to satisfy the constraint and does not extend to caller mutations after loading returns. Keep the promise limited to the declared identity constraints. It does not by itself require registered paths to exist or settle null and non-sequence handling; those boundaries remain under OD-3 and OD-6. Final validation examines effective typed values, while assembly-time checks retain their role in validating authored contributions. A final string value alone does not prove that the field was explicitly supplied in a source. The existing canonical final-value representation and shared publication path provide the implementation foundation. Diagnostics should identify the affected collection and item paths, using available origin information without exposing identity values. The guarantee requires neither a new mutation-history system nor exact causal reconstruction for records supplied by defaults or callbacks. Final enforcement for identities declared only in YAML is not settled here. ## Initial Architectural Direction Treat identity as a property of the destination collection, even if an import introduces its declaration. That property needs to cover both imported items and records authored directly at the same destination. Discovering collection declarations before composing the relevant scope would make their application independent of which contribution happens to be assembled first. The initial recommendation is to compose each tier's contributions strictly, then apply one collection operation from the higher tier against the lower sequence. An individual import would therefore express policy for its tier's contribution at that destination. It would not sequentially erase contributions from peer imports. Multiple agreeing declarations could coalesce; conflicting explicit identity or operation declarations would produce errors. For append, the ordering recommendation is to retain lower-tier item order and then the higher tier's assembled item order. For replacement by identity, retain the position of each replaced record and append new identities in higher-tier order. Existing dependency-ready scheduling would continue to determine assembly order within a tier. Whole-sequence replacement would retain only the higher tier's assembled sequence. Ordering and the treatment of peer declarations remain recommendations awaiting agreement, not changes to the current ordering contract. This requires collection policy to survive import expansion through precedence assembly. Correctness includes final values, projected item paths, source destinations, provenance, history, sensitivity, and passive resource contexts. Appending changes final indexes; replacing changes which contributions survive. The implementation must account for both through the existing loading lifecycle. ## Scope And Decision Ownership This is narrower than the deferred [identified-record sequences proposal](future.md#1-identified-record-sequences), which also considers recursive merging of matching records and record-specific inheritance. Those capabilities remain separate candidates. Compound keys, duplicate-rejecting append, and whole-record replacement by identity do not require adopting them. The proposal would extend identity declaration ownership and revise the cross-tier replacement decision in [ADR 0014](../adr/0014-sequence-placement-and-composition.md). If adopted, record the architectural change in a superseding decision and reconcile affected architecture policy before implementation. The current ADR and contracts remain authoritative until that decision is made. This draft neither changes them nor selects migration machinery. A staged implementation plan should follow resolution of the questions below. It should cover ordinary and layered loading, item-local scopes, source-version processing, identity validation, precedence, and inspection together. Public API design and test sequencing belong to that later plan. ## Open Decisions ### OD-1 Follow-Up. Final Validation For YAML-Only Identity Declarations Application-registered collections have the settled final-result guarantee above. Should a configuration-only identity declaration carry the same guarantee, or govern structural assembly alone? **Initial recommendation:** Settle this after choosing declaration scope in OD-2 and identity-policy lifetime across replacement in OD-3. Final enforcement needs an unambiguous effective policy for each retained collection. The application-owned guarantee can proceed independently of this choice. **Alternatives:** Applying the same final check to YAML-declared identities would give both declaration forms consistent enforcement, but requires resolving which policy survives replacement or follows nested records. Limiting YAML-only identity to assembly keeps policy lifetime narrower, but later operations could produce a final collection that no longer satisfies that identity declaration. ### OD-2. Where Does Configuration Declare Collection Policy? Is `key` on a `sequence-item` import sufficient, or should configuration have a collection declaration that also covers inline sequences, ordinary mapping imports containing sequences, and selected overlays? How does such a declaration address a collection inside an imported item? **Initial recommendation:** Define one destination-scoped collection policy model first. Evaluate a configuration-level declaration alongside import syntax against these workflows before selecting the surface. Collection identity must apply consistently to all participating records at that destination, including inline records. Resolve relative destinations and item-local scopes using their existing ownership boundaries. **Alternative:** Limit this feature to policies declared on item imports. This keeps the initial syntax small and directly serves imported prompt collections, but leaves inline-only collections and overlay-authored append without a natural declaration site. A separate collection declaration covers those cases at the cost of more configuration structure; an import shorthand would require a clear relationship to that declaration. ### OD-3. What Constitutes Identity, And When Can It Change? Settle compound-key field selection, value types, missing/null components, empty and repeated field lists, empty identity strings, and agreement between declarations. Decide whether whole-sequence replacement can introduce a different identity definition after discarding the lower collection when the application has imposed no identity constraint. An application-owned constraint remains authoritative. **Initial recommendation:** Interpret `key: id` as `[id]` and a compound key as an ordered tuple of direct, literal string fields. Retain exact string comparison and require every component to be present. Preserve tuple boundaries rather than concatenating values. Identity declarations participating in append or replacement by identity must agree; equal definitions within a scope coalesce and conflicting definitions fail. Detailed list validation and identity changes at whole-sequence replacement boundaries remain to be settled. **Alternative:** Permit additional scalar types or nested field paths. These support more schemas but add type-equality and addressing rules. A load-wide fixed identity definition simplifies agreement across tiers; allowing a replacement to establish a new definition gives independently authored layers more flexibility. ### OD-4. What Does An Import's Combination Choice Control? What happens when peer imports into the same destination request different operations, or when some contributions omit the operation? Settle ordering for replacement by identity, including where replaced records and new identities appear. **Initial recommendation:** Assemble one strict sequence per destination in each tier, then apply its agreed operation at the precedence boundary. Reject conflicting explicit choices within a tier. This preserves peer composition and prevents declaration or scheduling order from granting replacement authority. Define how omitted choices inherit the effective collection operation once its declaration site is selected. Duplicate identities among peers would remain strict-composition errors under all three cross-tier operations. For replacement by identity, keep matched records in their lower-tier positions and append new identities in higher-tier order. **Alternative:** Treat each import as a sequential collection operation. This permits deliberate resets among imports, but a replacement can discard peer contributions and would require an explicitly selected operation-order model. Another alternative is to allow the choice only on a collection declaration, which reduces conflicts between import annotations but changes the proposed authoring surface. For ordering, moving replaced records to their higher-tier contribution positions is another viable choice. It gives the higher layer more influence over ordering, but changing a record's payload could also move it relative to retained records. ### OD-5. What Is The Default Operation And Its Syntax? Choose syntax for all three operations and decide what omission means. **Initial recommendation:** Keep whole-sequence replacement as the cross-tier default and make append and replacement by identity explicit. Lists can represent complete ordered instructions as well as extensible catalogs; an explicit append communicates the intent to retain lower-layer records. Prefer one named operation with three values to interacting booleans; choose names that distinguish whole-sequence from whole-record replacement. **Alternative:** Default identified collections to append or replacement by identity and require explicit whole-sequence replacement. These defaults make progressive collections concise, but retain lower-layer records unless an author deliberately replaces the collection. Extending the boolean proposal with a separate duplicate-handling option is possible, but introduces interacting settings whose combinations need explanation and validation. ### OD-6. What Do Absence, Empty Sequences, And Null Mean? Can a declaration selecting nothing affect a lower collection? How do explicit empty sequences, null, and non-sequence values interact with all three operations? **Initial recommendation:** An optional import selecting nothing contributes nothing and does not clear lower values. An explicitly supplied empty sequence with whole-sequence replacement can clear a collection; append or replacement by identity with an empty sequence contributes no records. Preserve the distinction between absence and supplied values. Decide null and kind-mismatch handling explicitly rather than silently treating them as empty sequences. **Alternative:** Treat an explicitly declared replacement operation as a reset even when its import selects nothing. This provides a compact clearing mechanism, but makes an optional missing file or empty search destructive. Keeping clearing as an explicit value avoids that coupling. ### OD-7. Which Lifecycle Stages Participate? Do the new cross-tier operations extend only source-backed tiers, or also `Definition.Defaults`, inheritance, CLI overrides, and application-created records? At which boundaries must identity be validated, including helper-supplied fields and consumed source version headers? **Initial recommendation:** Start with source-backed strict composition and precedence assembly. Validate records at the relevant collection assembly boundaries and uniqueness across each combined result. Preserve existing typed-default and application-resolution semantics unless their inclusion is explicitly chosen. Decide selected-overlay coverage with OD-2; it is part of source precedence but currently has no import declaration site. Account for item-local helpers and source-version consumption before deciding identity is available, consistent with the current lifecycle. The settled OD-1 guarantee covers final validation of application constraints. Validating the final collection does not itself change how defaults, inheritance, CLI overrides, or callbacks combine values. **Alternative:** Extend the combination operations through defaulting, inheritance, overrides, and application resolution. This would support progressively extending collections across more of the lifecycle, but requires additional combination, callback, and inspection rules. Supporting typed default lists is a separate scope choice from extending source-backed baseline collections; the application-owned final-result guarantee remains independent of this combination-scope decision.