Add identified record inheritance to the sequence roadmap

docs/roadmap/future.md

@@ -22,6 +22,13 @@ composition. Precedence still replaces sequences as whole values. This entry
 tracks broader identified-record behavior that remains deferred and is not part
 of that implemented contract.
 
+The active [sequence roadmap](roadmap.md) now selects a narrower design for
+collection policy, compound scalar identities, final application-registered
+identity validation, and inheritance within identified sequences. It owns those
+selected behaviors and their remaining decisions. The original broader proposal
+below remains background for further consideration, particularly recursive
+combination of matching record bodies; it does not override the active roadmap.
+
 ### Motivation and proposed behavior
 
 Allow applications to declare that selected sequences contain records identified

docs/roadmap/roadmap.md

@@ -1,9 +1,10 @@
 # Sequence Identity And Combination Roadmap
 
-Status: settled feature roadmap, ready for a staged implementation plan. This
+Status: feature roadmap with the accepted scope extended to inheritance between
-is the targeted end state, not an implemented contract. The subsequent plan will
+identified sequence records. The remaining decision is recorded under
-specify execution order, the exact Go API, internal bookkeeping, and validation
+[Open Decisions](#open-decisions). This is the targeted end state, not an
-stages.
+implemented contract. The subsequent implementation plan will specify execution
+order, the exact Go API, internal bookkeeping, and validation stages.
 
 ## Purpose And Scope
 
@@ -12,7 +13,9 @@ records, declare identity in configuration, and progressively extend a baseline
 collection through higher layers. Authors can replace the entire collection,
 append distinct records, or replace complete records by identity while retaining
 unmatched records. Sibling inheritance uses the same sequence operations and
-identity-based defaults.
+identity-based defaults. Within an identified sequence, a record can explicitly
+inherit settings from another record in that sequence while keeping its own
+identity.
 
 The motivating use case is a prompt catalog whose source-backed baseline supplies
 records and whose application or installation layers add or supersede records.
@@ -26,9 +29,10 @@ CLI application moves later in the loading lifecycle.
 
 This scope is narrower than the deferred
 [identified-record sequences proposal](future.md#1-identified-record-sequences).
-Recursive merging of matching records and record-specific inheritance remain
+Automatic recursive merging of matching records remains outside this scope;
-separate candidates. The initial declaration surface is `configkit.sequences`;
+explicit inheritance between distinct records is included. The initial
-`key` and `mode` shorthand on imports is deferred. Application lists remain
+declaration surface is `configkit.sequences`; `key` and `mode` shorthand on
+imports is deferred. Application lists remain
 ordinary YAML sequences without control wrappers. Unkeyed sequence concatenation
 is outside this feature.
 
@@ -50,6 +54,7 @@ baseline:
 | Strict composition | Registered sequences append distinct string identities and reject duplicates; repeated unregistered sequences conflict. | Collections with effective identity append distinct records and reject duplicates. Applicable lower-tier identity is available before composing higher-tier peers. Unidentified sequences retain strict-conflict behavior. |
 | Source precedence and selected overlays | A present higher sequence replaces the lower sequence wholesale. | Collection policy selects `replace`, `append`, or `upsert`; overlays can declare policy for inline sequences. |
 | Sibling inheritance | An inheriting mapping recursively overlays its resolved parent; supplied sequences replace wholesale. | Mappings retain recursive combination, while sequences use destination policy and the same three operations. |
+| Sequence-member inheritance | A sequence element has no sibling scope for its own inheritance relationship. | An identified sequence provides a sibling scope: a record can inherit from one record in the same sequence, selected by complete identity. |
 | CLI overrides | Apply before inheritance and typed defaults/decoding. | Apply after inheritance and typed defaults/decoding, before application resolution. |
 | Identity enforcement | Registration governs strict sequence composition. | Structural identity checks cover identified combinations; application registrations additionally constrain final effective sequences. |
 
@@ -460,11 +465,13 @@ specified next.
 
 ## Sibling Inheritance
 
-Complete source-tier assembly before resolving inheritance. Retain the existing
+Complete source-tier assembly before resolving inheritance. Retain existing
-dependency-ready algorithm and sibling scope rules. At each inheritance combination,
+mapping-sibling scopes and add identified-sequence scopes as described below.
+Both use the existing dependency-ready algorithm. At each inheritance combination,
 the resolved parent's collection is the lower value and the inheriting mapping's
-local collection is the higher value. Mappings combine recursively; identified
+local collection is the higher value. Mappings combine recursively; upsert within
-sequence records remain whole-record replacements.
+an identified sequence remains whole-record replacement. Explicit record
+inheritance is a separate construction step after those replacements.
 
 Use the inheriting destination collection's policy: an applicable explicit mode,
 including one retained through source tiers, wins. Otherwise infer upsert with
@@ -511,6 +518,85 @@ An omitted local sequence retains the inherited sequence. Explicit `[]` clears i
 under replace and contributes no records under append or upsert. Retain controls
 until inheritance uses them, then exclude them from application decoding.
 
+### Inheritance Between Identified Sequence Records
+
+An identified sequence supplies a sibling scope for its records. Identity can
+come from an application registration or an applicable YAML declaration. A record
+may inherit from one other record in that same sequence using an `extends`
+mapping that selects the parent's complete identity:
+
+```yaml
+configkit:
+  sequences:
+    - destination: [prompts]
+      key: id
+prompts:
+  - id: base
+    model: standard
+    temperature: 0.5
+  - id: concise
+    extends: {id: base}
+    temperature: 0.2
+```
+
+The final `concise` record is:
+
+```yaml
+id: concise
+model: standard
+temperature: 0.2
+```
+
+The `base` record remains in the collection. Parents are ordinary records subject
+to the same decoding and validation as other records; there is no abstract
+template category.
+
+Use mapping-form references for both single and compound keys. With
+`key: [namespace, id]`, a reference can be
+`extends: {namespace: shared, id: base}`. Match components using the collection's
+field-set and scalar-equality rules, including mathematical numeric equality.
+`extends: {id: null}` selects a null-valued identity; `extends: null` cancels a
+relationship. The remaining question about assembling reference mappings is
+recorded under [Open Decisions](#open-decisions).
+
+Every record must supply its complete own identity before inheritance. Preserve
+the child's identity fields and inherit only the parent's nonidentity payload;
+inheritance cannot supply a missing child identity. Child payload fields take
+precedence, mappings combine recursively, and nested sequences use the inheriting
+destination's collection policy. Consume the record's directive before decoding.
+
+Support one parent within the same identified sequence, including forward
+references and chains. Missing parents and cycles, including self-reference,
+fail the load. Cross-collection references, position-based references, multiple
+parents, and inheritance between unidentified sequence members are outside this
+scope. Existing string-form references between mapping siblings retain their
+meaning; they do not become shorthand for sequence identity references.
+
+Resolve relationships against records surviving source-tier assembly and selected
+overlays. A higher layer can replace a parent before inheritance runs; removing
+that parent makes a surviving reference fail. Whole-record upsert discards the
+old record's directive along with its payload. Omitting `extends` in the
+replacement record does not retain the displaced record's relationship. An
+explicit reference to the record's own identity is a cycle, not a way to access
+its lower-tier predecessor.
+
+Reuse Kahn's algorithm with the existing stable canonical semantic-pointer key
+for dependency-ready ordering. Build parent lookup from the sequence's effective
+identity, and preserve the assembled list order independently of resolution
+order. Enclosing relationships resolve before nested relationships. Parent
+templates retain nested instructions for evaluation at each inheriting
+destination, following the existing destination-local inheritance rule. Copied
+collection identity, destination operation selection, and record-owned policy
+transfer follow the rules above.
+
+Record inheritance runs in the existing structural phase, before typed defaults,
+decoding, CLI overrides, and application resolution. Those later stages do not
+trigger another inheritance pass. Preserve inherited presence, origin and
+history evidence, sensitivity, declaring resource contexts, and existing
+expansion budgets just as for mapping-sibling inheritance. Identity lookup must
+not expose identity values in diagnostics. Existing index-based inspection paths
+remain sufficient; no new identity-selector path API is needed.
+
 ## Loading Lifecycle
 
 Use the following order:
@@ -518,8 +604,8 @@ Use the following order:
 1. Assemble source tiers and selected overlays, resolving policy before each
    tier's strict composition and applying collection operations at precedence
    boundaries.
-2. Resolve sibling inheritance, then apply typed defaults and decode the assembled
+2. Resolve sibling inheritance in mappings and identified sequences, then apply
-   configuration.
+   typed defaults and decode the assembled configuration.
 3. Apply explicit CLI overrides to that configuration and check supplied values
    against the application schema.
 4. Run application resolution against the overridden configuration.
@@ -590,6 +676,10 @@ stages can produce duplicate identities or otherwise violate a YAML-only identit
 Applications needing a final guarantee register the constraint through the
 application-owned interface.
 
+Adding record inheritance does not expand this guarantee: a YAML declaration
+can enable parent lookup without imposing final validation on values later
+produced by defaults, CLI overrides, or application resolution.
+
 ### Final Application Constraints
 
 On successful loading, final effective collections covered by application
@@ -597,6 +687,10 @@ registrations satisfy their declared identity constraints. Retain assembly-time
 checks and validate final effective sequences before publication, after defaults,
 inheritance, overrides, application resolution, and secret lookup.
 
+ConfigKit performs these identity checks; consumers register the constraint
+without needing to repeat duplicate detection in `Definition.Validate`.
+Application-owned semantic validation continues to cover domain-specific rules.
+
 This guarantee covers actual sequences, not path existence or allowed shape.
 It does not require every intermediate callback state to satisfy the constraint
 and does not cover caller mutations after loading returns. Final checks examine
@@ -643,8 +737,12 @@ order and test sequencing defined there:
   lower-tier structural failures.
 - **Combination and inheritance:** implement the three operations with strict peer
   handling, whole-record replacement, presence rules, stable ordering, and the
-  destination policy rules for inheritance. Preserve provenance, history,
+  destination policy rules for inheritance. Extend sibling-scope discovery and
-  sensitivity, paths, and resource contexts as records survive, move, or disappear.
+  parent lookup to identified sequences, validate mapping-form references, and
+  preserve child identities while copying parent payloads. Reuse the shared
+  dependency-ready scheduler and destination-local nested resolution. Preserve
+  provenance, history, sensitivity, paths, and resource contexts as records survive,
+  move, or disappear.
 - **Lifecycle and validation:** move CLI application after inheritance and typed
   defaults/decoding, preserve explicit-presence and source metadata for application
   resolution, reject CLI structural directives, and add final application identity
@@ -653,9 +751,12 @@ order and test sequencing defined there:
 - **Documentation and verification:** exercise ordinary and layered loading,
   imported and inline collections, item-local scopes, version-header consumption,
   selected overlays, inheritance, typed defaults, CLI, callbacks, and inspection.
+  Cover single and compound parent references, scalar equality, forward references
+  and chains, missing parents and cycles, parent replacement/removal, child identity
+  preservation, nested destination policy, cancellation, and stable output order.
   Update the public contract and consumer examples when behavior lands.
 
-The durable architectural changes are accepted in
+The collection-policy and CLI-ordering decisions are accepted in
 [ADR 0017](../adr/0017-collection-sequence-policy.md) for collection policy and
 [ADR 0018](../adr/0018-cli-after-structural-resolution.md) for CLI ordering.
 Their scoped supersession is recorded in the affected earlier ADRs, and the
@@ -664,6 +765,44 @@ These govern the target architecture; the configuration contract continues to
 describe implemented behavior until the feature lands. This roadmap does not
 select migration machinery.
 
-No open feature decisions remain. The implementation plan should translate this
+The added identified-sequence sibling scope requires an architectural follow-up
-end state into reviewable stages without adding new guarantees, restrictions, or
+before implementation: record the extension to the scope retained by
+[ADR 0009](../adr/0009-sibling-inheritance.md) and ADR 0017, and align the
+architecture policy. Their existing scope statements do not yet describe this
+addition. Keep the final application-registration guarantee unchanged.
+
+After settling the decision below, the implementation plan should translate this
+end state into reviewable stages without adding guarantees, restrictions, or
 combination semantics.
+
+## Open Decisions
+
+### Assembly And Validation Of Parent References
+
+The effective reference must identify one parent by its complete identity. Should
+each supplied reference be complete independently, or may item-local helper
+imports compose its fields before inheritance resolves it?
+
+**Recommendation:** use ordinary strict mapping composition for the reference,
+then require the effective mapping to contain exactly the collection's identity
+fields, with scalar values, before parent lookup. Disjoint helper contributions
+may assemble a compound reference; overlapping fields conflict under the usual
+strict rules, even when equal. This follows the existing ability of helpers to
+assemble a record and its identity, and avoids special atomic-composition rules
+for this one mapping. A partial contribution alone cannot select a parent.
+Authors must understand that disjoint contributions form one reference.
+
+Whole-record upsert still replaces the directive with its containing record,
+so this choice does not merge old and new reference components across matching
+records in different tiers. It also does not permit strict peers to contribute
+duplicate record identities. Validate the surviving effective reference in the
+inheritance phase, consistent with existing treatment of superseded relationships.
+
+**Alternative:** treat each non-null reference mapping as one complete, atomic
+selector. Require exactly the identity fields in each declaration; two supplied
+selectors conflict under strict composition even if their fields are disjoint.
+This makes the relationship self-contained and prevents helpers from accidentally
+constructing an unintended compound parent identity. It needs special treatment
+of directive mappings and prevents intentional composition that ordinary record
+fields support. Keep whole-record replacement and active-relationship timing
+unchanged under either choice.