Clarify v0.9.0 import and collection migration guidance

docs/development.md

@@ -167,6 +167,21 @@ The subsequent import-integration audit and completed remediation record are
 preserved at `6903c9e:docs/roadmap/remediation.md`; retrieve them with
 `git show 6903c9e:docs/roadmap/remediation.md`.
 
+The completed sequence-identity/collection-policy roadmap and implementation
+plan are preserved in Git history at `d0a605a`:
+
+```sh
+git show d0a605a:docs/roadmap/sequences.md
+git show d0a605a:docs/roadmap/implementation.md
+```
+
+Current behavior belongs to the configuration contract's
+[strict composition](config.md#strict-document-composition),
+[sibling inheritance](config.md#sibling-inheritance), and
+[CLI overrides](config.md#explicit-cli-overrides) sections.
+The completed remediation record, including the algorithm-audit findings, is
+preserved at `ea46001:docs/roadmap/remediation.md`.
+
 Potential enhancements are tracked in [future features](roadmap/future.md).
 
 Install the Go version required by `go.mod` or a compatible newer toolchain.

docs/releases/v0.9.0.md

@@ -0,0 +1,111 @@
+# ConfigKit v0.9.0
+
+This minor release expands import selection, adds author-declared collection
+identity and operations, and supports inheritance between identified records.
+It also moves CLI assignments after inheritance and typed decoding. These changes
+include breaking API and configuration behavior changes from v0.8.0; review the
+migration guidance below before upgrading.
+
+## Changes
+
+- Extend imports with named-file and directory lists, directory filename patterns,
+  exclusions, recursion and depth controls, and explicit hidden-file and symlink
+  discovery controls. Add `ImportDefaults` to ordinary and layered options, with
+  authored declarations taking precedence over application defaults. Imports are
+  optional by default. `SourceInfo.ImportSelections` exposes completed optional
+  declarations that selected no sources. See the
+  [import contract](../config.md#explicit-imports) and
+  [consumer guidance](../consumers/README.md#import-defaults-and-selection-inspection).
+- Use shared dependency-ready ordering for import composition and sibling
+  inheritance. Dependencies complete before dependents, with deterministic
+  selection among currently eligible contributions. Source acquisition and
+  callback order remain separate from assembly order. See the
+  [ordering decision](../adr/0015-shared-dependency-ready-ordering.md).
+- Add root `configkit.sequences` declarations for collection identity and
+  `replace`, `append`, or `upsert` operations, including in selected overlays.
+  Strict peers combine distinct identities and reject duplicates; precedence
+  and inheritance apply the destination's collection operation. Upsert replaces
+  a whole matching record at its retained position. Identity supports compound
+  field sets and scalar strings, numbers, booleans, and null. See the
+  [collection contract](../config.md#strict-document-composition).
+- Add `Definition.SequenceIdentities`, exact `SequenceIdentityAt` registrations,
+  and dynamic `SequenceIdentityMatching` selectors. Application registrations
+  govern structural collection operations and check existing matching final
+  typed sequences, including values from defaults, CLI assignments, and
+  application resolution. YAML-only declarations govern structural operations
+  without adding that final-result check.
+- Support record inheritance within an identified sequence using an `extends`
+  mapping containing the parent's complete identity. The child retains its own
+  identity, relationships use surviving assembled records, and output sequence
+  order is preserved. See [sibling inheritance](../config.md#sibling-inheritance)
+  and the maintained [sequence catalog](../consumers/README.md#identified-prompt-sequences).
+- Apply explicit CLI overrides to the decoded configuration before application
+  resolution. Mappings retain omitted fields and their presence metadata;
+  sequences replace whole lists independently of collection operations. See the
+  [CLI contract](../config.md#explicit-cli-overrides).
+- Correct native import alias and contained-symlink handling, source-version
+  path projection, and provenance/history through collection movement,
+  replacement, inheritance, and CLI assignment. Reduce repeated scans,
+  accumulated policy copying, and sparse-projection storage during construction.
+
+## Compatibility and upgrade
+
+This pre-v1 minor release intentionally changes the v0.8.0 contracts:
+
+1. **Migrate import requiredness.** The YAML `optional` field is removed and is
+   rejected. Replace `optional: true` with `required: false`; omission is equivalent
+   when the application does not default imports to required. Replace
+   `optional: false` with `required: true`. Imports that previously relied on
+   implicit requiredness now need `required: true`, or an application
+   `ImportDefaults.Required` default set to true. Review discovery filters:
+   `follow` and `hidden` default to true, and directory searches select all
+   filenames unless a pattern is supplied.
+2. **Replace sequence registrations.** `SequenceComposition` and
+   `Definition.SequenceCompositions` are removed without compatibility aliases.
+   For a previous registration at `prompts` using `id`, use:
+
+   ```go
+   rules.SequenceIdentities = []configkit.SequenceIdentity{
+       configkit.SequenceIdentityAt(configkit.Keys("prompts"), "id"),
+   }
+   ```
+
+   These registrations now also enforce identity in final typed sequences. Fix
+   missing or duplicate identities introduced by defaults, CLI assignments, or
+   application resolution. Registration does not require a collection to exist
+   or determine its allowed shape; schema and application validation retain
+   those responsibilities.
+3. **Review collection precedence and ordering.** An identified collection now
+   defaults to upsert across layers and inheritance when no explicit mode
+   applies, rather than unconditional whole-sequence replacement. Declare
+   `mode: replace` at the relevant destination where replacement is intended.
+   With append or upsert, `[]` contributes no records and does not clear the
+   collection; null is invalid. Use `mode: replace` with `[]` to clear it.
+   An explicit mode persists at its destination until changed; a parent's mode
+   does not transfer to a different inheriting destination. Unidentified
+   sequences retain strict peer conflicts and whole-value precedence
+   replacement. Dependency-ready assembly can change sequence positions and
+   diagnostic or history order compared with the former traversal order.
+4. **Review CLI recovery and inheritance assumptions.** CLI overrides can no
+   longer repair values that fail earlier decoding, bypass source or inheritance
+   errors, or add or cancel inheritance relationships. Supply structural
+   directives in source configuration. Application resolution still receives
+   explicit CLI values and may record subsequent normalization or derivation.
+5. **Update affected Go literals and diagnostic handling.** Use keyed literals
+   for expanded option and source-information structs. Collection validation adds
+   `CodeSequenceInvalid` (`sequence.invalid`) and `CodeSequenceConflict`
+   (`sequence.conflict`); update diagnostic handling where applicable.
+
+Select this version and rebuild:
+
+```sh
+go get gitea.maximumdirect.net/eric/ConfigKit@v0.9.0
+```
+
+The consumer minimum remains Go 1.26.0. The YAML dependency remains
+`go.yaml.in/yaml/v4 v4.0.0-rc.6`, and the maintainer tool dependency remains
+ReleaseKit v0.4.1. Linux amd64 and arm64 remain release-validated targets;
+macOS remains best-effort. Publication follows the existing
+[release procedure](../release.md).
+
+[COPYRIGHT](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.9.0/COPYRIGHT) | [LICENSE](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.9.0/LICENSE) | [ATTRIBUTION](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.9.0/docs/oss/attribution.md)

docs/roadmap/complexity.md

@@ -161,9 +161,10 @@ syntax, dependencies, or complexity-based gates.
 
 ## Relationship To Remediation And Other Work
 
-[Remediation](remediation.md) owns the identified correctness and algorithmic
-efficiency findings. Several overlap these files, particularly item-scope
-application matching and inheritance policy localization.
+The completed remediation record at `ea46001:docs/roadmap/remediation.md`
+documents the identified correctness and algorithmic efficiency findings.
+Several overlap these files, particularly item-scope application matching and
+inheritance policy localization.
 
 Use the documented contract and completed remediation as the behavioral baseline.
 Keep corrective changes reviewable under their findings, and retain their

docs/roadmap/remediation.md

@@ -1,905 +0,0 @@
-# Sequence Implementation Review Remediation
-
-Status: findings 1–17 are complete. The algorithm audit below identifies six
-additional open findings, numbered 18–23. This record introduces no new release
-gate.
-
-Reviewed commit: `0be7cdb00eff576fda6b30c0600b2134d4e2e086`.
-Implementation range: `bc6b977..0be7cdb`.
-Authority: the [sequence roadmap](sequences.md),
-[implementation plan](implementation.md),
-[ADR 0017](../adr/0017-collection-sequence-policy.md), and
-[ADR 0018](../adr/0018-cli-after-structural-resolution.md).
-
-The public facade, shared identity/composition machinery, scoped policies,
-identified-record inheritance, post-decode CLI phase, and final registered
-identity validation are present. The findings below concern remaining
-correctness, contract-alignment, documentation, and scaling gaps.
-
-Validation on the reviewed commit:
-
-- `./scripts/check.sh` passed with Go 1.26.10 on Linux.
-- `go test -race ./...` passed.
-- Public-API probes outside the repository reproduced findings 1–9. The probes
-  used labeled byte sources, defaults, and an in-memory filesystem; they did not
-  change implementation files or use external services.
-- Finding 10 follows directly from the maintained text; finding 11 follows from
-  the prefix-scan loop, not a timing comparison or performance threshold.
-
-Validation after the initial remediation: `./scripts/check.sh` and
-`go test -race ./...` passed with Go 1.26.10 on Linux. The original public-API
-probes for findings 1–9 now demonstrate the required corrected behavior.
-
-Validation after integration remediation: the everyday check, full race suite,
-and local documentation/public-boundary checks passed with the same toolchain.
-The original and follow-up public-API probes demonstrate corrected behavior for
-findings 1–9 and 12–15, including unrelated underlay payload at an old version
-index. The accumulated-copy correction is verified by code inspection and
-ownership regressions; the contract corrections are verified against the
-implemented operations and replay. Each additional finding has its own completion
-record and commit.
-
-P1 denotes the highest-priority correctness issue here; P2 denotes normal-priority
-work needed to complete the selected behavior. Preserve the agreed scope while
-fixing these issues. In particular, do not resolve failures by restricting valid
-identity fields, requiring repeated declarations, changing merge semantics, or
-adding limits.
-
-## 1. Preserve Retained Explicit Values In Nested CLI Mapping Assignments — P1
-
-Status: [x] Complete.
-
-Completion: override metadata now reads the actual target subtree through one
-shared lookup. Ordinary/layered regressions preserve retained values, including
-zero and false, their explicit origins, and absence-based resolver behavior.
-Verified `go test ./internal/engine -run 'Override|CLI' -count=1` and
-`./scripts/check.sh`.
-
-Owner: [`applyDecodedOverrides`](../../internal/engine/override_decode.go),
-especially lines 59–60 and the presence/origin update helpers.
-
-The helpers receive the complete configuration projection as `existing`, but
-interpret it as the subtree at the override path. A partial nested mapping
-therefore treats retained input children as absent and drops their presence and
-origins. `fillDefaultOrigins` subsequently labels those retained values as defaults.
-
-Reproduction: load the following into ordinary nested Go structs:
-
-```yaml
-settings:
-  nested:
-    a: 1
-    b: 2
-```
-
-Apply an override at `[settings]` with value
-`map[string]any{"nested": map[string]any{"a": 3}}`. The typed value of `b`
-initially remains 2, but `Metadata.Lookup([settings,nested,b])` reports
-`Explicit: false` and a default origin. Calling `Default` for that field in
-application resolution then changes the explicitly supplied 2 to the fallback
-value. This violates presence-based defaulting, not merely presentation.
-
-Required remediation: use the actual existing target subtree when updating
-presence/origins, preserving retained descendants and removing genuinely replaced
-ones. Keep override batches detached and preserve the selected post-decode order.
-Add regressions for nested partial mappings, retained zero/false values, metadata,
-and an absence-based resolver default that must not overwrite retained input.
-
-## 2. Read Upsert Origins At Input Paths And Write Them At Output Paths — P2
-
-Status: [x] Complete.
-
-Completion: sequence origin copying now separates incoming and effective paths
-for precedence and inheritance. A regression covers reordered matches, appended
-records, composite origins, and actual resource reads from distinct declaring
-directories. Verified focused origin/inheritance/overlay/provenance tests,
-`go test -race ./internal/engine`, and `./scripts/check.sh`.
-
-Owners: [`applyOverlaySequenceEvidence`](../../internal/engine/overlay.go),
-particularly lines 369 and 381, and the analogous sequence-origin copying in
-[`inheritance_collection.go`](../../internal/engine/inheritance_collection.go).
-
-The overlay helper supplies an output-index path to an origin lookup whose keys
-are still incoming-index paths. When those positions differ and another incoming
-record occupies the output index, metadata takes that other record's source
-location. The same defect was reproduced through mapping-sibling inheritance;
-its helper uses the same single-path copying pattern.
-
-Reproduction: an underlay supplies IDs `[a,b]`; the main source declares `key: id`
-and supplies `[b*,a*]`. The result is correctly `[a*,b*]`, but the value origin
-for final `a*` is the source line of `b*`, and vice versa. In the probe the two
-value lines were 7 and 9; final index 0 incorrectly reported line 7 instead of 9.
-With separately authored records this can select a different source/context.
-
-Required remediation: use the operation's input-to-output projection for origin
-subtree reads as well as writes. Preserve composite contributing origins, not
-just node-location fallbacks. Test reordered upserts and appended records with
-distinct source files/locations, including nested fields, inheritance operations,
-and declaring resource contexts.
-
-## 3. Apply Tier Sequence Operations To Historical Contributions — P2
-
-Status: [x] Complete.
-
-Completion: per-source history now composes strict item projections with the
-tier operation, including nested application prefixes and local sequence replay.
-Regressions cover all three operations, multiple peers, retained records, removed
-fields, source attribution, and a subsequent overlay without a false decoding
-event. Verified focused history tests, `go test -race ./internal/engine`, and
-`./scripts/check.sh`.
-
-Owner: [`applyOverlayWithCollectionPolicy`](../../internal/engine/overlay.go),
-lines 76–84, and existing sequence-history projection/replay.
-
-For composed tiers, `applications` is nonempty. This branch appends the tier's
-original strict-composition applications without incorporating the newly computed
-precedence sequence descriptors. Replay consequently performs a whole-list
-replacement or strict append rather than the actual cross-tier upsert/append.
-The later decoding reconciliation conceals the mismatch by attributing a
-structural combination to typed defaults/decoding.
-
-Reproduction: lower `[a,b]` and higher `[b*,a*]` correctly yield `[a*,b*]`.
-`Explain([items])` instead shows the main contribution producing `[b*,a*]`, then
-a `decoding` event reordering it to `[a*,b*]` with the reason "applied retained
-defaults and canonical typed decoding". No decoder performed that reordering.
-
-Required remediation: project each retained per-source application through the
-tier's selected operation while keeping peer composition distinct from tier
-precedence. Do not collapse authored sources or fake decoding events. Cover
-upsert, append, unmatched retained records, reordered matches, removed fields,
-multiple higher-tier peers, and a subsequent selected overlay. Replay should
-agree with structural assembly before real decoding reconciliation.
-
-## 4. Reproject Nested Source Destinations After Cross-Tier Item Movement — P2
-
-Status: [x] Complete.
-
-Completion: layered assembly projects incoming source destinations and version
-classification evidence with the same item assignments used for values/history.
-Earlier contributions retain their historical coordinates, physical identities
-and contexts. Main/underlay regressions cover reordered upserts, append, replace,
-nested helpers, and source-local version evidence. Verified focused destination,
-origin and source-version tests, focused race coverage, and `./scripts/check.sh`.
-
-Owners: [`projectAcquiredDestinations`](../../internal/engine/imports.go) and the
-sequence-operation handoff in [`assembleLayeredInput`](../../internal/engine/load_request.go).
-
-Source destinations are projected while composing their own tier, but are not
-reprojected when that tier's records move to accumulated-list positions. Nested
-helper source records can therefore point at a different final record.
-
-Reproduction: lower `items` contains `[a,b]`. Main imports a single `b` record
-with `placement: sequence-item` into `[items]`; that record imports a helper
-mapping supplying its value. Upsert retains `a` at index 0 and places the higher
-`b` at index 1. `Sources()` nevertheless reports the surviving helper's destination
-as `/items/0`, its tier-local position, rather than `/items/1`.
-
-Required remediation: carry the sequence operation's projection through nested
-source-destination/evidence updates at precedence boundaries. Keep physical
-source identity and passive resource contexts unchanged, and preserve historical
-meaning for displaced contributions rather than relabeling them as survivors.
-Test a retained lower record followed by an imported higher record, reordered
-matches, and nested helpers. Check version-evidence projection where applicable.
-
-## 5. Honor Identity Reset On Actual Inheritance Replacement — P2
-
-Status: [x] Complete.
-
-Completion: inheritance resolves identity against actual child-value presence,
-discarding parent YAML identity for an explicit replacement while preserving
-agreement for retained values and authoritative application constraints. Added
-policy-state and public-load regressions for changed/unkeyed/non-sequence
-replacements, application agreement/conflict, and mode-only/append/upsert cases.
-Verified focused inheritance tests and `./scripts/check.sh`.
-
-Owners: [`inheritanceCollectionResolution` and `mergeInheritanceCollectionPolicies`](../../internal/engine/inheritance_collection.go),
-especially lines 79–80 and 121.
-
-These paths reconcile the parent's YAML identity before accounting for actual
-replacement, and treat the parent's collection as retained even when the child
-supplies a replacing value. This is stronger than the selected identity lifetime
-and differs from source-tier replacement.
-
-Reproduction:
-
-```yaml
-configkit:
-  sequences:
-    - destination: [base, items]
-      key: id
-    - destination: [child, items]
-      key: name
-      mode: replace
-base:
-  items:
-    - id: a
-      value: parent
-child:
-  extends: base
-  items:
-    - name: b
-      value: child
-```
-
-Loading fails with `sequence.conflict` at `/child/items`. It should succeed:
-actual whole-sequence replacement may discard the parent's YAML identity and
-establish `name`. Authoritative application constraints must still apply.
-
-Required remediation: distinguish actual replacement from a mode-only declaration
-before identity agreement and transfer. Test a changed key, replacement with no
-new key becoming unkeyed, permitted non-sequence replacement, and an agreeing or
-conflicting application registration. Preserve rejection of identity changes
-without a replacement value and during append/upsert.
-
-## 6. Enforce Collection Operations Beneath Newly Added Ancestors — P2
-
-Status: [x] Complete.
-
-Completion: copying a mapping now visits nested collection operations with absent
-lower values, preserving ordinary ancestor removal and whole-record semantics.
-Incoming record-owned policy transfers through the same operation. Regressions
-cover absent/empty/null ancestors, identity and kind failures, empty and valid
-initialization, and later inheritance using an imported item's nested identity.
-Corrected an older test that deferred the required error until a later tier.
-Verified focused merge/collection/inheritance tests, focused ownership race
-coverage, and `./scripts/check.sh`.
-
-Owners: [`mergeRecursiveMappings`](../../internal/engine/recursive_merge.go) and
-[`mergeOverlayMappingsWithCollectionPolicy`](../../internal/engine/overlay.go).
-
-When an incoming ancestor mapping is absent below, the recursive merger clones
-the entire subtree and invokes only its ancestor-level added hook. Nested
-collection resolutions are not visited. This bypasses their operation rules;
-the subsequent declaration-only policy update does not replace those checks.
-
-Reproduction: combine a lower `{}` with this main tier:
-
-```yaml
-configkit:
-  sequences:
-    - destination: [group, items]
-      mode: append
-group:
-  items:
-    - id: a
-```
-
-Loading succeeds despite append having no identity definition. Placing the same
-sequence directly at `[items]` correctly fails. The validity of an operation
-must not depend on whether an enclosing mapping already existed.
-
-Required remediation: account for applicable nested collection policies when
-adding/copying an ancestor subtree, including owned-policy transfer. Test absent
-and present enclosing mappings, append/upsert without identity, explicit empty
-lists, invalid non-sequence contributions, and valid identified initialization.
-Do not turn this into a general path-existence or shape constraint: ordinary
-ancestor replacement/removal retains its documented meaning.
-
-## 7. Validate Final Typed Identity Fields As Payload, Not Directives — P2
-
-Status: [x] Complete.
-
-Completion: shared identity lookup now distinguishes structural records from
-effective payload/reference mappings. Final validation includes literal `extends`
-fields while structural source checks still exclude actual directives. Facade
-regressions cover defaults and recorded resolution, string/null components, and
-duplicate rejection. Verified those regressions and existing structural identity
-and inheritance coverage through `./scripts/check.sh`.
-
-Owners: [`validateFinalSequenceIdentities`](../../internal/engine/final_identity.go)
-and [`newSequenceMappingIndex`](../../internal/engine/sequence_identity.go).
-
-The final validator reuses structural record lookup, which excludes every
-`extends` entry. At this phase there is no structural processing: typed defaults
-and application resolution can produce ordinary payload with that literal name.
-The roadmap expressly distinguishes structural-control exclusion from final
-typed-payload validation; this reuse introduces an extra restriction.
-
-Reproduction: a defaults-only `Catalog` with `Items []map[string]any` containing
-`{"extends": "application-data"}`, registered with
-`SequenceIdentityAt(Keys("items"), "extends")`, fails with a missing identity
-field at `/items/0/extends`, even though the effective field is present.
-
-Required remediation: distinguish structural-payload lookup from final payload
-lookup. Preserve rejection when a source tries to use an actual consumed
-directive as identity. Test final fields supplied by defaults and recorded
-resolution, including a null component, without adding a broader name ban.
-
-## 8. Retain Nonfatal Main-Version Diagnostics In Layered Loads — P2
-
-Status: [x] Complete.
-
-Completion: layered assembly returns nonfatal main-version diagnostics on success
-and alongside later failures; the shared caller stops only for errors. Regression
-coverage verifies ordinary/layered warning parity, exactly one checker call, and
-warnings retained with later structural or decoding errors. Verified through
-`./scripts/check.sh`, including existing source-version ordering coverage.
-
-Owner: [`assembleLayeredInput`](../../internal/engine/load_request.go), lines 143–152.
-
-The moved default-mode version-check call returns its diagnostics only when they
-contain errors. Warning-only diagnostics are discarded. Ordinary loading still
-retains the same warnings.
-
-Reproduction: a `VersionRule.Check` that calls
-`validation.Warning(Keys("version"), "test warning")` and returns nil yields one
-`version.warning` through ordinary `Load`, but zero diagnostics through
-`LoadLayered` with the same main input and default source-version mode.
-
-Required remediation: carry warnings from the single main-tier check into the
-shared diagnostic accumulator on successful layered loads. Preserve checker
-ownership, invocation count, and the selected acquisition/composition ordering.
-Add ordinary/layered parity and warning-plus-later-error regressions.
-
-## 9. Identify Conflicting Application Registrations In Diagnostics — P2
-
-Status: [x] Complete.
-
-Completion: definition validation retains original one-based registration
-positions and names both positions in each overlap diagnostic. No input path or
-new public diagnostic structure is invented. A mixed-selector regression verifies
-multiple conflicts even after an earlier invalid registration is skipped.
-Verified registration/facade coverage through `./scripts/check.sh`.
-
-Owner: [`compileSequenceIdentityPolicy`](../../internal/engine/sequence_registration.go),
-lines 119–128.
-
-Incompatible dynamic overlaps produce only a generic `options.invalid` message,
-an empty configuration path, and no related information. Neither registration
-is identified. The roadmap requires definition-time errors to identify the
-conflicting registrations without inventing a concrete input destination.
-
-Reproduction: register `collections -> AnyMappingKey -> items` using `id`, and
-`collections -> foo -> items` using `name`. Even with `NoSource`, the error is
-only "overlapping sequence registrations must use the same identity fields".
-With multiple registrations the caller cannot identify the conflicting pair.
-
-Required remediation: retain original registration positions and identify both
-in safe definition diagnostics, for example through positional message context.
-No new public diagnostic structure is required solely for this. Test mixed exact
-and dynamic registrations and multiple conflicts, without asserting complete
-English wording or exposing identity values.
-
-## 10. Remove Stale Public Contract Descriptions — P2
-
-Status: [x] Complete.
-
-Completion: the public definition comment now describes final registered-identity
-validation and its shape/lifetime boundaries. Selected-overlay prose explicitly
-separates ordinary replacement from collection policy and links to the canonical
-operation/presence tables. Verified against the implementation, local-link tests,
-and `git diff --check`; no runtime behavior changed.
-
-Owners: [`Definition.SequenceIdentities`](../../lifecycle.go), lines 18–21, and
-the [selected-overlay contract](../config.md#selected-overlays).
-
-The exported definition-field comment still says registrations do not validate
-identities introduced by defaults, CLI, or callbacks. Final validation is now
-implemented and the registration constructors correctly document it.
-
-The selected-overlay table and following paragraph also still unconditionally
-describe sequences as atomic replacements and `[]` as clearing. A later paragraph
-describes collection operations, but does not make the earlier unconditional
-claims accurate: identified collections infer upsert, and `[]` then contributes
-no records rather than clearing.
-
-Required remediation: make the symbol comment describe its implemented final
-guarantee and boundaries, and qualify the overlay table/paragraph for collection
-policy. Link to the canonical collection tables rather than duplicating them.
-Keep YAML-only enforcement distinct from registered final validation.
-
-## 11. Remove The Quadratic Prefix Scan In Item Policy Projection — P2
-
-Status: [x] Complete.
-
-Completion: assigned sequence ranges now supply item offsets directly. Handoffs
-without a range lazily accumulate prefix lengths, visiting each preceding
-contribution once rather than rescanning every prefix. Regression coverage keeps
-inline and policy-less contributions in the correct positions with complete,
-partial, or absent ranges. Existing ownership/placement tests remain intact.
-Verified focused item-policy tests, `./scripts/check.sh`, `go test -race ./...`,
-and inspection of the remaining error-only prefix-scan callers. No new benchmark
-fixture or performance threshold was needed for this bounded change.
-
-Owner: [`projectNestedCollectionPolicies`](../../internal/engine/imports.go),
-especially line 968; the scanned helper is `priorSequenceLength` in the same file.
-
-For every item with local collection policy, the loop scans all preceding
-contributions to calculate `itemIndex`, then overwrites that result when the
-already-indexed sequence range is available. For N such items this performs
-`0 + 1 + ... + (N-1)` contribution visits, excluding path lookup costs, despite
-the common case already having the exact assigned index. This is introduced
-quadratic work in the feature's independently authored-record path.
-
-Required remediation: use the existing assigned range first. If a genuine
-fallback is necessary, maintain per-destination prefix lengths in one pass
-rather than rescanning every prefix. Preserve scope and ordering semantics;
-do not add limits, pooling, or a new public cache. Keep ownership/placement
-regressions and add a representative many-item/local-policy benchmark only if
-needed to exercise the path. A wall-clock threshold is not required.
-
-## Integration Audit Baseline And Scope
-
-Reviewed commit: `8d40983271de5239312eb0a2896737be88c5abdc`.
-
-This follow-up reviews the interaction of collection policy, identified-record
-inheritance, dynamic registrations, and post-decode CLI assignment with existing
-source placement, source-version checking, typed decoding, inspection, resource
-contexts, and ownership boundaries. The accepted ADRs and current configuration
-contract remain the authority; the findings require integration corrections,
-not additional consumer restrictions or new feature choices.
-
-Validation:
-
-- `./scripts/check.sh` and `go test -race ./...` pass with Go 1.26.10 on Linux.
-- Isolated public-API probes outside the repository reproduce findings 12–15.
-  They use labeled byte sources and an in-memory filesystem, with no external
-  services, real credentials, or repository implementation changes.
-- Finding 16 follows from the per-item deep-copy loop; no timing claim or
-  performance threshold is needed. Finding 17 follows from maintained contract
-  text that contradicts the implemented collection operations.
-- Positive comparisons include sequence-valued identity replacement, ancestor
-  retirement through source tiers, quoted versions after item movement, and an
-  append overlay that preserves the main record's version.
-
-## 12. Complete Identity Retirement Across Replacement Shapes — P2
-
-Status: [x] Complete.
-
-Completion: source composition and inheritance now prepare eligible lower policy
-from actual replacement paths before agreement. Removed identity and record state
-cannot constrain a new pending key or reappear in later inheritance. Shared mode
-selection preserves explicit-mode persistence and authoritative registrations.
-Regressions cover null/scalar/mapping/list replacement, ancestor removal with a
-new key, reintroduction, and retained/application-constrained conflicts. Verified
-focused retirement/inheritance tests, engine ownership race coverage, and
-`./scripts/check.sh`.
-
-Owners: [`resolveStrictCollectionPolicies`](../../internal/engine/compose.go),
-especially its `sequencePresent` reset condition;
-[`mergeInheritanceCollectionPolicies`](../../internal/engine/inheritance_collection.go);
-and ancestor-replacement handling in
-[`applyInheritanceScopeTemplates`](../../internal/engine/inheritance_schedule.go).
-
-Policy agreement and retirement still treat some actual replacements as retained
-collections. This has three observable effects under the same selected lifetime
-rule:
-
-1. Source precedence accepts a replacement sequence with a new YAML key, but
-   rejects the equivalent replacement by null or a scalar. For example, a lower
-   tier declaring `key: id` for `[items]` with `items: [{id: old}]`, followed by
-   a main tier declaring `key: name, mode: replace` with `items: null`, fails with
-   `sequence.conflict` at `/items`. Replacing null with `42` also fails; replacing
-   it with `[{name: new}]` succeeds. Null/scalar replacement should discard the
-   lower identity and retain the newly declared key as pending policy, subject
-   to the application's schema and constraints.
-2. Removing an ancestor while declaring a new pending key for its removed
-   descendant fails before the replacement occurs. A lower
-   `group: {items: [{id: old}]}` with key `id`, followed by `group: null` and a
-   root declaration of key `name` for `[group, items]`, fails at `/group/items`.
-   The analogous `base`/`child` inheritance case fails at `/child/group/items`.
-   The old collection is being removed rather than retained.
-3. Inheritance can resurrect removed identity even without a conflicting new
-   declaration. This input fails with a missing `id` at
-   `/grandchild/group/items/0/id`:
-
-   ```yaml
-   configkit:
-     sequences: [{destination: [base, group, items], key: id}]
-   base: {group: {items: [{id: old}]}}
-   child:
-     extends: base
-     group: null
-   grandchild:
-     extends: child
-     group: {items: [{name: new}]}
-   ```
-
-   The same remove/reintroduce operations through source tiers succeed. The
-   inherited descendant identity remains in policy state after its enclosing
-   mapping is removed, so the grandchild incorrectly receives the old key.
-
-Required remediation: account for actual value and ancestor replacement before
-deciding which lower identity participates in agreement, and retire removed
-descendant/record-owned state during inheritance as during source precedence.
-Retained inheritance templates must not reestablish discarded identity later.
-Preserve authoritative application constraints, independently owned mode
-persistence, pending declarations, and rejection of key changes when values are
-genuinely retained. Cover direct non-sequence replacement, ancestor removal with
-a new pending key, and removal followed by reintroduction across both contexts.
-Do not require repeated keys or narrow permitted null/scalar shapes to avoid
-these cases.
-
-## 13. Keep Metadata History In Post-Decode CLI Order — P2
-
-Status: [x] Complete.
-
-Completion: metadata history is assembled from the pre-CLI source view, with CLI
-events appended separately; inheritance is projected before CLI and resolution.
-Effective fields still describe post-CLI values. Regressions verify chronological
-origins and resolver dependencies for ordinary/layered mapping and record
-inheritance, including equal-valued assignments. Verified history/override
-coverage and `./scripts/check.sh`.
-
-Owners: [`historyWithInheritance`](../../internal/engine/inheritance_history.go)
-and the source/CLI metadata handoff in
-[`preparePostDecodeOverrideContext`](../../internal/engine/override_context.go).
-
-The value explanation follows the new lifecycle, but `Metadata.History` still
-inserts inheritance after stored CLI changes. `historyWithInheritance` splits
-only at `LayerResolution`, reflecting the former CLI-before-inheritance order.
-The reconstructed source view can also append an authored source origin after
-the CLI event.
-
-Reproduction: load `base: {value: parent}` and
-`child: {extends: base, value: authored}`, then override `[child, value]` with
-`"cli"`. The final value and `Metadata.Lookup` correctly report CLI provenance.
-`Explain` reports source → inheritance → CLI. `Metadata.History` instead reports
-source → CLI → source → inheritance and ends with a source/inheritance origin.
-This violates chronological metadata history and disagrees with the effective
-origin and the value explanation.
-
-Required remediation: assemble metadata history in the implemented source,
-inheritance, CLI, and resolution order, preserving the real origins and
-dependencies of those events. Cover ordinary/layered loading, mapping and record
-inheritance, equal-valued CLI writes, and a later recorded resolver change at the
-appropriate existing test owners. Do not impose a new exact event-count or
-message-format contract.
-
-## 14. Carry Main-Version Coordinates Through Sequence Movement — P2
-
-Status: [x] Complete.
-
-Completion: the effective version rule now follows the main declaration through
-strict placement, tier combination, and inheritance without changing source-local
-checker/evidence paths. Underlay and overlay ownership checks use assigned record
-positions, including whole-record removal; integer conversion remains confined
-to the actual main header. Regressions cover mounted/indexed paths, appended and
-matched records, integer/string parity, unrelated version payload, imported
-underlay claims, and inheritance. Corrected an older ownership fixture to target
-the actual matching main record. Verified source-version regressions,
-`./scripts/check.sh`, and `go test -race ./internal/engine`.
-
-Owners: [`effectiveSourceVersionRuleForPlacement`](../../internal/engine/source_version.go),
-the main-tier projection handoff in
-[`assembleLayeredInput`](../../internal/engine/load_request.go), and use of
-the effective rule in [`executeLoadRequest`](../../internal/engine/load.go).
-
-Source destinations and version-evidence sensitivity paths follow item movement,
-but the rule used for integer-version decoding retains the original item index.
-Consequently an accepted integer header can fail decoding after an otherwise
-valid upsert placement.
-
-Reproduction with the public API:
-
-- Main input is `id: main` and `version: 4`, placed as one sequence item at
-  `Keys("items")`.
-- A lower input contributes the sequence item `id: lower` at the same destination.
-- Register identity `id` for `[items]`, select `SourceVersionModeIfPresent`, and
-  use `VersionRule.Path = Keys("version")` with an accepting checker.
-- Decode into records with `ID string` and `Version string`.
-
-The checker accepts `4` exactly once. Upsert places the main record at index 1,
-but decoding fails with `decode.type` at `/items/1/version`. The integer-to-string
-exception is still tied to `/items/0/version`. Quoting the identical version as
-`'4'` succeeds. A source-local indexed header path, without item placement,
-reproduces the same problem when its record moves during upsert.
-
-Required remediation: carry the main declaration's effective coordinates through
-the existing item projections into decoding and audit the associated ownership
-protection handoffs for the same stale-coordinate assumption. Preserve
-source-local checker/evidence paths, checker invocation count, main-only
-ownership, and the narrow integer-to-string exception. Cover moved existing and
-newly appended main records, integer/string parity, and protection of the actual
-main header after movement. Do not prohibit supported indexed paths or mounted
-main records as a workaround.
-
-## 15. Exclude Parent Identity From Inherited History Payload — P2
-
-Status: [x] Complete.
-
-Completion: retained record-inheritance operations now exclude parent identity
-from their payload and origin projection. Replay preserves the child's own
-identity before taking field, record, sequence, or root snapshots, without
-inventing inheritance events for retained identity fields. Regressions cover
-single and compound identities, explicit child provenance, ordinary inherited
-payload, and the parent's original history. Verified inheritance/inspection
-coverage, `./scripts/check.sh`, and `go test -race ./internal/engine`.
-
-Owners: inheritance operation construction in
-[`applyInheritanceScopeTemplates`](../../internal/engine/inheritance_schedule.go),
-[`appendInheritanceExplanation`](../../internal/engine/inheritance_explain.go),
-and transferred-origin projection in
-[`inheritance_history.go`](../../internal/engine/inheritance_history.go).
-
-Identified-record value construction excludes the parent's identity fields, but
-the retained history operation still supplies the complete parent record and
-its identity origins to the older mapping-inheritance replay machinery.
-
-Reproduction:
-
-```yaml
-configkit:
-  sequences: [{destination: [items], key: id}]
-items:
-  - id: base
-    value: parent
-  - id: child
-    extends: {id: base}
-```
-
-The effective child ID and origin are correct. However,
-`Explain(Keys("items").Index(1).Key("id"))` reports `child` → `base` → `child`,
-attributing the intermediate value to an inheritance operation and the parent's
-ID location. `Metadata.History` likewise claims an inherited parent-ID
-contribution. That identity is expressly excluded from inherited payload; it was
-never an applied value at the child path.
-
-Required remediation: make history replay and transferred-origin projection use
-the same identity exclusion as value construction. Preserve the child's authored
-identity and presence, ordinary inherited payload history, and the parent record's
-own source history at its original path. Cover compound identities and identity
-fields alongside inherited payload at both record and ancestor queries; keep the
-existing inspection API and sensitivity behavior.
-
-## 16. Avoid Deep-Copying Accumulated Item Policies Per Record — P2
-
-Status: [x] Complete.
-
-Completion: item projection now takes one owned policy copy per destination,
-updates that local state incrementally, and stores it once after projection.
-Incoming scopes still cross detached ownership boundaries. Inspection confirms
-that accumulated record scopes are no longer deep-copied per item; no timing
-threshold or benchmark campaign was needed. Existing stable-index, sparse-range,
-fallback-offset, and ownership coverage passes, with a new regression for
-interleaved destinations, distinct record owners, and sibling/cross-load mutation
-independence. Verified focused item-policy tests, `./scripts/check.sh`, and
-`go test -race ./...`.
-
-Owners: [`projectNestedCollectionPolicies`](../../internal/engine/imports.go)
-and its use of copying `collectionPolicyScope.at` / `set` helpers in
-[`collection_policy_state.go`](../../internal/engine/collection_policy_state.go).
-
-The prefix-scan correction in finding 11 is present, but a separate quadratic
-cost remains on the same independently authored-record path. For every imported
-item with local policy, `at` deep-clones the destination's accumulated record
-order and every retained record's nested scope. After adding one owner, `set`
-deep-clones the growing policy again. With N fixed-size item-local policies at
-one destination, this copies approximately `1 + 2 + ... + N` retained scopes,
-even when every assigned sequence range is available.
-
-This follows directly from the loop and `cloneCollectionPolicy`; no busy-host
-timing comparison is required. It is an integration cost from reusing detached
-read/write helpers within private incremental construction, rather than from
-the required final ownership copies.
-
-Required remediation: construct each destination's owned record-policy state
-without repeatedly cloning all earlier records, retaining necessary copies at
-ownership boundaries. Preserve record ordering, distinct owners, sparse assigned
-positions, fallback offsets, and cross-load detachment. Reuse the existing item
-policy and ownership tests; add allocation/scaling evidence only where useful.
-Do not introduce public caches, pooling, new caps, or performance thresholds.
-
-## 17. Reconcile Legacy Sequence-History Contract Text — P2
-
-Status: [x] Complete.
-
-Completion: metadata and explanation prose now describe history through the
-selected collection operation, assigned item positions, and displaced payload.
-The text distinguishes sequence snapshots and atomic diffs from append/upsert
-combination and links to the canonical operation tables. Whole-record upsert,
-existing queries, and sensitivity guarantees are unchanged. Verified against
-replay/combination code, local documentation-link tests, and `git diff --check`;
-the unchanged runtime is covered by the preceding everyday and full race checks.
-
-Owner: the metadata/provenance and explanation sections of
-[`docs/config.md`](../config.md#layered-metadata).
-
-The metadata section still says that a later tier supplying a sequence records
-replacement at the sequence path and that history does not imply identity-based
-merging across tiers. That describes the former unconditional replacement model,
-not the implemented append/upsert history or the collection operation tables in
-the same document. The explanation paragraph about atomic sequences also needs
-review so readers can distinguish snapshot/diff granularity from collection
-combination semantics.
-
-Required remediation: describe history for the selected collection operation and
-link to the canonical operation tables. Preserve the existing atomic sequence
-snapshot/diff behavior and whole-record upsert semantics; correcting the prose
-does not authorize recursive record-field merging, a new query API, or broader
-history guarantees.
-
-## Algorithm Audit Baseline And Scope
-
-Reviewed commit: `c73daf891d0f29c96f7026bdabcccb571f44a751`.
-Feature baseline: `4f941f8ba4c9a9ed0afa87b07fe6ba29f131d1c6`.
-
-This audit examines the sequence identity/operation, scoped-policy, inheritance,
-inspection, and post-decode CLI algorithms, including the completed corrections
-above. The intervening discovery roadmap commits add no implementation to audit.
-The accepted scheduler, deterministic ordering, exact numeric normalization,
-selector trie, and deliberately pairwise registration-overlap validation remain
-selected design choices; they are not findings here.
-
-Evidence and validation:
-
-- Code inspection establishes the complexity of findings 19–23 on bounded-depth
-  inputs with linear-sized results. Required result/history copies and genuinely
-  growing inheritance output are excluded from the alleged excess work. Bounds
-  describe internal operations, not new public performance guarantees.
-- An isolated public-API probe reproduces finding 18 using an in-memory
-  filesystem, without external services or real credentials.
-- Temporary tests supplied through a Go overlay, outside the repository, checked
-  all 768 combinations of append/upsert/replace over the 16 unique-ID lists drawn
-  from three IDs. Stable ordering, duplicate rejection, and whole-record payload
-  replacement agreed with an independent reference calculation.
-- A structural overlay test confirmed that 4, 8, and 16 one-item contributions
-  retain 10, 36, and 136 reverse-index cells respectively in `indexSequenceRanges`.
-  These are representation counts, not timing or real-world benchmark results.
-- `go test ./internal/engine -run 'Sequence|Collection|Inheritance|Override|SourceVersion|Metadata|History' -count=1`
-  passed with Go 1.26.10 on Linux. Passing existing tests does not cover the
-  incorrect selector scope or establish efficient growth.
-
-The following are focused corrections within the selected model. This review
-changes only the remediation record.
-
-## 18. Match Application Registrations In Effective Item Coordinates — P2
-
-Status: [ ] Open.
-
-Owner: `composePendingItems` in
-[`imports.go`](../../internal/engine/imports.go), especially its local
-`composeDocumentsWithCollectionPolicy` call; the current
-`TestPendingItemCompositionAppliesApplicationIdentityConstraints` in
-[`item_policy_test.go`](../../internal/engine/item_policy_test.go) also needs review.
-
-Pending item/helper documents are composed in item-local coordinates while the
-unchanged global application selector policy is supplied to their composer.
-A registration for root `[prompts]` therefore also constrains an unrelated
-`[items, 0, prompts]` collection before placement. This contradicts complete,
-effective-destination matching and makes an otherwise unused registration reject
-valid configuration. Application selectors do not traverse sequence indexes.
-
-Reproduction: main imports `item.yml` with destination `[items]` and placement
-`sequence-item`. The item contains `id: outer` and
-`prompts: [{text: payload}]`. Loading succeeds without a registration; adding
-only `SequenceIdentityAt(Keys("prompts"), "id")` fails with `compose.invalid`
-at `/items/0/prompts/0/id`, although no root `prompts` collection exists.
-
-Required remediation: separate item-local YAML policy from application matching
-in effective coordinates, including nested helper scopes. Preserve applicable
-outer collection constraints without reinterpreting global selectors as local
-ones. Add ordinary/layered regressions for unused root and dynamic selectors
-alongside imported item-local collections. Correct tests that pin the accidental
-scope; do not change selector meaning or require consumers to remove valid rules.
-
-## 19. Avoid Dense Global-Position Storage For Sparse Item Projections — P2
-
-Status: [ ] Open.
-
-Owners: `newSequenceItemProjectionIndex` in
-[`sequence_projection.go`](../../internal/engine/sequence_projection.go),
-`indexSequenceRange` / `projectedCompositionOrigins` in
-[`compose.go`](../../internal/engine/compose.go), and contribution indexing in
-[`history.go`](../../internal/engine/history.go).
-
-The reverse projection slice is sized to `maxOutput + 1`, even for one incoming
-item assigned to a late assembled position. `indexSequenceRange` materializes
-these indexes for ordinary contiguous append ranges too. With N one-item sources
-contributing to one sequence, reverse slices retain `1 + 2 + ... + N` cells.
-Initialization time and simultaneously retained auxiliary storage are therefore
-quadratic despite only N actual mappings. Origin/history projection constructs
-the same representation through additional callers. Finding 16's policy-copy
-correction does not remove this independent cost.
-
-Required remediation: retain arithmetic start/end lookup for contiguous ranges
-and use storage proportional to actual mappings for scattered positions, such as
-a sparse reverse map. Audit shared callers and avoid rebuilding equivalent indexes
-inside item queries. Preserve bidirectional lookup, absent positions, upsert
-replacement evidence, ordering, and detached results. Verify the representation
-on many one-item sources and noncontiguous upserts; add no timing threshold or
-collection-size limit.
-
-## 20. Remove Full Metadata Scans Per CLI Replacement Leaf — P2
-
-Status: [ ] Open.
-
-Owners: `updateOverridePresence`, `deletePresencePathSubtree`, and
-`updateOverrideOrigins` in
-[`override_decode.go`](../../internal/engine/override_decode.go).
-
-Every replaced scalar or subtree scans all presence keys, then all origin keys,
-to find matching prefixes. A single partial-mapping override updating N scalar
-children in an N-field configuration performs quadratic map-entry visits. This
-does not require N CLI sources or growing historical output: the same effective
-maps remain approximately N entries throughout the update.
-
-Required remediation: delete affected entries by walking the already known
-existing subtree, or use one batch-local prefix index where needed. Retain
-necessary copies at ownership boundaries without rescanning unrelated paths for
-each leaf. Preserve retained explicit zero/false values, removal of replaced
-descendants, null/kind changes, origin attribution, and detached loads. Use the
-existing nested CLI and ownership regressions plus focused broad-mapping coverage;
-do not change override semantics or impose batch limits.
-
-## 21. Localize Inheritance Policy Work To The Relevant Scope — P2
-
-Status: [ ] Open.
-
-Owners: `inheritancePolicyScopeAtPath` in
-[`inheritance.go`](../../internal/engine/inheritance.go), graph preparation in
-[`inheritance_schedule.go`](../../internal/engine/inheritance_schedule.go),
-and policy handoff/retirement in
-[`inheritance_walk.go`](../../internal/engine/inheritance_walk.go).
-
-Scope localization scans every entry in the supplied policy table for each
-definition. Mapping graph preparation also clones the full supplied table, and
-the walker passes its global table to each independent scope and scans that table
-again during replacement. For S scopes or siblings and P unrelated collection
-policies, these paths perform repeated S-by-P work and copies.
-
-For example, N independent mapping groups can each contain a constant-sized
-`base`/`child extends base` pair and one YAML collection declaration beneath its
-base. Input, expanded values, and relationship evidence remain linear in N, but
-each group's graph/localization visits or clones the other groups' policies.
-This is separate from necessary expansion along a genuinely growing parent chain
-and from the selected dependency scheduler.
-
-Required remediation: partition or index policies by semantic scope once and
-carry the relevant state through traversal, including efficient descendant
-retirement as inheritance introduces or removes destinations. Clone owned
-in-scope state rather than unrelated global state. Preserve record owners,
-pending declarations, mode lifetime, destination-local registration matching,
-and independent templates. Verify wide independent groups and nested item scopes;
-do not replace the agreed scheduling algorithm or add global caches.
-
-## 22. Index Fallback Location Lookup During Effective Projection — P2
-
-Status: [ ] Open.
-
-Owner: `documentWithMetadataLocations` in
-[`override_context.go`](../../internal/engine/override_context.go), using
-`nearestDocumentLocation` in
-[`validation.go`](../../internal/engine/validation.go).
-
-The new effective-projection walk calls the unindexed nearest-location helper
-for each field lacking a real origin location, often once for its mapping entry
-and again for its value. Each call starts at the fallback root and linearly scans
-mapping entries. With N authored root fields and N different default-only root
-fields, each default's failed lookup scans all N authored keys: quadratic work
-for linear-sized input and output. The shared lifecycle calls this path even
-without CLI assignments or a resolver.
-
-Required remediation: share an indexed fallback lookup or traverse corresponding
-containers together while carrying the nearest available location. Existing
-`documentLookup` / mapping-index machinery can support bounded-depth lookups
-without repeated wide scans. Preserve nearest-ancestor fallback, source IDs,
-synthetic locations, secret-reference diagnostics, and publication ownership.
-Exercise mixed authored/default-only broad mappings; do not weaken attribution
-or skip needed location attachment to avoid the work.
-
-## 23. Index Repeated Collection-Policy Document Lookups — P2
-
-Status: [ ] Open.
-
-Owners: `mergeOverlayMappingsWithCollectionPolicy` in
-[`overlay.go`](../../internal/engine/overlay.go) and
-`mergeInheritanceCollectionPolicies` in
-[`inheritance_collection.go`](../../internal/engine/inheritance_collection.go).
-
-These routines loop over collection policies/declarations and call plain
-`documentAtPath` for each destination. That helper linearly searches every
-mapping on the path. N independent collections below N sibling mapping keys
-therefore cause quadratic mapping-entry searches when checking which lower
-policies the incoming tier touches. The inheritance parent/child lookups and
-retained-value declaration checks have the same pattern. YAML declarations alone
-can establish these policies; the deliberately pairwise application-registration
-overlap check is not involved.
-
-Required remediation: reuse a document lookup built once for each stable input
-tree, or use the already collected claims where their scope is sufficient.
-Index a mutated result only after its value-changing phase, rather than retaining
-a stale lookup. Preserve absent destinations, non-sequence pending declarations,
-literal key/index distinctions, retirement, and error locations. Verify broad
-independent collections across precedence and inheritance; no new public cache,
-shape requirement, or performance gate is needed.

docs/roadmap/sequences.md

@@ -1,998 +0,0 @@
-# Sequence Identity And Combination Roadmap
-
-Status: settled feature roadmap; implementation is specified in the
-[staged implementation plan](implementation.md). This document defines the
-targeted end state, not an implemented contract. The implementation plan owns
-execution order, exact Go names and signatures, bookkeeping, and validation.
-
-## Purpose And Scope
-
-Allow configuration authors to assemble collections from independently authored
-records, declare identity in configuration, and progressively extend a baseline
-collection through higher layers. Authors can replace the entire collection,
-append distinct records, or replace complete records by identity while retaining
-unmatched records. Sibling inheritance uses the same sequence operations and
-identity-based defaults. Within an identified sequence, a record can explicitly
-inherit settings from another record in that sequence while keeping its own
-identity.
-
-The motivating use case is a prompt catalog whose source-backed baseline supplies
-records and whose application or installation layers add or supersede records.
-Applications can optionally impose identity constraints that configuration must
-respect and that hold in the final loaded configuration. These constraints can
-cover exact destinations or collections beneath dynamically named mapping keys,
-without requiring the application to discover those names before loading.
-
-The feature covers ordinary and layered loading, imported items, inline
-sequences, selected overlays, and sibling inheritance. Typed defaults, CLI
-assignments, and application resolution retain their own combination semantics;
-CLI application moves later in the loading lifecycle.
-
-Automatic recursive merging of matching records remains deferred
-(see [future work](future.md#1-recursive-body-merging-for-identified-records));
-explicit inheritance between distinct records is included. The initial
-declaration surface is `configkit.sequences`; `key` and `mode` shorthand on
-imports is deferred. Application lists remain
-ordinary YAML sequences without control wrappers. Unkeyed sequence concatenation
-is outside this feature.
-
-## Current Baseline And Required Changes
-
-The [import contract](../config.md#explicit-imports) owns existing selectors,
-destinations, placement, and ordering. `placement: sequence-item` already places
-each selected source mapping as one item at its effective destination. Item-local
-helper imports can contribute fields before placement.
-
-The [composition contract](../config.md#strict-document-composition),
-[layered-loading contract](../config.md#layered-loading), and
-[inheritance contract](../config.md#sibling-inheritance) describe the implemented
-baseline:
-
-| Area | Current behavior | Target behavior |
-| --- | --- | --- |
-| Identity declaration | `Definition.SequenceCompositions` registers one literal string field at each registered mapping path. | YAML collection declarations and optional authoritative application registrations support field sets and scalar identity components. |
-| Application registration destinations | Registrations select exact literal mapping paths. | Exact and structural selectors can impose identity beneath arbitrary mapping keys, independently at each matched destination. |
-| Strict composition | Registered sequences append distinct string identities and reject duplicates; repeated unregistered sequences conflict. | Collections with effective identity append distinct records and reject duplicates. Applicable lower-tier identity is available before composing higher-tier peers. Unidentified sequences retain strict-conflict behavior. |
-| Source precedence and selected overlays | A present higher sequence replaces the lower sequence wholesale. | Collection policy selects `replace`, `append`, or `upsert`; overlays can declare policy for inline sequences. |
-| Sibling inheritance | An inheriting mapping recursively overlays its resolved parent; supplied sequences replace wholesale. | Mappings retain recursive combination, while sequences use destination policy and the same three operations. |
-| Sequence-member inheritance | A sequence element has no sibling scope for its own inheritance relationship. | An identified sequence provides a sibling scope: a record can inherit from one record in the same sequence, selected by complete identity. |
-| CLI overrides | Apply before inheritance and typed defaults/decoding. | Apply after inheritance and typed defaults/decoding, before application resolution. |
-| Identity enforcement | Registration governs strict sequence composition. | Structural identity checks cover identified combinations; application registrations additionally constrain final effective sequences. |
-
-Source-backed underlays participate in source assembly. `Definition.Defaults`
-provides typed defaults later; extending source combination does not turn typed
-default lists into source underlays.
-
-## Collection Declarations
-
-Use a source-root `configkit.sequences` block containing collection declarations.
-Each entry identifies a `destination` and may declare `key`, `mode`, or both.
-Identity and mode can be supplied independently. This form serves imported items,
-inline records, ordinary mapping imports, and inline sequences in selected
-overlays.
-
-```yaml
-configkit:
-  sequences:
-    - destination: [prompts]
-      key: [namespace, id]
-      mode: append
-  imports:
-    - dir: prompts
-      pattern: '*.yaml'
-      destination: [prompts]
-      placement: sequence-item
-```
-
-Each selected file contributes a record such as:
-
-```yaml
-namespace: support
-id: welcome
-text: Welcome
-```
-
-Here append retains a lower-layer collection and adds distinct higher-layer
-records. Identity and combination controls belong to the collection declaration,
-so imported and inline contributions share one policy surface.
-
-### Destinations And Scope
-
-`destination` uses the existing sequence of literal mapping keys. Resolve it
-relative to the declaring source's placement: a fragment mounted under `catalog`
-can declare `[prompts]` locally. A sequence-item source can declare a nested
-collection within its own item without referring to an eventual numeric index.
-Preserve source and item-local scope boundaries.
-
-A collection's scope is its resolved destination within its composition scope.
-Resolve relative destinations before reconciling declarations. Different imported
-items retain separate scopes for their nested collections. A declaration governs
-all contributions to its collection in the tier, including imported and inline
-records; it does not govern only records in the declaring file.
-
-Gather declarations before composing the relevant scope so their effect does not
-depend on which contribution is assembled first. Retain applicable controls
-through source precedence and inheritance, then keep them out of application
-decoding.
-
-Sequence declarations are independent of filesystem import enablement. Selected
-overlays may contain `configkit.sequences`, extending the current blanket
-prohibition on overlay `configkit` declarations. This does not enable imports
-in overlays.
-
-### Overlay Authoring
-
-For an existing sequence `[foo, bar]`, an overlay can replace `foo` in full:
-
-```yaml
-configkit:
-  sequences:
-    - destination: [prompts]
-      key: id
-      mode: upsert
-prompts:
-  - id: foo
-    text: Revised definition
-```
-
-The result is `[foo*, bar]`. The explicit upsert mode can be omitted when no
-different explicit mode applies from a lower tier: effective identity then
-selects upsert. To add a distinct record instead:
-
-```yaml
-configkit:
-  sequences:
-    - destination: [prompts]
-      key: id
-      mode: append
-prompts:
-  - id: baz
-    text: Another prompt
-```
-
-The result is `[foo, bar, baz]`. To replace the whole sequence:
-
-```yaml
-configkit:
-  sequences:
-    - destination: [prompts]
-      mode: replace
-prompts:
-  - id: bar
-    text: Bar definition
-  - id: baz
-    text: Baz definition
-```
-
-The result is `[bar, baz]`. Explicitly selecting the default operation is valid.
-With no applicable explicit mode, omitting this declaration selects replace for
-an unidentified collection and upsert for an identified one. A retained explicit
-mode continues to govern.
-
-## Identity Definitions And Constraints
-
-### Field Sets
-
-`key` accepts a string or a nonempty list of strings naming direct, literal
-fields in each record. A name such as `metadata.id` denotes that literal field,
-not a nested lookup. Treat field names as a set: remove repeated names and
-ignore authored order when comparing definitions.
-
-Thus `id`, `[id]`, and `[id, id]` are equivalent; `[namespace, id]`,
-`[id, namespace]`, and `[namespace, id, id]` are also equivalent. An empty list
-is invalid because it supplies no identity fields. These rules apply to both
-configuration declarations and application registrations.
-
-Use a consistent field order internally to construct identity tuples. Preserve
-component types and tuple boundaries so unstructured concatenation cannot cause
-otherwise distinct identities to collide.
-
-### Scalar Components And Equality
-
-Each identified record must be a mapping with every designated identity field
-present. Components may be strings, numbers, booleans, or explicit null. Missing
-fields and mapping or sequence components are invalid. Present null differs
-from absence; two single-field null identities are equal. Compound identities
-are equal when all corresponding components are equal.
-
-Strings compare exactly, including empty strings, without trimming, case folding,
-or normalization. Scalar categories remain distinct: `"1"`, `1`, `true`, and
-null are different identities.
-
-Numbers compare by mathematical value across integer, fractional, and exponent
-spellings: `1`, `1.0`, and `1e0` are equal, as are `0` and `-0`. Use exact numeric
-normalization for identity comparison without rounding through a floating-point
-conversion. This does not rewrite authored values or change existing typed
-decoding and projection semantics. The parser's rejection of non-finite numeric
-forms remains unchanged.
-
-### Application-Owned Constraints
-
-Retain an optional application interface for registering collections and imposing
-their identity definitions. Configuration may define identity where no application
-constraint exists. When a declaration omits `key`, an applicable registration
-supplies identity. An explicit equivalent definition is accepted, and a different
-definition fails under every mode, including replace.
-
-For example, if the application requires `id`, `key: [id, name]` is invalid:
-that compound identity would allow repeated `id` values with different names.
-A registration is an authoritative constraint, not an omission-filling default.
-Copying a collection through inheritance does not create another application
-registration; registrations constrain their registered destinations.
-
-Identity constraints validate actual sequences. They do not independently require
-a path to exist, be non-null, or have sequence shape. The application schema and
-validation own those requirements. Final-result enforcement for application
-registrations is defined under [Validation And Diagnostics](#validation-and-diagnostics).
-
-### Application Registration API
-
-Use one public sequence-identity registration type, collected in one field on
-`Definition`. Each registration contains an explicit destination selector and
-its identity fields. Exact and dynamic registrations share matching, overlap
-validation, identity comparison, and lifecycle behavior.
-
-Provide two constructors returning that same registration type:
-
-- An exact-path convenience constructor accepting a literal `Path` and identity
-  field names. It converts the path into a selector containing literal segments.
-- An explicit-selector constructor accepting a structural selector and identity
-  field names, for destinations that include arbitrary mapping keys.
-
-Both constructors accept identity fields as variadic strings, so one-field and
-compound identities use the same construction form. Apply the settled field-set
-rules, including deduplication. Invalid registrations, including an empty identity
-field set, fail during application-definition validation.
-
-Existing `Path` values retain their literal meaning throughout ConfigKit. The
-exact-path constructor supplies convenience at the registration boundary; dynamic
-matching uses the explicit selector type. Both forms can appear together in the
-same definition field.
-
-Replace the current registration shape with this model. Preserving the existing
-`SequenceComposition` fields and `Definition.SequenceCompositions` spelling is not
-a requirement, and no compatibility facade is needed solely to retain that syntax.
-Choose names centered on sequence identity, reflecting its role in composition,
-inheritance, and final validation. Exact Go type, field, and constructor names
-and signatures belong in the implementation plan.
-
-### Dynamic Application Registration Destinations
-
-Allow applications to register identity for collections whose containing mapping
-keys become known only during loading. For example:
-
-```yaml
-collections:
-  weatherreporter:
-    prompts:
-      - id: foo.bar
-        text: Report the weather.
-  homewatch:
-    prompts:
-      - id: foo.bar
-        text: Report the household status.
-```
-
-One application registration can require `id` for every namespace's `prompts`
-list. The equal IDs above are valid because each list has its own uniqueness
-scope. An optional root `prompts` list can have a separate exact registration.
-
-Use an explicit application-side structural selector composed of literal mapping
-keys and an arbitrary-mapping-key segment. Conceptually, the namespace rule is:
-
-```text
-Literal("collections") -> AnyMappingKey -> Literal("prompts")
-```
-
-This notation explains the selector; it is not a Go signature or YAML syntax.
-Each segment consumes exactly one mapping level. Match the complete destination,
-not an arbitrary prefix or suffix. Literal keys remain exact, including empty
-strings and keys containing punctuation such as `*`, `/`, or `.`. An arbitrary-key
-segment is distinct from any literal spelling and does not match a sequence
-index. Existing `Path` values retain their literal meaning.
-
-This capability belongs to the reusable application definition and applies to
-ordinary and layered loads. YAML collection destinations and import destinations
-retain their existing literal syntax. Recursive search, regular expressions,
-filesystem glob rules, sequence-index selectors, and custom matching callbacks
-are outside this addition.
-
-At each concrete matched destination, apply the same authoritative identity
-constraint as an exact registration. Use the settled field-set and scalar-equality
-rules. Do not pool identities across destinations or require records to repeat
-the containing mapping key as an identity component. Applications retain ownership
-of namespace interpretation, qualified runtime IDs, and domain validation.
-
-Matching supplies effective identity before relevant strict composition and
-precedence operations. Without an applicable explicit mode, this identity selects
-upsert under the existing default rule. It also supplies identity for inheritance
-within the matched sequence. YAML declarations must agree with it under every
-mode, including replace. These are ordinary application constraints with a broader
-destination selector, not a separate final-validation facility.
-
-### Registration Overlap
-
-Equivalent application constraints coalesce at any shared destination. Equivalence
-uses the identity field-set rules: for example, `id` and `[id, id]` agree.
-Incompatible constraints fail; exact registrations do not override broader
-selectors, and registration order grants no precedence.
-
-Detect incompatible overlapping registrations while validating the application
-definition, before source acquisition. For these fixed-depth mapping selectors,
-overlap exists when the selectors have the same number of segments and every
-corresponding pair can match a common key. Unequal literal keys at any position
-make the selectors disjoint; an arbitrary-key segment can overlap a literal or
-another arbitrary-key segment. Compare segment structure rather than rendered
-path strings. No configuration pre-read is needed.
-
-Thus an `id` rule for all namespace prompt lists conflicts with a `[id, name]`
-rule for one named namespace, even if that namespace is absent from a particular
-load. This is an invalid application definition. A valid selector matching no
-collections is harmless: registration does not require a path to exist, create
-collections, or impose shape and nullability requirements.
-
-### Matching Across The Loading Lifecycle
-
-Match application selectors against effective configuration destinations after
-accounting for input placement. Physical filenames and source-local paths do not
-determine applicability. Make constraints available before composing contributions
-at those destinations, including namespaces first introduced by imports, mounted
-inputs, or higher tiers. Apply matching constraints to key-only YAML declarations
-as well, even before their sequence exists. Consumers need neither a preliminary
-load nor source inspection to enumerate namespaces.
-
-During inheritance, apply registrations at the inheriting destination, including
-newly created matching destinations. Copying a collection does not transfer its
-application registrations to a different path. The existing YAML-policy ownership
-and transfer rules remain separate from application selector matching.
-
-During final validation, find matching actual sequences in the final effective
-configuration. This includes namespaces introduced or replaced by typed defaults,
-CLI overrides, and application resolution. Reusing only destinations found during
-source assembly would miss these values. Use the same selector meaning and
-identity validator in both phases; later assignments do not trigger composition
-or inheritance again. YAML-only identities retain their existing enforcement
-boundary.
-
-## Strict Composition And Precedence Operations
-
-### Compose Peers Before Applying Precedence
-
-Resolve applicable identity and mode policy, strictly compose the tier's peer
-contributions, then apply one selected operation per collection against the
-accumulated lower value. The tier operation does not sequentially replace peer
-source contributions. Selected overlay contributions use the same combination
-semantics.
-
-Within a strict tier, equivalent identity definitions coalesce and different
-field sets fail. Agreeing explicit modes coalesce; conflicting explicit modes
-fail. Omission expresses no preference. A root declaration or a later declaration
-has no precedence over conflicting peers. Explicitly selecting a default mode
-is still a real instruction and can conflict with another explicit mode.
-
-Identified peer sequences append distinct records. Duplicate identities among
-peers or within a contributed list fail under all three modes, including replace
-and upsert. Equal or disjoint record payloads do not excuse a duplicate identity.
-Unidentified sequences retain their current strict-composition conflict behavior.
-
-For example, higher-tier fragments `[C]` and `[D]` with an agreed identity assemble
-as `[C, D]`. Against lower `[A, B]`, append or upsert produces `[A, B, C, D]`,
-while replace produces `[C, D]`. Peers `[B]` and `[B*]` conflict even under
-upsert because neither has higher precedence.
-
-### Available Operations
-
-Use one spelling for each mode: `replace`, `append`, and `upsert`.
-
-| Mode | Behavior at source-precedence and inheritance boundaries |
-| --- | --- |
-| `replace` | Replace the complete lower sequence with the assembled higher sequence. |
-| `append` | Retain lower records and append higher records; duplicate identities fail. |
-| `upsert` | Replace matching records in full, retain unmatched lower records, and append new identities. |
-
-Append and upsert require effective identity. ConfigKit does not infer identity
-from an `id` field or record position. Whole-sequence replacement does not require
-identity agreement with lower YAML policy, but application constraints still apply.
-
-Letters denote record identities; `B*` has the same identity as `B` with a different
-payload:
-
-| Lower | Higher | Mode | Result |
-| --- | --- | --- | --- |
-| `[A, B]` | `[C]` | `append` | `[A, B, C]` |
-| `[A, B]` | `[C]` | `replace` | `[C]` |
-| `[A, B]` | `[B*]` | `append` | Duplicate-identity error |
-| `[A, B]` | `[B*]` | `replace` | `[B*]` |
-| `[A, B]` | `[B*]` | `upsert` | `[A, B*]` |
-| `[A, B]` | `[B*, C]` | `upsert` | `[A, B*, C]` |
-
-Upsert replaces the whole matched record. If lower `B` has `id`, `text`, and
-`temperature`, while higher `B` has only `id` and `text`, the result at this
-boundary has only `id` and `text`. Subsequent defaulting or other lifecycle
-operations retain their own semantics. Mappings continue to combine recursively;
-matching sequence records do not recursively merge fields.
-
-### Mode Selection And Persistence
-
-Select the operation in this order:
-
-1. Reconcile explicit modes within the current tier; conflicting peers fail.
-2. Use the current tier's explicit mode if supplied.
-3. Otherwise retain an applicable explicit mode from a lower tier.
-4. With no explicit mode, infer upsert when effective identity is available.
-5. Otherwise infer replace.
-
-Effective identity can come from an application registration, current declarations,
-or applicable lower policy, subject to the identity agreement and lifetime rules
-below. Determine the inferred mode from available identity before applying the
-operation. A plain higher-tier sequence can therefore use upsert without repeating
-a key declaration.
-
-An explicit mode persists at its applicable destination until a higher-priority
-explicit declaration changes it. New sequence values alone do not reset it, and
-an unrelated overlay leaves it intact. A mode-only declaration can change policy
-without supplying records; it neither clears the collection nor retroactively
-recombines earlier tiers. Later source contributions and inheritance use the
-resulting policy.
-
-| Lower explicit mode | Higher declaration | Effective mode |
-| --- | --- | --- |
-| `append` | Mode omitted | `append` |
-| `append` | `upsert` | `upsert` |
-| `upsert` | `replace` | `replace` |
-| None | Mode omitted, effective identity available | Inferred `upsert` |
-| None | Mode omitted, no effective identity | Inferred `replace` |
-
-Only explicit modes persist. Inferred defaults are recomputed from effective
-identity when needed. Explicit replace continues to apply in subsequent tiers
-until another mode is declared; introducing identity alone does not override it.
-Mode persistence does not preserve YAML identity discarded by replacement, nor
-policies owned by discarded records. Source precedence and inheritance use the
-same destination policy, without separate per-phase settings. CLI assignments
-remain outside mode selection.
-
-### Presence, Empty Sequences, Null, And Kind Mismatches
-
-At a collection's precedence boundary:
-
-| Higher contribution | `replace` | `append` / `upsert` |
-| --- | --- | --- |
-| Absent | Leave the lower value unchanged. | Leave the lower value unchanged. |
-| Explicit `[]` | Clear the sequence. | Contribute no records. |
-| Explicit null or another non-sequence value | Ordinary replacement, subject to application schema. | Error: the operation requires a sequence. |
-
-A declaration alone does not clear a collection. An optional import selecting
-nothing contributes nothing and does not erase existing records. An absent lower
-collection can be initialized by an incoming sequence under any mode, subject to
-identity requirements. Explicit lower null or another non-sequence value is not
-absence: append and upsert fail against it, while replacement remains available.
-An empty incoming sequence bypasses neither kind checks nor identity requirements.
-
-With effective identity and no explicit mode, `prompts: []` uses upsert and does
-not clear the collection. Clearing requires replace, selected in the current tier
-or retained from lower policy. Replacing an identified collection with null also
-requires replace and a schema permitting null. Collection null is distinct from
-a present null identity component inside a record.
-
-### Ordering
-
-Retain existing dependency-ready source scheduling within each tier and authored
-item order within lists. Gathering declarations does not change assembly order;
-identity values do not sort records.
-
-- Append keeps lower records in order, followed by higher records in assembled order.
-- Replace keeps only higher records in their assembled order.
-- Upsert keeps a matched record's lower position and appends new identities in
-  higher-tier order.
-
-For example, lower `[A, B, C]` and higher `[C*, A*, D]` produce `[A*, B, C*, D]`
-under upsert. Whole-sequence replacement gives the higher tier control of the
-complete sequence order.
-
-## Policy Agreement, Lifetime, And Ownership
-
-### Identity Agreement And Replacement
-
-Append and upsert use one agreed identity for all participating records:
-
-| Lower identity | Higher declaration | Result when combining records |
-| --- | --- | --- |
-| `id` | Omitted | Use `id`. |
-| `id` | `id`, `[id]`, or `[id, id]` | Accept the equivalent definition. |
-| `id` | `[namespace, id]` | Error: different definitions. |
-| None | `id` | Establish `id` and validate all participating records. |
-
-Introducing identity for an unkeyed lower sequence requires checking both lower
-and incoming records, including uniqueness within each contribution. With no
-identity from the application or either participating collection, append and
-upsert fail.
-
-YAML identity follows the collection that survives an operation:
-
-| Operation | Effective YAML identity |
-| --- | --- |
-| Append or upsert | Retain or establish the agreed identity. |
-| Whole-sequence replacement | Discard lower YAML identity and use the replacing tier's identity declaration, if any. |
-| Replacement by null or another non-sequence value | Discard lower YAML identity and retain any identity explicitly declared by the replacing tier. |
-
-Application constraints remain authoritative under every operation. Without an
-application constraint, replacement can introduce a different identity, such as
-changing from `id` to `name`. Replacing a sequence without a new key declaration
-or application registration produces an unkeyed sequence. No identity-removal
-directive is needed.
-
-The reset occurs only when the collection value is actually replaced. Declaring
-replace without supplying a replacement does not authorize changing the identity
-of retained records. Explicit mode lifetime is separate from YAML identity
-lifetime.
-
-### Ancestor Replacement
-
-Replacing an enclosing mapping can remove a collection without replacing that
-collection directly. Apply the same identity and ownership rules to this removal:
-discard the removed collection's lower YAML identity, and discard policies owned
-by any removed records. Reintroducing the same path later does not restore those
-discarded policies.
-
-Independently declared explicit modes retain their existing persistence rules.
-Removing a descendant collection does not remove a mode whose declaring scope
-survives. Application registrations remain applicable at matching destinations,
-including destinations reintroduced later. Identity newly declared by the
-replacing tier follows the existing declaration and ownership rules.
-
-For example, a root-level declaration establishes `key: id` and `mode: append`
-for `[catalog, prompts]`. A higher tier replaces `catalog` with null, removing
-the prompts collection and its YAML identity. The independently declared append
-mode survives. If a later tier recreates `catalog.prompts`, append requires
-identity supplied by a new declaration or an application registration. Without
-that identity, loading fails; the later tier can instead explicitly choose
-replace. The replacement of `catalog` itself follows ordinary ancestor-value
-precedence and does not attempt to append null to the removed prompts list.
-
-### Declarations Without Sequence Values
-
-A key declaration without accompanying records applies immediately to an existing
-retained sequence: check agreement and validate its records, without recombining
-earlier tiers. For example, a higher tier that only declares
-`destination: [prompts], key: id` identifies and validates the retained lower
-`prompts` sequence.
-
-If no collection exists, retain the declaration in its owning scope for later
-contributions. An otherwise empty underlay can establish identity for main-tier
-fragments. An explicit identity declaration can also remain pending at a
-non-sequence destination; it does not independently impose a shape requirement.
-The presence and kind rules still determine whether an attempted operation fails.
-Item-local declarations retain their ownership boundaries.
-
-### Lower-Tier Identity During Strict Composition
-
-Resolve applicable identity before composing a higher tier's peers, using
-application registrations, available lower-tier policy, and declarations gathered
-within the higher tier's scope. Authors need not repeat a baseline key solely
-because the higher tier has multiple fragments.
-
-For example, an underlay establishes `key: id` for `prompts`, and two main-tier
-fragments contribute `[foo]` and `[bar]` without a key declaration. They strictly
-compose as `[foo, bar]` before the main tier combines with the accumulated lower
-collection. If both contribute `foo`, composition fails; upsert does not give
-precedence to peers.
-
-Lower YAML identity remains subject to agreement, replacement, and ownership.
-A replacing collection must not be constrained by obsolete lower identity just
-because policy resolution precedes composition. Likewise, a replacement record
-cannot borrow nested identity owned by the discarded lower record, even when
-outer identity or numeric position matches. Application constraints still apply.
-Higher-tier declarations do not retroactively repair invalid lower-tier composition.
-
-### Nested Record Ownership
-
-Attach item-local policies to their owning contributions so they follow surviving
-records independently of numeric indexes. Whole-record upsert discards the old
-record's local policies along with its fields. Whole-sequence replacement discards
-policies owned by removed records. A replacement record supplies its own local
-declarations; equal outer identity or final index does not transfer discarded
-policies.
-
-For example, a source imported as one record into `prompts` can declare:
-
-```yaml
-configkit:
-  sequences:
-    - destination: [steps]
-      key: id
-      mode: append
-id: foo
-steps:
-  - id: prepare
-```
-
-The declaration governs `steps` inside that `foo` record. Retaining `foo` retains
-its local policy, including the explicit mode. Upserting a new `foo` discards the
-old `steps` and its declaration even though the outer record keeps its position.
-
-Policies independently applicable at a destination are not discarded merely
-because an individual record is replaced. Ownership determines which declarations
-are removed. Application registrations remain constraints at their registered
-destinations. Inheritance copies transfer identity and choose operations as
-specified next.
-
-## Sibling Inheritance
-
-Complete source-tier assembly before resolving inheritance. Retain existing
-mapping-sibling scopes and add identified-sequence scopes as described below.
-Both use the existing dependency-ready algorithm. At each inheritance combination,
-the resolved parent's collection is the lower value and the inheriting mapping's
-local collection is the higher value. Mappings combine recursively; upsert within
-an identified sequence remains whole-record replacement. Explicit record
-inheritance is a separate construction step after those replacements.
-
-Use the inheriting destination collection's policy: an applicable explicit mode,
-including one retained through source tiers, wins. Otherwise infer upsert with
-effective identity and replace without it. There is no separate default-append
-rule for inheritance. The same destination policy governs source-precedence and
-inheritance combinations.
-
-YAML identity follows a collection copied through inheritance, subject to agreement
-and replacement rules. A source collection's mode does not automatically become an
-instruction at a different inheriting destination. For example, source `steps`
-has `key: id` and `mode: append`; an inheriting mapping supplies local `steps`
-without an applicable explicit mode. The copied identity selects upsert there.
-An explicit destination append instead selects append. A matching destination key
-agrees; a different key conflicts when combining records, while explicit replace
-can establish new YAML identity subject to application constraints.
-
-For example:
-
-```yaml
-configkit:
-  sequences:
-    - destination: [parent, items]
-      key: id
-child:
-  items:
-    - id: foo
-      value: A
-    - id: bar
-      value: B
-parent:
-  extends: child
-  items:
-    - id: baz
-      value: C
-```
-
-Despite the names, `parent` inherits from `child`. Identity at `[parent, items]`
-selects upsert, producing `[foo, bar, baz]`. If the local sequence instead supplies
-a revised `foo`, the result is `[foo*, bar]`. Append rejects that repeated identity;
-replace retains only the local sequence.
-
-The same ordering, presence, and kind rules apply at inheritance boundaries.
-An omitted local sequence retains the inherited sequence. Explicit `[]` clears it
-under replace and contributes no records under append or upsert. Retain controls
-until inheritance uses them, then exclude them from application decoding.
-
-### Inheritance Between Identified Sequence Records
-
-An identified sequence supplies a sibling scope for its records. Identity can
-come from an application registration or an applicable YAML declaration. A record
-may inherit from one other record in that same sequence using an `extends`
-mapping that selects the parent's complete identity:
-
-```yaml
-configkit:
-  sequences:
-    - destination: [prompts]
-      key: id
-prompts:
-  - id: base
-    model: standard
-    temperature: 0.5
-  - id: concise
-    extends: {id: base}
-    temperature: 0.2
-```
-
-The final `concise` record is:
-
-```yaml
-id: concise
-model: standard
-temperature: 0.2
-```
-
-The `base` record remains in the collection. Parents are ordinary records subject
-to the same decoding and validation as other records; there is no abstract
-template category.
-
-Use mapping-form references for both single and compound keys. With
-`key: [namespace, id]`, a reference can be
-`extends: {namespace: shared, id: base}`. Match components using the collection's
-field-set and scalar-equality rules, including mathematical numeric equality.
-`extends: {id: null}` selects a null-valued identity; `extends: null` cancels a
-relationship.
-
-Parent-reference mappings use ordinary strict mapping composition. Item-local
-helper imports may supply disjoint fields of a compound reference; overlapping
-fields conflict, even when equal. For example, a helper's
-`extends: {namespace: shared}` and the importing record's `extends: {id: base}`
-compose into `extends: {namespace: shared, id: base}`. This assembles one record;
-it does not permit strict peers to contribute duplicate record identities.
-
-Before parent lookup, require the surviving effective reference mapping to contain
-exactly the collection's identity fields, with supported scalar values. Missing
-or extra fields fail. Validate the completed reference during inheritance, rather
-than requiring each helper contribution to be independently complete. Superseded
-references remain historical evidence; earlier strict-composition conflicts
-remain errors.
-
-Every record must supply its complete own identity before inheritance. Preserve
-the child's identity fields and inherit only the parent's nonidentity payload;
-inheritance cannot supply a missing child identity. Child payload fields take
-precedence, mappings combine recursively, and nested sequences use the inheriting
-destination's collection policy. Consume the record's directive before decoding.
-
-Support one parent within the same identified sequence, including forward
-references and chains. Missing parents and cycles, including self-reference,
-fail the load. Cross-collection references, position-based references, multiple
-parents, and inheritance between unidentified sequence members are outside this
-scope. Existing string-form references between mapping siblings retain their
-meaning; they do not become shorthand for sequence identity references.
-
-Resolve relationships against records surviving source-tier assembly and selected
-overlays. A higher layer can replace a parent before inheritance runs; removing
-that parent makes a surviving reference fail. Whole-record upsert discards the
-old record's directive along with its payload. Omitting `extends` in the
-replacement record does not retain the displaced record's relationship. An
-explicit reference to the record's own identity is a cycle, not a way to access
-its lower-tier predecessor.
-
-Reuse Kahn's algorithm with the existing stable canonical semantic-pointer key
-for dependency-ready ordering. Build parent lookup from the sequence's effective
-identity, and preserve the assembled list order independently of resolution
-order. Enclosing relationships resolve before nested relationships. Parent
-templates retain nested instructions for evaluation at each inheriting
-destination, following the existing destination-local inheritance rule. Copied
-collection identity, destination operation selection, and record-owned policy
-transfer follow the rules above.
-
-Record inheritance runs in the existing structural phase, before typed defaults,
-decoding, CLI overrides, and application resolution. Those later stages do not
-trigger another inheritance pass. Preserve inherited presence, origin and
-history evidence, sensitivity, declaring resource contexts, and existing
-expansion budgets just as for mapping-sibling inheritance. Identity lookup must
-not expose identity values in diagnostics. Existing index-based inspection paths
-remain sufficient; no new identity-selector path API is needed.
-
-## Loading Lifecycle
-
-Use the following order:
-
-1. Assemble source tiers and selected overlays, resolving policy before each
-   tier's strict composition and applying collection operations at precedence
-   boundaries.
-2. Resolve sibling inheritance in mappings and identified sequences, then apply
-   typed defaults and decode the assembled configuration.
-3. Apply explicit CLI overrides to that configuration and check supplied values
-   against the application schema.
-4. Run application resolution against the overridden configuration.
-5. Resolve effective secret references, validate the final configuration including
-   application-registered identities, and publish it.
-
-### Typed Defaults And Application Resolution
-
-Source-backed underlays, including application-supplied mapping sources,
-participate in sequence combination. Typed `Definition.Defaults` remains the
-initial decoded state: supplied sequences replace default sequences. Source
-append does not append to a typed default list.
-
-Application resolution retains assignment and recorded-transformation semantics.
-ConfigKit does not automatically combine a callback's assigned sequence with its
-previous value. Application-registered identities receive final validation
-independently of how defaults, CLI overrides, or callbacks produce their values.
-
-### CLI Precedence
-
-CLI overrides are the highest-precedence input to application resolution.
-Supplied sequences replace whole sequences, including sequences inside supplied
-mappings. Mappings retain recursive override behavior; empty sequences and
-permitted nulls retain explicit replacement meaning. Configuration append or
-upsert does not reinterpret CLI assignments. Inheritance and defaulting do not
-run again afterward: overriding assembled `[A, B]` with `[C]` presents `[C]` to
-application resolution without inheritance reintroducing prior records.
-
-Application resolution sees CLI values and their explicit-presence and source
-metadata. `Resolution.Default` preserves explicit CLI values; normalization and
-recorded application transformations may still change them. For example, a derived
-URL can use a CLI hostname, while an explicit CLI URL can be preserved by an
-absence-based default. Retain recording and provenance requirements for those
-transformations.
-
-Reject structural directives such as `extends` and `configkit` in CLI override
-paths or supplied values. CLI input supplies application values after structural
-processing; it cannot supply or cancel inheritance directives, change completed
-relationships, or prevent earlier inheritance failures.
-
-Typed decoding also precedes CLI application, so a CLI replacement cannot repair
-an earlier decoding failure. A source string in an integer field fails before a
-valid CLI integer can apply. This changes current override-before-decoding
-behavior. Semantic validation remains after overrides and application resolution.
-
-Secret lookup uses effective references after overrides and resolution. Neither
-stage bypasses final application identity constraints or semantic validation.
-
-## Validation And Diagnostics
-
-### Structural Identity Checks
-
-Allow item-local helper imports to complete and consume applicable non-main
-source-version headers before checking payload identity for collection assembly.
-A helper can supply an identity component; a consumed header cannot supply one
-after removal.
-
-Structural controls are likewise unavailable as source identity components.
-Check identity against application payload fields, excluding fields acting as
-structural directives even when the engine retains them for later processing.
-For example, `key: extends` with an item containing `extends: null` does not
-supply a null identity: that field cancels inheritance and is consumed. Report
-the ordinary missing-identity-component error at the collection/item path using
-available origins, without exposing values. Do not defer this failure until
-directive consumption or final application validation.
-
-This rule follows a field's structural role; it does not introduce a broader
-restriction on unrelated literal field names. Final application identity checks
-continue to examine the effective typed payload under their existing rules.
-
-Validate each identified contribution before combination and check uniqueness in
-the resulting identified collection, including inheritance operations. Upsert
-matches across precedence are replacements; duplicates within a contribution
-remain errors. Later inheritance, typed defaults, or callbacks cannot repair an
-identity missing at an earlier assembly boundary.
-
-YAML-only identities are enforced during structural composition, precedence
-assembly, and identified inheritance operations. They have no final-result
-guarantee after typed defaulting, CLI overrides, or application resolution. Those
-stages can produce duplicate identities or otherwise violate a YAML-only identity.
-Applications needing a final guarantee register the constraint through the
-application-owned interface.
-
-Adding record inheritance does not expand this guarantee: a YAML declaration
-can enable parent lookup without imposing final validation on values later
-produced by defaults, CLI overrides, or application resolution.
-
-### Final Application Constraints
-
-On successful loading, final effective collections covered by application
-registrations satisfy their declared identity constraints. Retain assembly-time
-checks and validate final effective sequences before publication, after defaults,
-inheritance, overrides, application resolution, and secret lookup.
-
-This covers both exact and dynamic application registrations. Evaluate dynamic
-selectors against the final effective configuration, with independent validation
-for each concrete matched sequence as described under
-[Matching Across The Loading Lifecycle](#matching-across-the-loading-lifecycle).
-
-ConfigKit performs these identity checks; consumers register the constraint
-without needing to repeat duplicate detection in `Definition.Validate`.
-Application-owned semantic validation continues to cover domain-specific rules.
-
-This guarantee covers actual sequences, not path existence or allowed shape.
-It does not require every intermediate callback state to satisfy the constraint
-and does not cover caller mutations after loading returns. Final checks examine
-effective typed values; assembly checks validate authored contributions. A final
-scalar alone does not prove that a source explicitly supplied that field.
-
-Use the existing canonical final-value representation and shared publication
-path as the foundation. The guarantee requires neither a new mutation-history
-system nor exact causal reconstruction for records supplied by defaults or
-callbacks.
-
-### Diagnostics And Inspection
-
-Identity diagnostics identify collection and item paths using available origin
-information without exposing identity values. Conflicting declarations identify
-the collection destination and declaration locations.
-
-For dynamic registrations, use concrete matched collection and item paths for
-value diagnostics, with available origins and existing sensitivity rules.
-Definition-time overlap errors identify the conflicting registrations; they do
-not require or invent a concrete destination from a loaded document.
-
-Correctness includes final values, projected item paths, source destinations,
-provenance, history, sensitivity, and passive resource contexts. Appending can
-change final indexes; replacing changes which contributions survive. Preserve
-the existing loading and inspection contracts through both operations, including
-policy ownership and inheritance transfer.
-
-## Work Needed From The Baseline
-
-The baseline needs the following connected capabilities to reach the target
-state. Execution order and test sequencing belong to the
-[implementation plan](implementation.md):
-
-- **Declarations and public API:** extend application registration to the settled
-  identity model with one public registration type and definition field, an
-  exact-path convenience constructor, and an explicit-selector constructor.
-  Preserve literal `Path` semantics. Parse and consume `configkit.sequences` across
-  supported source forms and selected overlays; preserve placement-relative and
-  item-local scopes. Keep sequence declarations independent of filesystem import
-  enablement.
-- **Identity and policy representation:** implement field-set comparison, typed
-  scalar identity tuples with exact numeric equality, declaration agreement,
-  pending declarations, explicit-mode persistence, replacement lifetime, and
-  record-owned policy transfer. Choose bookkeeping proportional to these semantics.
-  Handle descendant collection removal through ancestor replacement, preserving
-  independently declared modes and application constraints while discarding
-  removed collection identity and record-owned policies.
-- **Registration matching:** support literal and arbitrary-mapping-key segments,
-  detect incompatible selector overlap during definition validation, and resolve
-  applicable constraints at concrete placed destinations. Reuse the identity
-  rules and validator for exact and dynamic registrations. Keep matching separate
-  from source discovery and from YAML declaration ownership.
-- **Assembly handoffs:** make accumulated lower policy available before higher-tier
-  strict composition. Separate source acquisition from policy-aware composition
-  or otherwise revise assembly sequencing; current main-before-underlay composition
-  is not a design constraint. Preserve distinctions among acquisition order,
-  dependency-ready scheduling within tiers, and precedence order. Account for
-  source records, callbacks, main-version checks, and failure diagnostics. Retain
-  version ownership and validation semantics; higher replacement cannot hide
-  lower-tier structural failures.
-- **Combination and inheritance:** implement the three operations with strict peer
-  handling, whole-record replacement, presence rules, stable ordering, and the
-  destination policy rules for inheritance. Extend sibling-scope discovery and
-  parent lookup to identified sequences, validate mapping-form references, and
-  preserve child identities while copying parent payloads. Reuse the shared
-  dependency-ready scheduler and destination-local nested resolution. Preserve
-  provenance, history, sensitivity, paths, and resource contexts as records survive,
-  move, or disappear.
-- **Lifecycle and validation:** move CLI application after inheritance and typed
-  defaults/decoding, preserve explicit-presence and source metadata for application
-  resolution, reject CLI structural directives, and add final application identity
-  validation through the shared publication path. Keep YAML-only enforcement
-  within structural operations. Match dynamic registrations during assembly and
-  inheritance, and against the final effective configuration rather than a saved
-  list of source-backed destinations.
-  Exclude structural controls from source identity checks even while retaining
-  those controls for later inheritance processing.
-- **Documentation and verification:** exercise ordinary and layered loading,
-  imported and inline collections, item-local scopes, version-header consumption,
-  selected overlays, inheritance, typed defaults, CLI, callbacks, and inspection.
-  Cover single and compound parent references, scalar equality, forward references
-  and chains, missing parents and cycles, parent replacement/removal, child identity
-  preservation, nested destination policy, cancellation, and stable output order.
-  Verify helper-composed references, overlapping reference fields, missing or extra
-  fields, and replacement of references together with their containing records.
-  Verify independent namespace uniqueness, root and namespaced registrations,
-  literal punctuation keys, equivalent and incompatible overlaps, and conflicts
-  detected even without a matching input destination. Cover namespaces introduced
-  by imports, placement, higher tiers, inheritance, defaults, CLI, and callbacks;
-  preserve ordinary strict-peer and upsert behavior. Check YAML disagreement under
-  replace, harmless absent or empty collections, schema-owned shape requirements,
-  unrelated sequences remaining unselected, and concrete-path diagnostics.
-  Verify both constructors feed the same registration model, including mixed
-  exact and dynamic registrations, compound identities, and definition-time
-  rejection of empty identity field sets.
-  Cover ancestor replacement followed by collection reintroduction, including a
-  surviving append mode with and without renewed identity, explicit replacement,
-  and application-registered identity. Verify that `extends: null` cannot supply
-  an identity component and that structural-control exclusion uses ordinary
-  value-free missing-identity diagnostics.
-  Update the public contract and consumer examples when behavior lands.
-
-The collection-policy and CLI-ordering decisions are accepted in
-[ADR 0017](../adr/0017-collection-sequence-policy.md) for collection policy and
-[ADR 0018](../adr/0018-cli-after-structural-resolution.md) for CLI ordering.
-Their scoped supersession is recorded in the affected earlier ADRs, and the
-[architecture policy](../policy/architecture.md) reflects the accepted decisions.
-These govern the target architecture; the configuration contract continues to
-describe implemented behavior until the feature lands. This roadmap does not
-select migration machinery.
-
-ADR 0017 now includes inheritance within identified sequences;
-[ADR 0009](../adr/0009-sibling-inheritance.md) records that scope extension in its
-status while preserving its decision content. The architecture policy is aligned.
-Dynamic selectors extend the destinations covered by application registration;
-they retain the accepted application authority and final-validation boundary.
-
-No open feature decisions remain. The implementation plan translates this end
-state into reviewable stages without adding guarantees, restrictions, or
-combination semantics.

facade_item_registration_scope_test.go

@@ -0,0 +1,69 @@
+package configkit_test
+
+import (
+	"fmt"
+	"testing"
+	"testing/fstest"
+
+	configkit "gitea.maximumdirect.net/eric/ConfigKit"
+)
+
+func TestFacadeItemScopesDoNotReinterpretGlobalRegistrations(t *testing.T) {
+	t.Parallel()
+	for _, layered := range []bool{false, true} {
+		for _, dynamic := range []bool{false, true} {
+			for _, missingOuterID := range []bool{false, true} {
+				t.Run(fmt.Sprintf("layered=%t/dynamic=%t/missing-outer-id=%t", layered, dynamic, missingOuterID), func(t *testing.T) {
+					payload := "prompts: [{text: helper}]\n"
+					unused := configkit.SequenceIdentityAt(configkit.Keys("prompts"), "id")
+					if dynamic {
+						payload = "catalog: {local: {prompts: [{text: helper}]}}\n"
+						unused = configkit.SequenceIdentityMatching(configkit.SequenceSelector{
+							configkit.SequenceLiteralKey("catalog"), configkit.SequenceAnyMappingKey(), configkit.SequenceLiteralKey("prompts"),
+						}, "id")
+					}
+					item := "configkit: {imports: [{path: helper.yml}, {path: child.yml, destination: [children], placement: sequence-item}]}\n"
+					if !missingOuterID {
+						item += "id: outer\n"
+					}
+					files := configkit.NewFilesystem("items", fstest.MapFS{
+						"main.yml": {Data: []byte("configkit: {imports: [{path: item.yml, destination: [items], placement: sequence-item}]}\n")},
+						"item.yml": {Data: []byte(item)}, "helper.yml": {Data: []byte(payload)},
+						"child.yml":        {Data: []byte("id: child\nconfigkit: {imports: [{path: child-helper.yml}]}\n")},
+						"child-helper.yml": {Data: []byte(payload)},
+					})
+					definition := configkit.Definition[map[string]any]{SequenceIdentities: []configkit.SequenceIdentity{
+						unused, configkit.SequenceIdentityAt(configkit.Keys("items"), "id"),
+					}}
+					var result *configkit.Result[map[string]any]
+					var err error
+					if layered {
+						result, err = configkit.LoadLayered(configkit.Input{Source: files.File("main.yml"), Imports: true}, configkit.LayeredOptions[map[string]any]{Definition: definition})
+					} else {
+						result, err = configkit.Load(files.File("main.yml"), configkit.Options[map[string]any]{Imports: true, Definition: definition})
+					}
+					if missingOuterID {
+						failure := requireFacadeLoadError(t, err)
+						if result != nil || len(failure.Diagnostics()) != 1 || failure.Diagnostics()[0].Code != configkit.CodeComposeInvalid || failure.Diagnostics()[0].Path.String() != "/items/0/id" {
+							t.Fatalf("outer constraint diagnostics = %#v", failure.Diagnostics())
+						}
+						return
+					}
+					if err != nil {
+						t.Fatal(err)
+					}
+					for _, record := range []configkit.Path{configkit.Keys("items").Index(0), configkit.Keys("items").Index(0).Key("children").Index(0)} {
+						path := record
+						if dynamic {
+							path = path.Key("catalog").Key("local")
+						}
+						info, found := result.Metadata().Lookup(path.Key("prompts").Index(0).Key("text"))
+						if !found || !info.Explicit {
+							t.Fatalf("nested helper payload lost at %s", path)
+						}
+					}
+				})
+			}
+		}
+	}
+}

internal/engine/collection_lookup_test.go

@@ -0,0 +1,98 @@
+package engine
+
+import (
+	"fmt"
+	"strings"
+	"testing"
+
+	"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
+)
+
+func TestOverlayBroadCollectionPoliciesAndRetainedDeclarations(t *testing.T) {
+	var baseBody, overlayBody strings.Builder
+	basePolicies := collectionPolicyScope{}
+	var declarations []sequenceDeclarationSpec
+	for index := range 64 {
+		name := fmt.Sprintf("group/%03d~", index)
+		fmt.Fprintf(&baseBody, "%s: {items: [{id: base}], retained: [{id: retained}], scalar: pending}\n", name)
+		fmt.Fprintf(&overlayBody, "%s: {items: [{id: overlay}], added: true}\n", name)
+		basePolicies.set(Keys(name, "items"), collectionPolicy{
+			identityFields: []string{"id"}, identityPresent: true,
+			explicitMode: sequenceModeAppend, explicitModePresent: true,
+		})
+		for _, key := range []string{"retained", "missing", "scalar"} {
+			declarations = append(declarations, sequenceDeclarationSpec{
+				destination: Keys(name, key), identityFields: []string{"id"}, identityPresent: true,
+				mode: sequenceModeAppend, modePresent: true,
+			})
+		}
+	}
+	base, diagnostics := composeDocuments(parseCompositionDocument(t, "base", baseBody.String()))
+	if len(diagnostics) != 0 {
+		t.Fatal(diagnostics)
+	}
+	base.collectionPolicies = basePolicies
+	assembly := newAssemblyAccumulator(base)
+	overlay := parseCompositionDocument(t, "overlay", overlayBody.String())
+	diagnostics = assembly.applyOverlayWithCollectionPolicy(overlay, nil, collectionPolicyScope{}, nil, LayerOverlay, sequencePolicy{}, declarations)
+	if len(diagnostics) != 0 {
+		t.Fatal(diagnostics)
+	}
+	lookup := buildDocumentLookup(assembly.current.root)
+	for index := range 64 {
+		name := fmt.Sprintf("group/%03d~", index)
+		items, found := documentAtPathWithLookup(assembly.current.root, Keys(name, "items"), lookup)
+		if !found || items.Kind != document.Sequence || len(items.Sequence) != 2 || items.Sequence[0].Mapping[0].Value.Text != "base" || items.Sequence[1].Mapping[0].Value.Text != "overlay" {
+			t.Fatalf("collection %s did not append in order", name)
+		}
+		for _, key := range []string{"retained", "missing", "scalar"} {
+			policy, found := assembly.current.collectionPolicies.at(Keys(name, key))
+			if !found || !policy.identityPresent || !policy.explicitModePresent || policy.explicitMode != sequenceModeAppend {
+				t.Fatalf("retained/pending declaration lost at %s/%s", name, key)
+			}
+		}
+		if _, found := documentAtPathWithLookup(assembly.current.root, Keys(name, "missing"), lookup); found {
+			t.Fatal("pending declaration created a value")
+		}
+		if origin := assembly.current.origins[Keys(name, "items").Index(1).Key("id").String()]; origin.Layer != LayerOverlay {
+			t.Fatal("appended item lost overlay attribution")
+		}
+	}
+}
+
+func TestInheritanceBroadCollectionsInOneDefinition(t *testing.T) {
+	var body strings.Builder
+	policies := collectionPolicyScope{}
+	for _, definition := range []string{"base", "child"} {
+		fmt.Fprintf(&body, "%s:\n", definition)
+		if definition == "child" {
+			body.WriteString("  extends: base\n")
+		}
+		for index := range 64 {
+			name := fmt.Sprintf("group/%03d~", index)
+			fmt.Fprintf(&body, "  %s: {items: [{id: %s}]}\n", name, definition)
+			policy := collectionPolicy{identityFields: []string{"id"}, identityPresent: true}
+			if definition == "child" {
+				policy.explicitMode, policy.explicitModePresent = sequenceModeAppend, true
+			}
+			policies.set(Keys(definition, name, "items"), policy)
+			policies.set(Keys(definition, name, "pending"), collectionPolicy{identityFields: []string{"id"}, identityPresent: true})
+		}
+	}
+	root := parseCompositionDocument(t, "wide-collections", body.String())
+	result, diagnostics := expandInheritanceWithPolicies(root, nil, policies, defaultProjectionLimits())
+	if len(diagnostics) != 0 {
+		t.Fatal(diagnostics)
+	}
+	lookup := buildDocumentLookup(result.root)
+	for index := range 64 {
+		name := fmt.Sprintf("group/%03d~", index)
+		items, found := documentAtPathWithLookup(result.root, Keys("child", name, "items"), lookup)
+		if !found || len(items.Sequence) != 2 || items.Sequence[0].Mapping[0].Value.Text != "base" || items.Sequence[1].Mapping[0].Value.Text != "child" {
+			t.Fatalf("inherited collection %s did not append in order", name)
+		}
+		if policy, found := result.collectionPolicies.at(Keys("child", name, "pending")); !found || !policy.identityPresent {
+			t.Fatalf("pending inherited policy lost at %s", name)
+		}
+	}
+}

internal/engine/compose.go

@@ -1,6 +1,10 @@
 package engine
 
-import "gitea.maximumdirect.net/eric/ConfigKit/internal/document"
+import (
+	"maps"
+
+	"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
+)
 
 // composedDocument is the private handoff between source assembly and later
 // precedence layers. Claims remain ordered by source and declaration traversal;
@@ -89,19 +93,18 @@ type sequenceRange struct {
 	projection  sequenceItemProjectionIndex
 }
 
-func (assigned sequenceRange) projections() []sequenceItemProjection {
-	if assigned.items != nil {
-		return append([]sequenceItemProjection{}, assigned.items...)
-	}
-	return sequenceItemProjections(assigned.end-assigned.start, assigned.start)
-}
-
 func (assigned sequenceRange) outputForInput(inputIndex int) (int, bool) {
 	if assigned.projection.outputByInput != nil {
 		return assigned.projection.outputForInput(inputIndex)
 	}
 	if assigned.items != nil {
-		return newSequenceItemProjectionIndex(assigned.items).outputForInput(inputIndex)
+		output, found := 0, false
+		for _, item := range assigned.items {
+			if item.inputIndex == inputIndex && inputIndex >= 0 && item.outputIndex >= 0 {
+				output, found = item.outputIndex, true
+			}
+		}
+		return output, found
 	}
 	if inputIndex < 0 || inputIndex >= assigned.end-assigned.start {
 		return 0, false
@@ -114,7 +117,13 @@ func (assigned sequenceRange) inputForOutput(outputIndex int) (int, bool) {
 		return assigned.projection.inputForOutput(outputIndex)
 	}
 	if assigned.items != nil {
-		return newSequenceItemProjectionIndex(assigned.items).inputForOutput(outputIndex)
+		input, found := 0, false
+		for _, item := range assigned.items {
+			if item.outputIndex == outputIndex && outputIndex >= 0 && item.inputIndex >= 0 {
+				input, found = item.inputIndex, true
+			}
+		}
+		return input, found
 	}
 	if outputIndex < assigned.start || outputIndex >= assigned.end {
 		return 0, false
@@ -123,8 +132,8 @@ func (assigned sequenceRange) inputForOutput(outputIndex int) (int, bool) {
 }
 
 func indexSequenceRange(assigned sequenceRange) sequenceRange {
-	if assigned.projection.outputByInput == nil && assigned.projection.inputByOutput == nil {
-		assigned.projection = newSequenceItemProjectionIndex(assigned.projections())
+	if assigned.items != nil && assigned.projection.outputByInput == nil && assigned.projection.inputByOutput == nil {
+		assigned.projection = newSequenceItemProjectionIndex(assigned.items)
 	}
 	return assigned
 }
@@ -459,8 +468,8 @@ func cloneSequenceRanges(ranges []sequenceRange) []sequenceRange {
 		if assigned.items != nil {
 			cloned[i].items = append([]sequenceItemProjection{}, assigned.items...)
 		}
-		cloned[i].projection.outputByInput = append([]int(nil), assigned.projection.outputByInput...)
-		cloned[i].projection.inputByOutput = append([]int(nil), assigned.projection.inputByOutput...)
+		cloned[i].projection.outputByInput = maps.Clone(assigned.projection.outputByInput)
+		cloned[i].projection.inputByOutput = maps.Clone(assigned.projection.inputByOutput)
 	}
 	return cloned
 }
@@ -502,9 +511,15 @@ func projectedCompositionOrigins(inputs []*document.Node, ranges []sequenceRange
 	origins := make(map[string]Origin)
 	sequencesBySource := make([][]sequenceHistoryContribution, len(inputs))
 	for _, assigned := range ranges {
-		sequencesBySource[assigned.sourceIndex] = append(sequencesBySource[assigned.sourceIndex], sequenceHistoryContribution{
-			path: assigned.path.clone(), mode: sequenceHistoryMapped, items: assigned.projections(),
-		})
+		contribution := sequenceHistoryContribution{
+			path: assigned.path.clone(), mode: sequenceHistoryAppend,
+			offset: assigned.start, length: assigned.end - assigned.start,
+		}
+		if assigned.items != nil {
+			contribution.mode = sequenceHistoryMapped
+			contribution.items = assigned.items
+		}
+		sequencesBySource[assigned.sourceIndex] = append(sequencesBySource[assigned.sourceIndex], contribution)
 	}
 	for sourceIndex, input := range inputs {
 		application := historyApplication{value: input, sequences: sequencesBySource[sourceIndex]}

internal/engine/imports.go

@@ -319,7 +319,7 @@ func composeAcquiredInputWithPolicies(input assembledInput, lower collectionPoli
 	if len(scope.contributions) == 1 && scope.contributions[0].role == importRoleMain && scope.contributions[0].pendingItem != nil {
 		main := scope.contributions[0]
 		localScope := cloneAcquisitionScope(*main.pendingItem)
-		if diagnostics := traversal.composePendingItems(&localScope, policy); len(diagnostics) != 0 {
+		if diagnostics := traversal.composePendingItems(&localScope); len(diagnostics) != 0 {
 			return nil, prependCompositionDiagnosticPaths(diagnostics, main.destination.Index(0))
 		}
 		documents := make([]*document.Node, 0, len(localScope.contributions))
@@ -405,7 +405,7 @@ func composeAcquiredScopeWithPolicy(
 		accounting: accounting, limits: defaultImportLimits(), rule: rule,
 		versionState: versionState, importGraph: graph.clone(),
 	}
-	if diagnostics := traversal.composePendingItems(scope, policy); len(diagnostics) != 0 {
+	if diagnostics := traversal.composePendingItems(scope); len(diagnostics) != 0 {
 		return nil, diagnostics
 	}
 	documents := make([]*document.Node, 0, len(scope.contributions))
@@ -765,22 +765,25 @@ func (s *importTraversal) scheduleAcquisitionScopes(scope *acquisitionScope) {
 	}
 }
 
-func (s *importTraversal) composePendingItems(scope *acquisitionScope, policy sequencePolicy) []Diagnostic {
+func (s *importTraversal) composePendingItems(scope *acquisitionScope) []Diagnostic {
 	for index := range scope.contributions {
 		contribution := &scope.contributions[index]
 		if contribution.pendingItem == nil {
 			continue
 		}
 		itemScope := contribution.pendingItem
-		if diagnostics := s.composePendingItems(itemScope, policy); len(diagnostics) != 0 {
+		if diagnostics := s.composePendingItems(itemScope); len(diagnostics) != 0 {
 			return projectItemDiagnosticPaths(diagnostics, *scope, contribution.destination, index)
 		}
 		documents := make([]*document.Node, 0, len(itemScope.contributions))
 		for _, nested := range itemScope.contributions {
 			documents = append(documents, nested.document)
 		}
+		// An item scope is below a sequence index. Application selectors match
+		// mapping-only effective paths, so none can apply inside this scope.
+		// Its YAML declarations remain relative to the owning record.
 		completed, diagnostics := composeDocumentsWithCollectionPolicy(
-			policy, itemScope.collectionPolicies,
+			sequencePolicy{}, itemScope.collectionPolicies,
 			acquiredSequenceDeclarations(*itemScope), documents...,
 		)
 		if len(diagnostics) != 0 {

internal/engine/inheritance.go

@@ -114,26 +114,7 @@ func cloneInheritanceOperations(operations []inheritanceOperation) []inheritance
 }
 
 func inheritancePolicyScopeAtPath(scope collectionPolicyScope, prefix Path) collectionPolicyScope {
-	localized := collectionPolicyScope{}
-	for _, entry := range scope.collections {
-		if relative, inside := relativeHistoryPath(prefix, entry.destination); inside {
-			localized.set(relative, cloneCollectionPolicyForInheritance(entry.policy))
-			continue
-		}
-		relative, inside := relativeHistoryPath(entry.destination, prefix)
-		if !inside || len(relative.segments) == 0 || relative.segments[0].kind != pathSegmentIndex {
-			continue
-		}
-		record, found := entry.policy.recordPolicyAt(relative.segments[0].index)
-		if !found {
-			continue
-		}
-		remainder := Path{segments: append([]pathSegment(nil), relative.segments[1:]...)}
-		for _, nested := range inheritancePolicyScopeAtPath(record.scope, remainder).collections {
-			localized.set(nested.destination, cloneCollectionPolicyForInheritance(nested.policy))
-		}
-	}
-	return localized
+	return newInheritancePolicyIndex(scope).localize(prefix)
 }
 
 func projectInheritancePolicyTemplates(templates []collectionPolicyScope, prefix Path) []projectedCollectionPolicy {

internal/engine/inheritance_collection.go

@@ -103,15 +103,19 @@ func mergeInheritanceCollectionPolicies(
 	for key, entry := range child.collections {
 		paths[key] = entry.destination.clone()
 	}
+	var lookup *documentLookup
+	if len(paths) != 0 {
+		lookup = buildDocumentLookup(parentValue, childValue)
+	}
 	var diagnostics []Diagnostic
 	for _, path := range paths {
 		relative, inside := relativeHistoryPath(parentRootPath, path)
 		var retained, incoming *document.Node
 		if inside {
 			if !replaced.contains(path, false) {
-				retained, _ = documentAtPath(parentValue, relative)
+				retained, _ = documentAtPathWithLookup(parentValue, relative, lookup)
 			}
-			incoming, _ = documentAtPath(childValue, relative)
+			incoming, _ = documentAtPathWithLookup(childValue, relative, lookup)
 		}
 		resolution, lower, _, active, issues := inheritanceCollectionResolution(path, parent, child, applicationPolicy, incoming)
 		if len(issues) != 0 {

internal/engine/inheritance_policy_index.go

@@ -0,0 +1,123 @@
+package engine
+
+// inheritancePolicyIndex borrows immutable policy entries. Semantic path
+// segments keep mapping keys distinct from sequence indexes. Record-owned
+// scopes have their own indexes, just as they have their own policy tables.
+type inheritancePolicyIndex struct {
+	entry    *collectionPolicyEntry
+	children map[pathSegment]*inheritancePolicyIndex
+	records  map[int]*inheritancePolicyIndex
+}
+
+func newInheritancePolicyIndex(scope collectionPolicyScope) *inheritancePolicyIndex {
+	index := &inheritancePolicyIndex{}
+	for _, entry := range scope.collections {
+		index.set(entry)
+	}
+	return index
+}
+
+func (index *inheritancePolicyIndex) set(entry collectionPolicyEntry) {
+	node := index
+	for _, segment := range entry.destination.segments {
+		if node.children == nil {
+			node.children = make(map[pathSegment]*inheritancePolicyIndex)
+		}
+		if node.children[segment] == nil {
+			node.children[segment] = &inheritancePolicyIndex{}
+		}
+		node = node.children[segment]
+	}
+	node.entry = &entry
+	node.records = nil
+	for position, owner := range entry.policy.recordOrder {
+		scope := entry.policy.recordPolicies[owner]
+		if len(scope.collections) == 0 {
+			continue
+		}
+		if node.records == nil {
+			node.records = make(map[int]*inheritancePolicyIndex)
+		}
+		node.records[position] = newInheritancePolicyIndex(scope)
+	}
+}
+
+// localize copies only policies in the requested scope. Earlier record-owned
+// policies are considered before direct entries added by destination expansion.
+func (index *inheritancePolicyIndex) localize(prefix Path) collectionPolicyScope {
+	localized := collectionPolicyScope{}
+	node := index
+	for offset, segment := range prefix.segments {
+		if node == nil {
+			return localized
+		}
+		if segment.kind == pathSegmentIndex {
+			if record := node.records[segment.index]; record != nil {
+				remainder := Path{segments: prefix.segments[offset+1:]}
+				for _, entry := range record.localize(remainder).collections {
+					localized.set(entry.destination, entry.policy)
+				}
+			}
+		}
+		node = node.children[segment]
+	}
+	var collect func(*inheritancePolicyIndex)
+	collect = func(node *inheritancePolicyIndex) {
+		if node == nil {
+			return
+		}
+		if node.entry != nil {
+			relative, _ := relativeHistoryPath(prefix, node.entry.destination)
+			localized.set(relative, cloneCollectionPolicyForInheritance(node.entry.policy))
+		}
+		for _, child := range node.children {
+			collect(child)
+		}
+	}
+	collect(node)
+	return localized
+}
+
+func (index *inheritancePolicyIndex) absoluteScope(prefix Path) collectionPolicyScope {
+	scope := collectionPolicyScope{}
+	for _, entry := range index.localize(prefix).collections {
+		scope.set(appendDestination(prefix, entry.destination), entry.policy)
+	}
+	return scope
+}
+
+// replace updates the walker's owned table and its borrowed index together.
+// Retirement visits only direct entries below the changed destination.
+func (index *inheritancePolicyIndex) replace(scope *collectionPolicyScope, prefix Path, replacement collectionPolicyScope) {
+	node := index
+	var parent *inheritancePolicyIndex
+	var last pathSegment
+	for _, segment := range prefix.segments {
+		if node == nil {
+			break
+		}
+		parent, last, node = node, segment, node.children[segment]
+	}
+	var remove func(*inheritancePolicyIndex)
+	remove = func(node *inheritancePolicyIndex) {
+		if node == nil {
+			return
+		}
+		if node.entry != nil {
+			delete(scope.collections, node.entry.destination.String())
+		}
+		for _, child := range node.children {
+			remove(child)
+		}
+	}
+	remove(node)
+	if parent == nil {
+		*index = inheritancePolicyIndex{}
+	} else {
+		delete(parent.children, last)
+	}
+	for _, entry := range replacement.collections {
+		scope.set(entry.destination, entry.policy)
+		index.set(scope.collections[entry.destination.String()])
+	}
+}

internal/engine/inheritance_policy_index_test.go

@@ -0,0 +1,111 @@
+package engine
+
+import (
+	"fmt"
+	"strings"
+	"testing"
+)
+
+func TestInheritancePolicyIndexKeepsSemanticScopesAndOwnedTemplates(t *testing.T) {
+	policy := collectionPolicy{identityFields: []string{"id"}, identityPresent: true}
+	nested := collectionPolicyScope{}
+	nested.set(Keys("group", "base", "items"), policy)
+	outer := policy
+	record := newCollectionRecordPolicy()
+	record.scope = nested
+	outer.setRecordPolicyAt(0, record)
+	scope := collectionPolicyScope{}
+	scope.set(Keys("records"), outer)
+	scope.set(Keys("numeric", "0", "items"), policy)
+	index := newInheritancePolicyIndex(scope)
+
+	first := index.localize(Keys("records").Index(0).Key("group"))
+	second := index.localize(Keys("records").Index(0).Key("group"))
+	if len(first.collections) != 1 || len(second.collections) != 1 {
+		t.Fatal("record-owned scope was not localized")
+	}
+	entry := first.collections["/base/items"]
+	entry.policy.identityFields[0] = "changed"
+	if second.collections["/base/items"].policy.identityFields[0] != "id" || nested.collections["/group/base/items"].policy.identityFields[0] != "id" {
+		t.Fatal("localized policy shares mutable template state")
+	}
+	if len(index.localize(Keys("records", "0")).collections) != 0 || len(index.localize(Keys("numeric").Index(0)).collections) != 0 {
+		t.Fatal("mapping keys and sequence indexes were conflated")
+	}
+	if len(index.localize(Keys("numeric", "0")).collections) != 1 {
+		t.Fatal("numeric mapping key did not retain its policy")
+	}
+	left := index.localize(Keys("records"))
+	right := index.localize(Keys("records"))
+	leftOwner := left.collections[""].policy.recordOrder[0]
+	rightOwner := right.collections[""].policy.recordOrder[0]
+	if leftOwner == rightOwner || leftOwner == record.owner || rightOwner == record.owner {
+		t.Fatal("independent templates share a record owner")
+	}
+}
+
+func TestInheritancePolicyIndexReplacesOnlyTheChangedScope(t *testing.T) {
+	scope := collectionPolicyScope{}
+	policy := collectionPolicy{explicitMode: sequenceModeAppend, explicitModePresent: true}
+	for index := range 128 {
+		scope.set(Keys(fmt.Sprintf("group-%03d", index), "pending"), policy)
+	}
+	index := newInheritancePolicyIndex(scope)
+	changed := Keys("group-064")
+	if len(index.absoluteScope(changed).collections) != 1 {
+		t.Fatal("scope includes unrelated groups")
+	}
+	replacement := collectionPolicyScope{}
+	replacement.set(changed.Key("introduced"), policy)
+	index.replace(&scope, changed, replacement)
+	if len(scope.collections) != 128 {
+		t.Fatal("replacement lost unrelated pending declarations")
+	}
+	local := index.localize(changed)
+	if _, found := local.collections["/introduced"]; !found || len(local.collections) != 1 {
+		t.Fatal("retired policy survived or introduced policy was not indexed")
+	}
+	index.replace(&scope, changed, collectionPolicyScope{})
+	if len(index.localize(changed).collections) != 0 || len(scope.collections) != 127 {
+		t.Fatal("descendant retirement did not update the table and index together")
+	}
+	if len(index.localize(Keys("group-065")).collections) != 1 {
+		t.Fatal("retirement changed another scope")
+	}
+}
+
+func TestInheritanceWideIndependentYamlCollectionScopes(t *testing.T) {
+	var body strings.Builder
+	body.WriteString("configkit:\n  sequences:\n")
+	for index := range 64 {
+		fmt.Fprintf(&body, "    - {destination: [groups, group-%03d, base, items], key: id, mode: append}\n", index)
+	}
+	body.WriteString("groups:\n")
+	for index := range 64 {
+		fmt.Fprintf(&body, "  group-%03d:\n    base: {items: [{id: shared, value: parent}]}\n    child: {extends: base, items: [{id: shared, value: child}]}\n", index)
+	}
+	type item struct {
+		ID    string `yaml:"id"`
+		Value string `yaml:"value"`
+	}
+	type definition struct {
+		Items []item `yaml:"items"`
+	}
+	type config struct {
+		Groups map[string]map[string]definition `yaml:"groups"`
+	}
+	loaded, err := testLoad[config](Bytes("wide-inheritance", []byte(body.String())))
+	if err != nil {
+		t.Fatal(err)
+	}
+	metadata := loaded.Metadata()
+	for name, group := range loaded.Config.Groups {
+		if len(group["base"].Items) != 1 || group["base"].Items[0].Value != "parent" || len(group["child"].Items) != 1 || group["child"].Items[0].Value != "child" {
+			t.Fatalf("independent scope %s composed incorrectly: %#v", name, group)
+		}
+		path := Keys("groups", name, "child", "items").Index(0).Key("value")
+		if info, exists := metadata.Lookup(path); !exists || !info.Explicit || info.Origin.Layer != LayerBase {
+			t.Fatalf("inherited destination lost source metadata at %s", path)
+		}
+	}
+}

internal/engine/inheritance_schedule.go

@@ -47,9 +47,10 @@ func analyzeInheritanceGraphWithCollectionRules(scope *document.Node, scopePath
 		}}
 	}
 
+	policyIndex := newInheritancePolicyIndex(policies)
 	graph := &inheritanceGraph{
 		scope: scope, scopePath: scopePath,
-		collectionPolicies: cloneCollectionPolicyScopeForInheritance(policies), applicationPolicy: applicationPolicy,
+		collectionPolicies: policyIndex.absoluteScope(scopePath), applicationPolicy: applicationPolicy,
 		byName: make(map[string]int, len(scope.Mapping)),
 	}
 	entries := make(map[string]document.Entry, len(scope.Mapping))
@@ -69,7 +70,7 @@ func analyzeInheritanceGraphWithCollectionRules(scope *document.Node, scopePath
 		graph.byName[name] = index
 		graph.nodes = append(graph.nodes, inheritanceTemplate{
 			name: name, entry: entries[name], path: scopePath.Key(name),
-			collectionPolicies: inheritancePolicyScopeAtPath(policies, scopePath.Key(name)),
+			collectionPolicies: policyIndex.localize(scopePath.Key(name)),
 		})
 	}
 
@@ -355,6 +356,7 @@ func applyInheritanceScopeTemplates(adapter inheritanceScopeAdapter, order []int
 	root := adapter.rebuild(resolved)
 	result.collectionPolicies = collectionPolicyScope{}
 	result.origins = directInheritanceOrigins(&document.Node{Kind: root.Kind, Location: root.Location}, adapter.scopePath(), inputOrigins)
+	policyIndex := newInheritancePolicyIndex(adapter.collectionPolicies())
 	for _, slot := range adapter.slots() {
 		if slot.hasTemplate {
 			index := slot.template
@@ -364,7 +366,7 @@ func applyInheritanceScopeTemplates(adapter inheritanceScopeAdapter, order []int
 			}
 			continue
 		}
-		localPolicies := inheritancePolicyScopeAtPath(adapter.collectionPolicies(), slot.path)
+		localPolicies := policyIndex.localize(slot.path)
 		projectInheritancePolicyRoots(&result.collectionPolicies, localPolicies, slot.path)
 		for pointer, origin := range directInheritanceOrigins(slot.value, slot.path, inputOrigins) {
 			result.origins[pointer] = origin

internal/engine/inheritance_scope.go

@@ -109,10 +109,11 @@ func (scope identifiedSequenceInheritanceScope) applicationPolicy() sequencePoli
 func (scope identifiedSequenceInheritanceScope) slotOverhead() int { return 0 }
 func (scope identifiedSequenceInheritanceScope) templates() []inheritanceExecutionTemplate {
 	result := make([]inheritanceExecutionTemplate, len(scope.graph.nodes))
+	policyIndex := newInheritancePolicyIndex(scope.policies)
 	for index, node := range scope.graph.nodes {
 		result[index] = inheritanceExecutionTemplate{
 			entry: document.Entry{Value: node.item}, path: node.path,
-			collectionPolicies: inheritancePolicyScopeAtPath(scope.policies, node.path),
+			collectionPolicies: policyIndex.localize(node.path),
 			parent:             node.parent, hasParent: node.hasParent,
 			hasDirective: node.directive != nil, cancelled: node.cancelled,
 			directive: document.Entry{Value: node.directive}, directivePath: node.directivePath,

internal/engine/inheritance_walk.go

@@ -11,6 +11,7 @@ type inheritanceWalker struct {
 	nodes              int // logical non-directive nodes in the entire working tree
 	origins            map[string]Origin
 	collectionPolicies collectionPolicyScope
+	policyIndex        *inheritancePolicyIndex
 	applicationPolicy  sequencePolicy
 	limits             projectionLimits
 	retained           *retainedNodeCounter
@@ -66,6 +67,7 @@ func expandInheritanceWithCollectionRuleBudgets(input *document.Node, inputOrigi
 		applicationPolicy:  applicationPolicy,
 		retained:           newRetainedNodeCounter(historyLimit),
 	}
+	walker.policyIndex = newInheritancePolicyIndex(walker.collectionPolicies)
 	if problem := walker.visit(&root, Path{}, true); problem != nil {
 		return nil, []Diagnostic{*problem}
 	}
@@ -113,20 +115,13 @@ func (w *inheritanceWalker) visit(node **document.Node, path Path, unscoped bool
 				limits.nodes -= w.nodes - before
 			}
 			resolved, diagnostics := expandSiblingDefinitionsWithCollectionRules(
-				*node, path, limits, w.origins, w.collectionPolicies, w.applicationPolicy, w.retained,
+				*node, path, limits, w.origins, w.policyIndex.absoluteScope(path), w.applicationPolicy, w.retained,
 			)
 			if len(diagnostics) != 0 {
 				return &diagnostics[0]
 			}
 			*node = resolved.root
-			for pointer, entry := range w.collectionPolicies.collections {
-				if pathHasPrefix(entry.destination, path) {
-					delete(w.collectionPolicies.collections, pointer)
-				}
-			}
-			for _, entry := range resolved.collectionPolicies.collections {
-				w.collectionPolicies.set(entry.destination, entry.policy)
-			}
+			w.policyIndex.replace(&w.collectionPolicies, path, resolved.collectionPolicies)
 			deleteOriginSubtree(w.origins, path, original)
 			for pointer, origin := range resolved.origins {
 				w.origins[pointer] = origin
@@ -203,20 +198,13 @@ func (w *inheritanceWalker) expandIdentifiedSequence(node **document.Node, path
 		limits.nodes -= w.nodes - before
 	}
 	resolved, diagnostics := applyIdentifiedInheritanceTemplates(
-		graph, limits, w.origins, w.collectionPolicies, w.applicationPolicy, w.retained,
+		graph, limits, w.origins, w.policyIndex.absoluteScope(path), w.applicationPolicy, w.retained,
 	)
 	if len(diagnostics) != 0 {
 		return &diagnostics[0]
 	}
 	*node = resolved.root
-	for pointer, entry := range w.collectionPolicies.collections {
-		if pathHasPrefix(entry.destination, path) {
-			delete(w.collectionPolicies.collections, pointer)
-		}
-	}
-	for _, entry := range resolved.collectionPolicies.collections {
-		w.collectionPolicies.set(entry.destination, entry.policy)
-	}
+	w.policyIndex.replace(&w.collectionPolicies, path, resolved.collectionPolicies)
 	deleteOriginSubtree(w.origins, path, original)
 	for pointer, origin := range resolved.origins {
 		w.origins[pointer] = origin

internal/engine/item_policy_test.go

@@ -93,7 +93,7 @@ func TestPendingItemCompositionProjectsNestedIdentityDiagnostics(t *testing.T) {
 	}}}
 
 	traversal := importTraversal{accounting: &importAccounting{}, limits: defaultImportLimits()}
-	diagnostics := traversal.composePendingItems(&outer, sequencePolicy{})
+	diagnostics := traversal.composePendingItems(&outer)
 	if len(diagnostics) != 1 {
 		t.Fatalf("diagnostics = %#v, want one conflict", diagnostics)
 	}
@@ -105,7 +105,7 @@ func TestPendingItemCompositionProjectsNestedIdentityDiagnostics(t *testing.T) {
 	}
 }
 
-func TestPendingItemCompositionAppliesApplicationIdentityConstraints(t *testing.T) {
+func TestPendingItemCompositionAppliesOuterApplicationIdentityConstraints(t *testing.T) {
 	t.Parallel()
 
 	itemScope := acquisitionScope{contributions: []acquiredContribution{{
@@ -116,15 +116,15 @@ func TestPendingItemCompositionAppliesApplicationIdentityConstraints(t *testing.
 		destination: Keys("items"), pendingItem: &itemScope,
 	}}}
 	policy, diagnostics := compileSequenceIdentityPolicy([]sequenceIdentityRegistration{{
-		selector: sequenceIdentitySelector{literalSequenceSelector("prompts")}, identityFields: []string{"id"},
+		selector: sequenceIdentitySelector{literalSequenceSelector("items")}, identityFields: []string{"id"},
 	}})
 	if len(diagnostics) != 0 {
 		t.Fatalf("compile application identity: %#v", diagnostics)
 	}
 
-	traversal := importTraversal{accounting: &importAccounting{}, limits: defaultImportLimits()}
-	diagnostics = traversal.composePendingItems(&outer, policy)
-	if len(diagnostics) != 1 || diagnostics[0].Code != CodeComposeInvalid || diagnostics[0].Path.String() != "/items/0/prompts/0/id" {
+	_, diagnostics = composeAcquiredScopeWithPolicy(&outer, collectionPolicyScope{}, policy,
+		newImportInclusionGraph(), nil, nil, nil, &importAccounting{})
+	if len(diagnostics) != 1 || diagnostics[0].Code != CodeComposeInvalid || diagnostics[0].Path.String() != "/items/0/id" {
 		t.Fatalf("application identity diagnostics = %#v", diagnostics)
 	}
 }

internal/engine/metadata_locations_test.go

@@ -0,0 +1,79 @@
+package engine
+
+import (
+	"fmt"
+	"reflect"
+	"strings"
+	"testing"
+
+	"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
+)
+
+func TestMetadataLocationsAcrossBroadAuthoredAndDefaultMappings(t *testing.T) {
+	type config map[string]map[string]string
+	defaults := config{}
+	var body strings.Builder
+	for index := range 96 {
+		name := fmt.Sprintf("authored-%03d", index)
+		fmt.Fprintf(&body, "%s: {present: source}\n", name)
+		defaults[name] = map[string]string{"absent": "default"}
+		defaults[fmt.Sprintf("default-%03d", index)] = map[string]string{"absent": "default"}
+	}
+	input := parseDocumentForDecode(t, body.String())
+	decoded, issues := decodeConfiguration(mustSchemaFor[config](t), reflect.ValueOf(defaults), input)
+	if len(issues) != 0 {
+		t.Fatal(issues)
+	}
+	located := documentWithMetadataLocations(decoded.final, metadataFromDecoded(decoded), input)
+	inputIndex := buildDocumentLookup(input)
+	locatedIndex := buildDocumentLookup(located)
+	for index := range 96 {
+		authored := Keys(fmt.Sprintf("authored-%03d", index))
+		mapping, _ := documentAtPathWithLookup(input, authored, inputIndex)
+		present, _ := documentAtPathWithLookup(input, authored.Key("present"), inputIndex)
+		for _, test := range []struct {
+			path Path
+			want document.Location
+		}{
+			{authored.Key("present"), present.Location},
+			{authored.Key("absent"), mapping.Location},
+			{Keys(fmt.Sprintf("default-%03d", index), "absent"), input.Location},
+		} {
+			node, found := documentAtPathWithLookup(located, test.path, locatedIndex)
+			if !found || node.Location != test.want {
+				t.Fatalf("location at %s = %#v, want %#v", test.path, node, test.want)
+			}
+		}
+	}
+	walkDocument(located, Path{}, func(_ Path, node *document.Node) {
+		for _, entry := range node.Mapping {
+			if entry.KeyLocation != entry.Value.Location || entry.ValueLocation != entry.Value.Location {
+				t.Fatal("entry and value locations disagree")
+			}
+		}
+	})
+	before := decoded.final.Mapping[0].Value.Mapping[0].Value.Text
+	located.Mapping[0].Value.Mapping[0].Value.Text = "changed snapshot"
+	if decoded.final.Mapping[0].Value.Mapping[0].Value.Text != before || input.Mapping[0].Value.Mapping[0].Value.Text != "source" {
+		t.Fatal("location projection changed a borrowed input")
+	}
+}
+
+func TestIndexedNearestLocationsPreserveMissingAndSyntheticCoordinates(t *testing.T) {
+	root := parseDocumentForDecode(t, "mapping: {'0': value}\nitems: [{value: source}]\nscalar: stop\n")
+	lookup := buildDocumentLookup(root)
+	for _, path := range []Path{Path{}, Keys("missing"), Keys("mapping", "missing"), Keys("mapping", "0"), Keys("mapping").Index(0), Keys("items").Index(4), Keys("items").Index(0).Key("missing"), Keys("scalar", "child"), Path{}.Index(-1)} {
+		want, wantOK := nearestDocumentLocation(root, path)
+		got, gotOK := nearestDocumentLocationWithLookup(root, path, lookup)
+		if got != want || gotOK != wantOK {
+			t.Fatalf("nearest location changed for %s: %#v/%t, want %#v/%t", path, got, gotOK, want, wantOK)
+		}
+	}
+	final := parseDocumentForDecode(t, "value: cli\n")
+	synthetic := Location{SourceID: "cli-1"}
+	metadata := Metadata{fields: map[string]FieldInfo{"/value": {Origin: Origin{Layer: LayerCLI, Locations: []Location{synthetic}}}}}
+	located := documentWithMetadataLocations(final, metadata, root)
+	if location := located.Mapping[0].Value.Location; location.SourceID != "cli-1" || location.Line != 0 || location.Column != 0 {
+		t.Fatalf("synthetic CLI location was replaced by fallback: %#v", location)
+	}
+}

internal/engine/overlay.go

@@ -138,8 +138,12 @@ func mergeOverlayMappingsWithCollectionPolicy(
 	for key := range declarationsByPath {
 		registered[key] = true
 	}
+	var overlayLookup *documentLookup
+	if len(result.collectionPolicies.collections) != 0 {
+		overlayLookup = buildDocumentLookup(overlay)
+	}
 	for key := range result.collectionPolicies.collections {
-		if node, found := documentAtPath(overlay, result.collectionPolicies.collections[key].destination); found && node != nil {
+		if node, found := documentAtPathWithLookup(overlay, result.collectionPolicies.collections[key].destination, overlayLookup); found && node != nil {
 			registered[key] = true
 			registeredPaths[key] = result.collectionPolicies.collections[key].destination.clone()
 		}
@@ -210,6 +214,9 @@ func mergeOverlayMappingsWithCollectionPolicy(
 		return nil, canonicalDiagnostics(combinationDiagnostics)
 	}
 
+	// Values are now complete. Index lazily for declaration-only checks;
+	// this lookup is never retained across the value-changing merge above.
+	var retainedLookup *documentLookup
 	for key, grouped := range declarationsByPath {
 		if policyValues[key] {
 			continue
@@ -226,7 +233,10 @@ func mergeOverlayMappingsWithCollectionPolicy(
 		if found {
 			lowerPolicy = &lower
 		}
-		retained, _ := documentAtPath(assembly.current.root, path)
+		if retainedLookup == nil {
+			retainedLookup = buildDocumentLookup(assembly.current.root)
+		}
+		retained, _ := documentAtPathWithLookup(assembly.current.root, path, retainedLookup)
 		if retained != nil && retained.Kind == document.Sequence && len(current.effectiveIdentityFields) != 0 {
 			if _, issues := validateSequenceIdentity(retained, path, current.effectiveIdentityFields, sequenceIdentityDiagnosticCodes{invalid: CodeSequenceInvalid, conflict: CodeSequenceConflict}); len(issues) != 0 {
 				combinationDiagnostics = append(combinationDiagnostics, issues...)

internal/engine/override_context.go

@@ -171,16 +171,23 @@ func documentWithMetadataLocations(root *document.Node, metadata Metadata, fallb
 	if len(fallback) != 0 {
 		fallbackRoot = fallback[0]
 	}
+	var lookup *documentLookup
+	fallbackLocation := func(path Path) (document.Location, bool) {
+		if lookup == nil {
+			lookup = buildDocumentLookup(fallbackRoot)
+		}
+		return nearestDocumentLocationWithLookup(fallbackRoot, path, lookup)
+	}
 	var visit func(*document.Node, Path)
 	visit = func(node *document.Node, path Path) {
 		pointer := path.String()
 		if info, exists := metadata.fields[pointer]; exists {
 			if location, ok := metadataFieldLocation(metadata, pointer, info.Origin); ok {
 				node.Location = location
-			} else if location, ok := nearestDocumentLocation(fallbackRoot, path); ok {
+			} else if location, ok := fallbackLocation(path); ok {
 				node.Location = location
 			}
-		} else if location, ok := nearestDocumentLocation(fallbackRoot, path); ok {
+		} else if location, ok := fallbackLocation(path); ok {
 			node.Location = location
 		}
 		switch node.Kind {
@@ -192,11 +199,11 @@ func documentWithMetadataLocations(root *document.Node, metadata Metadata, fallb
 					if location, ok := metadataFieldLocation(metadata, childPath.String(), info.Origin); ok {
 						entry.KeyLocation = location
 						entry.ValueLocation = location
-					} else if location, ok := nearestDocumentLocation(fallbackRoot, childPath); ok {
+					} else if location, ok := fallbackLocation(childPath); ok {
 						entry.KeyLocation = location
 						entry.ValueLocation = location
 					}
-				} else if location, ok := nearestDocumentLocation(fallbackRoot, childPath); ok {
+				} else if location, ok := fallbackLocation(childPath); ok {
 					entry.KeyLocation = location
 					entry.ValueLocation = location
 				}

internal/engine/override_decode.go

@@ -2,7 +2,6 @@ package engine
 
 import (
 	"reflect"
-	"strings"
 
 	"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
 )
@@ -171,17 +170,16 @@ func updateOverridePresence(presence map[string]inputPresence, existing, incomin
 		return
 	}
 
-	deletePresencePathSubtree(presence, path)
+	deletePresencePathSubtree(presence, existing, path)
 	collectPresence(incoming, path, presence)
 }
 
-func deletePresencePathSubtree(presence map[string]inputPresence, path Path) {
-	pointer := path.String()
-	for candidate := range presence {
-		if candidate == pointer || strings.HasPrefix(candidate, pointer+"/") {
-			delete(presence, candidate)
-		}
-	}
+func deletePresencePathSubtree(presence map[string]inputPresence, existing *document.Node, path Path) {
+	// The typed pre-CLI tree contains every effective descendant. Visit only
+	// the replaced subtree rather than searching unrelated metadata entries.
+	walkDocument(existing, path, func(path Path, _ *document.Node) {
+		delete(presence, path.String())
+	})
 }
 
 func updateOverrideOrigins(origins map[string]Origin, existing, incoming *document.Node, path Path, location document.Location) {
@@ -200,12 +198,9 @@ func updateOverrideOrigins(origins map[string]Origin, existing, incoming *docume
 		return
 	}
 
-	pointer := path.String()
-	for candidate := range origins {
-		if candidate == pointer || strings.HasPrefix(candidate, pointer+"/") {
-			delete(origins, candidate)
-		}
-	}
+	walkDocument(existing, path, func(path Path, _ *document.Node) {
+		delete(origins, path.String())
+	})
 	collectOverlayOrigins(origins, incoming, path, LayerCLI)
 }
 

internal/engine/override_metadata_test.go

@@ -0,0 +1,82 @@
+package engine
+
+import (
+	"fmt"
+	"reflect"
+	"strings"
+	"testing"
+)
+
+func TestBroadPartialCLIOverrideRetainsUnchangedMetadata(t *testing.T) {
+	type child struct {
+		Name string `yaml:"name"`
+	}
+	type record struct {
+		Count   int     `yaml:"count"`
+		Zero    int     `yaml:"zero"`
+		Enabled bool    `yaml:"enabled"`
+		Items   []child `yaml:"items"`
+		Maybe   *child  `yaml:"maybe"`
+	}
+	type config struct {
+		Records map[string]record `yaml:"records"`
+	}
+	var body strings.Builder
+	body.WriteString("records:\n")
+	updates := make(map[string]any)
+	for index := range 96 {
+		name := fmt.Sprintf("key/%d~", index)
+		fmt.Fprintf(&body, "  %s: {count: 9, zero: 0, enabled: false, items: [{name: old}], maybe: {name: old}}\n", name)
+		if index%2 == 0 {
+			updates[name] = map[string]any{"count": 0, "items": []any{}, "maybe": nil}
+		}
+	}
+	input := parseDocumentForDecode(t, body.String())
+	schema := mustSchemaFor[config](t)
+	decoded, issues := decodeConfiguration(schema, reflect.ValueOf(config{}), input)
+	if len(issues) != 0 {
+		t.Fatal(issues)
+	}
+	batch, diagnostics := preparePostStructuralOverrides(overridePreparationInput{root: input}, nil,
+		[]Override{{Path: Keys("records"), Value: updates, Flag: "--records"}}, nil, defaultImportLimits())
+	if len(diagnostics) != 0 {
+		t.Fatal(diagnostics)
+	}
+	result, issues := applyDecodedOverrides(schema, decoded, batch, decodeContext{})
+	if len(issues) != 0 {
+		t.Fatal(issues)
+	}
+	for index := range 96 {
+		name := fmt.Sprintf("key/%d~", index)
+		path := Keys("records", name)
+		value := result.value.Interface().(config).Records[name]
+		for _, key := range []string{"zero", "enabled"} {
+			pointer := path.Key(key).String()
+			if _, explicit := result.presence[pointer]; !explicit || result.origins[pointer].Layer != LayerBase {
+				t.Fatalf("retained explicit field lost metadata at %s", pointer)
+			}
+		}
+		if index%2 != 0 {
+			if value.Count != 9 || len(value.Items) != 1 || value.Maybe == nil || result.origins[path.Key("count").String()].Layer != LayerBase {
+				t.Fatalf("omitted record changed at %s", path)
+			}
+			continue
+		}
+		if value.Count != 0 || len(value.Items) != 0 || value.Maybe != nil {
+			t.Fatalf("CLI replacement failed at %s: %#v", path, value)
+		}
+		for _, removed := range []Path{path.Key("items").Index(0), path.Key("items").Index(0).Key("name"), path.Key("maybe").Key("name")} {
+			_, present := result.presence[removed.String()]
+			_, origin := result.origins[removed.String()]
+			if present || origin {
+				t.Fatalf("removed descendant kept metadata at %s", removed)
+			}
+		}
+		for _, key := range []string{"count", "items", "maybe"} {
+			pointer := path.Key(key).String()
+			if presence, ok := result.presence[pointer]; !ok || presence.location.SourceID != batch.overrides[0].source.ID || result.origins[pointer].Layer != LayerCLI {
+				t.Fatalf("CLI field lost metadata at %s", pointer)
+			}
+		}
+	}
+}

internal/engine/sequence_projection.go

@@ -1,33 +1,17 @@
 package engine
 
 type sequenceItemProjectionIndex struct {
-	outputByInput []int
-	inputByOutput []int
+	outputByInput map[int]int
+	inputByOutput map[int]int
 }
 
 func newSequenceItemProjectionIndex(projections []sequenceItemProjection) sequenceItemProjectionIndex {
-	maxInput := -1
-	maxOutput := -1
-	for _, projection := range projections {
-		if projection.inputIndex > maxInput {
-			maxInput = projection.inputIndex
-		}
-		if projection.outputIndex > maxOutput {
-			maxOutput = projection.outputIndex
-		}
-	}
 	index := sequenceItemProjectionIndex{
-		outputByInput: make([]int, maxInput+1),
-		inputByOutput: make([]int, maxOutput+1),
-	}
-	for input := range index.outputByInput {
-		index.outputByInput[input] = -1
-	}
-	for output := range index.inputByOutput {
-		index.inputByOutput[output] = -1
+		outputByInput: make(map[int]int, len(projections)),
+		inputByOutput: make(map[int]int, len(projections)),
 	}
 	for _, projection := range projections {
-		if projection.inputIndex >= 0 && projection.inputIndex < len(index.outputByInput) && projection.outputIndex >= 0 && projection.outputIndex < len(index.inputByOutput) {
+		if projection.inputIndex >= 0 && projection.outputIndex >= 0 {
 			index.outputByInput[projection.inputIndex] = projection.outputIndex
 			index.inputByOutput[projection.outputIndex] = projection.inputIndex
 		}
@@ -36,15 +20,11 @@ func newSequenceItemProjectionIndex(projections []sequenceItemProjection) sequen
 }
 
 func (index sequenceItemProjectionIndex) outputForInput(inputIndex int) (int, bool) {
-	if inputIndex < 0 || inputIndex >= len(index.outputByInput) || index.outputByInput[inputIndex] < 0 {
-		return 0, false
-	}
-	return index.outputByInput[inputIndex], true
+	output, found := index.outputByInput[inputIndex]
+	return output, found
 }
 
 func (index sequenceItemProjectionIndex) inputForOutput(outputIndex int) (int, bool) {
-	if outputIndex < 0 || outputIndex >= len(index.inputByOutput) || index.inputByOutput[outputIndex] < 0 {
-		return 0, false
-	}
-	return index.inputByOutput[outputIndex], true
+	input, found := index.inputByOutput[outputIndex]
+	return input, found
 }

internal/engine/sequence_projection_test.go

@@ -0,0 +1,49 @@
+package engine
+
+import "testing"
+
+func TestSparseSequenceProjectionIndexesOnlyAssignedPositions(t *testing.T) {
+	index := newSequenceItemProjectionIndex([]sequenceItemProjection{
+		{inputIndex: 0, outputIndex: 9000}, {inputIndex: 3000, outputIndex: 2},
+	})
+	if len(index.outputByInput) != 2 || len(index.inputByOutput) != 2 {
+		t.Fatal("sparse projection retained unassigned positions")
+	}
+	for input, output := range map[int]int{0: 9000, 3000: 2} {
+		if got, found := index.outputForInput(input); !found || got != output {
+			t.Fatalf("input %d = %d, %t", input, got, found)
+		}
+		if got, found := index.inputForOutput(output); !found || got != input {
+			t.Fatalf("output %d = %d, %t", output, got, found)
+		}
+	}
+	for _, position := range []int{-1, 1, 10000} {
+		if _, found := index.outputForInput(position); found {
+			t.Fatalf("unassigned input %d exists", position)
+		}
+		if _, found := index.inputForOutput(position); found {
+			t.Fatalf("unassigned output %d exists", position)
+		}
+	}
+}
+
+func TestContiguousSequenceRangesKeepArithmeticLookup(t *testing.T) {
+	ranges := make([]sequenceRange, 128)
+	for position := range ranges {
+		ranges[position] = sequenceRange{sourceIndex: position, path: Keys("items"), start: position, end: position + 1}
+	}
+	for _, assigned := range indexSequenceRanges(ranges) {
+		if len(assigned.projection.outputByInput) != 0 || len(assigned.projection.inputByOutput) != 0 {
+			t.Fatal("contiguous range materialized a projection index")
+		}
+		if output, found := assigned.outputForInput(0); !found || output != assigned.start {
+			t.Fatal("arithmetic forward lookup lost its offset")
+		}
+		if input, found := assigned.inputForOutput(assigned.start); !found || input != 0 {
+			t.Fatal("arithmetic reverse lookup lost its offset")
+		}
+		if _, found := assigned.inputForOutput(assigned.end); found {
+			t.Fatal("range includes its exclusive end")
+		}
+	}
+}

internal/engine/validation.go

@@ -124,6 +124,10 @@ func validationMutationDiagnostic(path Path) Diagnostic {
 }
 
 func nearestDocumentLocation(root *document.Node, path Path) (document.Location, bool) {
+	return nearestDocumentLocationWithLookup(root, path, nil)
+}
+
+func nearestDocumentLocationWithLookup(root *document.Node, path Path, lookup *documentLookup) (document.Location, bool) {
 	if root == nil || !path.valid() {
 		return document.Location{}, false
 	}
@@ -135,18 +139,22 @@ func nearestDocumentLocation(root *document.Node, path Path) (document.Location,
 			if segment.kind == pathSegmentIndex {
 				return nearest, locationKnown(nearest)
 			}
-			found := false
-			for _, entry := range current.Mapping {
-				if entry.Key == segment.key {
-					current = entry.Value
-					nearest = current.Location
-					found = true
-					break
+			position := -1
+			if lookup != nil {
+				position = lookup.find(current, segment.key)
+			} else {
+				for index, entry := range current.Mapping {
+					if entry.Key == segment.key {
+						position = index
+						break
+					}
 				}
 			}
-			if !found {
+			if position < 0 {
 				return nearest, locationKnown(nearest)
 			}
+			current = current.Mapping[position].Value
+			nearest = current.Location
 		case document.Sequence:
 			index, ok := schemaIndex(segment)
 			if !ok || index >= len(current.Sequence) {