# Sequence-Item Placement and Identity-Based Composition Status: implemented. This roadmap records the target state and accepted behavior. The [configuration contract](../config.md#strict-document-composition) defines the current behavior, and the [implementation plan](implementation.md) records delivery sequencing and validation. ## Goal and current baseline Let consumers assemble a list of records from independently authored mapping files. Placement makes a source mapping one sequence item; identity-based strict composition combines distinct items and rejects repeated identities. These are complementary operations, with explicit consumer opt-in. Sources can explicitly place mappings as sequence items at non-root destinations, and definitions can register literal sequence paths with one literal string identity field for duplicate-rejecting strict composition. Ordinary unregistered sequences retain strict conflict behavior. [Precedence assembly](../config.md#layered-loading) and [selected overlays](../config.md#selected-overlays) continue to replace sequences wholesale. The canonical contract documents the exact rules and limits. This roadmap covers a narrower feature than the [identified record sequences candidate](future.md#1-identified-record-sequences). Composite identities, recursive merging of matching records, identity-based overlays, and record inheritance are not requested here. That scope statement does not establish permanent architectural prohibitions. ## Targeted behavior ### PromptKit use case Each file defines one prompt directly. `prompt_one.yml`: ```yaml id: prompt.one text: "foo" ``` `prompt_two.yml`: ```yaml id: prompt.two text: "bar" ``` The consumer selects item placement at `prompts` for both sources and registers `id` as that sequence's identity field. Within one strict composition tier, the result is: ```yaml prompts: - id: prompt.one text: "foo" - id: prompt.two text: "bar" ``` The files need neither a filename-derived wrapper key nor their own list wrapper. Existing sources that already contain `prompts: [...]` can contribute to the same registered sequence under the same composition rule. ### Explicit sequence-item placement - The destination identifies where content belongs; an explicit placement mode selects ordinary mapping placement or placement as one sequence item. Preserve existing behavior when the new mode is not selected. Do not infer the mode from document shape, filename, or the presence of an `id` field. - Support item placement for application-supplied inputs and YAML path/glob imports. Each matched file of an item-placement glob contributes one item, using existing match and traversal ordering. Physical import-path resolution remains relative to the declaring file. - Wrap the source's application mapping as one item at the destination sequence; the new operation is not flattening a source sequence or splitting a mapping into several items. Existing supported source forms should receive equivalent placement behavior; this does not expand accepted source-root shapes. - Item placement supplies structure, not identity values. Keep authored `id` and other fields in the item. Source directives and version headers retain their own processing rules, with the item-placement interpretation recorded under Settled Decisions. - Preserve existing absence distinctions: `NoSource()` and an unmatched optional import contribute nothing. An actual empty mapping contributes an empty item, which fails the agreed identity requirement when that item participates in a registered composition sequence. Item placement is independent of composition registration. Ordinary imports inside an item compose into its mapping; public source evidence reports the placement mode and destination container. The settled decisions below define these behaviors in detail. ### Identity-based strict composition - Consumers designate sequence paths and one direct string identity field for each. Unregistered sequences retain their current composition behavior. - Each participating record is a mapping with the designated string field. Missing, null, and non-string identities are invalid. Compare exact strings without trimming, case folding, normalization, or numeric coercion. Empty strings are valid identities; repeated empty identities conflict. - Append distinct records in contribution order, preserving each source list's order. Use existing deterministic traversal within a strict tier rather than sorting by identity. - Repeated identities conflict across contributions and within one contributing list, including a list with no peer contribution. Equal bodies and disjoint bodies both conflict; matching records do not recursively merge. - An empty sequence contributes no records. Two empty sequences compose, and an explicitly contributed empty sequence remains present. - Apply composition to main inputs and their imports and within each underlay group, including placed item contributions and already-authored sequences. Identity uniqueness is scoped to the sequence in that strict tier. - A composition error cannot be hidden by a higher-precedence replacement. Use normal load errors and the existing failed-load result contract. This differs from mapping composition at a shared key: mapping values can recurse, whereas a repeated record identity conflicts. It does not establish final uniqueness after defaults or application resolution. Identity values are not supplied by those later phases; final domain validation remains application-owned. ### Precedence and other operations Between underlay tiers, between underlays and the main tier, and in selected overlays, sequences continue to replace lower sequences wholesale. `[]` clears the lower list. The same identity can occur in different tiers without a cross-tier duplicate error. Item placement does not turn precedence assembly into append semantics. Whole-sequence CLI overrides, defaults, application resolution, and existing sibling inheritance retain their behavior. This feature does not make records inherit from one another by identity. Effective-configuration diffs continue to compare final sequences atomically. Replacement-only selected overlays and synthetic CLI overrides do not acquire identity validation from composition registration; existing decoding and application validation still apply. ### Existing inspection and ownership obligations A field at source path `text` can become `prompts[1].text`; a record originally at source index zero can acquire a different effective index after composition. Keep original physical locations and resource contexts associated with the right record while reflecting effective placement in presence and origin queries. History and explanations must represent accumulation within a tier and actual replacement between tiers. Replaying each contribution as a replacement would lose earlier records. Apply existing historical sensitivity and disclosure rules when indexes change or sequences are replaced, and preserve detached public records and existing input/result ownership. These are applications of current contracts, not a new promise of persistent identity-based paths across loads. ## Work needed The target state requires changes in these areas; execution order belongs to the [implementation plan](implementation.md): - **Public API and import syntax:** introduce the selected composition registration and placement controls at the existing root-owned boundaries; convert them into private engine state. Define invalid-option handling and retain current defaults for callers that do not opt in. - **Acquisition and placement:** support item contributions in ordinary and layered loading, input/import destinations, and glob expansion. Implement the selected nested-import and source-version interpretation without changing acquisition identity or physical resource resolution. - **Composition and retained evidence:** extend claim validation, tree assembly, origins, and history together. The current composer in `internal/engine/compose.go` collects claims by path and stores history applications as well as the final tree. Updating only the tree merge is insufficient. Keep tier boundaries explicit and preserve the existing limits and accounting through the new wrapper and accumulation operations. - **Inspection integration:** carry source-to-effective placement through history, explanations, sensitivity, resource contexts, and public source records. Use the smallest internal representation that preserves those existing contracts; this roadmap does not prescribe a new indexing or caching subsystem. - **Behavioral verification:** cover direct and glob-imported item sources, already-authored lists, mixed contributions, duplicate/malformed identities, empty and absent contributions, ordering, mounted paths, and nested imports. Verify whole-list replacement, errors hidden by later tiers, version and structural processing, remapped indexes, resource contexts, historical redaction, and detached records. Protect current unregistered behavior, defaults/resolution, and atomic diffs with proportionate regression coverage. - **Documentation and adoption:** provide a maintained PromptKit-shaped example and update public declaration comments, configuration/inspection contracts, consumer guidance, and relevant internal documentation when behavior lands. Record the significant design decision under the architecture policy and keep the broader candidate in `future.md` distinct from the implemented subset. Completion means the PromptKit example works through public inputs and imports, composition and replacement behave as agreed, and configuration values agree with their inspection evidence. Validation follows the [development guide](../development.md#baseline-validation). ## Settled Decisions The following choices are accepted for this feature. No substantive design choices remain open in this roadmap. The implementation plan selects exact public names, YAML option names, and existing diagnostic-code mappings. Internal representations remain implementation choices, subject to the existing contracts and the behavior recorded here. ### 1. Public controls and ownership Put sequence path/identity-field registrations on `Definition`. Put an explicit placement mode beside input destination controls, with equivalent YAML import syntax and support in ordinary loading. This separates reusable composition policy from placement of individual sources. Default placement remains unchanged. ### 2. Destination and registration paths Use literal mapping-key paths to identify destination sequences and match registrations at their effective location after placement. For example, a source mounted beneath `application` uses a registration at `application.prompts`. Preserve existing destination validation without adding new reserved names. Permit unused registrations, coalesce identical registrations, and reject conflicting identity fields for the same path. Ancestor/descendant registrations apply only where their literal paths reach sequences; they do not implicitly traverse every record. Sequence-index selectors and wildcard matching are outside this initial feature. Item placement identifies a sequence beneath the configuration root; a root sequence would expand the existing mapping-root model and is not included. These are the selected feature boundaries, not permanent architectural exclusions. Nested placement follows decision 4 without expanding registration addressing. ### 3. Item placement without identity registration Allow item placement independently. It contributes a singleton sequence without requiring an identity registration. Existing rules apply at unregistered paths: two present sequences conflict in one strict tier, and an upper-tier sequence replaces a lower one. Do not infer append semantics merely from item placement. ### 4. Imports declared inside an item-placed source Compose ordinary imports into the declaring item's mapping. For an import declared at the item root, an omitted destination or `destination: []` means that item context; `[metadata]` places the helper inside the item's `metadata`. Further relative destinations continue from the immediately importing document's placement within that context. Physical import paths still resolve from the declaring file. Item mode is explicit on each import, not inherited automatically. A collection manifest explicitly imports peer prompt files as items at `prompts`. A helper import supplies fields of the declaring item and need not declare its own ID. An explicitly item-placed import inside a prompt can contribute to a nested sequence. Registrations do not reach through outer sequence indexes in this initial feature, so those nested sequences retain atomic composition. Multiple contributions to such a nested sequence therefore follow ordinary strict conflict rules. This permits nested placement without introducing index selectors or wildcard matching. ### 5. Identity-validation boundary and structural processing Validate each completed record after applicable source-header processing and ordinary helper-import composition, before combining records in the collection and before later precedence replacement. A helper may supply the identity field; it need not be authored in the item file itself. Duplicate fields within the item still follow ordinary strict mapping composition. Defaults and application resolution cannot supply missing identities because they run later. Preserve existing structural semantics. An item has no mapping-sibling namespace for record inheritance, while mappings inside it can use existing inheritance. Consumed structural fields and headers do not become application identity values. Existing processing rules govern overlap with those controls; no new identity-field-name blacklist is introduced. ### 6. Empty identities and literal field names Accept empty string identities; repeated empty identities conflict. Interpret the designated direct field name literally, including unusual or empty YAML mapping keys where existing processing permits them. Presence and string type suffice for identity comparison. Nonempty-ID requirements, naming conventions, and whitespace restrictions remain application-owned validation choices. ### 7. Non-sequence values at registered paths Registration selects how actual sequences compose; it is not an additional schema declaration. Missing and non-sequence values retain existing composition and decoding behavior. Sequence/null and sequence/mapping overlaps still conflict in strict composition. Item placement contributes an actual sequence and follows these rules. Registration does not independently reject every non-sequence source value that a later tier might replace. ### 8. Validation of replacement-only sequences Apply identity validation to registered sequences participating in strict composition, including singleton contributions. Selected-overlay sequences and synthetic CLI replacements retain their existing validation. Defaults and application callbacks also retain their behavior. An overlay can therefore replace a list with duplicate IDs unless the consuming application's validator rejects them. The identity rule governs composition, not final configuration validity. ### 9. Source-version checks for item-placed main inputs Support item placement for main inputs as well as imported and underlay sources. Enabled source-version modes inspect declarations in source-local coordinates before placement. Default mode retains its existing effective-path interpretation. Accepted non-main headers are consumed; the main header remains application content, following existing rules. The retained main header's effective location includes its actual item index. Existing ownership protection follows that location and must not accidentally protect another record because its source index differs from its effective index. Preserve source-local version evidence and diagnostics where the existing contract uses them. This feature introduces neither a new version policy nor an exception to existing version-path protections. ### 10. Public placement evidence and diagnostics Keep `SourceInfo.Destination` describing the effective destination container and expose the placement mode. Use existing effective-path metadata to describe resulting item paths, retaining physical source locations and resource contexts. Do not promise one permanent item index per source: replacement can remove its contribution, and contribution order can differ between loads. Duplicate diagnostics identify conflicting locations and the sequence path without exposing potentially sensitive identity values. Reuse existing diagnostic categories where appropriate. Resolved historical item paths and persistent identity-addressed paths in source records are not part of this feature.