diff --git a/docs/roadmap/sequence-composition.md b/docs/roadmap/sequence-composition.md index 670e7a4..f61802e 100644 --- a/docs/roadmap/sequence-composition.md +++ b/docs/roadmap/sequence-composition.md @@ -1,8 +1,8 @@ # Sequence-Item Placement and Identity-Based Composition -Status: agreed feature direction; not implemented. This roadmap defines the -intended scope and remaining design choices. A separate staged implementation -plan will be prepared after the roadmap is finalized. +Status: design decisions settled; not implemented. This roadmap records the +agreed scope and behavior for final review. A separate staged implementation +plan will be prepared after that review. ## Goal and current baseline @@ -77,15 +77,17 @@ registered sequence under the same composition rule. placement behavior; this does not expand accepted source-root shapes. - Item placement supplies structure, not identity values. Keep authored `id` and other fields in the item. Source directives and version headers retain their - own processing rules; their interactions with the new placement context must - be resolved as described under Open Decisions. + own processing rules, with the item-placement interpretation recorded under + Settled Decisions. - Preserve existing absence distinctions: `NoSource()` and an unmatched optional import contribute nothing. An actual empty mapping contributes an empty item, which fails the agreed identity requirement when that item participates in a registered composition sequence. -Nested imports, placement without a composition registration, and the public -representation of item placement remain decisions rather than implicit rules. +Item placement is independent of composition registration. Ordinary imports +inside an item compose into its mapping; public source evidence reports the +placement mode and destination container. The settled decisions below define +these behaviors in detail. ### Identity-based strict composition @@ -94,7 +96,7 @@ representation of item placement remain decisions rather than implicit rules. - Each participating record is a mapping with the designated string field. Missing, null, and non-string identities are invalid. Compare exact strings without trimming, case folding, normalization, or numeric coercion. Empty - string handling remains open. + strings are valid identities; repeated empty identities conflict. - Append distinct records in contribution order, preserving each source list's order. Use existing deterministic traversal within a strict tier rather than sorting by identity. @@ -125,8 +127,9 @@ into append semantics. Whole-sequence CLI overrides, defaults, application resolution, and existing sibling inheritance retain their behavior. This feature does not make records inherit from one another by identity. Effective-configuration diffs continue to -compare final sequences atomically. Whether to validate identities in a -replacement-only overlay source is an open decision, separate from replacement. +compare final sequences atomically. Replacement-only selected overlays and +synthetic CLI overrides do not acquire identity validation from composition +registration; existing decoding and application validation still apply. ### Existing inspection and ownership obligations @@ -144,7 +147,7 @@ contracts, not a new promise of persistent identity-based paths across loads. ## Work needed -The staged plan will sequence these areas after the decisions below are settled: +After final roadmap review, the staged plan will sequence these areas: - **Public API and import syntax:** introduce the selected composition registration and placement controls at the existing root-owned boundaries; @@ -184,174 +187,133 @@ with their inspection evidence. Validation follows the [development guide](../development.md#baseline-validation). This documentation pass neither implements the feature nor runs performance benchmarks. -## Open Decisions +## Settled Decisions -Recommendations below are proposals, not accepted restrictions. Resolve them -before preparing the staged implementation plan. Exact public names and error -code spellings can be selected with the API design; implementation details that -do not affect observable behavior need not become additional contracts. +The following choices are accepted for this feature. No substantive design +choices remain open in this roadmap. Exact public names, YAML option names, +and error-code spellings will be selected during API design in the staged plan. +Internal representations remain implementation choices, subject to the existing +contracts and the behavior recorded here. ### 1. Public controls and ownership -**Recommendation:** put sequence path/identity-field registrations on `Definition` -and an explicit placement mode beside input destination controls, with equivalent -YAML import syntax. Expose the corresponding choice for ordinary loading too. -This separates reusable composition policy from placement of particular sources -and keeps default placement unchanged. - -**Alternative:** put composition registrations on load options, allowing different -composition rules for each invocation of one definition. That offers flexibility -but duplicates policy across ordinary/layered load forms. A dedicated item-input -constructor is also viable instead of a mode field, but needs equivalent import -syntax and a clear way to combine destination and source options. +Put sequence path/identity-field registrations on `Definition`. Put an explicit +placement mode beside input destination controls, with equivalent YAML import +syntax and support in ordinary loading. This separates reusable composition +policy from placement of individual sources. Default placement remains unchanged. ### 2. Destination and registration paths -**Recommendation:** use literal mapping-key paths to identify destination -sequences and match composition registrations after placement. This fits the -existing destination vocabulary and the requested `prompts` use case. Preserve -existing destination validation; do not silently add new reserved names. Keep -empty/root item destinations invalid if they would replace the currently required -mapping document root with a sequence. These are proposed scope choices, not -already-approved new validation rules. +Use literal mapping-key paths to identify destination sequences and match +registrations at their effective location after placement. For example, a source +mounted beneath `application` uses a registration at `application.prompts`. +Preserve existing destination validation without adding new reserved names. -Allow unused registrations for optional configuration. Coalesce identical -registrations and reject conflicting identity rules for the same path. For -ancestor/descendant registrations, apply a rule only where its literal path -actually reaches a sequence; it does not implicitly traverse every record. +Permit unused registrations, coalesce identical registrations, and reject +conflicting identity fields for the same path. Ancestor/descendant registrations +apply only where their literal paths reach sequences; they do not implicitly +traverse every record. -**Alternative:** support explicit sequence-index paths, including sequences inside -appended records, or sequence-valued configuration roots. These are expressive -extensions, but require decisions about source versus effective indexes and -root decoding. Nested-item destinations arising through imports also need the -context rule in decision 4; they must not accidentally introduce wildcard path -matching. Rejecting even identical duplicate registrations is another viable -option, but catches redundant setup at the cost of an additional restriction. +Sequence-index selectors and wildcard matching are outside this initial feature. +Item placement identifies a sequence beneath the configuration root; a root +sequence would expand the existing mapping-root model and is not included. +These are the selected feature boundaries, not permanent architectural exclusions. +Nested placement follows decision 4 without expanding registration addressing. ### 3. Item placement without identity registration -**Recommendation:** allow item placement independently. It contributes a singleton -sequence; without a registration, two present sequences still conflict during -strict composition, while precedence still replaces them. This keeps placement -orthogonal and allows a consumer to place a single item without inventing an -identity rule. - -**Alternative:** require every item destination to have a composition registration. -This catches a likely setup omission early, but unnecessarily couples placement -to identity validation for consumers that only need one item. Implicitly appending -unregistered sequences would change the agreed opt-in composition boundary. +Allow item placement independently. It contributes a singleton sequence without +requiring an identity registration. Existing rules apply at unregistered paths: +two present sequences conflict in one strict tier, and an upper-tier sequence +replaces a lower one. Do not infer append semantics merely from item placement. ### 4. Imports declared inside an item-placed source -**Recommendation:** compose ordinary imports into that source's item mapping; -interpret their relative destinations inside the item. An explicit item-placement -import then contributes an item to a sequence within that context. A collection -manifest can import multiple peer prompt files as items at `prompts`. Do not -implicitly inherit item mode and turn every helper import into another prompt. -This permits helper fragments without requiring each helper to declare an ID. +Compose ordinary imports into the declaring item's mapping. For an import +declared at the item root, an omitted destination or `destination: []` means that +item context; `[metadata]` places the helper inside the item's `metadata`. +Further relative destinations continue from the immediately importing document's +placement within that context. Physical import paths still resolve from the +declaring file. -This recommendation introduces an item-local composition context. Specify how -omitted and empty destinations resolve, how nested contexts are represented, -and how registrations address any nested destination sequences consistently -with decision 2 before accepting it. +Item mode is explicit on each import, not inherited automatically. A collection +manifest explicitly imports peer prompt files as items at `prompts`. A helper +import supplies fields of the declaring item and need not declare its own ID. -**Alternative:** keep inherited destinations anchored at the enclosing collection -and require an explicit mode on each imported contribution. That can simplify -peer-item expansion, but ordinary helper mappings would collide with the -collection sequence rather than extend the declaring record. Inheriting item -mode as well would make peer imports concise, but needs an explicit way to -request helpers within one record. Do not reject nested imports merely to avoid -settling their interpretation. +An explicitly item-placed import inside a prompt can contribute to a nested +sequence. Registrations do not reach through outer sequence indexes in this +initial feature, so those nested sequences retain atomic composition. Multiple +contributions to such a nested sequence therefore follow ordinary strict +conflict rules. This permits nested placement without introducing index selectors +or wildcard matching. ### 5. Identity-validation boundary and structural processing -**Recommendation:** validate each contributed record after its source directives -and applicable version-header processing, and after any item-local mapping -composition chosen in decision 4, but before collection composition and later -precedence replacement. Ordinary helper imports can therefore supply fields of -one record; each completed record still needs its identity. Defaults and -application callbacks do not supply missing identities at this stage. +Validate each completed record after applicable source-header processing and +ordinary helper-import composition, before combining records in the collection +and before later precedence replacement. A helper may supply the identity field; +it need not be authored in the item file itself. Duplicate fields within the +item still follow ordinary strict mapping composition. Defaults and application +resolution cannot supply missing identities because they run later. -Keep existing `extends` semantics: an item itself has no mapping-sibling namespace -for record inheritance, while mappings inside an item may have one. Consumed -structural fields or headers do not become application identity values. Determine -whether a proposed identity-field selection overlaps such controls through the -existing processing rules rather than inventing a blanket field-name blacklist. - -**Alternative:** require the directly authored item file to contain its own ID -before processing helpers. This simplifies early duplicate detection but adds a -restriction not needed for the requested assembled-record behavior. Validating -only the final configuration is not sufficient to catch the agreed strict-tier -conflicts that later replacements would hide. +Preserve existing structural semantics. An item has no mapping-sibling namespace +for record inheritance, while mappings inside it can use existing inheritance. +Consumed structural fields and headers do not become application identity +values. Existing processing rules govern overlap with those controls; no new +identity-field-name blacklist is introduced. ### 6. Empty identities and literal field names -**Recommendation:** accept empty string identity values, with repeated empty -values conflicting. Interpret the designated direct field name literally, -including unusual or empty YAML mapping keys where existing processing permits -them. Presence and string type suffice for comparison; applications can impose -stronger naming rules through their own validation. - -**Alternative:** require nonempty identity values or nonempty field names. This -can catch mistakes earlier but imposes additional domain conventions. Exact -comparison without trimming or normalization remains the agreed behavior. +Accept empty string identities; repeated empty identities conflict. Interpret +the designated direct field name literally, including unusual or empty YAML +mapping keys where existing processing permits them. Presence and string type +suffice for identity comparison. Nonempty-ID requirements, naming conventions, +and whitespace restrictions remain application-owned validation choices. ### 7. Non-sequence values at registered paths -**Recommendation:** let registration select how actual sequences compose, without -asserting that every source must contain a sequence at that path. Missing values -and non-sequence values retain existing composition and decoding behavior. -Sequence/null and sequence/mapping overlaps still conflict; item placement -contributes an actual sequence and follows those same rules. - -**Alternative:** reject every present non-null non-sequence source value at a -registered path, even if a higher tier would replace it. This catches misplaced -configuration earlier but makes registration an extra source-shape validator. -If selected, define null behavior explicitly rather than equating null with an -empty sequence. +Registration selects how actual sequences compose; it is not an additional +schema declaration. Missing and non-sequence values retain existing composition +and decoding behavior. Sequence/null and sequence/mapping overlaps still +conflict in strict composition. Item placement contributes an actual sequence +and follows these rules. Registration does not independently reject every +non-sequence source value that a later tier might replace. ### 8. Validation of replacement-only sequences -**Recommendation:** apply identity validation where source sequences participate -in strict composition, including singleton contributions, and preserve existing -validation for replacement-only selected overlays and synthetic CLI overrides. -Defaults and callbacks retain the agreed behavior. This keeps the new check tied -to its purpose and avoids broadening source validation merely for uniformity. -This revises the previous draft's recommendation to validate overlay identities. +Apply identity validation to registered sequences participating in strict +composition, including singleton contributions. Selected-overlay sequences and +synthetic CLI replacements retain their existing validation. Defaults and +application callbacks also retain their behavior. -**Alternative:** also require replacement-only source sequences to have valid, -unique identities. Authoring rules would be more uniform, but this would reject -some overlay configurations that do not need identity matching to replace a -list. It needs explicit acceptance; whole-sequence replacement does not imply it. +An overlay can therefore replace a list with duplicate IDs unless the consuming +application's validator rejects them. The identity rule governs composition, +not final configuration validity. ### 9. Source-version checks for item-placed main inputs -**Recommendation:** keep enabled source-header checks source-local before item -placement and preserve accepted header consumption/retention rules. Define the -main header's effective path, protected-path behavior, and reported evidence -using its actual item placement. Check ordinary and layered loading and all -source-version modes, including default-mode effective-path interpretation. -A sequence identity rule must not silently become a version policy. +Support item placement for main inputs as well as imported and underlay sources. +Enabled source-version modes inspect declarations in source-local coordinates +before placement. Default mode retains its existing effective-path interpretation. +Accepted non-main headers are consumed; the main header remains application +content, following existing rules. -**Alternative:** initially support item placement only for non-main contributions, -leaving the main document in its existing mapping placement model. This covers -a collection manifest importing prompt items but adds a source-role restriction -beyond the requested general input capability. Prefer resolving the main-path -semantics over imposing that restriction for implementation convenience. +The retained main header's effective location includes its actual item index. +Existing ownership protection follows that location and must not accidentally +protect another record because its source index differs from its effective +index. Preserve source-local version evidence and diagnostics where the existing +contract uses them. This feature introduces neither a new version policy nor +an exception to existing version-path protections. ### 10. Public placement evidence and diagnostics -**Recommendation:** retain `SourceInfo.Destination` as the effective -container destination and add the minimum placement information needed to -identify item mode. Use existing effective-path metadata for final indexes; -a source need not have one permanent item index after replacement. Diagnostics -should identify conflicting locations and paths without exposing identity -values before sensitivity is known. Reuse existing error categories where they -accurately describe the failure. +Keep `SourceInfo.Destination` describing the effective destination container and +expose the placement mode. Use existing effective-path metadata to describe +resulting item paths, retaining physical source locations and resource contexts. +Do not promise one permanent item index per source: replacement can remove its +contribution, and contribution order can differ between loads. -**Alternative:** report a resolved item path directly in source records, with -explicit semantics for replaced items, helper sources, and different historical -indexes. This is convenient for consumers but introduces a broader inspection -contract. A permanent identity-addressed source path is not needed for this use -case. Decide the public evidence shape without promising stable indexes across -loads or changes in contribution order. +Duplicate diagnostics identify conflicting locations and the sequence path +without exposing potentially sensitive identity values. Reuse existing diagnostic +categories where appropriate. Resolved historical item paths and persistent +identity-addressed paths in source records are not part of this feature.