From a5f0b92ee9be0ff3395d30de6942dc9259f021fa Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 23 Oct 2026 01:56:00 +0000 Subject: [PATCH] Add dynamic identity constraints to the sequence roadmap --- docs/roadmap/roadmap.md | 180 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 165 insertions(+), 15 deletions(-) diff --git a/docs/roadmap/roadmap.md b/docs/roadmap/roadmap.md index 605ffb9..e7946c5 100644 --- a/docs/roadmap/roadmap.md +++ b/docs/roadmap/roadmap.md @@ -1,9 +1,10 @@ # Sequence Identity And Combination Roadmap -Status: settled feature roadmap, ready for a staged implementation plan. This -is the targeted end state, not an implemented contract. The subsequent plan will -specify execution order, the exact Go API, internal bookkeeping, and validation -stages. +Status: feature roadmap with dynamic application identity constraints included. +The remaining registration API decision is recorded under +[Open Decisions](#open-decisions). This is the targeted end state, not an +implemented contract. The subsequent plan will specify execution order, exact +Go names and signatures, internal bookkeeping, and validation stages. ## Purpose And Scope @@ -19,7 +20,9 @@ identity. The motivating use case is a prompt catalog whose source-backed baseline supplies records and whose application or installation layers add or supersede records. Applications can optionally impose identity constraints that configuration must -respect and that hold in the final loaded configuration. +respect and that hold in the final loaded configuration. These constraints can +cover exact destinations or collections beneath dynamically named mapping keys, +without requiring the application to discover those names before loading. The feature covers ordinary and layered loading, imported items, inline sequences, selected overlays, and sibling inheritance. Typed defaults, CLI @@ -50,6 +53,7 @@ baseline: | Area | Current behavior | Target behavior | | --- | --- | --- | | Identity declaration | `Definition.SequenceCompositions` registers one literal string field at each registered mapping path. | YAML collection declarations and optional authoritative application registrations support field sets and scalar identity components. | +| Application registration destinations | Registrations select exact literal mapping paths. | Exact and structural selectors can impose identity beneath arbitrary mapping keys, independently at each matched destination. | | Strict composition | Registered sequences append distinct string identities and reject duplicates; repeated unregistered sequences conflict. | Collections with effective identity append distinct records and reject duplicates. Applicable lower-tier identity is available before composing higher-tier peers. Unidentified sequences retain strict-conflict behavior. | | Source precedence and selected overlays | A present higher sequence replaces the lower sequence wholesale. | Collection policy selects `replace`, `append`, or `upsert`; overlays can declare policy for inline sequences. | | Sibling inheritance | An inheriting mapping recursively overlays its resolved parent; supplied sequences replace wholesale. | Mappings retain recursive combination, while sequences use destination policy and the same three operations. | @@ -223,6 +227,104 @@ a path to exist, be non-null, or have sequence shape. The application schema and validation own those requirements. Final-result enforcement for application registrations is defined under [Validation And Diagnostics](#validation-and-diagnostics). +### Dynamic Application Registration Destinations + +Allow applications to register identity for collections whose containing mapping +keys become known only during loading. For example: + +```yaml +collections: + weatherreporter: + prompts: + - id: foo.bar + text: Report the weather. + homewatch: + prompts: + - id: foo.bar + text: Report the household status. +``` + +One application registration can require `id` for every namespace's `prompts` +list. The equal IDs above are valid because each list has its own uniqueness +scope. An optional root `prompts` list can have a separate exact registration. + +Use an explicit application-side structural selector composed of literal mapping +keys and an arbitrary-mapping-key segment. Conceptually, the namespace rule is: + +```text +Literal("collections") -> AnyMappingKey -> Literal("prompts") +``` + +This notation explains the selector; it is not a Go signature or YAML syntax. +Each segment consumes exactly one mapping level. Match the complete destination, +not an arbitrary prefix or suffix. Literal keys remain exact, including empty +strings and keys containing punctuation such as `*`, `/`, or `.`. An arbitrary-key +segment is distinct from any literal spelling and does not match a sequence +index. Existing `Path` values retain their literal meaning. + +This capability belongs to the reusable application definition and applies to +ordinary and layered loads. YAML collection destinations and import destinations +retain their existing literal syntax. Recursive search, regular expressions, +filesystem glob rules, sequence-index selectors, and custom matching callbacks +are outside this addition. + +At each concrete matched destination, apply the same authoritative identity +constraint as an exact registration. Use the settled field-set and scalar-equality +rules. Do not pool identities across destinations or require records to repeat +the containing mapping key as an identity component. Applications retain ownership +of namespace interpretation, qualified runtime IDs, and domain validation. + +Matching supplies effective identity before relevant strict composition and +precedence operations. Without an applicable explicit mode, this identity selects +upsert under the existing default rule. It also supplies identity for inheritance +within the matched sequence. YAML declarations must agree with it under every +mode, including replace. These are ordinary application constraints with a broader +destination selector, not a separate final-validation facility. + +### Registration Overlap + +Equivalent application constraints coalesce at any shared destination. Equivalence +uses the identity field-set rules: for example, `id` and `[id, id]` agree. +Incompatible constraints fail; exact registrations do not override broader +selectors, and registration order grants no precedence. + +Detect incompatible overlapping registrations while validating the application +definition, before source acquisition. For these fixed-depth mapping selectors, +overlap exists when the selectors have the same number of segments and every +corresponding pair can match a common key. Unequal literal keys at any position +make the selectors disjoint; an arbitrary-key segment can overlap a literal or +another arbitrary-key segment. Compare segment structure rather than rendered +path strings. No configuration pre-read is needed. + +Thus an `id` rule for all namespace prompt lists conflicts with a `[id, name]` +rule for one named namespace, even if that namespace is absent from a particular +load. This is an invalid application definition. A valid selector matching no +collections is harmless: registration does not require a path to exist, create +collections, or impose shape and nullability requirements. + +### Matching Across The Loading Lifecycle + +Match application selectors against effective configuration destinations after +accounting for input placement. Physical filenames and source-local paths do not +determine applicability. Make constraints available before composing contributions +at those destinations, including namespaces first introduced by imports, mounted +inputs, or higher tiers. Apply matching constraints to key-only YAML declarations +as well, even before their sequence exists. Consumers need neither a preliminary +load nor source inspection to enumerate namespaces. + +During inheritance, apply registrations at the inheriting destination, including +newly created matching destinations. Copying a collection does not transfer its +application registrations to a different path. The existing YAML-policy ownership +and transfer rules remain separate from application selector matching. + +During final validation, find matching actual sequences in the final effective +configuration. This includes namespaces introduced or replaced by typed defaults, +CLI overrides, and application resolution. Reusing only destinations found during +source assembly would miss these values. Use the same selector meaning and +identity validator in both phases; later assignments do not trigger composition +or inheritance again. YAML-only identities retain their existing enforcement +boundary. + ## Strict Composition And Precedence Operations ### Compose Peers Before Applying Precedence @@ -699,6 +801,11 @@ registrations satisfy their declared identity constraints. Retain assembly-time checks and validate final effective sequences before publication, after defaults, inheritance, overrides, application resolution, and secret lookup. +This covers both exact and dynamic application registrations. Evaluate dynamic +selectors against the final effective configuration, with independent validation +for each concrete matched sequence as described under +[Matching Across The Loading Lifecycle](#matching-across-the-loading-lifecycle). + ConfigKit performs these identity checks; consumers register the constraint without needing to repeat duplicate detection in `Definition.Validate`. Application-owned semantic validation continues to cover domain-specific rules. @@ -720,6 +827,11 @@ Identity diagnostics identify collection and item paths using available origin information without exposing identity values. Conflicting declarations identify the collection destination and declaration locations. +For dynamic registrations, use concrete matched collection and item paths for +value diagnostics, with available origins and existing sensitivity rules. +Definition-time overlap errors identify the conflicting registrations; they do +not require or invent a concrete destination from a loaded document. + Correctness includes final values, projected item paths, source destinations, provenance, history, sensitivity, and passive resource contexts. Appending can change final indexes; replacing changes which contributions survive. Preserve @@ -732,13 +844,19 @@ The implementation plan should cover these connected changes, with execution order and test sequencing defined there: - **Declarations and public API:** extend application registration to the settled - identity model; parse and consume `configkit.sequences` across supported source + identity model and explicit structural selectors while preserving literal + `Path` semantics. Parse and consume `configkit.sequences` across supported source forms and selected overlays; preserve placement-relative and item-local scopes. Keep sequence declarations independent of filesystem import enablement. - **Identity and policy representation:** implement field-set comparison, typed scalar identity tuples with exact numeric equality, declaration agreement, pending declarations, explicit-mode persistence, replacement lifetime, and record-owned policy transfer. Choose bookkeeping proportional to these semantics. +- **Registration matching:** support literal and arbitrary-mapping-key segments, + detect incompatible selector overlap during definition validation, and resolve + applicable constraints at concrete placed destinations. Reuse the identity + rules and validator for exact and dynamic registrations. Keep matching separate + from source discovery and from YAML declaration ownership. - **Assembly handoffs:** make accumulated lower policy available before higher-tier strict composition. Separate source acquisition from policy-aware composition or otherwise revise assembly sequencing; current main-before-underlay composition @@ -759,7 +877,9 @@ order and test sequencing defined there: defaults/decoding, preserve explicit-presence and source metadata for application resolution, reject CLI structural directives, and add final application identity validation through the shared publication path. Keep YAML-only enforcement - within structural operations. + within structural operations. Match dynamic registrations during assembly and + inheritance, and against the final effective configuration rather than a saved + list of source-backed destinations. - **Documentation and verification:** exercise ordinary and layered loading, imported and inline collections, item-local scopes, version-header consumption, selected overlays, inheritance, typed defaults, CLI, callbacks, and inspection. @@ -768,6 +888,13 @@ order and test sequencing defined there: preservation, nested destination policy, cancellation, and stable output order. Verify helper-composed references, overlapping reference fields, missing or extra fields, and replacement of references together with their containing records. + Verify independent namespace uniqueness, root and namespaced registrations, + literal punctuation keys, equivalent and incompatible overlaps, and conflicts + detected even without a matching input destination. Cover namespaces introduced + by imports, placement, higher tiers, inheritance, defaults, CLI, and callbacks; + preserve ordinary strict-peer and upsert behavior. Check YAML disagreement under + replace, harmless absent or empty collections, schema-owned shape requirements, + unrelated sequences remaining unselected, and concrete-path diagnostics. Update the public contract and consumer examples when behavior lands. The collection-policy and CLI-ordering decisions are accepted in @@ -779,12 +906,35 @@ These govern the target architecture; the configuration contract continues to describe implemented behavior until the feature lands. This roadmap does not select migration machinery. -Before implementation, finalize the ADR 0017 revision for inheritance within -identified sequences, update the supersession status of -[ADR 0009](../adr/0009-sibling-inheritance.md), and align the architecture policy. -Preserve ADR 0009's decision content and the final application-registration -guarantee. These architectural revisions remain subject to review. +ADR 0017 now includes inheritance within identified sequences; +[ADR 0009](../adr/0009-sibling-inheritance.md) records that scope extension in its +status while preserving its decision content. The architecture policy is aligned. +Dynamic selectors extend the destinations covered by application registration; +they retain the accepted application authority and final-validation boundary. -No open feature decisions remain. The implementation plan should translate this -end state into reviewable stages without adding guarantees, restrictions, or -combination semantics. +After settling the API decision below, the implementation plan should translate +this end state into reviewable stages without adding guarantees, restrictions, +or combination semantics. + +## Open Decisions + +### Registration API Shape + +The selector's behavior and definition-time overlap validation are settled. How +should the public registration represent exact destinations alongside selectors +containing arbitrary-mapping-key segments? + +**Recommendation:** use one registration model with an explicit selector type +whose segments distinguish literal keys from arbitrary mapping keys. Provide a +straightforward way to construct an exact selector from a literal `Path`. +Both forms then share identity fields, validation, and lifecycle behavior, while +ordinary `Path` remains unchanged. This requires a small new public type, but +keeps one registration surface and avoids mutually exclusive destination fields. +Exact Go names and constructor signatures belong in the implementation plan. + +**Alternative:** retain exact-path registrations and add a separate dynamic +registration form, normalizing both into the same internal selector model. +This keeps exact registration familiar for simple consumers, but introduces two +public forms with the same identity rules and requires overlap validation across +both. Both choices must preserve the settled matching and enforcement behavior; +neither introduces selector syntax into ordinary paths or YAML destinations.