From c7490b783b9a7a7da4430d5b1aa193e37b8ae018 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Wed, 14 Oct 2026 02:24:50 +0000 Subject: [PATCH] Propose collections built from mounted configuration fragments --- ...guration-collections-identified-records.md | 11 +- ...iguration-collections-mounted-fragments.md | 433 ++++++++++++++++++ docs/roadmap/namespaces.md | 10 +- 3 files changed, 447 insertions(+), 7 deletions(-) create mode 100644 docs/roadmap/configuration-collections-mounted-fragments.md diff --git a/docs/roadmap/configuration-collections-identified-records.md b/docs/roadmap/configuration-collections-identified-records.md index 94d1994..2728171 100644 --- a/docs/roadmap/configuration-collections-identified-records.md +++ b/docs/roadmap/configuration-collections-identified-records.md @@ -2,8 +2,11 @@ Status: proposed alternative; not implemented or accepted. This is a middle ground between [proposal 1](configuration-collections-current-configkit.md), -which uses current ConfigKit, and a future proposal 3 that may change source -and document organization more substantially. The parent +which uses current ConfigKit, and +[proposal 3](configuration-collections-mounted-fragments.md), which changes +source placement and document organization. Proposal 3 also adopts the later +requirement for one installed definition per prompt and an optional version; +this proposal retains versioned record identities for comparison. The parent [namespace and collection roadmap](namespaces.md) retains the accepted product decisions and the pending configuration decision. @@ -427,5 +430,5 @@ Local documentation links and whitespace checks passed. The temporary consumer and its module replacement are outside both repositories. This proposal remains for evaluation alongside proposal 1. The parent roadmap's -configuration decision remains open, and broader source-placement/versioning -changes are reserved for the later proposal 3 discussion. +configuration decision remains open. Proposal 3 develops the alternative based +on source placement; independent collection specification checking remains open. diff --git a/docs/roadmap/configuration-collections-mounted-fragments.md b/docs/roadmap/configuration-collections-mounted-fragments.md new file mode 100644 index 0000000..ab5e172 --- /dev/null +++ b/docs/roadmap/configuration-collections-mounted-fragments.md @@ -0,0 +1,433 @@ +# Proposal 3: Collections With Mounted Configuration Fragments + +Status: proposed alternative; not implemented. The authoring direction follows +the discussion's preference for collection-local mapping keys, one installed +definition per prompt, and an optional flat prompt `version` field. The overall +configuration standard and upstream API still require selection. Compare +[proposal 1](configuration-collections-current-configkit.md) and +[proposal 2](configuration-collections-identified-records.md); the parent +[namespace roadmap](namespaces.md) owns the shared decisions. + +## Recommendation + +Ask ConfigKit to support an explicit destination path for configuration inputs. +For example, inputs selected from a collection's `prompts` directory would +contribute their document bodies beneath the literal configuration path +`[collections, weatherreporter, prompts]`. Authors write local prompt names and +definitions without repeating either `prompts:` or the namespace in each file. + +Use a collection-shaped effective configuration. Each collection contains its +release metadata and ordinary mappings of backends, profiles, and prompts. +PromptKit derives qualified public IDs from the final collection and resource +keys after ConfigKit hands off the resolved configuration. + +There is one installed definition per qualified prompt ID. Its optional +`version` field describes that definition; it does not create another identity +or select a merge strategy. This removes the need for proposal 2's identified +record sequences and for the current nested prompt-version mapping. + +Placement belongs to ConfigKit. PromptKit must not parse, wrap, or rewrite each +source itself, nor impose duplicate-namespace checks before composition. This +preserves the accepted [configuration boundary](../adr/0001-configuration-boundary.md). + +## Current Baseline And Required Extension + +The reviewed ConfigKit checkout is v0.7.1, commit +`6fffde4af8f31f99d3ee295c426eb3234cb0fb77`, also the dependency selected by +PromptKit. Its [configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.7.1/docs/config.md) +and public `Input` declaration establish the relevant baseline: + +- Inputs select a source, import policy, and filesystem context. They do not + expose a destination in the effective configuration tree. +- YAML documents have mapping roots. Strict composition recursively combines + mappings and rejects every overlapping non-mapping pair, even equal values. +- Explicit precedence layers recursively overlay mappings; scalar, sequence, + and null values replace lower values. File order does not confer precedence. +- `extends` names an immediate mapping sibling and resolves after assembly. +- The configured specification-version path is checked on main only and + protected against contributions from fragments and lower layers. +- Effective fields retain source origins and filesystem resource contexts. + +The collection-shaped tree can be authored explicitly with current ConfigKit. +Removing the repeated wrappers from source files requires the proposed mount +capability. PromptKit also needs a new schema and runtime indexing changes; +none of the proposed PromptKit examples are supported by its current schema. + +## Authoring And Assembly + +A file such as `weatherreporter/prompts/alerts.yml` would contain: + +```yaml +summarize.alerts: + version: "1" + default_profile: provider/brief + messages: + - role: user + content: Summarize the weather alerts. + output: + format: text + validation_mode: none +``` + +The version may simply be omitted. Another collection can contribute the same +local prompt name, for example `homewatch/prompts/alerts.yml`: + +```yaml +summarize.alerts: + default_profile: provider/brief + messages: + - role: user + content: Summarize the household alerts. + output: + format: text + validation_mode: none +``` + +A provider collection's `profiles/brief.yml` uses the same local-key convention: + +```yaml +brief: + backend: provider/default + model: vendor/example-model + temperature: 0.2 +``` + +Its `backends/default.yml` could contain this synthetic, non-live endpoint: + +```yaml +default: + endpoint: https://inference.example.invalid/v1 +``` + +An optional `weatherreporter/collection.yml` contributes collection metadata: + +```yaml +version: "2.4.0" +``` + +Here that field is a collection release label because the document is mounted +at the collection node. It is not a PromptKit specification declaration. Keep +these two uses explicit in author documentation and packaging examples. + +The consuming application's source selection would express placements like +these. Paths are illustrative; the destination column is a list of literal +mapping keys, not a slash-delimited string or proposed API syntax. + +| Selected source | Destination | +| --- | --- | +| Application main document | `[]` (configuration root) | +| `weatherreporter/collection.yml` | `[collections, weatherreporter]` | +| YAML fragments under `weatherreporter/prompts` | `[collections, weatherreporter, prompts]` | +| YAML fragments under `homewatch/prompts` | `[collections, homewatch, prompts]` | +| YAML fragments under `provider/profiles` | `[collections, provider, profiles]` | +| YAML fragments under `provider/backends` | `[collections, provider, backends]` | + +The namespace is supplied by the assembly declaration. A collection-selection +helper could take it once and construct these destinations for conventional +subdirectories without reading or transforming document contents. ConfigKit +would still perform all placement and composition. Directory basenames do not +implicitly identify namespaces, and filenames do not identify resources. + +Recommend explicit source selection through the ConfigKit input API first. +PromptKit's directory helpers can enumerate files and attach a destination to +each input once that API exists. A YAML import destination could expose the +same operation later or alongside it; its concrete spelling is an upstream +decision. This proposal does not claim that existing `configkit.imports` +directives accept a destination today. + +Put peer collection inputs in one strict underlay group. Use later groups or +main for intentional application/operator overrides. Selecting a destination +does not select a precedence tier. Avoid loading the same files both through +a recursive whole-collection scan and through the individual mounted folders. + +The assembled application shape would be equivalent to: + +```yaml +version: 1 +collections: + weatherreporter: + version: "2.4.0" + prompts: + summarize.alerts: + version: "1" + default_profile: provider/brief + messages: + - role: user + content: Summarize the weather alerts. + output: + format: text + validation_mode: none + homewatch: + prompts: + summarize.alerts: + default_profile: provider/brief + messages: + - role: user + content: Summarize the household alerts. + output: + format: text + validation_mode: none + provider: + backends: + default: + endpoint: https://inference.example.invalid/v1 + profiles: + brief: + backend: provider/default + model: vendor/example-model + temperature: 0.2 +``` + +This yields `weatherreporter/summarize.alerts` and +`homewatch/summarize.alerts`, plus profile `provider/brief` and backend +`provider/default`. The provider's model string remains `vendor/example-model`; +it is not a PromptKit resource identifier. + +One fragment may contain several local definitions. A fragment starting directly +with `messages:` would need a destination that also supplies the local prompt +key. Generic mounting could permit that, but recommend the local-key mapping +above as the standard authoring form: identity travels with the definition and +files may be reorganized without changing it. Filename-derived IDs are not +needed. + +## Final Schema And Identity + +Recommend `collections` as a mapping of namespace to collection body, containing +optional release `version` and optional `backends`, `profiles`, and `prompts` +mappings. Collection release labels are descriptive, with no SemVer ordering +or dependency-solving semantics. Metadata is optional: an installed collection +can contain resources without claiming a published release. + +Each resource key inside a collection is local. The parent namespace and local +key follow the accepted slash-free component grammar; their public spelling is +exactly `namespace/local-name`. There are no duplicate embedded `id` or +`namespace` fields to reconcile. Dots remain ordinary characters. + +To retain unqualified application resources, recommend optional root-level +`backends`, `profiles`, and `prompts` mappings, also keyed by local name and +using the same definition shapes. They produce unqualified public IDs. Under +this proposal, namespaced definitions belong under `collections`, not a second +qualified-key spelling in those root maps. This avoids two configuration paths +producing the same public identity. This representation is a recommendation +for proposal 3, not a change to today's accepted format. + +Stored references such as `default_profile`, profile `backend`, and smoke-test +profile defaults remain exact public references. Nesting does not automatically +qualify a slashless reference or rebind it to caller scope. For example, a +profile in `provider` still declares `backend: provider/default`. Caller-scoped +lookup follows the existing namespace roadmap after final indexes are built. + +### One Installed Prompt And Optional Version + +The resource's qualified ID is its sole configuration identity. There is no +version-key mapping, implicit `latest`, version ordering, or parallel installed +history. Updating the optional version changes metadata on the same prompt. + +Recommend a nonblank opaque string when a prompt version is supplied, with no +SemVer constraint. Omission or explicit null means no declared version; null +can clear a lower-layer or inherited label. Quote numeric-looking labels such +as `"1"`. Do not synthesize a version for an unversioned prompt. These detailed +value rules are recommendations; the agreed requirement is that the field be +optional. + +Execution and inspection without a requested version select the installed +prompt directly. As an API migration recommendation, retain an optional +requested version as an exact assertion against the installed label: a match +succeeds; a mismatch or an unversioned definition fails the assertion. It never +searches for another version. Removing request-side version fields altogether +is a simpler alternative but requires more consumer changes; settle the public +API shape before implementation. Listings should identify one prompt per ID +and expose the optional declared version separately from identity. + +This intentionally supersedes multi-version preservation for this alternative. +Proposals 1 and 2 retain their earlier assumptions for comparison. The parent +roadmap's version-selection and listing language needs corresponding revision +if proposal 3 is adopted. + +## Required ConfigKit Mount Semantics + +The upstream feature should be generic source placement, not a PromptKit-aware +collection loader or application callback that transforms source documents. +The following is the recommended contract to agree and test upstream: + +1. An input has an optional destination of literal mapping-key segments. An + empty destination preserves existing root behavior. It is independent of + the filesystem path, filesystem containment root, and precedence group. +2. ConfigKit parses each source with its existing YAML rules and processes + source-root directives before placing application content at its destination. + Synthetic mapping ancestors participate in ordinary composition. It does + not turn dots or slashes inside a generic key into path separators. +3. Imports without an explicit destination inherit the importing input's + destination. If import placement is exposed, recommend its destination be + relative to the importing destination. Explicitly selected independent + inputs use root-relative destinations. This keeps a mounted import subtree + contained in its declared configuration context without guessing filenames. +4. Placement precedes strict composition and layer assembly. Contributions to + the same destination obey the existing mapping rules; conflicts with scalar + ancestors use those same strict-versus-overlay rules. No new merge, deletion, + or whole-definition replacement operation is implied. +5. Origins and diagnostics retain physical source locations and explain their + effective destination paths. Synthesized ancestors must not obscure the + contributing source. Resource contexts remain attached to the declaring + source: mounting does not relocate relative `content_file` references. +6. Apply protected specification-path checks to the destination-adjusted + application paths. A release `version` mounted at + `[collections, weatherreporter]` is not the protected root `version`. + Contributions that actually reach or replace the protected path remain + forbidden. Main's version checker still runs at its established stage; + mounting does not grant fragments their own compatibility checks. +7. Preserve source cycle/repetition, acquisition, and budget rules. Mounting + must not silently authorize loading the same physical source twice within + a group. Account for synthesized ancestors and destination depth within + ConfigKit's limits. Existing unmounted loads retain their behavior. + +Item 6 is a necessary upstream change, not something PromptKit can work around +by stripping a source header. Agree its interaction with imports and main's +version protection explicitly. Mounting main away from the root is unnecessary +for this PromptKit standard. + +### Composition And Overrides + +Different collection keys prevent accidental collisions between equal local +resource names. Two sources targeting the same namespace are not automatically +invalid: disjoint mapping contributions combine. Overlapping scalar fields, +message sequences, nulls, and equal release labels conflict within a strict +group. Across deliberate layers they follow ordinary precedence instead. + +A higher-layer prompt mapping is a partial override. Changing only its version +does not replace its messages or other settings. Omitting a field retains its +lower value, and an empty mapping does not clear a definition. Publishers and +operators must treat such changes as configuration customization, not as an +atomic installation of another prompt release. Do not introduce source audits +or a version-triggered replacement rule to conceal this distinction. + +### Inheritance + +Existing sibling inheritance remains useful within each collection's resource +mapping: `extends: brief` can name the local profile `brief`. Prompt versions, +now ordinary optional fields, would inherit and overlay like other fields; +`version: null` explicitly clears an inherited label. + +Cross-collection profiles are no longer immediate siblings. Consequently, +`extends: provider/brief` cannot reach that collection under current ConfigKit +inheritance. Recommend local sibling inheritance for the initial proposal; +applications can still execute or select the qualified `provider/brief` +profile directly. Migrating existing cross-collection inheritance requires +flattening those authored definitions or making a separate upstream change. + +An alternative is generic explicit-path inheritance in ConfigKit, with its own +syntax, cycle rules, and provenance contract. That would preserve reusable +cross-collection bases, but crosses a currently deferred ConfigKit boundary +and is additional work beyond mounting. PromptKit must not resolve such +relationships itself after handoff. Decide whether this capability is required +before selecting a migration plan. + +## Specification Versions And Collection Inventory + +Keep `version: 1` at the application root for the candidate PromptKit +specification, checked through ConfigKit's version mechanism. It describes +the assembled schema, independently of collection release and prompt labels. +The three proposals are competing candidate formats, not interchangeable +released meanings of specification version 1. + +Mounting alone does not solve independent collection compatibility checking. +A mounted `collection.yml` containing a release `version` does not declare or +validate a separate specification version. A self-contained collection entry +document with its own specification header would need a further ConfigKit +contract for checking and consuming that header before mounting its payload. +Do not infer compatibility from a directory name or release label, or have +PromptKit load and validate each manifest separately. Keep that upstream +extension open rather than bundling it implicitly into source placement. + +Recommend collection inventory derived from the final `collections` mapping: +namespace plus optional effective release label, sorted and detached from the +captured engine. Metadata-only and resource-only entries can both appear. +Root unqualified resources do not invent a named collection. Resource-based +namespace enumeration remains a separate listing operation. + +An inventory reports effective configuration declarations, not immutable +package identity, historical source ownership, or proof of unchanged published +contents. Operator overrides may alter resources while leaving a release +label in place. Source explanations remain ConfigKit inspection concerns. +Engine construction captures configuration; edits require a replacement engine. + +## Comparison And Migration + +| Concern | Proposal 1 | Proposal 2 | Proposal 3 | +| --- | --- | --- | --- | +| Author identity | Qualified mapping key | Namespace/local-ID fields | Local mapping key beneath a mounted collection | +| Prompt version | Required nested key | Required identity field | Optional descriptive field | +| Multiple installed versions per prompt | Supported | Supported | Not supported | +| Repeated resource wrapper in fragments | Required | Required | Omitted through placement | +| Composition machinery | Existing mappings | Identified record sequences | Existing mappings after placement | +| Main upstream change | None | Record composition and inheritance | Source placement, provenance, and adjusted path protection | +| Cross-collection inheritance | Existing qualified sibling keys | Proposed tuple references | Requires a separate extension or authored flattening | +| Independent collection specification check | Unavailable | Unavailable | Still a separate extension | + +Proposal 3 is the recommended direction for the stated one-installed-prompt +requirement. Its authoring economy follows from the configuration hierarchy, +and it retains ordinary map composition. The tradeoffs are explicit assembly +destinations, changed inheritance scope, and a deliberate reduction of prompt +version selection to one installed definition. + +Migration must choose one installed definition for each former multi-version +prompt, move any retained version into its optional field, and relocate +qualified resource keys into collection/local-key mappings. Update destination +declarations, stored references, request-version handling, inventories, and +configuration-inspection paths. Preserve provider model strings and source +resource bases. Namespace mounting does not automatically relocate schema or +artifact stores or rewrite references when a collection is renamed. + +Retain the accepted removal of mandatory catalog modules and built-in profiles. +This is a breaking configuration/API change under the +[release policy](../policy/release.md); implementation must include consumer +migration guidance and update current contracts when it lands. The proposal +does not change today's supported configuration or API. + +## Delivery, Validation, And Remaining Decisions + +Agree and implement the generic mount contract in ConfigKit first, then adopt +it in PromptKit's source helpers and final-configuration adapter. Build qualified +runtime indexes only after handoff. Keep collection installation, remote +fetching, automatic directory discovery, live reload, and dependency solving +outside this work. + +Upstream acceptance should exercise root and nested placement; overlapping +destinations within strict groups and across layers; mounted nested imports; +source cycles/repetition; destination-aware protected-version checks; source +locations and resource bases; sibling inheritance; and unchanged unmounted +behavior. Verify mounted metadata cannot bypass root version protection. + +PromptKit acceptance should cover the illustrated assembled tree, same local +names in different collections, unqualified resources, scoped selection, +versioned and unversioned prompts, explicit version assertions if retained, +inventory without release metadata, source-relative message content, and +intentional partial overrides. Do not add per-source PromptKit validation. + +Review verification on 2026-10-14: a temporary offline consumer of the local +ConfigKit checkout decoded all six YAML examples into generic mappings and +confirmed that a version-only overlay preserves the lower prompt body. Local +documentation links and whitespace checks passed. These checks establish the +current YAML and mapping behavior, not the unimplemented mount extension or +future PromptKit schema. The temporary consumer is outside both repositories. + +The remaining decisions are: + +- **Mount API and packaging:** prefer an input destination plus a conventional + explicitly selected directory layout. An import-destination directive would + let publishers express more assembly in configuration; settle its syntax + upstream. A manifest that computes its own mount from a declared namespace + would require further ConfigKit semantics and is not assumed here. +- **Cross-collection inheritance:** prefer existing local sibling inheritance + initially; add generic path references upstream if migration needs justify + the added contract. +- **Request versions:** prefer optional exact assertions for migration; removing + version selectors is simpler if consumers do not need that guard. +- **Independent specification headers:** keep one main-owned declaration for + this proposal; pursue a distinct ConfigKit extension if collections must + each assert compatibility independently. + +These recommendations are reviewable choices, not claims that upstream work +has been accepted. One installed definition and an optional prompt version are +the settled requirements for this alternative. The parent configuration +decision remains open while the three proposals are compared. diff --git a/docs/roadmap/namespaces.md b/docs/roadmap/namespaces.md index daaa2dd..2109f0e 100644 --- a/docs/roadmap/namespaces.md +++ b/docs/roadmap/namespaces.md @@ -489,9 +489,13 @@ dependencies instead. develops a concrete configuration standard for the existing composition rules. [Proposal 2: identified records](configuration-collections-identified-records.md) uses an opt-in upstream extension for identity-based record composition and -inheritance. Both retain main-owned specification versioning; broader source -and version changes can be evaluated in a later proposal 3. These are proposals -for comparison; this decision remains open. +inheritance. [Proposal 3: mounted fragments](configuration-collections-mounted-fragments.md) +uses explicit upstream source placement and collection-local mapping keys, +with one installed definition per prompt and an optional flat version field. +All three retain main-owned specification versioning; independent collection +checks remain a separate upstream question. These are proposals for comparison; +this decision remains open. Proposal 3 deliberately revises the version-selection +and listing assumptions elsewhere in this roadmap if that alternative is adopted. **Desired capability:** describe collection namespace and release version in configuration and expose an API through which downstream applications can list