promptkit/docs/roadmap/configuration-collections-mounted-fragments.md

# Proposal 3: Collections With Mounted Configuration Fragments

Status: selected configuration direction; not implemented in PromptKit. Required
ConfigKit capabilities are implemented on its development branch, reviewed at
`e66c371` on 2026-10-16, but are not in PromptKit's selected v0.7.1 dependency.
The recommended direction uses collection-local mapping keys, one installed
definition per prompt, and an optional flat prompt `version` field. The latest
review agrees on shared source checks using `IfPresent` and a distinct collection
`release_version` field. The six configuration decisions below are accepted;
exact public API shapes remain delivery details. The parent [namespace roadmap](namespaces.md) owns the shared decisions. Unselected
proposals 1 and 2 have been retired; their history and the separately captured
upstream request are recorded below.

## Configuration Simplicity

This development round simplifies configuration authoring and consumption:

- Load prompt, profile, and backend libraries as YAML configuration, removing
  mandatory catalog Go-module imports and bundled profiles.
- Remove the unused prompt-version selection scheme and its request API fields.
- Let publishers opt into specification checks by declaring a source-root
  `version`, commonly in a collection entry. Neither entries nor individual
  fragments must declare it; authors need not add headers to every file.

Prefer conventions and small convenience APIs over mandatory workflows.
Preserve consumer choice over sources, layout, namespaces, and precedence without
adding speculative options or stronger compatibility/provenance guarantees.

## Recommendation

Use ConfigKit's implemented input destinations and relative import destinations.
For example, a collection entry can import its `prompts` directory beneath
`[collections, weatherreporter, prompts]`. Authors write local prompt names and
definitions without repeating either `prompts:` or the namespace in each file.

Use a collection-shaped effective configuration. Each collection contains its
release metadata and ordinary mappings of backends, profiles, and prompts.
PromptKit derives qualified public IDs from the final collection and resource
keys after ConfigKit hands off the resolved configuration.

There is one installed definition per qualified prompt ID. Its optional
`version` field describes that definition; it does not create another identity
or select a merge strategy. This avoids identified record sequences and the
current nested prompt-version mapping.

Placement belongs to ConfigKit. PromptKit must not parse, wrap, or rewrite each
source itself, nor impose duplicate-namespace checks before composition. This
preserves the accepted [configuration boundary](../adr/0001-configuration-boundary.md).

## Reviewed Upstream Capabilities And Adoption Status

The reviewed ConfigKit development checkout is `e66c371`. Its public
configuration contract (`docs/config.md`, sections “Input destinations” and
“Shared source-version checks”) and ADRs 0011–0013 own the upstream behavior.
PromptKit still selects ConfigKit v0.7.1; choose a released dependency containing
these capabilities before implementation. This roadmap does not imply they are
already available through PromptKit or select an upstream release number.

The implemented public surface provides:

- `Input.Destination` for root-relative placement, and YAML import
  `destination: [prompts]` for placement relative to the immediate importing
  document. Glob matches share the declared destination; nested imports append
  to their immediate parent's destination.
- `Options.SourceVersionMode` and `LayeredOptions.SourceVersionMode`, using one
  shared `Definition.Version` rule. Enabled modes interpret the rule's path in
  each document's source-local coordinates before placement and composition.
- Consumption of accepted non-main specification headers, with the main header
  retained and protected at its effective application path.
- `SourceInfo.Destination` and `SourceInfo.VersionCheck` for detached inspection
  evidence, alongside existing source locations and resource contexts.

Strict mapping composition, explicit precedence, atomic sequences, and local
sibling inheritance retain their existing rules. ConfigKit owns the complete
load; PromptKit interprets and validates final effective values. No further
ConfigKit extension is required for the initial direction described here.

The new PromptKit schema and runtime indexes remain future work. The existing
[format reference](../formats.md) remains authoritative until adoption.

## Authoring And Assembly

A file such as `weatherreporter/prompts/alerts.yml` would contain:

```yaml
summarize.alerts:
  version: "1"
  default_profile: provider/brief
  messages:
    - role: user
      content: Summarize the weather alerts.
  output:
    format: text
    validation_mode: none
```

The version may simply be omitted. Another collection can contribute the same
local prompt name, for example `homewatch/prompts/alerts.yml`:

```yaml
summarize.alerts:
  default_profile: provider/brief
  messages:
    - role: user
      content: Summarize the household alerts.
  output:
    format: text
    validation_mode: none
```

A provider collection's `profiles/brief.yml` uses the same local-key convention:

```yaml
brief:
  backend: provider/default
  model: vendor/example-model
  temperature: 0.2
```

Its `backends/default.yml` could contain this synthetic, non-live endpoint:

```yaml
default:
  endpoint: https://inference.example.invalid/v1
```

A conventional `weatherreporter/collection.yml` entry can declare the shared
specification, optional collection release, and its layout:

```yaml
version: 1
release_version: "2.4.0"
configkit:
  imports:
    - glob: prompts/*.yml
      destination: [prompts]
```

A provider entry can import `profiles/*.yml` at `[profiles]` and
`backends/*.yml` at `[backends]` in the same way. Collection publishers choose
which categories to include. A required glob must match; unused categories can
be omitted or use ConfigKit's existing optional-import behavior. Ordinary
fragments may omit specification headers under the selected `IfPresent` mode.
An entry header is the recommended publishing convention, not a load-time
requirement selectively enforced on entries by this mode.

The application selects each entry, enables its imports, and supplies its
namespace through the public input destination. For example, the following
partial Go snippet uses implemented ConfigKit development-branch APIs:

```go
entry := configkit.Input{
    Source: configkit.File("/usr/local/share/collections/weatherreporter/collection.yml"),
    Imports: true,
    Destination: configkit.Keys("collections", "weatherreporter"),
}
```

For independent peer collections, one `Underlay` group's `Inputs` is a useful
starting point, not a required grouping. Consumers choose precedence and groups.
Configure `LayeredOptions.SourceVersionMode` as `SourceVersionModeIfPresent` and
`Definition.Version.Path` as `configkit.Keys("version")`, with PromptKit's
supported-specification checker. The application main input remains at the
root and declares `version: 1` in the examples below; both the main input and
its declaration are optional. Namespace and source selection require no manifest parsing in PromptKit.

| Selected source | Effective destination |
| --- | --- |
| Application main document | Configuration root |
| `weatherreporter/collection.yml` | `[collections, weatherreporter]` |
| Its `prompts/*.yml` imports | `[collections, weatherreporter, prompts]` |
| Homewatch entry and its prompt imports | `[collections, homewatch]` and `[collections, homewatch, prompts]` |
| Provider entry and its profile/backend imports | `[collections, provider]`, then `[profiles]` or `[backends]` appended |

Directly selected fragments with `Input.Destination` remain possible. Entry
imports are the recommended packaging convention because publishers can describe
their layout. Filenames and directory basenames do not define resource identity;
no implicit directory discovery, namespace inference, or collection installer is
introduced. A small helper should construct inputs from an explicitly selected
source and namespace, without inspecting manifests to compute destinations. `collection.yml`
is a convention, not a required filename. Direct fragments and programmatic
sources remain supported through the source API; no parallel collection loader
or mandatory filesystem layout is introduced.

Use later underlay groups or main for intentional application/operator overrides.
Placement does not establish precedence. Do not also recursively scan files
already selected by entry imports: ConfigKit's repeated-source rules still apply.

The entry's source-root `version` is checked and consumed before its payload
composes. Its `release_version` is retained as ordinary collection metadata.
A prompt's nested optional `version` is also ordinary payload. These are three
distinct concepts; source-root release metadata must no longer use `version`.

The assembled application shape would be equivalent to:

```yaml
version: 1
collections:
  weatherreporter:
    release_version: "2.4.0"
    prompts:
      summarize.alerts:
        version: "1"
        default_profile: provider/brief
        messages:
          - role: user
            content: Summarize the weather alerts.
        output:
          format: text
          validation_mode: none
  homewatch:
    prompts:
      summarize.alerts:
        default_profile: provider/brief
        messages:
          - role: user
            content: Summarize the household alerts.
        output:
          format: text
          validation_mode: none
  provider:
    backends:
      default:
        endpoint: https://inference.example.invalid/v1
    profiles:
      brief:
        backend: provider/default
        model: vendor/example-model
        temperature: 0.2
```

This yields `weatherreporter/summarize.alerts` and
`homewatch/summarize.alerts`, plus profile `provider/brief` and backend
`provider/default`. The provider's model string remains `vendor/example-model`;
it is not a PromptKit resource identifier.

One fragment may contain several local definitions. A fragment starting directly
with `messages:` would need a destination that also supplies the local prompt
key. ConfigKit mounting permits such placement, but with source checks enabled
its root `version` would be the specification header, not prompt metadata. Recommend
the local-key mapping above as the standard authoring form: identity travels
with the definition and files may be reorganized without changing it. Filename-derived IDs are not
needed.

## Final Schema And Identity

Recommend `collections` as a mapping of namespace to collection body, containing
optional `release_version` and optional `backends`, `profiles`, and `prompts`
mappings. Collection release labels are descriptive, with no SemVer ordering
or dependency-solving semantics. Metadata is optional: an installed collection
can contain resources without claiming a published release.

Each resource key inside a collection is local. The parent namespace and local
key follow the accepted slash-free component grammar; their public spelling is
exactly `namespace/local-name`. There are no duplicate embedded `id` or
`namespace` fields to reconcile. Dots remain ordinary characters.

To retain unqualified application resources, recommend optional root-level
`backends`, `profiles`, and `prompts` mappings, also keyed by local name and
using the same definition shapes. They produce unqualified public IDs. Under
this proposal, namespaced definitions belong under `collections`, not a second
qualified-key spelling in those root maps. This avoids two configuration paths
producing the same public identity. This representation belongs to the selected
future direction; it does not change today's implemented format.

Stored references such as `default_profile`, profile `backend`, and smoke-test
profile defaults remain exact public references. Nesting does not automatically
qualify a slashless reference or rebind it to caller scope. For example, a
profile in `provider` still declares `backend: provider/default`. Caller-scoped
lookup follows the existing namespace roadmap after final indexes are built.

### One Installed Prompt And Optional Version

The resource's qualified ID is its sole configuration identity. There is no
version-key mapping, implicit `latest`, version ordering, or parallel installed
history. Updating the optional version changes metadata on the same prompt.

An optional prompt version is an opaque string with no SemVer, nonblank, or
normalization requirement. Omission or explicit null means no declared label;
null can clear a lower-layer or inherited label. Quote numeric-looking labels
such as `"1"`. Preserve supplied strings, including empty strings, and distinguish
an absent label in inspection. Do not synthesize a version.

Remove request-side prompt-version fields, version selection, ordering, and
version defaults as part of the breaking change. Do not reinterpret those fields
as exact-match assertions. Execution and inspection resolve the installed prompt
by ID alone. Listings expose one prompt per ID with its optional descriptive
label separately. A label neither controls execution nor proves unchanged content.

## ConfigKit Composition Boundary

Use the implemented upstream contracts without duplicating them in PromptKit.
Input destinations use literal mapping-key segments, independent of filesystem
roots and precedence. Relative import destinations are sequences of strings;
omission and `[]` retain the importing destination. Null, scalar, and non-string
destination members are invalid. No logical path segment means parent traversal
or a reset to the application root.

Source-root import directives remain structural. Imported content contributes at
its resulting destination, while physical import paths and `content_file` bases
remain relative to their declaring sources. `SourceInfo.Destination` records
placement; normal origins and diagnostics retain real source locations. Selected
overlays and explicit CLI overrides remain rooted at the application root.

Mounting preserves source identity, cycle/repetition rules, and load limits.
An absent `NoSource` contributes no payload; an actual empty mapping can
materialize its destination containers without clearing lower contents.
ConfigKit's reserved structural keys continue to apply. PromptKit must not
strip headers, re-key source trees, merge fragments itself, or turn source
inspection into an additional enforcement pass.

### Composition And Overrides

Different collection keys prevent accidental collisions between equal local
resource names. Two sources targeting the same namespace are not automatically
invalid: disjoint mapping contributions combine. Overlapping scalar fields,
message sequences, nulls, and equal release labels conflict within a strict
group. Across deliberate layers they follow ordinary precedence instead.

A higher-layer prompt mapping is a partial override. Changing only its version
does not replace its messages or other settings. Omitting a field retains its
lower value, and an empty mapping does not clear a definition. Publishers and
operators must treat such changes as configuration customization, not as an
atomic installation of another prompt release. Do not introduce source audits
or a version-triggered replacement rule to conceal this distinction.

### Inheritance

Existing sibling inheritance remains useful within each collection's resource
mapping: `extends: brief` can name the local profile `brief`. Prompt versions,
now ordinary optional fields, would inherit and overlay like other fields;
`version: null` explicitly clears an inherited label.

Cross-collection profiles are no longer immediate siblings. Consequently,
`extends: provider/brief` cannot reach that collection under current ConfigKit
inheritance. Use existing local sibling inheritance for this development round;
applications can still select the qualified `provider/brief` profile directly.
Authors needing a customized derivative may publish an expanded definition.
Expansion is a migration option, not a mandatory authoring workflow.

Generic cross-path inheritance remains possible future ConfigKit work if a
concrete consumer need justifies it. It is not a prerequisite of this round.
PromptKit must not add its own inheritance resolver after ConfigKit handoff.

## Specification Versions And Collection Inventory

### Shared Specification Policy

Use one `Definition.Version` policy, with source-local path `version`, for the
candidate PromptKit specification. Initially accept specification text `1` from
string or strict decimal-integer declarations; reject explicit null and other
unsupported declarations through the ConfigKit checker. Accept an absent main
declaration, including an absent main input. Accept additional versions only when PromptKit actually supports their format semantics: accepting
a header does not migrate its payload or independently validate a fragment's
complete schema. The retired proposals were alternatives, not released formats
sharing specification version 1.

Use `SourceVersionModeIfPresent` without adding a PromptKit mode-selection
option in this round. A stricter option can be considered for a demonstrated
consumer need. The upstream modes remain distinct:

| Mode | Implemented behavior | Role in this proposal |
| --- | --- | --- |
| `Default` | Existing main-only effective-path check and protected-path behavior | Not the proposed shared-source policy |
| `IfPresent` | Checks present declarations; skips absent non-main declarations | Initial choice for compact fragments |
| `Required` | Requires declarations on every participating document | Upstream capability; exposing a PromptKit option is deferred |

Participating documents include actual main and underlay inputs, transitive and
glob imports, and the selected overlay. Defaults, synthetic CLI overrides,
resolution output, absent optional imports, and empty groups do not need headers.
Explicit null is present and reaches the checker; malformed selected values
fail ConfigKit's version-type validation. A rejected declaration fails loading
even if a higher layer would supersede that source's payload.

In enabled modes, accepted non-main headers are consumed before composition.
The main declaration remains and is protected at its destination-prefixed path.
PromptKit's recommended main is unmounted, so that protected path is `/version`.
Do not confuse header consumption with permission to override the remaining
protected application field.

`IfPresent` does not require a collection entry header and does not infer a
fragment's compatibility from its entry. A publisher can declare compatibility
in the entry while leaving imported files headerless; the assembled configuration
still undergoes normal validation. A declaration is not a guarantee of future
compatibility or independent schema validation of every imported fragment.
ConfigKit checks the declarations actually present. Requiring headers only on
entry files is not supported by the load-wide mode. Do not emulate it using a preliminary manifest load or validation of
source history. If that stronger guarantee becomes essential, seek an explicit
upstream contract instead.

### Naming And Optional Main Inputs

Reserve source-root `version` for specification declarations throughout the new
format. Use `release_version` for optional collection release labels and retain
nested prompt `version` for the optional installed-prompt label. An old release
file containing only `version: "2.4.0"` can fail the specification check; if its
text happens to be accepted, that field is consumed and lost as release metadata.
Migration must rename it rather than simply enable the new mode.

This reservation affects resource fragments whose local key is literally
`version`: a root mapping with that name is interpreted as a malformed header.
Allow that resource name through a nested resource mapping, for example
`prompts: {version: ...}` in a collection entry. Reserve the header position,
not the public resource name. Existing `configkit` and `extends` structural
reservations also remain applicable; destinations do not bypass them.

An absent header in an actual main document still reaches the checker under
`IfPresent`; `NoSource` invokes the absent-main checker as well. PromptKit's
checker accepts omission in both cases. Consumers may assemble only underlays,
fragments, or programmatic inputs without manufacturing a main document or
header. Do not inject a declaration or report undeclared sources as checked.
Present unsupported declarations still fail; final schema validation still runs.

### Inspection And Effective Collection Inventory

`SourceInfo.VersionCheck` is source evidence, not a generic success flag. A
non-nil record with `Present == false` can describe an omission for which the
checker did not run. Header paths and locations are source-local. Evidence is
detached and does not reread inputs; text is conservatively redacted during
callbacks and on failed loads, and successful final records respect completed
sensitivity classification. These records belong in explicit configuration
inspection, not in ordinary domain validation or collection-release identity.

Derive collection inventory from the final `collections` mapping: namespace
plus optional effective `release_version`, sorted and detached from the captured
engine. Metadata-only and resource-only entries can both appear. Root unqualified
resources do not invent a named collection; resource-based namespace enumeration
remains separate. Exact exported inventory types remain API design work.

An inventory describes effective configuration, not immutable package identity,
historical source ownership, or proof of unchanged published contents. Operator
overrides may change resources while retaining a release label. Engine
construction captures definitions; edits require a replacement engine.

## Retired Alternatives And Upstream Request

The unselected proposals were retired after the identified-record design was
captured for separate upstream consideration. Both original documents remain
available in PromptKit Git history at `b5866dd`:

```sh
git show b5866dd:docs/roadmap/configuration-collections-current-configkit.md
git show b5866dd:docs/roadmap/configuration-collections-identified-records.md
```

Proposal 1 used fully qualified mapping keys and the then-current ConfigKit
composition behavior. Proposal 2 used explicit identity fields in record
sequences and requested identity-based composition and inheritance. Neither is
an active PromptKit implementation option. The selected mounted-mapping design
meets the one-installed-prompt requirement with compact local keys and ordinary
mapping composition.

Proposal 2's generic extension is preserved in the ConfigKit repository as
`docs/roadmap/promptkit-identified-records-feature-request.md`. It requests
explicit record-identity registration, composition and overlay by identity,
record inheritance, deterministic ordering, and coherent inspection. That
request is for upstream evaluation, not an accepted ConfigKit capability or a
prerequisite of this roadmap. The subsequently accepted PromptKit decisions
are recorded below.

## Migration

Migration must choose one installed definition for each former multi-version
prompt, move any retained version into its optional field, and relocate
qualified resource keys into collection/local-key mappings. Update destination
declarations, stored references, inventories, and configuration-inspection paths. Rename collection release metadata to
`release_version`, configure the shared checker and mode, and account for the
source-root header reservation and optional main declarations. Remove obsolete
request-version fields and version-selection logic from consumer examples and
public APIs rather than retaining them as assertions. Preserve provider model
strings and source resource bases. Namespace mounting does not
automatically relocate schema or artifact stores or rewrite references when a
collection is renamed.

Retain the accepted removal of mandatory catalog modules and built-in profiles.
This is a breaking configuration/API change under the
[release policy](../policy/release.md); implementation must include consumer
migration guidance and update current contracts when it lands. The proposal
does not change today's supported configuration or API.

## Delivery And Validation

Select a released ConfigKit dependency containing the reviewed capabilities,
then implement PromptKit's schema, source-selection convenience APIs, and final
configuration adapter. Do not use a committed filesystem replacement as the
release integration. Build qualified runtime indexes after handoff. Collection
installation, remote fetching, implicit discovery, live reload, identity-based
list composition, and dependency solving remain outside this work.

PromptKit acceptance should exercise typed assembly of multiple collections,
relative/nested imports, source-local header checking and consumption, ordinary
release and prompt metadata, unqualified resources, scoped selection, optional
prompt versions, inventory without release metadata, source-relative content,
intentional partial overrides, headerless entries/fragments, omitted main inputs
and headers, and resources named `version` in nested mappings. Verify ID-only
lookup and descriptive labels without version selection or request assertions.
Do not duplicate ConfigKit's generic source validation or enforce domain rules through historical source inspection.

Review verification on 2026-10-16 against ConfigKit `e66c371`: focused public API
and consumer-example tests passed. A temporary typed consumer also passed
combined assembly for two collections with equal local prompt names, nested
relative imports and content-file contexts, optional prompt versions, both
enabled header modes, rejected headers despite upper-layer overrides, and
migration behavior for old release fields. The temporary consumer lives outside
both repositories. An additional offline check assembled this document's YAML
fragments through entry imports and matched the illustrated effective tree;
local links and whitespace checks passed. These checks verify upstream behavior
and the candidate schema, not an implemented PromptKit engine feature.

## Accepted Configuration Decisions

The following decisions settle the six questions previously listed here:

1. **Main declaration:** optional, including for programmatic or underlay-only
   construction. Validate supplied declarations without injecting missing ones.
2. **Public mode selection:** use `IfPresent`; defer a stricter PromptKit option
   until a consumer need justifies it. No per-file header requirement.
3. **Reserved header naming:** reserve source-root `version`, while allowing a
   resource named `version` through a nested resource mapping.
4. **Request versions:** remove request-side version fields and selection
   machinery. Retain optional descriptive prompt labels only.
5. **Cross-collection inheritance:** use current ConfigKit inheritance; direct
   profile references and authored expansion remain available. Defer a generic
   upstream extension without adding a PromptKit resolver.
6. **Consumer API and packaging:** provide a small source/namespace convenience
   API; recommend entry files and relative imports without requiring a filename,
   layout, main input, or single peer group. Preserve direct fragment loading,
   programmatic sources, and consumer-controlled precedence. Inventory reports
   effective collections with optional release metadata.

## Remaining Delivery Details

No unresolved configuration-policy decision remains from those six questions.
Implementation still needs to select a released ConfigKit dependency containing
the required capabilities, finalize helper signatures and inventory/optional-label
representations, and provide migration examples. These are delivery details,
not reasons to introduce additional workflows or speculative API options.

Stricter header modes and cross-path inheritance are deferred capabilities, not
implementation prerequisites. Update current contracts when behavior lands;
this roadmap records future work and does not itself implement the feature.