eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Prepare Promptkit v0.12.0 release

Browse Source
This commit is contained in:
Eric Rakestraw
2026-10-03 01:26:43 +00:00
parent 62a4a20529
commit 9903b28340
7 changed files with 91 additions and 1624 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -37,6 +37,10 @@ boundary and constraints that framework work must preserve.
## Release Guidance
Consumers upgrading to `v0.12.0` should read the
[v0.12.0 release notes](docs/releases/v0.12.0.md) for the intentional ConfigKit
configuration and public construction API migration.
The [v0.11.2 release notes](docs/releases/v0.11.2.md) cover policy alignment
and copied-policy attribution updates.

74
docs/releases/v0.12.0.md Normal file
View File

@@ -0,0 +1,74 @@
# Promptkit v0.12.0
This release replaces Promptkit's configuration loader with ConfigKit and
migrates maintained catalogs to keyed `/v3` assets. It intentionally changes
the public construction API and configuration format.
## Summary
Backends, profiles, and versioned prompts now share one layered configuration
snapshot built when `NewEngine` is called. ConfigKit supplies composition,
inheritance, presence, and source history; Promptkit validates the final
effective definitions and captures prompt content and templates. Consumers can
inspect the captured configuration explicitly without triggering a reload.
## Compatibility
This is an incompatible minor release under Promptkit's pre-1.0
[versioning policy](../policy/release.md#versioning-and-compatibility). The
former source options, `Config.PromptDir`, `Config.ProfileDir`, and the old
profile/backend convenience values have been removed. Configuration documents
now use literal keys beneath `backends`, `profiles`, and `prompts`; embedded
`id`/`version` fields and `base_profile` are no longer accepted. Source groups
compose strictly within a group and overlay in the supplied order. Explicit
zero, false, empty, and null values can now differ from omission; maps overlay
recursively, and an empty map does not clear an inherited map.
All final definitions, including unused ones, are validated during engine
construction. Prompt text and static templates are captured then; an existing
engine does not automatically reload changed configuration or message files.
Request artifacts, schemas, credentials, and provider operations retain their
runtime boundaries. See the [format reference](../formats.md) for the current
configuration contract and the [consumer migration guide](../consumers/pkg-promptkit.md#migrate-from-the-previous-configuration-api)
for API and source-selection changes.
Maintained OpenRouter and Rakestrawhome catalogs are selected as `/v3` modules,
both at `v3.20260930.0`. Their asset format changed; consumers that read catalog
assets directly must adopt the keyed format. The maintained Promptkit profile
IDs remain available under the [catalog compatibility policy](../policy/release.md#catalog-profile-compatibility),
which does not promise fixed profile data.
## Upgrade
```sh
go get gitea.maximumdirect.net/eric/promptkit@v0.12.0
```
Replace removed source and backend registration options with
`Config.Sources`, explicitly ordered ConfigKit underlays, and optionally a
highest-priority main input. Use `DirectoryInputs` or `FilesystemInputs` for
selected directories, `ProfileSource` and `BackendSource` for typed
programmatic values, or `configkit.Mapping` for explicit null and combined
documents. Move profile `base_profile` to `extends` and embedded identities to
mapping keys. Review zero/false/null values and any assumptions that replacing
a model or backend discards unrelated explicit settings. Rebuild the engine
when definitions or referenced message files change. The
[consumer guide](../consumers/pkg-promptkit.md#migrate-from-the-previous-configuration-api)
provides the migration details and maintained examples.
## Changes
- Add ConfigKit v0.5.0 as the configuration boundary and consume released
OpenRouter and Rakestrawhome `/v3` catalog modules.
- Add keyed backend, profile, and prompt definitions with final-value domain
validation, explicit presence/null behavior, and source-relative content
capture.
- Add `ConfigurationSources`, directory discovery helpers, programmatic
profile/backend source builders, and read-only configuration inspection.
- Preserve execution workflows while rendering captured templates, applying
backend defaults after inheritance, and honoring explicit suppression of
provider settings.
- Remove superseded loader, catalog preflight, and inheritance machinery;
update examples, format guidance, and internal documentation.
[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/COPYRIGHT) | [LICENSE](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/LICENSE) | [ATTRIBUTION](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/oss/attribution.md)

505
docs/roadmap/configkit-assessment.md
View File

@@ -1,505 +0,0 @@
# ConfigKit Adoption Assessment
Status: assessment aligned to released ConfigKit v0.5.0. Upstream capability
prerequisites are available; PromptKit integration and migration decisions remain
for the feature roadmap and implementation plan. No runtime changes are part of
this assessment.
## Goal And Recommendation
Use ConfigKit to standardize PromptKit's configuration semantics and simplify
its loading machinery, improving ConfigKit where a reusable capability is
justified. Preserve substantive consumer capabilities, not historical loading
sequences, file syntax, or Go signatures. Document intentional breaking changes
and their migration paths.
Recommend adoption through an eager, immutable definition-loading boundary.
ConfigKit should own YAML interpretation, source-tier combination, strict typed
decoding, sibling inheritance, presence, and source diagnostics. PromptKit should
own definition selection, LLM-specific rules, and execution. Do not retain the lazy loading
architecture merely to minimize migration, or introduce a second generic
configuration framework inside PromptKit.
The expected benefits are consistent configuration semantics for applications
using both libraries and removal of repeated discovery, partial metadata parsing,
lazy error arbitration, and repeated profile-chain resolution. Some of those
savings come from the loading redesign rather than ConfigKit itself. ConfigKit
adds ongoing value by owning the generic decoding behavior and its tests.
## Released ConfigKit Capabilities
ConfigKit v0.5.0 satisfies the two requested prerequisites and also implements
sibling inheritance. Adopt the released public contracts rather than waiting for
the earlier proposals to be implemented:
1. **Filesystem sources:** `NewFilesystem(label, fsys).File(name)` borrows an
`fs.FS` with reusable namespace identity. Imports and resource contexts retain
that namespace. `Result.SourceContext` and `Resolution.SourceContext` expose
passive `ResourceContext` values with `FS` and `BaseDir`.
2. **Source-backed underlays:** `LoadLayered` accepts a main `Input`
and ordered `Underlay` groups. Each group composes strictly; groups overlay
recursively from lowest to highest priority. `Input{Source: NoSource()}` omits
the main input. `Mapping(label, map[string]any)` snapshots supported built-in
scalar/list/map values with explicit presence; it does not ingest arbitrary
typed structs. Generic programmatic input is not represented as CLI overrides.
3. **Sibling inheritance:** the reserved `extends` key names one immediate sibling
mapping. Relationships resolve after precedence assembly and before typed
decoding, with inherited origins, history, and resource context preserved.
This is built-in behavior, not the opt-in/registered-collection API originally
suggested by PromptKit.
Adopt ConfigKit's recursive combination model rather than preserving PromptKit's
whole-profile shadowing or atomic provider-map replacement across source tiers.
No hidden replacement or inheritance pass should run after ConfigKit loading.
Use released sibling inheritance for profiles, leaving domain validation in
PromptKit. This assessment does not authorize code changes in either repository.
## Evidence And Scope
PromptKit's runtime baseline remains v0.11.2; this assessment has since received
documentation-only revisions. Updated upstream review on 2026-09-30:
- PromptKit: `12d6d80024e80c552d122a076c22429bf6f2f6c2` (v0.11.2).
- ConfigKit: `dc09959cca4e105f49ec316c6d4ba585bd2340bd` (v0.5.0), verified
against the published tag. Its clean checkout passed `go test ./...` during
the release review. This replaces the earlier v0.4.2 and uncommitted-roadmap
comparison as the upstream basis for adoption.
PromptKit evidence: [consumer workflows](../consumers/pkg-promptkit.md),
[file formats](../formats.md), [source implementation](../internal/sources.md),
[runner](../internal/runner.md), and the implementation owners identified below.
ConfigKit evidence at that release: `docs/config.md`, `docs/releases/v0.5.0.md`,
`layered.go`, `source.go`, `resource_context.go`, and ADR 0009 in the sibling
checkout. That checkout is review context, not a build dependency or required
directory layout. The released configuration contract owns exact upstream behavior;
the summaries here identify its consequences for the proposed PromptKit design.
This is a source-and-contract assessment, not a performance benchmark or full
correctness audit. It does not claim measured allocation, latency, or line-count
savings. Current code and canonical contracts remain authoritative until an
approved implementation changes them.
## 1. Capability Inventory
| Preserve the ability to… | Replaceable mechanics and representation |
| --- | --- |
| Define named, versioned prompts, inspect their declared inputs, and select a version explicitly | Directory scans on each operation, YAML metadata prepasses, filename layout, constructor signatures |
| Use inline and file-backed message templates, variables, input helpers, session templates, roles, and cache control | Timing of definition/content reads and private repository interfaces |
| Configure named profiles, alias/inherit another profile, and override selected settings | Zero-as-absence fields, per-call chain traversal, raw YAML decoding types |
| Supply embedded application defaults, operator definitions, maintained catalogs, and programmatic profiles | Repository-wrapper stacks and exact source-option API |
| Register custom/local backends, use built-ins or endpoint-only profiles, and control backend capacity | Configuration DTO shapes and registry assembly order |
| Prepare, inspect, execute, or freeze an execution for later single-claim execution | Source reloading during ordinary preparation; exact preparation error ordering |
| Override per-run settings, credentials, reasoning, session, and output contract; append rendered messages | Public value signatures if presence or clearer configuration separation requires revision |
| Supply inline/file artifacts and inject artifact readers, model clients, and HTTP clients | Assembly APIs; runtime collaborators need not be YAML-decodable |
| Validate output, use provider-native structured output, bounded repair, transport retries, cancellation, and observation | Internal wiring, not the substantive execution behaviors |
| Receive results, usage, identities, hashes, validation outcomes, and inspectable failures | Configuration error types and when definition failures appear |
Preserve independent profile and prompt inspection as useful operations. They
need not retain the old guarantee that engine construction has never examined
other definitions. Preserve custom backend identity and capacity behavior,
optional upstream credentials, and the distinction between failed generated
content and operational errors. This project does not authorize silently
dropping execution features to simplify the configuration layer.
Add explicit configuration inspection for consumers implementing operator-facing
CLI commands or debug output. This is a new capability motivated by this adoption,
not an assertion that the current prompt/profile inspection methods already
provide configuration provenance.
## 2. Responsibility Mapping
| Concern | Recommended owner and disposition |
| --- | --- |
| YAML grammar, scalar interpretation, one-document enforcement, unknown fields, typed ranges, null legality | Delegate to ConfigKit; remove corresponding PromptKit parser logic and redundant parser tests |
| Field presence, source coordinates, and configuration provenance | Delegate to ConfigKit; retain its inspection state with the loaded snapshot for explicit operator queries, separate from execution-ready values |
| Operator-facing configuration inspection | Provide a PromptKit consumer API reusing ConfigKit inspection facilities; applications own CLI commands, presentation, and disclosure policy |
| File/directory/embedded source selection and definition ID indexing | Retain a small PromptKit collection loader; ConfigKit loads documents, not a prompt catalog |
| Configuration filesystem sources, relative imports, and generic layer origins | Delegate to the prerequisite ConfigKit APIs; do not maintain parallel reader-based loading machinery |
| Definition discovery and relative message-content references | Retain PromptKit's domain resource adapters, using ConfigKit source context and standard `io/fs` facilities where sufficient |
| Profile-source priority and combination | PromptKit selects tiers; ConfigKit owns strict within-tier composition and recursive cross-tier overlay; eliminate repository fallback and whole-definition shadowing |
| Named profile inheritance | Delegate sibling graph resolution, combination, and derivation to ConfigKit's `extends`; remove the custom profile resolver |
| Backend selection and domain normalization | Retain PromptKit domain rules; use recorded application resolution for configuration transformations not already handled by structural inheritance |
| LLM setting bounds, role/session normalization, output rules, reserved provider fields | Retain shared PromptKit domain owners; ConfigKit validation callbacks may invoke them |
| Definition defaults | Use ConfigKit defaults where they do not erase inheritance presence; apply effective execution defaults after inheritance |
| Programmatic configuration | Prefer labeled mapping input with explicit presence; evaluate whether typed PromptKit convenience constructors justify conversion code, without YAML round trips or zero-value inference |
| Runtime clients, observers, artifact readers, request data | Retain normal Go injection and request APIs; do not load them through ConfigKit |
| Credentials | Retain optional environment-name and direct request-key semantics; do not replace these with ConfigKit's required typed secret resolution |
| Catalog manifest, ownership, reserved names, secret-safety checks | Retain the catalog adapter; reuse the common ConfigKit profile decoder |
| JSON Schema resources and generated-output validation | Retain JSON Schema implementation; these are not application configuration documents |
| Application imports, CLI overrides, deployment overlays, secrets-directory bootstrap | Leave with consuming applications unless a concrete PromptKit need is accepted |
| Generic source/decoding diagnostics | Reuse ConfigKit; add only PromptKit identity/context and domain relationship diagnostics |
## 3. Recommended Target Loading Model
### Definition Snapshot
Create one immutable definition snapshot during engine construction. Discover
the explicitly supplied sources, assemble their contributions through ConfigKit,
decode the effective configuration, normalize domain values, and resolve relationships
before publishing the engine. No provider call or credential availability check
belongs in this phase. Publish no partially initialized engine on failure.
Prefer keyed collections, such as `profiles.<name>`, assembled before one typed
loading lifecycle instead of independently decoding each profile and recombining
the results. Multiple files can contribute to those collections; this does not
require a monolithic file. Reconsider the previous one-typed-document-per-definition
recommendation. Settle prompt ID/version representation and catalog format migration
in the feature roadmap. Support OS directories, individual files, embedded/custom
`fs.FS` sources, and programmatic input through the new source model; source selection
and definition discovery remain PromptKit concerns, not a second composition engine.
ConfigKit expands structural inheritance before typed defaults/decoding. Perform
any remaining domain-specific resolution through its recorded lifecycle before
semantic validation and result publication. Reuse its presence and provenance;
do not add a generic merge/defaulting pass or reconstruct origins after loading.
Runtime client assembly and resource interpretation remain outside typed decoding.
Load and freeze referenced message text with definitions. Validate static prompt
structure and template syntax at loading, using the same template function setup
as execution; render only when request variables and artifacts exist. Preserve
source-relative content resolution and containment without expanding the threat
model. A snapshot is a set of captured reads, not an atomic filesystem transaction.
Resolve each effective content reference using its contributing source's context,
not the highest tier's directory. Use ConfigKit's released direct context
query rather than a consumer-maintained filesystem registry. Composite origins
must not be assigned an arbitrary directory; ambiguous or absent context needs
an actionable resource error or explicit consumer-supplied context. ConfigKit
does not snapshot external text: PromptKit opens it explicitly and freezes it
before engine publication. Caller-owned filesystem lifetimes remain caller-owned;
close resource handles acquired by PromptKit without retaining ConfigKit load handles.
Keep request artifacts and schema compilation at preparation. A request can
replace the output contract, so an unused default schema should not need eager
compilation. Prepared execution continues to freeze its effective resources.
The snapshot covers definitions/message text, not environment values, request
files, every schema file, or execution state.
### Composition And Validation
Use explicit source categories in the existing useful priority order:
programmatic profiles, operator source, application fallback, PromptKit-owned
profiles, then external catalogs. Express these as explicit ordered tiers, not
source discovery order. Within a tier, mappings compose recursively and overlapping
non-mapping contributions conflict, even when equal. Disjoint contributions to
the same profile may therefore combine; do not preserve a blanket duplicate-ID
rejection. Across tiers, mappings overlay recursively, retaining omitted lower
children. Effective prompt identities/versions must remain addressable, with an
omitted version ambiguous when multiple versions exist; finalize their keyed
representation without reintroducing whole-definition shadowing.
All sources undergo syntax, acquisition, and composition checks; higher tiers
cannot hide those failures. Decode and validate effective values after assembly,
not every intermediate tier against the final typed schema. A replaced lower
value may consequently have a type that would not fit the final schema. An unused
invalid effective definition still fails construction. Catalog ownership,
reserved-name, and secret-safety checks remain distinct source-policy requirements,
not a reason to impose intermediate typed validation on ordinary inputs.
Preserve profile templates that become usable through descendants or runtime
overrides: eager loading must not require every consumer profile to be an
independently runnable endpoint/model pair. Validate supplied fields and graph
integrity eagerly (ConfigKit handles missing bases and cycles), and check final execution
requirements after runtime selection/overrides. Prompt default-profile references
can be checked against the assembled collection; a prompt with no default remains
valid for callers that supply a profile explicitly.
ConfigKit's ordering is typed defaults, source-underlay tiers, strictly composed
main input, selected overlay, then CLI overrides. PromptKit's API must map its
source priorities explicitly onto supported roles; do not mislabel programmatic
input as CLI flags. Underlay-only loading avoids inventing a main document.
Imports remain opt-in, and the named-overlay convenience need not be exposed by
PromptKit merely because ConfigKit supports it.
A prompt's `version` is a domain identifier, not automatically a ConfigKit
application-schema version. Do not configure `VersionRule` for prompt versions.
If a separate configuration schema version is later needed, respect ConfigKit's
designated main-input ownership and pre-underlay check timing; do not assume
last-wins version selection across catalogs.
### Source Presence And Released Sibling Inheritance
Adopt ConfigKit source-layer semantics: absence retains lower input; explicit
zero, false, and empty strings replace it; mappings combine recursively; empty
mappings retain lower children; sequences replace as units; null explicitly
replaces, subject to final schema legality. No reset/delete operator is assumed.
Preserve presence through domain resolution, backend/framework
default application, request overrides, and outbound request construction.
Otherwise accepting `temperature: 0` during decoding would accomplish nothing.
Use ConfigKit metadata for both file and programmatic input. A final Go value
alone does not distinguish absent from explicit zero. Avoid eagerly defaulting
profile fields in ways that erase that distinction before relationship resolution.
Provider `extra_params` maps combine recursively across source tiers too: `{}`
does not clear lower keys. Retire the proposed atomic source-map replacement and
null-as-unspecified conventions; do not emulate them after loading. An operator
overriding only `model` also retains a lower `extends` declaration and other
omitted settings. Diagnose and document these consequences explicitly.
Replace `base_profile` with `extends` on sibling profile mappings. A string names
one exact immediate sibling; null cancels the relationship while preserving the
child's own settings. Omission retains a lower-tier directive. There is no ancestor
scope search or cross-collection reference. Superseded references are history,
not active relationships to validate; every effective relationship resolves,
including profiles not selected for execution.
Children recursively overlay their resolved parents with the same value rules:
zero/false/empty strings and sequences/null replace, mappings combine, and `{}`
does not clear inherited keys. Inherited values count as explicit input for
presence queries and absence-based defaults, even when false, zero, or null.
Do not repeat inheritance in PromptKit's application-resolution callback. Use
`Origin.Inheritance` and retained history for explicit operator inspection.
Inherited resource references retain their declaring source's context.
Enclosing inheritance resolves before nested relationships; nested inherited
directives resolve at each destination against its siblings. Defaults and
application-derived values do not trigger another inheritance pass. Model this
deliberately in PromptKit configuration rather than promising arbitrary graph
resolution over runtime values. Partial bases must fit the typed schema and
PromptKit's validation; ConfigKit grants them no special validation exemption.
`extends` is reserved in every input mapping, including provider-parameter maps,
across all source kinds. There is no opt-in switch or ordinary-data escape in the
released contract. Audit catalog and consumer payload needs before finalizing
their format; a literal provider field with that name cannot silently pass through
the configuration loader. Do not invent an encoding workaround in this assessment.
The existing request-level override API remains a separate runtime boundary.
The feature roadmap must review its interaction with configured values without
silently subjecting runtime request data to configuration inheritance directives.
### Reloading, Errors, And Application Integration
Later definition edits take effect through a newly constructed engine. Do not add
watchers, mutable registry reloads, or a second lazy compatibility mode. Applications
choose how to swap engines and let in-flight work finish. Old prepared handles
remain associated with their original engine. Simultaneously active engines have
separate backend capacity pools; migration guidance must not imply a global limit
across a reload overlap.
Return configuration failures from construction with file/path/location and
useful definition identity where available. Recommend preserving ConfigKit's
public `LoadError` through wrapping for `errors.As`, together with a PromptKit
configuration category; do not copy its entire diagnostic API into the facade.
Domain-only relationship failures remain PromptKit errors with relevant source
locations. Start with deterministic fail-fast loading across files and ConfigKit's
existing multi-diagnostic behavior within a load, rather than inventing a batch
diagnostic subsystem. Do not expose raw credentials or add implicit logging.
Runtime failures continue to distinguish invalid requests, missing selections,
rendering/artifact/schema errors, provider/transport errors, capacity rejection,
and content-validation results. Cancellation remains effective at
PromptKit-controlled boundaries; ConfigKit's current synchronous `Load` has no
context argument, so do not promise interruption inside it or create goroutines
to simulate that guarantee.
Applications using ConfigKit can present the same underlying loading diagnostics
for their application settings and PromptKit files. They can also translate
application-owned typed settings into PromptKit profiles/backends. Do not require
applications to encode Go clients into YAML or adopt ConfigKit merely to call
PromptKit's runtime API. The operator-inspection capability below supplies the
concrete consumer purpose for exposing effective configuration and its provenance.
### Explicit Configuration Inspection
Provide a consumer-facing API for inspecting effective configuration and its
provenance so applications can implement targeted CLI commands or debug output.
Keep three activities distinct:
- **Construction:** load and resolve configuration, retain ConfigKit's provenance
and inspection state, and publish execution-ready definitions. Resolve any
presence and origin information needed for definition construction here.
- **Ordinary execution:** use the ready definitions directly. Do not analyze
configuration provenance, traverse origin/history records, reconstruct
inheritance explanations, or render effective-configuration reports as part of
a prompt run. Carry execution-relevant presence explicitly in resolved values
rather than querying configuration metadata during each run.
- **Explicit inspection:** query effective settings and their contributing
sources or explanations only when a consumer invokes the inspection API.
Inspection uses the same loaded configuration snapshot as the engine; it does
not reread configuration files, open referenced resources, or contact providers.
Reuse ConfigKit's inspection and redaction facilities rather than maintaining a
second provenance system. The API should make targeted queries practical; exact
signatures, query scope, and safe result ownership belong in the feature roadmap.
Do not imply that configuration inspection predicts all runtime request overrides
or reveals the values of execution-time credentials. Existing prompt/profile
inspection, prepared-run details, and execution observations have distinct purposes;
configuration inspection does not replace them or add provenance to run events.
Applications decide whether and how to expose inspection through operator tools.
PromptKit adds no CLI, implicit debug logging, or automatic explanation output.
Redacted configuration can still contain sensitive application content, so callers
retain responsibility for access control and disclosure. Retention has a memory
cost, but explanation generation and provenance analysis stay off the run path.
## 4. Simplification Accounting
| Current implementation | Expected disposition |
| --- | --- |
| `internal/profile/filesystem_repository.go`: metadata scans, selected strict decode, stream-end checks, malformed-file selection arbitration | Replace with source acquisition, ConfigKit assembly/decoding, and indexed effective values; remove the metadata prepass and per-lookup filesystem path |
| `internal/promptdef/filesystem_repository.go`: metadata selection and repeated decoding/content opening | Replace with eager prompt loads and an ID/version index; keep domain normalization and contained content reading |
| Profile filesystem, `fs.FS`, eager, memory, and overlay repository variants | Collapse loading into source adapters plus immutable collections; preserve only interfaces that still serve a real boundary |
| `internal/profile/resolving_repository.go` | Remove custom graph traversal, cycle detection, and zero-sentinel merging in favor of ConfigKit sibling inheritance; retain only PromptKit domain validation |
| `internal/engine` raw repository-stack assembly | Replace with ConfigKit source tiers and one effective typed collection; engine capacity/client assembly remains |
| `internal/catalog` profile metadata extraction | Obtain explicit field information from ConfigKit; retain manifests, ownership, namespace, connection-metadata and secret checks |
| `internal/filecatalog` | Retain/reduce discovery and containment helpers actually needed by the eager loader; do not claim the entire package disappears |
| `internal/domain` source YAML tags and runtime/source coupling | Separate configuration DTOs from execution values where presence requires it; reuse domain validation rather than duplicating it |
| Direct `gopkg.in/yaml.v3` use for definition loading | Remove once all such consumers migrate; ConfigKit brings its own private YAML dependency |
New integration work is real: collection DTOs, ConfigKit calls and error wrapping,
source-context access, collection indexing, programmatic mapping ergonomics, and
conversion to runtime definitions. Catalog documents may need new keyed shapes;
do not hide that migration behind a permanent second loader.
Some defensive copies remain necessary at public/injected-client boundaries;
ConfigKit's detached load result is not permission to expose shared engine maps.
`internal/jsonvalue` still serves programmatic values, request parameters, and
prepared schema ownership, so adoption does not eliminate it.
Maintenance savings depend on actually deleting the old lazy paths and their
special-case tests. Move generic YAML/scalar behavior coverage to ConfigKit's
ownership; PromptKit retains representative adapter tests plus domain, precedence,
snapshot, inheritance, embedded-source, error, and execution integration tests.
Do not duplicate ConfigKit's entire parser suite or discard execution regression
coverage because construction fixtures need migration.
Runtime benefit: ordinary runs no longer rediscover definition directories,
redecode definitions, reread template text, or traverse inheritance chains.
Cost: construction loads even unused definitions and retains their text and
resolved values. Retain ConfigKit inspection state needed by the explicit operator
API without duplicating its history in runtime definitions or per-run values.
These are reasoned tradeoffs, not measured performance
claims. Adoption is not justified by the decoder replacement alone.
### Algorithmic Efficiency
Efficiency is an explicit design criterion for both the upstream work and this
shared-library refactor. Prefer optimal or near-optimal algorithms relative to
input and produced-data size, not a blanket O(n) claim with an undefined n.
- Acquire/parse each accepted source occurrence once, assemble input, and run
one typed loading lifecycle rather than decoding intermediate tiers. Reuse
captured content instead of reparsing it for metadata, selection, and validation.
- Build identity indexes in one pass, with expected constant-time map lookup.
Do not scan whole collections for every lookup or precedence decision.
- Delegate inheritance expansion to ConfigKit once during loading rather than
rebuilding its graph in PromptKit. Assess scaling against input, materialized
inherited values, nested destination resolution, and retained history; do not
claim unconditional O(V + E) total work from the count of profiles and edges.
ConfigKit bounds expansion and retained evidence through its existing budgets;
representative catalog sizes must fit them without new per-run processing.
- Avoid whole-collection copying per layer, repeated serialization, and repeated
prefix/history traversal. Analyze layer processing against total input nodes
and produced values/provenance, including intentionally retained history.
- Keep configuration discovery, decoding, and provenance analysis out of ordinary
execution. Perform explanation work only on explicit inspection calls. Copy only
where ownership requires it; do not trade caller isolation for speed.
These are design targets, not promised complexity for every operation. Sorting
for deterministic discovery may require O(F log F) for F files, and processing
message text costs work proportional to its bytes. Construction time, retained
memory, and frequent-call costs all matter. Prefer straightforward indexes and
clear immutable ownership over speculative global caches or intricate optimizers.
The implementation plan should include focused benchmarks for consequential
repeated-work risks, such as shared-ancestor profiles and many layers, rather than
machine-dependent timing gates or benchmarks for every helper.
## 5. Upstream Readiness And Remaining Opportunities
Filesystem sources, underlays, programmatic mappings, source contexts, and sibling
inheritance are released in v0.5.0. No additional upstream feature is currently a
demonstrated prerequisite. Context-aware loading remains a possible future request
if concrete consumers need it, not a reason to delay adoption. Existing loading
is synchronous; do not simulate cancellation with detached background work.
Do not propose a public raw AST, permissive identity-only decoder, or dynamic
subtree conversion solely to preserve PromptKit's current scanning design.
Do not upstream backend/model selection, credential timing for inference,
profile catalog reservations, or prompt-template functions.
ConfigKit and PromptKit both declare LGPL-3.0-or-later. Adoption must still update
dependency notices under the OSS policy and inspect the exact selected release
and its transitive notices. Their current Go minimums do not require a PromptKit
toolchain increase. Use a released module dependency, not a sibling-directory
replacement. Select v0.5.0 as the assessed adoption baseline; any newer selection
needs its own review. This assessment changes neither dependencies nor licenses.
## 6. Decisions And Migration
The agreed direction now incorporates ConfigKit's v0.5.0 contracts. This supersedes
the earlier recommendations for independently typed documents, whole-definition
shadowing, intermediate typed validation, and atomic maps/null-as-unspecified at
the source boundary.
1. **Loading lifetime and failure scope.** Keep eager construction and frozen
definitions/content. Syntax/acquisition/composition failures remain unmaskable;
typed and domain validation apply to effective values. A separate reusable
public snapshot lifecycle remains possible but is not currently needed.
2. **Source organization.** Prefer keyed collections composed across files and
tiers before typed loading. Settle exact profile and prompt/version shapes,
source-role mapping, and catalog migration in the feature roadmap. Independent
typed loads followed by custom combination are not the selected direction.
3. **Presence and relationships.** Accept recursive source-tier overlay, retained
children for empty mappings, explicit null replacement, and effective metadata.
Adopt built-in sibling inheritance, including reserved `extends`, recursive
mapping combination, null cancellation, and inherited explicit presence. Do
not preserve a parallel PromptKit structural resolver.
4. **Diagnostic boundary.** Keep wrapping ConfigKit's public loading errors rather
than mirroring them. Retain actionable domain relationship and resource errors
without duplicating ConfigKit's generic diagnostic/provenance system.
5. **Operator inspection.** Expose effective configuration and provenance through
an explicit consumer API backed by the engine's loaded snapshot. Retain the
required ConfigKit inspection state at construction; never analyze it during
ordinary prompt execution. Finalize targeted query shapes and ownership in the
feature roadmap, with presentation remaining application-owned.
Use released mapping-based programmatic input and direct filesystem resource-context
queries. If real consumer examples show
mapping construction to be repetitive, consider small typed PromptKit conveniences
with explicit presence rather than requiring an upstream typed-ingestion API.
Remaining PromptKit decisions are the exact collection/version format, catalog
migration, programmatic ergonomics, operator-inspection surface, and validation
of partial base profiles. Also settle how operators discard unwanted provider
settings or old relationships using the released null semantics without a hidden
replacement pass. Check the reserved-key impact on arbitrary provider maps. These
are integration decisions, not unresolved ConfigKit API proposals.
Expected consumer migration:
- Update definitions for ConfigKit's strict scalar rules, rejected YAML
anchors/aliases/tags, input limits, and null/type checks. Quote prompt version
strings rather than introducing a special numeric-version conversion merely
for historical YAML coercion.
- Move standalone definitions into the agreed keyed contribution format. Review
same-name contributions: disjoint fields within a tier can combine, and higher
tiers retain omitted lower fields, including `extends` references. Convert
`base_profile` to sibling `extends`; use `extends: null` to cancel a lower-tier
relationship, not to erase all settings already supplied to the child.
- Review zero/false/empty/null values explicitly. Provider maps merge across
tiers, `{}` does not clear them, and null replaces rather than means omission.
Use labeled programmatic mappings or approved presence-aware conveniences.
Apply those same combination rules to inherited settings. Rename ordinary
configuration keys named `extends`; examine provider maps for cases where
renaming would change their substantive meaning and resolve those explicitly.
- Handle effective-definition errors at construction, fix syntax/composition
failures even in overridden inputs, and deliberately rebuild/swap engines
after definition changes.
- Keep runtime credential lookup optional, avoid baking resolved secrets into
definition snapshots, and preserve request-scoped key overrides.
- Update maintained catalog data for collection shapes and any incompatible syntax;
coordinate catalog releases before the PromptKit dependency update. Preserve
maintained profile names and catalog ownership rules, not every old scalar
spelling. Do not validate live provider/model availability.
- Update error inspection, constructor/source options, and programmatic profile
literals as required. Keep representative preparation, execution, inspection,
custom backend, fallback, and embedded workflows in the migration guidance.
The subsequent feature roadmap should carry forward these choices, resolve the
remaining PromptKit integration decisions, and specify the target end state.
The staged implementation plan can now adopt released ConfigKit support without
waiting for upstream feature work, coordinating any catalog format releases before
their consumption. Cover deletions/replacements, contract tests, canonical documentation,
examples, attribution, and minor-release migration notes. Neither document should
become a reason to preserve obsolete compatibility machinery.

339
docs/roadmap/configkit.md
View File

@@ -1,339 +0,0 @@
# ConfigKit Adoption
Status: agreed feature roadmap. ConfigKit v0.5.0 supplies the required upstream
capabilities. The [implementation plan](implementation.md) owns execution steps.
The [configuration-boundary ADR](../adr/0001-configuration-boundary.md) records
the handoff decision. This document owns the intended end state, not
implementation authorization.
## Purpose
Use ConfigKit to standardize PromptKit's configuration semantics and simplify
its loading machinery, replacing custom logic wherever ConfigKit is the
appropriate owner. Preserve substantive consumer capabilities, not historical
file syntax, loading order, or Go signatures. Pre-1.0 breaking changes are
acceptable and must have migration guidance.
**This is an intentional breaking change. We are intentionally replacing PromptKit's configuration handling code in favor of adopting ConfigKit's simpler, standardized API. Our goal is to arrive at a new version of PromptKit that is leaner, simpler, and uses a ConfigKit-standard configuration syntax. We MUST NOT give any weight to preserving arbitrary implementation details from PromptKit's current codebase--those should be discarded in favor of the simplest viable implementation that supports PromptKit's substantive LLM-focused consumer capabilities.**
Applications using both libraries should share configuration interpretation,
layering, inheritance, presence, and inspection semantics. PromptKit should
concentrate on LLM-specific validation, resource interpretation, and execution.
The [adoption assessment](configkit-assessment.md) records the supporting analysis;
this roadmap owns the agreed feature end state. Where the earlier assessment
differs, this roadmap and the maintainer's direction above take precedence.
Do not preserve source-option wrappers, repository interfaces, incidental error
ordering, or old field shapes solely because they exist today. Reuse existing
code only when it remains the simplest appropriate owner of a substantive
capability. Migration documentation replaces compatibility machinery, not vice versa.
## Baseline And Dependency
The current PromptKit runtime baseline is v0.11.2. Its
[source implementation](../internal/sources.md), [format contract](../formats.md),
and [consumer guide](../consumers/pkg-promptkit.md) describe existing behavior,
not the target described here.
The reviewed ConfigKit release is v0.5.0 at
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`. It provides native `fs.FS` sources,
`LoadLayered`, source-backed underlays, labeled programmatic mappings, passive
resource contexts, and built-in sibling inheritance. Its released configuration
contract and GoDoc own exact upstream semantics. No further upstream feature is
currently a demonstrated prerequisite. Review any newer version before selection.
Adopt a released Go module, not a local `replace` or workspace requirement.
Update attribution and transitive license notices under the OSS policy. This
feature does not change either project's license or require a Go minimum increase
relative to the reviewed versions.
## Substantive Capabilities To Preserve
- Named/versioned prompts, declared-input inspection, templates, inline and
referenced message content, variables, input helpers, roles, and cache control.
- Maintained catalogs, application defaults, operator configuration, programmatic
profiles, aliases/inheritance, built-in/custom/local backends, and endpoint-only
profiles.
- Preparation without generation, ordinary execution, and frozen single-claim
prepared execution; independent prompt and profile inspection.
- Runtime execution-setting, reasoning, session, credential, and output-contract
overrides; appended already-rendered messages.
- Inline/file input artifacts and injected artifact, model, and HTTP collaborators.
- Backend capacity management, cancellation, transient retries, structural output
validation, provider-native structured output, bounded repair, and observations.
- Results, effective identities/settings, usage, hashes, and meaningful errors.
Retain optional environment credentials and direct request credentials. ConfigKit
secret resolution must not introduce a new requirement that an upstream API key
exist. Domain validation must not probe provider/model availability.
## Target Configuration Model
### One Shared Loading Lifecycle
Construction supplies selected configuration inputs to ConfigKit, which composes,
overlays, expands inheritance, and decodes them. PromptKit interprets and validates
the final effective configuration at the ConfigKit handoff and produces
execution-ready definitions. ConfigKit's effective-value resolution and validation
hooks may implement that handoff. Publish no partially initialized engine on failure.
ConfigKit owns generic parsing, combination, presence, origins, and inheritance.
PromptKit owns input selection/discovery, definition indexing, LLM-specific rules,
and translation of final values into runtime settings. Input adapters select or
encode contributions; they do not inspect or validate raw source contents.
Effective presence/null information and resource context supplied by ConfigKit
support interpretation at handoff. Historical inspection remains an explicitly
invoked operator capability, separate from domain enforcement.
Runtime clients and callbacks remain normal Go dependencies, not YAML types.
Use `backends.<id>`, `profiles.<id>`, and `prompts.<id>.<version>` keyed mappings
assembled before typed decoding rather than independently decoding definitions and combining them
afterward. Keys own identity; do not duplicate `id`/`version` fields. These paths
describe mapping levels, not dot-splitting of literal IDs or version keys.
Multiple files can contribute to a collection; a monolithic file is not required.
When a prompt ID has multiple versions, selection without a version remains
ambiguous. This shape fits sibling inheritance without extra identity reconciliation.
Supply explicit source groups through a small construction surface compatible
with ConfigKit's `Input`/underlay concepts. Offer convenient file/directory/embedded
entry points where useful, not a parallel set of historical loading semantics.
Accept ConfigKit's labeled mapping sources for programmatic configuration. Add
only small typed conveniences justified by the minimal local-backend and
application-profile workflows. Helpers encode explicit presence and feed the same
loading lifecycle; they neither merge configurations nor infer omission from zeros.
Backend helpers also produce inputs to this lifecycle. Backend definitions,
including catalog connection and capacity settings, cross the same ConfigKit
handoff as prompts and profiles. The immutable runtime registry is built from
effective backend values. HTTP/model clients and other runtime collaborators
remain outside typed configuration.
Keep the public package a deliberate facade and implementation private. Do not
expose internal engine/domain types or introduce a parallel generic loader merely
to preserve old source options. ConfigKit's supported public types may be used
where intentional integration is simpler than maintaining equivalent wrappers.
### Sources, Tiers, And Presence
Support ordinary files/directories, embedded or caller-supplied `fs.FS`, and
programmatic definitions. Explicitly selected directories may be discovered by
PromptKit; ConfigKit handles their configuration contributions. Do not infer
priority from traversal order.
Retain the useful priority categories, lowest to highest: external
catalogs, PromptKit-owned profiles, application fallback, operator configuration,
and programmatic configuration. Express intentional priority as ordered tiers.
Within each tier, independent contributions compose strictly; overlapping
non-mapping values conflict, even when equal. Disjoint fields under the same
identity can combine. Across tiers, mappings recursively overlay.
These rules also govern backend configuration; consumers can add backend IDs or
overlay existing definitions through their selected tiers. Registry construction
validates the resulting values, not the identity of their contributors.
Use ConfigKit's source roles honestly: underlay groups are ordered defaults,
main input overlays underlays, and optional named overlay/CLI facilities retain
their upstream meaning. Underlay-only loading is supported; do not manufacture
a main document or disguise programmatic values as CLI flags. Exposing every
ConfigKit convenience through PromptKit is not a requirement.
Adopt the shared value rules: omission retains lower input; explicit zero, false,
empty string, sequence, and null replace; mappings combine recursively; empty
mappings do not clear lower children. Null remains explicit and subject to the
final schema. Provider-parameter maps obey these rules too. No hidden whole-profile
replacement, arbitrary merge callback, or new reset/delete operator belongs here.
Preserve presence into execution-ready settings so an explicit zero does not
disappear during backend/default resolution or outbound request construction.
Use ConfigKit metadata at construction, not as a per-run lookup mechanism.
### Sibling Inheritance
Use ConfigKit's reserved `extends` rather than PromptKit's `base_profile` and
custom resolver. A string identifies an immediate sibling mapping; null cancels
the relationship without deleting the child's own settings. Layering completes
before inheritance. Only effective relationships resolve, but all of them do,
including entries never selected for execution.
Children recursively overlay resolved parents. Inherited values count as explicit
input. Origins and resource context remain attached to their declaring source.
Nested relationships follow ConfigKit's destination-local rules; defaults and
application-derived values do not trigger another inheritance pass.
The keyword is reserved throughout configuration, including dynamic provider
maps; it is not opt-in. Keep runtime request provider-parameter maps ordinary
JSON-compatible data outside configuration parsing. Consumers needing a literal
provider field named `extends` supply it at that runtime boundary. Do not rename
the outbound key or introduce a second opaque configuration format to escape it.
Document and test this distinction, including any affected maintained payloads.
### Validation And Errors
ConfigKit owns acquisition, syntax, composition, inheritance, and typed-decoding
failures. PromptKit preserves those errors and validates final effective domain
values. An unused invalid effective definition fails construction; a lower-layer
value legitimately replaced by ConfigKit is not a separate validation target.
This boundary applies equally to application and maintained catalog configuration.
PromptKit does not perform catalog preflight, raw-source or intermediate-state
validation, or history-based enforcement, including equivalent test-only checks.
Catalog maintainers own publishing checks in their respective repositories.
Domain validators remain shared with runtime request validation where appropriate.
Allow schema-valid partial profiles without an `abstract` flag. Validate supplied
domain values at loading; final runnable-target requirements are checked after
profile selection and runtime overrides at execution preparation.
Do not require credentials, artifacts, or provider connectivity during construction.
Preserve absence versus explicit null in resolved settings. Null on an optional
provider setting deliberately suppresses it; it must not restore the same setting
from backend defaults. A null provider map suppresses the configured lower map,
while null entries inside a provider map remain literal JSON null values. This
supports reusable bases and deliberate removal of stale settings without changing
ConfigKit's merge rules. Framework-required settings such as timeouts have explicit
field-specific null semantics rather than blanket nil defaulting.
Preserve ConfigKit public loading errors through wrapping for `errors.As`, with a
PromptKit configuration category. Use source-aware domain/resource failures where
ConfigKit is not the owner. Do not mirror its diagnostic framework or promise
aggregation beyond what the underlying operation provides. Runtime operational
errors remain distinct from generated-content validation failures.
Prompt versions are domain identifiers, not ConfigKit `VersionRule` declarations.
Do not introduce an application schema-version policy merely to adopt ConfigKit.
## Snapshot, Resource, And Execution Boundaries
The engine captures effective definitions and referenced message text during
construction. Validate static template syntax then; render with request data
later. Resolve content references from the effective field's contributing source
context, not the last layer's directory. Ambiguous/contextless resource references
need an explicit source context or an actionable error, not an invented directory.
ConfigKit returns passive resource contexts, not snapshots of referenced text.
PromptKit explicitly opens and closes its resource handles. Caller filesystem
ownership remains unchanged. Preserve appropriate source containment without
claiming that every `fs.FS` is a security sandbox. Captured reads are not an atomic
transaction over a concurrently changing filesystem.
Definition changes require new engine construction. No watcher, background
refresh, or lazy compatibility mode is added. Applications own engine replacement;
prepared handles remain bound to their original engine and overlapping engines
retain separate capacity pools.
Request artifacts and schema preparation remain invocation work. A request can
replace the output contract, so construction need not compile unused schemas.
Prepared execution continues to freeze its effective resources. Environment
credential lookup remains at the established execution/preparation boundary,
not frozen into the configuration snapshot.
Ordinary execution uses indexed ready definitions and resolved presence values.
It performs no configuration discovery, parsing, inheritance traversal, or
provenance analysis. Context cancellation remains cooperative at controlled
boundaries; do not add detached work to pretend synchronous ConfigKit loading is
instantly interruptible.
## Explicit Operator Inspection
Add an API consumers can use to implement targeted CLI commands or debug output
for effective configuration, contributing sources, and explanations. Retain
ConfigKit inspection state with the engine's configuration snapshot and reuse
its redaction and provenance facilities.
Inspection is explicitly invoked and uses that snapshot without reloading files,
opening resources, or contacting providers. It does not predict every runtime
override, reveal execution-time credential values, or replace prepared-run details
and execution observations. No provenance traversal, explanation generation, or
configuration-report rendering occurs during ordinary prompt runs.
Expose a read-only inspection view backed by retained ConfigKit state, with
targeted path queries, source/history information, and explicit redacted effective
and explanation output. Reuse suitable public ConfigKit inspection values rather
than mirroring them. Do not expose mutable runtime definitions, resource capabilities
merely for diagnostics, or request credentials. The scope is generic targeted
inspection, not just narrow profile/prompt origin summaries.
PromptKit ships no CLI and emits no automatic debug reports. Applications own
presentation, access control, and disclosure; redaction does not make all prompt
or configuration content safe to publish.
## Required Refactoring And Coordination
- Replace profile/prompt metadata prepasses, selected-file decoding, and per-call
scans with ConfigKit collection loading and immutable indexed definitions.
- Remove the custom profile inheritance resolver and error-driven repository
overlay stack. Do not retain an old-format compatibility engine alongside the
new semantics.
- Route catalog and programmatic backend definitions through ConfigKit; retire
the separate backend-manifest decoder and pre-handoff registration validation.
Retain backend endpoint, capacity, and provider-parameter domain validation on
the final effective values.
- Retain only needed discovery and message-resource adapters; remove direct YAML
dependency usage once no remaining owner needs it. Preserve JSON Schema,
runtime JSON-value ownership, domain validation, and execution collaborators.
- Adapt maintained catalogs and PromptKit-owned definitions to the agreed keyed
format and released inheritance semantics, including keyed backend assets.
Preserve maintained name commitments and the intended shipped backend settings.
Catalog source-policy enforcement belongs to catalog maintainers. Coordinate
compatible catalog releases before consuming the changed asset format.
Publish new keyed assets directly in the existing data-only catalog projects,
recording the format transition rather than maintaining duplicate legacy assets
or a permanent conversion adapter. Catalog maintainers select release/version
handling under their own policies; old consumers must not be assumed capable
of decoding the new shape. Pin the compatible released versions in PromptKit.
This document requests no immediate changes in those repositories.
- Keep configuration conversion, normalization, and inspection connected to
ConfigKit's lifecycle instead of reconstructing effective values/history after
loading. Translate once into execution-owned values with necessary copying.
- Update canonical formats, consumer guidance, GoDoc, internal inventory, examples,
architecture boundaries, dependencies, and OSS notices when implementation lands.
## Efficiency And Validation Expectations
Target linear or near-linear work relative to actual input, output, and retained
history, accounting separately for sorting, resource reads, and domain work.
Acquire/parse each accepted source occurrence once, avoid decoding intermediate
tiers, and index definition selection. Do not duplicate inheritance expansion or
copy complete accumulated collections for every added layer.
Materialized inherited data and provenance can exceed source size. Exercise real
catalog shapes within ConfigKit's budgets; do not claim unconditional O(n) runtime
or memory. Prefer clear ownership and straightforward indexes over global caches
or elaborate optimization. Operator inspection may perform work proportional to
the requested report; that work remains off the execution path.
Tests should use real ConfigKit with offline sources and synthetic provider
collaborators. PromptKit owns its adapter, domain, lifecycle, and execution tests,
not a duplicate ConfigKit parser/graph suite. Retire tests whose only purpose was
the removed loading machinery or source-policy enforcement. Exercise catalog
integration through the final assembled configuration and resulting behavior.
Preserve meaningful execution regressions.
Representative acceptance evidence must cover:
- Minimal local-backend construction and execution without elaborate configuration.
- Embedded defaults plus operator and programmatic contributions, including
accurate inherited relative-resource origins.
- Profile inheritance, changing model/backend, canceling old relationships, and
discarding unwanted settings with documented semantics.
- Explicit zero/false/null behavior reaching execution, construction failures,
independent snapshots, and unchanged substantive preparation/execution features.
- Targeted inspection with useful redacted origins and no provenance work during
ordinary runs.
Use focused scaling/allocation benchmarks for large collections, shared bases,
and many tiers where they expose repeated-work risks. Do not add timing-based
release gates or provider-network tests.
## Migration And Completion
Ship as an intentionally incompatible minor release with concrete migration
guidance. Cover keyed files, `base_profile` to `extends`, reserved-key behavior,
recursive maps, explicit null versus omission, changed error timing, programmatic
construction, backend layering in place of registration-only restrictions, and
explicit engine replacement. Explain that overriding a model
does not discard lower endpoint, reasoning, provider, or inheritance settings.
The end state is one shared configuration model with the old loading/resolution
machinery removed, useful operator inspection, and all substantive execution
workflows still available. The six previously open design choices have been
approved and incorporated into the owning sections above. This is not a
compatibility-preserving reinterpretation of the existing configuration engine.

25
docs/roadmap/deferred.md
View File

@@ -42,20 +42,21 @@ checkpoint equality or when a broader semantic identity design is selected.
### Eager Source Validation
**Reason for deferral:** Exact prompt and profile inspection may already
provide a sufficiently small validation surface. Experience from downstream
adoption should establish whether an engine-wide operation would add enough
value to justify its broader contract.
provide a sufficiently small validation surface alongside construction-time
definition validation. Experience from downstream adoption should establish
whether a separate operation would add enough value to justify its broader
contract.
Promptkit could provide an explicit offline operation that discovers and
structurally validates configured prompt, profile, and schema sources without
model generation. The normal `NewEngine` path would remain lazy.
Promptkit could provide an explicit offline operation that also checks schema
resources without model generation. `NewEngine` already validates effective
definitions and captures prompt content; schema resources remain
request-dependent.
An eager operation would need coherent handling for duplicate prompt IDs and
versions, strict YAML decoding, referenced content files, profile/backend
membership, schema syntax and transitive references, context cancellation,
and source-specific public errors. Credential declarations must remain
separate from credential values; checking current environment availability,
if supported at all, should be an explicit option and must not expose secrets.
Such an operation would need coherent handling for schema syntax and transitive
references, context cancellation, and source-specific public errors. Credential
declarations must remain separate from credential values; checking current
environment availability, if supported at all, should be an explicit option
and must not expose secrets.
Reconsider this idea after downstream use of `InspectPrompt`,
`InspectProfile`, and fixture-based preparation demonstrates a concrete gap.

642
docs/roadmap/implementation.md
View File

@@ -1,642 +0,0 @@
# ConfigKit Adoption Implementation Plan
Status: ready for staged implementation under the agreed configuration boundary.
This is a planning document; writing it does not implement the feature or
authorize publishing dependencies.
## Authority And Working Method
The [ConfigKit feature roadmap](configkit.md) owns the intended end state and
approved policy. Its intentional-breaking-change direction is authoritative:
preserve substantive prompting capabilities, not the old loader, signatures,
source wrappers, incidental error ordering, or configuration syntax.
The [configuration-boundary ADR](../adr/0001-configuration-boundary.md) records
the ownership decision: PromptKit interprets and validates the final effective
configuration supplied by ConfigKit. Presence/null information and effective
resource context are part of that handoff. Source history serves explicit
operator inspection, not validation or catalog enforcement. This applies to
tests as well as production code.
Implement the stages below in order, one stage per coding-agent prompt. Each
stage includes its tests and affected current-state documentation. Read
[development](../development.md) and the applicable policies before starting;
in particular, follow [agents](../policy/agents.md),
[architecture](../policy/architecture.md), [testing](../policy/testing.md),
[documentation](../policy/documentation.md), and [OSS](../policy/oss.md).
Catalog work also follows each catalog repository's own instructions and release
procedure. Do not turn this plan's internal organization into new public promises.
Begin each stage by inspecting the worktree and earlier stages. Commit completed
units under the agent policy. Keep stages buildable; temporary private wiring
needed to connect successive stages must be removed by Stage 11. Do not ship a
legacy mode, dual loader, or old-format conversion layer. Do not add prototype
applications or a generic configuration framework of our own.
Planning baselines:
- PromptKit runtime v0.11.2, commit
`12d6d80024e80c552d122a076c22429bf6f2f6c2`; approved roadmap at `521291d`.
- ConfigKit v0.5.0, commit
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`, module
`gitea.maximumdirect.net/eric/ConfigKit`.
- Both catalog dependencies currently select `/v2` at `v2.20260926.0`.
Use ConfigKit v0.5.0 for this work. Do not silently upgrade to a different
contract. The catalog asset migration needs coordinated releases, not a committed
workspace or local replacement. Changing the backend asset contract is broader
than the catalog profile-data compatibility exception: migrate the catalogs to
`/v3` with `v3.YYYYMMDD.N` tags, preserving their date/revision convention and
updating their own release documentation accordingly. Select exact dates and
unused revisions at publication; record versions and full commits consumed.
Publishing any repository requires separate authority.
## Implementation Decisions
### Construction Surface
Keep `NewEngine(Config, ...Option)` and the execution/injection surface where it
still serves its substantive purpose. Replace profile/prompt source options with
these fields and helpers; this is an intentional API break:
```go
type ConfigurationSources struct {
Underlays []configkit.Underlay
Main *configkit.Input
}
// Add Sources; remove PromptDir and ProfileDir.
// Retain SchemaDir, Timeout, HTTPClient, and TransientRetries.
type Config struct { /* existing non-definition settings; Sources ConfigurationSources */ }
func DirectoryInputs(directory string) ([]configkit.Input, error)
func FilesystemInputs(label string, filesystem fs.FS, root string) ([]configkit.Input, error)
```
The pointer distinguishes omitted main input without inspecting ConfigKit's
opaque Source. Nil main becomes `configkit.NoSource()`; a supplied invalid Input
is not silently ignored. Prepend two private underlays: external catalogs, then
PromptKit-owned definitions. Append caller underlays in their supplied order;
main is last and keeps its ConfigKit meaning. Imports follow each Input's flag.
Do not expose named overlays, CLI overrides, or VersionRule through another
PromptKit abstraction in this feature.
Show the usual application categories as caller underlays ordered fallback,
operator, programmatic, with main omitted. For a single highest-priority document,
main is also useful. These are explicit selections, not directory-name heuristics
or a second automatic precedence algorithm. Each directory helper returns inputs
for one strict-composition group, recursively finding `.yaml` and `.yml` files;
sorted discovery makes diagnostics reproducible, not one file higher priority.
An explicitly selected missing/unreadable root fails; an empty directory supplies
no contributions. Helpers set imports false; callers may enable them explicitly.
Use one ConfigKit filesystem namespace per helper invocation and root resource
contexts at the supplied directory. Single files use ConfigKit directly.
Remove `WithPromptFS`, `WithPromptFile`, `WithProfileFS`, `WithProfileFile`,
`WithFallbackProfileFS`, `WithProfiles`, and `WithBackend` at public cutover.
Retain schema source options and runtime collaborator injection. Backend
configuration moves into ConfigKit inputs as specified below. Do not require any
prompts to exist merely to construct an engine for profile inspection.
Programmatic configuration uses labeled `configkit.Mapping` inputs. Replace the
old zero-sentinel `Profile`/`OpenAICompatibleProfileConfig` pair with one small
`ProfileConfig` convenience and a source builder:
```go
type ProfileConfig struct {
Extends, Backend, Endpoint, Model *string
Temperature, TopP *float64
MaxTokens, TimeoutSeconds *int
ServiceTier, ReasoningEffort, APIKeyEnv *string
APIKeyRequired *bool
ExtraParams map[string]any
}
func ProfileSource(label string, profiles map[string]ProfileConfig) configkit.Source
```
Nil convenience fields mean omitted; nonnil pointers preserve zero/false/empty.
Nil ExtraParams omits the map; a nonnil empty map supplies an empty mapping, not a
clear operation. Use raw Mapping for explicit null, prompt definitions, or richer
configuration. Do not introduce a public generic optional-value type just for
this helper. Convert fields explicitly to ConfigKit-supported built-in values,
then call Mapping for snapshot ownership and load-time errors. Do not marshal
through YAML or infer absence from a value. Helpers encode values; domain
validation occurs on the final configuration at handoff.
Replace the registration-only Backend value with a presence-aware convenience:
```go
type BackendConfig struct {
Endpoint, APIKeyEnv *string
ConcurrencyLimit, QueueCapacity *int
ExtraParams map[string]any
}
func BackendSource(label string, backends map[string]BackendConfig) configkit.Source
func LocalBackend(endpoint string, concurrencyLimit int) BackendConfig
```
Use the same omission/pointer/map conventions as ProfileSource. LocalBackend
supplies endpoint and concurrency pointers and leaves the remaining fields
omitted; callers choose the ID as the map key. Neither helper validates settings
before ConfigKit processes them. For example, place BackendSource with
`map[string]BackendConfig{"local": LocalBackend(endpoint, 4)}` in an application
underlay. Raw Mapping supports explicit null and combined backend/profile/prompt
documents. Remove the old Backend registration value and its reservation rules;
retain useful built-in ID constants. All backend contributions follow the same
strict-composition and overlay rules as other configuration.
### Typed Configuration And Execution Values
Add `internal/configuration` as the cohesive ConfigKit adapter. Its private
document DTO has `backends: map[string]backendConfig`,
`profiles: map[string]profileConfig`, and
`prompts: map[string]map[string]promptConfig`. Collection entries are mappings,
not nullable definition pointers. No duplicate identity fields; ConfigKit consumes
`extends`, so no DTO field or second resolver handles it. Use `configkit.Keys`,
`Key`, and `Index` for semantic paths, never split IDs on dots.
Profile DTO fields match ProfileConfig's YAML spellings (`backend`,
`reasoning_effort`, etc.). Nullable pointers plus ConfigKit presence metadata
distinguish absent, value, and explicit null. Store that distinction in small
private execution-owned fields or presence bits, not retained metadata queries.
Runtime/domain packages must not import ConfigKit. Do not duplicate its tree,
merge engine, inheritance graph, diagnostics, or presence framework.
Backend DTO fields match BackendConfig using `endpoint`, `api_key_env`,
`concurrency_limit`, `queue_capacity`, and `extra_params`. Validate the final
endpoint as required, absolute HTTP(S) with the existing URL rules. An absent or
null API-key environment name means no lookup; a supplied name follows existing
normalization and naming rules. Absent/null concurrency means zero (unlimited);
supplied values must be nonnegative. Absent/null queue capacity selects the
existing default of 1024 for positive concurrency and no queue for unlimited
concurrency. Explicit queue capacity, including zero, requires positive
concurrency and must satisfy existing nonnegative/overflow checks. Null provider
maps mean no configured extra parameters; entries may contain JSON null.
Required endpoint failures, unknown fields, and domain errors apply to final
definitions, including unused ones. Registry construction consumes these final
normalized values; it does not replay registration or source-reservation checks.
Use ConfigKit's Resolve/Validate callbacks only on its effective post-layering,
post-inheritance typed values. These public lifecycle hooks implement the domain
handoff and record normalization; they are not hooks for inspecting earlier
contributions. Final presence lookup and resource-context lookup are sufficient
for interpretation. History/Explain/HistoricalPaths are operator inspection
facilities, not inputs to construction-time policy checks.
Use the existing prompt fields under each keyed version, without `id`/`version`.
Use pointers for `content`, `content_file`, and optional clearable strings;
preserve existing input, cache-control, output-contract, and role rules.
`messages` must be nonempty; each message has exactly one nonnull content source.
An explicitly supplied empty inline string is a valid template. Null content
removes that selection; null content_file removes its reference. Output remains a
required mapping with valid format and validation mode. Optional description,
session template, default profile, cache control, inputs, and schema path can be
cleared where their domain permits absence. Null required role/format/mode or a
null whole definition is invalid. Scalar `required` and `repair_attempts` retain
their false/zero defaults and are not nullable. Null inputs means no declarations.
Unknown fields are ConfigKit errors, including old `id`, `version`, and
`base_profile`. Prompt versions must be string mapping keys.
Literal definition keys remain exact; require nonblank identity but do not
normalize/rekey maps after ConfigKit has resolved relationships. Update selection
to use exact nonblank IDs/versions rather than silently trimming them into a
different identity. Scalar normalization such as endpoint trimming is recorded
through ConfigKit's resolution callbacks when
mutating its typed configuration. Message roles and cache control are normalized
on execution-owned copies instead: ConfigKit v0.5.0 treats sequence mutations as
whole-value replacements, losing unchanged content_file origins. Inspection
retains the effective input message metadata and its source context. Field
references to definition IDs remain exact as well.
Profile field rules after configuration overlay/inheritance:
| Fields | Absent | Explicit value | Explicit null |
| --- | --- | --- | --- |
| `backend` | No selected backend | Exact registered ID; blank means none | No backend |
| `endpoint` | Selected backend endpoint, if any | Normalize/validate nonblank URL; empty clears | Clear, do not recover backend endpoint |
| `model` | Unspecified | Nonblank supplied model; empty clears | Unspecified |
| `temperature`, `top_p`, `max_tokens` | Unspecified | Validate existing bounds; zero is explicit and sent | Suppress outbound control |
| `reasoning_effort`, `service_tier` | Unspecified | Existing normalization; blank suppresses | Suppress outbound control |
| `timeout_seconds` | Framework 600-second baseline | Nonnegative; zero disables generation deadline | Reset to framework baseline, not lower profile timeout |
| `api_key_env` | Selected backend's optional lookup name | Trimmed value; blank disables lookup | Disable lookup, do not recover backend name |
| `api_key_required` | False | Explicit boolean, including false overriding true | False |
| `extra_params` | Backend defaults | ConfigKit-composed profile map, recursively overlaid on backend map | Suppress entire configured provider map |
Profile API-key environment names follow the existing lookup contract, including
host-supported non-shell-identifier names. Do not extend the backend's naming
regular expression to profiles. Trimming, explicit clearing, optional lookup,
and direct request-key precedence remain unchanged.
Backend registry defaults are applied once during snapshot construction, after
effective profile inheritance. Keep them out of the inheritance input: a parent's
derived endpoint/key/defaults must not follow a child that changes backend.
Provider map null entries are literal JSON null; empty mappings retain children.
For backend-to-profile provider-map overlay, use a narrow JSON-value merge helper
with the same recursive-map/value-replacement semantics; it is not a configuration
loader. Both backend and profile configuration maps have already passed through
ConfigKit's reserved-key semantics; only request-level maps bypass that lifecycle.
Record any defaults/normalizations written into the typed configuration via
ConfigKit Resolve, with dependencies on relevant configuration paths. Do not
project runtime collaborators or credentials into the configuration tree just to
report them. Inspection includes effective backend configuration. It describes
resolved configuration, while InspectProfile and preparation describe
backend-completed execution settings. Make this boundary
explicit rather than inventing synthetic provenance for registry defaults.
Retain APIKeyRequired's direct-key workflow: true suppresses a backend-derived
environment source. An explicitly configured profile environment source, or a
request environment override, remains usable; false restores ordinary optional
credential semantics. No credential values belong in the DTO, inspection, or
construction-time environment resolution. Raw profile `api_key` remains invalid.
Validate all effective supplied values and referenced backend IDs at construction.
Do not require a complete model/endpoint there or reject unused partial profiles.
Do not require a prompt default_profile to be runnable before a request can
override it. Final target completeness is checked after runtime overrides.
InspectProfile can report a partial target without credential or provider access.
Keep request override contracts otherwise unchanged, including complete nonempty
runtime ExtraParams replacement and reasoning clearing. A runtime literal
`extra_params["extends"]` remains ordinary provider data. Do not expand this work
into a redesign of every request override.
### Snapshot And Inspection Design
The loader returns indexed, execution-owned definitions plus a private retained
ConfigKit result. An adapter may implement the runner's narrow lookup interfaces
over maps during migration; no repository chain, fallback-on-error, or per-call
configuration load survives. Preserve defensive ownership across injected clients
and public results. Cache prompt definition hashes at construction; hash effective
captured content, not a stale resource path. Hash bytes need not match the old
configuration format. Preserve rendered-prompt hashing semantics.
Resolve each effective content_file using its exact field origin and ConfigKit
SourceContext. Join its relative name to BaseDir inside the returned rooted FS;
reject absolute/outside-root paths and close every opened handle. Preserve path
text, including meaningful whitespace, after separately checking nonblankness.
Do not choose the child's source for an inherited reference. If there is no unique
contributing source with context, fail with an actionable path/source error; the
caller can supply an explicit Input filesystem context or override the field.
Retain no resource capability for the purpose of public diagnostics.
Capture message text and parse message/session template syntax at construction
with the same function names/options used for runtime rendering. Do not invoke
input helpers or require Vars/artifacts while parsing. Reuse compiled templates
where practical, binding request-specific input functions on safe per-call clones;
never mutate a shared template function closure with request state. Runtime
missing-variable/input failures remain runtime errors.
Expose a concrete read-only facade value:
```go
func (e *Engine) InspectConfiguration() (ConfigurationInspection, error)
func (v ConfigurationInspection) Metadata() configkit.Metadata
func (v ConfigurationInspection) Sources() []configkit.SourceInfo
func (v ConfigurationInspection) Diagnostics() []configkit.Diagnostic
func (v ConfigurationInspection) Explain(path configkit.Path) (configkit.Explanation, bool)
func (v ConfigurationInspection) WriteEffective(w io.Writer) error
func (v ConfigurationInspection) WriteExplanation(w io.Writer, path configkit.Path) error
```
Store an unexported narrow implementation reference; do not expose Result.Config,
ConfigKit SourceContext, mutable runtime values, or internal types. Delegate
queries and redaction to the retained result. Nil Engine returns ErrInvalidConfig;
the zero inspection view has empty query results and writer operations return
ErrInvalidConfig. Writer errors remain inspectable. Mark credential-shaped dynamic
keys sensitive using Definition.Secret; the existing secret-key vocabulary can
serve as a redaction predicate, not a source-rejection rule. Do not redact
`api_key_env` names as credential values. This is not a
promise that arbitrary prompt/provider content is safe to disclose.
Keep the runner independent of this view. Inspection calls perform no discovery,
resource reads, environment credential lookup, or provider operations. Schemas,
artifacts, preparation ownership, generation admission, retry/repair, observations,
and cancellation remain at their established runtime boundaries.
## Stage 1 — Add The Dependency And Typed Loading Core
**Completed:** 2026-09-30.
Scope: module metadata, `internal/configuration` DTOs, real ConfigKit loading,
and its focused tests. Leave engine assembly on the old path for this stage.
1. Add ConfigKit v0.5.0; update go.sum and factual OSS notices for it and its
selected transitive dependencies. Preserve project licenses and the Go floor.
2. Add keyed backend/profile/prompt DTOs and a private load entry point accepting
ConfigKit Inputs and underlays. Call LoadLayered once, with no VersionRule or
secret resolution bindings for API credentials. Use callback-scoped
Resolve/Validate correctly.
3. Implement effective presence extraction and the field rules above. Validate
final domain values through existing helpers, resolving backend references
against the final backend map; profiles have no final target requirement yet.
Record normalization/default mutations, not arbitrary DTO bookkeeping fields.
4. Preserve `*configkit.LoadError` through wrapping. Domain callback messages
identify paths/problems without interpolating raw values or credentials.
5. Add representative real-loader tests: keyed literal IDs, strict group conflict,
recursive cross-tier overlay, a shared base with explicit zero/null/false, an
unused invalid effective definition, and preservation of a ConfigKit load
error. Cover an invalid lower-layer domain value replaced by a valid final
value: construction succeeds. Do not reproduce ConfigKit's parser/inheritance
matrix or add preflight/source-policy tests.
Exit: focused tests and `go test ./...` pass; old engine behavior is not presented
as migrated. Internal inventory documents only the loader now actually present.
## Stage 2 — Add Source Discovery And Prompt Resource Capture
**Completed:** 2026-09-30.
Scope: source selection helpers and construction-time prompt materialization.
1. Implement private directory/FS discovery behind the chosen helper signatures;
wrap discovered paths as ConfigKit sources, retaining explicit root context.
Reuse useful filecatalog path handling without carrying profile metadata scans.
2. Materialize effective prompt message text using field origins and passive
contexts. Move reusable prompt/input/cache/output validation into the loader
or shared domain owner; do not copy entire legacy parsers.
3. Build immutable prompt indexes by exact ID/version and an ambiguity indicator
for omitted versions. Capture declared-input inspection data and prompt hashes.
4. Validate static template syntax using the renderer's grammar/function setup.
Refactor that setup into one private owner if needed; do not render at load.
5. Test native and embedded sources, imported/inherited relative content from a
different directory, explicit mapping context, contextless failure, handle
closure, malformed static syntax, and resource mutation after capture. Use
minimal fixtures, sharing scenarios where the same adapter is exercised.
Exit: loader creates usable frozen prompt definitions without request data.
`go test ./internal/configuration ./internal/prompt ./internal/promptdef` passes;
existing execution tests still work through the old assembly until cutover.
## Stage 3 — Make Execution Settings Presence-Aware
**Completed:** 2026-09-30.
Scope: domain settings, target resolution, wire-presence propagation, profile
inspection. No new source discovery in execution packages.
1. Build the immutable backend registry from the final loaded backend map. Add
execution-owned presence/suppression state and construct indexed profiles
from loader output plus that registry. Apply registry-derived
values after inheritance, using the field table above.
2. Replace scalar-zero/blank merge assumptions in target resolution. Configuration
explicit zero reaches injected clients and the built-in request body; null does
not resurrect a provider setting. Extend existing TargetPresence plumbing
through preparation/results as necessary rather than creating a parallel path.
3. Preserve request-level override precedence, API-key availability behavior,
endpoint-only profiles, backend identity/capacity, and partial-profile runtime
completion. Move final completeness checks after overrides. InspectProfile
returns partial settings without treating missing model/endpoint as load errors.
4. Test explicit numeric zero versus omission, map overlay and null, backend
change without derived defaults leaking from parent, timeout null/zero,
credential-source suppression, and completion through request overrides.
Include one built-in-client wire test, not just an injected-client assertion.
Exit: focused domain/usecase/client tests pass. Any temporary adaptation for the
old assembly is private, clearly limited to reaching Stage 7, and does not become
a compatibility promise. Update affected target/presence GoDoc with this change.
## Stage 4 — Migrate The OpenRouter Catalog Assets
**Completed:** 2026-09-30. Published OpenRouter catalog `v3.20260930.0`
from `f17d208cb7ad1fb043cda2266cd3011d11b40aa0`.
Scope: `../promptkit-backend-openrouter` only. Catalog maintainers own source and
publishing checks here; PromptKit consumes its configuration through ConfigKit.
1. Read that repository's instructions; preserve unrelated work. Convert each
profile asset to `profiles: {literal-id: {settings...}}`, preserving inventory
and values, changing `base_profile` to `extends`, and removing embedded ID.
Keep per-profile files and the exported FS/Root package; no dependency on
PromptKit and no second legacy asset tree.
2. Audit zero/empty values that previously meant unspecified. Remove those when
omission was intended; retain intentional explicit values. Review any dynamic
provider `extends` keys rather than silently changing outbound payloads.
3. Replace `catalog/backend.json` with `catalog/backend.yml` containing a
`backends: {openrouter: {settings...}}` mapping. Preserve endpoint, API-key
environment name, concurrency, queue, and extra-parameter values; remove the
duplicate id and old manifest schema_version. The file is an ordinary ConfigKit
input, not a separately decoded manifest. Update embedded assets, module path
and imports to `/v3`, tests, and catalog format/release documentation, with an
incompatible-format migration note. Keep FS/Root as the minimal export surface.
Catalog-local publishing checks stay in that repository. No provider probes.
4. Validate under its procedure and commit. If publication is authorized, publish
the next unused `v3.YYYYMMDD.N`; otherwise report the exact commit awaiting
publication. The date/revision is selected then, not guessed in this plan.
Exit: one reviewed migrated catalog commit; existing immutable tags still serve
old consumers. The new major module path identifies the changed asset contract;
old consumers retain the published `/v2` assets.
## Stage 5 — Migrate The Rakestrawhome Catalog Assets
**Completed:** 2026-09-30. Published Rakestrawhome catalog `v3.20260930.0`
from `993443d88c451901f4c917fa00c204fd3b54d28d`.
Scope: `../promptkit-backend-rakestrawhome` only. Apply the same keyed-format,
presence, notice, validation, and publication treatment as Stage 4 to its smaller
inventory, including the `/v3` transition and keyed `backends.rakestrawhome`
definition in `catalog/backend.yml`. Preserve backend endpoint, credentials
metadata, and capacity policy.
Do not couple its date/revision to OpenRouter's release.
Exit: both compatible catalog versions are published and ordinarily resolvable
before Stage 6 pins them. If permission or publication is unavailable, stop at
this external dependency checkpoint; do not commit a local replace, guessed
version, or legacy-to-new runtime converter to bypass it.
## Stage 6 — Integrate Catalog Sources And Maintained Profiles
**Completed:** 2026-09-30.
Scope: `internal/catalog`, maintained definitions, catalog integration tests,
dependency pins/notices. Keep this separate from public API migration.
1. Pin the published compatible catalog versions and exact attribution commits.
Update imports to `/v3`. Replace manifest decoding with ConfigKit file Inputs.
2. Supply backend and profile assets as ConfigKit Inputs in the first
underlay. Supply PromptKit-owned keyed Mapping definitions as the next group:
standard extends the selected catalog's current maintained base; auto retains
its current direct model/backend definition. Preserve both literal names.
3. Retire source-layout allowlists, catalog preflight, source ownership/reserved
prefix checks, connection-field prohibitions, and secret-key source rejection.
The adapter selects known asset entry points and discovers configuration files;
ConfigKit acquires their contents. Domain validation sees only the final
assembled configuration. Consumers may overlay catalog backends, profiles,
and their bases through the same configuration model.
4. Test real imported assets offline with the new loader, maintained-name
resolution, consumer override of a maintained base/backend, and rejection of
an invalid final backend. Test a replaced lower-layer domain error succeeds.
Do not independently validate catalog-only intermediate configurations, parse
raw assets in PromptKit tests, or audit origins to establish ownership.
Exit: catalog and owned definitions participate in the new keyed configuration
model without a legacy profile parser on that path. Catalog integration tests
pass against released modules with `GOWORK=off` and no replacements.
## Stage 7 — Cut Over Engine Assembly And Public Construction
**Completed:** 2026-09-30.
Scope: `engine.go`, configuration facade/helpers, `internal/engine`, facade
adapters, and migration of their direct test setup.
1. Introduce the Construction Surface exactly as specified above. Remove the
superseded source/profile options and convenience types rather than keeping
deprecated pass-through wrappers. Keep intentional runtime injection APIs.
2. Load the combined backend/profile/prompt configuration, construct its runtime
registry and indexed snapshot, then assemble renderer, validator, capacity
manager, and runner. Return no Engine on construction failure.
Translate construction errors to ErrInvalidConfig while preserving LoadError
via errors.As. Resource errors include semantic path and safe source context.
3. Connect indexed prompt/profile lookups to preparation and inspection. Missing
selections keep meaningful not-found errors; prompt version ambiguity stays a
selection error. Old parse/load error categories need not be manufactured for
errors now detected at construction.
4. Move root test fixtures/setup and maintained example construction to ConfigKit
sources, keyed definitions, ProfileSource, and BackendSource so all consumers
still compile. Update local-backend examples to choose their ID as a map key.
Update expected timing only for the deliberate eager-validation change. Keep
behavioral assertions for prompting, overrides, and errors.
5. Add public tests for a minimal custom local backend and programmatic profile,
plus embedded defaults/operator/programmatic tiers. No elaborate consumer
setup should be needed merely to send one prompt.
Exit: the public Engine uses only the new loading path. Root/package tests pass;
all package/example sources compile. Update GoDoc and consumer construction
guidance in this stage so removed options are not presented as usable.
## Stage 8 — Complete Frozen Rendering And Lifecycle Integration
**Completed:** 2026-09-30.
Scope: renderer/runtime consumption of the snapshot, ownership, and lifecycle
regressions. Stage 2 already owns acquisition and static syntax validation.
1. Use captured text and parsed templates without per-run configuration reads,
parsing, or inheritance. Bind input helpers per invocation safely; keep missing
request data errors and direct-session template bypass behavior.
2. Verify preparation and execution still handle artifacts and schema compilation
at invocation time, honor output overrides, freeze prepared resources, and
keep prepared handles bound to their engine. Credentials remain at runtime.
3. Test an old engine after configuration/resource edits and a newly constructed
engine observing the edits. Test concurrent renders with different Vars/input
helpers and no data leakage. Use an FS that rejects reads after construction
to demonstrate no configuration/content-file reopening during ordinary runs;
do not incorrectly prohibit request artifact/schema reads.
4. Preserve appended-message ordering, structural repair, retries, admission,
cancellation, observers, and result metadata through existing regression
tests; add only distinct missing cross-boundary cases.
Exit: focused race tests pass for assembled rendering/preparation and lifecycle.
No retained callbacks capture ConfigKit resolution contexts or request state.
## Stage 9 — Expose Explicit Configuration Inspection
**Completed:** 2026-09-30.
Scope: the read-only facade and retained ConfigKit inspection adapter.
1. Implement ConfigurationInspection and its specified methods without exposing
Result.Config or filesystem capabilities. Retain a private narrow view of the
ConfigKit result; no regeneration of origins from runtime values.
2. Ensure effective/explanation reports use ConfigKit redaction, including the
selected sensitivity predicate. Preserve safe value-free domain diagnostics.
3. Test a targeted literal-key path with layered/inherited history, detached
returned slices/records, writer failures, zero/nil behavior, and redacted secret
shaped provider values. Demonstrate inspection still works after source removal
and makes no filesystem/provider calls.
4. Document construction inspection versus InspectProfile and prepared details.
Consumers choose report disclosure; do not add CLI commands or automatic logs.
Exit: public inspection tests pass, including concurrent inspection/execution.
The runner has no reference to the inspector or ConfigKit metadata types.
## Stage 10 — Migrate Examples And Canonical Format Guidance
**Completed:** 2026-09-30.
Scope: maintained examples and user-facing documentation.
1. Complete the preparation, execution, and observation example migration begun
at public cutover. Preserve their distinct existing semantic assertions; extend
the preparation example with a small explicit inspection query if useful.
2. Update `docs/formats.md` to own the new keyed syntax, source-group semantics,
inheritance/presence/null table, eager validation, snapshot lifetime, and
resource-context rules. Link ConfigKit for generic grammar; do not copy its
complete parser/inheritance contract.
3. Update README and `docs/consumers/pkg-promptkit.md` with minimal local-backend,
file/embedded source, programmatic Mapping/ProfileSource, explicit inspection,
and engine-replacement workflows. Explain main versus ordered underlays and
backend layering in place of registration-only restrictions.
4. Add concise old-to-new migration guidance in the consumer guide: removed API
options, exact IDs, `base_profile` to `extends`, zero/false/null, recursive maps,
no clear via `{}`, reserved `extends` runtime escape, failure timing, and the
fact that changing a model/backend does not clear unrelated explicit fields.
Remove old automatic reload claims. Release notes later link to this owner.
Exit: all three offline examples run with their semantic checks; documentation
links and snippets match the implemented API. No unreleased version is tagged or
published, and no speculative release-note version is invented.
## Stage 11 — Remove Superseded Machinery And Consolidate Tests
**Completed:** 2026-09-30.
Scope: dead loading/resolution code and its obsolete tests, module cleanup, and
internal architectural documentation.
1. Delete lazy prompt/profile metadata prepasses, filesystem repositories,
fallback-on-error overlays, custom base-chain resolver/depth cap, zero-sentinel
profile merging, and temporary stage adapters once no caller needs them.
Remove the separate backend-manifest decoder, source-policy audits, and
pre-handoff backend registration validation. Keep final-value domain checks.
Retain narrow lookup interfaces only where they still simplify real consumers.
2. Retain useful filecatalog/schema utilities and JSON-value ownership/validation.
Remove direct yaml.v3 use and dependency if no remaining owner requires it;
do not claim all YAML dependencies disappear when ConfigKit still uses YAML.
3. Move meaningful regressions from deleted package tests to their new owning
boundary. Retire tests about old internals, repeated parser coverage,
source-policy enforcement, or superseded semantics. Do not replace lost runtime
protection with loader-only tests. Keep private assembly checks separate from
external facade tests.
4. Update `docs/internal/overview.md`, `docs/internal/sources.md`, relevant renderer
and engine documents, and the implemented-structure paragraphs in architecture
policy. Describe the new package/dependency boundary without turning names or
chosen adapters into permanent invariants. Update OSS notices with module tidy.
Exit: no legacy source path remains, no internal package imports the public root,
runtime/domain packages do not import ConfigKit, and `go test ./...` passes with
the reduced implementation/test inventory.
## Stage 12 — Validate Scale And The Complete Consumer Workflow
**Completed:** 2026-09-30.
Scope: final integration evidence and high-confidence simplification, not another
feature expansion or release gate.
1. Add focused benchmarks for construction with many independent definitions,
shared inherited bases, and several tiers; add an already-constructed selection
benchmark. Report allocations and input/output shape; inherited materialized
output/history may dominate. Do not impose timing thresholds or new caps.
2. Inspect for repeated full-map copies, per-run provenance calls, rescans,
repeated inheritance, and unnecessary double parsing. Fix demonstrated local
inefficiencies with straightforward ownership/indexing; no global cache.
3. Run the repository's existing complete local check script once on the final
candidate, including race tests and offline examples. Investigate failures;
do not weaken meaningful tests to get a green result. Stage intended new
files before the script's tracked-file checks.
4. Review the feature-roadmap capability list against actual public workflows:
independent inspection, partial profile completion, optional credentials,
source contexts, artifacts/schemas, prepared ownership, backend capacity,
overrides, appended messages, repair/retries, observations, and hashes.
5. Report completed stages, selected catalog versions, validation evidence, and
remaining issues. Minor-release documentation/version selection and roadmap
retirement belong to the separately authorized release preparation workflow.
Exit: the feature roadmap's end state is demonstrably implemented, with one
configuration lifecycle, explicit operator inspection, no legacy loader, and
no newly invented provider checks or human release gates.
The catalog-validation placement question is resolved by the configuration-boundary
ADR. No design question remains open here. Catalog publication authority and
ordinary module availability remain operational checkpoints for the catalog stages.

126
docs/roadmap/remediation.md
View File

@@ -1,126 +0,0 @@
# ConfigKit Adoption Review Findings
Review date: 2026-10-03. Reviewed implementation commits `2b847ac` through
`66c55b7` against the [feature roadmap](configkit.md),
[implementation plan](implementation.md), and accepted
[configuration-boundary ADR](../adr/0001-configuration-boundary.md).
## Summary
The principal architecture is in place: one ConfigKit load assembles backend,
profile, and prompt configuration; PromptKit validates effective values, captures
resources/templates, and builds execution-owned indexes. Catalog preflight and
legacy profile resolution are removed. Operator inspection delegates to the
retained ConfigKit result separately from ordinary execution.
Both findings have been addressed. The original review evidence is retained
below; the resolution section records the changes and regression coverage.
## 1. P2 — Message Normalization Loses Content-File Resource Origins
Location: `internal/configuration/prompts.go`, particularly `resolvePrompts`
lines 69–99 and `readContent`'s effective-origin lookup.
When a role or cache-control value needs normalization, `resolvePrompts` records
a normalization operation for the entire `messages` sequence in addition to the
field-level operations. The resulting metadata no longer provides the source
location needed by `readContent` for an unchanged `content_file`. Construction
then rejects an otherwise valid prompt.
Confirmed through the public API using FilesystemInputs over an fs.FS containing
this document and a readable `body.txt`:
```yaml
prompts:
p:
v1:
messages:
- role: " USER "
content_file: body.txt
output:
format: text
validation_mode: none
```
NewEngine fails with
`/prompts/p/v1/messages/0/content_file: content_file needs one contributing source with a filesystem context`.
Changing only the role to `user` succeeds. Role trimming/lowercasing and
source-relative message capture are both intended capabilities; requiring
already-normalized metadata is an accidental restriction.
Required outcome:
- Preserve ConfigKit's effective resource context while normalizing neighboring
message metadata. Review the broad sequence normalization record and how the
final input origin is retained across domain normalization.
- Use effective handoff metadata/context, not historical traversal, raw-source
inspection, source rereading, or an invented fallback directory.
- Add focused regressions combining file-backed content with role and
cache-control normalization, including inherited/imported content. Existing
tests cover normalization and resource capture separately but miss this
interaction. Verify inspection remains accurate for untouched content fields.
## 2. P2 — Profiles Gain An Unrequested Environment-Name Restriction
Location: `internal/configuration/load.go`, profile validation at lines 278–279,
using the package's `envName` regular expression.
Profile `api_key_env` values now must match `[A-Za-z_][A-Za-z0-9_]*`. The previous
profile path accepted environment lookup names without this shell-identifier
restriction; the regular expression previously governed backend registration.
The refactor extends it to profiles even though arbitrary environment names
supported by the host remain usable through Go's environment lookup and the
request override path.
Confirmed through NewEngine with an otherwise valid endpoint/model profile:
`api_key_env: NORMAL_KEY` succeeds, while `api_key_env: service.api-key` fails
construction. The latter is a usable environment name on the reviewed Linux
platform. This restriction is unrelated to configuration layering, ConfigKit
typing, or optional credential availability.
The implementation plan's reference to existing naming rules is not evidence
of an express user decision to introduce this new profile restriction. The
requested compatibility break permits adopting ConfigKit semantics; it does
not itself authorize additional unrelated name constraints.
Required outcome:
- Remove the newly introduced profile shell-identifier check, preserving the
established trimming, explicit clearing, optional environment lookup, and
direct-key precedence behavior.
- Add a public-path regression for a supported non-shell-identifier environment
name and for its unset/empty optional behavior, with controlled test environment
state and a synthetic credential. Keep any prerequisite validation limited to
actual supported environment lookup requirements.
- Correct the plan's profile-field guidance so later work does not reintroduce
this restriction. Treat any broader change to existing backend naming policy
separately rather than silently extending or weakening it in this fix.
## Validation Performed
- `go test ./...` passed on Go 1.26.8.
- `GOTOOLCHAIN=go1.25.5 go test ./...` passed on the declared minimum toolchain.
- `go test -race ./...` passed.
- Separate offline public-API reproductions confirmed both findings. Missing
message/session variables also remained runtime errors in targeted checks.
The passing suite does not cover the two failing cases above. No provider
requests were made. Beyond these findings, the reviewed implementation did not
show another actionable ownership-boundary violation or strengthened guarantee.
## Resolution
1. Message role/cache-control normalization now occurs on execution-owned copies
during materialization. ConfigKit's captured message sequence and its origins
remain intact. The imported/inherited content regression now includes role
and cache-control normalization and asserts the unchanged field's source in
inspection metadata. This also removes duplicate cache-control normalization.
2. Removed the profile environment-name regular expression check, leaving backend
validation unchanged. A public engine/HTTP-transport regression covers a
non-shell environment name, unset/empty/whitespace values, explicit clearing,
trimming, and direct-key precedence. The implementation plan now states this
distinction explicitly.
Both regressions failed against the reviewed implementation before the fixes.
After remediation, the full ordinary and race-enabled suites, affected packages
on Go 1.25.5, `go vet ./...`, and documentation/whitespace checks passed.