promptkit/docs/roadmap/configkit-assessment.md

# ConfigKit Adoption Assessment

Status: proposed for discussion, not an approved feature specification or an
implementation plan. No runtime changes are part of this assessment.

## Goal And Recommendation

Use ConfigKit to standardize PromptKit's configuration semantics and simplify
its loading machinery, improving ConfigKit where a reusable capability is
justified. Preserve substantive consumer capabilities, not historical loading
sequences, file syntax, or Go signatures. Document intentional breaking changes
and their migration paths.

Recommend adoption through an eager, immutable definition-loading boundary.
ConfigKit should own YAML interpretation, strict typed decoding, presence, and
source diagnostics. PromptKit should own definition identity, named profile
relationships, LLM-specific rules, and execution. Do not retain the lazy loading
architecture merely to minimize migration, or introduce a second generic
configuration framework inside PromptKit.

The expected benefits are consistent configuration semantics for applications
using both libraries and removal of repeated discovery, partial metadata parsing,
lazy error arbitration, and repeated profile-chain resolution. Some of those
savings come from the loading redesign rather than ConfigKit itself. ConfigKit
adds ongoing value by owning the generic decoding behavior and its tests.

## Evidence And Scope

Reviewed local baselines on 2026-09-29:

- PromptKit: `12d6d80024e80c552d122a076c22429bf6f2f6c2` (v0.11.2).
- ConfigKit: `67c03690181c5af8dde6be325abac241dae6732f` (v0.4.2).

PromptKit evidence: [consumer workflows](../consumers/pkg-promptkit.md),
[file formats](../formats.md), [source implementation](../internal/sources.md),
[runner](../internal/runner.md), and the implementation owners identified below.
ConfigKit evidence: its reviewed `docs/config.md`, `docs/roadmap/deferred.md`,
`lifecycle.go`, `source.go`, and ADR 0001 in the sibling checkout. That checkout
is review context, not a proposed build dependency or required directory layout.

ConfigKit currently exposes `Load[T]` over file, byte, reader, and defaults-only
sources. It supports strict decoding, presence/origin records, defaults,
recorded resolution, semantic validation, explicit imports, one selected
filesystem overlay, and secret handling. Its parser and generic conversion
machinery are private. It has no public arbitrary layer-composition, generic
inheritance, or `fs.FS` source API. Its deferred scope explicitly excludes several
of these generalizations; an upstream enhancement requires a deliberate decision.

This is a source-and-contract assessment, not a performance benchmark or full
correctness audit. It does not claim measured allocation, latency, or line-count
savings. Current code and canonical contracts remain authoritative until an
approved implementation changes them.

## 1. Capability Inventory

| Preserve the ability to… | Replaceable mechanics and representation |
| --- | --- |
| Define named, versioned prompts, inspect their declared inputs, and select a version explicitly | Directory scans on each operation, YAML metadata prepasses, filename layout, constructor signatures |
| Use inline and file-backed message templates, variables, input helpers, session templates, roles, and cache control | Timing of definition/content reads and private repository interfaces |
| Configure named profiles, alias/inherit another profile, and override selected settings | Zero-as-absence fields, per-call chain traversal, raw YAML decoding types |
| Supply embedded application defaults, operator definitions, maintained catalogs, and programmatic profiles | Repository-wrapper stacks and exact source-option API |
| Register custom/local backends, use built-ins or endpoint-only profiles, and control backend capacity | Configuration DTO shapes and registry assembly order |
| Prepare, inspect, execute, or freeze an execution for later single-claim execution | Source reloading during ordinary preparation; exact preparation error ordering |
| Override per-run settings, credentials, reasoning, session, and output contract; append rendered messages | Public value signatures if presence or clearer configuration separation requires revision |
| Supply inline/file artifacts and inject artifact readers, model clients, and HTTP clients | Assembly APIs; runtime collaborators need not be YAML-decodable |
| Validate output, use provider-native structured output, bounded repair, transport retries, cancellation, and observation | Internal wiring, not the substantive execution behaviors |
| Receive results, usage, identities, hashes, validation outcomes, and inspectable failures | Configuration error types and when definition failures appear |

Preserve independent profile and prompt inspection as useful operations. They
need not retain the old guarantee that engine construction has never examined
other definitions. Preserve custom backend identity and capacity behavior,
optional upstream credentials, and the distinction between failed generated
content and operational errors. This project does not authorize silently
dropping execution features to simplify the configuration layer.

## 2. Responsibility Mapping

| Concern | Recommended owner and disposition |
| --- | --- |
| YAML grammar, scalar interpretation, one-document enforcement, unknown fields, typed ranges, null legality | Delegate to ConfigKit; remove corresponding PromptKit parser logic and redundant parser tests |
| Field presence and source coordinates | Delegate to ConfigKit; retain only the compact records needed for resolution and diagnostics |
| File/directory/embedded source selection and definition ID indexing | Retain a small PromptKit collection loader; ConfigKit loads documents, not a prompt catalog |
| Filesystem traversal and relative content references | Retain PromptKit's source-boundary adapters; use standard `io/fs` facilities where sufficient |
| Profile-source priority and duplicate definition policy | Retain explicit PromptKit collection assembly; eliminate error-driven repository fallback |
| Named profile inheritance and backend selection | Retain a typed PromptKit resolver over the loaded collection, with explicit presence; consider generic inheritance upstream only with broader evidence |
| LLM setting bounds, role/session normalization, output rules, reserved provider fields | Retain shared PromptKit domain owners; ConfigKit validation callbacks may invoke them |
| Definition defaults | Use ConfigKit defaults where they do not erase inheritance presence; apply effective execution defaults after inheritance |
| Programmatic configuration | Retain typed construction; normalize to the same presence-aware internal definitions without YAML round trips |
| Runtime clients, observers, artifact readers, request data | Retain normal Go injection and request APIs; do not load them through ConfigKit |
| Credentials | Retain optional environment-name and direct request-key semantics; do not replace these with ConfigKit's required typed secret resolution |
| Catalog manifest, ownership, reserved names, secret-safety checks | Retain the catalog adapter; reuse the common ConfigKit profile decoder |
| JSON Schema resources and generated-output validation | Retain JSON Schema implementation; these are not application configuration documents |
| Application imports, CLI overrides, deployment overlays, secrets-directory bootstrap | Leave with consuming applications unless a concrete PromptKit need is accepted |
| Generic source/decoding diagnostics | Reuse ConfigKit; add only PromptKit identity/context and domain relationship diagnostics |

## 3. Recommended Target Loading Model

### Definition Snapshot

Create one immutable definition snapshot during engine construction. Discover
the explicitly supplied sources, decode each definition through ConfigKit,
normalize domain values, assemble identity indexes, and resolve profile chains
before publishing the engine. No provider call or credential availability check
belongs in this phase. Publish no partially initialized engine on failure.

Use one typed document per prompt or profile initially. Existing files need not
be converted into a monolithic application configuration merely to adopt
ConfigKit. Support OS directories, individual files, embedded/custom `fs.FS`
sources, and programmatic profiles. For `fs.FS`, a small adapter can open a file
and supply `configkit.Reader` with a safe source label; PromptKit retains root
and path context for referenced message files. Close handles owned by the adapter.
ConfigKit's existing public API is sufficient for this design.

Load and freeze referenced message text with definitions. Validate static prompt
structure and template syntax at loading, using the same template function setup
as execution; render only when request variables and artifacts exist. Preserve
source-relative content resolution and containment without expanding the threat
model. A snapshot is a set of captured reads, not an atomic filesystem transaction.

Keep request artifacts and schema compilation at preparation. A request can
replace the output contract, so an unused default schema should not need eager
compilation. Prepared execution continues to freeze its effective resources.
The snapshot covers definitions/message text, not environment values, request
files, every schema file, or execution state.

### Composition And Validation

Use explicit source categories in the existing useful priority order:
programmatic profiles, operator source, application fallback, PromptKit-owned
profiles, then external catalogs. Within a category, duplicate profile IDs fail;
across categories, a higher-priority definition replaces the whole lower one.
Prompt ID/version pairs must be unique; different versions remain available.
An omitted version remains ambiguous when more than one version is present.

Strictly decode and locally validate every supplied definition, including
shadowed definitions. Thus an unused malformed file is a construction failure,
not a reason to perform a partial metadata scan. Resolve relationships for the
effective selected profile collection; shadowed definitions do not contribute
inheritance edges to that collection. Keep the catalog adapter's independent
validation of maintained data before consumer overrides.

Preserve profile templates that become usable through descendants or runtime
overrides: eager loading must not require every consumer profile to be an
independently runnable endpoint/model pair. Validate supplied fields and graph
integrity eagerly (including missing bases and cycles), and check final execution
requirements after runtime selection/overrides. Prompt default-profile references
can be checked against the assembled collection; a prompt with no default remains
valid for callers that supply a profile explicitly.

This is named-definition precedence, not ConfigKit's application overlay system.
Do not generate temporary overlay files or serialize maps to YAML to imitate a
missing layer API. ConfigKit imports are also not directory discovery: their
strict composition has different semantics and need not be enabled here.

### Inheritance And Presence

Recommend omission-based inheritance: absent fields inherit; explicit values
replace inherited settings, including zero, false, and empty strings where the
domain permits them. Preserve presence through inheritance, backend/framework
default application, request overrides, and outbound request construction.
Otherwise accepting `temperature: 0` during decoding would accomplish nothing.

Use ConfigKit metadata for file presence and presence-aware typed fields for
programmatic definitions. Both paths feed one typed resolver. Decode raw profiles
without effective execution defaults so defaults do not become child overrides.
Avoid a parallel reflection-based merge engine in PromptKit.

Keep provider `extra_params` replacement atomic: an explicitly supplied map
replaces the inherited map, including an empty map. This matches the useful
ability to choose a complete provider parameter set and avoids accidental mixing
of incompatible options. Do not generalize this domain rule into arbitrary
ConfigKit merge strategies.

Recommend optional scalar pointers where null means no value/inherit, while
explicit zero/false/empty is a value. For optional maps, null likewise means no
override and `{}` explicitly clears. Required identity fields remain required;
an explicitly empty model or endpoint cannot produce an executable target.
Publish these conventions deliberately rather than pretending ConfigKit's
presence metadata automatically determines domain inheritance semantics.

Resolve and memoize each effective chain once per snapshot. Retain source and
field-origin information sufficient to diagnose a bad base or inherited value;
do not claim that per-document ConfigKit metadata already represents the complete
cross-document inheritance history.

### Reloading, Errors, And Application Integration

Later definition edits take effect through a newly constructed engine. Do not add
watchers, mutable registry reloads, or a second lazy compatibility mode. Applications
choose how to swap engines and let in-flight work finish. Old prepared handles
remain associated with their original engine. Simultaneously active engines have
separate backend capacity pools; migration guidance must not imply a global limit
across a reload overlap.

Return configuration failures from construction with file/path/location and
useful definition identity where available. Recommend preserving ConfigKit's
public `LoadError` through wrapping for `errors.As`, together with a PromptKit
configuration category; do not copy its entire diagnostic API into the facade.
Domain-only relationship failures remain PromptKit errors with relevant source
locations. Start with deterministic fail-fast loading across files and ConfigKit's
existing multi-diagnostic behavior within a load, rather than inventing a batch
diagnostic subsystem. Do not expose raw credentials or add implicit logging.

Runtime failures continue to distinguish invalid requests, missing selections,
rendering/artifact/schema errors, provider/transport errors, capacity rejection,
and content-validation results. Cancellation remains effective at
PromptKit-controlled boundaries; ConfigKit's current synchronous `Load` has no
context argument, so do not promise interruption inside it or create goroutines
to simulate that guarantee.

Applications using ConfigKit can present the same underlying loading diagnostics
for their application settings and PromptKit files. They can also translate
application-owned typed settings into PromptKit profiles/backends. Do not require
applications to encode Go clients into YAML or adopt ConfigKit merely to call
PromptKit's runtime API. No full effective-configuration/explanation API is proposed
without an identified consumer requirement.

## 4. Simplification Accounting

| Current implementation | Expected disposition |
| --- | --- |
| `internal/profile/filesystem_repository.go`: metadata scans, selected strict decode, stream-end checks, malformed-file selection arbitration | Replace with eager document loads and indexed values; remove the metadata prepass and per-lookup filesystem path |
| `internal/promptdef/filesystem_repository.go`: metadata selection and repeated decoding/content opening | Replace with eager prompt loads and an ID/version index; keep domain normalization and contained content reading |
| Profile filesystem, `fs.FS`, eager, memory, and overlay repository variants | Collapse loading into source adapters plus immutable collections; preserve only interfaces that still serve a real boundary |
| `internal/profile/resolving_repository.go` | Replace per-operation traversal and zero-sentinel merging with once-per-snapshot presence-aware typed resolution; graph logic remains |
| `internal/engine` raw repository-stack assembly | Replace with explicit collection construction and precedence; engine capacity/client assembly remains |
| `internal/catalog` profile metadata extraction | Obtain explicit field information from ConfigKit; retain manifests, ownership, namespace, connection-metadata and secret checks |
| `internal/filecatalog` | Retain/reduce discovery and containment helpers actually needed by the eager loader; do not claim the entire package disappears |
| `internal/domain` source YAML tags and runtime/source coupling | Separate configuration DTOs from execution values where presence requires it; reuse domain validation rather than duplicating it |
| Direct `gopkg.in/yaml.v3` use for definition loading | Remove once all such consumers migrate; ConfigKit brings its own private YAML dependency |

New integration work is real: typed source DTOs, ConfigKit calls and error wrapping,
source-context records, collection indexing, and conversion to runtime definitions.
Some defensive copies remain necessary at public/injected-client boundaries;
ConfigKit's detached load result is not permission to expose shared engine maps.
`internal/jsonvalue` still serves programmatic values, request parameters, and
prepared schema ownership, so adoption does not eliminate it.

Maintenance savings depend on actually deleting the old lazy paths and their
special-case tests. Move generic YAML/scalar behavior coverage to ConfigKit's
ownership; PromptKit retains representative adapter tests plus domain, precedence,
snapshot, inheritance, embedded-source, error, and execution integration tests.
Do not duplicate ConfigKit's entire parser suite or discard execution regression
coverage because construction fixtures need migration.

Runtime benefit: ordinary runs no longer rediscover definition directories,
redecode definitions, reread template text, or traverse inheritance chains.
Cost: construction loads even unused definitions and retains their text and
resolved values. ConfigKit also creates metadata and projections; keep only needed
records after loading. These are reasoned tradeoffs, not measured performance
claims. Adoption is not justified by the decoder replacement alone.

## 5. Upstream Opportunities

No ConfigKit enhancement is a demonstrated prerequisite for the recommended
per-document snapshot design. Do not manufacture one to move a small adapter
upstream. Reconsider the following if the agreed roadmap reveals concrete reuse:

| Candidate | Recommendation and tradeoff |
| --- | --- |
| Native `fs.FS` sources with source-relative resource context | Most plausible optional enhancement: embedded configurations are useful across Go libraries. Reader sources suffice today. A fuller design must define imports/overlays in that filesystem rather than merely add a constructor or leak parser internals. |
| Public in-memory layered loading with origins | Optional only if multiple consumers need application defaults and file/embedded layers composed together. It could remove adapters in those consumers, but is not necessary for whole-definition catalog shadowing. Review against ConfigKit's deliberate single-overlay scope. |
| Generic named inheritance/presence-aware typed overlay | Consider an upstream proposal if a coherent application-neutral operation emerges; another concrete use case would strengthen the case but is not a prerequisite. PromptKit's named graph, atomic provider map, and routing rules alone do not justify a configurable general merge framework. Current fixed overlay semantics are not this API. |
| Context-aware loading | Optional for reusable cancellable construction if large-source consumers need it. Promise cooperative cancellation only; no detached reads or instant interruption of arbitrary readers. |

Do not propose a public raw AST, permissive identity-only decoder, or dynamic
subtree conversion solely to preserve PromptKit's current scanning design.
Do not upstream backend/model selection, credential timing for inference,
profile catalog reservations, or prompt-template functions.

ConfigKit and PromptKit both declare LGPL-3.0-or-later. Adoption must still update
dependency notices under the OSS policy and inspect the exact selected release
and its transitive notices. Their current Go minimums do not require a PromptKit
toolchain increase. Use a released module dependency, not a sibling-directory
replacement. This review proposes no licensing change or upstream edits.

## 6. Decisions And Migration

The goal and permission to break compatibility are agreed. The following specific
choices are recommendations for discussion, not inferred approvals:

1. **Loading lifetime and failure scope.** Recommend eager construction, frozen
   definitions/content, and failure for malformed files anywhere in supplied
   collections. This deletes lazy arbitration and repeated scans. Alternative:
   explicit caller-loaded reusable definition snapshots separate from engine
   construction; useful when multiple engines share data, but adds a public
   lifecycle. Retaining automatic per-run reload is possible but sacrifices much
   of the simplification and is not recommended.
2. **Source organization.** Recommend individual typed definition documents with
   explicit collection precedence. This preserves embedded catalogs and distributed
   authoring without copying application configuration machinery. Alternative:
   one keyed PromptKit bundle composed using ConfigKit imports and overlays;
   offers richer unified provenance but needs decisions about embedded sources,
   provider-map replacement, profile identity, and catalog conversion. Do not
   assume existing ConfigKit APIs already solve those issues.
3. **Presence and inheritance.** Recommend explicit-value replacement, null as
   unspecified for optional settings, and atomic provider maps as described above.
   Alternative: use ConfigKit-style recursive mapping overlays, where null clears;
   this is more uniform with application overlays but changes provider-parameter
   behavior and requires a supported cross-document mechanism. Historical
   zero-as-absence is another viable option but loses useful standardization.
4. **Diagnostic boundary.** Recommend wrapping ConfigKit's public load errors
   rather than mirroring them. Alternative: translate into a small PromptKit-owned
   diagnostic value; that limits public dependency coupling but introduces mapping
   code and risks inconsistent diagnostics. Either option must cover domain errors
   and preserve actionable source information without inventing an audit subsystem.

Expected consumer migration:

- Update definitions for ConfigKit's strict scalar rules, rejected YAML
  anchors/aliases/tags, input limits, and null/type checks. Quote prompt version
  strings rather than introducing a special numeric-version conversion merely
  for historical YAML coercion.
- Remove unintended explicit zero/false/empty values that formerly meant
  inheritance; express deliberate overrides using the new presence-aware Go
  fields and file semantics. Review provider maps for explicit replacement.
- Handle definition errors at construction, clean up unused invalid files, and
  deliberately rebuild/swap engines after definition changes.
- Keep runtime credential lookup optional, avoid baking resolved secrets into
  definition snapshots, and preserve request-scoped key overrides.
- Update maintained catalog data if strict parsing reveals incompatible syntax;
  coordinate catalog releases before the PromptKit dependency update. Preserve
  maintained profile names and catalog ownership rules, not every old scalar
  spelling. Do not validate live provider/model availability.
- Update error inspection, constructor/source options, and programmatic profile
  literals as required. Keep representative preparation, execution, inspection,
  custom backend, fallback, and embedded workflows in the migration guidance.

The subsequent feature roadmap should settle these choices and specify the target
end state. The staged implementation plan can then sequence any accepted upstream
work, PromptKit deletions/replacements, contract tests, canonical documentation,
examples, attribution, and minor-release migration notes. Neither document should
become a reason to preserve obsolete compatibility machinery.