promptkit/docs/roadmap/configkit.md

ConfigKit Adoption

Status: feature roadmap for design agreement. ConfigKit v0.5.0 supplies the required upstream capabilities; significant PromptKit choices are collected under Open Decisions. A separate implementation plan will define stages, signatures, and execution details. No implementation is authorized by this document.

Purpose

Use ConfigKit to standardize PromptKit's configuration semantics and simplify its loading machinery, replacing custom logic wherever ConfigKit is the appropriate owner. Preserve substantive consumer capabilities, not historical file syntax, loading order, or Go signatures. Pre-1.0 breaking changes are acceptable and must have migration guidance.

Applications using both libraries should share configuration interpretation, layering, inheritance, presence, and inspection semantics. PromptKit should concentrate on LLM-specific validation, resource interpretation, and execution. The adoption assessment records the supporting analysis; this roadmap owns the proposed feature end state and remaining choices.

Baseline And Dependency

The current PromptKit runtime baseline is v0.11.2. Its source implementation, format contract, and consumer guide describe existing behavior, not the target described here.

The reviewed ConfigKit release is v0.5.0 at dc09959cca4e105f49ec316c6d4ba585bd2340bd. It provides native fs.FS sources, LoadLayered, source-backed underlays, labeled programmatic mappings, passive resource contexts, and built-in sibling inheritance. Its released configuration contract and GoDoc own exact upstream semantics. No further upstream feature is currently a demonstrated prerequisite. Review any newer version before selection.

Adopt a released Go module, not a local replace or workspace requirement. Update attribution and transitive license notices under the OSS policy. This feature does not change either project's license or require a Go minimum increase relative to the reviewed versions.

Substantive Capabilities To Preserve

• Named/versioned prompts, declared-input inspection, templates, inline and referenced message content, variables, input helpers, roles, and cache control.
• Maintained catalogs, application defaults, operator configuration, programmatic profiles, aliases/inheritance, built-in/custom/local backends, and endpoint-only profiles.
• Preparation without generation, ordinary execution, and frozen single-claim prepared execution; independent prompt and profile inspection.
• Runtime execution-setting, reasoning, session, credential, and output-contract overrides; appended already-rendered messages.
• Inline/file input artifacts and injected artifact, model, and HTTP collaborators.
• Backend capacity management, cancellation, transient retries, structural output validation, provider-native structured output, bounded repair, and observations.
• Results, effective identities/settings, usage, hashes, and meaningful errors.

Retain optional environment credentials and direct request credentials. ConfigKit secret resolution must not introduce a new requirement that an upstream API key exist. Domain validation must not probe provider/model availability.

Target Configuration Model

One Shared Loading Lifecycle

Construction acquires configuration contributions, uses ConfigKit to compose and overlay them, expands sibling inheritance, decodes the effective configuration, performs recorded domain resolution and validation, and produces execution-ready definitions. Publish no partially initialized engine on failure.

ConfigKit owns generic parsing, combination, presence, origins, and inheritance. PromptKit owns source selection/discovery, definition indexing, backend policy, LLM-specific rules, and translation into runtime values. Runtime clients and callbacks remain normal Go dependencies, not YAML configuration types.

Use keyed collections assembled before typed decoding rather than independently decoding definitions and combining them afterward. Multiple files can contribute to a collection; a monolithic file is not required. The exact collection layout and construction surface are open decisions below.

Keep the public package a deliberate facade and implementation private. Do not expose internal engine/domain types or introduce a parallel generic loader merely to preserve old source options. ConfigKit's supported public types may be used where intentional integration is simpler than maintaining equivalent wrappers.

Sources, Tiers, And Presence

Support ordinary files/directories, embedded or caller-supplied fs.FS, and programmatic definitions. Explicitly selected directories may be discovered by PromptKit; ConfigKit handles their configuration contributions. Do not infer priority from traversal order.

Retain the useful profile priority categories, lowest to highest: external catalogs, PromptKit-owned profiles, application fallback, operator configuration, and programmatic configuration. Express intentional priority as ordered tiers. Within each tier, independent contributions compose strictly; overlapping non-mapping values conflict, even when equal. Disjoint fields under the same identity can combine. Across tiers, mappings recursively overlay.

Use ConfigKit's source roles honestly: underlay groups are ordered defaults, main input overlays underlays, and optional named overlay/CLI facilities retain their upstream meaning. Underlay-only loading is supported; do not manufacture a main document or disguise programmatic values as CLI flags. Exposing every ConfigKit convenience through PromptKit is not a requirement.

Adopt the shared value rules: omission retains lower input; explicit zero, false, empty string, sequence, and null replace; mappings combine recursively; empty mappings do not clear lower children. Null remains explicit and subject to the final schema. Provider-parameter maps obey these rules too. No hidden whole-profile replacement, arbitrary merge callback, or new reset/delete operator belongs here.

Preserve presence into execution-ready settings so an explicit zero does not disappear during backend/default resolution or outbound request construction. Use ConfigKit metadata at construction, not as a per-run lookup mechanism.

Sibling Inheritance

Use ConfigKit's reserved extends rather than PromptKit's base_profile and custom resolver. A string identifies an immediate sibling mapping; null cancels the relationship without deleting the child's own settings. Layering completes before inheritance. Only effective relationships resolve, but all of them do, including entries never selected for execution.

Children recursively overlay resolved parents. Inherited values count as explicit input. Origins and resource context remain attached to their declaring source. Nested relationships follow ConfigKit's destination-local rules; defaults and application-derived values do not trigger another inheritance pass.

The keyword is reserved throughout configuration, including dynamic provider maps; it is not opt-in. The roadmap must resolve the ordinary-data limitation without silently renaming outbound provider fields or dropping extensibility.

Validation And Errors

Source acquisition, syntax, and strict-composition failures cannot be masked by higher tiers. Typed and semantic validation apply to effective values rather than every intermediate layer. An unused invalid effective definition fails construction. Separate catalog ownership/reserved-name/secret-safety checks remain source-policy responsibilities, not justification for duplicate generic decoding.

Domain validators remain shared with runtime request validation where appropriate. Partial profile bases must satisfy the configuration schema, while requirements that depend on actual selection and overrides are checked at execution preparation. Do not require credentials, artifacts, or provider connectivity during construction.

Preserve ConfigKit public loading errors through wrapping for errors.As, with a PromptKit configuration category. Use source-aware domain/resource failures where ConfigKit is not the owner. Do not mirror its diagnostic framework or promise aggregation beyond what the underlying operation provides. Runtime operational errors remain distinct from generated-content validation failures.

Prompt versions are domain identifiers, not ConfigKit VersionRule declarations. Do not introduce an application schema-version policy merely to adopt ConfigKit.

Snapshot, Resource, And Execution Boundaries

The engine captures effective definitions and referenced message text during construction. Validate static template syntax then; render with request data later. Resolve content references from the effective field's contributing source context, not the last layer's directory. Ambiguous/contextless resource references need an explicit source context or an actionable error, not an invented directory.

ConfigKit returns passive resource contexts, not snapshots of referenced text. PromptKit explicitly opens and closes its resource handles. Caller filesystem ownership remains unchanged. Preserve appropriate source containment without claiming that every fs.FS is a security sandbox. Captured reads are not an atomic transaction over a concurrently changing filesystem.

Definition changes require new engine construction. No watcher, background refresh, or lazy compatibility mode is added. Applications own engine replacement; prepared handles remain bound to their original engine and overlapping engines retain separate capacity pools.

Request artifacts and schema preparation remain invocation work. A request can replace the output contract, so construction need not compile unused schemas. Prepared execution continues to freeze its effective resources. Environment credential lookup remains at the established execution/preparation boundary, not frozen into the configuration snapshot.

Ordinary execution uses indexed ready definitions and resolved presence values. It performs no configuration discovery, parsing, inheritance traversal, or provenance analysis. Context cancellation remains cooperative at controlled boundaries; do not add detached work to pretend synchronous ConfigKit loading is instantly interruptible.

Explicit Operator Inspection

Add an API consumers can use to implement targeted CLI commands or debug output for effective configuration, contributing sources, and explanations. Retain ConfigKit inspection state with the engine's configuration snapshot and reuse its redaction and provenance facilities.

Inspection is explicitly invoked and uses that snapshot without reloading files, opening resources, or contacting providers. It does not predict every runtime override, reveal execution-time credential values, or replace prepared-run details and execution observations. No provenance traversal, explanation generation, or configuration-report rendering occurs during ordinary prompt runs.

PromptKit ships no CLI and emits no automatic debug reports. Applications own presentation, access control, and disclosure; redaction does not make all prompt or configuration content safe to publish. Exact inspection API scope is an open decision below.

Required Refactoring And Coordination

• Replace profile/prompt metadata prepasses, selected-file decoding, and per-call scans with ConfigKit collection loading and immutable indexed definitions.
• Remove the custom profile inheritance resolver and error-driven repository overlay stack. Do not retain an old-format compatibility engine alongside the new semantics.
• Retain only needed discovery and message-resource adapters; remove direct YAML dependency usage once no remaining owner needs it. Preserve JSON Schema, runtime JSON-value ownership, domain validation, and execution collaborators.
• Adapt maintained catalogs and PromptKit-owned definitions to the agreed keyed format and released inheritance semantics. Preserve maintained name commitments, backend ownership, and catalog namespace rules without upstream availability checks. Coordinate necessary catalog releases before consuming their new format. This document requests no immediate changes in those repositories.
• Keep configuration conversion, normalization, and inspection connected to ConfigKit's lifecycle instead of reconstructing effective values/history after loading. Translate once into execution-owned values with necessary copying.
• Update canonical formats, consumer guidance, GoDoc, internal inventory, examples, architecture boundaries, dependencies, and OSS notices when implementation lands.

Efficiency And Validation Expectations

Target linear or near-linear work relative to actual input, output, and retained history, accounting separately for sorting, resource reads, and domain work. Acquire/parse each accepted source occurrence once, avoid decoding intermediate tiers, and index definition selection. Do not duplicate inheritance expansion or copy complete accumulated collections for every added layer.

Materialized inherited data and provenance can exceed source size. Exercise real catalog shapes within ConfigKit's budgets; do not claim unconditional O(n) runtime or memory. Prefer clear ownership and straightforward indexes over global caches or elaborate optimization. Operator inspection may perform work proportional to the requested report; that work remains off the execution path.

Tests should use real ConfigKit with offline sources and synthetic provider collaborators. PromptKit owns its adapter, domain, lifecycle, and execution tests, not a duplicate ConfigKit parser/graph suite. Retire tests whose only purpose was the removed loading machinery. Preserve meaningful execution regressions.

Representative acceptance evidence must cover:

• Minimal local-backend construction and execution without elaborate configuration.
• Embedded defaults plus operator and programmatic contributions, including accurate inherited relative-resource origins.
• Profile inheritance, changing model/backend, canceling old relationships, and discarding unwanted settings with documented semantics.
• Explicit zero/false/null behavior reaching execution, construction failures, independent snapshots, and unchanged substantive preparation/execution features.
• Targeted inspection with useful redacted origins and no provenance work during ordinary runs.

Use focused scaling/allocation benchmarks for large collections, shared bases, and many tiers where they expose repeated-work risks. Do not add timing-based release gates or provider-network tests.

Migration And Completion

Ship as an intentionally incompatible minor release with concrete migration guidance. Cover keyed files, base_profile to extends, reserved-key behavior, recursive maps, explicit null versus omission, changed error timing, programmatic construction, and explicit engine replacement. Explain that overriding a model does not discard lower endpoint, reasoning, provider, or inheritance settings.

The end state is one shared configuration model with the old loading/resolution machinery removed, useful operator inspection, and all substantive execution workflows still available. The following decisions must be resolved before a decision-complete implementation plan is written; implementation stages do not belong in this feature roadmap.

Open Decisions

1. Collection Format And Source Surface

Recommendation: use profiles.<id> and prompts.<id>.<version> keyed mappings in documents that can contribute multiple entries. Keys own identity rather than duplicated id/version fields. Preserve omitted-version ambiguity when an ID has multiple versions. Supply explicit source groups through a small construction surface compatible with ConfigKit's Input/underlay concepts; retain convenient file/directory/embedded entry points without parallel semantics. This fits sibling inheritance and strict composition while minimizing identity reconciliation.

Alternative: keep a keyed profile collection but represent each prompt ID as an object with a nested versions collection. This offers room for ID-level metadata and shared settings, but adds another level and decisions about which fields inherit where. Neither format requires a single physical file. Choose one format and migrate catalogs rather than sustaining both through compatibility code.

2. Programmatic Ergonomics

Recommendation: accept ConfigKit's labeled mapping sources and provide only small typed PromptKit conveniences demonstrated by the minimal local-backend and application-profile workflows. Such helpers must encode explicit presence and feed the same loading lifecycle, not merge configurations or infer omission from zeros. Keep HTTP/model clients and runtime backend collaborators outside generic decoding.

Alternative: expose only mapping-based configuration plus runtime collaborators. This minimizes conversion code and is entirely consistent with ConfigKit, but can make common library use verbose and less discoverable. Compare concrete consumer examples before deciding which typed helpers earn their maintenance cost.

3. Partial Profiles, Nulls, And Effective Defaults

Recommendation: allow schema-valid partial profiles without an abstract flag; validate supplied domain values at loading and final runnable targets after profile selection and runtime overrides. Preserve absent versus explicit null in resolved settings. Treat null on optional provider settings as a deliberate suppression, not permission to restore the same value from backend defaults; a null provider map suppresses the configured lower map, while null entries inside that map remain literal JSON null values. Framework-required settings such as timeouts need an explicit field-by-field rule in the implementation plan, not blanket nil defaulting. This allows reusable bases and meaningful removal of stale inherited settings.

Alternative: require an explicit abstract-profile marker and define nullable fields as returning to backend/framework defaults. This makes executable-profile validation earlier and default recovery simple, but adds a configuration concept and makes suppression harder. Demonstrate model/backend switching without awkward reset tiers before choosing it. Neither option may redefine ConfigKit's merge rules.

4. Literal extends Inside Provider Data

Recommendation: reserve extends throughout file/mapping configuration as ConfigKit does, while keeping runtime request provider-parameter maps ordinary JSON-compatible data outside configuration parsing. Document this distinction; applications needing that literal provider key can supply it at the runtime boundary. This preserves outbound extensibility without adding a second parser or pretending ConfigKit offers an escape syntax.

Alternative: introduce an explicitly opaque JSON provider-parameter input for configuration, decoded by PromptKit as domain payload after structural loading. This would allow file-defined literal reserved keys, but adds a representation, validation/redaction responsibilities, and awkward layering behavior. Pursue it only if runtime supply does not satisfy concrete consumers; never silently rename the outbound key. Audit maintained catalog payloads before finalizing this choice.

5. Operator Inspection API

Recommendation: offer a read-only inspection view backed by retained ConfigKit state, with targeted path queries, source/history information, and explicit redacted effective/explanation output. Reuse suitable public ConfigKit inspection values instead of mirroring them. Do not expose mutable runtime definitions, resource capabilities merely for diagnostics, or request credentials. This gives consumers useful operator tooling with minimal duplicate machinery.

Alternative: return narrow PromptKit-owned profile/prompt inspection records containing only effective settings and selected origin summaries. This limits public dependency coupling and is easier to constrain, but loses generic path inspection and needs translation code as ConfigKit evolves. Define precisely what is omitted rather than claiming those summaries are full provenance.

6. Catalog Format Transition

Recommendation: release new keyed catalog assets in the existing catalog projects, preserving their data-only role and recording the format transition. Consume those releases directly after checking manifest/namespace/ownership rules. This removes long-term conversion machinery and aligns maintained and consumer definitions. The catalog maintainers must choose the applicable release/version handling under their own policies; PromptKit must not assume old consumers can decode the changed shape.

Alternative: publish new-format assets alongside legacy assets for a transition period. This protects independently upgrading old consumers but temporarily duplicates catalog maintenance and requires an explicit retirement decision. Avoid a permanent PromptKit adapter that reparses and serializes legacy definitions solely to postpone coordinated migration.