diff --git a/docs/roadmap/configkit.md b/docs/roadmap/configkit.md new file mode 100644 index 0000000..cc97bbe --- /dev/null +++ b/docs/roadmap/configkit.md @@ -0,0 +1,367 @@ +# ConfigKit Adoption + +Status: feature roadmap for design agreement. ConfigKit v0.5.0 supplies the +required upstream capabilities; significant PromptKit choices are collected +under Open Decisions. A separate implementation plan will define stages, +signatures, and execution details. No implementation is authorized by this document. + +## Purpose + +Use ConfigKit to standardize PromptKit's configuration semantics and simplify +its loading machinery, replacing custom logic wherever ConfigKit is the +appropriate owner. Preserve substantive consumer capabilities, not historical +file syntax, loading order, or Go signatures. Pre-1.0 breaking changes are +acceptable and must have migration guidance. + +Applications using both libraries should share configuration interpretation, +layering, inheritance, presence, and inspection semantics. PromptKit should +concentrate on LLM-specific validation, resource interpretation, and execution. +The [adoption assessment](configkit-assessment.md) records the supporting analysis; +this roadmap owns the proposed feature end state and remaining choices. + +## Baseline And Dependency + +The current PromptKit runtime baseline is v0.11.2. Its +[source implementation](../internal/sources.md), [format contract](../formats.md), +and [consumer guide](../consumers/pkg-promptkit.md) describe existing behavior, +not the target described here. + +The reviewed ConfigKit release is v0.5.0 at +`dc09959cca4e105f49ec316c6d4ba585bd2340bd`. It provides native `fs.FS` sources, +`LoadLayered`, source-backed underlays, labeled programmatic mappings, passive +resource contexts, and built-in sibling inheritance. Its released configuration +contract and GoDoc own exact upstream semantics. No further upstream feature is +currently a demonstrated prerequisite. Review any newer version before selection. + +Adopt a released Go module, not a local `replace` or workspace requirement. +Update attribution and transitive license notices under the OSS policy. This +feature does not change either project's license or require a Go minimum increase +relative to the reviewed versions. + +## Substantive Capabilities To Preserve + +- Named/versioned prompts, declared-input inspection, templates, inline and + referenced message content, variables, input helpers, roles, and cache control. +- Maintained catalogs, application defaults, operator configuration, programmatic + profiles, aliases/inheritance, built-in/custom/local backends, and endpoint-only + profiles. +- Preparation without generation, ordinary execution, and frozen single-claim + prepared execution; independent prompt and profile inspection. +- Runtime execution-setting, reasoning, session, credential, and output-contract + overrides; appended already-rendered messages. +- Inline/file input artifacts and injected artifact, model, and HTTP collaborators. +- Backend capacity management, cancellation, transient retries, structural output + validation, provider-native structured output, bounded repair, and observations. +- Results, effective identities/settings, usage, hashes, and meaningful errors. + +Retain optional environment credentials and direct request credentials. ConfigKit +secret resolution must not introduce a new requirement that an upstream API key +exist. Domain validation must not probe provider/model availability. + +## Target Configuration Model + +### One Shared Loading Lifecycle + +Construction acquires configuration contributions, uses ConfigKit to compose and +overlay them, expands sibling inheritance, decodes the effective configuration, +performs recorded domain resolution and validation, and produces execution-ready +definitions. Publish no partially initialized engine on failure. + +ConfigKit owns generic parsing, combination, presence, origins, and inheritance. +PromptKit owns source selection/discovery, definition indexing, backend policy, +LLM-specific rules, and translation into runtime values. Runtime clients and +callbacks remain normal Go dependencies, not YAML configuration types. + +Use keyed collections assembled before typed decoding rather than independently +decoding definitions and combining them afterward. Multiple files can contribute +to a collection; a monolithic file is not required. The exact collection layout +and construction surface are open decisions below. + +Keep the public package a deliberate facade and implementation private. Do not +expose internal engine/domain types or introduce a parallel generic loader merely +to preserve old source options. ConfigKit's supported public types may be used +where intentional integration is simpler than maintaining equivalent wrappers. + +### Sources, Tiers, And Presence + +Support ordinary files/directories, embedded or caller-supplied `fs.FS`, and +programmatic definitions. Explicitly selected directories may be discovered by +PromptKit; ConfigKit handles their configuration contributions. Do not infer +priority from traversal order. + +Retain the useful profile priority categories, lowest to highest: external +catalogs, PromptKit-owned profiles, application fallback, operator configuration, +and programmatic configuration. Express intentional priority as ordered tiers. +Within each tier, independent contributions compose strictly; overlapping +non-mapping values conflict, even when equal. Disjoint fields under the same +identity can combine. Across tiers, mappings recursively overlay. + +Use ConfigKit's source roles honestly: underlay groups are ordered defaults, +main input overlays underlays, and optional named overlay/CLI facilities retain +their upstream meaning. Underlay-only loading is supported; do not manufacture +a main document or disguise programmatic values as CLI flags. Exposing every +ConfigKit convenience through PromptKit is not a requirement. + +Adopt the shared value rules: omission retains lower input; explicit zero, false, +empty string, sequence, and null replace; mappings combine recursively; empty +mappings do not clear lower children. Null remains explicit and subject to the +final schema. Provider-parameter maps obey these rules too. No hidden whole-profile +replacement, arbitrary merge callback, or new reset/delete operator belongs here. + +Preserve presence into execution-ready settings so an explicit zero does not +disappear during backend/default resolution or outbound request construction. +Use ConfigKit metadata at construction, not as a per-run lookup mechanism. + +### Sibling Inheritance + +Use ConfigKit's reserved `extends` rather than PromptKit's `base_profile` and +custom resolver. A string identifies an immediate sibling mapping; null cancels +the relationship without deleting the child's own settings. Layering completes +before inheritance. Only effective relationships resolve, but all of them do, +including entries never selected for execution. + +Children recursively overlay resolved parents. Inherited values count as explicit +input. Origins and resource context remain attached to their declaring source. +Nested relationships follow ConfigKit's destination-local rules; defaults and +application-derived values do not trigger another inheritance pass. + +The keyword is reserved throughout configuration, including dynamic provider +maps; it is not opt-in. The roadmap must resolve the ordinary-data limitation +without silently renaming outbound provider fields or dropping extensibility. + +### Validation And Errors + +Source acquisition, syntax, and strict-composition failures cannot be masked by +higher tiers. Typed and semantic validation apply to effective values rather than +every intermediate layer. An unused invalid effective definition fails construction. +Separate catalog ownership/reserved-name/secret-safety checks remain source-policy +responsibilities, not justification for duplicate generic decoding. + +Domain validators remain shared with runtime request validation where appropriate. +Partial profile bases must satisfy the configuration schema, while requirements +that depend on actual selection and overrides are checked at execution preparation. +Do not require credentials, artifacts, or provider connectivity during construction. + +Preserve ConfigKit public loading errors through wrapping for `errors.As`, with a +PromptKit configuration category. Use source-aware domain/resource failures where +ConfigKit is not the owner. Do not mirror its diagnostic framework or promise +aggregation beyond what the underlying operation provides. Runtime operational +errors remain distinct from generated-content validation failures. + +Prompt versions are domain identifiers, not ConfigKit `VersionRule` declarations. +Do not introduce an application schema-version policy merely to adopt ConfigKit. + +## Snapshot, Resource, And Execution Boundaries + +The engine captures effective definitions and referenced message text during +construction. Validate static template syntax then; render with request data +later. Resolve content references from the effective field's contributing source +context, not the last layer's directory. Ambiguous/contextless resource references +need an explicit source context or an actionable error, not an invented directory. + +ConfigKit returns passive resource contexts, not snapshots of referenced text. +PromptKit explicitly opens and closes its resource handles. Caller filesystem +ownership remains unchanged. Preserve appropriate source containment without +claiming that every `fs.FS` is a security sandbox. Captured reads are not an atomic +transaction over a concurrently changing filesystem. + +Definition changes require new engine construction. No watcher, background +refresh, or lazy compatibility mode is added. Applications own engine replacement; +prepared handles remain bound to their original engine and overlapping engines +retain separate capacity pools. + +Request artifacts and schema preparation remain invocation work. A request can +replace the output contract, so construction need not compile unused schemas. +Prepared execution continues to freeze its effective resources. Environment +credential lookup remains at the established execution/preparation boundary, +not frozen into the configuration snapshot. + +Ordinary execution uses indexed ready definitions and resolved presence values. +It performs no configuration discovery, parsing, inheritance traversal, or +provenance analysis. Context cancellation remains cooperative at controlled +boundaries; do not add detached work to pretend synchronous ConfigKit loading is +instantly interruptible. + +## Explicit Operator Inspection + +Add an API consumers can use to implement targeted CLI commands or debug output +for effective configuration, contributing sources, and explanations. Retain +ConfigKit inspection state with the engine's configuration snapshot and reuse +its redaction and provenance facilities. + +Inspection is explicitly invoked and uses that snapshot without reloading files, +opening resources, or contacting providers. It does not predict every runtime +override, reveal execution-time credential values, or replace prepared-run details +and execution observations. No provenance traversal, explanation generation, or +configuration-report rendering occurs during ordinary prompt runs. + +PromptKit ships no CLI and emits no automatic debug reports. Applications own +presentation, access control, and disclosure; redaction does not make all prompt +or configuration content safe to publish. Exact inspection API scope is an open +decision below. + +## Required Refactoring And Coordination + +- Replace profile/prompt metadata prepasses, selected-file decoding, and per-call + scans with ConfigKit collection loading and immutable indexed definitions. +- Remove the custom profile inheritance resolver and error-driven repository + overlay stack. Do not retain an old-format compatibility engine alongside the + new semantics. +- Retain only needed discovery and message-resource adapters; remove direct YAML + dependency usage once no remaining owner needs it. Preserve JSON Schema, + runtime JSON-value ownership, domain validation, and execution collaborators. +- Adapt maintained catalogs and PromptKit-owned definitions to the agreed keyed + format and released inheritance semantics. Preserve maintained name commitments, + backend ownership, and catalog namespace rules without upstream availability + checks. Coordinate necessary catalog releases before consuming their new format. + This document requests no immediate changes in those repositories. +- Keep configuration conversion, normalization, and inspection connected to + ConfigKit's lifecycle instead of reconstructing effective values/history after + loading. Translate once into execution-owned values with necessary copying. +- Update canonical formats, consumer guidance, GoDoc, internal inventory, examples, + architecture boundaries, dependencies, and OSS notices when implementation lands. + +## Efficiency And Validation Expectations + +Target linear or near-linear work relative to actual input, output, and retained +history, accounting separately for sorting, resource reads, and domain work. +Acquire/parse each accepted source occurrence once, avoid decoding intermediate +tiers, and index definition selection. Do not duplicate inheritance expansion or +copy complete accumulated collections for every added layer. + +Materialized inherited data and provenance can exceed source size. Exercise real +catalog shapes within ConfigKit's budgets; do not claim unconditional O(n) runtime +or memory. Prefer clear ownership and straightforward indexes over global caches +or elaborate optimization. Operator inspection may perform work proportional to +the requested report; that work remains off the execution path. + +Tests should use real ConfigKit with offline sources and synthetic provider +collaborators. PromptKit owns its adapter, domain, lifecycle, and execution tests, +not a duplicate ConfigKit parser/graph suite. Retire tests whose only purpose was +the removed loading machinery. Preserve meaningful execution regressions. + +Representative acceptance evidence must cover: + +- Minimal local-backend construction and execution without elaborate configuration. +- Embedded defaults plus operator and programmatic contributions, including + accurate inherited relative-resource origins. +- Profile inheritance, changing model/backend, canceling old relationships, and + discarding unwanted settings with documented semantics. +- Explicit zero/false/null behavior reaching execution, construction failures, + independent snapshots, and unchanged substantive preparation/execution features. +- Targeted inspection with useful redacted origins and no provenance work during + ordinary runs. + +Use focused scaling/allocation benchmarks for large collections, shared bases, +and many tiers where they expose repeated-work risks. Do not add timing-based +release gates or provider-network tests. + +## Migration And Completion + +Ship as an intentionally incompatible minor release with concrete migration +guidance. Cover keyed files, `base_profile` to `extends`, reserved-key behavior, +recursive maps, explicit null versus omission, changed error timing, programmatic +construction, and explicit engine replacement. Explain that overriding a model +does not discard lower endpoint, reasoning, provider, or inheritance settings. + +The end state is one shared configuration model with the old loading/resolution +machinery removed, useful operator inspection, and all substantive execution +workflows still available. The following decisions must be resolved before a +decision-complete implementation plan is written; implementation stages do not +belong in this feature roadmap. + +## Open Decisions + +### 1. Collection Format And Source Surface + +**Recommendation:** use `profiles.` and `prompts..` keyed mappings +in documents that can contribute multiple entries. Keys own identity rather than +duplicated `id`/`version` fields. Preserve omitted-version ambiguity when an ID has +multiple versions. Supply explicit source groups through a small construction +surface compatible with ConfigKit's `Input`/underlay concepts; retain convenient +file/directory/embedded entry points without parallel semantics. This fits sibling +inheritance and strict composition while minimizing identity reconciliation. + +**Alternative:** keep a keyed profile collection but represent each prompt ID as +an object with a nested `versions` collection. This offers room for ID-level +metadata and shared settings, but adds another level and decisions about which +fields inherit where. Neither format requires a single physical file. Choose one +format and migrate catalogs rather than sustaining both through compatibility code. + +### 2. Programmatic Ergonomics + +**Recommendation:** accept ConfigKit's labeled mapping sources and provide only +small typed PromptKit conveniences demonstrated by the minimal local-backend and +application-profile workflows. Such helpers must encode explicit presence and feed +the same loading lifecycle, not merge configurations or infer omission from zeros. +Keep HTTP/model clients and runtime backend collaborators outside generic decoding. + +**Alternative:** expose only mapping-based configuration plus runtime collaborators. +This minimizes conversion code and is entirely consistent with ConfigKit, but can +make common library use verbose and less discoverable. Compare concrete consumer +examples before deciding which typed helpers earn their maintenance cost. + +### 3. Partial Profiles, Nulls, And Effective Defaults + +**Recommendation:** allow schema-valid partial profiles without an `abstract` flag; +validate supplied domain values at loading and final runnable targets after profile +selection and runtime overrides. Preserve absent versus explicit null in resolved +settings. Treat null on optional provider settings as a deliberate suppression, +not permission to restore the same value from backend defaults; a null provider +map suppresses the configured lower map, while null entries inside that map remain +literal JSON null values. Framework-required settings such as timeouts need an +explicit field-by-field rule in the implementation plan, not blanket nil defaulting. +This allows reusable bases and meaningful removal of stale inherited settings. + +**Alternative:** require an explicit abstract-profile marker and define nullable +fields as returning to backend/framework defaults. This makes executable-profile +validation earlier and default recovery simple, but adds a configuration concept +and makes suppression harder. Demonstrate model/backend switching without awkward +reset tiers before choosing it. Neither option may redefine ConfigKit's merge rules. + +### 4. Literal `extends` Inside Provider Data + +**Recommendation:** reserve `extends` throughout file/mapping configuration as +ConfigKit does, while keeping runtime request provider-parameter maps ordinary +JSON-compatible data outside configuration parsing. Document this distinction; +applications needing that literal provider key can supply it at the runtime +boundary. This preserves outbound extensibility without adding a second parser or +pretending ConfigKit offers an escape syntax. + +**Alternative:** introduce an explicitly opaque JSON provider-parameter input for +configuration, decoded by PromptKit as domain payload after structural loading. +This would allow file-defined literal reserved keys, but adds a representation, +validation/redaction responsibilities, and awkward layering behavior. Pursue it +only if runtime supply does not satisfy concrete consumers; never silently rename +the outbound key. Audit maintained catalog payloads before finalizing this choice. + +### 5. Operator Inspection API + +**Recommendation:** offer a read-only inspection view backed by retained ConfigKit +state, with targeted path queries, source/history information, and explicit redacted +effective/explanation output. Reuse suitable public ConfigKit inspection values +instead of mirroring them. Do not expose mutable runtime definitions, resource +capabilities merely for diagnostics, or request credentials. This gives consumers +useful operator tooling with minimal duplicate machinery. + +**Alternative:** return narrow PromptKit-owned profile/prompt inspection records +containing only effective settings and selected origin summaries. This limits +public dependency coupling and is easier to constrain, but loses generic path +inspection and needs translation code as ConfigKit evolves. Define precisely +what is omitted rather than claiming those summaries are full provenance. + +### 6. Catalog Format Transition + +**Recommendation:** release new keyed catalog assets in the existing catalog +projects, preserving their data-only role and recording the format transition. +Consume those releases directly after checking manifest/namespace/ownership rules. +This removes long-term conversion machinery and aligns maintained and consumer +definitions. The catalog maintainers must choose the applicable release/version +handling under their own policies; PromptKit must not assume old consumers can +decode the changed shape. + +**Alternative:** publish new-format assets alongside legacy assets for a transition +period. This protects independently upgrading old consumers but temporarily +duplicates catalog maintenance and requires an explicit retirement decision. +Avoid a permanent PromptKit adapter that reparses and serializes legacy definitions +solely to postpone coordinated migration.