Plan ConfigKit adoption for PromptKit

docs/roadmap/configkit.md

@@ -0,0 +1,367 @@
+# ConfigKit Adoption
+
+Status: feature roadmap for design agreement. ConfigKit v0.5.0 supplies the
+required upstream capabilities; significant PromptKit choices are collected
+under Open Decisions. A separate implementation plan will define stages,
+signatures, and execution details. No implementation is authorized by this document.
+
+## Purpose
+
+Use ConfigKit to standardize PromptKit's configuration semantics and simplify
+its loading machinery, replacing custom logic wherever ConfigKit is the
+appropriate owner. Preserve substantive consumer capabilities, not historical
+file syntax, loading order, or Go signatures. Pre-1.0 breaking changes are
+acceptable and must have migration guidance.
+
+Applications using both libraries should share configuration interpretation,
+layering, inheritance, presence, and inspection semantics. PromptKit should
+concentrate on LLM-specific validation, resource interpretation, and execution.
+The [adoption assessment](configkit-assessment.md) records the supporting analysis;
+this roadmap owns the proposed feature end state and remaining choices.
+
+## Baseline And Dependency
+
+The current PromptKit runtime baseline is v0.11.2. Its
+[source implementation](../internal/sources.md), [format contract](../formats.md),
+and [consumer guide](../consumers/pkg-promptkit.md) describe existing behavior,
+not the target described here.
+
+The reviewed ConfigKit release is v0.5.0 at
+`dc09959cca4e105f49ec316c6d4ba585bd2340bd`. It provides native `fs.FS` sources,
+`LoadLayered`, source-backed underlays, labeled programmatic mappings, passive
+resource contexts, and built-in sibling inheritance. Its released configuration
+contract and GoDoc own exact upstream semantics. No further upstream feature is
+currently a demonstrated prerequisite. Review any newer version before selection.
+
+Adopt a released Go module, not a local `replace` or workspace requirement.
+Update attribution and transitive license notices under the OSS policy. This
+feature does not change either project's license or require a Go minimum increase
+relative to the reviewed versions.
+
+## Substantive Capabilities To Preserve
+
+- Named/versioned prompts, declared-input inspection, templates, inline and
+  referenced message content, variables, input helpers, roles, and cache control.
+- Maintained catalogs, application defaults, operator configuration, programmatic
+  profiles, aliases/inheritance, built-in/custom/local backends, and endpoint-only
+  profiles.
+- Preparation without generation, ordinary execution, and frozen single-claim
+  prepared execution; independent prompt and profile inspection.
+- Runtime execution-setting, reasoning, session, credential, and output-contract
+  overrides; appended already-rendered messages.
+- Inline/file input artifacts and injected artifact, model, and HTTP collaborators.
+- Backend capacity management, cancellation, transient retries, structural output
+  validation, provider-native structured output, bounded repair, and observations.
+- Results, effective identities/settings, usage, hashes, and meaningful errors.
+
+Retain optional environment credentials and direct request credentials. ConfigKit
+secret resolution must not introduce a new requirement that an upstream API key
+exist. Domain validation must not probe provider/model availability.
+
+## Target Configuration Model
+
+### One Shared Loading Lifecycle
+
+Construction acquires configuration contributions, uses ConfigKit to compose and
+overlay them, expands sibling inheritance, decodes the effective configuration,
+performs recorded domain resolution and validation, and produces execution-ready
+definitions. Publish no partially initialized engine on failure.
+
+ConfigKit owns generic parsing, combination, presence, origins, and inheritance.
+PromptKit owns source selection/discovery, definition indexing, backend policy,
+LLM-specific rules, and translation into runtime values. Runtime clients and
+callbacks remain normal Go dependencies, not YAML configuration types.
+
+Use keyed collections assembled before typed decoding rather than independently
+decoding definitions and combining them afterward. Multiple files can contribute
+to a collection; a monolithic file is not required. The exact collection layout
+and construction surface are open decisions below.
+
+Keep the public package a deliberate facade and implementation private. Do not
+expose internal engine/domain types or introduce a parallel generic loader merely
+to preserve old source options. ConfigKit's supported public types may be used
+where intentional integration is simpler than maintaining equivalent wrappers.
+
+### Sources, Tiers, And Presence
+
+Support ordinary files/directories, embedded or caller-supplied `fs.FS`, and
+programmatic definitions. Explicitly selected directories may be discovered by
+PromptKit; ConfigKit handles their configuration contributions. Do not infer
+priority from traversal order.
+
+Retain the useful profile priority categories, lowest to highest: external
+catalogs, PromptKit-owned profiles, application fallback, operator configuration,
+and programmatic configuration. Express intentional priority as ordered tiers.
+Within each tier, independent contributions compose strictly; overlapping
+non-mapping values conflict, even when equal. Disjoint fields under the same
+identity can combine. Across tiers, mappings recursively overlay.
+
+Use ConfigKit's source roles honestly: underlay groups are ordered defaults,
+main input overlays underlays, and optional named overlay/CLI facilities retain
+their upstream meaning. Underlay-only loading is supported; do not manufacture
+a main document or disguise programmatic values as CLI flags. Exposing every
+ConfigKit convenience through PromptKit is not a requirement.
+
+Adopt the shared value rules: omission retains lower input; explicit zero, false,
+empty string, sequence, and null replace; mappings combine recursively; empty
+mappings do not clear lower children. Null remains explicit and subject to the
+final schema. Provider-parameter maps obey these rules too. No hidden whole-profile
+replacement, arbitrary merge callback, or new reset/delete operator belongs here.
+
+Preserve presence into execution-ready settings so an explicit zero does not
+disappear during backend/default resolution or outbound request construction.
+Use ConfigKit metadata at construction, not as a per-run lookup mechanism.
+
+### Sibling Inheritance
+
+Use ConfigKit's reserved `extends` rather than PromptKit's `base_profile` and
+custom resolver. A string identifies an immediate sibling mapping; null cancels
+the relationship without deleting the child's own settings. Layering completes
+before inheritance. Only effective relationships resolve, but all of them do,
+including entries never selected for execution.
+
+Children recursively overlay resolved parents. Inherited values count as explicit
+input. Origins and resource context remain attached to their declaring source.
+Nested relationships follow ConfigKit's destination-local rules; defaults and
+application-derived values do not trigger another inheritance pass.
+
+The keyword is reserved throughout configuration, including dynamic provider
+maps; it is not opt-in. The roadmap must resolve the ordinary-data limitation
+without silently renaming outbound provider fields or dropping extensibility.
+
+### Validation And Errors
+
+Source acquisition, syntax, and strict-composition failures cannot be masked by
+higher tiers. Typed and semantic validation apply to effective values rather than
+every intermediate layer. An unused invalid effective definition fails construction.
+Separate catalog ownership/reserved-name/secret-safety checks remain source-policy
+responsibilities, not justification for duplicate generic decoding.
+
+Domain validators remain shared with runtime request validation where appropriate.
+Partial profile bases must satisfy the configuration schema, while requirements
+that depend on actual selection and overrides are checked at execution preparation.
+Do not require credentials, artifacts, or provider connectivity during construction.
+
+Preserve ConfigKit public loading errors through wrapping for `errors.As`, with a
+PromptKit configuration category. Use source-aware domain/resource failures where
+ConfigKit is not the owner. Do not mirror its diagnostic framework or promise
+aggregation beyond what the underlying operation provides. Runtime operational
+errors remain distinct from generated-content validation failures.
+
+Prompt versions are domain identifiers, not ConfigKit `VersionRule` declarations.
+Do not introduce an application schema-version policy merely to adopt ConfigKit.
+
+## Snapshot, Resource, And Execution Boundaries
+
+The engine captures effective definitions and referenced message text during
+construction. Validate static template syntax then; render with request data
+later. Resolve content references from the effective field's contributing source
+context, not the last layer's directory. Ambiguous/contextless resource references
+need an explicit source context or an actionable error, not an invented directory.
+
+ConfigKit returns passive resource contexts, not snapshots of referenced text.
+PromptKit explicitly opens and closes its resource handles. Caller filesystem
+ownership remains unchanged. Preserve appropriate source containment without
+claiming that every `fs.FS` is a security sandbox. Captured reads are not an atomic
+transaction over a concurrently changing filesystem.
+
+Definition changes require new engine construction. No watcher, background
+refresh, or lazy compatibility mode is added. Applications own engine replacement;
+prepared handles remain bound to their original engine and overlapping engines
+retain separate capacity pools.
+
+Request artifacts and schema preparation remain invocation work. A request can
+replace the output contract, so construction need not compile unused schemas.
+Prepared execution continues to freeze its effective resources. Environment
+credential lookup remains at the established execution/preparation boundary,
+not frozen into the configuration snapshot.
+
+Ordinary execution uses indexed ready definitions and resolved presence values.
+It performs no configuration discovery, parsing, inheritance traversal, or
+provenance analysis. Context cancellation remains cooperative at controlled
+boundaries; do not add detached work to pretend synchronous ConfigKit loading is
+instantly interruptible.
+
+## Explicit Operator Inspection
+
+Add an API consumers can use to implement targeted CLI commands or debug output
+for effective configuration, contributing sources, and explanations. Retain
+ConfigKit inspection state with the engine's configuration snapshot and reuse
+its redaction and provenance facilities.
+
+Inspection is explicitly invoked and uses that snapshot without reloading files,
+opening resources, or contacting providers. It does not predict every runtime
+override, reveal execution-time credential values, or replace prepared-run details
+and execution observations. No provenance traversal, explanation generation, or
+configuration-report rendering occurs during ordinary prompt runs.
+
+PromptKit ships no CLI and emits no automatic debug reports. Applications own
+presentation, access control, and disclosure; redaction does not make all prompt
+or configuration content safe to publish. Exact inspection API scope is an open
+decision below.
+
+## Required Refactoring And Coordination
+
+- Replace profile/prompt metadata prepasses, selected-file decoding, and per-call
+  scans with ConfigKit collection loading and immutable indexed definitions.
+- Remove the custom profile inheritance resolver and error-driven repository
+  overlay stack. Do not retain an old-format compatibility engine alongside the
+  new semantics.
+- Retain only needed discovery and message-resource adapters; remove direct YAML
+  dependency usage once no remaining owner needs it. Preserve JSON Schema,
+  runtime JSON-value ownership, domain validation, and execution collaborators.
+- Adapt maintained catalogs and PromptKit-owned definitions to the agreed keyed
+  format and released inheritance semantics. Preserve maintained name commitments,
+  backend ownership, and catalog namespace rules without upstream availability
+  checks. Coordinate necessary catalog releases before consuming their new format.
+  This document requests no immediate changes in those repositories.
+- Keep configuration conversion, normalization, and inspection connected to
+  ConfigKit's lifecycle instead of reconstructing effective values/history after
+  loading. Translate once into execution-owned values with necessary copying.
+- Update canonical formats, consumer guidance, GoDoc, internal inventory, examples,
+  architecture boundaries, dependencies, and OSS notices when implementation lands.
+
+## Efficiency And Validation Expectations
+
+Target linear or near-linear work relative to actual input, output, and retained
+history, accounting separately for sorting, resource reads, and domain work.
+Acquire/parse each accepted source occurrence once, avoid decoding intermediate
+tiers, and index definition selection. Do not duplicate inheritance expansion or
+copy complete accumulated collections for every added layer.
+
+Materialized inherited data and provenance can exceed source size. Exercise real
+catalog shapes within ConfigKit's budgets; do not claim unconditional O(n) runtime
+or memory. Prefer clear ownership and straightforward indexes over global caches
+or elaborate optimization. Operator inspection may perform work proportional to
+the requested report; that work remains off the execution path.
+
+Tests should use real ConfigKit with offline sources and synthetic provider
+collaborators. PromptKit owns its adapter, domain, lifecycle, and execution tests,
+not a duplicate ConfigKit parser/graph suite. Retire tests whose only purpose was
+the removed loading machinery. Preserve meaningful execution regressions.
+
+Representative acceptance evidence must cover:
+
+- Minimal local-backend construction and execution without elaborate configuration.
+- Embedded defaults plus operator and programmatic contributions, including
+  accurate inherited relative-resource origins.
+- Profile inheritance, changing model/backend, canceling old relationships, and
+  discarding unwanted settings with documented semantics.
+- Explicit zero/false/null behavior reaching execution, construction failures,
+  independent snapshots, and unchanged substantive preparation/execution features.
+- Targeted inspection with useful redacted origins and no provenance work during
+  ordinary runs.
+
+Use focused scaling/allocation benchmarks for large collections, shared bases,
+and many tiers where they expose repeated-work risks. Do not add timing-based
+release gates or provider-network tests.
+
+## Migration And Completion
+
+Ship as an intentionally incompatible minor release with concrete migration
+guidance. Cover keyed files, `base_profile` to `extends`, reserved-key behavior,
+recursive maps, explicit null versus omission, changed error timing, programmatic
+construction, and explicit engine replacement. Explain that overriding a model
+does not discard lower endpoint, reasoning, provider, or inheritance settings.
+
+The end state is one shared configuration model with the old loading/resolution
+machinery removed, useful operator inspection, and all substantive execution
+workflows still available. The following decisions must be resolved before a
+decision-complete implementation plan is written; implementation stages do not
+belong in this feature roadmap.
+
+## Open Decisions
+
+### 1. Collection Format And Source Surface
+
+**Recommendation:** use `profiles.<id>` and `prompts.<id>.<version>` keyed mappings
+in documents that can contribute multiple entries. Keys own identity rather than
+duplicated `id`/`version` fields. Preserve omitted-version ambiguity when an ID has
+multiple versions. Supply explicit source groups through a small construction
+surface compatible with ConfigKit's `Input`/underlay concepts; retain convenient
+file/directory/embedded entry points without parallel semantics. This fits sibling
+inheritance and strict composition while minimizing identity reconciliation.
+
+**Alternative:** keep a keyed profile collection but represent each prompt ID as
+an object with a nested `versions` collection. This offers room for ID-level
+metadata and shared settings, but adds another level and decisions about which
+fields inherit where. Neither format requires a single physical file. Choose one
+format and migrate catalogs rather than sustaining both through compatibility code.
+
+### 2. Programmatic Ergonomics
+
+**Recommendation:** accept ConfigKit's labeled mapping sources and provide only
+small typed PromptKit conveniences demonstrated by the minimal local-backend and
+application-profile workflows. Such helpers must encode explicit presence and feed
+the same loading lifecycle, not merge configurations or infer omission from zeros.
+Keep HTTP/model clients and runtime backend collaborators outside generic decoding.
+
+**Alternative:** expose only mapping-based configuration plus runtime collaborators.
+This minimizes conversion code and is entirely consistent with ConfigKit, but can
+make common library use verbose and less discoverable. Compare concrete consumer
+examples before deciding which typed helpers earn their maintenance cost.
+
+### 3. Partial Profiles, Nulls, And Effective Defaults
+
+**Recommendation:** allow schema-valid partial profiles without an `abstract` flag;
+validate supplied domain values at loading and final runnable targets after profile
+selection and runtime overrides. Preserve absent versus explicit null in resolved
+settings. Treat null on optional provider settings as a deliberate suppression,
+not permission to restore the same value from backend defaults; a null provider
+map suppresses the configured lower map, while null entries inside that map remain
+literal JSON null values. Framework-required settings such as timeouts need an
+explicit field-by-field rule in the implementation plan, not blanket nil defaulting.
+This allows reusable bases and meaningful removal of stale inherited settings.
+
+**Alternative:** require an explicit abstract-profile marker and define nullable
+fields as returning to backend/framework defaults. This makes executable-profile
+validation earlier and default recovery simple, but adds a configuration concept
+and makes suppression harder. Demonstrate model/backend switching without awkward
+reset tiers before choosing it. Neither option may redefine ConfigKit's merge rules.
+
+### 4. Literal `extends` Inside Provider Data
+
+**Recommendation:** reserve `extends` throughout file/mapping configuration as
+ConfigKit does, while keeping runtime request provider-parameter maps ordinary
+JSON-compatible data outside configuration parsing. Document this distinction;
+applications needing that literal provider key can supply it at the runtime
+boundary. This preserves outbound extensibility without adding a second parser or
+pretending ConfigKit offers an escape syntax.
+
+**Alternative:** introduce an explicitly opaque JSON provider-parameter input for
+configuration, decoded by PromptKit as domain payload after structural loading.
+This would allow file-defined literal reserved keys, but adds a representation,
+validation/redaction responsibilities, and awkward layering behavior. Pursue it
+only if runtime supply does not satisfy concrete consumers; never silently rename
+the outbound key. Audit maintained catalog payloads before finalizing this choice.
+
+### 5. Operator Inspection API
+
+**Recommendation:** offer a read-only inspection view backed by retained ConfigKit
+state, with targeted path queries, source/history information, and explicit redacted
+effective/explanation output. Reuse suitable public ConfigKit inspection values
+instead of mirroring them. Do not expose mutable runtime definitions, resource
+capabilities merely for diagnostics, or request credentials. This gives consumers
+useful operator tooling with minimal duplicate machinery.
+
+**Alternative:** return narrow PromptKit-owned profile/prompt inspection records
+containing only effective settings and selected origin summaries. This limits
+public dependency coupling and is easier to constrain, but loses generic path
+inspection and needs translation code as ConfigKit evolves. Define precisely
+what is omitted rather than claiming those summaries are full provenance.
+
+### 6. Catalog Format Transition
+
+**Recommendation:** release new keyed catalog assets in the existing catalog
+projects, preserving their data-only role and recording the format transition.
+Consume those releases directly after checking manifest/namespace/ownership rules.
+This removes long-term conversion machinery and aligns maintained and consumer
+definitions. The catalog maintainers must choose the applicable release/version
+handling under their own policies; PromptKit must not assume old consumers can
+decode the changed shape.
+
+**Alternative:** publish new-format assets alongside legacy assets for a transition
+period. This protects independently upgrading old consumers but temporarily
+duplicates catalog maintenance and requires an explicit retirement decision.
+Avoid a permanent PromptKit adapter that reparses and serializes legacy definitions
+solely to postpone coordinated migration.