Finalize ConfigKit adoption decisions

docs/roadmap/configkit.md

@@ -1,8 +1,7 @@
 # ConfigKit Adoption
 
-Status: feature roadmap for design agreement. ConfigKit v0.5.0 supplies the
+Status: agreed feature roadmap. ConfigKit v0.5.0 supplies the
-required upstream capabilities; significant PromptKit choices are collected
+required upstream capabilities. A separate implementation plan will define stages,
-under Open Decisions. A separate implementation plan will define stages,
 signatures, and execution details. No implementation is authorized by this document.
 
 ## Purpose
@@ -13,11 +12,19 @@ appropriate owner. Preserve substantive consumer capabilities, not historical
 file syntax, loading order, or Go signatures. Pre-1.0 breaking changes are
 acceptable and must have migration guidance.
 
+**This is an intentional breaking change. We are intentionally replacing PromptKit's configuration handling code in favor of adopting ConfigKit's simpler, standardized API. Our goal is to arrive at a new version of PromptKit that is leaner, simpler, and uses a ConfigKit-standard configuration syntax. We MUST NOT give any weight to preserving arbitrary implementation details from PromptKit's current codebase--those should be discarded in favor of the simplest viable implementation that supports PromptKit's substantive LLM-focused consumer capabilities.**
+
 Applications using both libraries should share configuration interpretation,
 layering, inheritance, presence, and inspection semantics. PromptKit should
 concentrate on LLM-specific validation, resource interpretation, and execution.
 The [adoption assessment](configkit-assessment.md) records the supporting analysis;
-this roadmap owns the proposed feature end state and remaining choices.
+this roadmap owns the agreed feature end state. Where the earlier assessment
+differs, this roadmap and the maintainer's direction above take precedence.
+
+Do not preserve source-option wrappers, repository interfaces, incidental error
+ordering, or old field shapes solely because they exist today. Reuse existing
+code only when it remains the simplest appropriate owner of a substantive
+capability. Migration documentation replaces compatibility machinery, not vice versa.
 
 ## Baseline And Dependency
 
@@ -72,10 +79,23 @@ PromptKit owns source selection/discovery, definition indexing, backend policy,
 LLM-specific rules, and translation into runtime values. Runtime clients and
 callbacks remain normal Go dependencies, not YAML configuration types.
 
-Use keyed collections assembled before typed decoding rather than independently
+Use `profiles.<id>` and `prompts.<id>.<version>` keyed mappings assembled before
-decoding definitions and combining them afterward. Multiple files can contribute
+typed decoding rather than independently decoding definitions and combining them
-to a collection; a monolithic file is not required. The exact collection layout
+afterward. Keys own identity; do not duplicate `id`/`version` fields. These paths
-and construction surface are open decisions below.
+describe mapping levels, not dot-splitting of literal IDs or version keys.
+Multiple files can contribute to a collection; a monolithic file is not required.
+When a prompt ID has multiple versions, selection without a version remains
+ambiguous. This shape fits sibling inheritance without extra identity reconciliation.
+
+Supply explicit source groups through a small construction surface compatible
+with ConfigKit's `Input`/underlay concepts. Offer convenient file/directory/embedded
+entry points where useful, not a parallel set of historical loading semantics.
+
+Accept ConfigKit's labeled mapping sources for programmatic configuration. Add
+only small typed conveniences justified by the minimal local-backend and
+application-profile workflows. Helpers encode explicit presence and feed the same
+loading lifecycle; they neither merge configurations nor infer omission from zeros.
+HTTP/model clients and other runtime collaborators remain outside typed configuration.
 
 Keep the public package a deliberate facade and implementation private. Do not
 expose internal engine/domain types or introduce a parallel generic loader merely
@@ -126,8 +146,11 @@ Nested relationships follow ConfigKit's destination-local rules; defaults and
 application-derived values do not trigger another inheritance pass.
 
 The keyword is reserved throughout configuration, including dynamic provider
-maps; it is not opt-in. The roadmap must resolve the ordinary-data limitation
+maps; it is not opt-in. Keep runtime request provider-parameter maps ordinary
-without silently renaming outbound provider fields or dropping extensibility.
+JSON-compatible data outside configuration parsing. Consumers needing a literal
+provider field named `extends` supply it at that runtime boundary. Do not rename
+the outbound key or introduce a second opaque configuration format to escape it.
+Document and test this distinction, including any affected maintained payloads.
 
 ### Validation And Errors
 
@@ -138,10 +161,19 @@ Separate catalog ownership/reserved-name/secret-safety checks remain source-poli
 responsibilities, not justification for duplicate generic decoding.
 
 Domain validators remain shared with runtime request validation where appropriate.
-Partial profile bases must satisfy the configuration schema, while requirements
+Allow schema-valid partial profiles without an `abstract` flag. Validate supplied
-that depend on actual selection and overrides are checked at execution preparation.
+domain values at loading; final runnable-target requirements are checked after
+profile selection and runtime overrides at execution preparation.
 Do not require credentials, artifacts, or provider connectivity during construction.
 
+Preserve absence versus explicit null in resolved settings. Null on an optional
+provider setting deliberately suppresses it; it must not restore the same setting
+from backend defaults. A null provider map suppresses the configured lower map,
+while null entries inside a provider map remain literal JSON null values. This
+supports reusable bases and deliberate removal of stale settings without changing
+ConfigKit's merge rules. Framework-required settings such as timeouts need explicit
+field-by-field rules in the implementation plan rather than blanket nil defaulting.
+
 Preserve ConfigKit public loading errors through wrapping for `errors.As`, with a
 PromptKit configuration category. Use source-aware domain/resource failures where
 ConfigKit is not the owner. Do not mirror its diagnostic framework or promise
@@ -195,10 +227,17 @@ override, reveal execution-time credential values, or replace prepared-run detai
 and execution observations. No provenance traversal, explanation generation, or
 configuration-report rendering occurs during ordinary prompt runs.
 
+Expose a read-only inspection view backed by retained ConfigKit state, with
+targeted path queries, source/history information, and explicit redacted effective
+and explanation output. Reuse suitable public ConfigKit inspection values rather
+than mirroring them. Do not expose mutable runtime definitions, resource capabilities
+merely for diagnostics, or request credentials. Exact Go signatures belong in the
+implementation plan; the scope is generic targeted inspection, not just narrow
+profile/prompt origin summaries.
+
 PromptKit ships no CLI and emits no automatic debug reports. Applications own
 presentation, access control, and disclosure; redaction does not make all prompt
-or configuration content safe to publish. Exact inspection API scope is an open
+or configuration content safe to publish.
-decision below.
 
 ## Required Refactoring And Coordination
 
@@ -214,6 +253,11 @@ decision below.
   format and released inheritance semantics. Preserve maintained name commitments,
   backend ownership, and catalog namespace rules without upstream availability
   checks. Coordinate necessary catalog releases before consuming their new format.
+  Publish new keyed assets directly in the existing data-only catalog projects,
+  recording the format transition rather than maintaining duplicate legacy assets
+  or a permanent conversion adapter. Catalog maintainers select release/version
+  handling under their own policies; old consumers must not be assumed capable
+  of decoding the new shape. Pin the compatible released versions in PromptKit.
   This document requests no immediate changes in those repositories.
 - Keep configuration conversion, normalization, and inspection connected to
   ConfigKit's lifecycle instead of reconstructing effective values/history after
@@ -266,102 +310,7 @@ does not discard lower endpoint, reasoning, provider, or inheritance settings.
 
 The end state is one shared configuration model with the old loading/resolution
 machinery removed, useful operator inspection, and all substantive execution
-workflows still available. The following decisions must be resolved before a
+workflows still available. The six previously open design choices have been
-decision-complete implementation plan is written; implementation stages do not
+approved and incorporated into the owning sections above. Field-level details and signatures
-belong in this feature roadmap.
+belong in the decision-complete implementation plan, not a compatibility-preserving
-
+reinterpretation of this roadmap.
-## Open Decisions
-
-### 1. Collection Format And Source Surface
-
-**Recommendation:** use `profiles.<id>` and `prompts.<id>.<version>` keyed mappings
-in documents that can contribute multiple entries. Keys own identity rather than
-duplicated `id`/`version` fields. Preserve omitted-version ambiguity when an ID has
-multiple versions. Supply explicit source groups through a small construction
-surface compatible with ConfigKit's `Input`/underlay concepts; retain convenient
-file/directory/embedded entry points without parallel semantics. This fits sibling
-inheritance and strict composition while minimizing identity reconciliation.
-
-**Alternative:** keep a keyed profile collection but represent each prompt ID as
-an object with a nested `versions` collection. This offers room for ID-level
-metadata and shared settings, but adds another level and decisions about which
-fields inherit where. Neither format requires a single physical file. Choose one
-format and migrate catalogs rather than sustaining both through compatibility code.
-
-### 2. Programmatic Ergonomics
-
-**Recommendation:** accept ConfigKit's labeled mapping sources and provide only
-small typed PromptKit conveniences demonstrated by the minimal local-backend and
-application-profile workflows. Such helpers must encode explicit presence and feed
-the same loading lifecycle, not merge configurations or infer omission from zeros.
-Keep HTTP/model clients and runtime backend collaborators outside generic decoding.
-
-**Alternative:** expose only mapping-based configuration plus runtime collaborators.
-This minimizes conversion code and is entirely consistent with ConfigKit, but can
-make common library use verbose and less discoverable. Compare concrete consumer
-examples before deciding which typed helpers earn their maintenance cost.
-
-### 3. Partial Profiles, Nulls, And Effective Defaults
-
-**Recommendation:** allow schema-valid partial profiles without an `abstract` flag;
-validate supplied domain values at loading and final runnable targets after profile
-selection and runtime overrides. Preserve absent versus explicit null in resolved
-settings. Treat null on optional provider settings as a deliberate suppression,
-not permission to restore the same value from backend defaults; a null provider
-map suppresses the configured lower map, while null entries inside that map remain
-literal JSON null values. Framework-required settings such as timeouts need an
-explicit field-by-field rule in the implementation plan, not blanket nil defaulting.
-This allows reusable bases and meaningful removal of stale inherited settings.
-
-**Alternative:** require an explicit abstract-profile marker and define nullable
-fields as returning to backend/framework defaults. This makes executable-profile
-validation earlier and default recovery simple, but adds a configuration concept
-and makes suppression harder. Demonstrate model/backend switching without awkward
-reset tiers before choosing it. Neither option may redefine ConfigKit's merge rules.
-
-### 4. Literal `extends` Inside Provider Data
-
-**Recommendation:** reserve `extends` throughout file/mapping configuration as
-ConfigKit does, while keeping runtime request provider-parameter maps ordinary
-JSON-compatible data outside configuration parsing. Document this distinction;
-applications needing that literal provider key can supply it at the runtime
-boundary. This preserves outbound extensibility without adding a second parser or
-pretending ConfigKit offers an escape syntax.
-
-**Alternative:** introduce an explicitly opaque JSON provider-parameter input for
-configuration, decoded by PromptKit as domain payload after structural loading.
-This would allow file-defined literal reserved keys, but adds a representation,
-validation/redaction responsibilities, and awkward layering behavior. Pursue it
-only if runtime supply does not satisfy concrete consumers; never silently rename
-the outbound key. Audit maintained catalog payloads before finalizing this choice.
-
-### 5. Operator Inspection API
-
-**Recommendation:** offer a read-only inspection view backed by retained ConfigKit
-state, with targeted path queries, source/history information, and explicit redacted
-effective/explanation output. Reuse suitable public ConfigKit inspection values
-instead of mirroring them. Do not expose mutable runtime definitions, resource
-capabilities merely for diagnostics, or request credentials. This gives consumers
-useful operator tooling with minimal duplicate machinery.
-
-**Alternative:** return narrow PromptKit-owned profile/prompt inspection records
-containing only effective settings and selected origin summaries. This limits
-public dependency coupling and is easier to constrain, but loses generic path
-inspection and needs translation code as ConfigKit evolves. Define precisely
-what is omitted rather than claiming those summaries are full provenance.
-
-### 6. Catalog Format Transition
-
-**Recommendation:** release new keyed catalog assets in the existing catalog
-projects, preserving their data-only role and recording the format transition.
-Consume those releases directly after checking manifest/namespace/ownership rules.
-This removes long-term conversion machinery and aligns maintained and consumer
-definitions. The catalog maintainers must choose the applicable release/version
-handling under their own policies; PromptKit must not assume old consumers can
-decode the changed shape.
-
-**Alternative:** publish new-format assets alongside legacy assets for a transition
-period. This protects independently upgrading old consumers but temporarily
-duplicates catalog maintenance and requires an explicit retirement decision.
-Avoid a permanent PromptKit adapter that reparses and serializes legacy definitions
-solely to postpone coordinated migration.