eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Accept the ConfigKit configuration boundary and align the implementation plan

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-30 01:59:30 +00:00
parent 80131fa160
commit 5f7b19403f
3 changed files with 240 additions and 116 deletions
Download Patch File Download Diff File Expand all files Collapse all files

224
docs/roadmap/implementation.md
View File

@@ -1,8 +1,8 @@
# ConfigKit Adoption Implementation Plan
Status: ready for staged implementation, except for the explicitly identified
catalog-validation decision under Open Questions. This is a planning document;
writing it does not implement the feature or authorize publishing dependencies.
Status: ready for staged implementation under the agreed configuration boundary.
This is a planning document; writing it does not implement the feature or
authorize publishing dependencies.
## Authority And Working Method
@@ -10,6 +10,12 @@ The [ConfigKit feature roadmap](configkit.md) owns the intended end state and
approved policy. Its intentional-breaking-change direction is authoritative:
preserve substantive prompting capabilities, not the old loader, signatures,
source wrappers, incidental error ordering, or configuration syntax.
The [configuration-boundary ADR](../adr/0001-configuration-boundary.md) records
the ownership decision: PromptKit interprets and validates the final effective
configuration supplied by ConfigKit. Presence/null information and effective
resource context are part of that handoff. Source history serves explicit
operator inspection, not validation or catalog enforcement. This applies to
tests as well as production code.
Implement the stages below in order, one stage per coding-agent prompt. Each
stage includes its tests and affected current-state documentation. Read
@@ -37,9 +43,12 @@ Planning baselines:
Use ConfigKit v0.5.0 for this work. Do not silently upgrade to a different
contract. The catalog asset migration needs coordinated releases, not a committed
workspace or local replacement. Exact catalog release versions are selected at
publication under their existing date-based policy; record the versions and full
commits actually consumed. Publishing any repository requires separate authority.
workspace or local replacement. Changing the backend asset contract is broader
than the catalog profile-data compatibility exception: migrate the catalogs to
`/v3` with `v3.YYYYMMDD.N` tags, preserving their date/revision convention and
updating their own release documentation accordingly. Select exact dates and
unused revisions at publication; record versions and full commits consumed.
Publishing any repository requires separate authority.
## Implementation Decisions
@@ -83,9 +92,10 @@ Use one ConfigKit filesystem namespace per helper invocation and root resource
contexts at the supplied directory. Single files use ConfigKit directly.
Remove `WithPromptFS`, `WithPromptFile`, `WithProfileFS`, `WithProfileFile`,
`WithFallbackProfileFS`, and `WithProfiles` at public cutover. Retain schema source
options and collaborator/backend injection. Do not require any prompts to exist
merely to construct an engine for profile inspection.
`WithFallbackProfileFS`, `WithProfiles`, and `WithBackend` at public cutover.
Retain schema source options and runtime collaborator injection. Backend
configuration moves into ConfigKit inputs as specified below. Do not require any
prompts to exist merely to construct an engine for profile inspection.
Programmatic configuration uses labeled `configkit.Mapping` inputs. Replace the
old zero-sentinel `Profile`/`OpenAICompatibleProfileConfig` pair with one small
@@ -109,13 +119,36 @@ clear operation. Use raw Mapping for explicit null, prompt definitions, or riche
configuration. Do not introduce a public generic optional-value type just for
this helper. Convert fields explicitly to ConfigKit-supported built-in values,
then call Mapping for snapshot ownership and load-time errors. Do not marshal
through YAML or infer absence from a value. Keep the existing local-backend
constructor, which configures runtime backend registration rather than profiles.
through YAML or infer absence from a value. Helpers encode values; domain
validation occurs on the final configuration at handoff.
Replace the registration-only Backend value with a presence-aware convenience:
```go
type BackendConfig struct {
Endpoint, APIKeyEnv *string
ConcurrencyLimit, QueueCapacity *int
ExtraParams map[string]any
}
func BackendSource(label string, backends map[string]BackendConfig) configkit.Source
func LocalBackend(endpoint string, concurrencyLimit int) BackendConfig
```
Use the same omission/pointer/map conventions as ProfileSource. LocalBackend
supplies endpoint and concurrency pointers and leaves the remaining fields
omitted; callers choose the ID as the map key. Neither helper validates settings
before ConfigKit processes them. For example, place BackendSource with
`map[string]BackendConfig{"local": LocalBackend(endpoint, 4)}` in an application
underlay. Raw Mapping supports explicit null and combined backend/profile/prompt
documents. Remove the old Backend registration value and its reservation rules;
retain useful built-in ID constants. All backend contributions follow the same
strict-composition and overlay rules as other configuration.
### Typed Configuration And Execution Values
Add `internal/configuration` as the cohesive ConfigKit adapter. Its private
document DTO has `profiles: map[string]profileConfig` and
document DTO has `backends: map[string]backendConfig`,
`profiles: map[string]profileConfig`, and
`prompts: map[string]map[string]promptConfig`. Collection entries are mappings,
not nullable definition pointers. No duplicate identity fields; ConfigKit consumes
`extends`, so no DTO field or second resolver handles it. Use `configkit.Keys`,
@@ -128,6 +161,27 @@ private execution-owned fields or presence bits, not retained metadata queries.
Runtime/domain packages must not import ConfigKit. Do not duplicate its tree,
merge engine, inheritance graph, diagnostics, or presence framework.
Backend DTO fields match BackendConfig using `endpoint`, `api_key_env`,
`concurrency_limit`, `queue_capacity`, and `extra_params`. Validate the final
endpoint as required, absolute HTTP(S) with the existing URL rules. An absent or
null API-key environment name means no lookup; a supplied name follows existing
normalization and naming rules. Absent/null concurrency means zero (unlimited);
supplied values must be nonnegative. Absent/null queue capacity selects the
existing default of 1024 for positive concurrency and no queue for unlimited
concurrency. Explicit queue capacity, including zero, requires positive
concurrency and must satisfy existing nonnegative/overflow checks. Null provider
maps mean no configured extra parameters; entries may contain JSON null.
Required endpoint failures, unknown fields, and domain errors apply to final
definitions, including unused ones. Registry construction consumes these final
normalized values; it does not replay registration or source-reservation checks.
Use ConfigKit's Resolve/Validate callbacks only on its effective post-layering,
post-inheritance typed values. These public lifecycle hooks implement the domain
handoff and record normalization; they are not hooks for inspecting earlier
contributions. Final presence lookup and resource-context lookup are sufficient
for interpretation. History/Explain/HistoricalPaths are operator inspection
facilities, not inputs to construction-time policy checks.
Use the existing prompt fields under each keyed version, without `id`/`version`.
Use pointers for `content`, `content_file`, and optional clearable strings;
preserve existing input, cache-control, output-contract, and role rules.
@@ -169,13 +223,15 @@ derived endpoint/key/defaults must not follow a child that changes backend.
Provider map null entries are literal JSON null; empty mappings retain children.
For backend-to-profile provider-map overlay, use a narrow JSON-value merge helper
with the same recursive-map/value-replacement semantics; it is not a configuration
loader and must not interpret `extends` in backend runtime values.
loader. Both backend and profile configuration maps have already passed through
ConfigKit's reserved-key semantics; only request-level maps bypass that lifecycle.
Record any defaults/normalizations written into the typed configuration via
ConfigKit Resolve, with dependencies on relevant configuration paths. Do not
project runtime collaborators or credentials into the configuration tree just to
report them. Inspection describes resolved configuration, while InspectProfile
and preparation describe backend-completed execution settings. Make this boundary
report them. Inspection includes effective backend configuration. It describes
resolved configuration, while InspectProfile and preparation describe
backend-completed execution settings. Make this boundary
explicit rather than inventing synthetic provenance for registry defaults.
Retain APIKeyRequired's direct-key workflow: true suppresses a backend-derived
@@ -237,8 +293,9 @@ ConfigKit SourceContext, mutable runtime values, or internal types. Delegate
queries and redaction to the retained result. Nil Engine returns ErrInvalidConfig;
the zero inspection view has empty query results and writer operations return
ErrInvalidConfig. Writer errors remain inspectable. Mark credential-shaped dynamic
keys sensitive using Definition.Secret, reusing the existing catalog secret-key
vocabulary; do not redact `api_key_env` names as credential values. This is not a
keys sensitive using Definition.Secret; the existing secret-key vocabulary can
serve as a redaction predicate, not a source-rejection rule. Do not redact
`api_key_env` names as credential values. This is not a
promise that arbitrary prompt/provider content is safe to disclose.
Keep the runner independent of this view. Inspection calls perform no discovery,
@@ -253,18 +310,22 @@ and its focused tests. Leave engine assembly on the old path for this stage.
1. Add ConfigKit v0.5.0; update go.sum and factual OSS notices for it and its
selected transitive dependencies. Preserve project licenses and the Go floor.
2. Add the keyed DTOs and a private load entry point accepting ConfigKit Inputs
and underlays. Call LoadLayered once, with no VersionRule or secret resolution
bindings for API credentials. Use callback-scoped Resolve/Validate correctly.
3. Implement presence extraction and the field rules above. Validate supplied
domain values through existing domain helpers; no final target requirement.
2. Add keyed backend/profile/prompt DTOs and a private load entry point accepting
ConfigKit Inputs and underlays. Call LoadLayered once, with no VersionRule or
secret resolution bindings for API credentials. Use callback-scoped
Resolve/Validate correctly.
3. Implement effective presence extraction and the field rules above. Validate
final domain values through existing helpers, resolving backend references
against the final backend map; profiles have no final target requirement yet.
Record normalization/default mutations, not arbitrary DTO bookkeeping fields.
4. Preserve `*configkit.LoadError` through wrapping. Domain callback messages
identify paths/problems without interpolating raw values or credentials.
5. Add representative real-loader tests: keyed literal IDs, strict group conflict,
recursive cross-tier overlay, a shared base with explicit zero/null/false, an
unused invalid effective definition, and a malformed lower source not hidden
by an overlay. Do not reproduce ConfigKit's parser/inheritance matrix.
unused invalid effective definition, and preservation of a ConfigKit load
error. Cover an invalid lower-layer domain value replaced by a valid final
value: construction succeeds. Do not reproduce ConfigKit's parser/inheritance
matrix or add preflight/source-policy tests.
Exit: focused tests and `go test ./...` pass; old engine behavior is not presented
as migrated. Internal inventory documents only the loader now actually present.
@@ -297,8 +358,9 @@ existing execution tests still work through the old assembly until cutover.
Scope: domain settings, target resolution, wire-presence propagation, profile
inspection. No new source discovery in execution packages.
1. Add execution-owned presence/suppression state and construct indexed profiles
from loader output plus the immutable backend registry. Apply registry-derived
1. Build the immutable backend registry from the final loaded backend map. Add
execution-owned presence/suppression state and construct indexed profiles
from loader output plus that registry. Apply registry-derived
values after inheritance, using the field table above.
2. Replace scalar-zero/blank merge assumptions in target resolution. Configuration
explicit zero reaches injected clients and the built-in request body; null does
@@ -319,8 +381,8 @@ a compatibility promise. Update affected target/presence GoDoc with this change.
## Stage 4 — Migrate The OpenRouter Catalog Assets
Scope: `../promptkit-backend-openrouter` only. Resolve the catalog-validation open
question before implementing its consumption strategy in Stage 6.
Scope: `../promptkit-backend-openrouter` only. Catalog maintainers own source and
publishing checks here; PromptKit consumes its configuration through ConfigKit.
1. Read that repository's instructions; preserve unrelated work. Convert each
profile asset to `profiles: {literal-id: {settings...}}`, preserving inventory
@@ -330,22 +392,29 @@ question before implementing its consumption strategy in Stage 6.
2. Audit zero/empty values that previously meant unspecified. Remove those when
omission was intended; retain intentional explicit values. Review any dynamic
provider `extends` keys rather than silently changing outbound payloads.
3. Leave backend.json and its schema unchanged. Update catalog format notes and
applicable tests to the keyed format, with a prominent incompatible-format
migration note. Model availability is not probed.
3. Replace `catalog/backend.json` with `catalog/backend.yml` containing a
`backends: {openrouter: {settings...}}` mapping. Preserve endpoint, API-key
environment name, concurrency, queue, and extra-parameter values; remove the
duplicate id and old manifest schema_version. The file is an ordinary ConfigKit
input, not a separately decoded manifest. Update embedded assets, module path
and imports to `/v3`, tests, and catalog format/release documentation, with an
incompatible-format migration note. Keep FS/Root as the minimal export surface.
Catalog-local publishing checks stay in that repository. No provider probes.
4. Validate under its procedure and commit. If publication is authorized, publish
the next unused `v2.YYYYMMDD.N`; otherwise report the exact commit awaiting
the next unused `v3.YYYYMMDD.N`; otherwise report the exact commit awaiting
publication. The date/revision is selected then, not guessed in this plan.
Exit: one reviewed migrated catalog commit; existing immutable tags still serve
old consumers. The profile-data change does not change the Go module path/API or
the JSON backend manifest. Warn old loaders not to consume the new assets.
old consumers. The new major module path identifies the changed asset contract;
old consumers retain the published `/v2` assets.
## Stage 5 — Migrate The Rakestrawhome Catalog Assets
Scope: `../promptkit-backend-rakestrawhome` only. Apply the same keyed-format,
presence, notice, validation, and publication treatment as Stage 4 to its smaller
inventory. Preserve backend endpoint, credentials metadata, and capacity policy.
inventory, including the `/v3` transition and keyed `backends.rakestrawhome`
definition in `catalog/backend.yml`. Preserve backend endpoint, credentials
metadata, and capacity policy.
Do not couple its date/revision to OpenRouter's release.
Exit: both compatible catalog versions are published and ordinarily resolvable
@@ -359,18 +428,22 @@ Scope: `internal/catalog`, maintained definitions, catalog integration tests,
dependency pins/notices. Keep this separate from public API migration.
1. Pin the published compatible catalog versions and exact attribution commits.
Keep strict backend.json loading/validation and registry construction.
2. Replace catalog profile repositories with ConfigKit file Inputs for the first
Update imports to `/v3`. Replace manifest decoding with ConfigKit file Inputs.
2. Supply backend and profile assets as ConfigKit Inputs in the first
underlay. Supply PromptKit-owned keyed Mapping definitions as the next group:
standard extends the selected catalog's current maintained base; auto retains
its current direct model/backend definition. Preserve both literal names.
3. Implement the selected catalog source-policy strategy from Open Questions.
Preserve reserved-name, ownership, connection-metadata, and secret checks;
consumers may still override catalog profiles and their bases. Do not apply
catalog-only namespace restrictions to application sources.
3. Retire source-layout allowlists, catalog preflight, source ownership/reserved
prefix checks, connection-field prohibitions, and secret-key source rejection.
The adapter selects known asset entry points and discovers configuration files;
ConfigKit acquires their contents. Domain validation sees only the final
assembled configuration. Consumers may overlay catalog backends, profiles,
and their bases through the same configuration model.
4. Test real imported assets offline with the new loader, maintained-name
resolution, consumer override of a maintained base, and synthetic malformed
catalog fixtures for the source-policy risks. No upstream endpoint probes.
resolution, consumer override of a maintained base/backend, and rejection of
an invalid final backend. Test a replaced lower-layer domain error succeeds.
Do not independently validate catalog-only intermediate configurations, parse
raw assets in PromptKit tests, or audit origins to establish ownership.
Exit: catalog and owned definitions participate in the new keyed configuration
model without a legacy profile parser on that path. Catalog integration tests
@@ -384,8 +457,9 @@ adapters, and migration of their direct test setup.
1. Introduce the Construction Surface exactly as specified above. Remove the
superseded source/profile options and convenience types rather than keeping
deprecated pass-through wrappers. Keep intentional runtime injection APIs.
2. Assemble registry, combined configuration snapshot, renderer, validator,
capacity manager, and runner. Return no Engine on construction failure.
2. Load the combined backend/profile/prompt configuration, construct its runtime
registry and indexed snapshot, then assemble renderer, validator, capacity
manager, and runner. Return no Engine on construction failure.
Translate construction errors to ErrInvalidConfig while preserving LoadError
via errors.As. Resource errors include semantic path and safe source context.
3. Connect indexed prompt/profile lookups to preparation and inspection. Missing
@@ -393,7 +467,8 @@ adapters, and migration of their direct test setup.
selection error. Old parse/load error categories need not be manufactured for
errors now detected at construction.
4. Move root test fixtures/setup and maintained example construction to ConfigKit
sources, keyed definitions, and ProfileSource so all consumers still compile.
sources, keyed definitions, ProfileSource, and BackendSource so all consumers
still compile. Update local-backend examples to choose their ID as a map key.
Update expected timing only for the deliberate eager-validation change. Keep
behavioral assertions for prompting, overrides, and errors.
5. Add public tests for a minimal custom local backend and programmatic profile,
@@ -459,7 +534,8 @@ Scope: maintained examples and user-facing documentation.
complete parser/inheritance contract.
3. Update README and `docs/consumers/pkg-promptkit.md` with minimal local-backend,
file/embedded source, programmatic Mapping/ProfileSource, explicit inspection,
and engine-replacement workflows. Explain main versus ordered underlays.
and engine-replacement workflows. Explain main versus ordered underlays and
backend layering in place of registration-only restrictions.
4. Add concise old-to-new migration guidance in the consumer guide: removed API
options, exact IDs, `base_profile` to `extends`, zero/false/null, recursive maps,
no clear via `{}`, reserved `extends` runtime escape, failure timing, and the
@@ -478,14 +554,17 @@ internal architectural documentation.
1. Delete lazy prompt/profile metadata prepasses, filesystem repositories,
fallback-on-error overlays, custom base-chain resolver/depth cap, zero-sentinel
profile merging, and temporary stage adapters once no caller needs them.
Remove the separate backend-manifest decoder, source-policy audits, and
pre-handoff backend registration validation. Keep final-value domain checks.
Retain narrow lookup interfaces only where they still simplify real consumers.
2. Retain useful filecatalog/schema utilities and JSON-value ownership/validation.
Remove direct yaml.v3 use and dependency if no remaining owner requires it;
do not claim all YAML dependencies disappear when ConfigKit still uses YAML.
3. Move meaningful regressions from deleted package tests to their new owning
boundary. Retire tests only about old internals, repeated parser coverage, or
superseded semantics. Do not replace lost runtime protection with loader-only
tests. Keep private assembly checks separate from external facade tests.
boundary. Retire tests about old internals, repeated parser coverage,
source-policy enforcement, or superseded semantics. Do not replace lost runtime
protection with loader-only tests. Keep private assembly checks separate from
external facade tests.
4. Update `docs/internal/overview.md`, `docs/internal/sources.md`, relevant renderer
and engine documents, and the implemented-structure paragraphs in architecture
policy. Describe the new package/dependency boundary without turning names or
@@ -523,45 +602,6 @@ Exit: the feature roadmap's end state is demonstrably implemented, with one
configuration lifecycle, explicit operator inspection, no legacy loader, and
no newly invented provider checks or human release gates.
## Open Questions
### Catalog Source-Policy Validation Versus Single-Pass Loading
ConfigKit v0.5.0 performs typed callbacks only after effective overlay/inheritance;
it does not expose a raw-source policy callback. PromptKit currently checks each
catalog's resolved backend ownership and prohibited fields before consumer
overlays. Checking only the effective merged configuration would allow consumers
to mask a broken catalog contribution. Conversely, independently loading every
catalog and then loading the original sources again duplicates parsing and
inheritance work. Feeding the first load back as Mapping loses original source
provenance and changes when inheritance occurs. Neither workaround should be
silently described as the agreed single-pass design.
**Recommendation:** validate immutable catalog-only backend ownership and
source-policy rules in PromptKit's offline dependency contract tests, using real
ConfigKit and the selected released assets. At engine construction, retain
backend-manifest validation, strict source composition, effective domain
validation, and a history-based structural audit for reserved profile names,
forbidden connection fields, and prohibited secret-shaped keys contributed by
catalog sources (including masked paths). Use ConfigKit source IDs/history to
distinguish catalog contributions from permitted consumer overrides. Do not
parse Explanation YAML or rebuild an inheritance graph to recover historical
backend values. This keeps ordinary construction single-pass and treats pinned,
immutable dependency ownership as a tested distribution property. It changes the
location of part of existing catalog validation, so approval is needed before
Stage 6 rather than assuming it is merely an implementation detail.
**Viable alternative:** keep independent construction-time catalog preflight
loads through ConfigKit, followed by the combined load using original inputs.
This retains the current fail-closed runtime catalog checks and original
combined-load provenance, with simpler validation code. It deliberately relaxes
the acquire/parse-once efficiency objective for the bounded maintained catalogs
and adds repeated inheritance work; do not add a global cache to hide that cost.
If selected, document this narrow exception in the feature roadmap and benchmark
it. It need not affect single-pass loading of application sources.
An upstream raw-source policy hook could avoid the tradeoff, but it is a separate
ConfigKit feature request, not an assumed v0.5.0 capability or a reason to build
a replacement loader in PromptKit. All other decisions above are selected for
implementation; catalog publication permission is an operational checkpoint,
not an unresolved configuration-design choice.
The catalog-validation placement question is resolved by the configuration-boundary
ADR. No design question remains open here. Catalog publication authority and
ordinary module availability remain operational checkpoints for the catalog stages.