From 5f7b19403f7640f25d05c6c29273476eb9996024 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Wed, 30 Sep 2026 01:59:30 +0000 Subject: [PATCH] Accept the ConfigKit configuration boundary and align the implementation plan --- docs/adr/0001-configuration-boundary.md | 60 +++++++ docs/roadmap/configkit.md | 72 +++++--- docs/roadmap/implementation.md | 224 ++++++++++++++---------- 3 files changed, 240 insertions(+), 116 deletions(-) create mode 100644 docs/adr/0001-configuration-boundary.md diff --git a/docs/adr/0001-configuration-boundary.md b/docs/adr/0001-configuration-boundary.md new file mode 100644 index 0000000..df87aa7 --- /dev/null +++ b/docs/adr/0001-configuration-boundary.md @@ -0,0 +1,60 @@ +# 0001: Configuration Ownership At The ConfigKit Handoff + +## Status + +Accepted. + +## Date + +2026-09-30 + +## Context + +PromptKit is adopting ConfigKit to standardize configuration semantics and +simplify its loading machinery. Configuration includes prompts, profiles, and +backend definitions contributed by maintained catalogs and consuming applications. +ConfigKit supplies loading, composition, precedence, inheritance, and inspection +capabilities. PromptKit supplies LLM-specific interpretation and execution. + +A clear handoff assigns each library responsibility for the configuration it +owns and keeps validation aligned with the values an application actually uses. + +## Decision + +ConfigKit owns configuration processing up to the handoff of the final effective +configuration. PromptKit takes ownership at that handoff and applies its domain +interpretation and validation to the final configuration. + +The handoff includes effective presence/null information and resource context +supplied by ConfigKit. PromptKit uses this information to interpret effective +settings and resolve referenced resources. + +PromptKit passes through ConfigKit's inspection capabilities for explicit +operator use. Configuration history and source explanations serve that +inspection purpose; domain validation operates on the final effective values. + +This boundary applies uniformly to application configuration and maintained +catalog configuration, including backend definitions. + +## Alternatives Considered + +- **Independent source or catalog preflight validation:** preserves checks on + individual contributions, but duplicates configuration processing and evaluates + values that may be replaced before handoff. Raw-source and intermediate-state + validation are outside PromptKit's scope, including equivalent test-only checks. +- **History-based source auditing:** can identify overwritten contributions, but + couples domain enforcement to configuration history. Historical inspection is + an operator capability rather than a validation mechanism. + +## Consequences + +PromptKit's configuration logic and tests focus on the final effective values, +their domain meaning, and resulting execution behavior. ConfigKit owns failures +arising during its processing, which PromptKit preserves through its error +boundary. Catalog maintainers own their repositories' publishing checks. + +The adoption work routes backend configuration through the same handoff and +retires PromptKit's source-level and intermediate-configuration validation. +Effective resource interpretation and explicit operator inspection remain +supported. This concentrates configuration semantics in ConfigKit and reduces +duplicated machinery and maintenance in PromptKit. diff --git a/docs/roadmap/configkit.md b/docs/roadmap/configkit.md index 1443238..17b49f7 100644 --- a/docs/roadmap/configkit.md +++ b/docs/roadmap/configkit.md @@ -1,9 +1,10 @@ # ConfigKit Adoption Status: agreed feature roadmap. ConfigKit v0.5.0 supplies the required upstream -capabilities. The [implementation plan](implementation.md) owns execution steps -and identifies one remaining catalog-validation placement decision. This document -owns the intended end state, not implementation authorization. +capabilities. The [implementation plan](implementation.md) owns execution steps. +The [configuration-boundary ADR](../adr/0001-configuration-boundary.md) records +the handoff decision. This document owns the intended end state, not +implementation authorization. ## Purpose @@ -70,18 +71,23 @@ exist. Domain validation must not probe provider/model availability. ### One Shared Loading Lifecycle -Construction acquires configuration contributions, uses ConfigKit to compose and -overlay them, expands sibling inheritance, decodes the effective configuration, -performs recorded domain resolution and validation, and produces execution-ready -definitions. Publish no partially initialized engine on failure. +Construction supplies selected configuration inputs to ConfigKit, which composes, +overlays, expands inheritance, and decodes them. PromptKit interprets and validates +the final effective configuration at the ConfigKit handoff and produces +execution-ready definitions. ConfigKit's effective-value resolution and validation +hooks may implement that handoff. Publish no partially initialized engine on failure. ConfigKit owns generic parsing, combination, presence, origins, and inheritance. -PromptKit owns source selection/discovery, definition indexing, backend policy, -LLM-specific rules, and translation into runtime values. Runtime clients and -callbacks remain normal Go dependencies, not YAML configuration types. +PromptKit owns input selection/discovery, definition indexing, LLM-specific rules, +and translation of final values into runtime settings. Input adapters select or +encode contributions; they do not inspect or validate raw source contents. +Effective presence/null information and resource context supplied by ConfigKit +support interpretation at handoff. Historical inspection remains an explicitly +invoked operator capability, separate from domain enforcement. +Runtime clients and callbacks remain normal Go dependencies, not YAML types. -Use `profiles.` and `prompts..` keyed mappings assembled before -typed decoding rather than independently decoding definitions and combining them +Use `backends.`, `profiles.`, and `prompts..` keyed mappings +assembled before typed decoding rather than independently decoding definitions and combining them afterward. Keys own identity; do not duplicate `id`/`version` fields. These paths describe mapping levels, not dot-splitting of literal IDs or version keys. Multiple files can contribute to a collection; a monolithic file is not required. @@ -96,7 +102,11 @@ Accept ConfigKit's labeled mapping sources for programmatic configuration. Add only small typed conveniences justified by the minimal local-backend and application-profile workflows. Helpers encode explicit presence and feed the same loading lifecycle; they neither merge configurations nor infer omission from zeros. -HTTP/model clients and other runtime collaborators remain outside typed configuration. +Backend helpers also produce inputs to this lifecycle. Backend definitions, +including catalog connection and capacity settings, cross the same ConfigKit +handoff as prompts and profiles. The immutable runtime registry is built from +effective backend values. HTTP/model clients and other runtime collaborators +remain outside typed configuration. Keep the public package a deliberate facade and implementation private. Do not expose internal engine/domain types or introduce a parallel generic loader merely @@ -110,12 +120,15 @@ programmatic definitions. Explicitly selected directories may be discovered by PromptKit; ConfigKit handles their configuration contributions. Do not infer priority from traversal order. -Retain the useful profile priority categories, lowest to highest: external +Retain the useful priority categories, lowest to highest: external catalogs, PromptKit-owned profiles, application fallback, operator configuration, and programmatic configuration. Express intentional priority as ordered tiers. Within each tier, independent contributions compose strictly; overlapping non-mapping values conflict, even when equal. Disjoint fields under the same identity can combine. Across tiers, mappings recursively overlay. +These rules also govern backend configuration; consumers can add backend IDs or +overlay existing definitions through their selected tiers. Registry construction +validates the resulting values, not the identity of their contributors. Use ConfigKit's source roles honestly: underlay groups are ordered defaults, main input overlays underlays, and optional named overlay/CLI facilities retain @@ -155,11 +168,14 @@ Document and test this distinction, including any affected maintained payloads. ### Validation And Errors -Source acquisition, syntax, and strict-composition failures cannot be masked by -higher tiers. Typed and semantic validation apply to effective values rather than -every intermediate layer. An unused invalid effective definition fails construction. -Separate catalog ownership/reserved-name/secret-safety checks remain source-policy -responsibilities, not justification for duplicate generic decoding. +ConfigKit owns acquisition, syntax, composition, inheritance, and typed-decoding +failures. PromptKit preserves those errors and validates final effective domain +values. An unused invalid effective definition fails construction; a lower-layer +value legitimately replaced by ConfigKit is not a separate validation target. +This boundary applies equally to application and maintained catalog configuration. +PromptKit does not perform catalog preflight, raw-source or intermediate-state +validation, or history-based enforcement, including equivalent test-only checks. +Catalog maintainers own publishing checks in their respective repositories. Domain validators remain shared with runtime request validation where appropriate. Allow schema-valid partial profiles without an `abstract` flag. Validate supplied @@ -246,13 +262,18 @@ or configuration content safe to publish. - Remove the custom profile inheritance resolver and error-driven repository overlay stack. Do not retain an old-format compatibility engine alongside the new semantics. +- Route catalog and programmatic backend definitions through ConfigKit; retire + the separate backend-manifest decoder and pre-handoff registration validation. + Retain backend endpoint, capacity, and provider-parameter domain validation on + the final effective values. - Retain only needed discovery and message-resource adapters; remove direct YAML dependency usage once no remaining owner needs it. Preserve JSON Schema, runtime JSON-value ownership, domain validation, and execution collaborators. - Adapt maintained catalogs and PromptKit-owned definitions to the agreed keyed - format and released inheritance semantics. Preserve maintained name commitments, - backend ownership, and catalog namespace rules without upstream availability - checks. Coordinate necessary catalog releases before consuming their new format. + format and released inheritance semantics, including keyed backend assets. + Preserve maintained name commitments and the intended shipped backend settings. + Catalog source-policy enforcement belongs to catalog maintainers. Coordinate + compatible catalog releases before consuming the changed asset format. Publish new keyed assets directly in the existing data-only catalog projects, recording the format transition rather than maintaining duplicate legacy assets or a permanent conversion adapter. Catalog maintainers select release/version @@ -282,7 +303,9 @@ the requested report; that work remains off the execution path. Tests should use real ConfigKit with offline sources and synthetic provider collaborators. PromptKit owns its adapter, domain, lifecycle, and execution tests, not a duplicate ConfigKit parser/graph suite. Retire tests whose only purpose was -the removed loading machinery. Preserve meaningful execution regressions. +the removed loading machinery or source-policy enforcement. Exercise catalog +integration through the final assembled configuration and resulting behavior. +Preserve meaningful execution regressions. Representative acceptance evidence must cover: @@ -305,7 +328,8 @@ release gates or provider-network tests. Ship as an intentionally incompatible minor release with concrete migration guidance. Cover keyed files, `base_profile` to `extends`, reserved-key behavior, recursive maps, explicit null versus omission, changed error timing, programmatic -construction, and explicit engine replacement. Explain that overriding a model +construction, backend layering in place of registration-only restrictions, and +explicit engine replacement. Explain that overriding a model does not discard lower endpoint, reasoning, provider, or inheritance settings. The end state is one shared configuration model with the old loading/resolution diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 6c0f178..8b92af8 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,8 +1,8 @@ # ConfigKit Adoption Implementation Plan -Status: ready for staged implementation, except for the explicitly identified -catalog-validation decision under Open Questions. This is a planning document; -writing it does not implement the feature or authorize publishing dependencies. +Status: ready for staged implementation under the agreed configuration boundary. +This is a planning document; writing it does not implement the feature or +authorize publishing dependencies. ## Authority And Working Method @@ -10,6 +10,12 @@ The [ConfigKit feature roadmap](configkit.md) owns the intended end state and approved policy. Its intentional-breaking-change direction is authoritative: preserve substantive prompting capabilities, not the old loader, signatures, source wrappers, incidental error ordering, or configuration syntax. +The [configuration-boundary ADR](../adr/0001-configuration-boundary.md) records +the ownership decision: PromptKit interprets and validates the final effective +configuration supplied by ConfigKit. Presence/null information and effective +resource context are part of that handoff. Source history serves explicit +operator inspection, not validation or catalog enforcement. This applies to +tests as well as production code. Implement the stages below in order, one stage per coding-agent prompt. Each stage includes its tests and affected current-state documentation. Read @@ -37,9 +43,12 @@ Planning baselines: Use ConfigKit v0.5.0 for this work. Do not silently upgrade to a different contract. The catalog asset migration needs coordinated releases, not a committed -workspace or local replacement. Exact catalog release versions are selected at -publication under their existing date-based policy; record the versions and full -commits actually consumed. Publishing any repository requires separate authority. +workspace or local replacement. Changing the backend asset contract is broader +than the catalog profile-data compatibility exception: migrate the catalogs to +`/v3` with `v3.YYYYMMDD.N` tags, preserving their date/revision convention and +updating their own release documentation accordingly. Select exact dates and +unused revisions at publication; record versions and full commits consumed. +Publishing any repository requires separate authority. ## Implementation Decisions @@ -83,9 +92,10 @@ Use one ConfigKit filesystem namespace per helper invocation and root resource contexts at the supplied directory. Single files use ConfigKit directly. Remove `WithPromptFS`, `WithPromptFile`, `WithProfileFS`, `WithProfileFile`, -`WithFallbackProfileFS`, and `WithProfiles` at public cutover. Retain schema source -options and collaborator/backend injection. Do not require any prompts to exist -merely to construct an engine for profile inspection. +`WithFallbackProfileFS`, `WithProfiles`, and `WithBackend` at public cutover. +Retain schema source options and runtime collaborator injection. Backend +configuration moves into ConfigKit inputs as specified below. Do not require any +prompts to exist merely to construct an engine for profile inspection. Programmatic configuration uses labeled `configkit.Mapping` inputs. Replace the old zero-sentinel `Profile`/`OpenAICompatibleProfileConfig` pair with one small @@ -109,13 +119,36 @@ clear operation. Use raw Mapping for explicit null, prompt definitions, or riche configuration. Do not introduce a public generic optional-value type just for this helper. Convert fields explicitly to ConfigKit-supported built-in values, then call Mapping for snapshot ownership and load-time errors. Do not marshal -through YAML or infer absence from a value. Keep the existing local-backend -constructor, which configures runtime backend registration rather than profiles. +through YAML or infer absence from a value. Helpers encode values; domain +validation occurs on the final configuration at handoff. + +Replace the registration-only Backend value with a presence-aware convenience: + +```go +type BackendConfig struct { + Endpoint, APIKeyEnv *string + ConcurrencyLimit, QueueCapacity *int + ExtraParams map[string]any +} +func BackendSource(label string, backends map[string]BackendConfig) configkit.Source +func LocalBackend(endpoint string, concurrencyLimit int) BackendConfig +``` + +Use the same omission/pointer/map conventions as ProfileSource. LocalBackend +supplies endpoint and concurrency pointers and leaves the remaining fields +omitted; callers choose the ID as the map key. Neither helper validates settings +before ConfigKit processes them. For example, place BackendSource with +`map[string]BackendConfig{"local": LocalBackend(endpoint, 4)}` in an application +underlay. Raw Mapping supports explicit null and combined backend/profile/prompt +documents. Remove the old Backend registration value and its reservation rules; +retain useful built-in ID constants. All backend contributions follow the same +strict-composition and overlay rules as other configuration. ### Typed Configuration And Execution Values Add `internal/configuration` as the cohesive ConfigKit adapter. Its private -document DTO has `profiles: map[string]profileConfig` and +document DTO has `backends: map[string]backendConfig`, +`profiles: map[string]profileConfig`, and `prompts: map[string]map[string]promptConfig`. Collection entries are mappings, not nullable definition pointers. No duplicate identity fields; ConfigKit consumes `extends`, so no DTO field or second resolver handles it. Use `configkit.Keys`, @@ -128,6 +161,27 @@ private execution-owned fields or presence bits, not retained metadata queries. Runtime/domain packages must not import ConfigKit. Do not duplicate its tree, merge engine, inheritance graph, diagnostics, or presence framework. +Backend DTO fields match BackendConfig using `endpoint`, `api_key_env`, +`concurrency_limit`, `queue_capacity`, and `extra_params`. Validate the final +endpoint as required, absolute HTTP(S) with the existing URL rules. An absent or +null API-key environment name means no lookup; a supplied name follows existing +normalization and naming rules. Absent/null concurrency means zero (unlimited); +supplied values must be nonnegative. Absent/null queue capacity selects the +existing default of 1024 for positive concurrency and no queue for unlimited +concurrency. Explicit queue capacity, including zero, requires positive +concurrency and must satisfy existing nonnegative/overflow checks. Null provider +maps mean no configured extra parameters; entries may contain JSON null. +Required endpoint failures, unknown fields, and domain errors apply to final +definitions, including unused ones. Registry construction consumes these final +normalized values; it does not replay registration or source-reservation checks. + +Use ConfigKit's Resolve/Validate callbacks only on its effective post-layering, +post-inheritance typed values. These public lifecycle hooks implement the domain +handoff and record normalization; they are not hooks for inspecting earlier +contributions. Final presence lookup and resource-context lookup are sufficient +for interpretation. History/Explain/HistoricalPaths are operator inspection +facilities, not inputs to construction-time policy checks. + Use the existing prompt fields under each keyed version, without `id`/`version`. Use pointers for `content`, `content_file`, and optional clearable strings; preserve existing input, cache-control, output-contract, and role rules. @@ -169,13 +223,15 @@ derived endpoint/key/defaults must not follow a child that changes backend. Provider map null entries are literal JSON null; empty mappings retain children. For backend-to-profile provider-map overlay, use a narrow JSON-value merge helper with the same recursive-map/value-replacement semantics; it is not a configuration -loader and must not interpret `extends` in backend runtime values. +loader. Both backend and profile configuration maps have already passed through +ConfigKit's reserved-key semantics; only request-level maps bypass that lifecycle. Record any defaults/normalizations written into the typed configuration via ConfigKit Resolve, with dependencies on relevant configuration paths. Do not project runtime collaborators or credentials into the configuration tree just to -report them. Inspection describes resolved configuration, while InspectProfile -and preparation describe backend-completed execution settings. Make this boundary +report them. Inspection includes effective backend configuration. It describes +resolved configuration, while InspectProfile and preparation describe +backend-completed execution settings. Make this boundary explicit rather than inventing synthetic provenance for registry defaults. Retain APIKeyRequired's direct-key workflow: true suppresses a backend-derived @@ -237,8 +293,9 @@ ConfigKit SourceContext, mutable runtime values, or internal types. Delegate queries and redaction to the retained result. Nil Engine returns ErrInvalidConfig; the zero inspection view has empty query results and writer operations return ErrInvalidConfig. Writer errors remain inspectable. Mark credential-shaped dynamic -keys sensitive using Definition.Secret, reusing the existing catalog secret-key -vocabulary; do not redact `api_key_env` names as credential values. This is not a +keys sensitive using Definition.Secret; the existing secret-key vocabulary can +serve as a redaction predicate, not a source-rejection rule. Do not redact +`api_key_env` names as credential values. This is not a promise that arbitrary prompt/provider content is safe to disclose. Keep the runner independent of this view. Inspection calls perform no discovery, @@ -253,18 +310,22 @@ and its focused tests. Leave engine assembly on the old path for this stage. 1. Add ConfigKit v0.5.0; update go.sum and factual OSS notices for it and its selected transitive dependencies. Preserve project licenses and the Go floor. -2. Add the keyed DTOs and a private load entry point accepting ConfigKit Inputs - and underlays. Call LoadLayered once, with no VersionRule or secret resolution - bindings for API credentials. Use callback-scoped Resolve/Validate correctly. -3. Implement presence extraction and the field rules above. Validate supplied - domain values through existing domain helpers; no final target requirement. +2. Add keyed backend/profile/prompt DTOs and a private load entry point accepting + ConfigKit Inputs and underlays. Call LoadLayered once, with no VersionRule or + secret resolution bindings for API credentials. Use callback-scoped + Resolve/Validate correctly. +3. Implement effective presence extraction and the field rules above. Validate + final domain values through existing helpers, resolving backend references + against the final backend map; profiles have no final target requirement yet. Record normalization/default mutations, not arbitrary DTO bookkeeping fields. 4. Preserve `*configkit.LoadError` through wrapping. Domain callback messages identify paths/problems without interpolating raw values or credentials. 5. Add representative real-loader tests: keyed literal IDs, strict group conflict, recursive cross-tier overlay, a shared base with explicit zero/null/false, an - unused invalid effective definition, and a malformed lower source not hidden - by an overlay. Do not reproduce ConfigKit's parser/inheritance matrix. + unused invalid effective definition, and preservation of a ConfigKit load + error. Cover an invalid lower-layer domain value replaced by a valid final + value: construction succeeds. Do not reproduce ConfigKit's parser/inheritance + matrix or add preflight/source-policy tests. Exit: focused tests and `go test ./...` pass; old engine behavior is not presented as migrated. Internal inventory documents only the loader now actually present. @@ -297,8 +358,9 @@ existing execution tests still work through the old assembly until cutover. Scope: domain settings, target resolution, wire-presence propagation, profile inspection. No new source discovery in execution packages. -1. Add execution-owned presence/suppression state and construct indexed profiles - from loader output plus the immutable backend registry. Apply registry-derived +1. Build the immutable backend registry from the final loaded backend map. Add + execution-owned presence/suppression state and construct indexed profiles + from loader output plus that registry. Apply registry-derived values after inheritance, using the field table above. 2. Replace scalar-zero/blank merge assumptions in target resolution. Configuration explicit zero reaches injected clients and the built-in request body; null does @@ -319,8 +381,8 @@ a compatibility promise. Update affected target/presence GoDoc with this change. ## Stage 4 — Migrate The OpenRouter Catalog Assets -Scope: `../promptkit-backend-openrouter` only. Resolve the catalog-validation open -question before implementing its consumption strategy in Stage 6. +Scope: `../promptkit-backend-openrouter` only. Catalog maintainers own source and +publishing checks here; PromptKit consumes its configuration through ConfigKit. 1. Read that repository's instructions; preserve unrelated work. Convert each profile asset to `profiles: {literal-id: {settings...}}`, preserving inventory @@ -330,22 +392,29 @@ question before implementing its consumption strategy in Stage 6. 2. Audit zero/empty values that previously meant unspecified. Remove those when omission was intended; retain intentional explicit values. Review any dynamic provider `extends` keys rather than silently changing outbound payloads. -3. Leave backend.json and its schema unchanged. Update catalog format notes and - applicable tests to the keyed format, with a prominent incompatible-format - migration note. Model availability is not probed. +3. Replace `catalog/backend.json` with `catalog/backend.yml` containing a + `backends: {openrouter: {settings...}}` mapping. Preserve endpoint, API-key + environment name, concurrency, queue, and extra-parameter values; remove the + duplicate id and old manifest schema_version. The file is an ordinary ConfigKit + input, not a separately decoded manifest. Update embedded assets, module path + and imports to `/v3`, tests, and catalog format/release documentation, with an + incompatible-format migration note. Keep FS/Root as the minimal export surface. + Catalog-local publishing checks stay in that repository. No provider probes. 4. Validate under its procedure and commit. If publication is authorized, publish - the next unused `v2.YYYYMMDD.N`; otherwise report the exact commit awaiting + the next unused `v3.YYYYMMDD.N`; otherwise report the exact commit awaiting publication. The date/revision is selected then, not guessed in this plan. Exit: one reviewed migrated catalog commit; existing immutable tags still serve -old consumers. The profile-data change does not change the Go module path/API or -the JSON backend manifest. Warn old loaders not to consume the new assets. +old consumers. The new major module path identifies the changed asset contract; +old consumers retain the published `/v2` assets. ## Stage 5 — Migrate The Rakestrawhome Catalog Assets Scope: `../promptkit-backend-rakestrawhome` only. Apply the same keyed-format, presence, notice, validation, and publication treatment as Stage 4 to its smaller -inventory. Preserve backend endpoint, credentials metadata, and capacity policy. +inventory, including the `/v3` transition and keyed `backends.rakestrawhome` +definition in `catalog/backend.yml`. Preserve backend endpoint, credentials +metadata, and capacity policy. Do not couple its date/revision to OpenRouter's release. Exit: both compatible catalog versions are published and ordinarily resolvable @@ -359,18 +428,22 @@ Scope: `internal/catalog`, maintained definitions, catalog integration tests, dependency pins/notices. Keep this separate from public API migration. 1. Pin the published compatible catalog versions and exact attribution commits. - Keep strict backend.json loading/validation and registry construction. -2. Replace catalog profile repositories with ConfigKit file Inputs for the first + Update imports to `/v3`. Replace manifest decoding with ConfigKit file Inputs. +2. Supply backend and profile assets as ConfigKit Inputs in the first underlay. Supply PromptKit-owned keyed Mapping definitions as the next group: standard extends the selected catalog's current maintained base; auto retains its current direct model/backend definition. Preserve both literal names. -3. Implement the selected catalog source-policy strategy from Open Questions. - Preserve reserved-name, ownership, connection-metadata, and secret checks; - consumers may still override catalog profiles and their bases. Do not apply - catalog-only namespace restrictions to application sources. +3. Retire source-layout allowlists, catalog preflight, source ownership/reserved + prefix checks, connection-field prohibitions, and secret-key source rejection. + The adapter selects known asset entry points and discovers configuration files; + ConfigKit acquires their contents. Domain validation sees only the final + assembled configuration. Consumers may overlay catalog backends, profiles, + and their bases through the same configuration model. 4. Test real imported assets offline with the new loader, maintained-name - resolution, consumer override of a maintained base, and synthetic malformed - catalog fixtures for the source-policy risks. No upstream endpoint probes. + resolution, consumer override of a maintained base/backend, and rejection of + an invalid final backend. Test a replaced lower-layer domain error succeeds. + Do not independently validate catalog-only intermediate configurations, parse + raw assets in PromptKit tests, or audit origins to establish ownership. Exit: catalog and owned definitions participate in the new keyed configuration model without a legacy profile parser on that path. Catalog integration tests @@ -384,8 +457,9 @@ adapters, and migration of their direct test setup. 1. Introduce the Construction Surface exactly as specified above. Remove the superseded source/profile options and convenience types rather than keeping deprecated pass-through wrappers. Keep intentional runtime injection APIs. -2. Assemble registry, combined configuration snapshot, renderer, validator, - capacity manager, and runner. Return no Engine on construction failure. +2. Load the combined backend/profile/prompt configuration, construct its runtime + registry and indexed snapshot, then assemble renderer, validator, capacity + manager, and runner. Return no Engine on construction failure. Translate construction errors to ErrInvalidConfig while preserving LoadError via errors.As. Resource errors include semantic path and safe source context. 3. Connect indexed prompt/profile lookups to preparation and inspection. Missing @@ -393,7 +467,8 @@ adapters, and migration of their direct test setup. selection error. Old parse/load error categories need not be manufactured for errors now detected at construction. 4. Move root test fixtures/setup and maintained example construction to ConfigKit - sources, keyed definitions, and ProfileSource so all consumers still compile. + sources, keyed definitions, ProfileSource, and BackendSource so all consumers + still compile. Update local-backend examples to choose their ID as a map key. Update expected timing only for the deliberate eager-validation change. Keep behavioral assertions for prompting, overrides, and errors. 5. Add public tests for a minimal custom local backend and programmatic profile, @@ -459,7 +534,8 @@ Scope: maintained examples and user-facing documentation. complete parser/inheritance contract. 3. Update README and `docs/consumers/pkg-promptkit.md` with minimal local-backend, file/embedded source, programmatic Mapping/ProfileSource, explicit inspection, - and engine-replacement workflows. Explain main versus ordered underlays. + and engine-replacement workflows. Explain main versus ordered underlays and + backend layering in place of registration-only restrictions. 4. Add concise old-to-new migration guidance in the consumer guide: removed API options, exact IDs, `base_profile` to `extends`, zero/false/null, recursive maps, no clear via `{}`, reserved `extends` runtime escape, failure timing, and the @@ -478,14 +554,17 @@ internal architectural documentation. 1. Delete lazy prompt/profile metadata prepasses, filesystem repositories, fallback-on-error overlays, custom base-chain resolver/depth cap, zero-sentinel profile merging, and temporary stage adapters once no caller needs them. + Remove the separate backend-manifest decoder, source-policy audits, and + pre-handoff backend registration validation. Keep final-value domain checks. Retain narrow lookup interfaces only where they still simplify real consumers. 2. Retain useful filecatalog/schema utilities and JSON-value ownership/validation. Remove direct yaml.v3 use and dependency if no remaining owner requires it; do not claim all YAML dependencies disappear when ConfigKit still uses YAML. 3. Move meaningful regressions from deleted package tests to their new owning - boundary. Retire tests only about old internals, repeated parser coverage, or - superseded semantics. Do not replace lost runtime protection with loader-only - tests. Keep private assembly checks separate from external facade tests. + boundary. Retire tests about old internals, repeated parser coverage, + source-policy enforcement, or superseded semantics. Do not replace lost runtime + protection with loader-only tests. Keep private assembly checks separate from + external facade tests. 4. Update `docs/internal/overview.md`, `docs/internal/sources.md`, relevant renderer and engine documents, and the implemented-structure paragraphs in architecture policy. Describe the new package/dependency boundary without turning names or @@ -523,45 +602,6 @@ Exit: the feature roadmap's end state is demonstrably implemented, with one configuration lifecycle, explicit operator inspection, no legacy loader, and no newly invented provider checks or human release gates. -## Open Questions - -### Catalog Source-Policy Validation Versus Single-Pass Loading - -ConfigKit v0.5.0 performs typed callbacks only after effective overlay/inheritance; -it does not expose a raw-source policy callback. PromptKit currently checks each -catalog's resolved backend ownership and prohibited fields before consumer -overlays. Checking only the effective merged configuration would allow consumers -to mask a broken catalog contribution. Conversely, independently loading every -catalog and then loading the original sources again duplicates parsing and -inheritance work. Feeding the first load back as Mapping loses original source -provenance and changes when inheritance occurs. Neither workaround should be -silently described as the agreed single-pass design. - -**Recommendation:** validate immutable catalog-only backend ownership and -source-policy rules in PromptKit's offline dependency contract tests, using real -ConfigKit and the selected released assets. At engine construction, retain -backend-manifest validation, strict source composition, effective domain -validation, and a history-based structural audit for reserved profile names, -forbidden connection fields, and prohibited secret-shaped keys contributed by -catalog sources (including masked paths). Use ConfigKit source IDs/history to -distinguish catalog contributions from permitted consumer overrides. Do not -parse Explanation YAML or rebuild an inheritance graph to recover historical -backend values. This keeps ordinary construction single-pass and treats pinned, -immutable dependency ownership as a tested distribution property. It changes the -location of part of existing catalog validation, so approval is needed before -Stage 6 rather than assuming it is merely an implementation detail. - -**Viable alternative:** keep independent construction-time catalog preflight -loads through ConfigKit, followed by the combined load using original inputs. -This retains the current fail-closed runtime catalog checks and original -combined-load provenance, with simpler validation code. It deliberately relaxes -the acquire/parse-once efficiency objective for the bounded maintained catalogs -and adds repeated inheritance work; do not add a global cache to hide that cost. -If selected, document this narrow exception in the feature roadmap and benchmark -it. It need not affect single-pass loading of application sources. - -An upstream raw-source policy hook could avoid the tradeoff, but it is a separate -ConfigKit feature request, not an assumed v0.5.0 capability or a reason to build -a replacement loader in PromptKit. All other decisions above are selected for -implementation; catalog publication permission is an operational checkpoint, -not an unresolved configuration-design choice. +The catalog-validation placement question is resolved by the configuration-boundary +ADR. No design question remains open here. Catalog publication authority and +ordinary module availability remain operational checkpoints for the catalog stages.