promptkit/docs/roadmap/implementation.md

# ConfigKit Adoption Implementation Plan

Status: ready for staged implementation under the agreed configuration boundary.
This is a planning document; writing it does not implement the feature or
authorize publishing dependencies.

## Authority And Working Method

The [ConfigKit feature roadmap](configkit.md) owns the intended end state and
approved policy. Its intentional-breaking-change direction is authoritative:
preserve substantive prompting capabilities, not the old loader, signatures,
source wrappers, incidental error ordering, or configuration syntax.
The [configuration-boundary ADR](../adr/0001-configuration-boundary.md) records
the ownership decision: PromptKit interprets and validates the final effective
configuration supplied by ConfigKit. Presence/null information and effective
resource context are part of that handoff. Source history serves explicit
operator inspection, not validation or catalog enforcement. This applies to
tests as well as production code.

Implement the stages below in order, one stage per coding-agent prompt. Each
stage includes its tests and affected current-state documentation. Read
[development](../development.md) and the applicable policies before starting;
in particular, follow [agents](../policy/agents.md),
[architecture](../policy/architecture.md), [testing](../policy/testing.md),
[documentation](../policy/documentation.md), and [OSS](../policy/oss.md).
Catalog work also follows each catalog repository's own instructions and release
procedure. Do not turn this plan's internal organization into new public promises.

Begin each stage by inspecting the worktree and earlier stages. Commit completed
units under the agent policy. Keep stages buildable; temporary private wiring
needed to connect successive stages must be removed by Stage 11. Do not ship a
legacy mode, dual loader, or old-format conversion layer. Do not add prototype
applications or a generic configuration framework of our own.

Planning baselines:

- PromptKit runtime v0.11.2, commit
  `12d6d80024e80c552d122a076c22429bf6f2f6c2`; approved roadmap at `521291d`.
- ConfigKit v0.5.0, commit
  `dc09959cca4e105f49ec316c6d4ba585bd2340bd`, module
  `gitea.maximumdirect.net/eric/ConfigKit`.
- Both catalog dependencies currently select `/v2` at `v2.20260926.0`.

Use ConfigKit v0.5.0 for this work. Do not silently upgrade to a different
contract. The catalog asset migration needs coordinated releases, not a committed
workspace or local replacement. Changing the backend asset contract is broader
than the catalog profile-data compatibility exception: migrate the catalogs to
`/v3` with `v3.YYYYMMDD.N` tags, preserving their date/revision convention and
updating their own release documentation accordingly. Select exact dates and
unused revisions at publication; record versions and full commits consumed.
Publishing any repository requires separate authority.

## Implementation Decisions

### Construction Surface

Keep `NewEngine(Config, ...Option)` and the execution/injection surface where it
still serves its substantive purpose. Replace profile/prompt source options with
these fields and helpers; this is an intentional API break:

```go
type ConfigurationSources struct {
    Underlays []configkit.Underlay
    Main      *configkit.Input
}

// Add Sources; remove PromptDir and ProfileDir.
// Retain SchemaDir, Timeout, HTTPClient, and TransientRetries.
type Config struct { /* existing non-definition settings; Sources ConfigurationSources */ }

func DirectoryInputs(directory string) ([]configkit.Input, error)
func FilesystemInputs(label string, filesystem fs.FS, root string) ([]configkit.Input, error)
```

The pointer distinguishes omitted main input without inspecting ConfigKit's
opaque Source. Nil main becomes `configkit.NoSource()`; a supplied invalid Input
is not silently ignored. Prepend two private underlays: external catalogs, then
PromptKit-owned definitions. Append caller underlays in their supplied order;
main is last and keeps its ConfigKit meaning. Imports follow each Input's flag.
Do not expose named overlays, CLI overrides, or VersionRule through another
PromptKit abstraction in this feature.

Show the usual application categories as caller underlays ordered fallback,
operator, programmatic, with main omitted. For a single highest-priority document,
main is also useful. These are explicit selections, not directory-name heuristics
or a second automatic precedence algorithm. Each directory helper returns inputs
for one strict-composition group, recursively finding `.yaml` and `.yml` files;
sorted discovery makes diagnostics reproducible, not one file higher priority.
An explicitly selected missing/unreadable root fails; an empty directory supplies
no contributions. Helpers set imports false; callers may enable them explicitly.
Use one ConfigKit filesystem namespace per helper invocation and root resource
contexts at the supplied directory. Single files use ConfigKit directly.

Remove `WithPromptFS`, `WithPromptFile`, `WithProfileFS`, `WithProfileFile`,
`WithFallbackProfileFS`, `WithProfiles`, and `WithBackend` at public cutover.
Retain schema source options and runtime collaborator injection. Backend
configuration moves into ConfigKit inputs as specified below. Do not require any
prompts to exist merely to construct an engine for profile inspection.

Programmatic configuration uses labeled `configkit.Mapping` inputs. Replace the
old zero-sentinel `Profile`/`OpenAICompatibleProfileConfig` pair with one small
`ProfileConfig` convenience and a source builder:

```go
type ProfileConfig struct {
    Extends, Backend, Endpoint, Model *string
    Temperature, TopP *float64
    MaxTokens, TimeoutSeconds *int
    ServiceTier, ReasoningEffort, APIKeyEnv *string
    APIKeyRequired *bool
    ExtraParams map[string]any
}
func ProfileSource(label string, profiles map[string]ProfileConfig) configkit.Source
```

Nil convenience fields mean omitted; nonnil pointers preserve zero/false/empty.
Nil ExtraParams omits the map; a nonnil empty map supplies an empty mapping, not a
clear operation. Use raw Mapping for explicit null, prompt definitions, or richer
configuration. Do not introduce a public generic optional-value type just for
this helper. Convert fields explicitly to ConfigKit-supported built-in values,
then call Mapping for snapshot ownership and load-time errors. Do not marshal
through YAML or infer absence from a value. Helpers encode values; domain
validation occurs on the final configuration at handoff.

Replace the registration-only Backend value with a presence-aware convenience:

```go
type BackendConfig struct {
    Endpoint, APIKeyEnv *string
    ConcurrencyLimit, QueueCapacity *int
    ExtraParams map[string]any
}
func BackendSource(label string, backends map[string]BackendConfig) configkit.Source
func LocalBackend(endpoint string, concurrencyLimit int) BackendConfig
```

Use the same omission/pointer/map conventions as ProfileSource. LocalBackend
supplies endpoint and concurrency pointers and leaves the remaining fields
omitted; callers choose the ID as the map key. Neither helper validates settings
before ConfigKit processes them. For example, place BackendSource with
`map[string]BackendConfig{"local": LocalBackend(endpoint, 4)}` in an application
underlay. Raw Mapping supports explicit null and combined backend/profile/prompt
documents. Remove the old Backend registration value and its reservation rules;
retain useful built-in ID constants. All backend contributions follow the same
strict-composition and overlay rules as other configuration.

### Typed Configuration And Execution Values

Add `internal/configuration` as the cohesive ConfigKit adapter. Its private
document DTO has `backends: map[string]backendConfig`,
`profiles: map[string]profileConfig`, and
`prompts: map[string]map[string]promptConfig`. Collection entries are mappings,
not nullable definition pointers. No duplicate identity fields; ConfigKit consumes
`extends`, so no DTO field or second resolver handles it. Use `configkit.Keys`,
`Key`, and `Index` for semantic paths, never split IDs on dots.

Profile DTO fields match ProfileConfig's YAML spellings (`backend`,
`reasoning_effort`, etc.). Nullable pointers plus ConfigKit presence metadata
distinguish absent, value, and explicit null. Store that distinction in small
private execution-owned fields or presence bits, not retained metadata queries.
Runtime/domain packages must not import ConfigKit. Do not duplicate its tree,
merge engine, inheritance graph, diagnostics, or presence framework.

Backend DTO fields match BackendConfig using `endpoint`, `api_key_env`,
`concurrency_limit`, `queue_capacity`, and `extra_params`. Validate the final
endpoint as required, absolute HTTP(S) with the existing URL rules. An absent or
null API-key environment name means no lookup; a supplied name follows existing
normalization and naming rules. Absent/null concurrency means zero (unlimited);
supplied values must be nonnegative. Absent/null queue capacity selects the
existing default of 1024 for positive concurrency and no queue for unlimited
concurrency. Explicit queue capacity, including zero, requires positive
concurrency and must satisfy existing nonnegative/overflow checks. Null provider
maps mean no configured extra parameters; entries may contain JSON null.
Required endpoint failures, unknown fields, and domain errors apply to final
definitions, including unused ones. Registry construction consumes these final
normalized values; it does not replay registration or source-reservation checks.

Use ConfigKit's Resolve/Validate callbacks only on its effective post-layering,
post-inheritance typed values. These public lifecycle hooks implement the domain
handoff and record normalization; they are not hooks for inspecting earlier
contributions. Final presence lookup and resource-context lookup are sufficient
for interpretation. History/Explain/HistoricalPaths are operator inspection
facilities, not inputs to construction-time policy checks.

Use the existing prompt fields under each keyed version, without `id`/`version`.
Use pointers for `content`, `content_file`, and optional clearable strings;
preserve existing input, cache-control, output-contract, and role rules.
`messages` must be nonempty; each message has exactly one nonnull content source.
An explicitly supplied empty inline string is a valid template. Null content
removes that selection; null content_file removes its reference. Output remains a
required mapping with valid format and validation mode. Optional description,
session template, default profile, cache control, inputs, and schema path can be
cleared where their domain permits absence. Null required role/format/mode or a
null whole definition is invalid. Scalar `required` and `repair_attempts` retain
their false/zero defaults and are not nullable. Null inputs means no declarations.
Unknown fields are ConfigKit errors, including old `id`, `version`, and
`base_profile`. Prompt versions must be string mapping keys.

Literal definition keys remain exact; require nonblank identity but do not
normalize/rekey maps after ConfigKit has resolved relationships. Update selection
to use exact nonblank IDs/versions rather than silently trimming them into a
different identity. Existing value normalization (for example role lowercase or
endpoint trimming) is recorded through ConfigKit's resolution callbacks. Field
references to definition IDs remain exact as well.

Profile field rules after configuration overlay/inheritance:

| Fields | Absent | Explicit value | Explicit null |
| --- | --- | --- | --- |
| `backend` | No selected backend | Exact registered ID; blank means none | No backend |
| `endpoint` | Selected backend endpoint, if any | Normalize/validate nonblank URL; empty clears | Clear, do not recover backend endpoint |
| `model` | Unspecified | Nonblank supplied model; empty clears | Unspecified |
| `temperature`, `top_p`, `max_tokens` | Unspecified | Validate existing bounds; zero is explicit and sent | Suppress outbound control |
| `reasoning_effort`, `service_tier` | Unspecified | Existing normalization; blank suppresses | Suppress outbound control |
| `timeout_seconds` | Framework 600-second baseline | Nonnegative; zero disables generation deadline | Reset to framework baseline, not lower profile timeout |
| `api_key_env` | Selected backend's optional lookup name | Trimmed value; blank disables lookup | Disable lookup, do not recover backend name |
| `api_key_required` | False | Explicit boolean, including false overriding true | False |
| `extra_params` | Backend defaults | ConfigKit-composed profile map, recursively overlaid on backend map | Suppress entire configured provider map |

Backend registry defaults are applied once during snapshot construction, after
effective profile inheritance. Keep them out of the inheritance input: a parent's
derived endpoint/key/defaults must not follow a child that changes backend.
Provider map null entries are literal JSON null; empty mappings retain children.
For backend-to-profile provider-map overlay, use a narrow JSON-value merge helper
with the same recursive-map/value-replacement semantics; it is not a configuration
loader. Both backend and profile configuration maps have already passed through
ConfigKit's reserved-key semantics; only request-level maps bypass that lifecycle.

Record any defaults/normalizations written into the typed configuration via
ConfigKit Resolve, with dependencies on relevant configuration paths. Do not
project runtime collaborators or credentials into the configuration tree just to
report them. Inspection includes effective backend configuration. It describes
resolved configuration, while InspectProfile and preparation describe
backend-completed execution settings. Make this boundary
explicit rather than inventing synthetic provenance for registry defaults.

Retain APIKeyRequired's direct-key workflow: true suppresses a backend-derived
environment source. An explicitly configured profile environment source, or a
request environment override, remains usable; false restores ordinary optional
credential semantics. No credential values belong in the DTO, inspection, or
construction-time environment resolution. Raw profile `api_key` remains invalid.

Validate all effective supplied values and referenced backend IDs at construction.
Do not require a complete model/endpoint there or reject unused partial profiles.
Do not require a prompt default_profile to be runnable before a request can
override it. Final target completeness is checked after runtime overrides.
InspectProfile can report a partial target without credential or provider access.
Keep request override contracts otherwise unchanged, including complete nonempty
runtime ExtraParams replacement and reasoning clearing. A runtime literal
`extra_params["extends"]` remains ordinary provider data. Do not expand this work
into a redesign of every request override.

### Snapshot And Inspection Design

The loader returns indexed, execution-owned definitions plus a private retained
ConfigKit result. An adapter may implement the runner's narrow lookup interfaces
over maps during migration; no repository chain, fallback-on-error, or per-call
configuration load survives. Preserve defensive ownership across injected clients
and public results. Cache prompt definition hashes at construction; hash effective
captured content, not a stale resource path. Hash bytes need not match the old
configuration format. Preserve rendered-prompt hashing semantics.

Resolve each effective content_file using its exact field origin and ConfigKit
SourceContext. Join its relative name to BaseDir inside the returned rooted FS;
reject absolute/outside-root paths and close every opened handle. Preserve path
text, including meaningful whitespace, after separately checking nonblankness.
Do not choose the child's source for an inherited reference. If there is no unique
contributing source with context, fail with an actionable path/source error; the
caller can supply an explicit Input filesystem context or override the field.
Retain no resource capability for the purpose of public diagnostics.

Capture message text and parse message/session template syntax at construction
with the same function names/options used for runtime rendering. Do not invoke
input helpers or require Vars/artifacts while parsing. Reuse compiled templates
where practical, binding request-specific input functions on safe per-call clones;
never mutate a shared template function closure with request state. Runtime
missing-variable/input failures remain runtime errors.

Expose a concrete read-only facade value:

```go
func (e *Engine) InspectConfiguration() (ConfigurationInspection, error)
func (v ConfigurationInspection) Metadata() configkit.Metadata
func (v ConfigurationInspection) Sources() []configkit.SourceInfo
func (v ConfigurationInspection) Diagnostics() []configkit.Diagnostic
func (v ConfigurationInspection) Explain(path configkit.Path) (configkit.Explanation, bool)
func (v ConfigurationInspection) WriteEffective(w io.Writer) error
func (v ConfigurationInspection) WriteExplanation(w io.Writer, path configkit.Path) error
```

Store an unexported narrow implementation reference; do not expose Result.Config,
ConfigKit SourceContext, mutable runtime values, or internal types. Delegate
queries and redaction to the retained result. Nil Engine returns ErrInvalidConfig;
the zero inspection view has empty query results and writer operations return
ErrInvalidConfig. Writer errors remain inspectable. Mark credential-shaped dynamic
keys sensitive using Definition.Secret; the existing secret-key vocabulary can
serve as a redaction predicate, not a source-rejection rule. Do not redact
`api_key_env` names as credential values. This is not a
promise that arbitrary prompt/provider content is safe to disclose.

Keep the runner independent of this view. Inspection calls perform no discovery,
resource reads, environment credential lookup, or provider operations. Schemas,
artifacts, preparation ownership, generation admission, retry/repair, observations,
and cancellation remain at their established runtime boundaries.

## Stage 1 — Add The Dependency And Typed Loading Core

**Completed:** 2026-09-30.

Scope: module metadata, `internal/configuration` DTOs, real ConfigKit loading,
and its focused tests. Leave engine assembly on the old path for this stage.

1. Add ConfigKit v0.5.0; update go.sum and factual OSS notices for it and its
   selected transitive dependencies. Preserve project licenses and the Go floor.
2. Add keyed backend/profile/prompt DTOs and a private load entry point accepting
   ConfigKit Inputs and underlays. Call LoadLayered once, with no VersionRule or
   secret resolution bindings for API credentials. Use callback-scoped
   Resolve/Validate correctly.
3. Implement effective presence extraction and the field rules above. Validate
   final domain values through existing helpers, resolving backend references
   against the final backend map; profiles have no final target requirement yet.
   Record normalization/default mutations, not arbitrary DTO bookkeeping fields.
4. Preserve `*configkit.LoadError` through wrapping. Domain callback messages
   identify paths/problems without interpolating raw values or credentials.
5. Add representative real-loader tests: keyed literal IDs, strict group conflict,
   recursive cross-tier overlay, a shared base with explicit zero/null/false, an
   unused invalid effective definition, and preservation of a ConfigKit load
   error. Cover an invalid lower-layer domain value replaced by a valid final
   value: construction succeeds. Do not reproduce ConfigKit's parser/inheritance
   matrix or add preflight/source-policy tests.

Exit: focused tests and `go test ./...` pass; old engine behavior is not presented
as migrated. Internal inventory documents only the loader now actually present.

## Stage 2 — Add Source Discovery And Prompt Resource Capture

Scope: source selection helpers and construction-time prompt materialization.

1. Implement private directory/FS discovery behind the chosen helper signatures;
   wrap discovered paths as ConfigKit sources, retaining explicit root context.
   Reuse useful filecatalog path handling without carrying profile metadata scans.
2. Materialize effective prompt message text using field origins and passive
   contexts. Move reusable prompt/input/cache/output validation into the loader
   or shared domain owner; do not copy entire legacy parsers.
3. Build immutable prompt indexes by exact ID/version and an ambiguity indicator
   for omitted versions. Capture declared-input inspection data and prompt hashes.
4. Validate static template syntax using the renderer's grammar/function setup.
   Refactor that setup into one private owner if needed; do not render at load.
5. Test native and embedded sources, imported/inherited relative content from a
   different directory, explicit mapping context, contextless failure, handle
   closure, malformed static syntax, and resource mutation after capture. Use
   minimal fixtures, sharing scenarios where the same adapter is exercised.

Exit: loader creates usable frozen prompt definitions without request data.
`go test ./internal/configuration ./internal/prompt ./internal/promptdef` passes;
existing execution tests still work through the old assembly until cutover.

## Stage 3 — Make Execution Settings Presence-Aware

Scope: domain settings, target resolution, wire-presence propagation, profile
inspection. No new source discovery in execution packages.

1. Build the immutable backend registry from the final loaded backend map. Add
   execution-owned presence/suppression state and construct indexed profiles
   from loader output plus that registry. Apply registry-derived
   values after inheritance, using the field table above.
2. Replace scalar-zero/blank merge assumptions in target resolution. Configuration
   explicit zero reaches injected clients and the built-in request body; null does
   not resurrect a provider setting. Extend existing TargetPresence plumbing
   through preparation/results as necessary rather than creating a parallel path.
3. Preserve request-level override precedence, API-key availability behavior,
   endpoint-only profiles, backend identity/capacity, and partial-profile runtime
   completion. Move final completeness checks after overrides. InspectProfile
   returns partial settings without treating missing model/endpoint as load errors.
4. Test explicit numeric zero versus omission, map overlay and null, backend
   change without derived defaults leaking from parent, timeout null/zero,
   credential-source suppression, and completion through request overrides.
   Include one built-in-client wire test, not just an injected-client assertion.

Exit: focused domain/usecase/client tests pass. Any temporary adaptation for the
old assembly is private, clearly limited to reaching Stage 7, and does not become
a compatibility promise. Update affected target/presence GoDoc with this change.

## Stage 4 — Migrate The OpenRouter Catalog Assets

Scope: `../promptkit-backend-openrouter` only. Catalog maintainers own source and
publishing checks here; PromptKit consumes its configuration through ConfigKit.

1. Read that repository's instructions; preserve unrelated work. Convert each
   profile asset to `profiles: {literal-id: {settings...}}`, preserving inventory
   and values, changing `base_profile` to `extends`, and removing embedded ID.
   Keep per-profile files and the exported FS/Root package; no dependency on
   PromptKit and no second legacy asset tree.
2. Audit zero/empty values that previously meant unspecified. Remove those when
   omission was intended; retain intentional explicit values. Review any dynamic
   provider `extends` keys rather than silently changing outbound payloads.
3. Replace `catalog/backend.json` with `catalog/backend.yml` containing a
   `backends: {openrouter: {settings...}}` mapping. Preserve endpoint, API-key
   environment name, concurrency, queue, and extra-parameter values; remove the
   duplicate id and old manifest schema_version. The file is an ordinary ConfigKit
   input, not a separately decoded manifest. Update embedded assets, module path
   and imports to `/v3`, tests, and catalog format/release documentation, with an
   incompatible-format migration note. Keep FS/Root as the minimal export surface.
   Catalog-local publishing checks stay in that repository. No provider probes.
4. Validate under its procedure and commit. If publication is authorized, publish
   the next unused `v3.YYYYMMDD.N`; otherwise report the exact commit awaiting
   publication. The date/revision is selected then, not guessed in this plan.

Exit: one reviewed migrated catalog commit; existing immutable tags still serve
old consumers. The new major module path identifies the changed asset contract;
old consumers retain the published `/v2` assets.

## Stage 5 — Migrate The Rakestrawhome Catalog Assets

Scope: `../promptkit-backend-rakestrawhome` only. Apply the same keyed-format,
presence, notice, validation, and publication treatment as Stage 4 to its smaller
inventory, including the `/v3` transition and keyed `backends.rakestrawhome`
definition in `catalog/backend.yml`. Preserve backend endpoint, credentials
metadata, and capacity policy.
Do not couple its date/revision to OpenRouter's release.

Exit: both compatible catalog versions are published and ordinarily resolvable
before Stage 6 pins them. If permission or publication is unavailable, stop at
this external dependency checkpoint; do not commit a local replace, guessed
version, or legacy-to-new runtime converter to bypass it.

## Stage 6 — Integrate Catalog Sources And Maintained Profiles

Scope: `internal/catalog`, maintained definitions, catalog integration tests,
dependency pins/notices. Keep this separate from public API migration.

1. Pin the published compatible catalog versions and exact attribution commits.
   Update imports to `/v3`. Replace manifest decoding with ConfigKit file Inputs.
2. Supply backend and profile assets as ConfigKit Inputs in the first
   underlay. Supply PromptKit-owned keyed Mapping definitions as the next group:
   standard extends the selected catalog's current maintained base; auto retains
   its current direct model/backend definition. Preserve both literal names.
3. Retire source-layout allowlists, catalog preflight, source ownership/reserved
   prefix checks, connection-field prohibitions, and secret-key source rejection.
   The adapter selects known asset entry points and discovers configuration files;
   ConfigKit acquires their contents. Domain validation sees only the final
   assembled configuration. Consumers may overlay catalog backends, profiles,
   and their bases through the same configuration model.
4. Test real imported assets offline with the new loader, maintained-name
   resolution, consumer override of a maintained base/backend, and rejection of
   an invalid final backend. Test a replaced lower-layer domain error succeeds.
   Do not independently validate catalog-only intermediate configurations, parse
   raw assets in PromptKit tests, or audit origins to establish ownership.

Exit: catalog and owned definitions participate in the new keyed configuration
model without a legacy profile parser on that path. Catalog integration tests
pass against released modules with `GOWORK=off` and no replacements.

## Stage 7 — Cut Over Engine Assembly And Public Construction

Scope: `engine.go`, configuration facade/helpers, `internal/engine`, facade
adapters, and migration of their direct test setup.

1. Introduce the Construction Surface exactly as specified above. Remove the
   superseded source/profile options and convenience types rather than keeping
   deprecated pass-through wrappers. Keep intentional runtime injection APIs.
2. Load the combined backend/profile/prompt configuration, construct its runtime
   registry and indexed snapshot, then assemble renderer, validator, capacity
   manager, and runner. Return no Engine on construction failure.
   Translate construction errors to ErrInvalidConfig while preserving LoadError
   via errors.As. Resource errors include semantic path and safe source context.
3. Connect indexed prompt/profile lookups to preparation and inspection. Missing
   selections keep meaningful not-found errors; prompt version ambiguity stays a
   selection error. Old parse/load error categories need not be manufactured for
   errors now detected at construction.
4. Move root test fixtures/setup and maintained example construction to ConfigKit
   sources, keyed definitions, ProfileSource, and BackendSource so all consumers
   still compile. Update local-backend examples to choose their ID as a map key.
   Update expected timing only for the deliberate eager-validation change. Keep
   behavioral assertions for prompting, overrides, and errors.
5. Add public tests for a minimal custom local backend and programmatic profile,
   plus embedded defaults/operator/programmatic tiers. No elaborate consumer
   setup should be needed merely to send one prompt.

Exit: the public Engine uses only the new loading path. Root/package tests pass;
all package/example sources compile. Update GoDoc and consumer construction
guidance in this stage so removed options are not presented as usable.

## Stage 8 — Complete Frozen Rendering And Lifecycle Integration

Scope: renderer/runtime consumption of the snapshot, ownership, and lifecycle
regressions. Stage 2 already owns acquisition and static syntax validation.

1. Use captured text and parsed templates without per-run configuration reads,
   parsing, or inheritance. Bind input helpers per invocation safely; keep missing
   request data errors and direct-session template bypass behavior.
2. Verify preparation and execution still handle artifacts and schema compilation
   at invocation time, honor output overrides, freeze prepared resources, and
   keep prepared handles bound to their engine. Credentials remain at runtime.
3. Test an old engine after configuration/resource edits and a newly constructed
   engine observing the edits. Test concurrent renders with different Vars/input
   helpers and no data leakage. Use an FS that rejects reads after construction
   to demonstrate no configuration/content-file reopening during ordinary runs;
   do not incorrectly prohibit request artifact/schema reads.
4. Preserve appended-message ordering, structural repair, retries, admission,
   cancellation, observers, and result metadata through existing regression
   tests; add only distinct missing cross-boundary cases.

Exit: focused race tests pass for assembled rendering/preparation and lifecycle.
No retained callbacks capture ConfigKit resolution contexts or request state.

## Stage 9 — Expose Explicit Configuration Inspection

Scope: the read-only facade and retained ConfigKit inspection adapter.

1. Implement ConfigurationInspection and its specified methods without exposing
   Result.Config or filesystem capabilities. Retain a private narrow view of the
   ConfigKit result; no regeneration of origins from runtime values.
2. Ensure effective/explanation reports use ConfigKit redaction, including the
   selected sensitivity predicate. Preserve safe value-free domain diagnostics.
3. Test a targeted literal-key path with layered/inherited history, detached
   returned slices/records, writer failures, zero/nil behavior, and redacted secret
   shaped provider values. Demonstrate inspection still works after source removal
   and makes no filesystem/provider calls.
4. Document construction inspection versus InspectProfile and prepared details.
   Consumers choose report disclosure; do not add CLI commands or automatic logs.

Exit: public inspection tests pass, including concurrent inspection/execution.
The runner has no reference to the inspector or ConfigKit metadata types.

## Stage 10 — Migrate Examples And Canonical Format Guidance

Scope: maintained examples and user-facing documentation.

1. Complete the preparation, execution, and observation example migration begun
   at public cutover. Preserve their distinct existing semantic assertions; extend
   the preparation example with a small explicit inspection query if useful.
2. Update `docs/formats.md` to own the new keyed syntax, source-group semantics,
   inheritance/presence/null table, eager validation, snapshot lifetime, and
   resource-context rules. Link ConfigKit for generic grammar; do not copy its
   complete parser/inheritance contract.
3. Update README and `docs/consumers/pkg-promptkit.md` with minimal local-backend,
   file/embedded source, programmatic Mapping/ProfileSource, explicit inspection,
   and engine-replacement workflows. Explain main versus ordered underlays and
   backend layering in place of registration-only restrictions.
4. Add concise old-to-new migration guidance in the consumer guide: removed API
   options, exact IDs, `base_profile` to `extends`, zero/false/null, recursive maps,
   no clear via `{}`, reserved `extends` runtime escape, failure timing, and the
   fact that changing a model/backend does not clear unrelated explicit fields.
   Remove old automatic reload claims. Release notes later link to this owner.

Exit: all three offline examples run with their semantic checks; documentation
links and snippets match the implemented API. No unreleased version is tagged or
published, and no speculative release-note version is invented.

## Stage 11 — Remove Superseded Machinery And Consolidate Tests

Scope: dead loading/resolution code and its obsolete tests, module cleanup, and
internal architectural documentation.

1. Delete lazy prompt/profile metadata prepasses, filesystem repositories,
   fallback-on-error overlays, custom base-chain resolver/depth cap, zero-sentinel
   profile merging, and temporary stage adapters once no caller needs them.
   Remove the separate backend-manifest decoder, source-policy audits, and
   pre-handoff backend registration validation. Keep final-value domain checks.
   Retain narrow lookup interfaces only where they still simplify real consumers.
2. Retain useful filecatalog/schema utilities and JSON-value ownership/validation.
   Remove direct yaml.v3 use and dependency if no remaining owner requires it;
   do not claim all YAML dependencies disappear when ConfigKit still uses YAML.
3. Move meaningful regressions from deleted package tests to their new owning
   boundary. Retire tests about old internals, repeated parser coverage,
   source-policy enforcement, or superseded semantics. Do not replace lost runtime
   protection with loader-only tests. Keep private assembly checks separate from
   external facade tests.
4. Update `docs/internal/overview.md`, `docs/internal/sources.md`, relevant renderer
   and engine documents, and the implemented-structure paragraphs in architecture
   policy. Describe the new package/dependency boundary without turning names or
   chosen adapters into permanent invariants. Update OSS notices with module tidy.

Exit: no legacy source path remains, no internal package imports the public root,
runtime/domain packages do not import ConfigKit, and `go test ./...` passes with
the reduced implementation/test inventory.

## Stage 12 — Validate Scale And The Complete Consumer Workflow

Scope: final integration evidence and high-confidence simplification, not another
feature expansion or release gate.

1. Add focused benchmarks for construction with many independent definitions,
   shared inherited bases, and several tiers; add an already-constructed selection
   benchmark. Report allocations and input/output shape; inherited materialized
   output/history may dominate. Do not impose timing thresholds or new caps.
2. Inspect for repeated full-map copies, per-run provenance calls, rescans,
   repeated inheritance, and unnecessary double parsing. Fix demonstrated local
   inefficiencies with straightforward ownership/indexing; no global cache.
3. Run the repository's existing complete local check script once on the final
   candidate, including race tests and offline examples. Investigate failures;
   do not weaken meaningful tests to get a green result. Stage intended new
   files before the script's tracked-file checks.
4. Review the feature-roadmap capability list against actual public workflows:
   independent inspection, partial profile completion, optional credentials,
   source contexts, artifacts/schemas, prepared ownership, backend capacity,
   overrides, appended messages, repair/retries, observations, and hashes.
5. Report completed stages, selected catalog versions, validation evidence, and
   remaining issues. Minor-release documentation/version selection and roadmap
   retirement belong to the separately authorized release preparation workflow.

Exit: the feature roadmap's end state is demonstrably implemented, with one
configuration lifecycle, explicit operator inspection, no legacy loader, and
no newly invented provider checks or human release gates.

The catalog-validation placement question is resolved by the configuration-boundary
ADR. No design question remains open here. Catalog publication authority and
ordinary module availability remain operational checkpoints for the catalog stages.