# ConfigKit Adoption Implementation Plan ## Status, Authority, And Historical Plans Planned on 2026-09-26 against Scriptorium commit `43eea5c7c41d2a8134b3dece60f3af5c98b8fd1c`. No stage is implemented by this planning change. Execute Stage 1 through Stage 7 in order. Each stage is sized as one focused gpt-5.6-terra implementation task, including tests and current documentation. The maintainer has resolved the inspection-policy question: `extra_params` remain visible; only API-key values require secret protection. The [feature roadmap](configkit.md) owns scope, target behavior, and user intent. This plan owns implementation steps and the interface choices needed to execute them. Read [development](../development.md) and every policy before work: [agents](../policy/agents.md), [architecture](../policy/architecture.md), [documentation](../policy/documentation.md), [testing](../policy/testing.md), [release](../policy/release.md), and [OSS](../policy/oss.md). [ADR 0004](../adr/0004-definition-boundary.md) continues to govern Promptkit's definition boundary. Do not rewrite accepted ADR decisions. This filename previously contained completed Promptkit plans. Older roadmap links to this file refer to these historical records, not the ConfigKit stages: - Completed v0.11 implementation and release handoff: `git show 43eea5c7c41d2a8134b3dece60f3af5c98b8fd1c:docs/roadmap/implementation.md`. - Completed v0.9 implementation: `git show e4405966c87c8a7843b340146e90cc1e515bef42:docs/roadmap/implementation.md`. Those deliveries must not be repeated or lost from upcoming release preparation. These Git records are available locally without publishing the recent commits. ## Working Rules And Selected Design The dependency target is `gitea.maximumdirect.net/eric/ConfigKit v0.4.0`, commit `e6ced0d3d1194b58e0b3dc8970089471ba175258`. Use its public root package, imported as `configkit`. The roadmap links the exact upstream contracts. Do not opportunistically upgrade Promptkit or other dependencies, commit a workspace/replacement, or depend on sibling source at runtime. This is a clean configuration cutover, not a compatibility project. Native ConfigKit syntax, presence, nullability, resource limits, and precedence apply. Do not add a pre-parser, YAML serialization round trip, alternate merge engine, legacy mode, generic provider abstraction, or a new CLI framework. Keep application source discovery and domain rules in `internal/config`; CLI parsing/output stays in `internal/adapter/cli`. Promptkit remains responsible for prompt/profile/schema parsing, backend semantics, credentials, and engine behavior. Keep one shared server engine and existing HTTP artifact policy. No HTTP schema, execution-report, request-timeout, or retry-orchestration change is part of this plan. For every stage: 1. Inspect the working tree and relevant code/tests through the repository's code-discovery guidance. Preserve unrelated changes. 2. Implement the listed outcome and only supporting refactoring it needs. Internal names below are selected implementation guidance, not public APIs or immutable package-layout contracts. 3. Update canonical current-state docs when behavior lands. Until Stage 3 the new loader is internal preparation, not an advertised executable feature. Never claim later stages are implemented. 4. Run focused tests and the development guide's baseline validation for code changes. Tests stay offline, deterministic, and free of real credentials. Reuse existing coverage rather than cloning ConfigKit's parser test matrix. 5. Format changed Go files, inspect the diff, validate affected docs/examples, record stage status/evidence here, and commit the coherent unit. Do not push, tag, publish, or add new release gates. ### Application Configuration Definition Use a new `Document` type in `internal/config` for the native configuration shape. It replaces the old on-disk `Config` after CLI cutover. Keep the familiar YAML field names: | YAML path | Go shape | Default and selected meaning | | --- | --- | --- | | `prompt_dir`, `profile_dir` | `string` | Empty; no source configured. | | `schema_dir` | `string` | Existing schema-directory default. Empty remains empty at the application boundary; Promptkit owns any subsequent framework default. | | `transient_retries` | `*int` | Nil; omitted/null delegates, zero disables additional attempts. | | `server` | Non-pointer struct | Its children receive typed defaults; null is invalid. | | `server.addr` | `string` | Existing listen-address default when absent; explicit empty is passed unchanged to the standard HTTP server, not replaced with the Scriptorium default. | | `server.artifact_root` | `string` | Empty; HTTP file input remains disabled. | | Three `server.max_*_bytes` fields | `int64` | Existing defaults; explicit zero disables the respective limit; null is invalid. | | `defaults` | Non-pointer struct | Contains `render_format`; null is invalid. | | `defaults.render_format` | `string` | Existing text default; resolved value must be text or json. Explicit empty is invalid, not defaulted. | | `backends` | `map[string]BackendDocument` | Nil; null is permitted, an empty mapping follows ConfigKit semantics. | `BackendDocument` keeps `endpoint string`, `api_key_env string`, `extra_params map[string]any`, `concurrency_limit int`, and `queue_capacity *int`. Scalar nulls are invalid; nullable maps/pointers retain native semantics. No catalog IDs or Promptkit fields are added. The fresh defaults factory reads existing `internal/defaults` and format constants, not a duplicated set of literals. Normalization trims surrounding whitespace from application directory strings, artifact root, and server address; cleans only nonempty application paths; and lowercases/trims `defaults.render_format`. Empty paths remain empty: never let `filepath.Clean("")` turn disabled/unset into `.`. Record actual changes with `configkit.Normalize`, using value-free reasons. Do not normalize backend fields or parameter values on Promptkit's behalf. The read-only validator reports negative application size limits, negative nonnull application retries, and invalid render format through `Validation.Error` at their field paths. It does not print rejected values. Do not call the existing format parser's empty-as-default branch for this field. Do not add endpoint, backend ID, credential-presence, directory-existence, address-format, or positive-only-capacity checks. Existing Promptkit and command-specific validation retains those responsibilities where applicable. ## Stage 1: Add The Dependency And Native Application Definition **Status: Complete.** The pinned dependency, native definition, focused presence/normalization/validation tests, and commit-specific attribution are in place. The existing executable loader remains active pending the command cutover. Primary files: `go.mod`, `go.sum`, new focused files under `internal/config`, and `docs/oss/attribution.md`. 1. Add ConfigKit at the selected release. Retain Promptkit v0.11.0 and its selected catalogs. Provision dependencies outside offline test execution. ConfigKit's yaml/v4 release-candidate dependency is the reviewed upstream choice, not an instruction to upgrade its parser independently. 2. Implement `Document`, `BackendDocument`, the fresh defaults factory, recorded resolver, and read-only validator described above. Keep the old loader operational until Stage 3 so this stage remains independently buildable; this temporary development state is not a shipped compatibility mode or a second user-selectable configuration path. 3. Test the new definition using real `configkit.Load[Document]` with `Bytes` and `NoSource`. Cover explicit zero/empty/null distinctions, retry/queue nil-versus-zero, safe application validation, and a normalized path whose effective output agrees with the returned value. Include nested unknown fields and a representative dynamic numeric parameter. 4. Add ConfigKit and yaml/v4 attribution from their selected source material. Resolve the imported yaml/v4 commit from release/module metadata. Preserve actual copyright statements; do not infer a project statement from the FSF license-document copyright. Use applicable SPDX expressions, local standard texts, and commit-pinned upstream license/notice links. Reuse GPL, LGPL, MIT, and Apache texts where applicable, including companion links. Retain yaml.v3 attribution because Promptkit still incorporates it. 5. Document the internal definition as preparation in the relevant internal guide without claiming the CLI has switched. No current configuration contract changes yet. Acceptance: the dependency and definition compile, focused tests demonstrate native semantics and audited normalization, and attribution follows the OSS policy. Existing executable behavior still passes its tests. ## Stage 2: Build Discovery And The Shared Loading Boundary **Status: Complete.** Ordered source discovery, one ConfigKit load, typed diagnostics, and settings conversion are covered by focused file/default and override tests. Existing commands still use the old entry point until cutover. Primary files: `internal/config` and its tests; `docs/internal/sources.md`. 1. Add `LoadOptions` with `Path string`, `Explicit bool`, `Overlay string`, and `Overrides []configkit.Override`, plus `Load(LoadOptions) (*configkit.Result[Document], error)`. Discovery chooses a source; the one ConfigKit call owns actual file reading and decoding. Return the result so consumers retain the frozen inspection snapshot as well as the typed configuration. 2. With `Explicit=true`, pass the selected path to `configkit.File`; an explicitly empty path is invalid, not implicit discovery or a fallback to the old system filename. Do not silently trim a source filename into a different filename. ConfigKit validates and resolves file sources. 3. With `Explicit=false`, ignore the option's path value and examine the existing ordered implicit search locations. Only nonexistence permits examining the next candidate. Other stat failures are errors. Select the first existing candidate and let ConfigKit load it. A selected file's later disappearance/read/parse failure does not restart discovery. 4. If neither implicit candidate exists, use `NoSource`. Initially leave imports disabled and the overlay empty; Stage 4 enables composition. Defaults-only loads still accept explicit application overrides. Never reinterpret an empty file as `NoSource`. 5. Pass overrides into the ConfigKit call before decoding/validation; do not validate a separately merged lower configuration first. Native parsing and composition failures still follow the upstream lifecycle and are not bypassed by CLI overrides. 6. Add a small `Settings(Document) AppSettings` conversion for existing executable wiring. It copies resolved fields, maps the already-validated format, and converts the backend map into ID-sorted backend settings. It performs no file IO, normalization, credential lookup, semantic backend checks, or additional defaults. Dynamic values retain their decoded shapes. 7. Preserve ConfigKit's typed `*LoadError` through wrapping so the CLI can render safe structured diagnostics. Discovery-only errors are concise and quote paths safely; no new public error taxonomy is needed. 8. Test ordered discovery with temporary paths rather than host system files, explicit missing/empty paths, empty-file rejection versus `{}`, defaults-only loading, source records, override-before-validation behavior, sorted backend conversion, and preservation of zero/nullable values. Remove no existing entry point until all its CLI callers migrate in Stage 3. Acceptance: real files and defaults-only inputs produce one reusable result, runtime conversion agrees with it, and tests do not depend on machine-wide configuration or secrets. The old and new internals are temporarily buildable, with no new user-visible mode. ## Stage 3: Cut Existing Commands Over And Remove The Legacy Loader **Status: Not started.** Primary files: `internal/adapter/cli/run.go`, `inspect.go`, `internal/config/config.go`, and associated tests. 1. Route run, render, serve, inspect prompt, and inspect profile through the shared `Load` and `Settings` boundary. Preserve the explicit presence of `--config` in every parser. Centralize the small application-flag mapping rather than reproduce it in each command. 2. Use `flag.FlagSet.Visit` (or the existing explicit-set helper) to create one `configkit.Override` per explicitly supplied application flag. Use `configkit.Keys`, built-in scalar values, and a safe `Flag` label containing only the flag name. Standard flag parsing's last occurrence supplies the single override; do not emit duplicate records or parser defaults. | Flag | Application path | Existing command scope after cutover | | --- | --- | --- | | `--prompt-dir` | `prompt_dir` | run, render, serve, inspect prompt | | `--profile-dir` | `profile_dir` | run, render, serve, inspect profile | | `--schema-dir` | `schema_dir` | run, render, serve | | `--addr` | `server.addr` | serve | | `--artifact-root` | `server.artifact_root` | serve | | `--max-request-bytes` | `server.max_request_bytes` | serve | | `--max-artifact-bytes` | `server.max_artifact_bytes` | serve | | `--max-response-bytes` | `server.max_response_bytes` | serve | Register the schema-directory flag on render as well as run/serve so the advertised application path override is consistently usable. This is not a general expansion of runtime/provider flags to other commands. 3. Keep `--format`, `--out`, request `--transient-retries`, prompt/profile selection, model overrides, timeouts, inputs, variables, and reporting options outside the application configuration tree. In particular, `render --format` overrides that invocation's output format without rewriting `defaults.render_format`; omitted format inherits its resolved application value. Existing invocation-only parsing remains unchanged. 4. Remove post-load directory trimming/cleaning/defaulting from argument finalizers, including unconditional schema-path cleaning. Keep only invocation-owned transformations such as output-file paths. Explicit empty profile/artifact-root overrides must clear file-provided values; explicit empty prompt directory must reach the existing execution prerequisite rather than revive the lower value. 5. Keep existing command prerequisites: run/render/serve and prompt inspection need their effective prompt directory as currently required; profile inspection does not acquire a prompt-directory requirement. All commands still perform the common application schema validation. Do not introduce eager filesystem or credential validation for inspection. 6. Add one CLI diagnostic path using `errors.As` and `configkit.WriteDiagnostics` for load errors. Carry successful load warnings to the command boundary and render them once on stderr before execution or output. Handle diagnostic writer errors as command failures. Do not flatten structured failures into raw parser text or print rejected values. Preserve existing process exit classifications; application/parse/load failures are exit 1, not generated-output validation exit 2. 7. Remove the old YAML decoder, `strictInteger.UnmarshalYAML`, old on-disk types, `applyConfig`, `ApplyCLIOverrides`, and unused compatibility sentinels/helpers after migrating all callers/tests. Keep only one defaults factory. Remove the direct yaml.v3 import; module tidy may retain yaml.v3 as an indirect Promptkit dependency. 8. Update tests to the deliberate breaking semantics, not the legacy behavior. Protect explicit empty path clearing, scalar-null rejection, unknown fields, rejected empty/multi-document input, one representative YAML policy change, unchanged request-versus-engine retry precedence, absent optional credentials, and command-specific inspection prerequisites. 9. Update configuration and CLI contracts, source/adapter internals, architecture ownership, and the contributor reading guide for the delivered cutover. Preserve existing application defaults, Promptkit definition support, HTTP containment, shared capacity, and report behavior. Adjust the current CLI claim that every command requires a prompt directory to match actual command-specific requirements. Acceptance: all existing commands use ConfigKit, no legacy parser or second application merge remains, relevant tests pass, and diagnostics and docs describe the native breaking behavior. Imports/overlays and new inspection commands are not yet advertised. ## Stage 4: Enable Explicit Imports And Selected Configuration Overlays **Status: Not started.** Primary files: shared loader, CLI source-selection wiring, configuration and CLI tests, maintained configuration examples. 1. Enable `Options.Imports` for file-backed loads. Leave it false for `NoSource`, which rejects imports. Use ConfigKit's default canonical selected-file directory as the root; do not supply a wider filesystem root or add application traversal code. 2. Wire `LoadOptions.Overlay` into `Options.Overlay`. Register `--config-overlay` on all five existing configuration-loading command modes. Treat the value exactly as an upstream overlay name: no trimming, case folding, extension fallback, path expansion, or local inheritance. Empty and exact lowercase default mean no active overlay under v0.4.0. A requested active overlay without a selected base source fails through ConfigKit; do not manufacture an empty base file or root. 3. Use native `configkit.imports` directives for explicit paths/globs. The application enables the feature without requiring a second CLI opt-in; absent directives cause no discovery. Retain conflict-based base composition and native overlay replacement/null semantics. 4. Preserve working-directory-relative application path values even when authored in fragments or overlays. Configuration-source containment is not the HTTP artifact policy and is not applied to prompt/schema paths. 5. Add focused real-file tests for disjoint backend fragments, an import conflict, overlay replacement, explicit CLI precedence, missing selected overlay, and no-base overlay failure. Verify `--profile` still selects a Promptkit execution profile independently of `--config-overlay`. One real command workflow proves wiring; upstream owns exhaustive glob, symlink, cycle, and parser-limit tests. 6. Add a small synthetic import/overlay example under `examples/`; preserve a simple single-file quickstart. Update config/CLI contracts and operations guidance for the new features, root selection, and relative-value policy. Acceptance: supported commands load the same native composition pipeline; imports are explicit, overlays remain distinct from execution profiles, and no new scanning, remote acquisition, or path policy is introduced. ## Stage 5: Add Application Validation, Effective Output, And Source Inspection **Status: Not started.** Primary files: a focused `internal/adapter/cli/config.go`, shared config definition, CLI dispatch/help/tests, and canonical CLI/config documentation. 1. Apply the maintainer's selected inspection policy: no redaction for `extra_params`, including historical values. API-key values are the only relevant secrets, but this schema stores only `api_key_env` reference names. Keep those names visible and never resolve them in the ConfigKit load. Do not add secret tags to the reference name or parameter container, a catch-all secret predicate, sensitive-key guessing, or a selective masking interface. Actual credentials remain with Promptkit; raw API-key fields remain rejected. Use upstream inspection renderers without adding a second value-rendering path or a synthetic credential field. 2. Add a top-level `config` command family using the existing standard-library flag style. The selected interfaces are: | Command | Success stdout | | --- | --- | | `scriptorium config validate [flags]` | Empty; exit status communicates application validation success. | | `scriptorium config show [flags]` | `Result.WriteEffective` redacted YAML. | | `scriptorium config sources [flags]` | One newline-terminated JSON array of source records. | | `scriptorium config explain --path [flags]` | Added in Stage 6. | These are prospective interface specifications, not commands to execute before their stages are implemented. 3. All configuration subcommands accept `--config`, `--config-overlay`, and the eight application override flags in Stage 3's table, regardless of which execution command normally uses them. They do not accept request, credential, backend-editing, output-format, or output-file flags. Shell redirection is sufficient for these inspection outputs. No positional arguments are accepted. 4. Load exactly once through the common loader. Do not construct a Promptkit engine, restricted artifact reader, listener, or client. Missing prompt directories, unselected/absent credentials, and framework backend validity are not additional application-inspection gates. Invalid application types/ranges/format still fail. Validation describes application load validity, not end-to-end executability. 5. For sources, map `Result.Sources()` to a small local DTO with JSON keys `id`, `label`, `layer` (always present), and `path` (omit when empty). Preserve upstream source order; emit `[]\n` for defaults-only input without overrides. Include synthetic CLI sources where present. Do not marshal opaque ConfigKit implementation objects. The standard JSON encoder escapes control characters; no custom terminal rendering is necessary. 6. Diagnostics/warnings go only to stderr through the shared diagnostic helper. Successful operations exit 0; parse/load/output/diagnostic-write failures exit 1. No exit 2 meaning is introduced for configuration validation. Ordinary writer failures may leave partial output; do not promise atomic streams or buffering of an entire configuration. 7. Test defaults-only show/sources, a composed load with CLI provenance, explicit empty clearing, normalized effective values, failed loads producing no successful stdout, and writer failure. Set a synthetic API-key environment variable: effective output must show its reference name, not its value. Repeat with the variable absent to establish that inspection does not require credentials. Assert that ordinary synthetic parameter values remain visible and runtime ExtraParams stay intact; no real environment or provider is used. 8. Update CLI/config contracts, README navigation, subprocess integration, operations, and internal inventory to describe only these delivered modes. Retain the README's license/attribution ending. Acceptance: validation, effective output, and source inspection use the same resolved configuration as execution, preserve stream separation, and honor the selected API-key boundary without masking or changing backend parameter data. ## Stage 6: Add Per-Path Explanation And Verify Credential Boundaries **Status: Not started.** Primary files: configuration command implementation/tests and CLI/config docs. 1. Implement `config explain` with the same selectors and override flags as Stage 5 plus a required, explicitly supplied `--path`. Parse using `configkit.ParsePath`; syntax is JSON Pointer, not dotted notation, a file path, or a URL fragment. Explicit `--path ""` selects the root; an omitted flag is an error. Preserve escaped and numeric mapping-key semantics. 2. Use the successful shared result's `WriteExplanation`. Explain known effective and historical paths, including removed values. An unknown or malformed path exits 1 with no successful explanation output. Quote user-supplied paths safely if included in application diagnostics. Do not rebuild history, look up environment values, or reload sources. 3. Keep the upstream human-readable renderer and its redaction semantics rather than inventing a JSON explanation schema or freezing the entire rendered prose as an application guarantee. Writer failures exit 1 and can leave partial output, as with effective output. 4. Test a value's default/base/overlay/CLI history, recorded normalization, explicit equal-value writes, an escaped backend key, root selection, and an unknown path. Cover a displaced historical backend parameter value: it remains visible, including in a container/root explanation. Use distinct synthetic environment names and key values to verify reference names remain explainable but resolved API-key bytes are absent from all history/output. 5. Update CLI/config docs and operations guidance with path syntax, historical-value meaning, representation differences, and sensitive identifiers/paths. Do not claim that redaction detects arbitrary secrets outside classified paths. Acceptance: explain uses the existing snapshot and upstream semantics; its history matches the selected inputs and normalization, parameters remain visible, and explanation neither resolves nor discloses API-key values. ## Stage 7: Verify Integration, Finish Documentation, And Record Release Handoff **Status: Not started.** 1. Review the feature roadmap against actual code. Confirm all five original command modes and all four new configuration subcommands use one definition and load pipeline. Remove any transitional loader, duplicated defaults, unused compatibility helpers, or post-snapshot configuration mutation. Do not add architecture machinery merely to assert a particular file layout. 2. Exercise maintained single-file and import/overlay examples through the implemented configuration commands and a no-provider render workflow. Use focused backend integration evidence that dynamic integer parameters reach Promptkit correctly and missing optional credentials retain their established meaning. Use a local HTTP server only where provider serialization must be checked; never a real service or credential. 3. Retain existing HTTP concurrency, containment, error mapping, retry, timeout, and report tests. Add a representative configuration-to-server wiring test only if existing coverage misses changed settings propagation; do not reproduce all upstream scheduling or HTTP cases. 4. Run the development baseline on final code: `go test ./...`, `go test -race ./...`, `go vet ./...`, and `go build ./cmd/scriptorium`. Check formatting, affected examples/links, and `git diff --check`. Do not stage the built binary or scratch files. Compare any new standard license text byte-for-byte; preserve upstream whitespace rather than editing license wording to satisfy a style check. 5. Ensure current configuration, CLI, subprocess, operations, internal docs, contributor routing, and architecture ownership agree. Testing policy should identify ConfigKit as the owner of exhaustive upstream loading tests, while Scriptorium owns application adoption tests. Change policy dates when revising policies, not base versions unless adopting new templates. Keep accepted ADR text and published release notes immutable. 6. Record the release handoff below and mark only delivered stages complete. Update the feature roadmap's status, including the resolved inspection decision. Preserve deferred work. The next breaking release follows the release policy; version selection, candidate publication validation, release notes at that version, tag creation, and publishing are not this sprint. 7. Commit the final coherent handoff and report validation evidence, remaining limitations, and any outstanding work without claiming source/binary publication or complete legal compliance. Acceptance: the ConfigKit-native target is delivered with proportionate regression evidence and accurate documentation, and the workspace contains no temporary dependency substitutions or uncommitted implementation artifacts. ### Release Handoff To Retain The upcoming release needs migration guidance for empty files versus `{}`, explicit empty values, nullable versus scalar fields, strict YAML/scalars, reserved keys and upstream size limits, and the invalid explicit-empty config path. Include imports/overlays, application path interpretation, CLI presence, inspection command scope, and the selected API-key-only secret policy. Explicit empty server address reaches standard HTTP server behavior instead of restoring Scriptorium's default; operators should supply their intended listen address. Keep request retry overrides and Promptkit credential semantics distinct from application settings. Explain that configuration validation is not a provider probe or complete Promptkit definition validator, and that inspection output is not guaranteed reloadable. Include dependency/attribution changes and the existing LICENSE | ATTRIBUTION footer. Preserve the still-relevant Promptkit v0.11 release handoff identified above when preparing a combined next release. ## Open Questions None remain for this scope. On 2026-09-26 the maintainer selected no redaction for `extra_params` and specified that the API key is the only secret requiring ConfigKit redaction. Stages 5–6 implement that decision through visible parameter values/reference names and the existing separation of actual credentials from application configuration. Do not reopen it or substitute blanket masking. This does not claim automatic detection of credentials embedded in arbitrary strings or provider parameters. The application schema still rejects raw API-key fields, and Promptkit remains responsible for resolving named credential sources.