scriptorium/docs/roadmap/configkit.md

ConfigKit Adoption

Status And Intent

Accepted feature direction, drafted on 2026-09-26; not implemented. This roadmap defines the target behavior and changes from the reviewed baseline. A separate implementation plan will own sequencing and concrete implementation steps. The existing implementation.md concerns the completed Promptkit upgrade and is not an implementation plan for this work.

The user is adopting ConfigKit across an application suite to standardize configuration semantics. Scriptorium will make a clean cutover to ConfigKit's native conventions, including intentional breaking changes. Preserving the legacy loader's behavior is not a goal: there will be no parallel legacy loader, compatibility mode, or recreation of the old merging rules.

Alignment applies to application configuration, not to Promptkit-owned prompt, profile, or schema definitions. Departures from ConfigKit conventions require an application-specific reason or a genuine ownership boundary, not backward compatibility alone. The architecture policy and ADR 0004 remain the scope boundaries.

Reviewed Baseline

Scriptorium consumes Promptkit v0.11.0. Its internal/config package uses yaml.v3 directly and manually applies defaults, file settings, and CLI overrides. It has a parser-specific integer decoder, ignores empty string overrides, and treats an empty configuration file as defaults-only input. It has no application configuration imports, selected overlays, or provenance/inspection interface. The configuration reference, CLI reference, and source internals own the current behavior.

The reviewed ConfigKit release is v0.4.0 at commit e6ced0d3d1194b58e0b3dc8970089471ba175258. Adopt that published release as the initial target, rather than a sibling checkout or unpublished replacement. Its canonical references are:

• Configuration contract.
• Consumer adoption guide.
• Release notes.

ConfigKit requires Go 1.25, within Scriptorium's existing toolchain requirement. It selects go.yaml.in/yaml/v4 v4.0.0-rc.6. Removing Scriptorium's direct parser usage does not remove yaml.v3 from the dependency graph: Promptkit still uses it. The initial review passed ConfigKit's default test suite and Scriptorium's configuration and CLI tests; that is baseline evidence, not migration validation.

Target Ownership And Loading Model

Owner	Responsibility
ConfigKit	Application YAML policy, source acquisition, composition, overlays, typed decoding, explicit override application, provenance, and redacted inspection.
Scriptorium	Source discovery, application schema and defaults, CLI parsing, recorded application normalization, semantic validation, command prerequisites, and runtime conversion.
Promptkit	Framework definition formats, backend semantics, credential lookup, engine defaults, and prompt execution.

internal/config becomes a small application-specific ConfigKit integration, not a second general-purpose configuration framework. Execution and inspection share one configuration definition and loading pipeline. Defaults, selected base and imports, selected overlay, and explicit application CLI overrides participate in ConfigKit's lifecycle; resolution and validation follow its upstream rules. Normalization that changes inspected values is recorded through its resolver. Runtime conversion does not perform a hidden second merge or apply new defaults after the inspection snapshot has been created.

Scriptorium retains its application field names and domain meanings unless a specific change below requires otherwise. ConfigKit's public root API replaces direct YAML decoding and parser-specific hooks. Application defaults remain Scriptorium-owned; omitted framework settings still delegate to Promptkit. Backend conversion retains deterministic registration and engine-scoped shared capacity state. This work does not change the HTTP request contract or move configuration loading into each HTTP request.

Native Configuration Semantics

The target accepts ConfigKit's upstream YAML grammar, schema binding, limits, composition, and value-presence rules without a legacy preprocessing layer. Migration-visible changes include:

• An existing empty file is invalid; {} expresses an empty application mapping. Defaults-only operation with no discovered file uses NoSource instead.
• Explicit empty strings, false values, and zeroes replace defaults. They are not silently treated as omission. Application validation still decides whether a resulting value is meaningful for its field or selected command.
• Null follows the application's declared nullable types. It is not a universal spelling for omission. Scalars and structs do not accept null merely because the former parser tolerated it.
• Anchors, aliases, merge keys, explicit tags, additional documents, unsupported scalar spellings, and coercions rejected by ConfigKit are rejected here too.
• ConfigKit's reserved configkit key and resource limits apply, including in dynamic application data. These constraints do not apply to independently loaded Promptkit definitions or HTTP request bodies.
• Dynamic integers in backend extra_params use ConfigKit's int64 representation. Runtime mapping preserves their supported value semantics rather than reserializing through YAML or demanding the old Go scalar type.

Presence remains meaningful at the framework boundary: omitted/null transient_retries delegates to Promptkit, while explicit zero disables retries; omitted backend queue capacity differs from explicit zero. Application HTTP size defaults and zero-as-unlimited behavior remain. No new value ranges or backend restrictions are introduced merely to adopt the library.

Discovery, Imports, Overlays, And Paths

Source discovery remains Scriptorium-owned. The target retains the documented explicit --config selection and ordered system-location search. If neither implicit candidate exists, loading uses defaults only; a selected file that fails loading is not silently replaced with defaults. No implicit working-directory configuration search, environment overlay, or directory scan is added.

Application configuration supports ConfigKit's explicit path and glob imports. Imports compose disjoint definitions; repeated non-mapping definitions are conflicts, not last-writer-wins overrides. There is no implicit conf.d scan.

An optional --config-overlay selects ConfigKit's one named overlay using its standard overlays/<name>.yml location, naming rules, and merge behavior. No overlay is selected by default. Selecting an active overlay without a base source follows ConfigKit's rejection of overlays with NoSource; Scriptorium does not invent a root. Application overlays do not replace --profile, profile_dir, or Promptkit's execution-profile selection.

Use the selected file's canonical directory as ConfigKit's default import and overlay root. This root governs configuration source traversal only. Values of prompt_dir, profile_dir, schema_dir, and server.artifact_root retain working-directory-relative interpretation; absolute values remain absolute. This is an explicit application path policy, not an upstream requirement or a legacy parsing mode. ConfigKit's consumer examples demonstrate alternatives but do not mandate source-file-relative application paths. This adoption does not extend import containment to application data or alter HTTP artifact containment.

CLI Overrides And Configuration Inspection

Only flags explicitly supplied by the user become ConfigKit application override records. Parser defaults do not overwrite file or overlay settings. Explicit empty values are not dropped. The existing invocation-specific prompt, profile, input, model, timeout, output, and reporting options remain invocation state rather than new application configuration fields. In particular, the run/render retry flag remains a request override; it does not mutate the engine's application retry setting or backend state.

Expose application configuration validation, effective redacted YAML, source provenance, and explanation of an individual configuration path. These are included features, not merely library capabilities available for future work. They use ConfigKit diagnostics and inspection records from the shared loading pipeline, including defaults, imports, overlays, explicit CLI values, and recorded normalization. No separate inspection-only interpretation is maintained.

Configuration inspection does not execute prompts, make provider calls, or resolve API-key values. Application validation is not a promise that every Promptkit backend or definition is valid or that a provider is reachable. Command-specific execution prerequisites remain with the relevant command; inspection can explain an incomplete application configuration without requiring an executable prompt selection. The implementation plan will specify command names, flag availability, output streams, and exit behavior without adding a second CLI framework or an HTTP configuration endpoint.

Safe diagnostics must not restore raw parser errors or configuration values that ConfigKit deliberately suppresses. Effective output and explanation use the upstream redacted renderers; they are inspection output, not guaranteed reloadable configuration. Dynamic-value sensitivity needs the explicit choice recorded under Open Questions before inspection is implemented.

Credential Boundary And Deferred Features

Existing api_key_env values remain environment-name strings passed to Promptkit. They do not become ConfigKit Secret fields. ConfigKit eagerly resolves effective typed references and requires nonempty values; that would change Promptkit's optional/required and selected-credential behavior. This departure from the secret-reference feature is an ownership decision, not compatibility plumbing. Raw API-key fields remain unsupported under the existing architecture policy.

Secrets-directory bootstrap and application schema versioning are deferred until there is a concrete application need. Their availability in ConfigKit does not make them mandatory. Hot reload, runtime resource replacement, remote imports, automatic migrations, and general environment-variable overlays are outside this adoption. These are scope limits for this feature, not new permanent prohibitions on future explicitly requested work.

Required Changes And Completion Criteria

• Replace manual decoding/default/override machinery with the shared ConfigKit integration and retain only application-specific discovery, rules, and mapping.
• Support imports and selected overlays across commands that load application configuration, and deliver the configuration inspection capabilities above.
• Update application configuration types, presence handling, CLI explicit-set mapping, and diagnostics to match the selected native semantics.
• Update maintained examples and canonical configuration, CLI, operations, architecture, and internal documentation when behavior lands. Upstream generic contracts remain linked rather than copied into a competing specification.
• Add proportionate integration coverage for the cutover: defaults-only input, explicit values and nulls, imports/conflicts, overlays, CLI precedence, inspection agreement and redaction, dynamic backend parameters, unchanged credential ownership, and representative execution/server startup. ConfigKit owns exhaustive parser and composition tests; Scriptorium tests its adoption and application boundaries.
• Pin the published dependency and maintain ConfigKit and YAML attribution under the OSS policy, using imported-commit source/notice links and applicable standard and companion license texts.
• Document migration in a release appropriate for the intentional breaking configuration changes under the release policy. Release selection and publication remain separate preparation work; this roadmap adds no release gate, network probe, or new distribution format.

Completion means one ConfigKit-native application configuration path serves execution and inspection, the selected user-facing features work, and migrated examples and regression coverage demonstrate the documented semantics. It does not require adopting every ConfigKit feature or changing Promptkit definitions.

Open Questions

Sensitivity Of Arbitrary Backend Parameters

ConfigKit redacts classified paths; it cannot infer secrets in arbitrary provider-specific extra_params. The new inspection surface needs an explicit classification policy, including superseded values shown in explanation.

Recommended: classify each backend's entire extra_params container as sensitive for inspection. That avoids guessing provider-specific secret names and covers historical values without restricting which parameters may be passed to Promptkit. The tradeoff is that benign provider options are also hidden.

Alternative: maintain an explicit application classification of sensitive paths, leaving other parameter values visible. This gives more useful inspection but requires a clear mechanism for classifying arbitrary provider options and ongoing maintenance; key-name heuristics alone are not a reliable substitute. This is an inspection-policy choice, not authority to reject new backend parameter keys.