audita/docs/roadmap/configkit.md

ConfigKit Adoption Roadmap

Last updated: 2026-09-24

Status: Prototype boundary and selected overlays verified. The ordered production handoff below is planned, not implemented; no production migration or release has begun.

Purpose and Direction

Adopt ConfigKit to harmonize Audita's configuration policy with the other applications in the suite. Consistent configuration conventions are the primary benefit; reducing application-owned configuration infrastructure is a secondary benefit and a useful measure of integration quality.

This is an intentional breaking migration. There is no goal of backward compatibility with existing installations, configuration files, CLI flags, environment variables, or configuration-inspection output. Provide a migration guide instead of legacy loaders, aliases, dual operating modes, or an ongoing compatibility layer. Retain existing syntax only when it independently suits the new model, not because compatibility requires it.

The initial review and prototype used ConfigKit v0.3.1. The prototype now pins v0.4.0 and verifies its selected-overlay path; original v0.3.1 findings remain historical. The installed application must depend on a versioned module, not a sibling checkout or local module replacement.

The shared boundary/prototype unit is complete. Its selected contract, inventory, and implementation handoff refine the provisional directions here; production configuration migration has not begun.

Proposed End State

• ConfigKit owns strict YAML loading, input presence, default application, explicit override mechanics, recorded resolution, secret references, and safe inspection.
• Audita owns its configuration types, field meanings, defaults, schema-version policy, file selection, CLI parsing, domain validation, and runtime initialization.
• Adopt ConfigKit's precedence and validation lifecycle rather than reproduce Audita's current per-layer validation and environment-overlay behavior.
• General configuration environment overrides are retired. Credentials use typed environment-name secret references; supplied references must resolve according to ConfigKit's contract. Review config-file selection separately as an application-owned startup decision.
• Standardize duration representations, explicit empty/null/zero handling, and inheritance on the shared conventions. Express genuine Audita domain rules through the supported resolution and validation APIs.
• Review the configuration-related CLI afresh. Useful explicit flags can become ConfigKit override records; old names and aliases are not requirements. Direct credential-value flags should give way to the secret-reference convention.
• Use ConfigKit's redacted effective output and provide source/explanation access where useful. Existing JSON inspection output need not be preserved. Coordinate diagnostic configuration snapshots with the new model.
• Delete superseded loading and override machinery. Do not hide the old model behind adapters that defeat the adoption's consistency benefit.

Keep Audita's domain capabilities, such as ordered and repeated correction modules, expressible in the new model. Their existing configuration spelling is not binding. Transcript/glossary formats, processing algorithms, and output schemas are outside this migration unless later planning identifies a concrete dependency or an explicitly coordinated change.

Selected Overlays for Testing

Enable ConfigKit's optional selected-overlay feature in the migration. A testing configuration that selects cheaper, faster LLMs is the initial use case. ConfigKit overlays modify application settings; PromptKit execution profiles still own model and generation settings. Keep profiles_dir, the PromptKit profiles/ directory, and proposal/validation profile names unchanged by this terminology change.

The planned interface is --overlay <name> on process, config validate, and config print-effective. Treat selection as invocation state passed to Options.Overlay, not as another persistent setting or environment overlay. Follow ConfigKit v0.4.0's selection contract: omission, an empty name, or default selects no overlay; a named selection loads overlays/<name>.yml beneath the selected configuration root. A missing selected overlay is an error. Use the library's single-overlay semantics without adding stacking, inheritance, or legacy-directory fallback.

Precedence is application defaults, main configuration, selected overlay, then explicit setting flags. Perform Audita's role inheritance and semantic validation on the resolved configuration. ConfigKit's base parsing and schema-version checks still precede overlay application; overlays cannot replace the schema version or contain ConfigKit directives. General environment-variable overrides remain retired. Imports and secret-directory bootstrap remain disabled.

Provide a maintained overlays/testing.yml example that selects testing execution profiles through llm.proposal_profile and llm.validation_profile. Model names and generation controls belong in the corresponding PromptKit profile files. Set both role selections in this example so a base configuration's explicit validation profile cannot accidentally retain the production model. Normal omission-based validation-profile inheritance continues to apply to the final effective settings. The example must make its backend/credential requirements clear and use no real secrets. Model choice is operator-controlled; “testing” does not promise a price, performance level, or equivalent correction quality.

Overlays may modify other ordinary application settings supported by the schema; they are not restricted to profile selection. The resulting configuration must still satisfy shared backend capacity, credential, and domain rules. Validation, effective-config inspection, selected-profile inspection, and processing must all use the same selected overlay. Preserve overlay provenance in configuration explanations and diagnostics while retaining secret redaction.

The completed prototype extension and production-planning handoff are recorded in the prototype roadmap and its ordered implementation plan. Production CLI changes below remain subsequent migration work.

Implementation and acceptance work:

• The isolated prototype pins ConfigKit v0.4.0 and includes the testing-overlay fixture. Preserve the original v0.3.1 evidence as historical; the extension results establish the selected-overlay behavior.
• Verify unchanged base selection when no overlay is requested, both testing role selections, and explicit setting flags taking precedence over the overlay.
• Verify role inheritance after overlay application, effective profile/backend assembly, and secret resolution from the surviving configuration.
• Reject missing/invalid overlay selections and prohibited version/directive changes through ConfigKit; rely on its tests for exhaustive merge/parser cases.
• Verify redacted inspection and overlay provenance, then run both LLM roles through the offline provider using the selected testing profiles. Provider requests must use their model selections; no paid calls are needed.
• Add production CLI coverage for the three commands and document selection, precedence, example layout, and migration guidance when implemented.

This expands the next configuration unit; it does not reopen the completed prototype's original scope or change the ConfigKit → PromptKit → combined release sequence. The production LLM migration remains necessary before promising the full new runtime behavior.

Ordered Production Migration Handoff

The migration inventory is the field-by-field old-to-new map. The selected boundary owns defaults and semantic choices; the v0.4.0 overlay results prove the representative load, inspection, and HTTP path. These packages are future work. Complete each dependency and its tests before removing the code it replaces. Do not treat the isolated prototype as a second production runtime.

1. Define and load the complete version-2 configuration

Depends on: the verified prototype. Owns: internal/core/config and its tests. Replace the version-1 YAML/env application path with one ConfigKit v0.4.0 definition and a version-2 rule. Pin the published module in the root Go module and raise the production Go minimum as needed. Keep the existing YAML dependency used for glossary parsing. ConfigKit imports and secret-directory bootstrap stay disabled.

The definition must cover every retained domain setting in the inventory: ordered/repeated pipeline.modules, output.schema, validation prompt budget, chunking, normalization, thresholds, context, and diagnostics, plus profiles_dir, backend endpoint/concurrency/queue capacity/optional credential, and LLM proposal and validation profile selection, transient retries, repair attempts, and total operation timeout. Apply the inventory's defaults once. Preserve presence: omitted validation profile inherits the final proposal profile; explicit empty/null validation fails. Omitted or null section target means automatic, zero fails. Explicit empty module list means no modules. Optional credential null means unauthenticated; supplied references must resolve to nonempty values. Keep the selected numeric-second normalization fields numeric, and use a duration string for llm.operation_timeout; do not revive version-1 duration coercion. Validate backend limits, references, profile selections, thresholds, and cross-field domain rules on the resolved result. Preserve ConfigKit provenance for derived values.

Acceptance/tests: internal/core/config tests use temporary files to cover version rejection, strict fields, defaults, omission/null/zero, module order, final role inheritance, overlays and explicit override precedence, secret replacement/missing/null, and retained domain validation. Compare behavior to the prototype extension rather than duplicating ConfigKit's parser matrix. No legacy environment overlay or old-schema acceptance remains.

2. Select files and map invocation input

Depends on: package 1. Owns: internal/cli/run.go, internal/cli/process_flags.go, internal/cli tests, and cmd/audita subprocess tests. Select exactly one main file: explicit --config, then the two system locations in the selected boundary. Remove AUDITA_CONFIG; a missing explicit path fails, while absent system files are skipped. Require an explicit file for config validate. Parse help before loading configuration.

Add invocation-only --overlay <name> to process, config validate, and config print-effective, passing it to Options.Overlay for the same selected main file. Omitted, empty, or default selects the base. Keep transcript, glossary, output, and report paths outside persistent configuration. Map only flags actually supplied by the user into ConfigKit overrides; parser defaults must never defeat file or overlay values. Retain useful domain-setting flags from the inventory. Add profile-selection flags. Remove direct API-key/model/endpoint, old retry/role/global concurrency flags and aliases rather than translating them silently. Retire all general AUDITA_* configuration overlays listed in the inventory; environment names retain meaning only when explicitly referenced as secrets.

Acceptance/tests: CLI and subprocess tests cover discovery order, explicit missing files, all three commands with base/testing/default/missing selections, overlay then explicit-setting precedence, omitted flags, usage/error streams and exit classifications. Reject old flags and implicit environment configuration. The same loaded result must feed validation, effective inspection, and processing.

3. Assemble profiles and safe inspection

Depends on: packages 1–2. Owns: a small CLI/runtime assembly boundary, config inspection, and diagnostic snapshot wiring in internal/cli and internal/core/diagnostics; verify against internal/framework/processreport where snapshots are referenced. Resolve profiles_dir relative to the selected main file, including when an overlay is selected; absolute paths remain valid. Inspect both selected PromptKit profiles and verify their named registered backend, endpoint equality, and absence of a competing profile credential lookup. Treat ConfigKit settings and PromptKit generation controls as separate views.

Use WriteEffective for redacted effective YAML and expose relevant explanation/provenance and selected-profile model/backend controls through an Audita-owned safe view. A ConfigKit snapshot alone cannot claim to describe the full execution target. Validation and inspection resolve surviving secret references without provider calls; a missing or empty surviving reference fails, while a replaced reference is not resolved. Snapshot and surfaced error paths must redact secret values and preserve overlay/CLI origin. Avoid serializing raw library objects into stable report formats.

Acceptance/tests: config-command and diagnostics tests verify relative paths, same selection across views, overlay/CLI origin, surviving credentials, unauthenticated null, and no secret values in YAML, explanations, selected-profile inspection, diagnostics, or errors. Update internal/cli/run_test.go, internal/core/config/*_test.go, and report/diagnostics tests that own these outputs.

4. Convert runtime inputs without claiming unavailable execution

Depends on: packages 1–3. Owns: a narrow conversion at the CLI to internal/framework/llm/runner boundary, with focused internal/cli and internal/framework/llm tests. Convert retained domain settings to their current domain structs. Convert selected profile/backend endpoint, model, resolved secret, and per-attempt timeout to existing client setup values for assembly checks; keep library types out of modules, validators, transcript schemas, and ledgers.

The interim conversion has this explicit scope; "assemble" means values can be validated and converted, not that processing is enabled:

New setting or control	Interim disposition
profiles_dir, proposal/validation profile IDs	Honor for relative-path profile loading, final inheritance, inspection, and target assembly.
backends.<id>.endpoint and credential	Honor for selected target validation and client setup; keep ConfigKit secret resolution and redaction.
Profile model and timeout_seconds	Convert for old client setup checks; old transport can consume these per-attempt values.
validation.max_prompt_tokens; retained domain, output, and diagnostics fields	Honor through existing domain/batching/report inputs after validation.
backends.<id>.concurrency and queue_capacity	Validate and inspect; old global/role pools cannot enforce independent per-backend admission or queue semantics.
llm.transient_retries	Validate and inspect; old retry eligibility and explicit-zero behavior differ.
llm.repair_attempts	Validate and inspect; old runtime has no PromptKit structural-repair contract.
llm.operation_timeout	Validate and inspect; old per-call timeout is not the full logical-operation deadline.
Other PromptKit profile generation controls and execution observations	Inspect only; old transport/reporting cannot claim equivalent behavior.

The old global/role pools and retry loop must not be presented as equivalents. The selected interim behavior is to reject process before any LLM request with a clear unsupported-runtime classification while ConfigKit has landed but PromptKit has not. config validate and effective/profile inspection remain usable. This is an unreleased development interval, not a user-visible mode or compatibility switch. The process gate is removed only when the PromptKit package below proves the complete settings through controlled calls.

Acceptance/tests: conversion tests verify the representable values and redaction; a process subprocess test proves the interim rejection occurs before provider transport or output mutation. No test should assert that old retries or schedulers implement new settings. The combined migration cannot release with this gate in place.

5. Remove superseded configuration paths and update maintained contracts

Depends on: packages 1–4. Owns: removal of old file/env merge, aliases, direct-key flags, legacy redaction/inspection paths, and obsolete tests in internal/core/config and internal/cli. Retain Audita's domain checks, normalization/chunking, output encoding, module and validator behavior. Update examples/ with version-2 base configuration, overlays/testing.yml, complete PromptKit profiles, and secret-free credentials by reference. Update the canonical docs/config.md, docs/cli.md, operations/integration/internal references as their contracts change, and provide one migration guide with the complete inventory's file/flag/environment mapping, duration and empty-list changes, credentials, profiles/overlays, inspection, and changed runtime/diagnostic semantics. Do not leave dual loaders or a permanent translation mode.

Acceptance/tests: maintained examples validate offline; removed flags and old schema are rejected; relevant config/CLI/subprocess/report tests pass; documentation links and git diff --check pass. The interim process restriction remains explicit until package 6 and is not documented as a released product feature.

6. Replace execution with PromptKit, then prepare the combined release

Depends on: packages 1–5. This is the later PromptKit work unit, not part of this ConfigKit implementation. Replace the old provider transport, prompt loader, structural-output handling, and scheduler composition as described by the coordinated adoption roadmap and execution/observation handoff. Prove both roles' profile/backend settings, per-backend capacity, explicit-zero retry, positive repair, total-operation deadline, domain decisions, and safe observations with controlled transport. Remove the interim converter/process gate and superseded LLM machinery only after those tests pass. Update report and diagnostic contracts and the migration guide. The combined release and license/ distribution review follow last under the release policy; no publication is authorized by this plan.

Historical Provisional Outline

The completed shared-boundary and prototype unit supplies the initial target settings model and integration evidence. Extend that evidence with the selected-overlay work above during the next configuration unit.

1. Define the Contract and Coordinate Sequencing

Inventory existing configuration fields, invocation options, environment settings, secret handling, inheritance rules, and inspection consumers. Produce an old-to-new mapping for migration guidance and identify which concepts should be retained, redesigned, or removed. This is an inventory of needs, not a compatibility checklist.

Specify the target ConfigKit conventions and the boundary between configuration and invocation-only state. Decide file discovery, the new file schema version, CLI shape, validation/inspection behavior, and optional feature scope. Identify application requirements that the library cannot express cleanly before building workarounds.

Coordinate these decisions with the other planned large changes to Audita. Finalize an implementation plan and concrete acceptance criteria before beginning migration.

2. Prove a Representative Configuration Path

After detailed planning, implement a bounded path from file selection and CLI inputs through ConfigKit to Audita's runtime configuration. Exercise defaults, selected overlays, proposal/validation inheritance, concurrency, duration values, secrets, and redacted inspection using representative valid and invalid inputs.

Use the result to assess whether domain rules remain clear and substantial old infrastructure can be removed. Resolve genuine library gaps deliberately, including upstream discussion or changes when justified; avoid rebuilding a parallel loader. A prototype is evidence for the design, not a permanent compatibility mode.

3. Complete the Migration

Convert the remaining configuration and command paths, remove superseded code and legacy interfaces, and update maintained examples and canonical documentation alongside the implementation. Replace tests that enforce retired conventions with coverage of the new contract; preserve meaningful domain and runtime coverage.

Provide a concise migration guide covering old-to-new fields, flags, environment settings, durations, credentials, validation, and inspection behavior. Automated conversion tooling is not assumed to be necessary or in scope.

4. Validate and Release the Breaking Change

Verify representative loading and CLI workflows, domain resolution, safe secret handling, diagnostics, examples, and runtime integration with deterministic tests. Confirm obsolete loaders and aliases are gone and that documentation describes the implemented model. Use the established release procedure.

Recommend Audita v2.0.0 and a new configuration-file schema version. Clearly reject unsupported old file versions and direct users to the migration guide. Final release numbering and whether this ships with other breaking changes remain sequencing decisions; no publication is authorized by this roadmap.

Sequencing Considerations

Changes to module configuration, runtime settings, CLI design, or diagnostic reporting are likely to overlap this migration. Design their shared target model first. If another change will substantially reshape those areas, avoid migrating the same concepts twice. Work independent of configuration can proceed separately.

The coordinated library-adoption roadmap owns the proposed sequence with PromptKit: shared boundary design and a PromptKit prototype, then ConfigKit implementation, then PromptKit implementation, followed by a combined v2.0.0 release. Revisit that sequence alongside other major changes. Keep implementation units reviewable even when the release combines them.

The separate runtime behavior decisions track remaining questions and the selected retry direction. The coordinated plan adopts PromptKit's explicit retry semantics and enables structural repair from the outset. Include separate transient-retry and positive repair budgets, plus the total-operation deadline policy, in the shared settings design. The selected defaults and ownership are recorded in the boundary findings. Do not implement the runtime change merely as a side effect of replacing the configuration loader. Retention and transcript timestamp decisions are not automatically included.

Implementation Boundaries

• Pin ConfigKit v0.4.0 in production and align Go/toolchain requirements. ConfigKit requires Go 1.25; the combined PromptKit prototype requires Go 1.25.5. Config-only adoption does not automatically remove Audita's existing YAML dependency used for glossary parsing.
• Implement the selected type shape, final role inheritance, and interim runtime gate above without leaking library concerns throughout the pipeline.
• Enable selected overlays as specified above. Keep imports and secret-directory bootstrap disabled unless a separate concrete requirement justifies them.
• Validation and inspection resolve surviving credential references and fail on missing or empty values; no credential-free bypass is planned.
• Add the safe selected-profile view and ConfigKit provenance to inspection and diagnostics as specified in package 3, then account for changed formats in the migration guide. No schedule or effort estimate is committed.

The eventual completion test is that Audita follows the suite's ConfigKit conventions while retaining its intended domain capabilities, with a clear migration guide and no ongoing obligation to support the previous configuration model.