8.0 KiB
Coordinated ConfigKit and PromptKit Adoption
Last updated: 2026-09-23
Status: Agreed high-level direction; provisional roadmap. Detailed design, implementation tasks, dependency versions, and acceptance criteria must be revisited before migration begins. This document authorizes neither implementation nor release publication and sets no schedule.
Goal and Ownership
Adopt ConfigKit and PromptKit to align Audita with the application suite's shared configuration and LLM execution conventions. Remove overlapping application infrastructure while keeping transcript-correction responsibilities in Audita.
| Owner | Intended responsibilities |
|---|---|
| ConfigKit | Application configuration loading, presence/default semantics, explicit overrides, recorded resolution, secret references, and safe configuration inspection |
| PromptKit | Prompt/profile formats and rendering, provider calls, structural output validation, and shared execution capabilities supplied by the selected release |
| Audita | Configuration field meanings, CLI parsing and file selection, transcript normalization/chunking, module ordering, domain validators, correction application, reports, and diagnostic artifact lifecycle |
This roadmap owns sequencing across the two adoptions. The
ConfigKit roadmap owns the configuration migration's proposed scope.
PromptKit consumer requests are recorded upstream in
../promptkit/docs/roadmap/audita-feature-requests.md; they remain proposals until
accepted and released there. Do not treat proposed upstream capabilities as present.
Migration Posture
Treat adoption as an intentional breaking transition. Preserve intended domain capabilities, not legacy configuration files, flags, environment variables, or configuration-inspection formats. Supply one migration guide rather than parallel legacy loaders, compatibility aliases, or permanent translation layers.
Plan to ship both adoptions together as Audita v2.0.0, without an intermediate public release. Implementation should still consist of coherent, reviewable units. Revisit the final release scope and sequencing alongside other large planned changes, especially changes to module or pipeline configuration.
Proposed Sequence
1. Design the Shared Configuration Boundary
Define the target settings model for both libraries before implementing either migration. Decide how Audita selects proposal and validation execution profiles, supplies credentials, selects retry behavior, and assigns per-backend concurrency limits. Give each setting one authoritative home; avoid duplicate model, endpoint, credential, or timeout definitions with competing precedence.
Keep ConfigKit application configuration distinct from PromptKit execution profiles. In particular, optional ConfigKit configuration profiles and PromptKit execution profiles are different concepts. Select an explicit profile-ownership model; complete Audita-owned execution profiles are the initial recommendation, with catalog-driven selection a deliberate alternative.
Capacity needs are simple: protect local backends from exceeding their available simultaneous-query capacity and give paid API backends a reasonable finite cap. Proposal and validation calls using the same backend share its limit. Separate proposal/validation budgets and additional fine-grained Audita scheduling controls are not required. Account for the engine-local scope of PromptKit's existing limits.
2. Prove the PromptKit Boundary
Before finalizing the new application configuration schema, exercise one proposal call and one LLM-validator call through PromptKit using controlled offline responses. Verify profile selection, credentials, embedded prompts and schemas, shared backend capacity, failed output validation, cancellation, and diagnostic mapping.
Establish how retries and content repair differ and how their budgets and execution observations reach Audita. Keep optional content repair disabled initially unless a specific need justifies it. Audita's domain acceptance checks remain necessary even when generated output satisfies a JSON schema.
Use the prototype to resolve genuine boundary gaps and inform configuration design. It is evidence for the migration, not a permanent alternate runtime. Avoid finalizing ConfigKit-facing settings around the old LLM runtime only to redesign them later.
3. Complete ConfigKit Adoption
Implement the agreed application configuration contract following the ConfigKit roadmap. Retire legacy environment overlays and obsolete CLI/configuration conventions, update maintained examples and canonical references, and begin the combined migration guide.
If the existing LLM runtime remains temporarily during this stage, keep translation into it small and explicitly transitional. Do not preserve its old settings merely to make the temporary bridge convenient. No compatibility mode is owed to users.
4. Complete PromptKit Adoption
Move prompt rendering, execution profiles, provider calls, and structural output validation onto the selected PromptKit release. Remove overlapping transport, prompt-loading, and scheduling machinery once the replacement is verified.
Retain Audita's transcript pipeline, semantic and deterministic validators, correction application, reports, ledger, and artifact retention. Map PromptKit results, errors, metadata, and execution observations into those responsibilities. Remove the temporary bridge from the preceding stage.
5. Validate and Release the Combined Migration
Validate the assembled configuration-to-execution workflows, including local and API backend limits, failure/cancellation behavior, credential handling, diagnostics, and domain outcomes. Use deterministic tests and controlled endpoints rather than paid calls as acceptance gates. Update documentation and examples together with implemented behavior and remove obsolete contracts and code.
Complete one migration guide covering files, CLI, environment settings, credentials, execution profiles, inspection, and changed diagnostics. Release the combined breaking transition under the established release procedure, with no CI waiting added to release completion.
Parallel Upstream Preparation
PromptKit work can proceed alongside Audita's design and ConfigKit implementation. Prioritize evaluation of transient-failure retries, actionable failure classification, and execution observations, supported by a representative consumer example. Existing per-backend capacity controls are the intended foundation; additional scheduling features are not assumed necessary.
Resolve the proposed LGPL 3.0 licensing change and applicable distribution terms before selecting PromptKit for adoption. Recheck released APIs, toolchain requirements, and dependency versions at implementation time. Pin published library releases; production builds must not depend on sibling checkouts or local module replacements.
If a required upstream capability is not ready, complete independent work and reassess sequencing explicitly rather than quietly recreate that capability as permanent Audita infrastructure. The prototype should identify what is genuinely required for adoption versus useful later enhancement.
Coordination and Open Decisions
- Finalize the configuration/profile boundary, defaults, credential ownership, and retry semantics before committing to the new file schema and CLI design.
- Define how failed validation, exhausted retries, and partial execution observations map to Audita warnings, rejected work, and failed runs.
- Coordinate the runtime retry-count question with this design. Retention and transcript timestamp questions are not automatically included.
- Revisit ordering when other major changes are defined. Prefer designing shared module/pipeline settings once, while keeping unrelated work independent.
- Flesh out task-level scope, tests, and acceptance criteria before implementation. Completion requires the intended ownership split, removed transitional machinery, verified assembled behavior, and one coherent migration path for users.