10 KiB
Coordinated ConfigKit and PromptKit Adoption
Last updated: 2026-09-24
Status: Shared boundary and offline prototype completed; production migrations remain planned. The v0.4.0 overlay evidence extends the completed prototype, and the ordered ConfigKit handoff specifies the next implementation work. This document authorizes no release publication and sets no schedule.
Goal and Ownership
Adopt ConfigKit and PromptKit to align Audita with the application suite's shared configuration and LLM execution conventions. Remove overlapping application infrastructure while keeping transcript-correction responsibilities in Audita.
| Owner | Intended responsibilities |
|---|---|
| ConfigKit | Application configuration loading, presence/default semantics, explicit overrides, recorded resolution, secret references, and safe configuration inspection |
| PromptKit | Prompt/profile formats and rendering, provider calls, structural output validation, and shared execution capabilities supplied by the selected release |
| Audita | Configuration field meanings, CLI parsing and file selection, transcript normalization/chunking, module ordering, domain validators, correction application, reports, and diagnostic artifact lifecycle |
This roadmap owns sequencing across the two adoptions. The ConfigKit roadmap owns the configuration migration's proposed scope. PromptKit v0.11.0 is the reviewed baseline for the adoption prototype. It supplies transient HTTP retries, execution observations, typed failure categories, an offline proposal/assessment example, and LGPL-3.0-or-later licensing. The earlier upstream consumer requests are no longer outstanding prerequisites on those points. Audita has not yet adopted either library; recheck the selected releases during detailed planning.
Migration Posture
Treat adoption as an intentional breaking transition. Preserve intended domain capabilities, not legacy configuration files, flags, environment variables, or configuration-inspection formats. Supply one migration guide rather than parallel legacy loaders, compatibility aliases, or permanent translation layers.
Plan to ship both adoptions together as Audita v2.0.0, without an intermediate public release. Implementation should still consist of coherent, reviewable units. Revisit the final release scope and sequencing alongside other large planned changes, especially changes to module or pipeline configuration.
Proposed Sequence
The shared-boundary and prototype roadmap records the completed unit covering stages 1 and 2 below. The findings select its target contract and identify the next ConfigKit implementation work.
1. Design the Shared Configuration Boundary
Define the target settings model for both libraries before implementing either migration. Decide how Audita selects proposal and validation execution profiles, supplies credentials, selects transient-retry and structural-repair budgets, and assigns per-backend concurrency limits. Give each setting one authoritative home; avoid duplicate model, endpoint, credential, or timeout definitions with competing precedence.
Keep ConfigKit application configuration distinct from PromptKit execution profiles. ConfigKit v0.4.0 calls its optional configuration layers overlays. Enable selected overlays for testing as specified in the ConfigKit roadmap; these select application settings, including PromptKit execution profiles. Select an explicit profile-ownership model; complete Audita-owned execution profiles are the initial recommendation, with catalog-driven selection a deliberate alternative.
Capacity needs are simple: protect local backends from exceeding their available simultaneous-query capacity and give paid API backends a reasonable finite cap. Proposal and validation calls using the same backend share its limit. Separate proposal/validation budgets and additional fine-grained Audita scheduling controls are not required. Account for the engine-local scope of PromptKit's existing limits.
2. Prove the PromptKit Boundary
Before finalizing the new application configuration schema, exercise one proposal call and one LLM-validator call through PromptKit using controlled offline responses. Verify profile selection, credentials, embedded prompts and schemas, shared backend capacity, failed output validation, cancellation, and diagnostic mapping.
Enable structural repair from the outset for proposal and LLM-validator outputs. Additional attempts to obtain valid output are desired behavior, not a deferred optimization. Select explicit positive repair budgets during detailed planning and exercise invalid output that becomes valid after repair, as well as exhausted repair budgets.
Keep transient HTTP retries and structural repair distinct. With the reviewed library, each generation, including a repair, has its own transient-retry budget; plan and document the combined request allowance and overall deadline. Adopt PromptKit's explicit zero-disables-retries semantics instead of Audita's legacy zero-as-default behavior. The library's narrower transient retry policy does not retry transport failures or malformed provider responses; do not restore the old policy through an overlapping application retry loop.
Verify that initial, retry, and repair calls share backend limits and that their observations reach Audita without double-counting usage. Audita's domain acceptance checks remain necessary after structural repair succeeds; valid JSON does not imply an acceptable correction or validator decision.
Use the prototype to resolve genuine boundary gaps and inform configuration design. It is evidence for the migration, not a permanent alternate runtime. Avoid finalizing ConfigKit-facing settings around the old LLM runtime only to redesign them later.
3. Complete ConfigKit Adoption
Implement the agreed application configuration contract following the ConfigKit roadmap. Retire legacy environment overlays and obsolete CLI/configuration conventions, update maintained examples and canonical references, and begin the combined migration guide. Include the selected testing-overlay path and its offline integration evidence from the ConfigKit roadmap.
ConfigKit and PromptKit adoption form one refactor. These sections describe implementation order, not separately runnable milestones. Intermediate commits need not build or run. Build directly toward the final integration without temporary adapters, compatibility shims, or an interim processing gate; validate the complete application before declaring the refactor finished.
4. Complete PromptKit Adoption
Move prompt rendering, execution profiles, provider calls, and structural output validation onto the selected PromptKit release. Remove overlapping transport, prompt-loading, and scheduling machinery once the replacement is verified.
Retain Audita's transcript pipeline, semantic and deterministic validators, correction application, reports, ledger, and artifact retention. Map PromptKit results, errors, metadata, and execution observations into those responsibilities. Connect the new configuration directly to this execution path and remove the superseded configuration and runtime machinery.
5. Validate and Release the Combined Migration
Validate the assembled configuration-to-execution workflows, including local and API backend limits, successful and exhausted repairs, transient retries, failure/cancellation behavior, credential handling, diagnostics, and domain outcomes. Use deterministic tests and controlled endpoints rather than paid calls as acceptance gates. Update documentation and examples together with implemented behavior and remove obsolete contracts and code.
Complete one migration guide covering files, CLI, environment settings, credentials, execution profiles, inspection, and changed diagnostics. Release the combined breaking transition under the established release procedure, with no CI waiting added to release completion.
Reviewed Upstream Readiness
The v0.11.0 review covered the relevant implementation, the full test suite, targeted race tests, and the offline proposal/assessment example; those checks passed. Existing per-backend controls cover the requested capacity model, including retry and repair interactions. No further upstream feature request is currently a prerequisite to the prototype. Keep the planned adoption sequence unchanged.
Remaining integration work includes selecting a total-operation deadline alongside per-attempt timeouts, using shared backend identities with finite limits, and mapping PromptKit observations to Audita-owned diagnostic records. Observation callbacks are synchronous and may overlap across runs; collection must be short and safe for concurrent use. Distinguish unknown or incomplete usage from known zero and do not treat observations as a complete billing ledger or a stable serialized report format.
The licensing change has shipped as LGPL-3.0-or-later. Account for applicable notices and source/rebuild distribution requirements in Audita's release design; relicensing no longer needs to be requested upstream. Recheck released APIs, toolchain requirements, and dependency versions at implementation time. Pin published library releases; production builds must not depend on sibling checkouts or local module replacements.
If the prototype reveals a genuine remaining gap, evaluate it explicitly rather than quietly recreate shared library functionality as permanent Audita infrastructure.
Coordination and Open Decisions
- Use the selected configuration/profile boundary, defaults, credential flow, and retry/repair/deadline contract in the findings when detailing the ConfigKit production migration.
- Implement the outcome and observation mapping recorded in the findings, retaining Audita domain validation and diagnostic redaction.
- Resolve the runtime retry-count follow-up through this migration. Retention and transcript timestamp questions are not automatically included.
- Revisit ordering when other major changes are defined. Prefer designing shared module/pipeline settings once, while keeping unrelated work independent.
- Flesh out task-level scope, tests, and acceptance criteria before implementation. Completion requires the intended ownership split, removed transitional machinery, verified assembled behavior, and one coherent migration path for users.