8.0 KiB
Coordinated ConfigKit and PromptKit Adoption
Last updated: 2026-09-24
Status: Final target selected; production implementation pending. The shared boundary and offline prototype, including ConfigKit v0.4.0 overlays, are verified. Implementation stages are owned by the implementation plan. This roadmap defines the end state and user intent; it authorizes no publication.
Goal and Ownership
Adopt ConfigKit and PromptKit to align Audita with the application suite's shared configuration and LLM execution conventions. Consistency is the primary benefit; removing overlapping application infrastructure is a secondary benefit. Preserve Audita's transcript-correction capabilities and domain protections.
| Owner | Responsibilities |
|---|---|
| ConfigKit | Strict application configuration loading, presence/default semantics, explicit overrides, recorded resolution, secret references, and safe inspection |
| PromptKit | Prompt/profile formats and rendering, provider calls, structural output validation and repair, transient retries, engine-local backend admission, and execution observations |
| Audita | Setting meanings, file/CLI selection, role/profile association, normalization/chunking, ordered modules, domain validation, correction application, reports, diagnostic redaction, and artifact lifecycle |
The configuration roadmap owns configuration and inspection policy. The boundary findings own the selected defaults, full migration inventory, failure mapping, and prototype evidence. The selected production versions are ConfigKit v0.4.0 and PromptKit v0.11.0, with Go 1.25.5 as the minimum. Use published modules, never sibling checkouts or local replacements. The existing YAML dependency remains while glossary parsing needs it.
Migration Posture
This is an intentional breaking transition targeting Audita v2.0.0. No backward compatibility is required for existing installations, config files, CLI flags, environment overlays, or inspection output. Provide one migration guide, without legacy loaders, aliases, dual modes, or automatic conversion tooling.
Both libraries belong to one refactor. Intermediate commits need not build or run; there is no supported intermediate product state. No temporary adapters, compatibility shims, or processing gates are needed. The complete application must work and pass its checks before the refactor is finished. Release compliance and publication come last, with no intermediate public release.
Configuration and Execution Target
ConfigKit selected overlays enable a testing configuration that chooses cheaper or faster models. The example selects both proposal and validation profiles. ConfigKit application overlays and PromptKit execution profiles remain separate concepts. Complete operator-maintained profiles own model/generation controls; embedded application prompts and schemas are not operator-selected files. No catalog wiring, profile-inheritance feature, ConfigKit imports, or secret-directory bootstrap is part of this migration.
Capacity controls serve two needs: respecting a local service's hard simultaneous- query limit and bounding concurrent paid API activity. One finite limit per backend is shared across both roles and every initial, retry, and repair attempt. Limits are engine-local, not coordinated across processes. No additional global or per-role limiter or fine-grained scheduling policy is required. Profiles must use registered backends and ConfigKit-resolved credentials; endpoint-only bypass and competing profile credential lookup are excluded.
Structural repair is enabled by default for both roles because additional attempts to obtain valid output are desired. Transient retries and structural repairs have separate budgets; explicit zero disables the respective behavior. Each generation, including a repair, receives its own transient-retry budget. The Audita logical-call deadline covers preparation, waiting, backoff, all generations, and repairs; profile HTTP timeouts apply per attempt. Exact defaults and bounds are in the selected contract. No Audita outer retry loop restores the old broader retry policy.
Audita retains all four correction modules, five LLM-validator prompts, deterministic validators, confidence/applicability checks, glossary protection, validation batching/cardinality, deterministic proposal and ledger ordering, transcript encoders, and artifact retention. PromptKit-valid JSON still passes Audita's domain checks. Repair exhaustion, provider failure, and domain rejection follow the selected failure table. Retention and transcript timestamp changes remain separate work; this migration resolves the retry-count follow-up.
Reporting and Diagnostic Target
Keep stable Audita-owned serialized records; do not expose raw library structs.
Use process report schema v2 with the existing domain outcomes and a new
llm_summary separating logical calls, dispatched provider attempts, transient
retries, structural repairs, and usage. Transcript schema versions remain unchanged.
The implementation plan specifies the concrete new fields and artifact wiring.
Usage distinguishes known zero, unknown, and incomplete totals. Preserve observations from failed calls even when no final result exists. Count terminal totals once; attempt observations provide diagnosis without being added to those totals again. This is operational evidence, not a complete billing ledger.
Replace global/role scheduler utilization with per-backend configured capacity and observed queue/generation/retry/logical-call timing. Do not fabricate measurements that the event API does not provide. Keep domain timing and deterministic correction ledger ordering. Record prompt definition/rendered hashes and selected profile/model without promising the old hash values. Callbacks are short, concurrency-safe, and free of blocking I/O or waits on their own execution.
Effective configuration and selected-profile snapshots remain distinct safe views. Secret values must not escape through inspection, logs, reports, runtime errors, or retained diagnostic files. Observation records exclude payloads, URLs, raw provider messages, prompts, and credentials. Existing artifact lifecycle and partial-failure handling remain Audita responsibilities.
Completion Criteria and Release Boundary
- Both libraries own their selected responsibilities in the production application. Superseded loaders, transport, renderers, schedulers, aliases, and the isolated prototype are gone once its useful evidence is represented in production tests.
- Offline tests prove configuration-to-execution routing for both roles, selected overlays and explicit overrides, shared backend capacity, retry/repair outcomes, cancellation/deadlines, domain decisions, redaction, and truthful partial reports. Tests exercise real internal collaborators and controlled HTTP boundaries.
- Maintained examples and canonical docs describe the implemented final model. One migration guide covers files, flags, environment inputs, profiles, credentials, overlays, inspection, and changed runtime/report/diagnostic semantics.
- The assembled application passes its required checks; Linux amd64 and arm64 build. macOS arm64 remains best effort. No live-provider or CI-success gate is added.
PromptKit v0.11.0's reviewed retry, observation, and capacity APIs remove the earlier upstream feature prerequisites; no upstream edit is required for adoption. The prototype proved representative behavior, not release compliance or model quality.
The selected PromptKit release uses LGPL-3.0-or-later. A later release unit must review the actual production dependency graph and distribution requirements, including notices and source/rebuild materials, before binary publication. This implementation plan does not claim that review is complete. Release publication uses the existing procedure and completes on successful source/tag push, without waiting for CI or artifact delivery.