130 lines
8.1 KiB
Markdown
130 lines
8.1 KiB
Markdown
# Coordinated ConfigKit and PromptKit Adoption
|
|
|
|
**Last updated:** 2026-09-26
|
|
|
|
**Status:** Production implementation complete. ConfigKit v0.4.1 and PromptKit
|
|
v0.11.1 are adopted; [historical implementation evidence](https://gitea.maximumdirect.net/eric/audita/src/commit/cb405f6/docs/roadmap/implementation.md) records the
|
|
completed checks. The current `implementation.md` covers span deletion instead.
|
|
Release compliance and publication remain separate work.
|
|
|
|
## Goal and Ownership
|
|
|
|
Adopt ConfigKit and PromptKit to align Audita with the application suite's shared
|
|
configuration and LLM execution conventions. Consistency is the primary benefit;
|
|
removing overlapping application infrastructure is a secondary benefit. Preserve
|
|
Audita's transcript-correction capabilities and domain protections.
|
|
|
|
| Owner | Responsibilities |
|
|
| --- | --- |
|
|
| ConfigKit | Strict application configuration loading, presence/default semantics, explicit overrides, recorded resolution, secret references, and safe inspection |
|
|
| PromptKit | Prompt/profile formats and rendering, provider calls, structural output validation and repair, transient retries, engine-local backend admission, and execution observations |
|
|
| Audita | Setting meanings, file/CLI selection, role/profile association, normalization/chunking, ordered modules, domain validation, correction application, reports, diagnostic redaction, and artifact lifecycle |
|
|
|
|
The [configuration roadmap](configkit.md) owns configuration and inspection policy.
|
|
The [boundary findings](library-boundary-findings.md) own the selected defaults,
|
|
full migration inventory, failure mapping, and prototype evidence. The selected
|
|
production versions are ConfigKit v0.4.1 and PromptKit v0.11.1, with Go 1.25.5 as
|
|
the minimum. Use published modules, never sibling checkouts or local replacements.
|
|
The existing YAML dependency remains while glossary parsing needs it.
|
|
|
|
## Migration Posture
|
|
|
|
This is an intentional breaking transition targeting Audita v2.0.0. No backward
|
|
compatibility is required for existing installations, config files, CLI flags,
|
|
environment overlays, or inspection output. Provide one migration guide, without
|
|
legacy loaders, aliases, dual modes, or automatic conversion tooling.
|
|
|
|
Both libraries belong to one refactor. Intermediate commits need not build or run;
|
|
there is no supported intermediate product state. No temporary adapters, compatibility
|
|
shims, or processing gates are needed. The complete application must work and pass
|
|
its checks before the refactor is finished. Release compliance and publication
|
|
come last, with no intermediate public release.
|
|
|
|
## Configuration and Execution Target
|
|
|
|
ConfigKit selected overlays enable a testing configuration that chooses cheaper or
|
|
faster models. The example selects both proposal and validation profiles. ConfigKit
|
|
application overlays and PromptKit execution profiles remain separate concepts.
|
|
Complete operator-maintained profiles own model/generation controls; embedded
|
|
application prompts and schemas are not operator-selected files. No catalog wiring,
|
|
profile-inheritance feature, ConfigKit imports, or secret-directory bootstrap is
|
|
part of this migration.
|
|
|
|
Capacity controls serve two needs: respecting a local service's hard simultaneous-
|
|
query limit and bounding concurrent paid API activity. One finite limit per backend
|
|
is shared across both roles and every initial, retry, and repair attempt. Limits
|
|
are engine-local, not coordinated across processes. No additional global or
|
|
per-role limiter or fine-grained scheduling policy is required. Profiles must use
|
|
registered backends and ConfigKit-resolved credentials; endpoint-only bypass and
|
|
competing profile credential lookup are excluded.
|
|
|
|
Structural repair is enabled by default for both roles because additional attempts
|
|
to obtain valid output are desired. Transient retries and structural repairs have
|
|
separate budgets; explicit zero disables the respective behavior. Each generation,
|
|
including a repair, receives its own transient-retry budget. The Audita logical-call
|
|
deadline covers preparation, waiting, backoff, all generations, and repairs; profile
|
|
HTTP timeouts apply per attempt. Exact defaults and bounds are in the
|
|
[selected contract](library-boundary-findings.md#configuration-and-profile-contract).
|
|
No Audita outer retry loop restores the old broader retry policy.
|
|
|
|
Audita retains all four correction modules, five LLM-validator prompts,
|
|
deterministic validators, confidence/applicability checks, glossary protection,
|
|
validation batching/cardinality, deterministic proposal and ledger ordering,
|
|
transcript encoders, and artifact retention. PromptKit-valid JSON still passes
|
|
Audita's domain checks. Repair exhaustion, provider failure, and domain rejection
|
|
follow the [selected failure table](library-boundary-findings.md#execution-and-failure-mapping).
|
|
Retention and transcript timestamp changes remain separate work; this migration
|
|
resolves the [retry-count follow-up](runtime-behavior.md).
|
|
|
|
## Reporting and Diagnostic Target
|
|
|
|
Keep stable Audita-owned serialized records; do not expose raw library structs.
|
|
Use process report schema `v2` with the existing domain outcomes and a new
|
|
`llm_summary` separating logical calls, dispatched provider attempts, transient
|
|
retries, structural repairs, and usage. Transcript schema versions remain unchanged.
|
|
The implementation plan specifies the concrete new fields and artifact wiring.
|
|
|
|
Usage distinguishes known zero, unknown, and incomplete totals. Preserve observations
|
|
from failed calls even when no final result exists. Count terminal totals once;
|
|
attempt observations provide diagnosis without being added to those totals again.
|
|
This is operational evidence, not a complete billing ledger.
|
|
|
|
Replace global/role scheduler utilization with per-backend configured capacity and
|
|
observed queue/generation/retry/logical-call timing. Do not fabricate measurements
|
|
that the event API does not provide. Keep domain timing and deterministic correction
|
|
ledger ordering. Record prompt definition/rendered hashes and selected profile/model
|
|
without promising the old hash values. Callbacks are short, concurrency-safe, and
|
|
free of blocking I/O or waits on their own execution.
|
|
|
|
Effective configuration and selected-profile snapshots remain distinct safe views.
|
|
Secret values must not escape through inspection, logs, reports, runtime errors,
|
|
or retained diagnostic files. Observation records exclude payloads, URLs, raw
|
|
provider messages, prompts, and credentials. Existing artifact lifecycle and
|
|
partial-failure handling remain Audita responsibilities.
|
|
|
|
## Completion Criteria and Release Boundary
|
|
|
|
- Both libraries own their selected responsibilities in the production application.
|
|
Superseded loaders, transport, renderers, schedulers, aliases, and the isolated
|
|
prototype are gone once its useful evidence is represented in production tests.
|
|
- Offline tests prove configuration-to-execution routing for both roles, selected
|
|
overlays and explicit overrides, shared backend capacity, retry/repair outcomes,
|
|
cancellation/deadlines, domain decisions, redaction, and truthful partial reports.
|
|
Tests exercise real internal collaborators and controlled HTTP boundaries.
|
|
- Maintained examples and canonical docs describe the implemented final model.
|
|
One migration guide covers files, flags, environment inputs, profiles, credentials,
|
|
overlays, inspection, and changed runtime/report/diagnostic semantics.
|
|
- The assembled application passes its required checks; Linux amd64 and arm64 build.
|
|
macOS arm64 remains best effort. No live-provider or CI-success gate is added.
|
|
|
|
PromptKit v0.11.0's reviewed retry, observation, and capacity APIs remove the earlier
|
|
upstream feature prerequisites; no upstream edit is required for adoption. The
|
|
prototype proved representative behavior, not release compliance or model quality.
|
|
|
|
The selected PromptKit release uses LGPL-3.0-or-later. A later release unit must
|
|
review the actual production dependency graph and distribution requirements,
|
|
including notices and source/rebuild materials, before binary publication. This
|
|
implementation plan does not claim that review is complete. Release publication
|
|
uses the existing [procedure](../release.md) and completes on successful source/tag
|
|
push, without waiting for CI or artifact delivery.
|