Plan coordinated ConfigKit and PromptKit adoption

This commit is contained in:
2026-09-23 15:38:41 +00:00
parent e8bc005040
commit 6b33254a82
2 changed files with 148 additions and 3 deletions

View File

@@ -115,9 +115,11 @@ reporting are likely to overlap this migration. Design their shared target model
first. If another change will substantially reshape those areas, avoid migrating
the same concepts twice. Work independent of configuration can proceed separately.
The migration may provide the configuration foundation for later features, or join
other changes in one major release. Neither ordering is selected yet. Prefer
reviewable implementation units even if the final release combines several changes.
The [coordinated library-adoption roadmap](library-adoption.md) owns the proposed
sequence with PromptKit: shared boundary design and a PromptKit prototype, then
ConfigKit implementation, then PromptKit implementation, followed by a combined
v2.0.0 release. Revisit that sequence alongside other major changes. Keep
implementation units reviewable even when the release combines them.
The separate [runtime behavior decisions](runtime-behavior.md) remain unresolved.
Coordinate the retry-count question with the new configuration semantics where

View File

@@ -0,0 +1,143 @@
# Coordinated ConfigKit and PromptKit Adoption
**Last updated:** 2026-09-23
**Status:** Agreed high-level direction; provisional roadmap. Detailed design,
implementation tasks, dependency versions, and acceptance criteria must be revisited
before migration begins. This document authorizes neither implementation nor release
publication and sets no schedule.
## Goal and Ownership
Adopt ConfigKit and PromptKit to align Audita with the application suite's shared
configuration and LLM execution conventions. Remove overlapping application
infrastructure while keeping transcript-correction responsibilities in Audita.
| Owner | Intended responsibilities |
| --- | --- |
| ConfigKit | Application configuration loading, presence/default semantics, explicit overrides, recorded resolution, secret references, and safe configuration inspection |
| PromptKit | Prompt/profile formats and rendering, provider calls, structural output validation, and shared execution capabilities supplied by the selected release |
| Audita | Configuration field meanings, CLI parsing and file selection, transcript normalization/chunking, module ordering, domain validators, correction application, reports, and diagnostic artifact lifecycle |
This roadmap owns sequencing across the two adoptions. The
[ConfigKit roadmap](configkit.md) owns the configuration migration's proposed scope.
PromptKit consumer requests are recorded upstream in
`../promptkit/docs/roadmap/audita-feature-requests.md`; they remain proposals until
accepted and released there. Do not treat proposed upstream capabilities as present.
## Migration Posture
Treat adoption as an intentional breaking transition. Preserve intended domain
capabilities, not legacy configuration files, flags, environment variables, or
configuration-inspection formats. Supply one migration guide rather than parallel
legacy loaders, compatibility aliases, or permanent translation layers.
Plan to ship both adoptions together as Audita v2.0.0, without an intermediate
public release. Implementation should still consist of coherent, reviewable units.
Revisit the final release scope and sequencing alongside other large planned
changes, especially changes to module or pipeline configuration.
## Proposed Sequence
### 1. Design the Shared Configuration Boundary
Define the target settings model for both libraries before implementing either
migration. Decide how Audita selects proposal and validation execution profiles,
supplies credentials, selects retry behavior, and assigns per-backend concurrency
limits. Give each setting one authoritative home; avoid duplicate model, endpoint,
credential, or timeout definitions with competing precedence.
Keep ConfigKit application configuration distinct from PromptKit execution profiles.
In particular, optional ConfigKit configuration profiles and PromptKit execution
profiles are different concepts. Select an explicit profile-ownership model;
complete Audita-owned execution profiles are the initial recommendation, with
catalog-driven selection a deliberate alternative.
Capacity needs are simple: protect local backends from exceeding their available
simultaneous-query capacity and give paid API backends a reasonable finite cap.
Proposal and validation calls using the same backend share its limit. Separate
proposal/validation budgets and additional fine-grained Audita scheduling controls
are not required. Account for the engine-local scope of PromptKit's existing limits.
### 2. Prove the PromptKit Boundary
Before finalizing the new application configuration schema, exercise one proposal
call and one LLM-validator call through PromptKit using controlled offline responses.
Verify profile selection, credentials, embedded prompts and schemas, shared backend
capacity, failed output validation, cancellation, and diagnostic mapping.
Establish how retries and content repair differ and how their budgets and execution
observations reach Audita. Keep optional content repair disabled initially unless a
specific need justifies it. Audita's domain acceptance checks remain necessary even
when generated output satisfies a JSON schema.
Use the prototype to resolve genuine boundary gaps and inform configuration design.
It is evidence for the migration, not a permanent alternate runtime. Avoid finalizing
ConfigKit-facing settings around the old LLM runtime only to redesign them later.
### 3. Complete ConfigKit Adoption
Implement the agreed application configuration contract following the
[ConfigKit roadmap](configkit.md). Retire legacy environment overlays and obsolete
CLI/configuration conventions, update maintained examples and canonical references,
and begin the combined migration guide.
If the existing LLM runtime remains temporarily during this stage, keep translation
into it small and explicitly transitional. Do not preserve its old settings merely
to make the temporary bridge convenient. No compatibility mode is owed to users.
### 4. Complete PromptKit Adoption
Move prompt rendering, execution profiles, provider calls, and structural output
validation onto the selected PromptKit release. Remove overlapping transport,
prompt-loading, and scheduling machinery once the replacement is verified.
Retain Audita's transcript pipeline, semantic and deterministic validators,
correction application, reports, ledger, and artifact retention. Map PromptKit
results, errors, metadata, and execution observations into those responsibilities.
Remove the temporary bridge from the preceding stage.
### 5. Validate and Release the Combined Migration
Validate the assembled configuration-to-execution workflows, including local and
API backend limits, failure/cancellation behavior, credential handling, diagnostics,
and domain outcomes. Use deterministic tests and controlled endpoints rather than
paid calls as acceptance gates. Update documentation and examples together with
implemented behavior and remove obsolete contracts and code.
Complete one migration guide covering files, CLI, environment settings, credentials,
execution profiles, inspection, and changed diagnostics. Release the combined
breaking transition under the established [release procedure](../release.md), with
no CI waiting added to release completion.
## Parallel Upstream Preparation
PromptKit work can proceed alongside Audita's design and ConfigKit implementation.
Prioritize evaluation of transient-failure retries, actionable failure classification,
and execution observations, supported by a representative consumer example. Existing
per-backend capacity controls are the intended foundation; additional scheduling
features are not assumed necessary.
Resolve the proposed LGPL 3.0 licensing change and applicable distribution terms
before selecting PromptKit for adoption. Recheck released APIs, toolchain requirements,
and dependency versions at implementation time. Pin published library releases;
production builds must not depend on sibling checkouts or local module replacements.
If a required upstream capability is not ready, complete independent work and
reassess sequencing explicitly rather than quietly recreate that capability as
permanent Audita infrastructure. The prototype should identify what is genuinely
required for adoption versus useful later enhancement.
## Coordination and Open Decisions
- Finalize the configuration/profile boundary, defaults, credential ownership, and
retry semantics before committing to the new file schema and CLI design.
- Define how failed validation, exhausted retries, and partial execution observations
map to Audita warnings, rejected work, and failed runs.
- Coordinate the [runtime retry-count question](runtime-behavior.md) with this design.
Retention and transcript timestamp questions are not automatically included.
- Revisit ordering when other major changes are defined. Prefer designing shared
module/pipeline settings once, while keeping unrelated work independent.
- Flesh out task-level scope, tests, and acceptance criteria before implementation.
Completion requires the intended ownership split, removed transitional machinery,
verified assembled behavior, and one coherent migration path for users.