Plan coordinated ConfigKit and PromptKit adoption
This commit is contained in:
@@ -115,9 +115,11 @@ reporting are likely to overlap this migration. Design their shared target model
|
|||||||
first. If another change will substantially reshape those areas, avoid migrating
|
first. If another change will substantially reshape those areas, avoid migrating
|
||||||
the same concepts twice. Work independent of configuration can proceed separately.
|
the same concepts twice. Work independent of configuration can proceed separately.
|
||||||
|
|
||||||
The migration may provide the configuration foundation for later features, or join
|
The [coordinated library-adoption roadmap](library-adoption.md) owns the proposed
|
||||||
other changes in one major release. Neither ordering is selected yet. Prefer
|
sequence with PromptKit: shared boundary design and a PromptKit prototype, then
|
||||||
reviewable implementation units even if the final release combines several changes.
|
ConfigKit implementation, then PromptKit implementation, followed by a combined
|
||||||
|
v2.0.0 release. Revisit that sequence alongside other major changes. Keep
|
||||||
|
implementation units reviewable even when the release combines them.
|
||||||
|
|
||||||
The separate [runtime behavior decisions](runtime-behavior.md) remain unresolved.
|
The separate [runtime behavior decisions](runtime-behavior.md) remain unresolved.
|
||||||
Coordinate the retry-count question with the new configuration semantics where
|
Coordinate the retry-count question with the new configuration semantics where
|
||||||
|
|||||||
143
docs/roadmap/library-adoption.md
Normal file
143
docs/roadmap/library-adoption.md
Normal file
@@ -0,0 +1,143 @@
|
|||||||
|
# Coordinated ConfigKit and PromptKit Adoption
|
||||||
|
|
||||||
|
**Last updated:** 2026-09-23
|
||||||
|
|
||||||
|
**Status:** Agreed high-level direction; provisional roadmap. Detailed design,
|
||||||
|
implementation tasks, dependency versions, and acceptance criteria must be revisited
|
||||||
|
before migration begins. This document authorizes neither implementation nor release
|
||||||
|
publication and sets no schedule.
|
||||||
|
|
||||||
|
## Goal and Ownership
|
||||||
|
|
||||||
|
Adopt ConfigKit and PromptKit to align Audita with the application suite's shared
|
||||||
|
configuration and LLM execution conventions. Remove overlapping application
|
||||||
|
infrastructure while keeping transcript-correction responsibilities in Audita.
|
||||||
|
|
||||||
|
| Owner | Intended responsibilities |
|
||||||
|
| --- | --- |
|
||||||
|
| ConfigKit | Application configuration loading, presence/default semantics, explicit overrides, recorded resolution, secret references, and safe configuration inspection |
|
||||||
|
| PromptKit | Prompt/profile formats and rendering, provider calls, structural output validation, and shared execution capabilities supplied by the selected release |
|
||||||
|
| Audita | Configuration field meanings, CLI parsing and file selection, transcript normalization/chunking, module ordering, domain validators, correction application, reports, and diagnostic artifact lifecycle |
|
||||||
|
|
||||||
|
This roadmap owns sequencing across the two adoptions. The
|
||||||
|
[ConfigKit roadmap](configkit.md) owns the configuration migration's proposed scope.
|
||||||
|
PromptKit consumer requests are recorded upstream in
|
||||||
|
`../promptkit/docs/roadmap/audita-feature-requests.md`; they remain proposals until
|
||||||
|
accepted and released there. Do not treat proposed upstream capabilities as present.
|
||||||
|
|
||||||
|
## Migration Posture
|
||||||
|
|
||||||
|
Treat adoption as an intentional breaking transition. Preserve intended domain
|
||||||
|
capabilities, not legacy configuration files, flags, environment variables, or
|
||||||
|
configuration-inspection formats. Supply one migration guide rather than parallel
|
||||||
|
legacy loaders, compatibility aliases, or permanent translation layers.
|
||||||
|
|
||||||
|
Plan to ship both adoptions together as Audita v2.0.0, without an intermediate
|
||||||
|
public release. Implementation should still consist of coherent, reviewable units.
|
||||||
|
Revisit the final release scope and sequencing alongside other large planned
|
||||||
|
changes, especially changes to module or pipeline configuration.
|
||||||
|
|
||||||
|
## Proposed Sequence
|
||||||
|
|
||||||
|
### 1. Design the Shared Configuration Boundary
|
||||||
|
|
||||||
|
Define the target settings model for both libraries before implementing either
|
||||||
|
migration. Decide how Audita selects proposal and validation execution profiles,
|
||||||
|
supplies credentials, selects retry behavior, and assigns per-backend concurrency
|
||||||
|
limits. Give each setting one authoritative home; avoid duplicate model, endpoint,
|
||||||
|
credential, or timeout definitions with competing precedence.
|
||||||
|
|
||||||
|
Keep ConfigKit application configuration distinct from PromptKit execution profiles.
|
||||||
|
In particular, optional ConfigKit configuration profiles and PromptKit execution
|
||||||
|
profiles are different concepts. Select an explicit profile-ownership model;
|
||||||
|
complete Audita-owned execution profiles are the initial recommendation, with
|
||||||
|
catalog-driven selection a deliberate alternative.
|
||||||
|
|
||||||
|
Capacity needs are simple: protect local backends from exceeding their available
|
||||||
|
simultaneous-query capacity and give paid API backends a reasonable finite cap.
|
||||||
|
Proposal and validation calls using the same backend share its limit. Separate
|
||||||
|
proposal/validation budgets and additional fine-grained Audita scheduling controls
|
||||||
|
are not required. Account for the engine-local scope of PromptKit's existing limits.
|
||||||
|
|
||||||
|
### 2. Prove the PromptKit Boundary
|
||||||
|
|
||||||
|
Before finalizing the new application configuration schema, exercise one proposal
|
||||||
|
call and one LLM-validator call through PromptKit using controlled offline responses.
|
||||||
|
Verify profile selection, credentials, embedded prompts and schemas, shared backend
|
||||||
|
capacity, failed output validation, cancellation, and diagnostic mapping.
|
||||||
|
|
||||||
|
Establish how retries and content repair differ and how their budgets and execution
|
||||||
|
observations reach Audita. Keep optional content repair disabled initially unless a
|
||||||
|
specific need justifies it. Audita's domain acceptance checks remain necessary even
|
||||||
|
when generated output satisfies a JSON schema.
|
||||||
|
|
||||||
|
Use the prototype to resolve genuine boundary gaps and inform configuration design.
|
||||||
|
It is evidence for the migration, not a permanent alternate runtime. Avoid finalizing
|
||||||
|
ConfigKit-facing settings around the old LLM runtime only to redesign them later.
|
||||||
|
|
||||||
|
### 3. Complete ConfigKit Adoption
|
||||||
|
|
||||||
|
Implement the agreed application configuration contract following the
|
||||||
|
[ConfigKit roadmap](configkit.md). Retire legacy environment overlays and obsolete
|
||||||
|
CLI/configuration conventions, update maintained examples and canonical references,
|
||||||
|
and begin the combined migration guide.
|
||||||
|
|
||||||
|
If the existing LLM runtime remains temporarily during this stage, keep translation
|
||||||
|
into it small and explicitly transitional. Do not preserve its old settings merely
|
||||||
|
to make the temporary bridge convenient. No compatibility mode is owed to users.
|
||||||
|
|
||||||
|
### 4. Complete PromptKit Adoption
|
||||||
|
|
||||||
|
Move prompt rendering, execution profiles, provider calls, and structural output
|
||||||
|
validation onto the selected PromptKit release. Remove overlapping transport,
|
||||||
|
prompt-loading, and scheduling machinery once the replacement is verified.
|
||||||
|
|
||||||
|
Retain Audita's transcript pipeline, semantic and deterministic validators,
|
||||||
|
correction application, reports, ledger, and artifact retention. Map PromptKit
|
||||||
|
results, errors, metadata, and execution observations into those responsibilities.
|
||||||
|
Remove the temporary bridge from the preceding stage.
|
||||||
|
|
||||||
|
### 5. Validate and Release the Combined Migration
|
||||||
|
|
||||||
|
Validate the assembled configuration-to-execution workflows, including local and
|
||||||
|
API backend limits, failure/cancellation behavior, credential handling, diagnostics,
|
||||||
|
and domain outcomes. Use deterministic tests and controlled endpoints rather than
|
||||||
|
paid calls as acceptance gates. Update documentation and examples together with
|
||||||
|
implemented behavior and remove obsolete contracts and code.
|
||||||
|
|
||||||
|
Complete one migration guide covering files, CLI, environment settings, credentials,
|
||||||
|
execution profiles, inspection, and changed diagnostics. Release the combined
|
||||||
|
breaking transition under the established [release procedure](../release.md), with
|
||||||
|
no CI waiting added to release completion.
|
||||||
|
|
||||||
|
## Parallel Upstream Preparation
|
||||||
|
|
||||||
|
PromptKit work can proceed alongside Audita's design and ConfigKit implementation.
|
||||||
|
Prioritize evaluation of transient-failure retries, actionable failure classification,
|
||||||
|
and execution observations, supported by a representative consumer example. Existing
|
||||||
|
per-backend capacity controls are the intended foundation; additional scheduling
|
||||||
|
features are not assumed necessary.
|
||||||
|
|
||||||
|
Resolve the proposed LGPL 3.0 licensing change and applicable distribution terms
|
||||||
|
before selecting PromptKit for adoption. Recheck released APIs, toolchain requirements,
|
||||||
|
and dependency versions at implementation time. Pin published library releases;
|
||||||
|
production builds must not depend on sibling checkouts or local module replacements.
|
||||||
|
|
||||||
|
If a required upstream capability is not ready, complete independent work and
|
||||||
|
reassess sequencing explicitly rather than quietly recreate that capability as
|
||||||
|
permanent Audita infrastructure. The prototype should identify what is genuinely
|
||||||
|
required for adoption versus useful later enhancement.
|
||||||
|
|
||||||
|
## Coordination and Open Decisions
|
||||||
|
|
||||||
|
- Finalize the configuration/profile boundary, defaults, credential ownership, and
|
||||||
|
retry semantics before committing to the new file schema and CLI design.
|
||||||
|
- Define how failed validation, exhausted retries, and partial execution observations
|
||||||
|
map to Audita warnings, rejected work, and failed runs.
|
||||||
|
- Coordinate the [runtime retry-count question](runtime-behavior.md) with this design.
|
||||||
|
Retention and transcript timestamp questions are not automatically included.
|
||||||
|
- Revisit ordering when other major changes are defined. Prefer designing shared
|
||||||
|
module/pipeline settings once, while keeping unrelated work independent.
|
||||||
|
- Flesh out task-level scope, tests, and acceptance criteria before implementation.
|
||||||
|
Completion requires the intended ownership split, removed transitional machinery,
|
||||||
|
verified assembled behavior, and one coherent migration path for users.
|
||||||
Reference in New Issue
Block a user