Plan coordinated ConfigKit and PromptKit adoption
This commit is contained in:
@@ -115,9 +115,11 @@ reporting are likely to overlap this migration. Design their shared target model
|
||||
first. If another change will substantially reshape those areas, avoid migrating
|
||||
the same concepts twice. Work independent of configuration can proceed separately.
|
||||
|
||||
The migration may provide the configuration foundation for later features, or join
|
||||
other changes in one major release. Neither ordering is selected yet. Prefer
|
||||
reviewable implementation units even if the final release combines several changes.
|
||||
The [coordinated library-adoption roadmap](library-adoption.md) owns the proposed
|
||||
sequence with PromptKit: shared boundary design and a PromptKit prototype, then
|
||||
ConfigKit implementation, then PromptKit implementation, followed by a combined
|
||||
v2.0.0 release. Revisit that sequence alongside other major changes. Keep
|
||||
implementation units reviewable even when the release combines them.
|
||||
|
||||
The separate [runtime behavior decisions](runtime-behavior.md) remain unresolved.
|
||||
Coordinate the retry-count question with the new configuration semantics where
|
||||
|
||||
143
docs/roadmap/library-adoption.md
Normal file
143
docs/roadmap/library-adoption.md
Normal file
@@ -0,0 +1,143 @@
|
||||
# Coordinated ConfigKit and PromptKit Adoption
|
||||
|
||||
**Last updated:** 2026-09-23
|
||||
|
||||
**Status:** Agreed high-level direction; provisional roadmap. Detailed design,
|
||||
implementation tasks, dependency versions, and acceptance criteria must be revisited
|
||||
before migration begins. This document authorizes neither implementation nor release
|
||||
publication and sets no schedule.
|
||||
|
||||
## Goal and Ownership
|
||||
|
||||
Adopt ConfigKit and PromptKit to align Audita with the application suite's shared
|
||||
configuration and LLM execution conventions. Remove overlapping application
|
||||
infrastructure while keeping transcript-correction responsibilities in Audita.
|
||||
|
||||
| Owner | Intended responsibilities |
|
||||
| --- | --- |
|
||||
| ConfigKit | Application configuration loading, presence/default semantics, explicit overrides, recorded resolution, secret references, and safe configuration inspection |
|
||||
| PromptKit | Prompt/profile formats and rendering, provider calls, structural output validation, and shared execution capabilities supplied by the selected release |
|
||||
| Audita | Configuration field meanings, CLI parsing and file selection, transcript normalization/chunking, module ordering, domain validators, correction application, reports, and diagnostic artifact lifecycle |
|
||||
|
||||
This roadmap owns sequencing across the two adoptions. The
|
||||
[ConfigKit roadmap](configkit.md) owns the configuration migration's proposed scope.
|
||||
PromptKit consumer requests are recorded upstream in
|
||||
`../promptkit/docs/roadmap/audita-feature-requests.md`; they remain proposals until
|
||||
accepted and released there. Do not treat proposed upstream capabilities as present.
|
||||
|
||||
## Migration Posture
|
||||
|
||||
Treat adoption as an intentional breaking transition. Preserve intended domain
|
||||
capabilities, not legacy configuration files, flags, environment variables, or
|
||||
configuration-inspection formats. Supply one migration guide rather than parallel
|
||||
legacy loaders, compatibility aliases, or permanent translation layers.
|
||||
|
||||
Plan to ship both adoptions together as Audita v2.0.0, without an intermediate
|
||||
public release. Implementation should still consist of coherent, reviewable units.
|
||||
Revisit the final release scope and sequencing alongside other large planned
|
||||
changes, especially changes to module or pipeline configuration.
|
||||
|
||||
## Proposed Sequence
|
||||
|
||||
### 1. Design the Shared Configuration Boundary
|
||||
|
||||
Define the target settings model for both libraries before implementing either
|
||||
migration. Decide how Audita selects proposal and validation execution profiles,
|
||||
supplies credentials, selects retry behavior, and assigns per-backend concurrency
|
||||
limits. Give each setting one authoritative home; avoid duplicate model, endpoint,
|
||||
credential, or timeout definitions with competing precedence.
|
||||
|
||||
Keep ConfigKit application configuration distinct from PromptKit execution profiles.
|
||||
In particular, optional ConfigKit configuration profiles and PromptKit execution
|
||||
profiles are different concepts. Select an explicit profile-ownership model;
|
||||
complete Audita-owned execution profiles are the initial recommendation, with
|
||||
catalog-driven selection a deliberate alternative.
|
||||
|
||||
Capacity needs are simple: protect local backends from exceeding their available
|
||||
simultaneous-query capacity and give paid API backends a reasonable finite cap.
|
||||
Proposal and validation calls using the same backend share its limit. Separate
|
||||
proposal/validation budgets and additional fine-grained Audita scheduling controls
|
||||
are not required. Account for the engine-local scope of PromptKit's existing limits.
|
||||
|
||||
### 2. Prove the PromptKit Boundary
|
||||
|
||||
Before finalizing the new application configuration schema, exercise one proposal
|
||||
call and one LLM-validator call through PromptKit using controlled offline responses.
|
||||
Verify profile selection, credentials, embedded prompts and schemas, shared backend
|
||||
capacity, failed output validation, cancellation, and diagnostic mapping.
|
||||
|
||||
Establish how retries and content repair differ and how their budgets and execution
|
||||
observations reach Audita. Keep optional content repair disabled initially unless a
|
||||
specific need justifies it. Audita's domain acceptance checks remain necessary even
|
||||
when generated output satisfies a JSON schema.
|
||||
|
||||
Use the prototype to resolve genuine boundary gaps and inform configuration design.
|
||||
It is evidence for the migration, not a permanent alternate runtime. Avoid finalizing
|
||||
ConfigKit-facing settings around the old LLM runtime only to redesign them later.
|
||||
|
||||
### 3. Complete ConfigKit Adoption
|
||||
|
||||
Implement the agreed application configuration contract following the
|
||||
[ConfigKit roadmap](configkit.md). Retire legacy environment overlays and obsolete
|
||||
CLI/configuration conventions, update maintained examples and canonical references,
|
||||
and begin the combined migration guide.
|
||||
|
||||
If the existing LLM runtime remains temporarily during this stage, keep translation
|
||||
into it small and explicitly transitional. Do not preserve its old settings merely
|
||||
to make the temporary bridge convenient. No compatibility mode is owed to users.
|
||||
|
||||
### 4. Complete PromptKit Adoption
|
||||
|
||||
Move prompt rendering, execution profiles, provider calls, and structural output
|
||||
validation onto the selected PromptKit release. Remove overlapping transport,
|
||||
prompt-loading, and scheduling machinery once the replacement is verified.
|
||||
|
||||
Retain Audita's transcript pipeline, semantic and deterministic validators,
|
||||
correction application, reports, ledger, and artifact retention. Map PromptKit
|
||||
results, errors, metadata, and execution observations into those responsibilities.
|
||||
Remove the temporary bridge from the preceding stage.
|
||||
|
||||
### 5. Validate and Release the Combined Migration
|
||||
|
||||
Validate the assembled configuration-to-execution workflows, including local and
|
||||
API backend limits, failure/cancellation behavior, credential handling, diagnostics,
|
||||
and domain outcomes. Use deterministic tests and controlled endpoints rather than
|
||||
paid calls as acceptance gates. Update documentation and examples together with
|
||||
implemented behavior and remove obsolete contracts and code.
|
||||
|
||||
Complete one migration guide covering files, CLI, environment settings, credentials,
|
||||
execution profiles, inspection, and changed diagnostics. Release the combined
|
||||
breaking transition under the established [release procedure](../release.md), with
|
||||
no CI waiting added to release completion.
|
||||
|
||||
## Parallel Upstream Preparation
|
||||
|
||||
PromptKit work can proceed alongside Audita's design and ConfigKit implementation.
|
||||
Prioritize evaluation of transient-failure retries, actionable failure classification,
|
||||
and execution observations, supported by a representative consumer example. Existing
|
||||
per-backend capacity controls are the intended foundation; additional scheduling
|
||||
features are not assumed necessary.
|
||||
|
||||
Resolve the proposed LGPL 3.0 licensing change and applicable distribution terms
|
||||
before selecting PromptKit for adoption. Recheck released APIs, toolchain requirements,
|
||||
and dependency versions at implementation time. Pin published library releases;
|
||||
production builds must not depend on sibling checkouts or local module replacements.
|
||||
|
||||
If a required upstream capability is not ready, complete independent work and
|
||||
reassess sequencing explicitly rather than quietly recreate that capability as
|
||||
permanent Audita infrastructure. The prototype should identify what is genuinely
|
||||
required for adoption versus useful later enhancement.
|
||||
|
||||
## Coordination and Open Decisions
|
||||
|
||||
- Finalize the configuration/profile boundary, defaults, credential ownership, and
|
||||
retry semantics before committing to the new file schema and CLI design.
|
||||
- Define how failed validation, exhausted retries, and partial execution observations
|
||||
map to Audita warnings, rejected work, and failed runs.
|
||||
- Coordinate the [runtime retry-count question](runtime-behavior.md) with this design.
|
||||
Retention and transcript timestamp questions are not automatically included.
|
||||
- Revisit ordering when other major changes are defined. Prefer designing shared
|
||||
module/pipeline settings once, while keeping unrelated work independent.
|
||||
- Flesh out task-level scope, tests, and acceptance criteria before implementation.
|
||||
Completion requires the intended ownership split, removed transitional machinery,
|
||||
verified assembled behavior, and one coherent migration path for users.
|
||||
Reference in New Issue
Block a user