From 6b33254a82ac2691191e0c0ac98540f98a978f6e Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Wed, 23 Sep 2026 15:38:41 +0000 Subject: [PATCH] Plan coordinated ConfigKit and PromptKit adoption --- docs/roadmap/configkit.md | 8 +- docs/roadmap/library-adoption.md | 143 +++++++++++++++++++++++++++++++ 2 files changed, 148 insertions(+), 3 deletions(-) create mode 100644 docs/roadmap/library-adoption.md diff --git a/docs/roadmap/configkit.md b/docs/roadmap/configkit.md index 9bbd05e..b8da66c 100644 --- a/docs/roadmap/configkit.md +++ b/docs/roadmap/configkit.md @@ -115,9 +115,11 @@ reporting are likely to overlap this migration. Design their shared target model first. If another change will substantially reshape those areas, avoid migrating the same concepts twice. Work independent of configuration can proceed separately. -The migration may provide the configuration foundation for later features, or join -other changes in one major release. Neither ordering is selected yet. Prefer -reviewable implementation units even if the final release combines several changes. +The [coordinated library-adoption roadmap](library-adoption.md) owns the proposed +sequence with PromptKit: shared boundary design and a PromptKit prototype, then +ConfigKit implementation, then PromptKit implementation, followed by a combined +v2.0.0 release. Revisit that sequence alongside other major changes. Keep +implementation units reviewable even when the release combines them. The separate [runtime behavior decisions](runtime-behavior.md) remain unresolved. Coordinate the retry-count question with the new configuration semantics where diff --git a/docs/roadmap/library-adoption.md b/docs/roadmap/library-adoption.md new file mode 100644 index 0000000..b69db9d --- /dev/null +++ b/docs/roadmap/library-adoption.md @@ -0,0 +1,143 @@ +# Coordinated ConfigKit and PromptKit Adoption + +**Last updated:** 2026-09-23 + +**Status:** Agreed high-level direction; provisional roadmap. Detailed design, +implementation tasks, dependency versions, and acceptance criteria must be revisited +before migration begins. This document authorizes neither implementation nor release +publication and sets no schedule. + +## Goal and Ownership + +Adopt ConfigKit and PromptKit to align Audita with the application suite's shared +configuration and LLM execution conventions. Remove overlapping application +infrastructure while keeping transcript-correction responsibilities in Audita. + +| Owner | Intended responsibilities | +| --- | --- | +| ConfigKit | Application configuration loading, presence/default semantics, explicit overrides, recorded resolution, secret references, and safe configuration inspection | +| PromptKit | Prompt/profile formats and rendering, provider calls, structural output validation, and shared execution capabilities supplied by the selected release | +| Audita | Configuration field meanings, CLI parsing and file selection, transcript normalization/chunking, module ordering, domain validators, correction application, reports, and diagnostic artifact lifecycle | + +This roadmap owns sequencing across the two adoptions. The +[ConfigKit roadmap](configkit.md) owns the configuration migration's proposed scope. +PromptKit consumer requests are recorded upstream in +`../promptkit/docs/roadmap/audita-feature-requests.md`; they remain proposals until +accepted and released there. Do not treat proposed upstream capabilities as present. + +## Migration Posture + +Treat adoption as an intentional breaking transition. Preserve intended domain +capabilities, not legacy configuration files, flags, environment variables, or +configuration-inspection formats. Supply one migration guide rather than parallel +legacy loaders, compatibility aliases, or permanent translation layers. + +Plan to ship both adoptions together as Audita v2.0.0, without an intermediate +public release. Implementation should still consist of coherent, reviewable units. +Revisit the final release scope and sequencing alongside other large planned +changes, especially changes to module or pipeline configuration. + +## Proposed Sequence + +### 1. Design the Shared Configuration Boundary + +Define the target settings model for both libraries before implementing either +migration. Decide how Audita selects proposal and validation execution profiles, +supplies credentials, selects retry behavior, and assigns per-backend concurrency +limits. Give each setting one authoritative home; avoid duplicate model, endpoint, +credential, or timeout definitions with competing precedence. + +Keep ConfigKit application configuration distinct from PromptKit execution profiles. +In particular, optional ConfigKit configuration profiles and PromptKit execution +profiles are different concepts. Select an explicit profile-ownership model; +complete Audita-owned execution profiles are the initial recommendation, with +catalog-driven selection a deliberate alternative. + +Capacity needs are simple: protect local backends from exceeding their available +simultaneous-query capacity and give paid API backends a reasonable finite cap. +Proposal and validation calls using the same backend share its limit. Separate +proposal/validation budgets and additional fine-grained Audita scheduling controls +are not required. Account for the engine-local scope of PromptKit's existing limits. + +### 2. Prove the PromptKit Boundary + +Before finalizing the new application configuration schema, exercise one proposal +call and one LLM-validator call through PromptKit using controlled offline responses. +Verify profile selection, credentials, embedded prompts and schemas, shared backend +capacity, failed output validation, cancellation, and diagnostic mapping. + +Establish how retries and content repair differ and how their budgets and execution +observations reach Audita. Keep optional content repair disabled initially unless a +specific need justifies it. Audita's domain acceptance checks remain necessary even +when generated output satisfies a JSON schema. + +Use the prototype to resolve genuine boundary gaps and inform configuration design. +It is evidence for the migration, not a permanent alternate runtime. Avoid finalizing +ConfigKit-facing settings around the old LLM runtime only to redesign them later. + +### 3. Complete ConfigKit Adoption + +Implement the agreed application configuration contract following the +[ConfigKit roadmap](configkit.md). Retire legacy environment overlays and obsolete +CLI/configuration conventions, update maintained examples and canonical references, +and begin the combined migration guide. + +If the existing LLM runtime remains temporarily during this stage, keep translation +into it small and explicitly transitional. Do not preserve its old settings merely +to make the temporary bridge convenient. No compatibility mode is owed to users. + +### 4. Complete PromptKit Adoption + +Move prompt rendering, execution profiles, provider calls, and structural output +validation onto the selected PromptKit release. Remove overlapping transport, +prompt-loading, and scheduling machinery once the replacement is verified. + +Retain Audita's transcript pipeline, semantic and deterministic validators, +correction application, reports, ledger, and artifact retention. Map PromptKit +results, errors, metadata, and execution observations into those responsibilities. +Remove the temporary bridge from the preceding stage. + +### 5. Validate and Release the Combined Migration + +Validate the assembled configuration-to-execution workflows, including local and +API backend limits, failure/cancellation behavior, credential handling, diagnostics, +and domain outcomes. Use deterministic tests and controlled endpoints rather than +paid calls as acceptance gates. Update documentation and examples together with +implemented behavior and remove obsolete contracts and code. + +Complete one migration guide covering files, CLI, environment settings, credentials, +execution profiles, inspection, and changed diagnostics. Release the combined +breaking transition under the established [release procedure](../release.md), with +no CI waiting added to release completion. + +## Parallel Upstream Preparation + +PromptKit work can proceed alongside Audita's design and ConfigKit implementation. +Prioritize evaluation of transient-failure retries, actionable failure classification, +and execution observations, supported by a representative consumer example. Existing +per-backend capacity controls are the intended foundation; additional scheduling +features are not assumed necessary. + +Resolve the proposed LGPL 3.0 licensing change and applicable distribution terms +before selecting PromptKit for adoption. Recheck released APIs, toolchain requirements, +and dependency versions at implementation time. Pin published library releases; +production builds must not depend on sibling checkouts or local module replacements. + +If a required upstream capability is not ready, complete independent work and +reassess sequencing explicitly rather than quietly recreate that capability as +permanent Audita infrastructure. The prototype should identify what is genuinely +required for adoption versus useful later enhancement. + +## Coordination and Open Decisions + +- Finalize the configuration/profile boundary, defaults, credential ownership, and + retry semantics before committing to the new file schema and CLI design. +- Define how failed validation, exhausted retries, and partial execution observations + map to Audita warnings, rejected work, and failed runs. +- Coordinate the [runtime retry-count question](runtime-behavior.md) with this design. + Retention and transcript timestamp questions are not automatically included. +- Revisit ordering when other major changes are defined. Prefer designing shared + module/pipeline settings once, while keeping unrelated work independent. +- Flesh out task-level scope, tests, and acceptance criteria before implementation. + Completion requires the intended ownership split, removed transitional machinery, + verified assembled behavior, and one coherent migration path for users.