eric/audita
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Simplify the library migration to target the final integration

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-24 01:09:43 +00:00
parent 5b7d2e6e81
commit 116648fa78
5 changed files with 75 additions and 137 deletions
Download Patch File Download Diff File Expand all files Collapse all files

161
docs/roadmap/configkit.md
View File

@@ -136,8 +136,13 @@ The [migration inventory](library-boundary-findings.md#migration-inventory) is t
field-by-field old-to-new map. The [selected boundary](library-boundary-findings.md#configuration-and-profile-contract)
owns defaults and semantic choices; the [v0.4.0 overlay results](library-boundary-findings.md#selected-overlay-extension-2026-09-24)
prove the representative load, inspection, and HTTP path. These packages are
future work. Complete each dependency and its tests before removing the code it
replaces. Do not treat the isolated prototype as a second production runtime.
future work within one coordinated refactor to adopt both libraries. The order
organizes implementation dependencies, not independently runnable product states.
Intermediate commits need not build or run; do not add temporary scaffolding,
compatibility shims, or a processing gate to support them. Remove superseded code
as its replacement lands. Run focused checks as components become testable, and
validate the complete application before declaring the refactor finished. The
isolated prototype is evidence, not a second production runtime.
### 1. Define and load the complete version-2 configuration
@@ -224,43 +229,31 @@ inspection, diagnostics, or errors. Update `internal/cli/run_test.go`,
`internal/core/config/*_test.go`, and report/diagnostics tests that own these
outputs.
### 4. Convert runtime inputs without claiming unavailable execution
### 4. Connect the final configuration directly to PromptKit execution
**Depends on:** packages 1–3. **Owns:** a narrow conversion at the CLI to
`internal/framework/llm`/runner boundary, with focused `internal/cli` and
`internal/framework/llm` tests. Convert retained domain settings to their current
domain structs. Convert selected profile/backend endpoint, model, resolved secret,
and per-attempt timeout to existing client setup values for assembly checks; keep
library types out of modules, validators, transcript schemas, and ledgers.
**Depends on:** packages 1–3. **Owns:** the final CLI/runtime assembly boundary,
`internal/framework/llm`, prompt/schema loading, runner wiring, and relevant tests.
Implement PromptKit adoption as part of the same refactor. Convert domain settings
into Audita's existing domain structs and supply the selected profiles, registered
backends, resolved secrets, retry/repair budgets, and logical-operation deadlines
directly to PromptKit. Keep library types out of correction algorithms and ledgers.
The interim conversion has this explicit scope; "assemble" means values can be
validated and converted, not that processing is enabled:
Migrate all module and validator prompts, preserve their domain checks, and map
PromptKit results, classified failures, and observations into Audita reporting and
redacted diagnostics. Replace the old provider transport, prompt loader,
structural-output handling, and global/role scheduler composition. Follow the
[coordinated execution scope](library-adoption.md#4-complete-promptkit-adoption)
and [execution/observation contract](library-boundary-findings.md#execution-and-failure-mapping).
Do not build an adapter to feed the old LLM runtime or introduce an interim
`process` rejection mode.
| New setting or control | Interim disposition |
| --- | --- |
| `profiles_dir`, proposal/validation profile IDs | Honor for relative-path profile loading, final inheritance, inspection, and target assembly. |
| `backends.<id>.endpoint` and `credential` | Honor for selected target validation and client setup; keep ConfigKit secret resolution and redaction. |
| Profile model and `timeout_seconds` | Convert for old client setup checks; old transport can consume these per-attempt values. |
| `validation.max_prompt_tokens`; retained domain, output, and diagnostics fields | Honor through existing domain/batching/report inputs after validation. |
| `backends.<id>.concurrency` and `queue_capacity` | Validate and inspect; old global/role pools cannot enforce independent per-backend admission or queue semantics. |
| `llm.transient_retries` | Validate and inspect; old retry eligibility and explicit-zero behavior differ. |
| `llm.repair_attempts` | Validate and inspect; old runtime has no PromptKit structural-repair contract. |
| `llm.operation_timeout` | Validate and inspect; old per-call timeout is not the full logical-operation deadline. |
| Other PromptKit profile generation controls and execution observations | Inspect only; old transport/reporting cannot claim equivalent behavior. |
The old global/role pools and retry loop must not be presented as equivalents. The
selected interim behavior is to reject `process` before any LLM request with a
clear unsupported-runtime classification while ConfigKit has landed but PromptKit
has not. `config validate` and effective/profile inspection remain usable. This
is an unreleased development interval, not a user-visible mode or compatibility
switch. The process gate is removed only when the PromptKit package below proves
the complete settings through controlled calls.
**Acceptance/tests:** conversion tests verify the representable values and
redaction; a process subprocess test proves the interim rejection occurs before
provider transport or output mutation. No test should assert that old retries or
schedulers implement new settings. The combined migration cannot release with
this gate in place.
**Acceptance/tests:** controlled HTTP tests prove both roles' final model/backend
selection, overlays and explicit overrides, finite per-backend capacity, explicit
zero retries, positive structural repair, total-operation deadlines, cancellation,
domain acceptance/rejection, and safe observations including partial failures.
Processing uses the new configuration and execution path together. Reuse existing
domain coverage and the prototype's integration evidence; update tests tied to the
retired transport and scheduler contracts.
### 5. Remove superseded configuration paths and update maintained contracts
@@ -278,80 +271,21 @@ semantics. Do not leave dual loaders or a permanent translation mode.
**Acceptance/tests:** maintained examples validate offline; removed flags and old
schema are rejected; relevant config/CLI/subprocess/report tests pass; documentation
links and `git diff --check` pass. The interim process restriction remains explicit
until package 6 and is not documented as a released product feature.
links and `git diff --check` pass. Remove the isolated prototype when its useful
coverage and examples have moved into the production suite.
### 6. Replace execution with PromptKit, then prepare the combined release
### 6. Validate the combined end state and prepare release
**Depends on:** packages 1–5. This is the later PromptKit work unit, not part of
this ConfigKit implementation. Replace the old provider transport, prompt loader,
structural-output handling, and scheduler composition as described by the
[coordinated adoption roadmap](library-adoption.md#4-complete-promptkit-adoption)
and [execution/observation handoff](library-boundary-findings.md#execution-and-failure-mapping).
Prove both roles' profile/backend settings, per-backend capacity, explicit-zero
retry, positive repair, total-operation deadline, domain decisions, and safe
observations with controlled transport. Remove the interim converter/process gate
and superseded LLM machinery only after those tests pass. Update report and
diagnostic contracts and the migration guide. The combined release and license/
distribution review follow last under the release policy; no publication is
authorized by this plan.
**Depends on:** packages 1–5. Verify the assembled version-2 configuration,
overlay, profile, execution, reporting, and artifact workflows. Run the relevant
full build/test checks, confirm obsolete configuration and LLM machinery is gone,
and verify maintained examples and the migration guide against the final behavior.
There is no requirement to validate or preserve intermediate product states.
## Historical Provisional Outline
The completed [shared-boundary and prototype unit](library-boundary.md) supplies
the initial target settings model and integration evidence. Extend that evidence
with the selected-overlay work above during the next configuration unit.
### 1. Define the Contract and Coordinate Sequencing
Inventory existing configuration fields, invocation options, environment settings,
secret handling, inheritance rules, and inspection consumers. Produce an old-to-new
mapping for migration guidance and identify which concepts should be retained,
redesigned, or removed. This is an inventory of needs, not a compatibility checklist.
Specify the target ConfigKit conventions and the boundary between configuration
and invocation-only state. Decide file discovery, the new file schema version,
CLI shape, validation/inspection behavior, and optional feature scope. Identify
application requirements that the library cannot express cleanly before building
workarounds.
Coordinate these decisions with the other planned large changes to Audita. Finalize
an implementation plan and concrete acceptance criteria before beginning migration.
### 2. Prove a Representative Configuration Path
After detailed planning, implement a bounded path from file selection and CLI
inputs through ConfigKit to Audita's runtime configuration. Exercise defaults,
selected overlays, proposal/validation inheritance, concurrency, duration values,
secrets, and redacted inspection using representative valid and invalid inputs.
Use the result to assess whether domain rules remain clear and substantial old
infrastructure can be removed. Resolve genuine library gaps deliberately, including
upstream discussion or changes when justified; avoid rebuilding a parallel loader.
A prototype is evidence for the design, not a permanent compatibility mode.
### 3. Complete the Migration
Convert the remaining configuration and command paths, remove superseded code and
legacy interfaces, and update maintained examples and canonical documentation
alongside the implementation. Replace tests that enforce retired conventions with
coverage of the new contract; preserve meaningful domain and runtime coverage.
Provide a concise migration guide covering old-to-new fields, flags, environment
settings, durations, credentials, validation, and inspection behavior. Automated
conversion tooling is not assumed to be necessary or in scope.
### 4. Validate and Release the Breaking Change
Verify representative loading and CLI workflows, domain resolution, safe secret
handling, diagnostics, examples, and runtime integration with deterministic tests.
Confirm obsolete loaders and aliases are gone and that documentation describes the
implemented model. Use the established [release procedure](../release.md).
Recommend Audita v2.0.0 and a new configuration-file schema version. Clearly reject
unsupported old file versions and direct users to the migration guide. Final
release numbering and whether this ships with other breaking changes remain
sequencing decisions; no publication is authorized by this roadmap.
The refactor is complete when both ConfigKit and PromptKit are adopted and the
complete application passes its required checks. Release compliance, license/
distribution review, and publication follow last under the established release
procedure. No publication is authorized by this plan.
## Sequencing Considerations
@@ -362,9 +296,9 @@ the same concepts twice. Work independent of configuration can proceed separatel
The [coordinated library-adoption roadmap](library-adoption.md) owns the proposed
sequence with PromptKit: shared boundary design and a PromptKit prototype, then
ConfigKit implementation, then PromptKit implementation, followed by a combined
v2.0.0 release. Revisit that sequence alongside other major changes. Keep
implementation units reviewable even when the release combines them.
ConfigKit and PromptKit implementation in dependency order within one refactor,
followed by a combined v2.0.0 release. Intermediate commits are work in progress,
not supported configurations or runnable milestones.
The separate [runtime behavior decisions](runtime-behavior.md) track remaining
questions and the selected retry direction.
@@ -373,8 +307,7 @@ structural repair from the outset. Include separate transient-retry and positive
repair budgets, plus the total-operation deadline policy, in the shared settings
design. The selected defaults and ownership are recorded in the
[boundary findings](library-boundary-findings.md#configuration-and-profile-contract).
Do not implement the runtime change
merely as a side effect of replacing the configuration loader.
Implement the runtime change explicitly as part of the combined refactor.
Retention and transcript timestamp decisions are not automatically included.
## Implementation Boundaries
@@ -382,8 +315,8 @@ Retention and transcript timestamp decisions are not automatically included.
- Pin ConfigKit v0.4.0 in production and align Go/toolchain requirements. ConfigKit requires
Go 1.25; the combined PromptKit prototype requires Go 1.25.5. Config-only
adoption does not automatically remove Audita's existing YAML dependency used for glossary parsing.
- Implement the selected type shape, final role inheritance, and interim runtime
gate above without leaking library concerns throughout the pipeline.
- Implement the selected configuration and final PromptKit runtime directly,
without temporary runtime adapters, compatibility shims, or process gates.
- Enable selected overlays as specified above. Keep imports and secret-directory
bootstrap disabled unless a separate concrete requirement justifies them.
- Validation and inspection resolve surviving credential references and fail on