Finalize the ConfigKit and PromptKit implementation plan

This commit is contained in:
2026-09-24 01:23:32 +00:00
parent 116648fa78
commit 1ad2119283
5 changed files with 642 additions and 659 deletions

View File

@@ -2,9 +2,9 @@
**Last updated:** 2026-09-24 **Last updated:** 2026-09-24
**Status:** Prototype boundary and selected overlays verified. The ordered **Status:** Final target selected; production implementation pending. The boundary
production handoff below is planned, not implemented; no production migration or and selected overlays are verified. The [implementation plan](implementation.md)
release has begun. owns execution stages; no production migration or release has begun.
## Purpose and Direction ## Purpose and Direction
@@ -27,10 +27,10 @@ application must depend on a versioned module, not a sibling checkout or local
module replacement. module replacement.
The shared boundary/prototype unit is complete. Its [selected contract, inventory, The shared boundary/prototype unit is complete. Its [selected contract, inventory,
and implementation handoff](library-boundary-findings.md) refine the provisional and evidence](library-boundary-findings.md) define the detailed target alongside
directions here; production configuration migration has not begun. this roadmap. Production configuration migration has not begun.
## Proposed End State ## Target End State
- ConfigKit owns strict YAML loading, input presence, default application, explicit - ConfigKit owns strict YAML loading, input presence, default application, explicit
override mechanics, recorded resolution, secret references, and safe inspection. override mechanics, recorded resolution, secret references, and safe inspection.
@@ -40,17 +40,17 @@ directions here; production configuration migration has not begun.
Audita's current per-layer validation and environment-overlay behavior. Audita's current per-layer validation and environment-overlay behavior.
- General configuration environment overrides are retired. Credentials use typed - General configuration environment overrides are retired. Credentials use typed
environment-name secret references; supplied references must resolve according environment-name secret references; supplied references must resolve according
to ConfigKit's contract. Review config-file selection separately as an to ConfigKit's contract. File selection follows the application-owned
application-owned startup decision. discovery rules in the selected boundary.
- Standardize duration representations, explicit empty/null/zero handling, and - Standardize duration representations, explicit empty/null/zero handling, and
inheritance on the shared conventions. Express genuine Audita domain rules inheritance on the shared conventions. Express genuine Audita domain rules
through the supported resolution and validation APIs. through the supported resolution and validation APIs.
- Review the configuration-related CLI afresh. Useful explicit flags can become - Retained domain-setting flags and proposal/validation profile flags become
ConfigKit override records; old names and aliases are not requirements. Direct explicit ConfigKit overrides according to the migration inventory. Remove old
credential-value flags should give way to the secret-reference convention. provider/key/retry/concurrency flags and aliases. Credentials use secret references.
- Use ConfigKit's redacted effective output and provide source/explanation access - Use ConfigKit's redacted effective output and provide source/explanation access
where useful. Existing JSON inspection output need not be preserved. Coordinate through the selected views below. Legacy JSON inspection is replaced.
diagnostic configuration snapshots with the new model. Diagnostic configuration snapshots use the same new model.
- Delete superseded loading and override machinery. Do not hide the old model - Delete superseded loading and override machinery. Do not hide the old model
behind adapters that defeat the adoption's consistency benefit. behind adapters that defeat the adoption's consistency benefit.
@@ -102,228 +102,69 @@ effective-config inspection, selected-profile inspection, and processing must al
use the same selected overlay. Preserve overlay provenance in configuration use the same selected overlay. Preserve overlay provenance in configuration
explanations and diagnostics while retaining secret redaction. explanations and diagnostics while retaining secret redaction.
The completed prototype extension and production-planning handoff are recorded The [recorded overlay results](library-boundary-findings.md#selected-overlay-extension-2026-09-24)
in the [prototype roadmap](library-boundary.md#next-unit-testing-overlays-and-production-migration-plan) prove the representative load, inspection, and HTTP path. Production acceptance
and its ordered [implementation plan](implementation.md). Production CLI changes requires the same selection to reach both actual LLM roles, with explicit flags
below remain subsequent migration work. winning over overlays and surviving secret references resolved/redacted correctly.
The isolated prototype is evidence, not a second supported runtime.
Implementation and acceptance work: ## Inspection and Invocation Contract
- The isolated prototype pins ConfigKit v0.4.0 and includes the testing-overlay `process`, `config validate`, and `config print-effective` share file/overlay
fixture. Preserve the original v0.3.1 evidence as historical; the selection and retained setting overrides. Invocation I/O paths remain process-only.
[extension results](library-boundary-findings.md#selected-overlay-extension-2026-09-24) Only explicitly supplied flags override settings. Validation requires an explicit
establish the selected-overlay behavior. config file; processing and inspection use the selected discovery order. Help is
- Verify unchanged base selection when no overlay is requested, both testing role available before configuration loading. All three commands validate selected
selections, and explicit setting flags taking precedence over the overlay. profiles without requiring a provider call during setup.
- Verify role inheritance after overlay application, effective profile/backend
assembly, and secret resolution from the surviving configuration.
- Reject missing/invalid overlay selections and prohibited version/directive changes
through ConfigKit; rely on its tests for exhaustive merge/parser cases.
- Verify redacted inspection and overlay provenance, then run both LLM roles through
the offline provider using the selected testing profiles. Provider requests must
use their model selections; no paid calls are needed.
- Add production CLI coverage for the three commands and document selection,
precedence, example layout, and migration guidance when implemented.
This expands the next configuration unit; it does not reopen the completed `config print-effective --view settings|profiles|explain` exposes three YAML views:
prototype's original scope or change the ConfigKit → PromptKit → combined release
sequence. The production LLM migration remains necessary before promising the full
new runtime behavior.
## Ordered Production Migration Handoff - `settings` (default): ConfigKit's redacted effective configuration.
- `profiles`: an Audita-owned summary of each selected role's profile, backend,
model, temperature, token limit, top-p, timeout, service tier, and reasoning
effort where present. Arbitrary extra provider parameters and credentials are
omitted; this is a selected-controls view, not every execution input.
- `explain`: field paths, sources, and derivation records from ConfigKit, without
resolved secret values.
The [migration inventory](library-boundary-findings.md#migration-inventory) is the A ConfigKit snapshot alone does not describe profile-owned controls. Profile
field-by-field old-to-new map. The [selected boundary](library-boundary-findings.md#configuration-and-profile-contract) inspection must enforce the registered-backend and credential boundary. Diagnostic
owns defaults and semantic choices; the [v0.4.0 overlay results](library-boundary-findings.md#selected-overlay-extension-2026-09-24) snapshots use these safe views and preserve overlay/explicit-override provenance.
prove the representative load, inspection, and HTTP path. These packages are The migration guide explains the intentional replacement of legacy inspection JSON.
future work within one coordinated refactor to adopt both libraries. The order
organizes implementation dependencies, not independently runnable product states.
Intermediate commits need not build or run; do not add temporary scaffolding,
compatibility shims, or a processing gate to support them. Remove superseded code
as its replacement lands. Run focused checks as components become testable, and
validate the complete application before declaring the refactor finished. The
isolated prototype is evidence, not a second production runtime.
### 1. Define and load the complete version-2 configuration ## Integration and Completion Criteria
**Depends on:** the verified prototype. **Owns:** `internal/core/config` and its ConfigKit and PromptKit form one intentional breaking refactor. Intermediate
tests. Replace the version-1 YAML/env application path with one ConfigKit v0.4.0 commits need not build or run. The target contains no temporary old-runtime adapter,
definition and a version-2 rule. Pin the published module in the root Go module compatibility shim, processing gate, dual loader, or automatic legacy conversion.
and raise the production Go minimum as needed. Keep the existing YAML dependency The [combined feature roadmap](library-adoption.md) owns execution/reporting intent;
used for glossary parsing. ConfigKit imports and secret-directory bootstrap stay [implementation.md](implementation.md) alone owns stage order and implementation tasks.
disabled.
The definition must cover every retained domain setting in the inventory: The final application loads all retained domain settings, assembles both selected
ordered/repeated `pipeline.modules`, `output.schema`, validation prompt budget, profiles, and processes through PromptKit. Controlled HTTP evidence must connect
chunking, normalization, thresholds, context, and diagnostics, plus `profiles_dir`, configuration selection to actual requests, domain outcomes, safe diagnostics,
backend endpoint/concurrency/queue capacity/optional credential, and LLM proposal and shared finite backend capacity. Maintained examples and one migration guide
and validation profile selection, transient retries, repair attempts, and total cover the complete file/flag/environment transition. Obsolete application-owned
operation timeout. Apply the inventory's defaults once. Preserve presence: loading, transport, rendering, and scheduler machinery is removed.
omitted validation profile inherits the **final** proposal profile; explicit
empty/null validation fails. Omitted or null section target means automatic, zero
fails. Explicit empty module list means no modules. Optional credential null means
unauthenticated; supplied references must resolve to nonempty values. Keep the
selected numeric-second normalization fields numeric, and use a duration string
for `llm.operation_timeout`; do not revive version-1 duration coercion. Validate
backend limits, references, profile selections, thresholds, and cross-field domain
rules on the resolved result. Preserve ConfigKit provenance for derived values.
**Acceptance/tests:** `internal/core/config` tests use temporary files to cover The combined target is Audita v2.0.0. Release compliance and publication remain
version rejection, strict fields, defaults, omission/null/zero, module order, last and separate from implementing the refactor. Retention and transcript timestamp
final role inheritance, overlays and explicit override precedence, secret changes are not included; the selected retry/repair contract resolves the retry
replacement/missing/null, and retained domain validation. Compare behavior to the follow-up in [runtime behavior](runtime-behavior.md).
prototype extension rather than duplicating ConfigKit's parser matrix. No legacy
environment overlay or old-schema acceptance remains.
### 2. Select files and map invocation input ## Scope Boundaries
**Depends on:** package 1. **Owns:** `internal/cli/run.go`,
`internal/cli/process_flags.go`, `internal/cli` tests, and `cmd/audita` subprocess
tests. Select exactly one main file: explicit `--config`, then the two system
locations in the selected boundary. Remove `AUDITA_CONFIG`; a missing explicit
path fails, while absent system files are skipped. Require an explicit file for
`config validate`. Parse help before loading configuration.
Add invocation-only `--overlay <name>` to `process`, `config validate`, and
`config print-effective`, passing it to `Options.Overlay` for the same selected
main file. Omitted, empty, or `default` selects the base. Keep transcript,
glossary, output, and report paths outside persistent configuration. Map only
flags actually supplied by the user into ConfigKit overrides; parser defaults
must never defeat file or overlay values. Retain useful domain-setting flags from
the inventory. Add profile-selection flags. Remove direct API-key/model/endpoint,
old retry/role/global concurrency flags and aliases rather than translating them
silently. Retire all general `AUDITA_*` configuration overlays listed in the
inventory; environment names retain meaning only when explicitly referenced as
secrets.
**Acceptance/tests:** CLI and subprocess tests cover discovery order, explicit
missing files, all three commands with base/testing/default/missing selections,
overlay then explicit-setting precedence, omitted flags, usage/error streams and
exit classifications. Reject old flags and implicit environment configuration.
The same loaded result must feed validation, effective inspection, and processing.
### 3. Assemble profiles and safe inspection
**Depends on:** packages 1–2. **Owns:** a small CLI/runtime assembly boundary,
config inspection, and diagnostic snapshot wiring in `internal/cli` and
`internal/core/diagnostics`; verify against `internal/framework/processreport`
where snapshots are referenced. Resolve `profiles_dir` relative to the selected
**main** file, including when an overlay is selected; absolute paths remain valid.
Inspect both selected PromptKit profiles and verify their named registered backend,
endpoint equality, and absence of a competing profile credential lookup. Treat
ConfigKit settings and PromptKit generation controls as separate views.
Use `WriteEffective` for redacted effective YAML and expose relevant
explanation/provenance and selected-profile model/backend controls through an
Audita-owned safe view. A ConfigKit snapshot alone cannot claim to describe the
full execution target. Validation and inspection resolve surviving secret
references without provider calls; a missing or empty surviving reference fails,
while a replaced reference is not resolved. Snapshot and surfaced error paths
must redact secret **values** and preserve overlay/CLI origin. Avoid serializing
raw library objects into stable report formats.
**Acceptance/tests:** config-command and diagnostics tests verify relative paths,
same selection across views, overlay/CLI origin, surviving credentials,
unauthenticated null, and no secret values in YAML, explanations, selected-profile
inspection, diagnostics, or errors. Update `internal/cli/run_test.go`,
`internal/core/config/*_test.go`, and report/diagnostics tests that own these
outputs.
### 4. Connect the final configuration directly to PromptKit execution
**Depends on:** packages 1–3. **Owns:** the final CLI/runtime assembly boundary,
`internal/framework/llm`, prompt/schema loading, runner wiring, and relevant tests.
Implement PromptKit adoption as part of the same refactor. Convert domain settings
into Audita's existing domain structs and supply the selected profiles, registered
backends, resolved secrets, retry/repair budgets, and logical-operation deadlines
directly to PromptKit. Keep library types out of correction algorithms and ledgers.
Migrate all module and validator prompts, preserve their domain checks, and map
PromptKit results, classified failures, and observations into Audita reporting and
redacted diagnostics. Replace the old provider transport, prompt loader,
structural-output handling, and global/role scheduler composition. Follow the
[coordinated execution scope](library-adoption.md#4-complete-promptkit-adoption)
and [execution/observation contract](library-boundary-findings.md#execution-and-failure-mapping).
Do not build an adapter to feed the old LLM runtime or introduce an interim
`process` rejection mode.
**Acceptance/tests:** controlled HTTP tests prove both roles' final model/backend
selection, overlays and explicit overrides, finite per-backend capacity, explicit
zero retries, positive structural repair, total-operation deadlines, cancellation,
domain acceptance/rejection, and safe observations including partial failures.
Processing uses the new configuration and execution path together. Reuse existing
domain coverage and the prototype's integration evidence; update tests tied to the
retired transport and scheduler contracts.
### 5. Remove superseded configuration paths and update maintained contracts
**Depends on:** packages 1–4. **Owns:** removal of old file/env merge, aliases,
direct-key flags, legacy redaction/inspection paths, and obsolete tests in
`internal/core/config` and `internal/cli`. Retain Audita's domain checks,
normalization/chunking, output encoding, module and validator behavior. Update
`examples/` with version-2 base configuration, `overlays/testing.yml`, complete
PromptKit profiles, and secret-free credentials by reference. Update the canonical
`docs/config.md`, `docs/cli.md`, operations/integration/internal references as
their contracts change, and provide one migration guide with the complete
inventory's file/flag/environment mapping, duration and empty-list changes,
credentials, profiles/overlays, inspection, and changed runtime/diagnostic
semantics. Do not leave dual loaders or a permanent translation mode.
**Acceptance/tests:** maintained examples validate offline; removed flags and old
schema are rejected; relevant config/CLI/subprocess/report tests pass; documentation
links and `git diff --check` pass. Remove the isolated prototype when its useful
coverage and examples have moved into the production suite.
### 6. Validate the combined end state and prepare release
**Depends on:** packages 1–5. Verify the assembled version-2 configuration,
overlay, profile, execution, reporting, and artifact workflows. Run the relevant
full build/test checks, confirm obsolete configuration and LLM machinery is gone,
and verify maintained examples and the migration guide against the final behavior.
There is no requirement to validate or preserve intermediate product states.
The refactor is complete when both ConfigKit and PromptKit are adopted and the
complete application passes its required checks. Release compliance, license/
distribution review, and publication follow last under the established release
procedure. No publication is authorized by this plan.
## Sequencing Considerations
Changes to module configuration, runtime settings, CLI design, or diagnostic
reporting are likely to overlap this migration. Design their shared target model
first. If another change will substantially reshape those areas, avoid migrating
the same concepts twice. Work independent of configuration can proceed separately.
The [coordinated library-adoption roadmap](library-adoption.md) owns the proposed
sequence with PromptKit: shared boundary design and a PromptKit prototype, then
ConfigKit and PromptKit implementation in dependency order within one refactor,
followed by a combined v2.0.0 release. Intermediate commits are work in progress,
not supported configurations or runnable milestones.
The separate [runtime behavior decisions](runtime-behavior.md) track remaining
questions and the selected retry direction.
The coordinated plan adopts PromptKit's explicit retry semantics and enables
structural repair from the outset. Include separate transient-retry and positive
repair budgets, plus the total-operation deadline policy, in the shared settings
design. The selected defaults and ownership are recorded in the
[boundary findings](library-boundary-findings.md#configuration-and-profile-contract).
Implement the runtime change explicitly as part of the combined refactor.
Retention and transcript timestamp decisions are not automatically included.
## Implementation Boundaries
- Pin ConfigKit v0.4.0 in production and align Go/toolchain requirements. ConfigKit requires - Pin ConfigKit v0.4.0 in production and align Go/toolchain requirements. ConfigKit requires
Go 1.25; the combined PromptKit prototype requires Go 1.25.5. Config-only Go 1.25; the combined PromptKit prototype requires Go 1.25.5. Adopting these libraries
adoption does not automatically remove Audita's existing YAML dependency used for glossary parsing. does not automatically remove Audita's existing YAML dependency used for glossary parsing.
- Implement the selected configuration and final PromptKit runtime directly, - Implement the selected configuration and final PromptKit runtime directly,
without temporary runtime adapters, compatibility shims, or process gates. without temporary runtime adapters, compatibility shims, or process gates.
- Enable selected overlays as specified above. Keep imports and secret-directory - Enable selected overlays as specified above. Keep imports and secret-directory
bootstrap disabled unless a separate concrete requirement justifies them. bootstrap disabled unless a separate concrete requirement justifies them.
- Validation and inspection resolve surviving credential references and fail on - Validation and inspection resolve surviving credential references and fail on
missing or empty values; no credential-free bypass is planned. missing or empty values; no credential-free bypass is planned.
- Add the safe selected-profile view and ConfigKit provenance to inspection and - Inspection and diagnostics use the safe selected-profile view and ConfigKit
diagnostics as specified in package 3, then account for changed formats in the provenance described above. Changed formats are covered by the migration guide.
migration guide. No schedule or effort estimate is committed.
The eventual completion test is that Audita follows the suite's ConfigKit The eventual completion test is that Audita follows the suite's ConfigKit
conventions while retaining its intended domain capabilities, with a clear conventions while retaining its intended domain capabilities, with a clear

View File

@@ -1,306 +1,527 @@
# Implementation Plan: Testing Overlays and Migration Handoff # Implementation Plan: Adopt ConfigKit and PromptKit
**Last updated:** 2026-09-24 **Last updated:** 2026-09-24
**Status:** Complete (2026-09-24). All six stages below are finished. This plan implements the next **Status:** Ready for implementation. All stages below are pending. This replaces
unit in the [prototype roadmap](library-boundary.md#next-unit-testing-overlays-and-production-migration-plan), the completed testing-overlay prototype plan; its results remain in the
not the production ConfigKit or PromptKit migrations. [boundary findings](library-boundary-findings.md#selected-overlay-extension-2026-09-24).
**Production handoff revision:** The completed prototype stages remain recorded ## Scope and Working Instructions
below. The current [production plan](configkit.md#ordered-production-migration-handoff)
targets both libraries in one refactor; intermediate commits need not build or run.
It supersedes the earlier interim-runtime design. No temporary adapter,
compatibility shim, or processing gate is required.
Execute Stage 1 through Stage 6 in order. Each stage is sized as a separate coding Implement the final configuration and execution model in the
prompt for a gpt-5.6-terra agent and ends with a reviewable result. When assigned [combined feature roadmap](library-adoption.md) and [configuration roadmap](configkit.md).
one stage, finish that stage and report the next stage; when assigned the entire The [selected boundary](library-boundary-findings.md#configuration-and-profile-contract),
plan, continue through all stages. Do not restart completed work: inspect current [migration inventory](library-boundary-findings.md#migration-inventory), and
files, recorded results, and Git state before deciding what remains. [failure mapping](library-boundary-findings.md#execution-and-failure-mapping)
are the detailed behavioral inputs. Do not repeat the prototype or reopen settled
choices without a concrete contradiction in the pinned libraries.
Before each stage, follow `AGENTS.md`, [development guidance](../development.md), This plan is for a gpt-5.6-terra coding agent. Execute Stage 1 through Stage 12 in
and the [agent policy](../policy/agents.md). Preserve unrelated changes. Use the order. Each stage is one bounded implementation prompt. When assigned the whole
knowledge graph for code discovery; examples may be excluded from its index, in plan, continue between stages without a permission round. When assigned one stage,
which case inspect the known prototype files directly. Follow the repository's finish that stage's scope and report the next stage. Mark completed stages with a
testing and documentation policies. Complete relevant checks and make a plain short result and actual validation evidence; do not call a pending integration
English local commit for each coherent stage under the standing commit policy. check passed.
No push, tag, release, or upstream-library edit is included.
Update each stage's status line with its result and verification evidence when Read `AGENTS.md`, [development guidance](../development.md), and all documents under
finished; keep detailed design/results in their owning roadmap documents. Do not `docs/policy/`. Use graph-first code discovery. Follow the
mark a stage complete while required checks are failing. Fix in-scope failures; [agent](../policy/agents.md), [architecture](../policy/architecture.md),
record any genuine upstream gap and its effect on later stages before continuing [testing](../policy/testing.md), [documentation](../policy/documentation.md), and
work that depends on it. Routine implementation choices do not need fresh approval. [release](../policy/release.md) policies. The intentional dependency, configuration,
and scheduler changes here supersede the corresponding old implementation choices;
update canonical descriptions as the new behavior lands.
The [ConfigKit roadmap](configkit.md#selected-overlays-for-testing) owns the overlay The stages divide one refactor, not supported intermediate releases. Intermediate
contract. The [boundary findings](library-boundary-findings.md) own existing choices commits may fail to build or run. Do not write temporary adapters, compatibility
and evidence. Preserve the original v0.3.1 results as historical. ConfigKit v0.4.0 shims, duplicate execution paths, stubs returning invented successes, or an interim
renames selected configuration profiles to overlays; do not rename PromptKit's processing gate. Remove superseded code when replacing it. Run focused checks
`profiles/`, `profiles_dir`, or execution-profile fields. where the affected packages are testable; record checks deferred because later
stages must reconnect callers. Such a completed stage is still work in progress
on the overall refactor. Commit coherent stages with plain-English messages under
the standing policy. Do not spend a stage making an intermediate application work.
All implementation in this plan stays under `examples/library-adoption/`, plus Keep tests deterministic, offline, and credential-independent. Reuse prototype
relevant roadmap/README updates. Keep the root Go module, production CLI/runtime, scenarios at their production boundary rather than copy the nested application or
correction algorithms, and upstream libraries unchanged. Use the existing built-in retest all library internals. Preserve domain tests unless the selected contract
PromptKit HTTP path with a controlled transport, synthetic credentials, and no live actually changes. Update each affected canonical reference alongside its owning
provider calls. Keep imports and secret-directory bootstrap disabled. This work stage; Stage 11 finishes the cross-document migration guide and examples.
does not introduce general environment overlays or backwards compatibility.
Unless explicitly stated otherwise, shell commands below run from No upstream edits, paid calls, push, tag, publication, retention-policy change,
`examples/library-adoption/`. Use an installed Go 1.25.5 or newer. Acquire dependencies timestamp fix, or unrelated pipeline redesign is included. Release compliance
with `GOWORK=off go mod download all` when needed; then use the offline command and publication remain a later release unit. Completion here means both libraries
prefix below. Root `go test ./...` does not cover this nested module. are adopted, obsolete machinery is removed, and the final application is verified.
## Stage 1: Upgrade the Prototype Dependency ## Implementation Decisions
**Status:** Complete (2026-09-24). **Depends on:** nothing. ConfigKit v0.4.0 pinned; These decisions close the remaining application-design details; background and
the nested offline tests and demonstration passed with the expected 1/4/0/2 result. user-facing policy belong to the feature roadmaps.
Read the prototype README, `go.mod`, `go.sum`, and the current ConfigKit roadmap. - Pin the published modules used by the verified prototype: ConfigKit v0.4.0 and
Confirm the working tree and run the existing nested-module tests once as a PromptKit v0.11.0. Set the root minimum to Go 1.25.5. Keep yaml.v3 while glossary
baseline. Upgrade ConfigKit to the published v0.4.0 release using module tooling; parsing uses it. No sibling replacements or dependency on Audita's own old release.
leave PromptKit and Audita dependency versions unchanged. Review any indirect - Keep application loading in `internal/core/config`, embedded prompt definitions
module changes and retain only changes needed by this upgrade. Do not add local in `internal/prompts`, schemas in `internal/framework/responseschema`, and the
replacements or an active Go workspace. PromptKit integration in `internal/framework/llm`. Use one engine per process
invocation, shared by both role-bound clients; neither modules nor validators
construct engines. The CLI composes these pieces. Avoid a generic service layer.
- Retain the `CompleteStructured(ctx, request, out)` boundary, using Audita-owned
prompt ID/version, string variables, schema identity, and diagnostic context.
Replace the old rendered-message/model request fields. Model/profile/credential
selection belongs to the role-bound adapter, not individual callers. Consolidate
the equivalent validator types onto the framework contract where needed, without
an import cycle. No PromptKit types in correction algorithms or ledger records.
- Store PromptKit definitions for all nine existing prompt IDs with their existing
`v1` selection. Replace the custom `hardening` template function with an
application-supplied `HardeningPolicy` string variable sourced from the one
embedded hardening fragment. The adapter supplies that reserved variable after
caller variables. PromptKit renders all messages; do not retain an Audita template
engine or introduce generated prompt files requiring a second build step.
- Use one shared configuration-input parser for `process`, `config validate`, and
`config print-effective`. All three accept `--config`, `--overlay`, and the
retained setting overrides in the inventory, including the two profile flags.
Only `process` accepts transcript/glossary/output/report paths. Only explicitly
visited setting flags become ConfigKit overrides. Do not add retry/capacity/key
flags beyond the agreed inventory.
- `config validate` requires an explicit nonempty config path. The other two
commands use the selected discovery order. All validate the resolved settings
and both selected profiles without provider calls, including an empty module
pipeline. Help succeeds before file discovery or secret/profile loading.
- `config print-effective` adds `--view settings|profiles|explain`, defaulting to
`settings`. Settings uses ConfigKit `WriteEffective` YAML. Profiles emits a safe
Audita-owned YAML summary for the proposal and validation roles. Explain emits
a YAML sequence of field path, source, and derivation records from ConfigKit's
recorded resolution, with no resolved secret values. Unknown views are usage
errors. This avoids mixing multiple formats into the default output or adding
another command tree. Profiles includes selected ID, backend, model, temperature,
max tokens, top-p, per-attempt timeout, service tier, and reasoning effort when
present; omit arbitrary extra provider parameters and credentials. It is a
selected-controls summary, not a dump of every profile input.
- Keep CLI exit classes 0/1/2 and transcript/report stream behavior. Invalid user
settings/profiles are configuration errors (2); broken embedded assets and
runtime/provider failures are runtime errors (1). Serialization/write failures
remain runtime errors. Do not classify malformed HTTP envelopes as repairable
model output.
- Publish process reports as `audita-process-report` schema `v2`. Preserve existing
domain fields and ledger ordering, add a top-level `llm_summary`, and replace
obsolete scheduler utilization fields as specified in Stage 10. Transcript
encoder versions stay unchanged. No legacy report mode is needed.
No source changes were needed in the earlier isolated review. If a change is now ## Stage 1: Replace the Configuration Model and Loader
necessary, investigate and record the concrete reason instead of assuming that
ConfigKit execution overlays and PromptKit profiles share an API.
Verification: **Status:** Pending. **Depends on:** the completed prototype only.
**Owns:** root `go.mod`/`go.sum`, `internal/core/config`, and config tests.
```sh 1. Pin both modules and the Go minimum above. Align build/CI toolchain declarations
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOSUMDB=off go test ./... with that minimum; preserve the existing platform set and release gates.
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOSUMDB=off go run . 2. Replace version-1 YAML/env merge and duration coercion with one ConfigKit
``` definition covering every retained field in the migration inventory. Use the
prototype's types/lifecycle as evidence, then add the full domain configuration.
Remove legacy environment loaders and alias resolution; do not keep two loaders.
3. Implement defaults, presence-aware backend defaults, final validation-profile
inheritance, and recorded derivations. Keep `chunking.target_sections` optional,
explicit empty module lists empty, and module order/repeats intact. Validate
domain constraints after overlay and explicit override resolution. Preserve
numeric normalization seconds; operation timeout is a duration string.
4. Use typed optional secret references. Resolve every surviving reference, including
unselected backends; missing/empty references fail, replacement resolves only the
replacement, and null permits unauthenticated use. Enable selected overlays;
leave imports and secret-directory bootstrap disabled.
5. Expose the ConfigKit result to CLI composition for effective output and provenance;
convert retained settings into existing domain inputs without transporting the
library result through correction code. Update `docs/config.md`'s implemented
loader/schema description, clearly marking unfinished CLI integration if needed.
Acceptance: the manifest and sums pin v0.4.0, tests pass, and the demonstration **Acceptance and validation:** focused config tests cover strict version/fields,
still yields one accepted correction, four attempts, zero transient retries, and selected defaults, omission/null/zero, final role inheritance, empty/repeated
two structural repairs. Update the README/findings to distinguish the new pin from modules, overlay/override precedence, secret replacement/null/missing, and retained
historical v0.3.1 evidence. Do not claim overlay integration yet. cross-field rules. Old schema fails with migration guidance. General environment
variables have no implicit effect. Run `go test ./internal/core/config` when this
package is independently buildable; record caller compile failures for later stages.
Do not duplicate ConfigKit's exhaustive YAML/merge tests.
## Stage 2: Add Overlay Loading and Configuration Fixtures ## Stage 2: Replace CLI Configuration Selection and Overrides
**Status:** Complete (2026-09-24). **Depends on:** Stage 1. On-disk overlay and **Status:** Pending. **Depends on:** Stage 1.
profile fixtures, precedence, inheritance, rejection, and runtime assembly tests **Owns:** `internal/cli/run.go`, `process_flags.go`, configuration command parsing,
pass in the nested module (`go test ./...`). and corresponding CLI/subprocess tests.
Start with `config.go`, `config_test.go`, `config.yaml`, and `profiles/`. Extend 1. Implement the shared configuration-input parser described above. Preserve the
`loadConfig` with an explicit overlay-selection argument or a small invocation existing supported positional-argument arrangement for `process`, required
options value. Pass it to ConfigKit `Options.Overlay`; selection does not belong glossary, output/report behavior, and root/subcommand help. Remove obsolete
in the persistent `Config` struct. Update existing callers to request no overlay LLM/model/key/timeout/retry/concurrency flags and aliases outright.
without changing their behavior. Avoid a parallel merge implementation. 2. Select exactly one file: explicit `--config`, then
`/usr/local/etc/audita/config.yml`, then `/etc/audita/config.yml`. Missing system
candidates are skipped; selected inaccessible files and explicit empty/missing
paths fail. With no file, report a configuration error; do not select a provider.
Remove `AUDITA_CONFIG`. Test discovery using an injected path list/filesystem
seam rather than modifying the host's system configuration.
3. Pass invocation overlay selection to ConfigKit and only supplied setting flags
to its overrides. `--modules ''` must clear the pipeline; explicit numeric zero
reaches validation rather than being treated as absent. Relative profile bases
derive from the selected main file, never the overlay directory.
4. Wire the parsed result into the final process/config command entry points.
Profile-dependent command completion belongs to Stage 4; do not emulate the old
runtime to make the process command work now. Update `docs/cli.md` for the final
input contract as parsing lands.
Add these prototype fixtures: **Acceptance and validation:** parser/loading tests prove help without usable
configuration, discovery/error cases, explicit-only overrides, retained domain
flags, removed flag rejection, and ignored legacy environment values. Table-drive
shared cases; retain representative subprocess checks rather than triplicate every
parser test across commands. Config commands reject positional arguments and
process-only flags. Defer assembled command checks until Stage 8 if imports still
prevent compilation.
- `overlays/testing.yml`, selecting `proposal-testing` and `assessment-testing` ## Stage 3: Migrate Embedded Prompt Assets and Call Contracts
through the existing two `llm` role fields. Do not put `version`, model settings,
or ConfigKit directives in this overlay.
- Complete PromptKit profiles under `profiles/` with those IDs and distinct
synthetic model IDs, such as `synthetic-grammar-testing` and
`synthetic-review-testing`. Both use the existing `offline` backend and keep the
current positive repair policy and finite shared capacity.
Preserve base `config.yaml` as the no-overlay fixture, including its explicitly **Status:** Pending. **Depends on:** Stage 1; execute after Stage 2.
selected assessment profile. This makes the risk of accidentally leaving the **Owns:** `internal/prompts`, `internal/framework/responseschema`,
validation role on its base model observable. Profile paths remain relative to `internal/framework/contracts`, validator contract aliases, and prompt tests.
main configuration according to the selected application contract, not relative
to the overlay file.
Add focused configuration tests proving: 1. Convert all four module and five validator prompts listed in
[the prompt registry](../internal/prompts.md) to PromptKit definitions. Preserve
instructions, variables, glossary/description context, prompt IDs, and the two
existing JSON response schemas. Embed assets once and expose their filesystem
to the runtime. Do not copy the prototype's duplicate schema directory.
2. Keep output contracts JSON Schema based, but supply the full effective output
contract and repair count on each runtime request. Do not create separate
operator-facing repair defaults per prompt. Use the hardening variable decision
above to retain a single hardening source.
3. Change the structured-call request to prompt identity, variables, response schema
identity, and diagnostic context. Keep decoded output and typed Audita metadata
as the return boundary. Include role/module instance/section/batch/validator
context where available for attribution; do not derive identities from messages.
4. Replace the old parser/renderer and hash assumptions. Keep small metadata and
schema lookup helpers where Audita needs them. Update the prompt reference.
- Omitted selection, empty selection, and `default` select the base configuration; **Acceptance and validation:** a table covering all nine definitions prepares
`testing` selects both new profiles. representative variable sets through PromptKit with controlled profile fixtures;
- Explicit ConfigKit overrides win over overlay values. Only supplied flags will checks include hardening, section/glossary/description data, missing variables,
become overrides in Stage 3; parser defaults must not enter this layer. correct schema association, and rendered content preserving the previous task
- With validation selection absent from both base and overlay, validation inherits instructions. Use reviewed expectations, not a retained legacy renderer or old
the final proposal selection, including an explicit override. An existing Audita module dependency. Existing schema/domain tests remain meaningful. Callers
explicit base validation selection remains explicit unless replaced by overlay are deliberately migrated in Stages 6–7.
or CLI. Empty/null validation values retain the established rejection behavior.
- Runtime profile assembly resolves both selected profiles to the shared backend.
Use temporary on-disk layouts for overlay tests so ConfigKit owns file discovery. ## Stage 4: Assemble Profiles and Finish Offline Inspection
Existing byte-source tests can remain for cases that do not select an overlay;
if a byte source needs an overlay, supply ConfigKit's supported filesystem context
rather than relying on the process working directory.
Acceptance: the new fixtures load and assemble correctly, the focused precedence/ **Status:** Pending. **Depends on:** Stages 1–3.
inheritance tests pass, and `go test ./...` in the nested module still passes. **Owns:** engine/profile assembly in `internal/framework/llm`, CLI config commands,
Do not yet claim that the testing models reached the HTTP transport. and inspection tests.
## Stage 3: Demonstrate and Verify Model Switching 1. Construct one PromptKit engine with the embedded assets, complete profile files,
and registered application backends. Reuse the prototype's supported filesystem,
backend, retry-config, and inspection APIs. Accept an HTTP client at the external
boundary for tests; production uses the library transport.
2. Resolve `profiles_dir` against the main config file. Inspect both selected roles,
require a registered backend, reject endpoint-only bypass and profile credential
lookup, and compare endpoints after trimming surrounding whitespace and trailing
slashes, as in the prototype. Do not invent profile inheritance/catalog wiring
or reject unrelated unselected profile content beyond library behavior.
3. Complete `config validate` and the three effective-output views. Settings,
profiles, and explanations share the same load/assembly path used by process.
Keep the `config is valid` success message. Redact configured secret values in
any surfaced library error; do not stringify raw configuration/profile objects.
4. Add the internal safe-view helpers later used for snapshots. Document the view
interface and its limits in `docs/cli.md` and `docs/config.md`.
**Status:** Complete (2026-09-24). **Depends on:** Stage 2. Nested tests inspect **Acceptance and validation:** offline command tests prove all views and validation
outbound base/testing/override model IDs, including repairs; all three documented agree for base/testing/explicit-profile selections; main-file-relative and absolute
offline demonstrations passed with one accepted correction and 4/0/2 attempts, profile paths work; missing profiles, backend mismatch, competing credentials,
retries, and repairs. missing surviving secrets, and invalid views fail with the selected exit class.
Use a transport that fails the test if called. Inspect outputs/errors for synthetic
secret absence and overlay/CLI provenance, including inherited validation selection.
Start with `main.go`, `offline.go`, `runtime.go`, and `workflow_test.go`. Add minimal ## Stage 5: Implement the Final PromptKit Structured-Call Adapter
standard-library argument handling to the prototype executable:
- `--overlay <name>` selects the ConfigKit overlay. **Status:** Pending. **Depends on:** Stages 3–4.
- `--proposal-profile <id>` and `--validation-profile <id>` produce ConfigKit **Owns:** `internal/framework/llm`, the typed failure boundary in
overrides only when explicitly supplied. `internal/framework/structuredoutput`, and adapter tests.
- Keep the existing fixture-based invocation from the prototype directory. There
is no need to reproduce production file discovery or its subcommand tree here.
Use a small testable entry function if needed to check flag handling without 1. Implement role-bound clients over the shared engine. Start the operation context
repeated subprocess builds. Preserve ordinary flag error handling. Keep execution and Audita logical-call record before preparation; inherit earlier caller deadlines.
on the synthetic transport for every supported demonstration invocation; do not Select the role profile and backend secret, supply reserved hardening variables,
add a live-network mode. Extend that transport to recognize the testing models and and pass an explicit full schema/repair contract. Use `PrepareExecution`, its
preserve the initial-invalid-output/repair behavior for each role. `Details`, then `RunPrepared` so diagnostic messages/hashes describe the actual
execution snapshot. Defer handle discard; do not call `Prepare` and then `Run`
with an independently reloaded profile. Before dispatch, verify the prepared
target still matches the role's registered backend/endpoint and has no competing
credential lookup. A changed target fails safely rather than sending a backend
key to a different target. This is the final diagnostic/security boundary, not
an intermediate adapter.
2. Check both the returned error and `result.Validation.IsValid`: exhausted repair
can return a result with failed validation and no error. Decode only validated
JSON into the requested Audita output. Use the selected failure table:
exhausted output repair becomes the Audita malformed-output classification; provider/transport/protocol/retry/admission/deadline failures
remain runtime failures. Preserve typed causes for classification and safe
messages for users. Explicit zero retries must not become retries-exhausted.
3. Preserve prepared messages/hashes and available final content and profile/model
metadata through Audita-owned types. Provide a per-call observer hook and logical
start/finish record bound to Audita diagnostic context for Stage 9; do not
create another retry loop, scheduler, provider envelope parser, or schema engine.
4. Remove the old OpenAI-compatible client and its role config resolution. Retain
only applicable redaction and diagnostics helpers. Update the LLM integration
reference to delegate wire/retry mechanics to the pinned library.
Add a compact safe selection summary to demonstration output: requested overlay **Acceptance and validation:** controlled HTTP tests cover successful decode,
and effective profile, backend, and model for each role, alongside the existing positive repair and exhaustion for both schema types, 429/5xx retry recovery,
domain outcome and attempt/retry/repair totals. It must reflect the assembled explicit zero, exhausted positive budget, transport/malformed-envelope failure,
configuration rather than echoing flag values. Inspection APIs and tests in the combined retry-plus-repair bounds, preparation failure without dispatch, exact
next stage suffice; do not create production config commands in this prototype. prepared request metadata, target-change rejection, and operation cancellation
during a long retry wait. Assert classifications and meaningful counts, not timing-sensitive sleeps
or complete error strings. Reuse the prototype sequences; no need to reproduce
PromptKit's entire status-code matrix.
Test through the assembled path and inspect actual outbound HTTP request bodies: ## Stage 6: Migrate All Proposal Callers
- No overlay sends the base proposal and assessment models. **Status:** Pending. **Depends on:** Stage 5.
- `testing` sends the testing proposal and assessment models, including repair **Owns:** `internal/modules`, `internal/framework/proposal_generation`, and their tests.
requests, and still produces one accepted correction with two repairs.
- An explicitly supplied role-profile flag overrides that role's testing selection;
the other role retains its overlay selection. An omitted flag preserves overlay
selection. Match reported selections to observed outbound model IDs.
Use distinct models per role for these demonstration cases. Do not expand the fake 1. Switch glossary, homophones, spoken_word, and grammar proposal construction to
provider into a general simulation framework. Reuse existing prompt/schema/domain prompt IDs and variables. Preserve section/glossary encoding, context trimming,
helpers and existing capacity/observation coverage. replacement policy, confidence filtering, and applicability checks.
2. Feed repaired structured output into the existing domain path. Exhausted
structural repair produces the existing section warning and no proposals;
runtime failures propagate. Schema-valid but inapplicable proposals do not
trigger semantic retries or additional repairs.
3. Update fake clients and module prompt tests to the final request contract.
Delete duplicate message rendering and obsolete transport/schema checks; retain
domain protection and prompt variable coverage. Update module internals docs.
Acceptance: nested-module tests pass and these invocations visibly show the expected **Acceptance and validation:** all four modules have an exercised variable/schema
selections and successful domain result: mapping; existing domain suites retain their assertions. Representative controlled
adapter coverage proves repaired proposals reach domain checks and that a low-
confidence/inapplicable proposal remains rejected. Preserve section/proposal
ordering. Run the module and proposal-generation suites as imports become buildable.
```sh ## Stage 7: Migrate LLM Validators Without Changing Domain Decisions
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOSUMDB=off go run .
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOSUMDB=off go run . --overlay testing
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOSUMDB=off go run . --overlay testing --proposal-profile proposal
```
Update the prototype README with these implemented commands and fixture layout. **Status:** Pending. **Depends on:** Stage 6.
State that synthetic testing IDs demonstrate operator-selectable routing, not **Owns:** `internal/framework/validators`, validation prompt builders/batching,
measured price, speed, or model quality. and validator tests.
## Stage 4: Verify Failure, Credential, and Inspection Boundaries 1. Migrate spoken_form_plausibility, meaning_reversal_review, editorial_review,
grammar_review, and spoken_word_review to prompt IDs/variables and the common
structured boundary. Keep deterministic validators independent of PromptKit.
2. Preserve batching by `validation.max_prompt_tokens`, proposal preview, stable
proposal indexes, exact decision cardinality, and unknown/duplicate index checks.
Keep the current domain token-estimation method; do not add a second template
renderer or treat this budget as a provider output-token limit.
3. Apply the failure mapping: exhausted repairs reject/warn the batch; schema-valid
bad cardinality or negative assessments reject without semantic retry. Provider,
malformed HTTP, deadline, and admission failures propagate as runtime errors.
Update any old tests treating malformed provider envelopes as warning-only.
4. Remove redundant validator client/model configuration types and keep execution
class distinctions. Update validator reference/internal documentation.
**Status:** Complete (2026-09-24). **Depends on:** Stage 3. Focused offline tests **Acceptance and validation:** preserve existing batching no-drop/no-duplicate,
cover ConfigKit overlay errors, backend validation, credential replacement and domain reason, glossary protection, and cardinality tests. Cover all five prompt
null handling, provenance, inspection, and redaction. Nested `go test -race ./...` mappings, positive repair before domain validation, exhausted repair, and runtime
and `go vet ./...` passed on linux/amd64 with Go 1.26.8. failure propagation through a representative validator. Run validator tests;
full runner behavior is Stage 8.
Extend focused tests in the prototype; keep them offline and independent of host ## Stage 8: Connect the Runner and Remove the Old Scheduling Hierarchy
configuration and credentials. Exercise the consumer's assembled loader/runtime
rather than duplicating ConfigKit's exhaustive parser or merge tests.
Required evidence: **Status:** Pending. **Depends on:** Stages 2 and 4–7.
**Owns:** CLI process assembly, `internal/framework/runner`, old scheduler contracts,
and runner/CLI integration tests.
- Missing and invalid overlay selections fail with ConfigKit classifications. 1. Pass the shared engine's two role clients into the runner. Remove global and
Representative attempts to replace the schema version or introduce ConfigKit role schedulers, scheduler interfaces/fields, legacy concurrency configuration,
directives fail. Do not create application fallbacks for those failures. and measured scheduler wrappers. PromptKit backend admission is the only LLM
- An overlay can change an ordinary non-profile setting, such as a backend limit, service-capacity mechanism. Do not remove unrelated deterministic work or alter
and the effective setting still undergoes application validation. This proves module sequencing merely because the scheduler disappears.
overlays have not been accidentally restricted to profile selection. 2. Preserve concurrent section work, deterministic collection/indexing, ordered
- A replaced credential reference resolves only its surviving replacement; an module application, validator execution classes, and cancellation of remaining
unresolved displaced reference does not make the load fail. A surviving missing work on runtime failure. Capacity rejection fails the run as selected; do not
or empty secret reference still fails. Explicit null retains unauthenticated silently retry or add another queue to conceal it.
backend semantics. Use synthetic environment names and values under test control. 3. Wire final loaded domain settings, role clients, and diagnostics context into
- Switching role profiles alone does not remove other credential references still normal processing. No old-runtime branch or interim gate remains. Keep existing
present in configuration. Document this distinction and test the relevant case; transcript encoding, output channels, partial domain results, and artifact
do not bypass ConfigKit validation for unselected backends as an incidental fix. retention behavior. Ensure terminal failures cannot emit a purportedly complete
- `WriteEffective` and relevant explanation/metadata APIs identify overlay origin; corrected transcript.
a CLI replacement identifies CLI origin. Inspect selected PromptKit profiles from 4. Update architecture policy for shared library dependencies and engine-local
that same loaded result and verify model/backend agreement with runtime selection. backend admission; preserve the existing ownership, determinism, and redaction
- Synthetic credential values do not appear in effective configuration, explanations, invariants. Update pipeline/runtime internal guidance.
profile inspection, observations, or surfaced errors exercised by these scenarios.
Confirm the controlled transport receives the effective synthetic credential.
Maintain the existing retry/repair/capacity tests. Add only the overlay-specific **Acceptance and validation:** controlled concurrent requests from both roles to
integration coverage needed to protect this boundary; do not repeat the complete one backend never exceed its configured cap, including retries and repairs;
scheduler matrix for each fixture. separate backend identities have independent capacity. Queued cancellation,
deadline and zero-wait-capacity rejection do not leak capacity; later work can
succeed. Existing deterministic runner/domain tests pass after deleting scheduler-
implementation assertions where the affected packages compile. Exercise a real
CLI-to-local-HTTP workflow with both roles when reporting dependencies permit;
otherwise record that check for Stage 12. Do not restore old utilization fields
or add placeholders merely to make this intermediate commit build.
Verification: ## Stage 9: Collect Execution Observations Safely
```sh **Status:** Pending. **Depends on:** Stage 8.
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOSUMDB=off go test -race ./... **Owns:** Audita-owned execution record types and collector in the LLM boundary,
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOSUMDB=off go vet ./... runner attribution, and concurrency/observation tests.
```
Acceptance: all cases above have passing behavioral evidence, the existing suite 1. Translate PromptKit events into small Audita-owned records at the observer
remains green, and concurrent observation collection remains race-free. Record boundary. Use the prototype's short mutex-protected collection pattern; callbacks
actual gaps instead of silently relaxing tests or claiming complete integration. do no filesystem I/O, provider calls, or waiting on their own run. Freeze/copy
snapshots after active work has joined.
2. Bind observations to logical call, role (`proposal` or `validation`), module
instance, section, validator and batch where applicable, then retain library
run/backend/profile and attempt/generation identity. Keep provider attempts distinct from logical calls.
3. Record dispatch, queue/generation/retry/wall durations, outcome/failure category,
HTTP status, retry exhaustion, attempts/retries/repairs, and usage value/known/
complete per field. Use terminal events once for provider totals; never add
attempt usage on top. `PrepareExecution` emits no events and `RunPrepared` timing
excludes preparation. Audita's surrounding call record therefore owns the logical
count and wall time, including preparation failures; do not invent library events.
A failure before dispatch has zero attempts and known-zero complete usage.
Preserve completed observations on nil-result failures and mark usage incomplete
when dispatched attempts lack measurements. Across calls, sum known values,
set known if any contributing dispatched usage is known, and set complete only
when all contributing dispatched usage is complete, independently per field.
An aggregate with no dispatched attempts is known-zero and complete.
4. Keep non-LLM runner wall/validator timing. Remove instrumentation that counts
scheduler permits as calls. Do not infer queue peaks, exact active concurrency,
billing totals, or timing fields the released event API does not provide.
## Stage 5: Record Overlay Findings and Executable Instructions **Acceptance and validation:** adapter/collector tests prove known zero differs
from unknown, partial usage survives failed repairs, undispatched admissions are
not HTTP attempts, terminal totals are not doubled, and parallel callbacks retain
attribution safely. Run targeted race tests for the affected LLM/runner packages.
No raw provider messages, URLs, keys, prompts, or payloads enter observation records.
**Status:** Complete (2026-09-24). **Depends on:** Stage 4. Dated v0.4.0 evidence, ## Stage 10: Update Reports and Diagnostic Artifacts
test coverage, commands, limitations, README guidance, and roadmap links updated;
affected links and `git diff --check` verified.
This stage changes documentation, not production code. Read the new tests/results, **Status:** Pending. **Depends on:** Stage 9.
then update the existing owners rather than creating another parallel design: **Owns:** `internal/core/reporting`, `internal/core/diagnostics`,
`internal/framework/processreport`, runtime diagnostic writers, and CLI artifact wiring.
- Append a dated v0.4.0/overlay result section to `library-boundary-findings.md`. 1. Bump the process report version to `v2`; keep the existing transcript/ledger
Record commands, toolchain/platform, outcomes, test-to-requirement coverage, structures. Add `llm_summary` with `logical_calls`, `provider_attempts`,
limitations, and any unresolved gaps. Keep original v0.3.1 evidence identifiable. `transient_retries`, `structural_repairs`, and `usage`. A logical call counts
- Update the prototype README to match the actual executable and fixtures; check once even when it fails before dispatch. Usage has `prompt`, `completion`,
every documented command. Explain the separate ConfigKit overlay and PromptKit `total`, `cached`, and `cache_write`, each `{value, known, complete}`.
profile directories and the credential behavior proved in Stage 4. Omit the whole summary only when execution observations are unavailable; a
- Record the overlay implementation as verified in `library-boundary.md` and link successfully initialized run with no LLM calls has known-zero complete totals.
the new evidence from `library-adoption.md`. The next unit still has its Stage 6 2. Replace obsolete utilization concurrency/pool fields with `backends`, keyed by
planning deliverable outstanding; keep production migration pending. backend ID, containing configured `concurrency`/`queue_capacity` and the same
execution summary plus `capacity_wait_ms`, `generation_ms`, `retry_delay_ms`,
and `logical_wall_ms`. Sum library terminal timings once per call for capacity,
generation, and retry time; use Audita's whole-call elapsed time for logical wall
time. Attribute pre-dispatch failures to the already selected role backend.
Durations can overlap and must not be presented as a decomposition of elapsed
process time. Preserve domain module/validator timing and run wall time.
Existing role/module call counts now count logical calls; remove their obsolete
scheduler-wait and transport-execution fields rather than assign misleading new
meanings. Remove estimated active-concurrency fields throughout utilization.
3. Write `llm-execution.json` as `{calls: [...], events: [...]}` in the run directory.
Calls contain Audita logical identity, attribution, elapsed milliseconds, final
classification, summary, and available prompt/profile/model metadata. Events
contain the safe Stage 9 records, including attempt and terminal records, joined
by call ID. A preparation failure has a call record and no fabricated events.
Use millisecond integer durations in serialized records; absent attempt/section/
batch identity is omitted, not encoded as a misleading zero. Add optional
`llm_execution_path` and `selected_profiles_path` to diagnostic path metadata.
Write `selected-profiles.yaml` from the safe view and use ConfigKit YAML for
the effective snapshot (`effective-config.redacted.yaml`). Keep invocation
overlay selection and recorded origins in safe invocation/config diagnostics.
Adjust centralized filename constants and readers together; do not emit old and
new snapshots in parallel.
4. Preserve redacted request/response/error diagnostic content from prepared
details and available final results, including prompt definition/rendered hashes
and selected profile/model. Do not promise legacy hash equality or complete
provider payloads for every retry. Use the separate observation artifact for retry/repair
attempts. Redact all resolved backend secrets, including unselected ones, at
every serialization/error boundary; observation records remain payload-free.
5. Update output-file, operations, and diagnostics/reporting contracts. Preserve
failure report/ledger behavior and the distinction between primary execution
failures and secondary diagnostic write errors. Do not change retention policy.
Acceptance: the README commands are reproducible, the new overlay evidence is **Acceptance and validation:** report/diagnostic tests check v2 mapping, summary
clearly separated from historical results, and each Stage 2–4 requirement maps semantics, per-backend grouping, safe serialization, nil-result partial failures,
to a passing test or observed demonstration result. Verify affected links and and existing deterministic ledger order. A CLI failure fixture echoes a synthetic
`git diff --check`. Do not repeat unchanged code checks solely for this commit. secret in a provider error; stdout/stderr/report and retained artifacts must not
leak it. Inspect known-zero and unavailable summaries structurally, not by complete
JSON golden dumps. Reuse Stage 9 evidence rather than retest every event sequence.
## Stage 6: Prepare the Production Migration Plan and Close the Unit ## Stage 11: Finish Examples, Migration Guidance, and Removal
**Status:** Complete (2026-09-24). **Depends on:** Stage 5. The ordered ConfigKit **Status:** Pending. **Depends on:** Stages 1–10.
production handoff records bounded packages, runtime integration, tests, **Owns:** maintained examples, affected canonical documentation, obsolete code/tests,
dependencies, and acceptance criteria. Prototype evidence and scope audit passed; prototype removal, and final dependency cleanup.
links and `git diff --check` verified.
Expand `configkit.md` into a concrete, ordered production migration plan using 1. Replace maintained config examples with schema 2, complete PromptKit profile
the existing inventory and verified overlay evidence. Follow the development files, and `overlays/testing.yml` selecting both roles. Use one shared backend
reading guide for configuration/CLI and LLM runtime work; use graph-first discovery where roles share a service; provide secret-reference and unauthenticated local
to verify affected production components and their tests. This is a planning stage: examples without real secrets. Explain that operators choose real models and
do not implement the production migration or overwrite this execution plan with testing cost/quality is not guaranteed. Validate examples offline with controlled
new production stages. Keep future work packages explicitly pending. synthetic environment values; no service is required for profile inspection.
2. Write `docs/migrations/v2.md` as the one migration guide. Cover every old file/
flag/environment item in the inventory, durations, explicit empties/null,
credentials, profiles/overlays, inspection views, finite capacity, retry/repair/
deadline semantics, and changed report/diagnostic shapes. Do not add conversion
tooling or compatibility aliases. Link current contracts for complete references.
3. Finish README prerequisites/examples and canonical CLI/config, integration,
internal component, and architecture references. Audit build/CI toolchain images
for Go 1.25.5 or newer; keep only required Linux amd64/arm64 targets and best-effort
macOS arm64. Do not rewrite immutable historical release notes.
4. Remove `examples/library-adoption` once its unique useful scenarios live in
production tests. Remove the nested dependency on old Audita and duplicate
assets/fixtures. Update historical roadmap links to recorded evidence or Git
history rather than leaving broken links to deleted files.
5. Remove superseded loaders, client, renderers, schedulers, aliases, and tests
asserting retired behavior. Tidy/verify the root dependency graph and confirm
no local replacements. Keep yaml.v3 if glossary parsing still uses it. Mark the
retry-count follow-up in `runtime-behavior.md` resolved without including its
unrelated retention/timestamp proposals.
The production handoff must specify bounded work packages, dependencies, affected **Acceptance and validation:** maintained examples validate and all guide links
components, tests, and acceptance criteria for: resolve. Search for old CLI/env inputs, loader/client/scheduler names and prototype
references; remaining mentions must be intentional migration/history material.
Review deleted tests for lost domain/concurrency/redaction protection. Run affected
example/config/subprocess checks and `git diff --check`; final full-suite execution
belongs to Stage 12.
- The full version-2 configuration definition: existing domain fields, defaults, ## Stage 12: Verify the Combined Application and Close the Refactor
presence/null/zero behavior, role inheritance, overlays, secrets, and validation.
- File discovery and invocation parsing; `--overlay` behavior shared by process,
config validation, and effective inspection; explicit-setting override mapping;
removal of legacy environment overlays, aliases, and direct-key flags.
- Profile loading/inspection and configuration snapshots, including relative-path
bases, secret redaction, provenance, and clear separation of ConfigKit settings
from PromptKit execution controls.
- Direct integration of the final configuration with PromptKit execution. Specify
the final behavior and tests for each setting; do not preserve the old runtime
through temporary adapters or add interim processing gates.
- Removal of superseded configuration machinery, retained domain checks, maintained
examples, canonical documentation updates, and the combined migration guide.
- Removal of superseded LLM machinery and release follow-up. Preserve
the agreed ordering and defer release compliance/publication to the final unit.
Resolve final integration decisions against the evidence. Record genuine open **Status:** Pending. **Depends on:** Stages 1–11.
questions without creating scaffolding to support intermediate commits. **Owns:** final integration tests, final defect fixes within this scope, and status/evidence.
Final audit: map every acceptance criterion in the prototype extension to actual 1. Complete representative production subprocess workflows against controlled HTTP:
files and command results. Verify new relative links/anchors, run `git diff --check`, base selection, testing overlay, explicit profile override, and a retained
and review that no root runtime/dependency or upstream files changed. Reuse passing domain override. Observe actual outbound models for both roles, an accepted
checks for unchanged code; rerun the affected tests if this stage changes code or correction after repair, a warning-only domain/repair rejection, and a runtime
fixtures to correct a finding. A root build/test run is not a substitute for the failure with safe partial diagnostics. Cover an empty module pipeline and a
nested-module checks, and this documentation stage adds no release gate. repeated-module sequence without duplicating all adapter scenarios.
2. Run root `go test ./...`, `go build -o ./bin/audita ./cmd/audita`, and focused race
checks for the final concurrent adapter/runner/collector changes. Verify both
Linux targets compile using `bash scripts/build-release.sh v2.0.0 <temp-dir>`;
this names local build outputs and creates no tag or publication. No native Mac, paid-provider, or CI-completion gate.
If the supported minimum toolchain was not exercised earlier, verify the root
suite with installed Go 1.25.5; do not silently claim this from a newer compiler.
3. Fix actual failures and rerun affected checks; repeat the full suite when those
fixes materially change assembled behavior. Do not rerun unchanged tests solely
to mark multiple planning steps complete. Verify maintained documentation links,
examples, and whitespace once for the final content.
4. Record concise final evidence and known limits here, mark the production feature
roadmaps implemented, and commit the completed refactor. Both libraries must
own their selected responsibilities, processing must work, and no temporary
or legacy compatibility machinery may remain.
5. Hand off release preparation separately: proposed Audita v2.0.0, one migration
guide, actual pinned production dependency/license inventory, and the remaining
LGPL notice/source/rebuild-distribution review. Do not claim license compliance
from prototype tests or publish a release. The later release unit uses the
existing minimal automated procedure and ends on a successful source/tag push,
without waiting for CI.
Mark this prototype-extension/planning unit complete in `library-boundary.md` only **Acceptance:** the final production application and required Linux builds pass;
after the audit passes, and link the production handoff from `library-adoption.md`. configuration selection, both LLM roles, domain outcomes, cancellation/capacity,
Production implementation remains pending. secret handling, reports and retained artifacts match the selected contract.
Recorded checks describe what actually ran. The workspace is committed and clean.
Release-specific compliance and publication are explicitly still pending.
Acceptance: a reviewer can reproduce base/testing/override selection offline, ## Open Questions
trace each new claim to evidence, and hand the production ConfigKit work to a coding
agent in bounded units. Commit the completed documentation, report remaining None block this implementation plan. Configuration, inspection, runtime boundaries,
limitations and working-tree state, and stop at the end of this plan. report representation, and stage ownership are selected above. Release distribution
compliance remains outside this implementation sprint and must be resolved against
the final production dependency graph before publishing binaries; it is not a
reason to build compatibility scaffolding or delay the refactor.

View File

@@ -2,179 +2,128 @@
**Last updated:** 2026-09-24 **Last updated:** 2026-09-24
**Status:** Shared boundary and offline prototype completed; production migrations **Status:** Final target selected; production implementation pending. The shared
remain planned. The [v0.4.0 overlay evidence](library-boundary-findings.md#selected-overlay-extension-2026-09-24) boundary and offline prototype, including ConfigKit v0.4.0 overlays, are verified.
extends the completed prototype, and the [ordered ConfigKit handoff](configkit.md#ordered-production-migration-handoff) [Implementation stages](implementation.md) are owned by the implementation plan.
specifies the next implementation work. This document authorizes no release This roadmap defines the end state and user intent; it authorizes no publication.
publication and sets no schedule.
## Goal and Ownership ## Goal and Ownership
Adopt ConfigKit and PromptKit to align Audita with the application suite's shared Adopt ConfigKit and PromptKit to align Audita with the application suite's shared
configuration and LLM execution conventions. Remove overlapping application configuration and LLM execution conventions. Consistency is the primary benefit;
infrastructure while keeping transcript-correction responsibilities in Audita. removing overlapping application infrastructure is a secondary benefit. Preserve
Audita's transcript-correction capabilities and domain protections.
| Owner | Intended responsibilities | | Owner | Responsibilities |
| --- | --- | | --- | --- |
| ConfigKit | Application configuration loading, presence/default semantics, explicit overrides, recorded resolution, secret references, and safe configuration inspection | | ConfigKit | Strict application configuration loading, presence/default semantics, explicit overrides, recorded resolution, secret references, and safe inspection |
| PromptKit | Prompt/profile formats and rendering, provider calls, structural output validation, and shared execution capabilities supplied by the selected release | | PromptKit | Prompt/profile formats and rendering, provider calls, structural output validation and repair, transient retries, engine-local backend admission, and execution observations |
| Audita | Configuration field meanings, CLI parsing and file selection, transcript normalization/chunking, module ordering, domain validators, correction application, reports, and diagnostic artifact lifecycle | | Audita | Setting meanings, file/CLI selection, role/profile association, normalization/chunking, ordered modules, domain validation, correction application, reports, diagnostic redaction, and artifact lifecycle |
This roadmap owns sequencing across the two adoptions. The The [configuration roadmap](configkit.md) owns configuration and inspection policy.
[ConfigKit roadmap](configkit.md) owns the configuration migration's proposed scope. The [boundary findings](library-boundary-findings.md) own the selected defaults,
PromptKit v0.11.0 is the reviewed baseline for the adoption prototype. It supplies full migration inventory, failure mapping, and prototype evidence. The selected
transient HTTP retries, execution observations, typed failure categories, an offline production versions are ConfigKit v0.4.0 and PromptKit v0.11.0, with Go 1.25.5 as
proposal/assessment example, and LGPL-3.0-or-later licensing. The earlier upstream the minimum. Use published modules, never sibling checkouts or local replacements.
consumer requests are no longer outstanding prerequisites on those points. The existing YAML dependency remains while glossary parsing needs it.
Audita has not yet adopted either library; recheck the selected releases during
detailed planning.
## Migration Posture ## Migration Posture
Treat adoption as an intentional breaking transition. Preserve intended domain This is an intentional breaking transition targeting Audita v2.0.0. No backward
capabilities, not legacy configuration files, flags, environment variables, or compatibility is required for existing installations, config files, CLI flags,
configuration-inspection formats. Supply one migration guide rather than parallel environment overlays, or inspection output. Provide one migration guide, without
legacy loaders, compatibility aliases, or permanent translation layers. legacy loaders, aliases, dual modes, or automatic conversion tooling.
Plan to ship both adoptions together as Audita v2.0.0, without an intermediate Both libraries belong to one refactor. Intermediate commits need not build or run;
public release. Implementation should still consist of coherent, reviewable units. there is no supported intermediate product state. No temporary adapters, compatibility
Revisit the final release scope and sequencing alongside other large planned shims, or processing gates are needed. The complete application must work and pass
changes, especially changes to module or pipeline configuration. its checks before the refactor is finished. Release compliance and publication
come last, with no intermediate public release.
## Proposed Sequence ## Configuration and Execution Target
The [shared-boundary and prototype roadmap](library-boundary.md) records the completed ConfigKit selected overlays enable a testing configuration that chooses cheaper or
unit covering stages 1 and 2 below. The [findings](library-boundary-findings.md) faster models. The example selects both proposal and validation profiles. ConfigKit
select its target contract and identify the next ConfigKit implementation work. application overlays and PromptKit execution profiles remain separate concepts.
Complete operator-maintained profiles own model/generation controls; embedded
application prompts and schemas are not operator-selected files. No catalog wiring,
profile-inheritance feature, ConfigKit imports, or secret-directory bootstrap is
part of this migration.
### 1. Design the Shared Configuration Boundary Capacity controls serve two needs: respecting a local service's hard simultaneous-
query limit and bounding concurrent paid API activity. One finite limit per backend
is shared across both roles and every initial, retry, and repair attempt. Limits
are engine-local, not coordinated across processes. No additional global or
per-role limiter or fine-grained scheduling policy is required. Profiles must use
registered backends and ConfigKit-resolved credentials; endpoint-only bypass and
competing profile credential lookup are excluded.
Define the target settings model for both libraries before implementing either Structural repair is enabled by default for both roles because additional attempts
migration. Decide how Audita selects proposal and validation execution profiles, to obtain valid output are desired. Transient retries and structural repairs have
supplies credentials, selects transient-retry and structural-repair budgets, and separate budgets; explicit zero disables the respective behavior. Each generation,
assigns per-backend concurrency limits. Give each setting one authoritative home; including a repair, receives its own transient-retry budget. The Audita logical-call
avoid duplicate model, endpoint, credential, or timeout definitions with competing deadline covers preparation, waiting, backoff, all generations, and repairs; profile
precedence. HTTP timeouts apply per attempt. Exact defaults and bounds are in the
[selected contract](library-boundary-findings.md#configuration-and-profile-contract).
No Audita outer retry loop restores the old broader retry policy.
Keep ConfigKit application configuration distinct from PromptKit execution profiles. Audita retains all four correction modules, five LLM-validator prompts,
ConfigKit v0.4.0 calls its optional configuration layers overlays. Enable selected deterministic validators, confidence/applicability checks, glossary protection,
overlays for testing as specified in the [ConfigKit roadmap](configkit.md#selected-overlays-for-testing); validation batching/cardinality, deterministic proposal and ledger ordering,
these select application settings, including PromptKit execution profiles. transcript encoders, and artifact retention. PromptKit-valid JSON still passes
Select an explicit profile-ownership model; complete Audita-owned execution profiles are the initial recommendation, with Audita's domain checks. Repair exhaustion, provider failure, and domain rejection
catalog-driven selection a deliberate alternative. follow the [selected failure table](library-boundary-findings.md#execution-and-failure-mapping).
Retention and transcript timestamp changes remain separate work; this migration
resolves the [retry-count follow-up](runtime-behavior.md).
Capacity needs are simple: protect local backends from exceeding their available ## Reporting and Diagnostic Target
simultaneous-query capacity and give paid API backends a reasonable finite cap.
Proposal and validation calls using the same backend share its limit. Separate
proposal/validation budgets and additional fine-grained Audita scheduling controls
are not required. Account for the engine-local scope of PromptKit's existing limits.
### 2. Prove the PromptKit Boundary Keep stable Audita-owned serialized records; do not expose raw library structs.
Use process report schema `v2` with the existing domain outcomes and a new
`llm_summary` separating logical calls, dispatched provider attempts, transient
retries, structural repairs, and usage. Transcript schema versions remain unchanged.
The implementation plan specifies the concrete new fields and artifact wiring.
Before finalizing the new application configuration schema, exercise one proposal Usage distinguishes known zero, unknown, and incomplete totals. Preserve observations
call and one LLM-validator call through PromptKit using controlled offline responses. from failed calls even when no final result exists. Count terminal totals once;
Verify profile selection, credentials, embedded prompts and schemas, shared backend attempt observations provide diagnosis without being added to those totals again.
capacity, failed output validation, cancellation, and diagnostic mapping. This is operational evidence, not a complete billing ledger.
Enable structural repair from the outset for proposal and LLM-validator outputs. Replace global/role scheduler utilization with per-backend configured capacity and
Additional attempts to obtain valid output are desired behavior, not a deferred observed queue/generation/retry/logical-call timing. Do not fabricate measurements
optimization. Select explicit positive repair budgets during detailed planning and that the event API does not provide. Keep domain timing and deterministic correction
exercise invalid output that becomes valid after repair, as well as exhausted ledger ordering. Record prompt definition/rendered hashes and selected profile/model
repair budgets. without promising the old hash values. Callbacks are short, concurrency-safe, and
free of blocking I/O or waits on their own execution.
Keep transient HTTP retries and structural repair distinct. With the reviewed Effective configuration and selected-profile snapshots remain distinct safe views.
library, each generation, including a repair, has its own transient-retry budget; Secret values must not escape through inspection, logs, reports, runtime errors,
plan and document the combined request allowance and overall deadline. Adopt or retained diagnostic files. Observation records exclude payloads, URLs, raw
PromptKit's explicit zero-disables-retries semantics instead of Audita's legacy provider messages, prompts, and credentials. Existing artifact lifecycle and
zero-as-default behavior. The library's narrower transient retry policy does not partial-failure handling remain Audita responsibilities.
retry transport failures or malformed provider responses; do not restore the old
policy through an overlapping application retry loop.
Verify that initial, retry, and repair calls share backend limits and that their ## Completion Criteria and Release Boundary
observations reach Audita without double-counting usage. Audita's domain acceptance
checks remain necessary after structural repair succeeds; valid JSON does not imply
an acceptable correction or validator decision.
Use the prototype to resolve genuine boundary gaps and inform configuration design. - Both libraries own their selected responsibilities in the production application.
It is evidence for the migration, not a permanent alternate runtime. Avoid finalizing Superseded loaders, transport, renderers, schedulers, aliases, and the isolated
ConfigKit-facing settings around the old LLM runtime only to redesign them later. prototype are gone once its useful evidence is represented in production tests.
- Offline tests prove configuration-to-execution routing for both roles, selected
overlays and explicit overrides, shared backend capacity, retry/repair outcomes,
cancellation/deadlines, domain decisions, redaction, and truthful partial reports.
Tests exercise real internal collaborators and controlled HTTP boundaries.
- Maintained examples and canonical docs describe the implemented final model.
One migration guide covers files, flags, environment inputs, profiles, credentials,
overlays, inspection, and changed runtime/report/diagnostic semantics.
- The assembled application passes its required checks; Linux amd64 and arm64 build.
macOS arm64 remains best effort. No live-provider or CI-success gate is added.
### 3. Complete ConfigKit Adoption PromptKit v0.11.0's reviewed retry, observation, and capacity APIs remove the earlier
upstream feature prerequisites; no upstream edit is required for adoption. The
prototype proved representative behavior, not release compliance or model quality.
Implement the agreed application configuration contract following the The selected PromptKit release uses LGPL-3.0-or-later. A later release unit must
[ConfigKit roadmap](configkit.md). Retire legacy environment overlays and obsolete review the actual production dependency graph and distribution requirements,
CLI/configuration conventions, update maintained examples and canonical references, including notices and source/rebuild materials, before binary publication. This
and begin the combined migration guide. Include the selected testing-overlay path implementation plan does not claim that review is complete. Release publication
and its offline integration evidence from the ConfigKit roadmap. uses the existing [procedure](../release.md) and completes on successful source/tag
push, without waiting for CI or artifact delivery.
ConfigKit and PromptKit adoption form one refactor. These sections describe
implementation order, not separately runnable milestones. Intermediate commits
need not build or run. Build directly toward the final integration without
temporary adapters, compatibility shims, or an interim processing gate; validate
the complete application before declaring the refactor finished.
### 4. Complete PromptKit Adoption
Move prompt rendering, execution profiles, provider calls, and structural output
validation onto the selected PromptKit release. Remove overlapping transport,
prompt-loading, and scheduling machinery once the replacement is verified.
Retain Audita's transcript pipeline, semantic and deterministic validators,
correction application, reports, ledger, and artifact retention. Map PromptKit
results, errors, metadata, and execution observations into those responsibilities.
Connect the new configuration directly to this execution path and remove the
superseded configuration and runtime machinery.
### 5. Validate and Release the Combined Migration
Validate the assembled configuration-to-execution workflows, including local and
API backend limits, successful and exhausted repairs, transient retries,
failure/cancellation behavior, credential handling, diagnostics, and domain outcomes.
Use deterministic tests and controlled endpoints rather than paid calls as acceptance
gates. Update documentation and examples together with
implemented behavior and remove obsolete contracts and code.
Complete one migration guide covering files, CLI, environment settings, credentials,
execution profiles, inspection, and changed diagnostics. Release the combined
breaking transition under the established [release procedure](../release.md), with
no CI waiting added to release completion.
## Reviewed Upstream Readiness
The v0.11.0 review covered the relevant implementation, the full test suite,
targeted race tests, and the offline proposal/assessment example; those checks
passed. Existing per-backend controls cover the requested capacity model, including
retry and repair interactions. No further upstream feature request is currently a
prerequisite to the prototype. Keep the planned adoption sequence unchanged.
Remaining integration work includes selecting a total-operation deadline alongside
per-attempt timeouts, using shared backend identities with finite limits, and mapping
PromptKit observations to Audita-owned diagnostic records. Observation callbacks are
synchronous and may overlap across runs; collection must be short and safe for
concurrent use. Distinguish unknown or incomplete usage from known zero and do not treat
observations as a complete billing ledger or a stable serialized report format.
The licensing change has shipped as LGPL-3.0-or-later. Account for applicable
notices and source/rebuild distribution requirements in Audita's release design;
relicensing no longer needs to be requested upstream. Recheck released APIs,
toolchain requirements, and dependency versions at implementation time. Pin
published library releases; production builds must not depend on sibling checkouts
or local module replacements.
If the prototype reveals a genuine remaining gap, evaluate it explicitly rather
than quietly recreate shared library functionality as permanent Audita infrastructure.
## Coordination and Open Decisions
- Use the selected configuration/profile boundary, defaults, credential flow, and
retry/repair/deadline contract in the [findings](library-boundary-findings.md)
when detailing the ConfigKit production migration.
- Implement the outcome and observation mapping recorded in the findings, retaining
Audita domain validation and diagnostic redaction.
- Resolve the [runtime retry-count follow-up](runtime-behavior.md) through this migration.
Retention and transcript timestamp questions are not automatically included.
- Revisit ordering when other major changes are defined. Prefer designing shared
module/pipeline settings once, while keeping unrelated work independent.
- Flesh out task-level scope, tests, and acceptance criteria before implementation.
Completion requires the intended ownership split, removed transitional machinery,
verified assembled behavior, and one coherent migration path for users.

View File

@@ -311,51 +311,23 @@ claims of completion for the prototype.
The new profile/credential/capacity and deadline choices are explicit above; no The new profile/credential/capacity and deadline choices are explicit above; no
unresolved upstream requirement prevents starting the ConfigKit implementation unresolved upstream requirement prevents starting the ConfigKit implementation
plan. Production report field names and final runtime wiring follow the outcome plan. Production report field names and final runtime wiring are selected in the
and observation contract here. The coordinated refactor targets both libraries [implementation plan](implementation.md), following the outcome and observation
directly; intermediate commits do not require a working legacy runtime. contract here. The coordinated refactor targets both libraries directly; intermediate commits do not require a working legacy runtime.
### Component-Level Next Unit ### Production Implementation Reference
Implement ConfigKit adoption first, against this target model: The [implementation plan](implementation.md) owns ordered production work,
component changes, tests, and acceptance criteria. The [combined roadmap](library-adoption.md)
and [configuration roadmap](configkit.md) own the final target. This evidence does
not require an interim runtime bridge or separately runnable configuration migration.
| Component | Migration action | Retained domain scope includes transcript/glossary schemas, normalization/chunking,
| --- | --- | output encoding, proposals and apply-time checks, deterministic validators, ordered
| `internal/core/config` | Replace file/env merge and custom duration parsing with ConfigKit definitions, presence-aware resolution, domain validation, and explicit override paths. Retain domain defaults and validation rules identified in the inventory. | modules, LLM-validator batching/cardinality, ledger ordering, and artifact lifecycle.
| `internal/cli` | Separate invocation paths from persistent settings; select one file; parse only supplied setting flags into overrides; remove legacy LLM/role-budget/direct-key flags and all environment overlays. Assemble/inspect selected profiles for validation. | The production integration replaces loading, provider execution, prompt rendering,
| CLI config inspection and diagnostics snapshot | Use ConfigKit redacted output plus Audita-owned selected-profile summary; update command tests and diagnostic snapshot consumers. | and scheduler composition while preserving those responsibilities. Retention and
| Configuration tests/examples | Replace v1 discovery/overlay/alias expectations with v2 presence, inheritance, secret, profile, and override contracts; preserve domain validation tests. Add actual production discovery and CLI tests not exercised by this prototype. | timestamp changes remain separate work.
The PromptKit portion of the refactor replaces `internal/framework/llm` provider execution
and scheduler composition, `internal/prompts` parsing/rendering, and duplicated
structural-output validation. Keep a small Audita adapter behind the existing
structured-call boundary where that contract remains sufficient; revise its return
metadata only as needed for repair and observation outcomes. Avoid library types
in module proposals, domain validators, transcript schemas, and correction ledgers.
Retain `internal/core/schema`, normalization/chunking/output encoding,
`internal/framework/proposals`, deterministic validators, module identity/order,
LLM-validator domain batching and cardinality, runner ordering/apply logic, report
assembly, and artifact lifecycle. Update their integration wiring rather than
rewriting those algorithms. Move all nine prompt definitions and schema references,
checking each module/validator's variables and helpers; this prototype proves only
one pair. Migrate actual diagnostic redaction and error classification, then remove
the old transport/schedulers/prompt loader. No temporary runtime bridge is needed.
Documentation work during the respective implementation stages: update
`docs/config.md`, `docs/cli.md`, examples, README prerequisites, internal config/prompt/
LLM/runner guidance, and output/LLM/subprocess integration contracts. Create one v2
migration guide covering file shape, duration/empty-list changes, removed environment
and CLI inputs, backend/profile/secret setup, effective inspection, retry/repair
behavior, and changed usage/concurrency diagnostics. Revise architecture policy's
existing scheduler language when production uses PromptKit; the current policy
continues to describe the current runtime. Retention and timestamp fixes remain
separate roadmap work.
The combined release remains last, with no intermediate publication. Raise the
production Go minimum when dependencies land; review CI/build images and supported
Linux amd64/arm64 builds then. Prototype execution on Linux amd64 is not proof of
arm64 release readiness or live model output quality.
### Distribution Follow-Up ### Distribution Follow-Up
@@ -416,5 +388,5 @@ The tests use a controlled built-in HTTP transport and synthetic credentials;
there were no live provider calls. The prototype still covers one representative there were no live provider calls. The prototype still covers one representative
proposal/assessment pair, not all production modules, prompts, CLI paths, or proposal/assessment pair, not all production modules, prompts, CLI paths, or
diagnostic artifacts. Root `go test ./...` does not run this nested module. No diagnostic artifacts. Root `go test ./...` does not run this nested module. No
upstream API gap appeared in this extension. The [ordered production handoff](configkit.md#ordered-production-migration-handoff) upstream API gap appeared in this extension. The [production implementation plan](implementation.md)
keeps implementation and release pending. keeps implementation and release pending.

View File

@@ -11,9 +11,9 @@ is isolated from production. No publication or upstream changes were made.
## Next Unit: Testing Overlays and Production Migration Plan ## Next Unit: Testing Overlays and Production Migration Plan
**Status:** Complete. The [ordered production handoff](configkit.md#ordered-production-migration-handoff) **Status:** Complete. The [production implementation plan](implementation.md)
is ready for subsequent work; [implementation.md](implementation.md) records this owns subsequent work; the [overlay findings](library-boundary-findings.md#selected-overlay-extension-2026-09-24)
unit's checks. The original work packages below remain record this unit's checks. The original work packages below remain
historical scope for the completed first unit. historical scope for the completed first unit.
Extend the existing prototype to prove the [selected-overlay contract](configkit.md#selected-overlays-for-testing) Extend the existing prototype to prove the [selected-overlay contract](configkit.md#selected-overlays-for-testing)
@@ -68,7 +68,7 @@ their existing names and meaning.
Define and prove the boundary between Audita, ConfigKit, and PromptKit before Define and prove the boundary between Audita, ConfigKit, and PromptKit before
replacing Audita's production configuration or LLM runtime. This unit covers the replacing Audita's production configuration or LLM runtime. This unit covers the
shared design and representative prototype in stages 1 and 2 of the shared design and representative prototype supporting the
[coordinated adoption roadmap](library-adoption.md). The [coordinated adoption roadmap](library-adoption.md). The
[ConfigKit roadmap](configkit.md) continues to own the broader configuration scope. [ConfigKit roadmap](configkit.md) continues to own the broader configuration scope.