Files
audita/docs/roadmap/implementation.md

567 lines
37 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Implementation Plan: Adopt ConfigKit and PromptKit
**Last updated:** 2026-09-24
**Status:** Ready for implementation. All stages below are pending. This replaces
the completed testing-overlay prototype plan; its results remain in the
[boundary findings](library-boundary-findings.md#selected-overlay-extension-2026-09-24).
## Scope and Working Instructions
Implement the final configuration and execution model in the
[combined feature roadmap](library-adoption.md) and [configuration roadmap](configkit.md).
The [selected boundary](library-boundary-findings.md#configuration-and-profile-contract),
[migration inventory](library-boundary-findings.md#migration-inventory), and
[failure mapping](library-boundary-findings.md#execution-and-failure-mapping)
are the detailed behavioral inputs. Do not repeat the prototype or reopen settled
choices without a concrete contradiction in the pinned libraries.
This plan is for a gpt-5.6-terra coding agent. Execute Stage 1 through Stage 12 in
order. Each stage is one bounded implementation prompt. When assigned the whole
plan, continue between stages without a permission round. When assigned one stage,
finish that stage's scope and report the next stage. Mark completed stages with a
short result and actual validation evidence; do not call a pending integration
check passed.
Read `AGENTS.md`, [development guidance](../development.md), and all documents under
`docs/policy/`. Use graph-first code discovery. Follow the
[agent](../policy/agents.md), [architecture](../policy/architecture.md),
[testing](../policy/testing.md), [documentation](../policy/documentation.md), and
[release](../policy/release.md) policies. The intentional dependency, configuration,
and scheduler changes here supersede the corresponding old implementation choices;
update canonical descriptions as the new behavior lands.
The stages divide one refactor, not supported intermediate releases. Intermediate
commits may fail to build or run. Do not write temporary adapters, compatibility
shims, duplicate execution paths, stubs returning invented successes, or an interim
processing gate. Remove superseded code when replacing it. Run focused checks
where the affected packages are testable; record checks deferred because later
stages must reconnect callers. Such a completed stage is still work in progress
on the overall refactor. Commit coherent stages with plain-English messages under
the standing policy. Do not spend a stage making an intermediate application work.
Keep tests deterministic, offline, and credential-independent. Reuse prototype
scenarios at their production boundary rather than copy the nested application or
retest all library internals. Preserve domain tests unless the selected contract
actually changes. Update each affected canonical reference alongside its owning
stage; Stage 11 finishes the cross-document migration guide and examples.
No upstream edits, paid calls, push, tag, publication, retention-policy change,
timestamp fix, or unrelated pipeline redesign is included. Release compliance
and publication remain a later release unit. Completion here means both libraries
are adopted, obsolete machinery is removed, and the final application is verified.
## Implementation Decisions
These decisions close the remaining application-design details; background and
user-facing policy belong to the feature roadmaps.
- Pin the published modules used by the verified prototype: ConfigKit v0.4.0 and
PromptKit v0.11.0. Set the root minimum to Go 1.25.5. Keep yaml.v3 while glossary
parsing uses it. No sibling replacements or dependency on Audita's own old release.
- Keep application loading in `internal/core/config`, embedded prompt definitions
in `internal/prompts`, schemas in `internal/framework/responseschema`, and the
PromptKit integration in `internal/framework/llm`. Use one engine per process
invocation, shared by both role-bound clients; neither modules nor validators
construct engines. The CLI composes these pieces. Avoid a generic service layer.
- Retain the `CompleteStructured(ctx, request, out)` boundary, using Audita-owned
prompt ID/version, string variables, schema identity, and diagnostic context.
Replace the old rendered-message/model request fields. Model/profile/credential
selection belongs to the role-bound adapter, not individual callers. Consolidate
the equivalent validator types onto the framework contract where needed, without
an import cycle. No PromptKit types in correction algorithms or ledger records.
- Store PromptKit definitions for all nine existing prompt IDs with their existing
`v1` selection. Replace the custom `hardening` template function with an
application-supplied `HardeningPolicy` string variable sourced from the one
embedded hardening fragment. The adapter supplies that reserved variable after
caller variables. PromptKit renders all messages; do not retain an Audita template
engine or introduce generated prompt files requiring a second build step.
- Use one shared configuration-input parser for `process`, `config validate`, and
`config print-effective`. All three accept `--config`, `--overlay`, and the
retained setting overrides in the inventory, including the two profile flags.
Only `process` accepts transcript/glossary/output/report paths. Only explicitly
visited setting flags become ConfigKit overrides. Do not add retry/capacity/key
flags beyond the agreed inventory.
- `config validate` requires an explicit nonempty config path. The other two
commands use the selected discovery order. All validate the resolved settings
and both selected profiles without provider calls, including an empty module
pipeline. Help succeeds before file discovery or secret/profile loading.
- `config print-effective` adds `--view settings|profiles|explain`, defaulting to
`settings`. Settings uses ConfigKit `WriteEffective` YAML. Profiles emits a safe
Audita-owned YAML summary for the proposal and validation roles. Explain emits
a YAML sequence of field path, source, and derivation records from ConfigKit's
recorded resolution, with no resolved secret values. Unknown views are usage
errors. This avoids mixing multiple formats into the default output or adding
another command tree. Profiles includes selected ID, backend, model, temperature,
max tokens, top-p, per-attempt timeout, service tier, and reasoning effort when
present; omit arbitrary extra provider parameters and credentials. It is a
selected-controls summary, not a dump of every profile input.
- Keep CLI exit classes 0/1/2 and transcript/report stream behavior. Invalid user
settings/profiles are configuration errors (2); broken embedded assets and
runtime/provider failures are runtime errors (1). Serialization/write failures
remain runtime errors. Do not classify malformed HTTP envelopes as repairable
model output.
- Publish process reports as `audita-process-report` schema `v2`. Preserve existing
domain fields and ledger ordering, add a top-level `llm_summary`, and replace
obsolete scheduler utilization fields as specified in Stage 10. Transcript
encoder versions stay unchanged. No legacy report mode is needed.
## Stage 1: Replace the Configuration Model and Loader
**Status:** Complete (2026-09-24). Replaced the version-1 loader with the ConfigKit
schema-2 model, selected overlays, typed secret references, final role inheritance,
and retained domain validation. Pinned both libraries and Go 1.25.5, and aligned
the release build image. `go test ./internal/core/config` and `git diff --check`
passed. `go test ./...` currently fails in old CLI, validator, and diagnostic
callers of removed fields; those connections are assigned to later stages.
**Depends on:** the completed prototype only.
**Owns:** root `go.mod`/`go.sum`, `internal/core/config`, and config tests.
1. Pin both modules and the Go minimum above. Align build/CI toolchain declarations
with that minimum; preserve the existing platform set and release gates.
2. Replace version-1 YAML/env merge and duration coercion with one ConfigKit
definition covering every retained field in the migration inventory. Use the
prototype's types/lifecycle as evidence, then add the full domain configuration.
Remove legacy environment loaders and alias resolution; do not keep two loaders.
3. Implement defaults, presence-aware backend defaults, final validation-profile
inheritance, and recorded derivations. Keep `chunking.target_sections` optional,
explicit empty module lists empty, and module order/repeats intact. Validate
domain constraints after overlay and explicit override resolution. Preserve
numeric normalization seconds; operation timeout is a duration string.
4. Use typed optional secret references. Resolve every surviving reference, including
unselected backends; missing/empty references fail, replacement resolves only the
replacement, and null permits unauthenticated use. Enable selected overlays;
leave imports and secret-directory bootstrap disabled.
5. Expose the ConfigKit result to CLI composition for effective output and provenance;
convert retained settings into existing domain inputs without transporting the
library result through correction code. Update `docs/config.md`'s implemented
loader/schema description, clearly marking unfinished CLI integration if needed.
**Acceptance and validation:** focused config tests cover strict version/fields,
selected defaults, omission/null/zero, final role inheritance, empty/repeated
modules, overlay/override precedence, secret replacement/null/missing, and retained
cross-field rules. Old schema fails with migration guidance. General environment
variables have no implicit effect. Run `go test ./internal/core/config` when this
package is independently buildable; record caller compile failures for later stages.
Do not duplicate ConfigKit's exhaustive YAML/merge tests.
## Stage 2: Replace CLI Configuration Selection and Overrides
**Status:** Complete (2026-09-24). Added one parser for all three commands,
strict file discovery, optional overlay selection, and visited-only setting
overrides; removed retired flags and legacy process flag extraction. Updated CLI
and configuration references. Isolated parser/loading tests passed with
`go test internal/cli/config_input.go internal/cli/config_input_test.go`, and
`git diff --check` passed. `go test ./internal/cli` remains unbuildable because
the old runtime, validators, and diagnostics still use removed configuration
fields; assembled command checks remain deferred to later stages.
**Depends on:** Stage 1.
**Owns:** `internal/cli/run.go`, `process_flags.go`, configuration command parsing,
and corresponding CLI/subprocess tests.
1. Implement the shared configuration-input parser described above. Preserve the
existing supported positional-argument arrangement for `process`, required
glossary, output/report behavior, and root/subcommand help. Remove obsolete
LLM/model/key/timeout/retry/concurrency flags and aliases outright.
2. Select exactly one file: explicit `--config`, then
`/usr/local/etc/audita/config.yml`, then `/etc/audita/config.yml`. Missing system
candidates are skipped; selected inaccessible files and explicit empty/missing
paths fail. With no file, report a configuration error; do not select a provider.
Remove `AUDITA_CONFIG`. Test discovery using an injected path list/filesystem
seam rather than modifying the host's system configuration.
3. Pass invocation overlay selection to ConfigKit and only supplied setting flags
to its overrides. `--modules ''` must clear the pipeline; explicit numeric zero
reaches validation rather than being treated as absent. Relative profile bases
derive from the selected main file, never the overlay directory.
4. Wire the parsed result into the final process/config command entry points.
Profile-dependent command completion belongs to Stage 4; do not emulate the old
runtime to make the process command work now. Update `docs/cli.md` for the final
input contract as parsing lands.
**Acceptance and validation:** parser/loading tests prove help without usable
configuration, discovery/error cases, explicit-only overrides, retained domain
flags, removed flag rejection, and ignored legacy environment values. Table-drive
shared cases; retain representative subprocess checks rather than triplicate every
parser test across commands. Config commands reject positional arguments and
process-only flags. Defer assembled command checks until Stage 8 if imports still
prevent compilation.
## Stage 3: Migrate Embedded Prompt Assets and Call Contracts
**Status:** Complete (2026-09-24). All nine embedded prompts are PromptKit
definitions with the shared hardening variable, existing `v1` IDs, and canonical
JSON Schema associations. Replaced the template renderer and its hash metadata;
the structured-call request now carries prompt, schema, variables, and attribution
through one framework contract. `go test ./internal/prompts
./internal/framework/responseschema ./internal/framework/structuredoutput` and
`git diff --check` passed. `go test ./...` still fails at old module, validator,
runtime, and diagnostic callers awaiting their assigned migrations.
**Depends on:** Stage 1; execute after Stage 2.
**Owns:** `internal/prompts`, `internal/framework/responseschema`,
`internal/framework/contracts`, validator contract aliases, and prompt tests.
1. Convert all four module and five validator prompts listed in
[the prompt registry](../internal/prompts.md) to PromptKit definitions. Preserve
instructions, variables, glossary/description context, prompt IDs, and the two
existing JSON response schemas. Embed assets once and expose their filesystem
to the runtime. Do not copy the prototype's duplicate schema directory.
2. Keep output contracts JSON Schema based, but supply the full effective output
contract and repair count on each runtime request. Do not create separate
operator-facing repair defaults per prompt. Use the hardening variable decision
above to retain a single hardening source.
3. Change the structured-call request to prompt identity, variables, response schema
identity, and diagnostic context. Keep decoded output and typed Audita metadata
as the return boundary. Include role/module instance/section/batch/validator
context where available for attribution; do not derive identities from messages.
4. Replace the old parser/renderer and hash assumptions. Keep small metadata and
schema lookup helpers where Audita needs them. Update the prompt reference.
**Acceptance and validation:** a table covering all nine definitions prepares
representative variable sets through PromptKit with controlled profile fixtures;
checks include hardening, section/glossary/description data, missing variables,
correct schema association, and rendered content preserving the previous task
instructions. Use reviewed expectations, not a retained legacy renderer or old
Audita module dependency. Existing schema/domain tests remain meaningful. Callers
are deliberately migrated in Stages 6–7.
## Stage 4: Assemble Profiles and Finish Offline Inspection
**Status:** Complete (2026-09-24). Added one engine and profile inspection path
for both roles, requiring registered matching backends and backend-owned
credentials. Config commands and process select that path; settings, safe selected
profiles, and provenance are exposed as separate YAML views with secret redaction.
Focused offline checks passed with `go test internal/framework/llm/runtime.go
internal/framework/llm/safe_view.go internal/framework/llm/runtime_inspection_test.go`
and `go test internal/cli/config_input.go internal/cli/config_input_test.go`;
`git diff --check` passed. Assembled command tests remain deferred until the
old caller packages are reconnected in later stages.
**Depends on:** Stages 1–3.
**Owns:** engine/profile assembly in `internal/framework/llm`, CLI config commands,
and inspection tests.
1. Construct one PromptKit engine with the embedded assets, complete profile files,
and registered application backends. Reuse the prototype's supported filesystem,
backend, retry-config, and inspection APIs. Accept an HTTP client at the external
boundary for tests; production uses the library transport.
2. Resolve `profiles_dir` against the main config file. Inspect both selected roles,
require a registered backend, reject endpoint-only bypass and profile credential
lookup, and compare endpoints after trimming surrounding whitespace and trailing
slashes, as in the prototype. Do not invent profile inheritance/catalog wiring
or reject unrelated unselected profile content beyond library behavior.
3. Complete `config validate` and the three effective-output views. Settings,
profiles, and explanations share the same load/assembly path used by process.
Keep the `config is valid` success message. Redact configured secret values in
any surfaced library error; do not stringify raw configuration/profile objects.
4. Add the internal safe-view helpers later used for snapshots. Document the view
interface and its limits in `docs/cli.md` and `docs/config.md`.
**Acceptance and validation:** offline command tests prove all views and validation
agree for base/testing/explicit-profile selections; main-file-relative and absolute
profile paths work; missing profiles, backend mismatch, competing credentials,
missing surviving secrets, and invalid views fail with the selected exit class.
Use a transport that fails the test if called. Inspect outputs/errors for synthetic
secret absence and overlay/CLI provenance, including inherited validation selection.
## Stage 5: Implement the Final PromptKit Structured-Call Adapter
**Status:** Complete (2026-09-24). Added role-bound prepared execution with
target verification, validated-only decoding, typed safe failures, protected
prepared/final-content snapshots, and logical-call and execution-event hooks.
Removed the old direct HTTP client and role resolver; updated LLM integration
documentation. `go test ./internal/framework/llm` and
`go test ./internal/framework/structuredoutput` passed, including controlled
HTTP repair/retry, target-drift, transport/protocol, and cancellation cases;
`git diff --check` passed. Caller integration remains for Stages 6–8.
**Depends on:** Stages 3–4.
**Owns:** `internal/framework/llm`, the typed failure boundary in
`internal/framework/structuredoutput`, and adapter tests.
1. Implement role-bound clients over the shared engine. Start the operation context
and Audita logical-call record before preparation; inherit earlier caller deadlines.
Select the role profile and backend secret, supply reserved hardening variables,
and pass an explicit full schema/repair contract. Use `PrepareExecution`, its
`Details`, then `RunPrepared` so diagnostic messages/hashes describe the actual
execution snapshot. Defer handle discard; do not call `Prepare` and then `Run`
with an independently reloaded profile. Before dispatch, verify the prepared
target still matches the role's registered backend/endpoint and has no competing
credential lookup. A changed target fails safely rather than sending a backend
key to a different target. This is the final diagnostic/security boundary, not
an intermediate adapter.
2. Check both the returned error and `result.Validation.IsValid`: exhausted repair
can return a result with failed validation and no error. Decode only validated
JSON into the requested Audita output. Use the selected failure table:
exhausted output repair becomes the Audita malformed-output classification; provider/transport/protocol/retry/admission/deadline failures
remain runtime failures. Preserve typed causes for classification and safe
messages for users. Explicit zero retries must not become retries-exhausted.
3. Preserve prepared messages/hashes and available final content and profile/model
metadata through Audita-owned types. Provide a per-call observer hook and logical
start/finish record bound to Audita diagnostic context for Stage 9; do not
create another retry loop, scheduler, provider envelope parser, or schema engine.
4. Remove the old OpenAI-compatible client and its role config resolution. Retain
only applicable redaction and diagnostics helpers. Update the LLM integration
reference to delegate wire/retry mechanics to the pinned library.
**Acceptance and validation:** controlled HTTP tests cover successful decode,
positive repair and exhaustion for both schema types, 429/5xx retry recovery,
explicit zero, exhausted positive budget, transport/malformed-envelope failure,
combined retry-plus-repair bounds, preparation failure without dispatch, exact
prepared request metadata, target-change rejection, and operation cancellation
during a long retry wait. Assert classifications and meaningful counts, not timing-sensitive sleeps
or complete error strings. Reuse the prototype sequences; no need to reproduce
PromptKit's entire status-code matrix.
## Stage 6: Migrate All Proposal Callers
**Status:** Pending. **Depends on:** Stage 5.
**Owns:** `internal/modules`, `internal/framework/proposal_generation`, and their tests.
1. Switch glossary, homophones, spoken_word, and grammar proposal construction to
prompt IDs and variables. Preserve section/glossary encoding, context trimming,
replacement policy, confidence filtering, and applicability checks.
2. Feed repaired structured output into the existing domain path. Exhausted
structural repair produces the existing section warning and no proposals;
runtime failures propagate. Schema-valid but inapplicable proposals do not
trigger semantic retries or additional repairs.
3. Update fake clients and module prompt tests to the final request contract.
Delete duplicate message rendering and obsolete transport/schema checks; retain
domain protection and prompt variable coverage. Update module internals docs.
**Acceptance and validation:** all four modules have an exercised variable/schema
mapping; existing domain suites retain their assertions. Representative controlled
adapter coverage proves repaired proposals reach domain checks and that a low-
confidence/inapplicable proposal remains rejected. Preserve section/proposal
ordering. Run the module and proposal-generation suites as imports become buildable.
## Stage 7: Migrate LLM Validators Without Changing Domain Decisions
**Status:** Pending. **Depends on:** Stage 6.
**Owns:** `internal/framework/validators`, validation prompt builders/batching,
and validator tests.
1. Migrate spoken_form_plausibility, meaning_reversal_review, editorial_review,
grammar_review, and spoken_word_review to prompt IDs/variables and the common
structured boundary. Keep deterministic validators independent of PromptKit.
2. Preserve batching by `validation.max_prompt_tokens`, proposal preview, stable
proposal indexes, exact decision cardinality, and unknown/duplicate index checks.
Keep the current domain token-estimation method; do not add a second template
renderer or treat this budget as a provider output-token limit.
3. Apply the failure mapping: exhausted repairs reject/warn the batch; schema-valid
bad cardinality or negative assessments reject without semantic retry. Provider,
malformed HTTP, deadline, and admission failures propagate as runtime errors.
Update any old tests treating malformed provider envelopes as warning-only.
4. Remove redundant validator client/model configuration types and keep execution
class distinctions. Update validator reference/internal documentation.
**Acceptance and validation:** preserve existing batching no-drop/no-duplicate,
domain reason, glossary protection, and cardinality tests. Cover all five prompt
mappings, positive repair before domain validation, exhausted repair, and runtime
failure propagation through a representative validator. Run validator tests;
full runner behavior is Stage 8.
## Stage 8: Connect the Runner and Remove the Old Scheduling Hierarchy
**Status:** Pending. **Depends on:** Stages 2 and 4–7.
**Owns:** CLI process assembly, `internal/framework/runner`, old scheduler contracts,
and runner/CLI integration tests.
1. Pass the shared engine's two role clients into the runner. Remove global and
role schedulers, scheduler interfaces/fields, legacy concurrency configuration,
and measured scheduler wrappers. PromptKit backend admission is the only LLM
service-capacity mechanism. Do not remove unrelated deterministic work or alter
module sequencing merely because the scheduler disappears.
2. Preserve concurrent section work, deterministic collection/indexing, ordered
module application, validator execution classes, and cancellation of remaining
work on runtime failure. Capacity rejection fails the run as selected; do not
silently retry or add another queue to conceal it.
3. Wire final loaded domain settings, role clients, and diagnostics context into
normal processing. No old-runtime branch or interim gate remains. Keep existing
transcript encoding, output channels, partial domain results, and artifact
retention behavior. Ensure terminal failures cannot emit a purportedly complete
corrected transcript.
4. Update architecture policy for shared library dependencies and engine-local
backend admission; preserve the existing ownership, determinism, and redaction
invariants. Update pipeline/runtime internal guidance.
**Acceptance and validation:** controlled concurrent requests from both roles to
one backend never exceed its configured cap, including retries and repairs;
separate backend identities have independent capacity. Queued cancellation,
deadline and zero-wait-capacity rejection do not leak capacity; later work can
succeed. Existing deterministic runner/domain tests pass after deleting scheduler-
implementation assertions where the affected packages compile. Exercise a real
CLI-to-local-HTTP workflow with both roles when reporting dependencies permit;
otherwise record that check for Stage 12. Do not restore old utilization fields
or add placeholders merely to make this intermediate commit build.
## Stage 9: Collect Execution Observations Safely
**Status:** Pending. **Depends on:** Stage 8.
**Owns:** Audita-owned execution record types and collector in the LLM boundary,
runner attribution, and concurrency/observation tests.
1. Translate PromptKit events into small Audita-owned records at the observer
boundary. Use the prototype's short mutex-protected collection pattern; callbacks
do no filesystem I/O, provider calls, or waiting on their own run. Freeze/copy
snapshots after active work has joined.
2. Bind observations to logical call, role (`proposal` or `validation`), module
instance, section, validator and batch where applicable, then retain library
run/backend/profile and attempt/generation identity. Keep provider attempts distinct from logical calls.
3. Record dispatch, queue/generation/retry/wall durations, outcome/failure category,
HTTP status, retry exhaustion, attempts/retries/repairs, and usage value/known/
complete per field. Use terminal events once for provider totals; never add
attempt usage on top. `PrepareExecution` emits no events and `RunPrepared` timing
excludes preparation. Audita's surrounding call record therefore owns the logical
count and wall time, including preparation failures; do not invent library events.
A failure before dispatch has zero attempts and known-zero complete usage.
Preserve completed observations on nil-result failures and mark usage incomplete
when dispatched attempts lack measurements. Across calls, sum known values,
set known if any contributing dispatched usage is known, and set complete only
when all contributing dispatched usage is complete, independently per field.
An aggregate with no dispatched attempts is known-zero and complete.
4. Keep non-LLM runner wall/validator timing. Remove instrumentation that counts
scheduler permits as calls. Do not infer queue peaks, exact active concurrency,
billing totals, or timing fields the released event API does not provide.
**Acceptance and validation:** adapter/collector tests prove known zero differs
from unknown, partial usage survives failed repairs, undispatched admissions are
not HTTP attempts, terminal totals are not doubled, and parallel callbacks retain
attribution safely. Run targeted race tests for the affected LLM/runner packages.
No raw provider messages, URLs, keys, prompts, or payloads enter observation records.
## Stage 10: Update Reports and Diagnostic Artifacts
**Status:** Pending. **Depends on:** Stage 9.
**Owns:** `internal/core/reporting`, `internal/core/diagnostics`,
`internal/framework/processreport`, runtime diagnostic writers, and CLI artifact wiring.
1. Bump the process report version to `v2`; keep the existing transcript/ledger
structures. Add `llm_summary` with `logical_calls`, `provider_attempts`,
`transient_retries`, `structural_repairs`, and `usage`. A logical call counts
once even when it fails before dispatch. Usage has `prompt`, `completion`,
`total`, `cached`, and `cache_write`, each `{value, known, complete}`.
Omit the whole summary only when execution observations are unavailable; a
successfully initialized run with no LLM calls has known-zero complete totals.
2. Replace obsolete utilization concurrency/pool fields with `backends`, keyed by
backend ID, containing configured `concurrency`/`queue_capacity` and the same
execution summary plus `capacity_wait_ms`, `generation_ms`, `retry_delay_ms`,
and `logical_wall_ms`. Sum library terminal timings once per call for capacity,
generation, and retry time; use Audita's whole-call elapsed time for logical wall
time. Attribute pre-dispatch failures to the already selected role backend.
Durations can overlap and must not be presented as a decomposition of elapsed
process time. Preserve domain module/validator timing and run wall time.
Existing role/module call counts now count logical calls; remove their obsolete
scheduler-wait and transport-execution fields rather than assign misleading new
meanings. Remove estimated active-concurrency fields throughout utilization.
3. Write `llm-execution.json` as `{calls: [...], events: [...]}` in the run directory.
Calls contain Audita logical identity, attribution, elapsed milliseconds, final
classification, summary, and available prompt/profile/model metadata. Events
contain the safe Stage 9 records, including attempt and terminal records, joined
by call ID. A preparation failure has a call record and no fabricated events.
Use millisecond integer durations in serialized records; absent attempt/section/
batch identity is omitted, not encoded as a misleading zero. Add optional
`llm_execution_path` and `selected_profiles_path` to diagnostic path metadata.
Write `selected-profiles.yaml` from the safe view and use ConfigKit YAML for
the effective snapshot (`effective-config.redacted.yaml`). Keep invocation
overlay selection and recorded origins in safe invocation/config diagnostics.
Adjust centralized filename constants and readers together; do not emit old and
new snapshots in parallel.
4. Preserve redacted request/response/error diagnostic content from prepared
details and available final results, including prompt definition/rendered hashes
and selected profile/model. Do not promise legacy hash equality or complete
provider payloads for every retry. Use the separate observation artifact for retry/repair
attempts. Redact all resolved backend secrets, including unselected ones, at
every serialization/error boundary; observation records remain payload-free.
5. Update output-file, operations, and diagnostics/reporting contracts. Preserve
failure report/ledger behavior and the distinction between primary execution
failures and secondary diagnostic write errors. Do not change retention policy.
**Acceptance and validation:** report/diagnostic tests check v2 mapping, summary
semantics, per-backend grouping, safe serialization, nil-result partial failures,
and existing deterministic ledger order. A CLI failure fixture echoes a synthetic
secret in a provider error; stdout/stderr/report and retained artifacts must not
leak it. Inspect known-zero and unavailable summaries structurally, not by complete
JSON golden dumps. Reuse Stage 9 evidence rather than retest every event sequence.
## Stage 11: Finish Examples, Migration Guidance, and Removal
**Status:** Pending. **Depends on:** Stages 1–10.
**Owns:** maintained examples, affected canonical documentation, obsolete code/tests,
prototype removal, and final dependency cleanup.
1. Replace maintained config examples with schema 2, complete PromptKit profile
files, and `overlays/testing.yml` selecting both roles. Use one shared backend
where roles share a service; provide secret-reference and unauthenticated local
examples without real secrets. Explain that operators choose real models and
testing cost/quality is not guaranteed. Validate examples offline with controlled
synthetic environment values; no service is required for profile inspection.
2. Write `docs/migrations/v2.md` as the one migration guide. Cover every old file/
flag/environment item in the inventory, durations, explicit empties/null,
credentials, profiles/overlays, inspection views, finite capacity, retry/repair/
deadline semantics, and changed report/diagnostic shapes. Do not add conversion
tooling or compatibility aliases. Link current contracts for complete references.
3. Finish README prerequisites/examples and canonical CLI/config, integration,
internal component, and architecture references. Audit build/CI toolchain images
for Go 1.25.5 or newer; keep only required Linux amd64/arm64 targets and best-effort
macOS arm64. Do not rewrite immutable historical release notes.
4. Remove `examples/library-adoption` once its unique useful scenarios live in
production tests. Remove the nested dependency on old Audita and duplicate
assets/fixtures. Update historical roadmap links to recorded evidence or Git
history rather than leaving broken links to deleted files.
5. Remove superseded loaders, client, renderers, schedulers, aliases, and tests
asserting retired behavior. Tidy/verify the root dependency graph and confirm
no local replacements. Keep yaml.v3 if glossary parsing still uses it. Mark the
retry-count follow-up in `runtime-behavior.md` resolved without including its
unrelated retention/timestamp proposals.
**Acceptance and validation:** maintained examples validate and all guide links
resolve. Search for old CLI/env inputs, loader/client/scheduler names and prototype
references; remaining mentions must be intentional migration/history material.
Review deleted tests for lost domain/concurrency/redaction protection. Run affected
example/config/subprocess checks and `git diff --check`; final full-suite execution
belongs to Stage 12.
## Stage 12: Verify the Combined Application and Close the Refactor
**Status:** Pending. **Depends on:** Stages 1–11.
**Owns:** final integration tests, final defect fixes within this scope, and status/evidence.
1. Complete representative production subprocess workflows against controlled HTTP:
base selection, testing overlay, explicit profile override, and a retained
domain override. Observe actual outbound models for both roles, an accepted
correction after repair, a warning-only domain/repair rejection, and a runtime
failure with safe partial diagnostics. Cover an empty module pipeline and a
repeated-module sequence without duplicating all adapter scenarios.
2. Run root `go test ./...`, `go build -o ./bin/audita ./cmd/audita`, and focused race
checks for the final concurrent adapter/runner/collector changes. Verify both
Linux targets compile using `bash scripts/build-release.sh v2.0.0 <temp-dir>`;
this names local build outputs and creates no tag or publication. No native Mac, paid-provider, or CI-completion gate.
If the supported minimum toolchain was not exercised earlier, verify the root
suite with installed Go 1.25.5; do not silently claim this from a newer compiler.
3. Fix actual failures and rerun affected checks; repeat the full suite when those
fixes materially change assembled behavior. Do not rerun unchanged tests solely
to mark multiple planning steps complete. Verify maintained documentation links,
examples, and whitespace once for the final content.
4. Record concise final evidence and known limits here, mark the production feature
roadmaps implemented, and commit the completed refactor. Both libraries must
own their selected responsibilities, processing must work, and no temporary
or legacy compatibility machinery may remain.
5. Hand off release preparation separately: proposed Audita v2.0.0, one migration
guide, actual pinned production dependency/license inventory, and the remaining
LGPL notice/source/rebuild-distribution review. Do not claim license compliance
from prototype tests or publish a release. The later release unit uses the
existing minimal automated procedure and ends on a successful source/tag push,
without waiting for CI.
**Acceptance:** the final production application and required Linux builds pass;
configuration selection, both LLM roles, domain outcomes, cancellation/capacity,
secret handling, reports and retained artifacts match the selected contract.
Recorded checks describe what actually ran. The workspace is committed and clean.
Release-specific compliance and publication are explicitly still pending.
## Open Questions
None block this implementation plan. Configuration, inspection, runtime boundaries,
report representation, and stage ownership are selected above. Release distribution
compliance remains outside this implementation sprint and must be resolved against
the final production dependency graph before publishing binaries; it is not a
reason to build compatibility scaffolding or delay the refactor.