audita/docs/roadmap/remediation.md

Library Adoption Review: Required Remediation

Reviewed: 2026-09-24, source commit 7cefb3f. Scope: The twelve implementation commits after 1ad2119, compared with the implementation plan, combined target, and configuration target. Status: Findings confirmed; fixes have not been implemented.

Architecture Assessment

The intended ownership split is substantially implemented. ConfigKit v0.4.0 owns schema-2 loading, overlays, presence/default resolution, explicit overrides, credentials, and configuration inspection. One shared PromptKit v0.11.0 engine owns both roles' profiles, embedded prompt rendering, prepared execution, provider transport, retry/repair behavior, and per-backend admission. Audita retains domain configuration, CLI composition, ordered modules, deterministic validators, proposal/application checks, validation batching/cardinality, reporting, and artifact lifecycle. The root module requires Go 1.25.5 and has no local replacement.

Legacy loaders, provider transport, template rendering, global/role schedulers, and the isolated prototype were removed. All nine prompt definitions preserve the original instruction text after accounting for the new hardening variable and whitespace. There is no need to revisit the library-adoption design or add compatibility machinery. The findings below are concrete defects or unfulfilled parts of the selected final contract.

R1 — P1: Redact Model-Derived Content in Every Retained Report Artifact

Category: Incomplete implementation of Stage 10's redaction boundary.

Evidence: Both success and failure paths in runProcess write BuildCorrectionLedger(...) directly through RunDirectory.WriteJSONArtifact. The success-path write is at line 348. That writer serializes its input without redaction. Validator decision/rejection messages contain provider-supplied reason text. Redacting the separate process report does not protect the ledger.

A controlled HTTP subprocess reproduction returned a schema-valid negative assessment whose reason contained the synthetic configured backend credential. Processing succeeded and the process report was redacted, but correction-ledger.json retained the credential verbatim. The existing provider- error test does not exercise this successful, schema-valid response path.

Required change: Apply the configured-secret redaction boundary to the ledger and other model-derived diagnostic/report outputs before persistence. Audit the new execution/utilization and selected-profile artifact paths for the same bypass; metadata-only output is not a substitute for enforcing the selected secret-value contract. Use all resolved backend credentials, including unselected backends. Keep processing/domain data separate from its sanitized diagnostic representation. Coordinate the shared redaction implementation with R2.

Acceptance: A controlled provider echoes synthetic credentials in a valid assessment reason. Neither success nor partial-failure reports, ledgers, interaction artifacts, or other affected diagnostic outputs contain those values. Include an unselected backend credential. Preserve valid serialization and domain outcomes; do not expand tests into exhaustive permutations of every artifact.

R2 — P1: Remove the Report Redactor's Unredacted Fallback

Category: New code-quality/security defect in Stage 10.

Evidence: redactProcessReport replaces secret substrings in the serialized JSON document, then decodes it back into the report. If decoding fails, it returns the original report. Replacements can corrupt JSON syntax or typed values rather than just redact strings.

A focused reproduction loaded an otherwise valid config with the synthetic credential 2, then redacted a report containing config_version: 2 and the string echo:2. Replacing the numeric token made the JSON invalid; the helper returned the original echo:2. Short numeric credentials are accepted by the selected secret-reference contract, so the implementation cannot silently assume otherwise. Quotes, escapes, overlapping credentials, and replacement order also need handling at the serialization boundary.

Required change: Redact string content without changing JSON structure or numeric/boolean types. Never return or write the original sensitive object when redaction/serialization fails. Share the appropriate string-redaction logic across report and artifact writers rather than maintaining incompatible versions. Handle escaped content and overlapping values deliberately; retain typed error causes without exposing provider error text.

Acceptance: Focused tests cover a numeric credential coinciding with report numbers, quoted/backslash-containing credentials, and overlapping values. Output remains valid with the original numeric/boolean meanings, secrets are removed, and a redaction failure cannot emit the original object. Exercise the actual external and run-local report writing paths with representative coverage.

R3 — P1: Cancel Remaining Work When a Validator Fails

Category: Pre-existing runner behavior left inconsistent with the selected Stage 8 final state and architecture policy; not newly introduced by PromptKit.

Evidence: runModulePipeline puts validation errors into a buffered channel, but reads that channel only after vwg.Wait() (lines 414–421). setErr therefore cannot cancel the shared context in response to a validator failure until all validation workers have finished. Proposal work can continue launching validation in the meantime.

A controlled two-section reproduction made one validator fail while its peer waited for context cancellation or an explicit test release. The peer was not canceled; the pipeline returned its error only after the peer was manually released. In production, a failed validation operation can thus leave other queued or active provider calls consuming capacity and money until they finish or reach their own deadlines. Adapter-level cancellation tests do not cover this runner behavior.

Required change: Propagate validation failures to shared cancellation as soon as they occur, while retaining the originating error, joining active workers, and collecting available results deterministically. Use a coordinator or properly synchronized failure state; simply calling the current setErr from workers would also require protecting concurrent reads of firstErr. Do not add a scheduler, outer retry loop, or compatibility layer.

Acceptance: A runner integration test with a failing validator and a blocked peer proves cancellation arrives without releasing the peer manually. Cover queued provider work and preservation of the originating failure/partial observations. The affected runner/runtime tests pass under the race detector, and failure emits no purportedly complete corrected transcript.

R4 — P2: Separate Module Identity from Section-Specific Diagnostic Names

Category: New observation/report attribution defect in Stages 9–10.

Evidence: validateSectionCandidates passes ModuleInstanceForStages as the validation request's module identity. This value includes :section-NNNN. The validator call copies it into DiagnosticContext.ModuleInstance and leaves SectionIndex unset. BuildUtilization groups these records by that field, so proposal and validation calls for one real module are reported under different module identities.

The subprocess reproduction with one grammar module and one section produced:

Reported module instance	Proposal calls	Validation calls
grammar	1	0
grammar:section-0000	0	1

The latter is not a configured module instance. Its timing cannot join the real module result, and the actual module incorrectly reports zero validation calls. Multi-section runs multiply these spurious entries. Explicit section attribution is also lost from validation call/event records.

Required change: Carry actual module instance and section index separately through the validator request and structured-call context. Keep section-qualified artifact names as a separate naming concern so files remain distinct. Group counts and timing by the actual module instance. Update tests that currently use batch index as a substitute for section identity.

Acceptance: A multi-section workflow with repeated module instances yields one utilization entry per actual instance, correct proposal/validation totals, and explicit section/batch identity in validation observations. Artifact names remain unique and correction-ledger ordering is unchanged.

R5 — P2: Index Execution Events Instead of Repeatedly Scanning Them

Category: New report-building performance/maintainability defect in Stage 10.

Evidence: BuildLLMExecution scans every event for every call. BuildUtilization additionally tests every event against a linear hasCall search for each backend's calls. With C calls and E events, these paths perform O(C × E) work even for one backend. Retries and repairs increase E. This work occurs after processing, before the CLI completes.

A synthetic review probe using one terminal event per call measured:

Calls/events	Execution mapping	Utilization mapping
1,000	14 ms	6 ms
10,000	1.61 s	0.52 s
20,000	8.08 s	2.42 s

These are local diagnostic measurements, not performance guarantees; the nested scans establish the underlying growth independently of machine timing. Actual snapshots usually contain more than one event per call.

Required change: Index calls/events by call ID once and aggregate by backend and actual module identity using direct lookups. Reuse terminal records for totals without rescanning all events for each call. Preserve event/call ordering and knownness/completeness semantics. Straightforward maps are sufficient; no reporting framework or new persistent cache is needed.

Acceptance: Existing report semantics remain intact, including preparation failures, undispatched calls, and incomplete usage. A local benchmark or profiling comparison demonstrates removal of the quadratic scans. Do not add fragile wall-clock timing assertions to the default test suite.

Review Verification and Limits

The existing root go test ./..., focused race tests for LLM/runner/subprocess packages, go vet ./..., and Linux amd64/arm64 builds passed. Temporary review probes reproduced R1–R4 and measured R5; those probe files were removed afterward. The retained suite currently passes despite these defects. Add durable regression coverage with the fixes, using the narrowest useful production boundaries.

Only this remediation document is changed by the review. No application fixes, upstream changes, live provider calls, release publication, or compatibility work were performed. Release-distribution compliance remains the explicitly deferred release task and is not counted as an implementation defect here.