Add utilization diagnostics and correction ledger

docs/diagnostics.md

@@ -0,0 +1,98 @@
+# Audita Diagnostics
+
+This document describes the run-directory diagnostics artifacts produced by `audita process`.
+
+## Purpose
+
+Diagnostics provide machine-readable run context and execution artifacts for:
+- failure debugging;
+- validator/correction review;
+- post-run performance analysis.
+
+Diagnostics are written under the configured work directory (`--work-dir`) when run-directory initialization succeeds.
+
+## Core artifacts
+
+Typical artifacts in each run directory:
+- `source-transcript.json`
+- `source-transcript-parsed.json`
+- `normalized-transcript.json`
+- `normalization-summary.json`
+- `chunking-summary.json`
+- `invocation.json`
+- `effective-config.json` (redacted)
+- module/validator LLM interaction artifacts
+- `report.json`
+- `error.log` on failure
+
+## Utilization diagnostics artifact
+
+Artifact:
+- `utilization-diagnostics.json`
+
+High-level fields:
+- `effective_concurrency`:
+  - total/proposal/validation LLM concurrency limits in effect.
+- `run_timing`:
+  - run wall time;
+  - scheduler queue wait time;
+  - LLM execution time;
+  - deterministic validator time;
+  - max/average in-flight LLM calls.
+- `llm_calls`:
+  - total proposal and validation LLM call counts.
+- `modules`:
+  - module-level timing summaries.
+- `validators`:
+  - per-validator timing summaries keyed by stable validator key.
+
+## Correction ledger artifact
+
+Artifact:
+- `correction-ledger.json`
+
+Ledger records are flattened review entries derived from module results and include:
+- module/proposal identity (`module_key`, `module_instance`, `proposal_index`, `segment_id`);
+- correction text fields and replacement policy when available;
+- disposition:
+  - `applied`
+  - `rejected`
+  - `skipped`
+  - `failed`
+- stable reason codes/messages;
+- deterministic and LLM validator decision snapshots using stable validator keys.
+
+Validator rejection and proposal-application skip are distinct dispositions.
+
+## Report references
+
+`report.json` and optional `--report-json` output include diagnostics metadata paths for:
+- utilization diagnostics artifact;
+- correction ledger artifact;
+- existing transcript/normalization/chunking/invocation/effective-config artifacts.
+
+## Retention behavior
+
+Run-directory retention follows configured policy:
+- `always`: keep all run directories;
+- `never`: keep successful run directories;
+- `auto`: keep failed runs and successful runs with skipped/rejected corrections.
+
+## Redaction guarantees
+
+API keys and other configured secrets are redacted from:
+- `effective-config.json`;
+- LLM interaction diagnostics artifacts;
+- reports and surfaced errors.
+
+## Debugging guide
+
+When debugging:
+- slow runs:
+  - inspect `utilization-diagnostics.json` (`run_timing`, `modules`, `validators`, in-flight metrics).
+- validator rejections:
+  - inspect `correction-ledger.json` rejected entries and matching validator decisions;
+  - inspect validator response diagnostics payloads.
+- application skips:
+  - inspect `correction-ledger.json` skipped entries and skip reason codes;
+  - compare with validator decisions to distinguish validation rejection vs apply-time skip.