Add utilization diagnostics and correction ledger

docs/architecture.md

@@ -646,6 +646,8 @@ Current per-run artifacts include:
 - `normalized-transcript.json`
 - `normalization-summary.json`
 - `chunking-summary.json`
+- `utilization-diagnostics.json`
+- `correction-ledger.json`
 - `invocation.json`
 - `effective-config.json` (redacted credentials)
 - `report.json`
@@ -660,6 +662,8 @@ Current process reports include diagnostics metadata references for:
 - normalized transcript artifact path;
 - normalization summary artifact path;
 - chunking summary artifact path;
+- utilization diagnostics artifact path;
+- correction ledger artifact path;
 - invocation metadata artifact path;
 - redacted effective-config artifact path;
 - error-log artifact path on failure.
@@ -675,6 +679,59 @@ Current process reports also include:
   - report schema version;
   - selected output schema;
   - config file version when config file input is used.
+- review/observability artifacts:
+  - run-level and module-level utilization/timing summaries;
+  - flattened correction ledger entries for applied/rejected/skipped/failed correction dispositions.
+
+Utilization diagnostics collection:
+- collection is performed in the runner path via lightweight instrumentation around LLM scheduler and structured-client execution (`internal/framework/runner`);
+- instrumentation is observational only and does not change scheduler acquisition/release semantics or module execution order;
+- serialized artifact: `utilization-diagnostics.json`.
+
+Utilization diagnostics high-level shape:
+- `effective_concurrency`:
+  - `total_llm`, `proposal_llm`, `validation_llm`;
+- `run_timing`:
+  - `run_wall_time_ms`;
+  - `scheduler_queue_wait_ms`;
+  - `llm_execution_time_ms`;
+  - `deterministic_validation_time_ms`;
+  - `max_in_flight_llm_calls`;
+  - `average_in_flight_llm_calls`;
+- `llm_calls`:
+  - `total_proposal_calls`;
+  - `total_validation_calls`;
+- `modules`:
+  - per-module key/instance timing summaries including module wall time and per-module call counts;
+- `validators`:
+  - per-validator summaries keyed by stable validator key with elapsed time and LLM-backed marker.
+
+Correction ledger construction:
+- ledger entries are built from runner module results in the CLI report/diagnostics path (`internal/cli/review_artifacts.go`);
+- serialized artifact: `correction-ledger.json`;
+- one flattened record per applied/validator-rejected/application-skipped outcome where data is available, plus module-failed records for failed module instances.
+
+Correction ledger high-level shape:
+- run/module/proposal identity:
+  - `run_id`, `module_key`, `module_instance`, `proposal_index`, `segment_id`;
+- correction payload:
+  - `original_text`, `proposed_corrected_text`, `applied_corrected_text` (when applied), `replacement_policy`;
+- disposition:
+  - `disposition` in `{applied,rejected,skipped,failed}`;
+  - `disposition_reason_code`, `disposition_message`;
+- validator decision snapshots:
+  - `deterministic_validator_decisions[]`;
+  - `llm_validator_decisions[]`;
+  - each decision uses stable validator keys and reason codes.
+
+Identity and metadata boundaries:
+- stable module keys/instance names and stable validator keys are included directly in ledger records;
+- prompt metadata and structured response schema metadata remain in LLM interaction diagnostics payloads and are not duplicated into every ledger row;
+- reports reference artifact paths for utilization and ledger files through diagnostics metadata.
+
+Redaction and retention:
+- secret redaction guarantees continue to apply to diagnostics/report artifacts;
+- utilization and ledger artifacts are emitted within the existing run-directory retention model (`auto|always|never`) and are retained/removed with the run directory.
 
 Current report schema metadata values:
 - `report_metadata.report_schema_name = "audita-process-report"`

docs/diagnostics.md

@@ -0,0 +1,98 @@
+# Audita Diagnostics
+
+This document describes the run-directory diagnostics artifacts produced by `audita process`.
+
+## Purpose
+
+Diagnostics provide machine-readable run context and execution artifacts for:
+- failure debugging;
+- validator/correction review;
+- post-run performance analysis.
+
+Diagnostics are written under the configured work directory (`--work-dir`) when run-directory initialization succeeds.
+
+## Core artifacts
+
+Typical artifacts in each run directory:
+- `source-transcript.json`
+- `source-transcript-parsed.json`
+- `normalized-transcript.json`
+- `normalization-summary.json`
+- `chunking-summary.json`
+- `invocation.json`
+- `effective-config.json` (redacted)
+- module/validator LLM interaction artifacts
+- `report.json`
+- `error.log` on failure
+
+## Utilization diagnostics artifact
+
+Artifact:
+- `utilization-diagnostics.json`
+
+High-level fields:
+- `effective_concurrency`:
+  - total/proposal/validation LLM concurrency limits in effect.
+- `run_timing`:
+  - run wall time;
+  - scheduler queue wait time;
+  - LLM execution time;
+  - deterministic validator time;
+  - max/average in-flight LLM calls.
+- `llm_calls`:
+  - total proposal and validation LLM call counts.
+- `modules`:
+  - module-level timing summaries.
+- `validators`:
+  - per-validator timing summaries keyed by stable validator key.
+
+## Correction ledger artifact
+
+Artifact:
+- `correction-ledger.json`
+
+Ledger records are flattened review entries derived from module results and include:
+- module/proposal identity (`module_key`, `module_instance`, `proposal_index`, `segment_id`);
+- correction text fields and replacement policy when available;
+- disposition:
+  - `applied`
+  - `rejected`
+  - `skipped`
+  - `failed`
+- stable reason codes/messages;
+- deterministic and LLM validator decision snapshots using stable validator keys.
+
+Validator rejection and proposal-application skip are distinct dispositions.
+
+## Report references
+
+`report.json` and optional `--report-json` output include diagnostics metadata paths for:
+- utilization diagnostics artifact;
+- correction ledger artifact;
+- existing transcript/normalization/chunking/invocation/effective-config artifacts.
+
+## Retention behavior
+
+Run-directory retention follows configured policy:
+- `always`: keep all run directories;
+- `never`: keep successful run directories;
+- `auto`: keep failed runs and successful runs with skipped/rejected corrections.
+
+## Redaction guarantees
+
+API keys and other configured secrets are redacted from:
+- `effective-config.json`;
+- LLM interaction diagnostics artifacts;
+- reports and surfaced errors.
+
+## Debugging guide
+
+When debugging:
+- slow runs:
+  - inspect `utilization-diagnostics.json` (`run_timing`, `modules`, `validators`, in-flight metrics).
+- validator rejections:
+  - inspect `correction-ledger.json` rejected entries and matching validator decisions;
+  - inspect validator response diagnostics payloads.
+- application skips:
+  - inspect `correction-ledger.json` skipped entries and skip reason codes;
+  - compare with validator decisions to distinguish validation rejection vs apply-time skip.

docs/public-contract.md

@@ -13,6 +13,7 @@ This contract covers:
 - stable validator key identifiers in report/diagnostics records
 - prompt metadata identifiers in diagnostics
 - diagnostics directory behavior
+- utilization diagnostics and correction-ledger artifact presence/pathing in diagnostics metadata
 - stdout/stderr and exit-code behavior
 - secret redaction guarantees
 - compatibility and deprecation policy
@@ -92,6 +93,7 @@ Current values:
 `--report-json` output and diagnostics run-dir `report.json` use the same report schema metadata.
 
 Validator decision/rejection records in reports use stable validator keys in `validator_name`.
+Report diagnostics metadata includes artifact-path fields for utilization diagnostics and correction ledger when diagnostics initialization succeeds.
 
 ## Diagnostics directory behavior
 
@@ -99,6 +101,8 @@ When diagnostics directory creation succeeds, Audita writes run artifacts includ
 - invocation metadata
 - redacted effective config
 - transcript/normalization/chunking artifacts
+- utilization diagnostics (`utilization-diagnostics.json`)
+- correction ledger (`correction-ledger.json`)
 - report and failure error log (when applicable)
 - module/LLM diagnostics artifacts as available
 

docs/roadmap.md

@@ -811,6 +811,17 @@ Improve operational observability so Audita runs can be debugged and performance
 
 This phase should follow the registry/prompt work so diagnostics can include stable module, validator, prompt, and schema identifiers.
 
+## Implementation status (2026-05-13)
+
+This workstream is now implemented:
+- run diagnostics include `utilization-diagnostics.json` with run-level and module-level timing/utilization data;
+- utilization diagnostics include per-validator timing summaries keyed by stable validator keys;
+- process reports include diagnostics references for utilization and correction-ledger artifacts;
+- run diagnostics include `correction-ledger.json` with flattened correction disposition records (`applied`, `rejected`, `skipped`, `failed`) keyed by stable module instance and proposal index;
+- successful runs emit both artifacts; failure paths emit partial artifacts when runner state exists.
+
+This status update applies only to utilization diagnostics and correction-ledger artifacts. Release-hardening evaluation fixtures and later roadmap items remain planned.
+
 ## Scheduler utilization diagnostics
 
 Add module-level and run-level metrics for LLM scheduling and execution.

docs/subprocess-operations.md

@@ -46,11 +46,12 @@ Recommended additions:
 - Run-directory `report.json` is written independently under diagnostics.
 - Best-effort failure reports are emitted when possible without masking the primary failure.
 - Report write failures return nonzero with clear stderr messaging.
+- Report diagnostics metadata references run-directory artifacts including utilization diagnostics and correction ledger paths when available.
 
 ## Diagnostics directory behavior
 
 - Each run creates (when possible) a per-run diagnostics directory.
-- Typical artifacts include transcript, normalization, chunking, invocation, effective config, LLM diagnostics, `report.json`, and `error.log` on failure.
+- Typical artifacts include transcript, normalization, chunking, invocation, effective config, LLM diagnostics, `utilization-diagnostics.json`, `correction-ledger.json`, `report.json`, and `error.log` on failure.
 - Failed runs retain diagnostics.
 - Under `auto` retention, successful runs with skipped/rejected corrections are retained; clean successful runs may be removed.
 

docs/validators.md

@@ -103,6 +103,7 @@ These are separate outcomes and are reported separately.
 
 - report validator decision/rejection entries use stable validator keys in `validator_name`.
 - validator LLM diagnostics include validator identity in interaction metadata and structured response schema metadata.
+- correction ledger entries include deterministic and LLM validator decision snapshots keyed by the same stable validator keys, and keep validator rejection distinct from application-level skip.
 
 ## Configurable knobs that remain supported