audita/docs/diagnostics.md

Audita Diagnostics

This document describes the run-directory diagnostics artifacts produced by audita process.

Purpose

Diagnostics provide machine-readable run context and execution artifacts for:

• failure debugging;
• validator/correction review;
• post-run performance analysis.

Diagnostics are written under the configured work directory (--work-dir) when run-directory initialization succeeds.

Core artifacts

Typical artifacts in each run directory:

• source-transcript.json
• source-transcript-parsed.json
• normalized-transcript.json
• normalization-summary.json
• chunking-summary.json
• invocation.json
• effective-config.json (redacted)
• module/validator LLM interaction artifacts
• report.json
• error.log on failure

Utilization diagnostics artifact

Artifact:

• utilization-diagnostics.json

High-level fields:

• effective_concurrency:
  • total/proposal/validation LLM concurrency limits in effect.
• run_timing:
  • run wall time;
  • scheduler queue wait time;
  • LLM execution time;
  • deterministic validator time;
  • max/average in-flight LLM calls.
• llm_calls:
  • total proposal and validation LLM call counts.
• modules:
  • module-level timing summaries.
• validators:
  • per-validator timing summaries keyed by stable validator key.

Correction ledger artifact

Artifact:

• correction-ledger.json

Ledger records are flattened review entries derived from module results and include:

• module/proposal identity (module_key, module_instance, proposal_index, segment_id);
• correction text fields and replacement policy when available;
• disposition:
  • applied
  • rejected
  • skipped
  • failed
• stable reason codes/messages;
• deterministic and LLM validator decision snapshots using stable validator keys.

Validator rejection and proposal-application skip are distinct dispositions.

Report references

report.json and optional --report-json output include diagnostics metadata paths for:

• utilization diagnostics artifact;
• correction ledger artifact;
• existing transcript/normalization/chunking/invocation/effective-config artifacts.

Retention behavior

Run-directory retention follows configured policy:

• always: keep all run directories;
• never: keep successful run directories;
• auto: keep failed runs and successful runs with skipped/rejected corrections.

Redaction guarantees

API keys and other configured secrets are redacted from:

• effective-config.json;
• LLM interaction diagnostics artifacts;
• reports and surfaced errors.

Debugging guide

When debugging:

• slow runs:
  • inspect utilization-diagnostics.json (run_timing, modules, validators, in-flight metrics).
• validator rejections:
  • inspect correction-ledger.json rejected entries and matching validator decisions;
  • inspect validator response diagnostics payloads.
• application skips:
  • inspect correction-ledger.json skipped entries and skip reason codes;
  • compare with validator decisions to distinguish validation rejection vs apply-time skip.