diff --git a/docs/operations.md b/docs/operations.md new file mode 100644 index 0000000..c6ebe92 --- /dev/null +++ b/docs/operations.md @@ -0,0 +1,113 @@ +# Audita Operations + +## Scope + +This document covers operational behavior for `audita process` as currently implemented: +- run lifecycle; +- output and report files; +- diagnostics artifacts; +- run-directory retention behavior; +- failure inspection and recovery. + +For command syntax, see [`docs/cli.md`](cli.md). + +## Process Run Lifecycle + +A `process` run performs these high-level steps: +1. load effective config (defaults + optional file + env + CLI); +2. create a per-run diagnostics directory; +3. load transcript JSON and glossary YAML; +4. parse/validate input schemas; +5. normalize transcript and compute chunking; +6. run configured modules/validators; +7. serialize output schema and write transcript output; +8. build and write process report; +9. apply run-directory retention. + +If a failure happens after diagnostics initialization, the run writes failure details and returns nonzero. + +## Output Files + +Transcript output: +- when `--output ` is set, corrected transcript JSON is written to that file; +- when `--output` is omitted, corrected transcript JSON is written to stdout. + +Report output: +- when `--report-json ` is set, Audita writes a process report JSON file; +- the run directory also writes its own `report.json` artifact. + +On success with `--output`, stdout is expected to be empty. + +## Diagnostics Directory + +By default, runs use `work_dir` from effective config (default `/tmp/audita`). +Each run directory is created under the work dir using a generated ID like `run-`. + +Top-level diagnostics artifacts: +- `source-transcript.json` +- `source-transcript-parsed.json` +- `normalized-transcript.json` +- `normalization-summary.json` +- `chunking-summary.json` +- `utilization-diagnostics.json` +- `correction-ledger.json` +- `invocation.json` +- `effective-config.json` (redacted) +- `report.json` +- `error.log` (failure runs) + +Report diagnostics metadata includes resolved paths to these artifacts. + +## Correction Ledger and Utilization Diagnostics + +`correction-ledger.json` records correction dispositions: +- `applied` +- `skipped` +- `rejected` +- `failed` + +`utilization-diagnostics.json` records effective concurrency and execution timing summaries for run/module/validator activity. + +## Retention Behavior + +Retention is controlled by `work_dir_retention` (`auto|always|never`). + +Current behavior: +- failed runs are always retained; +- `always`: successful runs are retained; +- `auto`: successful runs are retained only when skipped/rejected corrections occurred; clean successful runs are removed; +- `never`: successful runs are currently retained (same net retention outcome as `always` in current implementation). + +Even when a successful run directory is removed under `auto`, an explicit `--report-json` file is still preserved at its target path. + +## Failure Inspection + +For failed runs: +1. read stderr for the top-level failure and diagnostics path; +2. open `error.log` in the reported run directory; +3. inspect run `report.json` (`status`, `error_phase`, `error_message`); +4. inspect related artifacts referenced by report diagnostics metadata. + +Typical `error_phase` values include: +- `transcript_read` +- `glossary_read` +- `transcript_schema` +- `glossary_schema` +- `chunking` +- `runner_setup` +- `runner_execution` +- `output_schema` +- `serialization` +- `output_write` +- `stdout_write` + +## Recovery Guidance + +Safe recovery pattern: +1. correct the immediate input/config/output-path problem; +2. rerun with `--work-dir-retention always` during debugging; +3. once stable, restore your normal retention mode. + +Not implemented: +- resume/checkpoint APIs +- remote diagnostics/report storage diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md new file mode 100644 index 0000000..41f7b4f --- /dev/null +++ b/docs/troubleshooting.md @@ -0,0 +1,153 @@ +# Audita Troubleshooting + +## Scope + +This guide lists recurring implemented failure modes for `audita process` and `audita config`. + +For each entry: symptom, likely cause, inspect, and fix. + +## Config Validation Fails + +Symptom: +- `audita config validate --config ` exits nonzero. + +Likely causes: +- missing `version`; +- unsupported config version; +- unknown YAML field; +- unsupported module key or output schema; +- invalid numeric/range/concurrency/retention values. + +Inspect: +1. rerun `audita config validate --config ` and read stderr. +2. if needed, inspect effective config with `audita config print-effective --config `. + +Fix: +- set `version: 1`; +- remove unknown fields; +- use supported module keys and output schemas (`bare-segments`, `audita-v1`); +- correct invalid values to satisfy validation constraints. + +## Config File Resolution Errors + +Symptom: +- `audita process` fails before processing with config-related errors like `config file not found`. + +Likely causes: +- `--config` points to a missing path; +- `AUDITA_CONFIG` points to a missing path; +- unreadable config path. + +Inspect: +1. confirm `--config` or `AUDITA_CONFIG` path exists; +2. run `audita config validate --config ` directly. + +Fix: +- correct the path or unset invalid `AUDITA_CONFIG`; +- fix permissions for the config file. + +## Transcript or Glossary Schema Errors + +Symptom: +- stderr includes `transcript_schema` or `glossary_schema` and run exits nonzero. + +Likely causes: +- transcript is not valid JSON or has invalid segment fields; +- glossary is not valid YAML or has missing required glossary entry fields. + +Inspect: +1. check stderr for parser/validation details; +2. if diagnostics were created, inspect `error.log` and run `report.json` (`error_phase`); +3. inspect `source-transcript.json` and `source-transcript-parsed.json` in the run directory. + +Fix: +- correct transcript JSON shape/content; +- correct glossary YAML shape/content and required entry fields; +- rerun validation with known-good tiny examples for comparison: + - `examples/tiny-transcript.json` + - `examples/tiny-glossary.yaml` + +## LLM Runtime/Backend Failures + +Symptom: +- stderr includes `runner_execution` (or backend timeout/error details) and nonzero exit. + +Likely causes: +- unreachable/failed LLM endpoint; +- timeout/cancellation; +- runtime module execution failure. + +Inspect: +1. inspect stderr for backend message details; +2. inspect run `report.json` (`error_phase`, `module_results`); +3. inspect diagnostics payloads and `error.log`. + +Fix: +- verify model/base URL/API key settings; +- increase timeout if needed; +- rerun with `--work-dir-retention always` while debugging. + +## Output File Write Failure + +Symptom: +- stderr includes `failed to write output file` and run exits nonzero. + +Likely causes: +- output path directory missing; +- insufficient filesystem permissions; +- invalid output target path. + +Inspect: +1. check `--output` target directory exists and is writable; +2. inspect run diagnostics `error.log` and report `error_phase`. + +Fix: +- write to a valid writable path; +- create missing directories; +- adjust permissions. + +## Report File Write Failure + +Symptom: +- stderr includes `failed to write report JSON file` and run exits nonzero. + +Likely causes: +- invalid or unwritable `--report-json` target path. + +Inspect: +1. verify parent directory exists and is writable; +2. inspect diagnostics `error.log` for `report_write` context. + +Fix: +- choose a writable report path; +- create missing directories; +- rerun. + +## Unsupported Output Schema + +Symptom: +- stderr includes `unsupported output schema` and run exits nonzero. + +Likely causes: +- unsupported `--output-schema` value; +- unsupported `output.schema` in config. + +Inspect: +1. check CLI/config schema key; +2. run `audita config validate --config ` when config is involved. + +Fix: +- use `bare-segments` or `audita-v1`. + +## Diagnostics Directory Lookup + +Symptom: +- run fails and you need artifacts for debugging. + +Inspect: +1. read stderr for `audita process: diagnostics: `; +2. open `/report.json` and `/error.log`; +3. use diagnostics paths embedded in report metadata for artifact lookup. + +Fix: +- rerun with `--work-dir-retention always` to preserve run directories during investigation.