eric/audita
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Document resilient reports and remove obsolete warning material

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-24 17:55:37 +00:00
parent 2f036b0356
commit a770c77a23
26 changed files with 346 additions and 536 deletions
Download Patch File Download Diff File Expand all files Collapse all files

154
docs/operations.md
View File

@@ -1,113 +1,73 @@
# Audita Operations
This document owns runtime workflow, physical artifact placement, retention, and
recovery. Use the [CLI reference](cli.md) for invocation and stream/exit semantics,
[configuration](config.md) for settings, and [output-file contracts](integrations/output-files.md)
for logical formats.
This guide owns physical output placement and failure handling. The
[CLI reference](cli.md) owns flags and streams, [configuration](config.md) owns
settings, and the [output contract](integrations/output-files.md) owns schemas.
## Normal Workflow
1. Choose a configuration and validate it using the
[configuration commands](cli.md#commands-and-help).
2. Prepare input files conforming to the [input contract](integrations/transcript-glossary-files.md).
3. Run the [processing command](cli.md#commands-and-help), choosing explicit output
and report destinations when those files must survive run-directory cleanup.
4. Inspect the corrected transcript and process report. Review rejected or skipped
corrections when the report indicates them; successful execution does not mean
every proposed correction was applied.
1. Validate a version-2 configuration and its selected profiles with
`audita config validate --config config.yml`.
2. Prepare a transcript and glossary following the [input contract](integrations/transcript-glossary-files.md).
3. Run `audita process input.json --config config.yml --glossary glossary.yaml --output corrected.json`.
4. Check the exit status, stderr summary, corrected transcript, and reports.
A completed run can contain recoverable model errors and still apply
corrections that passed every required validator.
Processing creates a run directory, loads and normalizes the inputs, chunks the
transcript, executes the correction pipeline, writes results, and applies
retention. Early configuration failures happen before run-directory creation.
Failures after initialization attempt to preserve diagnostic evidence; individual
artifact writes can fail, so a run directory need not contain every listed file.
## Required Output Files
## Output Files
Reporting is enabled by default. For file transcript output, `errors.yml`,
`recoveries.yml`, and `report.md` go in the transcript output's parent directory.
For stdout transcript output, they go in `reporting.stdout_dir` (default `.`),
overridden by `--report-dir`. `--reporting=false` suppresses all three reports
and removes their path requirements. It does not suppress the stderr summary or
fatal errors. Stdout contains only transcript JSON; file output leaves it empty.
The caller chooses transcript and optional external report destinations using the
[CLI](cli.md#streams-and-exit-status). These are separate from the per-run
diagnostic directory unless the caller deliberately places them inside it.
Choose existing, writable parent directories. Output/report files can be overwritten
by subsequent runs using the same destinations.
A later report or cleanup failure can occur after transcript output was written.
Inspect the process result and available report before treating the files as a
completed run. Retention of the diagnostic directory does not archive caller-selected
output files elsewhere.
Existing report directories and transcript-output parents must be writable.
Audita checks active destinations against input, configuration, and selected
profile files before model dispatch. It preflights file transcript output
without truncating an existing file. With reporting enabled, all three reports
are initialized as `in_progress` before input processing. Output is encoded
fully before publication. Each file is replaced through a temporary file in
its destination directory: transcript first, then `errors.yml`,
`recoveries.yml`, and `report.md`. This is per-file replacement, not a
multi-file transaction. A late report error may leave the transcript published;
the process exits nonzero and attempts failed-status reports. On fatal processing
or cancellation, no completed transcript is published, but partial reports are
finalized where possible. An uncatchable termination can leave `in_progress`.
## Diagnostics Directory
The [diagnostics configuration](config.md#settings)
selects the parent work directory. Runs create children named like
`run-<unix-nanoseconds>`. The runtime creates the work directory if needed.
Report paths may be relative when the supplied work directory is relative;
resolve them using the invocation's working directory.
`--debug` enables detailed evidence independently of reporting. One directory
named `run-<id>` is created under `diagnostics.work_dir` (default `/tmp/audita`)
and retained on success or failure. Without debug, no directory or payload
files are created. The parent is created when needed only for debug. Failure to
write optional debug evidence prints a redacted stderr diagnostic but does not
change correction decisions or the exit status.
| Artifact in the run directory | Purpose |
| --- | --- |
| `source-transcript.json` | Original transcript input |
| `source-transcript-parsed.json` | Parsed source representation |
| `normalized-transcript.json` | Transcript after normalization |
| `normalization-summary.json` | Normalization counters |
| `chunking-summary.json` | Section/token summary |
| `invocation.json` | Invocation metadata |
| `effective-config.redacted.yaml` | Redacted ConfigKit configuration snapshot |
| `config-origins.yaml` | Safe configuration origin and resolution metadata |
| `selected-profiles.yaml` | Safe selected role profiles and model controls |
| `report.json` | Run's [process report](integrations/output-files.md#process-report) |
| `correction-ledger.json` | [Correction dispositions](integrations/output-files.md#correction-ledger) |
| `utilization-diagnostics.json` | [Backend execution and domain timing observations](integrations/output-files.md#utilization-diagnostics) |
| `llm-execution.json` | Safe logical-call and execution event records |
| `error.log` | Failure details when written |
LLM request, response, and error diagnostics are also written within the run
directory. Their nested naming and payloads are debugging details. Prefer paths
provided by the report and its decision/warning records over reconstructing those
paths. The report describes intended artifact locations, not proof that every
write succeeded or that the files still exist after retention.
## Retention Behavior
[Configuration](config.md#settings) owns accepted
retention values and the default. Current lifecycle behavior is:
| Run outcome / selection | Directory outcome |
| --- | --- |
| Failed run | Retained regardless of selection |
| Successful run, `always` | Retained |
| Successful run, `auto` | Retained if corrections were skipped or rejected; otherwise removed |
| Successful run, `never` | Currently retained, the same outcome as `always` |
The name `never` does not currently mean deletion. Warnings alone do not trigger
retention under `auto` unless they also produce skipped or rejected corrections.
An external report outside the run directory survives automatic cleanup, but its
artifact paths can then point to removed files.
## Artifact Handling
Run artifacts can contain full transcripts, glossary-related context, and LLM
prompts/responses. Configured-key redaction does not anonymize those contents.
Choose a work directory whose ownership and permissions fit the transcript data,
and retain or share artifacts intentionally. Credential-supply mechanisms are
owned by [configuration](config.md#credentials).
For a debugging run, select retained diagnostics using the
[retention flag](cli.md#setting-overrides). Copy needed artifacts before
removing completed run directories manually. Avoid deleting a directory while
its run is active. Audita does not provide a separate archival/backup or remote
artifact storage facility.
Debug evidence includes invocation and effective settings, configuration
origins, selected profiles, redacted source and normalized snapshots,
normalization and initial chunk summaries, all captured proposed candidates
and validator decisions in `correction-ledger.json`, execution/utilization
records, and model interaction files. Individual artifact writes can fail; a
directory's existence does not prove every file was written. These files can
contain transcript text and prompt/response payloads. Configured backend
credentials are redacted, but the transcript itself is not anonymized. Protect
and retain the directory according to the sensitivity of the input.
## Failure Inspection and Recovery
1. Read stderr and the exit classification defined by the [CLI](cli.md#streams-and-exit-status).
2. Locate the external report, or the run directory identified by the error when
available. If a late error omits the path, inspect the selected work directory.
3. Read the available report and error log. Use
[report status and error fields](integrations/output-files.md#process-report)
to identify the failed phase, then inspect referenced artifacts.
4. Correct the concrete input, configuration, endpoint, or filesystem issue and
rerun the command with diagnostics retained if further investigation is needed.
Exit `0` means processing and every enabled required output completed. Exit
`1` means cancellation, fatal processing, or required-output failure. Exit `2`
identifies usage or configuration/destination selection errors. Recoverable
model-operation failures appear in `errors.yml` as one record per affected
work unit; they do not automatically make the run fail. `recoveries.yml`
records successful operations that needed retry, repair, or proposal filtering.
`report.md` shows every initial chunk and configured module instance, including
interrupted and unattempted work. Ordinary reports omit transcript excerpts and
full decision ledgers; enable debug when those are needed.
Each invocation starts a new run. There is no implemented resume/checkpoint API;
recovery is a rerun, not continuation from an artifact. Failures during early
configuration or directory creation may leave no report or diagnostic files.
Read stderr and report status first. If a required output failed after a
transcript was published, do not treat the file alone as proof of a completed
run. Fix the input, service, or destination problem and rerun. Audita has no
resume/checkpoint API.