113 lines
6.5 KiB
Markdown
113 lines
6.5 KiB
Markdown
# Output Files and Reports
|
||
|
||
## Corrected Transcript
|
||
|
||
The corrected transcript uses the selected `output.schema`: `bare-segments`
|
||
or `audita-v1`. Its schema and text rules are unchanged by resilience
|
||
reporting. See [configuration](../config.md#settings) for selection and
|
||
[operations](../operations.md#required-output-files) for placement and writes.
|
||
|
||
With reporting enabled, Audita produces three ordinary files:
|
||
|
||
- `errors.yml`: affected work units and run-level failures;
|
||
- `recoveries.yml`: successful logical model operations with repair, service
|
||
retry, or response filtering;
|
||
- `report.md`: a concise overview and complete per-chunk matrices.
|
||
|
||
The YAML documents have Audita-owned schemas `audita-errors` and
|
||
`audita-recoveries`, each with integer `version: 1`. Both contain, in order,
|
||
`schema`, `version`, `run_id`, `status`, `started_at`, `completed_at`, `summary`,
|
||
and an always-present `errors` or `recoveries` sequence. Status is
|
||
`in_progress`, `completed`, or `failed`. Times are UTC RFC3339 strings;
|
||
`completed_at` is null until finalization. A recoverable model error can appear
|
||
with `status: completed`.
|
||
|
||
## Summary Counts
|
||
|
||
Both documents carry the same summary, in this order:
|
||
|
||
| Count | Meaning |
|
||
| --- | --- |
|
||
| `work_units_total` | Configured module instances × initial chunks. |
|
||
| `work_units_started`, `work_units_completed`, `work_units_interrupted`, `work_units_not_started` | Explicit processing states. |
|
||
| `work_units_with_errors` | Distinct units with recoverable operation failures. |
|
||
| `run_errors` | Fatal processing or output failures outside that unit count. |
|
||
| `corrections_applied` | Approved proposals applied, not replacement occurrences. |
|
||
| `llm_operations` | Structured-client invocations, including pre-dispatch failures. |
|
||
| `proposal_responses` | Structurally usable proposal responses. |
|
||
| `recoveries` | Successful operations with one or more recovery reasons. |
|
||
| `proposal_responses_with_filtering` | Proposal responses that filtered at least one defective proposal. |
|
||
|
||
Unknown counts before processing are `null`; completed empty counts are `0`.
|
||
Retries and repair generations are details of one operation, not additional
|
||
operations. Provider token usage, execution timing, and attempt-level evidence
|
||
remain in opt-in debug files.
|
||
|
||
## Error and Recovery Records
|
||
|
||
An `errors` record with `scope: work_unit` identifies zero-based `stage_index`
|
||
and `chunk_index`, module key/instance, initial segment range, applied-correction
|
||
count, and a `failures` sequence. Each failure identifies the operation
|
||
(`proposal`, `validation`, or `validation_input`), optional validator and
|
||
zero-based batch index, safe reason/message, sorted proposal indices when
|
||
known, and execution counts when available. Multiple failed batches in one
|
||
work unit form one error record. `scope: run` instead records phase, safe reason
|
||
and message, optional origin, and output path where relevant. It can describe
|
||
a fatal error or secondary output-write failure without inventing a failed
|
||
model work unit. Raw provider errors and transcript text are absent.
|
||
For `validation_decisions_invalid`, proposal indices identify only candidates
|
||
whose required decisions were unavailable. Other candidates in the same batch
|
||
may still be approved or rejected normally.
|
||
|
||
Each recovery identifies its work unit and successful operation, with sorted
|
||
`reasons`, a safe message, available execution counts, and sorted
|
||
`filtered_proposals` details. One recovery can contain `output_repaired`,
|
||
`service_recovered`, and `proposals_filtered` together. Filtering covers
|
||
defective response proposals; low confidence, protected-term rejection,
|
||
ordinary editorial rejection, and an apply-time conflict caused by an earlier
|
||
accepted correction are not filtering recoveries. Failed operations are errors,
|
||
not recoveries. The initial chunk identity remains stable even if earlier
|
||
modules change its text.
|
||
`validation_decisions_filtered` records unexpected validator decision indices
|
||
ignored from a successful validation batch. It may accompany repair or service
|
||
recovery reasons, but it does not add `filtered_proposals` or increment
|
||
`proposal_responses_with_filtering`. A batch with unavailable requested decisions
|
||
is an error even when unexpected indices were also ignored.
|
||
|
||
## Markdown Overview
|
||
|
||
`report.md` includes run ID, status, corrections, work coverage, error and
|
||
recovery totals, input/output paths, and full work-unit error and recovery
|
||
matrices. Rows show every initial chunk with 1-based chunk number and initial
|
||
segment range; columns follow configured module-instance order, including
|
||
repeats. Completed error cells are `0` or `1`; recovery cells are operation
|
||
counts. `—` means not started and `n*` means interrupted with n facts recorded.
|
||
An empty stage or chunk set is identified as having no work units. Short reason breakdowns count
|
||
records containing each reason; a recovery can have several reasons, so those
|
||
counts need not sum to the recovery total. The report refers to both YAML
|
||
files. It does not include transcript excerpts or correction lists.
|
||
|
||
## Debug-Only Files
|
||
|
||
With `--debug`, `correction-ledger.json` contains proposed candidates and
|
||
their application, rejection, or interruption details. `llm-execution.json`
|
||
holds logical calls and PromptKit execution events; `utilization-diagnostics.json`
|
||
groups backend capacity and timing observations while preserving known and
|
||
complete usage semantics. Interaction request/response artifacts, selected
|
||
profiles, configuration provenance, and input snapshots are also debug-only.
|
||
These files are diagnostic evidence, not an alternate ordinary report schema.
|
||
Each candidate-backed ledger entry includes `proposed_corrected_text`, the
|
||
literal replacement span, even when it is `""` for a deletion. Applied entries
|
||
also include `applied_corrected_text`, likewise `""` for an applied deletion.
|
||
Rejected, unavailable, skipped, and interrupted entries omit the applied field; a module
|
||
failure without a candidate omits both. These fields are replacement spans,
|
||
not full resulting-segment text. Deterministic decision records now identify
|
||
the blank-result guard as `non_empty_resulting_segment` (formerly
|
||
`non_empty_corrected_text`); its `empty_resulting_segment` reason is unchanged.
|
||
Validator decision records carry `disposition`: `approved`, `rejected`, or
|
||
`unavailable`. A missing or duplicate required decision is `unavailable`, not
|
||
an editorial rejection. The candidate ledger uses the same distinction in its
|
||
top-level disposition. Interaction response artifacts retain supplied model
|
||
decisions and the reconciliation result, including unavailable and ignored
|
||
indices; unexpected indices never create candidate ledger entries.
|