eric/audita
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Document resilient reports and remove obsolete warning material

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-24 17:55:37 +00:00
parent 2f036b0356
commit a770c77a23
26 changed files with 346 additions and 536 deletions
Download Patch File Download Diff File Expand all files Collapse all files

98
docs/internal/pipeline.md
View File

@@ -1,66 +1,44 @@
# Audita Internal Pipeline
## Scope
The [input contract](../integrations/transcript-glossary-files.md) owns input
formats; [configuration](../config.md) owns settings; the
[output contract](../integrations/output-files.md) owns serialized results.
This document describes processing collaborators and state transitions. External
input definitions belong to the [input-file contract](../integrations/transcript-glossary-files.md),
settings to [configuration](../config.md), and serialized results to the
[output-file contract](../integrations/output-files.md).
1. CLI filesystem adapters read transcript and glossary files. Schema parsing
produces source objects; transcript segments enter canonical normalization.
2. Normalization sorts and merges segments using configured time and token
limits, reassigning IDs where needed.
3. The runner partitions the normalized transcript exactly once before its
module loop. Initial section identity, segment membership, and token
estimates stay fixed while text evolves.
4. Module instances execute in configured order. Sections may propose
concurrently, but proposal indices follow section order and response order.
5. Deterministic validators precede LLM-backed validators. Every submitted
candidate receives one decision per validator. Recoverable failed batches
mark their candidates unavailable; successful survivors continue through
all required validators.
6. Only fully approved proposals reach application. Corrections apply in
proposal-index order within their owning section. Existing replacement
policies and apply-time safety checks resolve stale conflicts.
7. CLI encodes the final transcript and publishes active files through the
output session. `framework/processreport` builds one snapshot for YAML,
Markdown, and CLI totals. Debug evidence is captured separately when enabled.
## Inputs
`runner.Run` returns the final working transcript, initial section inventory,
explicit `not_started`/`interrupted`/`completed` work units, compact model
operation outcomes, and per-module results. Candidate payloads are retained
in the returned output only when debug capture is requested. A work unit can
complete with model errors and applied corrections. Recoverable model failures
never trigger shared cancellation. Fatal setup, internal, application, or
parent-cancellation errors cancel peers, join workers, and preserve earlier
committed module text for partial reports; no completed transcript is published.
Pipeline inputs are:
- effective config (`internal/core/config`);
- transcript JSON (`internal/core/schema`);
- glossary YAML (`internal/core/schema`).
Required transcript/report write failures are fatal. Optional debug writes
report safely to stderr without changing decisions or exit status. See
[operations](../operations.md) for file placement and failure handling, and
[architecture policy](../policy/architecture.md#correction-and-concurrency-invariants)
for invariants.
## Pipeline Phases
1. **Input loading:** CLI filesystem adapters read inputs; schema parsing produces
source transcript and glossary objects. Source segments are converted to the
canonical transcript shape before normalization.
2. **Normalization:** the normalizer sorts and merges segments using its configured
time/token rules, reassigns IDs, and produces summary counters.
3. **Chunking:** the chunker partitions the normalized transcript and returns
sections and token estimates, or fails when constraints are impossible.
4. **Proposal generation:** the runner executes configured module instances in
sequence. Sections can generate proposals concurrently; results are collected
for deterministic indexing.
5. **Validation:** deterministic validators run before LLM-backed validators.
LLM validation is batched by prompt budget; candidate decisions feed application.
6. **Application:** `internal/framework/proposals` clones transcript state and
applies approved proposals in index order through replacement-policy checks.
7. **Serialization and reporting:** the CLI encodes the final transcript, maps
runner output into reports, writes artifacts, and delegates retention to
diagnostics. See [report assembly](diagnostics-reporting.md).
## Runner Outputs
`runner.Run` returns:
- final transcript;
- per-module results.
CLI/reporting then map these results into process reports and diagnostics
artifacts. PromptKit execution observations are collected at the role-client
boundary for the new report summary.
## Failure Behavior
CLI orchestration labels failures at the phase boundary. Runner execution failures
can return completed or partial module results, which report assembly preserves.
[Report fields](../integrations/output-files.md#process-report) define how consumers
see these outcomes; [CLI semantics](../cli.md#streams-and-exit-status)
define streams and exits. Diagnostic write failures do not always abort processing.
## Ordering and State
The runner mutates its working transcript between module instances, after collecting
section results and validating proposals. Proposal indexes follow section order;
reports are derived from actual run results. Global requirements are owned by the
[architecture policy](../policy/architecture.md#correction-and-concurrency-invariants).
## Key Tests
- `internal/framework/runner/runner_test.go`
- `internal/framework/proposal_generation/*_test.go`
- `cmd/audita/main_integration_test.go`
Key tests: `internal/framework/runner/*_test.go`,
`internal/framework/validators/*_test.go`, and
`cmd/audita/main_integration_test.go`.