Migrate internal architecture docs to docs/internal
This commit is contained in:
79
docs/internal/pipeline.md
Normal file
79
docs/internal/pipeline.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# Audita Internal Pipeline
|
||||
|
||||
## Scope
|
||||
|
||||
This document describes the implemented `audita process` pipeline.
|
||||
|
||||
## Inputs
|
||||
|
||||
Pipeline inputs are:
|
||||
- effective config (`internal/core/config`);
|
||||
- transcript JSON (`internal/core/schema`);
|
||||
- glossary YAML (`internal/core/schema`).
|
||||
|
||||
## Pipeline Phases
|
||||
|
||||
1. Input loading and schema validation
|
||||
- transcript and glossary files are read and parsed.
|
||||
- schema failures stop the run with `transcript_schema` or `glossary_schema`.
|
||||
|
||||
2. Normalization
|
||||
- canonical transcript segments are normalized by configured gap/duration/token settings.
|
||||
- normalization summary artifacts are written.
|
||||
|
||||
3. Chunking
|
||||
- normalized transcript is chunked with configured max/min tokens and target sections.
|
||||
|
||||
4. Module proposal generation
|
||||
- runner executes configured module instances in sequence.
|
||||
- each module proposes corrections per section.
|
||||
- per-section proposal generation can run concurrently.
|
||||
|
||||
5. Validator filtering
|
||||
- validators run on candidate proposals before apply.
|
||||
- deterministic validators run before LLM-backed validators.
|
||||
- LLM validator inputs are batched by max prompt token limit.
|
||||
|
||||
6. Deterministic apply
|
||||
- approved proposals are applied via replacement policy.
|
||||
- applied/skipped/rejected outcomes are recorded.
|
||||
|
||||
7. Output and reporting
|
||||
- final transcript is serialized with selected output schema.
|
||||
- report, utilization diagnostics, and correction ledger are written.
|
||||
- retention policy is applied to run directory.
|
||||
|
||||
## Runner Outputs
|
||||
|
||||
`runner.Run` returns:
|
||||
- final transcript;
|
||||
- per-module results;
|
||||
- utilization diagnostics.
|
||||
|
||||
CLI/reporting then map this into process report and diagnostics artifacts.
|
||||
|
||||
## Failure Behavior
|
||||
|
||||
Representative failure phases include:
|
||||
- `run_dir_creation`
|
||||
- `transcript_read`, `glossary_read`
|
||||
- `transcript_schema`, `glossary_schema`
|
||||
- `chunking`
|
||||
- `runner_setup`, `runner_execution`
|
||||
- `output_schema`, `serialization`, `output_write`, `stdout_write`
|
||||
|
||||
When diagnostics are available, failure stderr includes diagnostics path.
|
||||
|
||||
## Invariants
|
||||
|
||||
- module execution order follows configured module sequence;
|
||||
- proposal/validator nondeterminism is isolated before deterministic apply;
|
||||
- proposal indices are assigned deterministically by section order;
|
||||
- output/report artifacts are generated from run results, not speculative state.
|
||||
|
||||
## Key Tests
|
||||
|
||||
- `internal/framework/runner/runner_test.go`
|
||||
- `internal/framework/proposal_generation/*_test.go`
|
||||
- `internal/cli/run_test.go`
|
||||
- `cmd/audita/main_integration_test.go`
|
||||
Reference in New Issue
Block a user