eric/audita
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Make module-stage LLM handling resilient and report warnings

Browse Source
This commit is contained in:
Eric Rakestraw
2026-05-23 10:07:06 -05:00
parent a84941d681
commit a3655f5540
43 changed files with 856 additions and 217 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
docs/architecture/validators.md
View File

@@ -8,6 +8,7 @@ For LLM-backed validator prompt asset details, see [`docs/prompts.md`](prompts.m
Built-in validator construction is package-owned under `internal/validators/<validator_key>`:
- `internal/validators/confidence_threshold`
- `internal/validators/proposal_shape`
- `internal/validators/original_text_presence`
- `internal/validators/non_empty_corrected_text`
- `internal/validators/no_effect`
@@ -45,12 +46,14 @@ Current 1.0 boundary:
### Deterministic validators
- `proposal_shape`
- rejects malformed proposal fields before other validators run.
- `confidence_threshold`
- checks proposal confidence against module-specific configured threshold.
- `original_text_presence`
- ensures target segment exists and `original_text` exists in current working segment text.
- `non_empty_corrected_text`
- rejects blank/whitespace-only `corrected_text`.
- rejects proposals whose previewed resulting segment text would be empty or whitespace-only.
- `no_effect`
- rejects proposals where `original_text == corrected_text`.
- `protected_terms`
@@ -71,6 +74,7 @@ Current 1.0 boundary:
Current built-in chains resolved from `internal/validators/chains.go`:
- `glossary`
- `proposal_shape`
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
@@ -80,6 +84,7 @@ Current built-in chains resolved from `internal/validators/chains.go`:
- `meaning_reversal_review`
- `homophones`
- `proposal_shape`
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
@@ -89,6 +94,7 @@ Current built-in chains resolved from `internal/validators/chains.go`:
- `meaning_reversal_review`
- `spoken_word`
- `proposal_shape`
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
@@ -98,6 +104,7 @@ Current built-in chains resolved from `internal/validators/chains.go`:
- `meaning_reversal_review`
- `grammar`
- `proposal_shape`
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
@@ -119,7 +126,9 @@ Both variants preserve existing behavior and report the stable key `protected_te
- modules execute serially;
- section proposal work can run concurrently within a module;
- deterministic validators run before LLM-backed validators;
- malformed/missing/duplicate/unknown LLM validator decisions fail safely;
- malformed module proposal payloads are downgraded to section-scoped module warnings with zero proposals for the affected section rather than module failure;
- malformed/missing/duplicate/unknown LLM validator decisions reject the affected validator batch with warnings instead of failing the module;
- oversized single-proposal validator inputs reject only the affected proposal under that validator;
- approved proposals are applied once per module after section work settles.
## Validator rejections vs proposal-application skips
@@ -128,12 +137,15 @@ Both variants preserve existing behavior and report the stable key `protected_te
- proposal is denied by validator-chain review and appears in validator rejection reporting with validator key and reason code.
- proposal-application skip:
- proposal passed validators but could not be applied under replacement-policy semantics (for example no matching span at apply time).
- module warning:
- malformed proposal-generation payloads and malformed validator batches are recorded in module warning records and diagnostics without writing success stderr.
These are separate outcomes and are reported separately.
## Reporting and diagnostics identity
- report validator decision/rejection entries use stable validator keys in `validator_name`.
- report module results include warning records for malformed module-stage LLM payloads.
- validator LLM diagnostics include validator identity in interaction metadata and structured response schema metadata.
- correction ledger entries include deterministic and LLM validator decision snapshots keyed by the same stable validator keys, and keep validator rejection distinct from application-level skip.