155 lines
5.6 KiB
Markdown
155 lines
5.6 KiB
Markdown
# Audita Validators
|
|
|
|
This document describes Audita's built-in validator registry and module validator chains.
|
|
|
|
For LLM-backed validator prompt asset details, see [`docs/prompts.md`](prompts.md).
|
|
|
|
## Package ownership
|
|
|
|
Built-in validator construction is package-owned under `internal/validators/<validator_key>`:
|
|
- `internal/validators/confidence_threshold`
|
|
- `internal/validators/original_text_presence`
|
|
- `internal/validators/non_empty_corrected_text`
|
|
- `internal/validators/no_effect`
|
|
- `internal/validators/protected_terms`
|
|
- `internal/validators/spoken_form_plausibility`
|
|
- `internal/validators/meaning_reversal_review`
|
|
- `internal/validators/editorial_review`
|
|
- `internal/validators/grammar_review`
|
|
- `internal/validators/spoken_word_review`
|
|
|
|
Registry and chain wiring stay in:
|
|
- `internal/validators/registry.go`
|
|
- `internal/validators/chains.go`
|
|
|
|
Shared validator runtime mechanics stay in `internal/framework/validators`:
|
|
- request/result/decision models
|
|
- decision cardinality helpers
|
|
- protected vocabulary helpers
|
|
- shared LLM validator runtime, batching, and diagnostics helpers
|
|
|
|
Execution classification metadata is defined in `internal/validators/metadata`:
|
|
- `deterministic`
|
|
- `llm_backed`
|
|
|
|
Runner ordering uses this metadata so deterministic validators run before LLM-backed validators without concrete framework type assertions.
|
|
|
|
## Scope
|
|
|
|
Validator chains are built-in runtime behavior.
|
|
|
|
Current 1.0 boundary:
|
|
- built-in validator keys and built-in module chains are stable runtime identifiers;
|
|
- thresholds and batching knobs remain configurable where already supported;
|
|
- arbitrary user-defined validator chains are deferred.
|
|
|
|
## Built-in validator keys
|
|
|
|
### Deterministic validators
|
|
|
|
- `confidence_threshold`
|
|
- checks proposal confidence against module-specific configured threshold.
|
|
- `original_text_presence`
|
|
- ensures target segment exists and `original_text` exists in current working segment text.
|
|
- `non_empty_corrected_text`
|
|
- rejects blank/whitespace-only `corrected_text`.
|
|
- `no_effect`
|
|
- rejects proposals where `original_text == corrected_text`.
|
|
- `protected_terms`
|
|
- protects glossary-derived terms from unsafe mutations in non-glossary modules.
|
|
- glossary stages use glossary-specific protection logic but still report this same stable key.
|
|
|
|
### LLM-backed validators
|
|
|
|
- `spoken_form_plausibility`
|
|
- checks whether proposed spoken-form change remains plausible in transcript context.
|
|
- `meaning_reversal_review`
|
|
- checks for likely meaning reversal or semantic contradiction.
|
|
- `editorial_review`
|
|
- performs conservative editorial safety review.
|
|
- `grammar_review`
|
|
- checks grammar-stage proposals for grammar-focused safety constraints.
|
|
- `spoken_word_review`
|
|
- checks spoken-word-stage proposals for dysfluency-cleanup safety constraints.
|
|
|
|
## Built-in module chains
|
|
|
|
Current built-in chains resolved from `internal/validators/chains.go`:
|
|
|
|
- `glossary`
|
|
- `no_effect`
|
|
- `original_text_presence`
|
|
- `confidence_threshold`
|
|
- `protected_terms`
|
|
- `non_empty_corrected_text`
|
|
- `spoken_form_plausibility`
|
|
- `meaning_reversal_review`
|
|
|
|
- `homophones`
|
|
- `no_effect`
|
|
- `original_text_presence`
|
|
- `confidence_threshold`
|
|
- `protected_terms`
|
|
- `non_empty_corrected_text`
|
|
- `spoken_form_plausibility`
|
|
- `meaning_reversal_review`
|
|
|
|
- `spoken_word`
|
|
- `no_effect`
|
|
- `original_text_presence`
|
|
- `confidence_threshold`
|
|
- `protected_terms`
|
|
- `non_empty_corrected_text`
|
|
- `spoken_word_review`
|
|
- `meaning_reversal_review`
|
|
|
|
- `grammar`
|
|
- `no_effect`
|
|
- `original_text_presence`
|
|
- `confidence_threshold`
|
|
- `protected_terms`
|
|
- `non_empty_corrected_text`
|
|
- `grammar_review`
|
|
- `meaning_reversal_review`
|
|
|
|
## Protected terms construction
|
|
|
|
`protected_terms` has explicit constructors:
|
|
- general constructor used by non-glossary modules through the built-in registry
|
|
- glossary-stage constructor used by glossary chain resolution
|
|
|
|
Both variants preserve existing behavior and report the stable key `protected_terms`.
|
|
|
|
## Execution semantics
|
|
|
|
- modules execute serially;
|
|
- section proposal work can run concurrently within a module;
|
|
- deterministic validators run before LLM-backed validators;
|
|
- malformed/missing/duplicate/unknown LLM validator decisions fail safely;
|
|
- approved proposals are applied once per module after section work settles.
|
|
|
|
## Validator rejections vs proposal-application skips
|
|
|
|
- validator rejection:
|
|
- proposal is denied by validator-chain review and appears in validator rejection reporting with validator key and reason code.
|
|
- proposal-application skip:
|
|
- proposal passed validators but could not be applied under replacement-policy semantics (for example no matching span at apply time).
|
|
|
|
These are separate outcomes and are reported separately.
|
|
|
|
## Reporting and diagnostics identity
|
|
|
|
- report validator decision/rejection entries use stable validator keys in `validator_name`.
|
|
- validator LLM diagnostics include validator identity in interaction metadata and structured response schema metadata.
|
|
- correction ledger entries include deterministic and LLM validator decision snapshots keyed by the same stable validator keys, and keep validator rejection distinct from application-level skip.
|
|
|
|
Prompt assets are unchanged by the validator package-ownership refactor and remain built-in under `internal/prompts`.
|
|
|
|
## Configurable knobs that remain supported
|
|
|
|
- per-module confidence thresholds (`thresholds.*` / equivalent env+CLI overrides)
|
|
- validation batching limits (`validation_max_prompt_tokens` / equivalent env+CLI overrides)
|
|
- validation LLM model/base URL/timeout/retries/concurrency settings
|
|
|
|
These tune validator behavior without exposing arbitrary user-defined chains.
|