110 lines
4.5 KiB
Markdown
110 lines
4.5 KiB
Markdown
# Audita Validators
|
|
|
|
## Scope
|
|
|
|
This document describes internal validator composition, execution order, and
|
|
decision handling. These registry keys identify implementation components; they
|
|
are not independently selectable through [configuration](../config.md#settings).
|
|
|
|
## Ownership
|
|
|
|
Built-in validator keys and chains:
|
|
- `internal/validators`
|
|
|
|
Shared validator runtime mechanics:
|
|
- `internal/framework/validators`
|
|
|
|
Execution-class metadata:
|
|
- `internal/validators/metadata`
|
|
|
|
## Built-In Validator Keys
|
|
|
|
Deterministic:
|
|
- `proposal_shape`
|
|
- `confidence_threshold`
|
|
- `original_text_presence`
|
|
- `non_empty_resulting_segment` — rejects edits that would leave the segment
|
|
blank, regardless of replacement-span length.
|
|
- `no_effect`
|
|
- `protected_terms`
|
|
|
|
LLM-backed:
|
|
- `spoken_form_plausibility`
|
|
- `meaning_reversal_review`
|
|
- `editorial_review`
|
|
|
|
## Built-In Chains
|
|
|
|
Module chains are defined in `internal/validators/chains.go`.
|
|
Glossary, homophones, spoken_word, and grammar each resolve a fixed ordered chain.
|
|
|
|
## Runtime Execution
|
|
|
|
For each module section:
|
|
1. run deterministic validators;
|
|
2. run LLM-backed validators;
|
|
3. record decisions and compact operation outcomes;
|
|
4. carry only approved proposals forward.
|
|
|
|
Decision cardinality is enforced inside Audita: each candidate proposal must
|
|
receive exactly one outcome per validator it reaches. The runner still rejects
|
|
duplicate, missing, or unknown identities in an internal validator result.
|
|
|
|
## LLM Validator Batching
|
|
|
|
LLM validators:
|
|
- build canonical validation request payloads;
|
|
- batch using `validation.max_prompt_tokens` and the existing domain token estimator;
|
|
- select an embedded prompt ID and pass payload/description variables to the
|
|
role-bound structured client with the validator-decision schema.
|
|
|
|
Oversized single inputs are rejected with a consolidated
|
|
`validation_input_too_large` operation, without a provider call. Recoverable
|
|
model failures make only their batch candidates unavailable while other
|
|
batches continue. PromptKit validates each complete response against the
|
|
validator-decision JSON Schema and performs configured structural repair before
|
|
Audita receives decoded decisions. Audita then reconciles those decisions with
|
|
the correction indices requested in that batch: one usable entry keeps its
|
|
approval or editorial rejection; a missing or duplicate requested entry makes
|
|
only that candidate unavailable; unexpected indices are ignored. An empty list
|
|
makes all requested candidates unavailable. Results are ordered by request
|
|
identity and satisfy strict internal cardinality before the runner advances
|
|
approved candidates. An unavailable candidate never advances as approved.
|
|
Invalid confidence in a decoded test double also makes only its candidate
|
|
unavailable; production schema validation still rejects out-of-range confidence
|
|
before reconciliation. A partially unavailable batch records one failed
|
|
operation with only those requested indices, while a successful batch with
|
|
ignored extras records a decision-filtering recovery. Fatal errors stop the
|
|
chain without returning unchecked candidates as approved. Reconciliation makes
|
|
no semantic retry or extra model call; guided recovery remains
|
|
[deferred](../roadmap/future.md#domain-aware-response-recovery).
|
|
|
|
## Decision and Rejection Reporting
|
|
|
|
Runner collects decisions, rejected candidates, and operation outcomes. Ledger
|
|
assembly groups decisions by deterministic versus LLM-backed execution metadata.
|
|
The serialized names and fields belong to the
|
|
[report and ledger contract](../integrations/output-files.md); implementation mapping
|
|
is covered by [report assembly](diagnostics-reporting.md).
|
|
|
|
## Changing Validators
|
|
|
|
1. Implement validator behavior under `internal/validators/<validator_key>`;
|
|
update registry and chain wiring in `internal/validators`.
|
|
2. Put shared runtime mechanics in `internal/framework/validators` only when
|
|
needed. Keep deterministic versus LLM-backed execution metadata explicit.
|
|
3. Preserve exposed identities and decision semantics under the
|
|
[architecture policy](../policy/architecture.md#correction-and-concurrency-invariants).
|
|
4. Review validator, registry, chain, batching, malformed-output, diagnostics,
|
|
and decision-cardinality tests; update meaningful coverage under the
|
|
[testing policy](../policy/testing.md).
|
|
5. Update implementation guidance here and any affected external contract in its
|
|
[canonical owner](../policy/documentation.md#canonical-ownership).
|
|
|
|
## Key Tests
|
|
|
|
- `internal/validators/*_test.go`
|
|
- `internal/framework/validators/*_test.go`
|
|
- `internal/framework/processreport/correction_ledger_test.go`
|
|
- `internal/cli/run_test.go`
|