157 lines
7.4 KiB
Markdown
157 lines
7.4 KiB
Markdown
# Audita Validators
|
|
|
|
## Scope
|
|
|
|
This document describes internal validator composition, execution order, and
|
|
decision handling. These registry keys identify implementation components; they
|
|
are not independently selectable through [configuration](../config.md#settings).
|
|
|
|
## Ownership
|
|
|
|
Built-in validator keys and chains:
|
|
- `internal/validators`
|
|
|
|
Shared validator runtime mechanics:
|
|
- `internal/framework/validators`
|
|
|
|
Execution-class metadata:
|
|
- `internal/validators/metadata`
|
|
|
|
## Built-In Validator Keys
|
|
|
|
Deterministic:
|
|
- `proposal_shape`
|
|
- `confidence_threshold`
|
|
- `original_text_presence`
|
|
- `non_empty_resulting_segment` — rejects edits that would leave the segment
|
|
blank, regardless of replacement-span length.
|
|
- `no_effect`
|
|
- `protected_terms`
|
|
|
|
LLM-backed:
|
|
- `spoken_form_plausibility`
|
|
- `meaning_reversal_review`
|
|
- `editorial_review`
|
|
|
|
## Built-In Chains
|
|
|
|
Module chains are defined in `internal/validators/chains.go`.
|
|
Glossary, homophones, spoken_word, and grammar each resolve a fixed ordered chain.
|
|
|
|
## Runtime Execution
|
|
|
|
For each module section:
|
|
1. run deterministic validators;
|
|
2. run LLM-backed validators;
|
|
3. record decisions and compact operation outcomes;
|
|
4. carry only approved proposals forward.
|
|
|
|
Decision cardinality is enforced inside Audita: each candidate proposal must
|
|
receive exactly one outcome per validator it reaches. The runner still rejects
|
|
duplicate, missing, or unknown identities in an internal validator result.
|
|
|
|
## LLM Validator Batching
|
|
|
|
LLM validators:
|
|
- build canonical validation request payloads;
|
|
- batch using `validation.max_prompt_tokens` and the existing domain token estimator;
|
|
- select an embedded prompt ID and pass payload/description variables to the
|
|
role-bound structured client with the validator-decision schema.
|
|
|
|
Oversized single inputs are rejected with a consolidated
|
|
`validation_input_too_large` operation, without a provider call. Recoverable
|
|
model failures make only their batch candidates unavailable while other
|
|
batches continue. PromptKit validates each complete response against the
|
|
validator-decision JSON Schema and performs configured structural repair before
|
|
Audita receives decoded decisions. Each actual batch presents candidates
|
|
numbered from 1, independent of their internal proposal indices. Audita then
|
|
reconciles those local references: one usable entry keeps its
|
|
approval or editorial rejection; a missing or duplicate requested entry makes
|
|
only that candidate unavailable; unexpected indices are ignored. An empty list
|
|
makes all requested candidates unavailable. Decisions and unavailable indices
|
|
map back to internal proposal identities; results are ordered by request identity
|
|
and satisfy strict internal cardinality before the runner advances
|
|
approved candidates. An unavailable candidate never advances as approved.
|
|
Invalid confidence in a decoded test double also makes only its candidate
|
|
unavailable; production schema validation still rejects out-of-range confidence
|
|
before reconciliation. A partially unavailable batch records one failed
|
|
operation with only those requested indices, while a successful batch with
|
|
ignored extras records a decision-filtering recovery. Fatal errors stop the
|
|
chain without returning unchecked candidates as approved. A local result builder
|
|
retains already-collected decisions and operations on early returns; full
|
|
cardinality checks and final sorting remain explicit on completed paths.
|
|
Reconciliation makes no semantic retry or extra model call; guided recovery remains
|
|
[deferred](../roadmap/future.md#domain-aware-response-recovery).
|
|
|
|
Debug request metadata records `proposal_indices` in local-reference order
|
|
(reference 1 is the first entry). Raw decoded responses and ignored indices use
|
|
local references; reconciled decisions and unavailable indices use internal
|
|
proposal indices. Batch token estimates use the same locally numbered payload
|
|
as prompt encoding.
|
|
|
|
## Decision and Rejection Reporting
|
|
|
|
Runner collects decisions, rejected candidates, and operation outcomes. Ledger
|
|
assembly groups decisions by deterministic versus LLM-backed execution metadata.
|
|
The serialized names and fields belong to the
|
|
[report and ledger contract](../integrations/output-files.md); implementation mapping
|
|
is covered by [report assembly](diagnostics-reporting.md).
|
|
|
|
## Changing Validators
|
|
|
|
1. Implement validator behavior under `internal/validators/<validator_key>`;
|
|
update registry and chain wiring in `internal/validators`.
|
|
2. Put shared runtime mechanics in `internal/framework/validators` only when
|
|
needed. Keep deterministic versus LLM-backed execution metadata explicit.
|
|
3. Preserve exposed identities and decision semantics under the
|
|
[architecture policy](../policy/architecture.md#correction-and-concurrency-invariants).
|
|
4. Review validator, registry, chain, batching, malformed-output, diagnostics,
|
|
and decision-cardinality tests; update meaningful coverage under the
|
|
[testing policy](../policy/testing.md).
|
|
5. Update implementation guidance here and any affected external contract in its
|
|
[canonical owner](../policy/documentation.md#canonical-ownership).
|
|
|
|
## Key Tests
|
|
|
|
- `internal/validators/*_test.go`
|
|
- `internal/framework/validators/*_test.go`
|
|
- `internal/framework/processreport/correction_ledger_test.go`
|
|
- `internal/cli/run_test.go`
|
|
|
|
## Supplemental Validation Context
|
|
|
|
Request construction prepares one segment-position lookup and protected vocabulary
|
|
for all candidates. Glossary selection remains candidate-specific, including the
|
|
proposed replacement text.
|
|
|
|
Each candidate includes the immediately preceding and following source segments
|
|
available within its fixed chunk, labeled `preceding` or `following` with speaker,
|
|
text, and categories, plus glossary entries whose names, aliases, or
|
|
recognized plurals appear in its original/resulting text or those neighbors.
|
|
Glossary matching reuses the protected vocabulary; entries retain glossary order.
|
|
Model-facing candidate and neighboring context omit internal segment IDs.
|
|
This supplies evidence such as a troupe definition for a proposed `Jesters`
|
|
correction or an adjacent speaker turn explaining deliberate repetition. A correction
|
|
already clear from its own segment does not require such evidence. No audio,
|
|
out-of-chunk context, or complete transcript is implied.
|
|
|
|
Context is serialized inside the validation item and included in the existing
|
|
payload token estimate. If one enriched item exceeds the configured limit, optional
|
|
neighbors are removed from the end first, then glossary entries from the end,
|
|
until it fits or only required candidate data remains. Required before/after text
|
|
is never truncated. Existing oversized-candidate handling still applies if that
|
|
required data does not fit. Context is supplemental, not a promise that every
|
|
relevant definition or neighboring utterance always fits. This uses the existing
|
|
payload estimator, not a new guarantee about the provider's full context window.
|
|
|
|
## Protected-Name Repetition
|
|
|
|
Spoken-word cleanup may reduce repeated protected-name occurrences when every
|
|
original protected identity still appears in the resulting segment and surviving
|
|
spellings satisfy existing capitalization checks. Full-segment preview supports
|
|
both a replacement of the repeated phrase and deletion of a redundant span. This
|
|
exception only relaxes the deterministic count guard; editorial and meaning-reversal
|
|
validation still decide whether the repetition was accidental and the edit preserves
|
|
meaning. Intentional emphasis and substantive references remain protected by those
|
|
reviews. Other modules retain their existing protection rules.
|