Refactor validators into built-in registry chains

2026-05-13 18:36:46 +00:00
parent 1bc5936681
commit d6126bf52b
21 changed files with 559 additions and 83 deletions
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -132,6 +132,10 @@ internal/framework/validators/
  llm_batching.go
  llm_validators.go

+internal/validators/
+  registry.go
+  chains.go
+
 internal/framework/llm/
  openai_compatible_client.go
  scheduler.go
@@ -428,6 +432,63 @@ These primitives are wired into the production runner and report model. The gram

 Validator rejections are reported distinctly from proposal-application skips.

+Validator composition is now explicit and registry-backed through `internal/validators`:
+- built-in validator registry with stable keys and lookup/build failure for unknown keys;
+- built-in chain definitions per production module key;
+- production modules resolve validator chains from those built-in definitions.
+
+Stable built-in validator keys:
+- deterministic:
+  - `confidence_threshold`
+  - `original_text_presence`
+  - `non_empty_corrected_text`
+  - `no_effect`
+  - `protected_terms`
+- LLM-backed:
+  - `spoken_form_plausibility`
+  - `meaning_reversal_review`
+  - `editorial_review`
+  - `grammar_review`
+  - `spoken_word_review`
+
+Built-in module chains:
+- `glossary`:
+  - `no_effect`
+  - `original_text_presence`
+  - `confidence_threshold`
+  - `protected_terms`
+  - `non_empty_corrected_text`
+  - `spoken_form_plausibility`
+  - `meaning_reversal_review`
+- `homophones`:
+  - `no_effect`
+  - `original_text_presence`
+  - `confidence_threshold`
+  - `protected_terms`
+  - `non_empty_corrected_text`
+  - `spoken_form_plausibility`
+  - `meaning_reversal_review`
+- `spoken_word`:
+  - `no_effect`
+  - `original_text_presence`
+  - `confidence_threshold`
+  - `protected_terms`
+  - `non_empty_corrected_text`
+  - `spoken_word_review`
+  - `meaning_reversal_review`
+- `grammar`:
+  - `no_effect`
+  - `original_text_presence`
+  - `confidence_threshold`
+  - `protected_terms`
+  - `non_empty_corrected_text`
+  - `grammar_review`
+  - `meaning_reversal_review`
+
+1.0 boundary:
+- validator chains are built-in and not user-configurable from config/CLI.
+- existing threshold and batching knobs remain configurable.
+
 ## Implemented LLM-backed validator infrastructure
 `internal/framework/validators` now includes LLM-backed validator support:
 - typed request/response models for structured LLM validation;
@@ -563,6 +624,7 @@ Current process reports also include:
 - run-level module summary totals and failed module instance metadata.
 - module-level validator decisions and validator rejections.
 - optional decision-level diagnostic artifact paths for validator LLM interactions when available.
+- stable validator keys in `validator_name` fields for validator decisions/rejections.
 - explicit report metadata:
  - report schema name;
  - report schema version;
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -182,3 +182,4 @@ Current guidance:
 - prefer file config for baseline behavior;
 - keep environment variables for secrets/deployment-specific overrides;
 - use CLI flags for per-run overrides.
+- validator chains are built-in and are not user-configurable in config.
--- a/docs/public-contract.md
+++ b/docs/public-contract.md
@@ -10,6 +10,7 @@ This contract covers:
 - transcript/glossary input forms
 - transcript output schema selection
 - process report schema metadata
+- stable validator key identifiers in report/diagnostics records
 - diagnostics directory behavior
 - stdout/stderr and exit-code behavior
 - secret redaction guarantees
@@ -89,6 +90,8 @@ Current values:

 `--report-json` output and diagnostics run-dir `report.json` use the same report schema metadata.

+Validator decision/rejection records in reports use stable validator keys in `validator_name`.
+
 ## Diagnostics directory behavior

 When diagnostics directory creation succeeds, Audita writes run artifacts including:
--- a/docs/roadmap.md
+++ b/docs/roadmap.md
@@ -511,6 +511,20 @@ Add explicit report schema metadata, for example:

 Make validators as easy to reason about as modules.

+## Implementation status (2026-05-13)
+
+This workstream is now implemented for built-in validator composition:
+- first-class built-in validator registry exists in `internal/validators`;
+- stable validator keys are defined and used for built-in chains and runtime validator names;
+- explicit built-in module validator-chain definitions are implemented and resolved through registry-backed chain wiring;
+- production module constructors use built-in chain resolution instead of ad hoc manual validator construction;
+- runner execution preserves deterministic ordering (deterministic validators before LLM-backed validators);
+- report validator decision/rejection records now carry stable validator keys.
+
+Current boundary:
+- validator chains are built-in and not user-configurable.
+- prompt-asset registries, prompt metadata, scheduler utilization diagnostics, correction ledgers, and generated summaries remain planned.
+
 Validators are now central runtime components. They are reused across modules, have deterministic and LLM-backed implementations, produce diagnostics, and affect final correction acceptance. They should therefore have stable identities, registry metadata, and composable chain definitions.

 ## Package structure
--- a/docs/validators.md
+++ b/docs/validators.md
@@ -0,0 +1,111 @@
+# Audita Validators
+
+This document describes Audita's built-in validator registry and module validator chains.
+
+## Scope
+
+Validator chains are built-in runtime behavior.
+
+Current 1.0 boundary:
+- built-in validator keys and built-in module chains are stable runtime identifiers;
+- thresholds and batching knobs remain configurable where already supported;
+- arbitrary user-defined validator chains are deferred.
+
+## Built-in validator keys
+
+### Deterministic validators
+
+- `confidence_threshold`
+  - checks proposal confidence against module-specific configured threshold.
+- `original_text_presence`
+  - ensures target segment exists and `original_text` exists in current working segment text.
+- `non_empty_corrected_text`
+  - rejects blank/whitespace-only `corrected_text`.
+- `no_effect`
+  - rejects proposals where `original_text == corrected_text`.
+- `protected_terms`
+  - protects glossary-derived terms from unsafe mutations in non-glossary modules.
+  - glossary stages use glossary-specific protection logic but still report this same stable key.
+
+### LLM-backed validators
+
+- `spoken_form_plausibility`
+  - checks whether proposed spoken-form change remains plausible in transcript context.
+- `meaning_reversal_review`
+  - checks for likely meaning reversal or semantic contradiction.
+- `editorial_review`
+  - performs conservative editorial safety review.
+- `grammar_review`
+  - checks grammar-stage proposals for grammar-focused safety constraints.
+- `spoken_word_review`
+  - checks spoken-word-stage proposals for dysfluency-cleanup safety constraints.
+
+## Built-in module chains
+
+Current built-in chains resolved from `internal/validators/chains.go`:
+
+- `glossary`
+  - `no_effect`
+  - `original_text_presence`
+  - `confidence_threshold`
+  - `protected_terms`
+  - `non_empty_corrected_text`
+  - `spoken_form_plausibility`
+  - `meaning_reversal_review`
+
+- `homophones`
+  - `no_effect`
+  - `original_text_presence`
+  - `confidence_threshold`
+  - `protected_terms`
+  - `non_empty_corrected_text`
+  - `spoken_form_plausibility`
+  - `meaning_reversal_review`
+
+- `spoken_word`
+  - `no_effect`
+  - `original_text_presence`
+  - `confidence_threshold`
+  - `protected_terms`
+  - `non_empty_corrected_text`
+  - `spoken_word_review`
+  - `meaning_reversal_review`
+
+- `grammar`
+  - `no_effect`
+  - `original_text_presence`
+  - `confidence_threshold`
+  - `protected_terms`
+  - `non_empty_corrected_text`
+  - `grammar_review`
+  - `meaning_reversal_review`
+
+## Execution semantics
+
+- modules execute serially;
+- section proposal work can run concurrently within a module;
+- deterministic validators run before LLM-backed validators;
+- malformed/missing/duplicate/unknown LLM validator decisions fail safely;
+- approved proposals are applied once per module after section work settles.
+
+## Validator rejections vs proposal-application skips
+
+- validator rejection:
+  - proposal is denied by validator-chain review and appears in validator rejection reporting with validator key and reason code.
+- proposal-application skip:
+  - proposal passed validators but could not be applied under replacement-policy semantics (for example no matching span at apply time).
+
+These are separate outcomes and are reported separately.
+
+## Reporting and diagnostics identity
+
+- report validator decision/rejection entries use stable validator keys in `validator_name`.
+- validator LLM diagnostics include validator identity in interaction metadata and structured response schema metadata.
+
+## Configurable knobs that remain supported
+
+- per-module confidence thresholds (`thresholds.*` / equivalent env+CLI overrides)
+- validation batching limits (`validation_max_prompt_tokens` / equivalent env+CLI overrides)
+- validation LLM model/base URL/timeout/retries/concurrency settings
+
+These tune validator behavior without exposing arbitrary user-defined chains.