Refactor validators into built-in registry chains

This commit is contained in:
2026-05-13 18:36:46 +00:00
parent 1bc5936681
commit d6126bf52b
21 changed files with 559 additions and 83 deletions

View File

@@ -150,6 +150,7 @@ audita config print-effective --config audita.yml
For full config-file schema and examples, see [`docs/configuration.md`](docs/configuration.md). For full config-file schema and examples, see [`docs/configuration.md`](docs/configuration.md).
For output-schema details, see [`docs/output-schemas.md`](docs/output-schemas.md). For output-schema details, see [`docs/output-schemas.md`](docs/output-schemas.md).
For built-in validator keys and chain definitions, see [`docs/validators.md`](docs/validators.md).
For CLI/process compatibility guarantees, see [`docs/public-contract.md`](docs/public-contract.md). For CLI/process compatibility guarantees, see [`docs/public-contract.md`](docs/public-contract.md).
### Modules ### Modules

View File

@@ -132,6 +132,10 @@ internal/framework/validators/
llm_batching.go llm_batching.go
llm_validators.go llm_validators.go
internal/validators/
registry.go
chains.go
internal/framework/llm/ internal/framework/llm/
openai_compatible_client.go openai_compatible_client.go
scheduler.go scheduler.go
@@ -428,6 +432,63 @@ These primitives are wired into the production runner and report model. The gram
Validator rejections are reported distinctly from proposal-application skips. Validator rejections are reported distinctly from proposal-application skips.
Validator composition is now explicit and registry-backed through `internal/validators`:
- built-in validator registry with stable keys and lookup/build failure for unknown keys;
- built-in chain definitions per production module key;
- production modules resolve validator chains from those built-in definitions.
Stable built-in validator keys:
- deterministic:
- `confidence_threshold`
- `original_text_presence`
- `non_empty_corrected_text`
- `no_effect`
- `protected_terms`
- LLM-backed:
- `spoken_form_plausibility`
- `meaning_reversal_review`
- `editorial_review`
- `grammar_review`
- `spoken_word_review`
Built-in module chains:
- `glossary`:
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
- `protected_terms`
- `non_empty_corrected_text`
- `spoken_form_plausibility`
- `meaning_reversal_review`
- `homophones`:
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
- `protected_terms`
- `non_empty_corrected_text`
- `spoken_form_plausibility`
- `meaning_reversal_review`
- `spoken_word`:
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
- `protected_terms`
- `non_empty_corrected_text`
- `spoken_word_review`
- `meaning_reversal_review`
- `grammar`:
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
- `protected_terms`
- `non_empty_corrected_text`
- `grammar_review`
- `meaning_reversal_review`
1.0 boundary:
- validator chains are built-in and not user-configurable from config/CLI.
- existing threshold and batching knobs remain configurable.
## Implemented LLM-backed validator infrastructure ## Implemented LLM-backed validator infrastructure
`internal/framework/validators` now includes LLM-backed validator support: `internal/framework/validators` now includes LLM-backed validator support:
- typed request/response models for structured LLM validation; - typed request/response models for structured LLM validation;
@@ -563,6 +624,7 @@ Current process reports also include:
- run-level module summary totals and failed module instance metadata. - run-level module summary totals and failed module instance metadata.
- module-level validator decisions and validator rejections. - module-level validator decisions and validator rejections.
- optional decision-level diagnostic artifact paths for validator LLM interactions when available. - optional decision-level diagnostic artifact paths for validator LLM interactions when available.
- stable validator keys in `validator_name` fields for validator decisions/rejections.
- explicit report metadata: - explicit report metadata:
- report schema name; - report schema name;
- report schema version; - report schema version;

View File

@@ -182,3 +182,4 @@ Current guidance:
- prefer file config for baseline behavior; - prefer file config for baseline behavior;
- keep environment variables for secrets/deployment-specific overrides; - keep environment variables for secrets/deployment-specific overrides;
- use CLI flags for per-run overrides. - use CLI flags for per-run overrides.
- validator chains are built-in and are not user-configurable in config.

View File

@@ -10,6 +10,7 @@ This contract covers:
- transcript/glossary input forms - transcript/glossary input forms
- transcript output schema selection - transcript output schema selection
- process report schema metadata - process report schema metadata
- stable validator key identifiers in report/diagnostics records
- diagnostics directory behavior - diagnostics directory behavior
- stdout/stderr and exit-code behavior - stdout/stderr and exit-code behavior
- secret redaction guarantees - secret redaction guarantees
@@ -89,6 +90,8 @@ Current values:
`--report-json` output and diagnostics run-dir `report.json` use the same report schema metadata. `--report-json` output and diagnostics run-dir `report.json` use the same report schema metadata.
Validator decision/rejection records in reports use stable validator keys in `validator_name`.
## Diagnostics directory behavior ## Diagnostics directory behavior
When diagnostics directory creation succeeds, Audita writes run artifacts including: When diagnostics directory creation succeeds, Audita writes run artifacts including:

View File

@@ -511,6 +511,20 @@ Add explicit report schema metadata, for example:
Make validators as easy to reason about as modules. Make validators as easy to reason about as modules.
## Implementation status (2026-05-13)
This workstream is now implemented for built-in validator composition:
- first-class built-in validator registry exists in `internal/validators`;
- stable validator keys are defined and used for built-in chains and runtime validator names;
- explicit built-in module validator-chain definitions are implemented and resolved through registry-backed chain wiring;
- production module constructors use built-in chain resolution instead of ad hoc manual validator construction;
- runner execution preserves deterministic ordering (deterministic validators before LLM-backed validators);
- report validator decision/rejection records now carry stable validator keys.
Current boundary:
- validator chains are built-in and not user-configurable.
- prompt-asset registries, prompt metadata, scheduler utilization diagnostics, correction ledgers, and generated summaries remain planned.
Validators are now central runtime components. They are reused across modules, have deterministic and LLM-backed implementations, produce diagnostics, and affect final correction acceptance. They should therefore have stable identities, registry metadata, and composable chain definitions. Validators are now central runtime components. They are reused across modules, have deterministic and LLM-backed implementations, produce diagnostics, and affect final correction acceptance. They should therefore have stable identities, registry metadata, and composable chain definitions.
## Package structure ## Package structure

111
docs/validators.md Normal file
View File

@@ -0,0 +1,111 @@
# Audita Validators
This document describes Audita's built-in validator registry and module validator chains.
## Scope
Validator chains are built-in runtime behavior.
Current 1.0 boundary:
- built-in validator keys and built-in module chains are stable runtime identifiers;
- thresholds and batching knobs remain configurable where already supported;
- arbitrary user-defined validator chains are deferred.
## Built-in validator keys
### Deterministic validators
- `confidence_threshold`
- checks proposal confidence against module-specific configured threshold.
- `original_text_presence`
- ensures target segment exists and `original_text` exists in current working segment text.
- `non_empty_corrected_text`
- rejects blank/whitespace-only `corrected_text`.
- `no_effect`
- rejects proposals where `original_text == corrected_text`.
- `protected_terms`
- protects glossary-derived terms from unsafe mutations in non-glossary modules.
- glossary stages use glossary-specific protection logic but still report this same stable key.
### LLM-backed validators
- `spoken_form_plausibility`
- checks whether proposed spoken-form change remains plausible in transcript context.
- `meaning_reversal_review`
- checks for likely meaning reversal or semantic contradiction.
- `editorial_review`
- performs conservative editorial safety review.
- `grammar_review`
- checks grammar-stage proposals for grammar-focused safety constraints.
- `spoken_word_review`
- checks spoken-word-stage proposals for dysfluency-cleanup safety constraints.
## Built-in module chains
Current built-in chains resolved from `internal/validators/chains.go`:
- `glossary`
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
- `protected_terms`
- `non_empty_corrected_text`
- `spoken_form_plausibility`
- `meaning_reversal_review`
- `homophones`
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
- `protected_terms`
- `non_empty_corrected_text`
- `spoken_form_plausibility`
- `meaning_reversal_review`
- `spoken_word`
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
- `protected_terms`
- `non_empty_corrected_text`
- `spoken_word_review`
- `meaning_reversal_review`
- `grammar`
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
- `protected_terms`
- `non_empty_corrected_text`
- `grammar_review`
- `meaning_reversal_review`
## Execution semantics
- modules execute serially;
- section proposal work can run concurrently within a module;
- deterministic validators run before LLM-backed validators;
- malformed/missing/duplicate/unknown LLM validator decisions fail safely;
- approved proposals are applied once per module after section work settles.
## Validator rejections vs proposal-application skips
- validator rejection:
- proposal is denied by validator-chain review and appears in validator rejection reporting with validator key and reason code.
- proposal-application skip:
- proposal passed validators but could not be applied under replacement-policy semantics (for example no matching span at apply time).
These are separate outcomes and are reported separately.
## Reporting and diagnostics identity
- report validator decision/rejection entries use stable validator keys in `validator_name`.
- validator LLM diagnostics include validator identity in interaction metadata and structured response schema metadata.
## Configurable knobs that remain supported
- per-module confidence thresholds (`thresholds.*` / equivalent env+CLI overrides)
- validation batching limits (`validation_max_prompt_tokens` / equivalent env+CLI overrides)
- validation LLM model/base URL/timeout/retries/concurrency settings
These tune validator behavior without exposing arbitrary user-defined chains.

View File

@@ -3048,6 +3048,9 @@ func TestRunProcessExplicitHomophonesProtectedGlossaryTermRejected(t *testing.T)
if module.ValidatorRejected[0].ReasonCode != validators.ReasonProtectedGlossaryTerm { if module.ValidatorRejected[0].ReasonCode != validators.ReasonProtectedGlossaryTerm {
t.Fatalf("expected protected glossary term rejection, got %+v", module.ValidatorRejected[0]) t.Fatalf("expected protected glossary term rejection, got %+v", module.ValidatorRejected[0])
} }
if module.ValidatorRejected[0].ValidatorName != "protected_terms" {
t.Fatalf("expected stable validator key protected_terms, got %+v", module.ValidatorRejected[0])
}
} }
func TestRunProcessExplicitHomophonesMalformedLLMOutputFailsWithErrorLog(t *testing.T) { func TestRunProcessExplicitHomophonesMalformedLLMOutputFailsWithErrorLog(t *testing.T) {

View File

@@ -14,6 +14,6 @@
"total_skipped_changes": 1, "total_skipped_changes": 1,
"validator_rejected_reason_codes": ["llm_rejected"], "validator_rejected_reason_codes": ["llm_rejected"],
"expected_proposal_calls": ["grammar:proposal"], "expected_proposal_calls": ["grammar:proposal"],
"expected_validation_calls": ["grammar:section-0000:grammar_only_guard:batch-0000"] "expected_validation_calls": ["grammar:section-0000:grammar_review:batch-0000"]
} }
} }

View File

@@ -59,7 +59,7 @@ func (v OriginalTextPresenceValidator) Validate(_ context.Context, req Request)
type NonEmptyCorrectionValidator struct{} type NonEmptyCorrectionValidator struct{}
func (v NonEmptyCorrectionValidator) Name() string { return "non_empty_correction" } func (v NonEmptyCorrectionValidator) Name() string { return "non_empty_corrected_text" }
func (v NonEmptyCorrectionValidator) Validate(_ context.Context, req Request) (Result, error) { func (v NonEmptyCorrectionValidator) Validate(_ context.Context, req Request) (Result, error) {
decisions := make([]Decision, 0, len(req.CandidateProposal)) decisions := make([]Decision, 0, len(req.CandidateProposal))
@@ -97,7 +97,7 @@ func (v NoEffectValidator) Validate(_ context.Context, req Request) (Result, err
type ProtectedGlossaryTermValidator struct{} type ProtectedGlossaryTermValidator struct{}
func (v ProtectedGlossaryTermValidator) Name() string { return "protected_glossary_terms" } func (v ProtectedGlossaryTermValidator) Name() string { return "protected_terms" }
func (v ProtectedGlossaryTermValidator) Validate(_ context.Context, req Request) (Result, error) { func (v ProtectedGlossaryTermValidator) Validate(_ context.Context, req Request) (Result, error) {
if req.ModuleKey == "glossary" { if req.ModuleKey == "glossary" {
@@ -127,7 +127,7 @@ func (v ProtectedGlossaryTermValidator) Validate(_ context.Context, req Request)
type GlossaryStageProtectedGlossaryTermValidator struct{} type GlossaryStageProtectedGlossaryTermValidator struct{}
func (v GlossaryStageProtectedGlossaryTermValidator) Name() string { func (v GlossaryStageProtectedGlossaryTermValidator) Name() string {
return "glossary_stage_protected_glossary_terms" return "protected_terms"
} }
func (v GlossaryStageProtectedGlossaryTermValidator) Validate(_ context.Context, req Request) (Result, error) { func (v GlossaryStageProtectedGlossaryTermValidator) Validate(_ context.Context, req Request) (Result, error) {

View File

@@ -8,7 +8,7 @@ import (
"gitea.maximumdirect.net/eric/audita/internal/framework/contracts" "gitea.maximumdirect.net/eric/audita/internal/framework/contracts"
"gitea.maximumdirect.net/eric/audita/internal/framework/proposal_generation" "gitea.maximumdirect.net/eric/audita/internal/framework/proposal_generation"
"gitea.maximumdirect.net/eric/audita/internal/framework/proposals" "gitea.maximumdirect.net/eric/audita/internal/framework/proposals"
"gitea.maximumdirect.net/eric/audita/internal/framework/validators" builtinvalidators "gitea.maximumdirect.net/eric/audita/internal/validators"
) )
type Module struct { type Module struct {
@@ -16,25 +16,12 @@ type Module struct {
} }
func New() (*Module, error) { func New() (*Module, error) {
spokenForm, err := validators.NewLLMBackedValidator("spoken_form_plausibility_review", validators.LLMValidatorTypeSpokenFormPlausibility, "") chain, err := builtinvalidators.ResolveBuiltInChain("glossary", builtinvalidators.NewBuiltInRegistry())
if err != nil { if err != nil {
return nil, err return nil, err
} }
meaningReversal, err := validators.NewLLMBackedValidator("meaning_reversal_review", validators.LLMValidatorTypeMeaningReversal, "")
if err != nil {
return nil, err
}
return &Module{ return &Module{
validators: []contracts.Validator{ validators: chain,
validators.NoEffectValidator{},
validators.OriginalTextPresenceValidator{},
validators.ConfidenceThresholdValidator{},
validators.GlossaryStageProtectedGlossaryTermValidator{},
validators.NonEmptyCorrectionValidator{},
spokenForm,
meaningReversal,
},
}, nil }, nil
} }

View File

@@ -132,9 +132,9 @@ func TestGlossaryModuleValidatorChain(t *testing.T) {
"no_effect", "no_effect",
"original_text_presence", "original_text_presence",
"confidence_threshold", "confidence_threshold",
"glossary_stage_protected_glossary_terms", "protected_terms",
"non_empty_correction", "non_empty_corrected_text",
"spoken_form_plausibility_review", "spoken_form_plausibility",
"meaning_reversal_review", "meaning_reversal_review",
} }
if strings.Join(got, ",") != strings.Join(want, ",") { if strings.Join(got, ",") != strings.Join(want, ",") {

View File

@@ -8,7 +8,7 @@ import (
"gitea.maximumdirect.net/eric/audita/internal/framework/contracts" "gitea.maximumdirect.net/eric/audita/internal/framework/contracts"
"gitea.maximumdirect.net/eric/audita/internal/framework/proposal_generation" "gitea.maximumdirect.net/eric/audita/internal/framework/proposal_generation"
"gitea.maximumdirect.net/eric/audita/internal/framework/proposals" "gitea.maximumdirect.net/eric/audita/internal/framework/proposals"
"gitea.maximumdirect.net/eric/audita/internal/framework/validators" builtinvalidators "gitea.maximumdirect.net/eric/audita/internal/validators"
) )
type Module struct { type Module struct {
@@ -16,25 +16,12 @@ type Module struct {
} }
func New() (*Module, error) { func New() (*Module, error) {
grammarOnlyGuard, err := validators.NewLLMBackedValidator("grammar_only_guard", validators.LLMValidatorTypeGrammarReview, "") chain, err := builtinvalidators.ResolveBuiltInChain("grammar", builtinvalidators.NewBuiltInRegistry())
if err != nil { if err != nil {
return nil, err return nil, err
} }
meaningReversal, err := validators.NewLLMBackedValidator("meaning_reversal_review", validators.LLMValidatorTypeMeaningReversal, "")
if err != nil {
return nil, err
}
return &Module{ return &Module{
validators: []contracts.Validator{ validators: chain,
validators.NoEffectValidator{},
validators.OriginalTextPresenceValidator{},
validators.ConfidenceThresholdValidator{},
validators.ProtectedGlossaryTermValidator{},
validators.NonEmptyCorrectionValidator{},
grammarOnlyGuard,
meaningReversal,
},
}, nil }, nil
} }

View File

@@ -134,9 +134,9 @@ func TestGrammarModuleValidatorChain(t *testing.T) {
"no_effect", "no_effect",
"original_text_presence", "original_text_presence",
"confidence_threshold", "confidence_threshold",
"protected_glossary_terms", "protected_terms",
"non_empty_correction", "non_empty_corrected_text",
"grammar_only_guard", "grammar_review",
"meaning_reversal_review", "meaning_reversal_review",
} }
if strings.Join(got, ",") != strings.Join(want, ",") { if strings.Join(got, ",") != strings.Join(want, ",") {

View File

@@ -8,7 +8,7 @@ import (
"gitea.maximumdirect.net/eric/audita/internal/framework/contracts" "gitea.maximumdirect.net/eric/audita/internal/framework/contracts"
"gitea.maximumdirect.net/eric/audita/internal/framework/proposal_generation" "gitea.maximumdirect.net/eric/audita/internal/framework/proposal_generation"
"gitea.maximumdirect.net/eric/audita/internal/framework/proposals" "gitea.maximumdirect.net/eric/audita/internal/framework/proposals"
"gitea.maximumdirect.net/eric/audita/internal/framework/validators" builtinvalidators "gitea.maximumdirect.net/eric/audita/internal/validators"
) )
type Module struct { type Module struct {
@@ -16,25 +16,12 @@ type Module struct {
} }
func New() (*Module, error) { func New() (*Module, error) {
spokenForm, err := validators.NewLLMBackedValidator("spoken_form_plausibility_review", validators.LLMValidatorTypeSpokenFormPlausibility, "") chain, err := builtinvalidators.ResolveBuiltInChain("homophones", builtinvalidators.NewBuiltInRegistry())
if err != nil { if err != nil {
return nil, err return nil, err
} }
meaningReversal, err := validators.NewLLMBackedValidator("meaning_reversal_review", validators.LLMValidatorTypeMeaningReversal, "")
if err != nil {
return nil, err
}
return &Module{ return &Module{
validators: []contracts.Validator{ validators: chain,
validators.NoEffectValidator{},
validators.OriginalTextPresenceValidator{},
validators.ConfidenceThresholdValidator{},
validators.ProtectedGlossaryTermValidator{},
validators.NonEmptyCorrectionValidator{},
spokenForm,
meaningReversal,
},
}, nil }, nil
} }

View File

@@ -148,9 +148,9 @@ func TestHomophonesModuleValidatorChain(t *testing.T) {
"no_effect", "no_effect",
"original_text_presence", "original_text_presence",
"confidence_threshold", "confidence_threshold",
"protected_glossary_terms", "protected_terms",
"non_empty_correction", "non_empty_corrected_text",
"spoken_form_plausibility_review", "spoken_form_plausibility",
"meaning_reversal_review", "meaning_reversal_review",
} }
if strings.Join(got, ",") != strings.Join(want, ",") { if strings.Join(got, ",") != strings.Join(want, ",") {
@@ -276,13 +276,13 @@ func TestHomophonesProtectedGlossaryTermBehaviorRejectsUnsafeCorrection(t *testi
} }
var protectedValidator contracts.Validator var protectedValidator contracts.Validator
for _, v := range module.Validators() { for _, v := range module.Validators() {
if v.Name() == "protected_glossary_terms" { if v.Name() == "protected_terms" {
protectedValidator = v protectedValidator = v
break break
} }
} }
if protectedValidator == nil { if protectedValidator == nil {
t.Fatal("expected protected_glossary_terms validator") t.Fatal("expected protected_terms validator")
} }
working := &schema.Transcript{Segments: []schema.Segment{ working := &schema.Transcript{Segments: []schema.Segment{

View File

@@ -8,7 +8,7 @@ import (
"gitea.maximumdirect.net/eric/audita/internal/framework/contracts" "gitea.maximumdirect.net/eric/audita/internal/framework/contracts"
"gitea.maximumdirect.net/eric/audita/internal/framework/proposal_generation" "gitea.maximumdirect.net/eric/audita/internal/framework/proposal_generation"
"gitea.maximumdirect.net/eric/audita/internal/framework/proposals" "gitea.maximumdirect.net/eric/audita/internal/framework/proposals"
"gitea.maximumdirect.net/eric/audita/internal/framework/validators" builtinvalidators "gitea.maximumdirect.net/eric/audita/internal/validators"
) )
type Module struct { type Module struct {
@@ -16,25 +16,12 @@ type Module struct {
} }
func New() (*Module, error) { func New() (*Module, error) {
spokenWordReview, err := validators.NewLLMBackedValidator("spoken_word_review", validators.LLMValidatorTypeSpokenWordReview, "") chain, err := builtinvalidators.ResolveBuiltInChain("spoken_word", builtinvalidators.NewBuiltInRegistry())
if err != nil { if err != nil {
return nil, err return nil, err
} }
meaningReversal, err := validators.NewLLMBackedValidator("meaning_reversal_review", validators.LLMValidatorTypeMeaningReversal, "")
if err != nil {
return nil, err
}
return &Module{ return &Module{
validators: []contracts.Validator{ validators: chain,
validators.NoEffectValidator{},
validators.OriginalTextPresenceValidator{},
validators.ConfidenceThresholdValidator{},
validators.ProtectedGlossaryTermValidator{},
validators.NonEmptyCorrectionValidator{},
spokenWordReview,
meaningReversal,
},
}, nil }, nil
} }

View File

@@ -136,8 +136,8 @@ func TestSpokenWordModuleValidatorChain(t *testing.T) {
"no_effect", "no_effect",
"original_text_presence", "original_text_presence",
"confidence_threshold", "confidence_threshold",
"protected_glossary_terms", "protected_terms",
"non_empty_correction", "non_empty_corrected_text",
"spoken_word_review", "spoken_word_review",
"meaning_reversal_review", "meaning_reversal_review",
} }
@@ -264,13 +264,13 @@ func TestSpokenWordProtectedGlossaryTermBehaviorRejectsUnsafeCorrection(t *testi
} }
var protectedValidator contracts.Validator var protectedValidator contracts.Validator
for _, v := range module.Validators() { for _, v := range module.Validators() {
if v.Name() == "protected_glossary_terms" { if v.Name() == "protected_terms" {
protectedValidator = v protectedValidator = v
break break
} }
} }
if protectedValidator == nil { if protectedValidator == nil {
t.Fatal("expected protected_glossary_terms validator") t.Fatal("expected protected_terms validator")
} }
working := &schema.Transcript{Segments: []schema.Segment{ working := &schema.Transcript{Segments: []schema.Segment{

View File

@@ -0,0 +1,84 @@
package validators
import (
"fmt"
"gitea.maximumdirect.net/eric/audita/internal/framework/contracts"
frameworkvalidators "gitea.maximumdirect.net/eric/audita/internal/framework/validators"
)
var builtInChains = map[string][]string{
"glossary": {
KeyNoEffect,
KeyOriginalTextPresence,
KeyConfidenceThreshold,
KeyProtectedTerms,
KeyNonEmptyCorrectedText,
KeySpokenFormPlausibility,
KeyMeaningReversalReview,
},
"homophones": {
KeyNoEffect,
KeyOriginalTextPresence,
KeyConfidenceThreshold,
KeyProtectedTerms,
KeyNonEmptyCorrectedText,
KeySpokenFormPlausibility,
KeyMeaningReversalReview,
},
"spoken_word": {
KeyNoEffect,
KeyOriginalTextPresence,
KeyConfidenceThreshold,
KeyProtectedTerms,
KeyNonEmptyCorrectedText,
KeySpokenWordReview,
KeyMeaningReversalReview,
},
"grammar": {
KeyNoEffect,
KeyOriginalTextPresence,
KeyConfidenceThreshold,
KeyProtectedTerms,
KeyNonEmptyCorrectedText,
KeyGrammarReview,
KeyMeaningReversalReview,
},
}
func BuiltInChainKeys(moduleKey string) ([]string, error) {
keys, ok := builtInChains[moduleKey]
if !ok {
return nil, fmt.Errorf("no built-in validator chain for module %q", moduleKey)
}
out := make([]string, len(keys))
copy(out, keys)
return out, nil
}
func ResolveBuiltInChain(moduleKey string, registry *Registry) ([]contracts.Validator, error) {
keys, err := BuiltInChainKeys(moduleKey)
if err != nil {
return nil, err
}
if registry == nil {
registry = NewBuiltInRegistry()
}
out := make([]contracts.Validator, 0, len(keys))
for _, key := range keys {
if moduleKey == "glossary" && key == KeyProtectedTerms {
// Glossary stages preserve current stricter protection semantics while
// reporting the stable protected_terms key.
v := frameworkvalidators.GlossaryStageProtectedGlossaryTermValidator{}
out = append(out, v)
continue
}
v, buildErr := registry.MustBuild(key)
if buildErr != nil {
return nil, buildErr
}
out = append(out, v)
}
return out, nil
}

View File

@@ -0,0 +1,66 @@
package validators_test
import (
"testing"
"gitea.maximumdirect.net/eric/audita/internal/framework/contracts"
"gitea.maximumdirect.net/eric/audita/internal/modules/glossary"
"gitea.maximumdirect.net/eric/audita/internal/modules/grammar"
"gitea.maximumdirect.net/eric/audita/internal/modules/homophones"
"gitea.maximumdirect.net/eric/audita/internal/modules/spoken_word"
builtinvalidators "gitea.maximumdirect.net/eric/audita/internal/validators"
)
func TestProductionModulesUseRegisteredValidatorKeys(t *testing.T) {
registry := builtinvalidators.NewBuiltInRegistry()
keys := map[string]struct{}{}
for _, key := range registry.RegisteredKeys() {
keys[key] = struct{}{}
}
cases := []struct {
name string
build func() ([]contracts.Validator, error)
}{
{name: "glossary", build: func() ([]contracts.Validator, error) {
m, err := glossary.New()
if err != nil {
return nil, err
}
return m.Validators(), nil
}},
{name: "homophones", build: func() ([]contracts.Validator, error) {
m, err := homophones.New()
if err != nil {
return nil, err
}
return m.Validators(), nil
}},
{name: "spoken_word", build: func() ([]contracts.Validator, error) {
m, err := spoken_word.New()
if err != nil {
return nil, err
}
return m.Validators(), nil
}},
{name: "grammar", build: func() ([]contracts.Validator, error) {
m, err := grammar.New()
if err != nil {
return nil, err
}
return m.Validators(), nil
}},
}
for _, tc := range cases {
validators, err := tc.build()
if err != nil {
t.Fatalf("build module %s: %v", tc.name, err)
}
for _, v := range validators {
if _, ok := keys[v.Name()]; !ok {
t.Fatalf("module %s uses unregistered validator key %q", tc.name, v.Name())
}
}
}
}

View File

@@ -0,0 +1,104 @@
package validators
import (
"fmt"
"strings"
"gitea.maximumdirect.net/eric/audita/internal/framework/contracts"
frameworkvalidators "gitea.maximumdirect.net/eric/audita/internal/framework/validators"
)
const (
KeyConfidenceThreshold = "confidence_threshold"
KeyOriginalTextPresence = "original_text_presence"
KeyNonEmptyCorrectedText = "non_empty_corrected_text"
KeyNoEffect = "no_effect"
KeyProtectedTerms = "protected_terms"
KeySpokenFormPlausibility = "spoken_form_plausibility"
KeyMeaningReversalReview = "meaning_reversal_review"
KeyEditorialReview = "editorial_review"
KeyGrammarReview = "grammar_review"
KeySpokenWordReview = "spoken_word_review"
)
type BuiltInValidatorDefinition struct {
Key string
Build func() (contracts.Validator, error)
LLMBacked bool
}
type Registry struct {
definitions map[string]BuiltInValidatorDefinition
}
func NewBuiltInRegistry() *Registry {
defs := []BuiltInValidatorDefinition{
{Key: KeyConfidenceThreshold, Build: func() (contracts.Validator, error) { return frameworkvalidators.ConfidenceThresholdValidator{}, nil }},
{Key: KeyOriginalTextPresence, Build: func() (contracts.Validator, error) { return frameworkvalidators.OriginalTextPresenceValidator{}, nil }},
{Key: KeyNonEmptyCorrectedText, Build: func() (contracts.Validator, error) { return frameworkvalidators.NonEmptyCorrectionValidator{}, nil }},
{Key: KeyNoEffect, Build: func() (contracts.Validator, error) { return frameworkvalidators.NoEffectValidator{}, nil }},
{Key: KeyProtectedTerms, Build: func() (contracts.Validator, error) { return frameworkvalidators.ProtectedGlossaryTermValidator{}, nil }},
{Key: KeySpokenFormPlausibility, LLMBacked: true, Build: func() (contracts.Validator, error) {
return frameworkvalidators.NewLLMBackedValidator(KeySpokenFormPlausibility, frameworkvalidators.LLMValidatorTypeSpokenFormPlausibility, "")
}},
{Key: KeyMeaningReversalReview, LLMBacked: true, Build: func() (contracts.Validator, error) {
return frameworkvalidators.NewLLMBackedValidator(KeyMeaningReversalReview, frameworkvalidators.LLMValidatorTypeMeaningReversal, "")
}},
{Key: KeyEditorialReview, LLMBacked: true, Build: func() (contracts.Validator, error) {
return frameworkvalidators.NewLLMBackedValidator(KeyEditorialReview, frameworkvalidators.LLMValidatorTypeEditorialReview, "")
}},
{Key: KeyGrammarReview, LLMBacked: true, Build: func() (contracts.Validator, error) {
return frameworkvalidators.NewLLMBackedValidator(KeyGrammarReview, frameworkvalidators.LLMValidatorTypeGrammarReview, "")
}},
{Key: KeySpokenWordReview, LLMBacked: true, Build: func() (contracts.Validator, error) {
return frameworkvalidators.NewLLMBackedValidator(KeySpokenWordReview, frameworkvalidators.LLMValidatorTypeSpokenWordReview, "")
}},
}
m := make(map[string]BuiltInValidatorDefinition, len(defs))
for _, def := range defs {
m[def.Key] = def
}
return &Registry{definitions: m}
}
func (r *Registry) Lookup(key string) (BuiltInValidatorDefinition, bool) {
if r == nil {
return BuiltInValidatorDefinition{}, false
}
def, ok := r.definitions[strings.TrimSpace(key)]
return def, ok
}
func (r *Registry) MustBuild(key string) (contracts.Validator, error) {
if r == nil {
return nil, fmt.Errorf("validator registry is nil")
}
def, ok := r.Lookup(key)
if !ok {
return nil, fmt.Errorf("unknown validator key %q", key)
}
v, err := def.Build()
if err != nil {
return nil, fmt.Errorf("build validator %q: %w", key, err)
}
if v == nil {
return nil, fmt.Errorf("build validator %q: returned nil validator", key)
}
if v.Name() != def.Key {
return nil, fmt.Errorf("validator key/name mismatch for %q: got %q", key, v.Name())
}
return v, nil
}
func (r *Registry) RegisteredKeys() []string {
if r == nil {
return nil
}
keys := make([]string, 0, len(r.definitions))
for k := range r.definitions {
keys = append(keys, k)
}
return keys
}

View File

@@ -0,0 +1,79 @@
package validators
import (
"testing"
)
func TestBuiltInRegistryRegistersAllKeys(t *testing.T) {
r := NewBuiltInRegistry()
for _, key := range []string{
KeyConfidenceThreshold,
KeyOriginalTextPresence,
KeyNonEmptyCorrectedText,
KeyNoEffect,
KeyProtectedTerms,
KeySpokenFormPlausibility,
KeyMeaningReversalReview,
KeyEditorialReview,
KeyGrammarReview,
KeySpokenWordReview,
} {
def, ok := r.Lookup(key)
if !ok {
t.Fatalf("expected validator key %q to be registered", key)
}
v, err := def.Build()
if err != nil {
t.Fatalf("build validator %q: %v", key, err)
}
if v.Name() != key {
t.Fatalf("expected validator name %q, got %q", key, v.Name())
}
}
}
func TestBuiltInRegistryUnknownKeyFails(t *testing.T) {
r := NewBuiltInRegistry()
if _, err := r.MustBuild("missing"); err == nil {
t.Fatalf("expected unknown validator key error")
}
}
func TestBuiltInChainKeysResolveForProductionModules(t *testing.T) {
for _, moduleKey := range []string{"glossary", "homophones", "spoken_word", "grammar"} {
keys, err := BuiltInChainKeys(moduleKey)
if err != nil {
t.Fatalf("resolve keys for %q: %v", moduleKey, err)
}
if len(keys) == 0 {
t.Fatalf("expected non-empty chain for %q", moduleKey)
}
}
}
func TestResolveBuiltInChainUsesRegisteredKeys(t *testing.T) {
r := NewBuiltInRegistry()
for _, moduleKey := range []string{"glossary", "homophones", "spoken_word", "grammar"} {
chain, err := ResolveBuiltInChain(moduleKey, r)
if err != nil {
t.Fatalf("resolve chain for %q: %v", moduleKey, err)
}
if len(chain) == 0 {
t.Fatalf("expected non-empty chain for %q", moduleKey)
}
for _, v := range chain {
if v == nil {
t.Fatalf("nil validator in %q chain", moduleKey)
}
}
}
}
func TestBuiltInChainUnknownModuleFails(t *testing.T) {
if _, err := BuiltInChainKeys("unknown"); err == nil {
t.Fatalf("expected unknown module chain failure")
}
if _, err := ResolveBuiltInChain("unknown", NewBuiltInRegistry()); err == nil {
t.Fatalf("expected unknown module chain failure")
}
}