eric/audita
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Complete Phase 8 deterministic validators

Browse Source
This commit is contained in:
Eric Rakestraw
2026-05-12 00:26:27 +00:00
parent 28fe899aa1
commit aeb31f1c0d
13 changed files with 940 additions and 115 deletions
Download Patch File Download Diff File Expand all files Collapse all files

59
docs/rewrite-notes.md
View File

@@ -47,12 +47,16 @@ Implemented:
- Production runner orchestration over a mutable working transcript.
- Module-level report structures and run-level module summaries.
- CLI runner integration point via injectable module factory/registry (used by deterministic tests).
- Runtime validator models and deterministic validator implementations.
- Validator cardinality enforcement (missing/duplicate/unknown proposal index errors).
- Deterministic validator-chain execution in the production runner.
- Module reports including validator decisions and validator rejections.
- Broad deterministic and CLI/subprocess test coverage for implemented phases through `go test ./...`.
Not yet implemented in runtime pipeline:
- Real correction modules.
- Validator-chain execution.
- Structured LLM client integration.
- LLM-backed validators.
- Prompt/response diagnostics for LLM calls.
- End-to-end transcript polishing behavior.
@@ -161,21 +165,12 @@ Not implemented in Phase 7 (by design):
Current runtime behavior note:
- Default user-facing CLI behavior remains deterministic normalization/chunking output unless test-only module injection is used during tests.
## Remaining work plan
Next recommended phase: **Phase 8 (runtime validator framework and deterministic validators)**.
## Phase 8: Runtime validator framework and deterministic validators
Completed.
### Purpose
Wire validator-chain execution into the runner using deterministic validators first. This phase establishes the safety model before any real LLM proposal generation is introduced.
### Scope
Implement:
- Runtime validator interfaces if the existing contracts need refinement.
- Validation request/result models.
Implemented:
- Runtime validator request/result models in `internal/framework/validators`.
- Deterministic validator reason codes for stable reporting.
- Validator cardinality enforcement:
- one decision per candidate proposal index
- missing indexes are errors
@@ -183,33 +178,25 @@ Implement:
- unknown indexes are errors
- Deterministic validators:
- confidence threshold
- original-text presence
- original-text presence against working transcript
- non-empty correction
- identical text/no-effect rejection
- protected glossary term logic, if it can be implemented deterministically from current glossary schema
- Validator ordering.
- Runner integration so candidate proposals pass through validators before application.
- Module report fields for validator approvals/rejections.
- identical/no-effect rejection
- conservative protected glossary-term guard
- Ordered validator-chain execution in the production runner.
- Runner behavior where only validator-approved proposals proceed to proposal application.
- Module-level reporting of validator decisions and validator rejections, distinct from application-level skips.
- Deterministic fake-module tests covering approvals, rejections, validator order/filtering, and cardinality failure pipeline-stop behavior.
Do not implement:
- LLM-backed validators.
- Real modules.
- Real LLM proposal generation.
Not implemented in Phase 8 (by design):
- LLM-backed validators (Phase 10).
- Structured LLM client implementation or scheduler behavior (Phase 9).
- Real correction modules.
- Prompt/response diagnostics.
- End-to-end transcript polishing.
### Expected behavior at end of phase
## Remaining work plan
Fake modules can generate deterministic proposals, those proposals can be filtered by deterministic validators, and only approved proposals are applied.
### Definition of done
- Validator chains run in the production runner.
- Deterministic validators are implemented and tested.
- Validator cardinality enforcement is tested.
- Rejected proposals appear in module reports with stable reasons.
- Approved proposals are applied through existing proposal application semantics.
- No real LLM calls occur.
- `go test ./...` passes.
Next recommended phase: **Phase 9 (structured LLM client and scheduler infrastructure)**.
## Phase 9: Structured LLM client and scheduler infrastructure