Removed the old python code and cleaned up outdated documentation
This commit is contained in:
@@ -1,84 +0,0 @@
|
||||
# Migrating from Python Audita to Go Audita
|
||||
|
||||
This guide describes the operational migration from the legacy Python Audita implementation to the Go Audita implementation in this repository.
|
||||
|
||||
## Status summary
|
||||
|
||||
- The Go CLI is now the primary Audita implementation.
|
||||
- The Go `audita process` command is intended to replace the Python CLI for normal operation.
|
||||
- The Python implementation under [`python/`](../python/) is preserved as a legacy/reference implementation for parity history and troubleshooting context.
|
||||
|
||||
## What changes for operators
|
||||
|
||||
Use the Go binary as the integration target in orchestrators and parent processes.
|
||||
|
||||
Default Go runtime behavior executes the full module sequence:
|
||||
1. `glossary`
|
||||
2. `homophones`
|
||||
3. `glossary`
|
||||
4. `spoken_word`
|
||||
5. `grammar`
|
||||
|
||||
Repeated stages are resolved deterministically in reports (for example `glossary_1`, `glossary_2`).
|
||||
|
||||
## Recommended invocation pattern
|
||||
|
||||
For orchestrated runs, use explicit output files:
|
||||
|
||||
```sh
|
||||
audita process <transcript.json> \
|
||||
--glossary <glossary.yaml> \
|
||||
--output <corrected-transcript.json> \
|
||||
--report-json <report.json>
|
||||
```
|
||||
|
||||
Why:
|
||||
- `--output` keeps stdout empty on success, simplifying subprocess integration.
|
||||
- `--report-json` provides machine-readable run metadata independent of stderr.
|
||||
|
||||
Without `--output`, stdout contains transcript JSON only on successful runs.
|
||||
|
||||
## Diagnostics and retention behavior
|
||||
|
||||
- Each run creates a diagnostics run directory when initialization succeeds.
|
||||
- Failed runs retain diagnostics and include `error.log`.
|
||||
- Retention mode is controlled by `AUDITA_WORK_DIR_RETENTION` / `--work-dir-retention`:
|
||||
- `always`: keep all run directories.
|
||||
- `never`: keep successful run directories.
|
||||
- `auto`: keep failed runs and successful runs with skipped/rejected corrections.
|
||||
|
||||
For subprocess behavior and pipe-handling guidance, see:
|
||||
- [`docs/subprocess-operations.md`](subprocess-operations.md)
|
||||
|
||||
## Parity notes and known differences
|
||||
|
||||
Python-vs-Go parity fixtures and intentional differences are documented in:
|
||||
- [`docs/python-parity.md`](python-parity.md)
|
||||
|
||||
Known open parity gaps are tracked there and should be treated as real gaps, not intentional differences. Current documented gaps include:
|
||||
- broader direct import/use of Python fixture corpus;
|
||||
- a repository-standard Python+Go side-by-side runner command;
|
||||
- wider transcript/glossary corpus coverage.
|
||||
|
||||
## Testing expectations
|
||||
|
||||
- Normal `go test ./...` does not require real LLM credentials.
|
||||
- Normal `go test ./...` does not require Python dependencies.
|
||||
- Deterministic fake-LLM fixtures are used for routine CI-friendly testing.
|
||||
|
||||
## Legacy Python status
|
||||
|
||||
The Python implementation remains in-repo as a legacy/reference baseline. It is not the primary operational path.
|
||||
|
||||
Do not route new production orchestration to Python unless you are doing explicit parity/debug work.
|
||||
|
||||
## Rollout checklist
|
||||
|
||||
Use this checklist when switching an environment from Python invocation to Go invocation:
|
||||
|
||||
1. Run `go test ./...`.
|
||||
2. Build the Go binary (`go build -o ./bin/audita ./cmd/audita`).
|
||||
3. Run one representative fixture through `audita process`.
|
||||
4. Verify `--report-json` output is written and machine-readable.
|
||||
5. Verify failure runs print diagnostics path to stderr and retain diagnostics with `error.log`.
|
||||
6. Update orchestrator configuration to call the Go binary and pass `--output` and `--report-json`.
|
||||
@@ -1,78 +0,0 @@
|
||||
# Python vs Go Parity Notes
|
||||
|
||||
This document tracks parity-fixture coverage and intentional differences between historical Python behavior and the current Audita runtime.
|
||||
|
||||
## Scope
|
||||
|
||||
- Uses deterministic fixture-driven tests in `internal/cli/testdata/parity`.
|
||||
- Uses fake structured LLM responses for proposal generation and LLM validators.
|
||||
- Verifies functional contract fields (module order, instance naming, applied/skipped counts, report status, diagnostics presence).
|
||||
- Does not require real LLM credentials or Python dependencies during `go test ./...`.
|
||||
|
||||
## Intentional Differences
|
||||
|
||||
The following differences are expected and treated as intentional unless they break contract behavior:
|
||||
|
||||
1. JSON formatting and field ordering
|
||||
- Serialized JSON whitespace and object key order may differ.
|
||||
- Parity tests compare JSON semantically, not byte-for-byte.
|
||||
|
||||
2. Diagnostics path values
|
||||
- Absolute run-directory paths, run IDs, and temp directory roots differ by runtime and platform.
|
||||
- Parity checks assert artifact presence/shape, not exact absolute paths.
|
||||
|
||||
3. Time-variant metadata
|
||||
- Timestamps (`started_at`, `completed_at`) and generated run IDs are runtime-specific.
|
||||
- Parity checks ignore exact timestamp/run-id values.
|
||||
|
||||
4. Provider metadata/token accounting
|
||||
- Provider/token usage metadata may vary by adapter behavior and is not asserted as strict parity fields.
|
||||
|
||||
5. Internal adapter implementation details
|
||||
- Go uses its own structured LLM adapter implementation details while preserving the same high-level contract semantics.
|
||||
|
||||
## Current Fixture Coverage
|
||||
|
||||
Current parity fixtures cover:
|
||||
|
||||
- Transcript schema handling failure path.
|
||||
- Glossary schema handling failure path.
|
||||
- Default full module sequence shape:
|
||||
- `glossary_1`, `homophones`, `glossary_2`, `spoken_word`, `grammar`.
|
||||
- Mutable transcript handoff across default stages.
|
||||
- Module-specific behavior inside the default sequence:
|
||||
- glossary correction
|
||||
- homophone-style correction
|
||||
- spoken-word cleanup
|
||||
- grammar cleanup
|
||||
- Protected glossary-term guardrail behavior.
|
||||
- Deterministic validator rejection behavior.
|
||||
- LLM validator decision/rejection behavior.
|
||||
- Application-level skip behavior (`ambiguous_original_text`).
|
||||
- Mid-pipeline failure with partial progress preserved in reports.
|
||||
- Diagnostics artifact presence and secret-redaction checks.
|
||||
|
||||
## Open Parity Gaps (Not Intentional)
|
||||
|
||||
These are known parity expansion opportunities and should not be labeled as intentional compatibility differences:
|
||||
|
||||
1. Broader Python fixture import
|
||||
- The current Go parity fixtures are native fixture cases; they do not yet ingest all existing Python test fixtures directly.
|
||||
|
||||
2. Side-by-side runner command
|
||||
- No repository-standard Python+Go side-by-side parity command is required or enforced yet.
|
||||
|
||||
3. Wider transcript corpus
|
||||
- Current fixtures are representative but not exhaustive across all transcript/glossary edge combinations.
|
||||
|
||||
## How To Extend
|
||||
|
||||
1. Add a new `*.case.json` file under `internal/cli/testdata/parity`.
|
||||
2. Add referenced transcript/glossary/fake-LLM response files.
|
||||
3. Encode deterministic expectations in the case:
|
||||
- module order and instance names
|
||||
- transcript output
|
||||
- applied/skipped/rejected counts
|
||||
- report status and failure metadata
|
||||
- diagnostics artifact presence/redaction markers
|
||||
4. Run `go test ./...`.
|
||||
@@ -1,770 +0,0 @@
|
||||
# Audita Rewrite Notes (Historical)
|
||||
|
||||
This document is a historical record of the Python-to-Go rewrite project.
|
||||
It is retained for engineering context, not as primary runtime guidance.
|
||||
|
||||
## Definition of done for the Go rewrite
|
||||
|
||||
The Go rewrite is complete when both of the following are true:
|
||||
|
||||
1. Feature parity with the initial Python implementation:
|
||||
- `audita process` performs end-to-end transcript polishing, not only deterministic preprocessing.
|
||||
- The default module sequence is implemented and active in the runtime path:
|
||||
- `glossary`
|
||||
- `homophones`
|
||||
- `glossary`
|
||||
- `spoken_word`
|
||||
- `grammar`
|
||||
- Repeated module instances are resolved deterministically, for example `glossary_1` and `glossary_2`.
|
||||
- Real LLM-backed proposal generation is implemented through an OpenAI-compatible structured-output client.
|
||||
- Deterministic validators and LLM-backed validators are implemented and enforced in the runtime path.
|
||||
- Proposal application preserves the safety-first semantics of the Python implementation.
|
||||
- Structured run reports and diagnostics are sufficient for debugging successful runs, skipped corrections, and failures.
|
||||
- The Go CLI can replace the Python CLI in the surrounding orchestration pipeline without downstream contract surprises.
|
||||
|
||||
2. Adherence to the intended application architecture in `docs/architecture.md`:
|
||||
- Sequential module pipeline over a mutable working transcript.
|
||||
- Bounded intra-module concurrency only.
|
||||
- Provider-neutral LLM abstraction.
|
||||
- Separate proposal and validation LLM settings.
|
||||
- Prompt/response diagnostics for LLM stages.
|
||||
- Module-level run reports with applied and skipped changes.
|
||||
- Strict stdout/stderr discipline for subprocess callers.
|
||||
- No hidden service dependency; the CLI remains the primary integration surface.
|
||||
|
||||
## Final status (Phase 19 complete)
|
||||
|
||||
The Go rewrite definition of done is met, with documented parity caveats tracked in `docs/python-parity.md`.
|
||||
|
||||
- Phase 19 is complete.
|
||||
- Go Audita is the active implementation.
|
||||
- The default full module pipeline is implemented (`glossary,homophones,glossary,spoken_word,grammar`).
|
||||
- Parity fixtures and operational hardening coverage exist in normal `go test ./...`.
|
||||
- Python is retained as a legacy/reference implementation and is not the primary operational path.
|
||||
|
||||
## Current implementation status
|
||||
|
||||
The Go rewrite is complete and feature-complete for the intended CLI runtime architecture.
|
||||
|
||||
Implemented:
|
||||
- CLI command surface for `audita process`.
|
||||
- Config/env/flag loading and validation.
|
||||
- Transcript/glossary schema parsing and validation.
|
||||
- Deterministic normalization with summary stats.
|
||||
- Deterministic chunking with summary stats.
|
||||
- Per-run diagnostics directory plus source/normalization/chunking artifacts.
|
||||
- Redacted invocation/effective-config diagnostics metadata artifacts.
|
||||
- Process report output through `--report-json` and run-dir `report.json`.
|
||||
- Report-level diagnostics artifact references.
|
||||
- Framework foundation packages for contracts and proposal preview/apply semantics.
|
||||
- Production runner orchestration over a mutable working transcript.
|
||||
- Module-level report structures and run-level module summaries.
|
||||
- CLI runner integration point via injectable module factory/registry (used by deterministic tests).
|
||||
- Runtime validator models and deterministic validator implementations.
|
||||
- Validator cardinality enforcement (missing/duplicate/unknown proposal index errors).
|
||||
- Deterministic validator-chain execution in the production runner.
|
||||
- Module reports including validator decisions and validator rejections.
|
||||
- LLM-backed validator request/response models and prompt builders.
|
||||
- LLM validator batching by validation prompt-token budget.
|
||||
- LLM validator runtime integration through structured LLM client abstraction and scheduler hooks.
|
||||
- LLM validator prompt/response diagnostics artifact wiring with secret redaction.
|
||||
- Shared LLM proposal-generation helper with structured correction-set parsing.
|
||||
- Deterministic proposal-index assignment for shared proposal generation.
|
||||
- Proposal-generation prompt/response diagnostics artifact wiring with secret redaction.
|
||||
- Production module-registry scaffolding with known key recognition and explicit unsupported/unimplemented errors.
|
||||
- Production grammar module package with Python-aligned prompt intent and guardrails.
|
||||
- Explicit `--modules grammar` runtime path through runner, shared proposal generation, validators, application, reporting, and diagnostics.
|
||||
- Production glossary module package with Python-aligned prompt intent and guardrails.
|
||||
- Glossary-derived deterministic protected-term extraction and validator integration.
|
||||
- Explicit `--modules glossary` runtime path through runner, shared proposal generation, validators, application, reporting, and diagnostics.
|
||||
- Repeated glossary stage support with deterministic instance names (`glossary_1`, `glossary_2`), including mutable working-transcript handoff.
|
||||
- Production homophones module package with Python-aligned prompt intent and guardrails.
|
||||
- Explicit `--modules homophones` runtime path through runner, shared proposal generation, validators, application, reporting, and diagnostics.
|
||||
- Focused multi-module runtime tests for already-implemented interoperability (for example `glossary,homophones`) without claiming full default-pipeline completion.
|
||||
- Production spoken_word module package with Python-aligned prompt intent and guardrails.
|
||||
- Explicit `--modules spoken_word` runtime path through runner, shared proposal generation, validators, application, reporting, and diagnostics.
|
||||
- Focused multi-module runtime tests for already-implemented interoperability (for example `spoken_word,grammar`) without claiming full default-pipeline completion.
|
||||
- All production modules now exist (`glossary`, `homophones`, `spoken_word`, `grammar`) and are integrated into the default full-sequence runtime path.
|
||||
- Default runtime sequence is now active and ordered as:
|
||||
- `glossary`
|
||||
- `homophones`
|
||||
- `glossary`
|
||||
- `spoken_word`
|
||||
- `grammar`
|
||||
- Repeated glossary stages resolve and report deterministically as `glossary_1` and `glossary_2`.
|
||||
- Full-pipeline module reports and run-level summaries aggregate applied/skipped/failed metadata across all module instances.
|
||||
- Mid-pipeline failure reporting preserves partial progress and failed-module metadata.
|
||||
- Full-pipeline diagnostics include proposal/validator prompt-response artifacts with redaction.
|
||||
- Skip-aware retention uses actual module skipped/rejected correction data.
|
||||
- Broad deterministic and CLI/subprocess test coverage for implemented phases through `go test ./...`.
|
||||
- Internal typed structured LLM contract (`StructuredLLMClient.CompleteStructured(ctx, req, out)`).
|
||||
- `internal/framework/llm` instructor-go-backed adapter with:
|
||||
- configurable base URL/model/retries/mode/timeout
|
||||
- optional API key support for local-compatible endpoints
|
||||
- API-key redaction in returned errors
|
||||
- typed structured decode into caller-provided outputs.
|
||||
- LLM scheduler/semaphore infrastructure for bounded concurrency with context-aware acquisition and reliable release.
|
||||
- LLM effective-config resolution helpers:
|
||||
- primary config resolution
|
||||
- validation config inheritance from primary when validation fields are unset
|
||||
- validation override behavior when validation fields are set.
|
||||
- Generic JSON diagnostics primitives for LLM interactions (request metadata, request payload, response payload, optional error payload) with secret redaction.
|
||||
|
||||
Phase 19 documentation and rollout items are implemented:
|
||||
- Go-first README/build/test/usage guidance.
|
||||
- subprocess/orchestrator operational guidance.
|
||||
- Python-to-Go migration guidance.
|
||||
- explicit legacy/reference Python status documentation.
|
||||
|
||||
## Completed phases
|
||||
|
||||
### Phase 1: Go CLI skeleton
|
||||
|
||||
Completed.
|
||||
|
||||
Implemented:
|
||||
- Go module and `cmd/audita` entrypoint.
|
||||
- `audita process` command surface.
|
||||
- Core flags and config wiring.
|
||||
- Subprocess-safe command behavior foundation.
|
||||
|
||||
### Phase 2: Schemas and file I/O
|
||||
|
||||
Completed.
|
||||
|
||||
Implemented:
|
||||
- Transcript parsing for bare arrays and `{ "segments": [...] }` input.
|
||||
- Glossary YAML parsing.
|
||||
- Transcript and glossary validation.
|
||||
- Canonical transcript output serialization.
|
||||
|
||||
### Phase 3: Deterministic normalization
|
||||
|
||||
Completed.
|
||||
|
||||
Implemented:
|
||||
- Chronological sorting.
|
||||
- Same-speaker segment merging.
|
||||
- Gap-sensitive join behavior.
|
||||
- Duration and token-budget merge constraints.
|
||||
- Sequential normalized segment IDs.
|
||||
- Normalization summary stats.
|
||||
|
||||
### Phase 4: Chunking and token estimation
|
||||
|
||||
Completed.
|
||||
|
||||
Implemented:
|
||||
- Deterministic heuristic token estimation.
|
||||
- Contiguous transcript sectioning.
|
||||
- Min/max section token behavior.
|
||||
- Optional target section handling.
|
||||
- Chunking summaries and diagnostics artifacts.
|
||||
|
||||
### Phase 5: Proposal model and application semantics
|
||||
|
||||
Completed.
|
||||
|
||||
Implemented:
|
||||
- Correction proposal and enriched proposal models.
|
||||
- Replacement policies:
|
||||
- `require_unique`
|
||||
- `replace_all`
|
||||
- Safe preview logic.
|
||||
- Deterministic proposal application.
|
||||
- Applied/skipped change records.
|
||||
- Stable skip reasons.
|
||||
|
||||
### Phase 6: Reports and diagnostics
|
||||
|
||||
Completed for the current deterministic runtime scope.
|
||||
|
||||
Implemented:
|
||||
- Per-run diagnostics directory creation.
|
||||
- Source transcript artifacts.
|
||||
- Parsed source transcript artifacts.
|
||||
- Normalized transcript artifact.
|
||||
- Normalization summary artifact.
|
||||
- Chunking summary artifact.
|
||||
- Redacted invocation metadata artifact.
|
||||
- Redacted effective-config artifact.
|
||||
- Run-dir `report.json`.
|
||||
- Optional external `--report-json`.
|
||||
- Failure `error.log`.
|
||||
- Report-level diagnostics artifact references.
|
||||
- Retention decision model with future skipped-correction hook.
|
||||
|
||||
Intentionally deferred:
|
||||
- Module prompt/response diagnostics artifacts are not produced yet because module execution and LLM calls are not implemented in the runtime path.
|
||||
|
||||
### Phase 7: Pipeline runner with deterministic test modules
|
||||
|
||||
Completed.
|
||||
|
||||
Implemented:
|
||||
- `internal/framework/runner` production package with sequential module orchestration.
|
||||
- Deterministic module run-spec resolution and repeated instance naming (`glossary_1`, `glossary_2`, etc.).
|
||||
- Mutable working transcript handoff across module instances.
|
||||
- Proposal application through `internal/framework/proposals`.
|
||||
- Per-module applied/skipped change capture and module status/timing metadata.
|
||||
- Partial-progress return on module failure, with pipeline stop on first failure.
|
||||
- Process report support for module-level results and run-level module summaries.
|
||||
- CLI runtime integration point via injectable module factory/registry, exercised by deterministic fake-module tests.
|
||||
|
||||
Not implemented in Phase 7 (by design):
|
||||
- Real `glossary`, `homophones`, `spoken_word`, `grammar` production modules.
|
||||
- Validators (Phase 8).
|
||||
- Structured LLM calls or scheduler behavior.
|
||||
- Prompt/response diagnostics.
|
||||
- End-to-end transcript polishing.
|
||||
|
||||
Current runtime behavior note:
|
||||
- Default user-facing CLI behavior now executes the full production module sequence unless `--modules` explicitly overrides it.
|
||||
|
||||
## Phase 8: Runtime validator framework and deterministic validators
|
||||
Completed.
|
||||
|
||||
Implemented:
|
||||
- Runtime validator request/result models in `internal/framework/validators`.
|
||||
- Deterministic validator reason codes for stable reporting.
|
||||
- Validator cardinality enforcement:
|
||||
- one decision per candidate proposal index
|
||||
- missing indexes are errors
|
||||
- duplicate indexes are errors
|
||||
- unknown indexes are errors
|
||||
- Deterministic validators:
|
||||
- confidence threshold
|
||||
- original-text presence against working transcript
|
||||
- non-empty correction
|
||||
- identical/no-effect rejection
|
||||
- conservative protected glossary-term guard
|
||||
- Ordered validator-chain execution in the production runner.
|
||||
- Runner behavior where only validator-approved proposals proceed to proposal application.
|
||||
- Module-level reporting of validator decisions and validator rejections, distinct from application-level skips.
|
||||
- Deterministic fake-module tests covering approvals, rejections, validator order/filtering, and cardinality failure pipeline-stop behavior.
|
||||
|
||||
Not implemented in Phase 8 (by design):
|
||||
- LLM-backed validators (Phase 10).
|
||||
- Structured LLM runtime wiring (Phase 9 follow-up).
|
||||
- Real correction modules.
|
||||
- Prompt/response diagnostics runtime wiring.
|
||||
- End-to-end transcript polishing.
|
||||
|
||||
## Phase 9: Structured LLM client and scheduler infrastructure
|
||||
|
||||
### Status
|
||||
|
||||
Completed for Phase 9 infrastructure scope.
|
||||
|
||||
Implemented in this phase so far:
|
||||
- Added internal structured LLM contract support for caller-provided typed outputs.
|
||||
- Added `internal/framework/llm` adapter backed by `github.com/jxnl/instructor-go`.
|
||||
- Confirmed OpenAI-compatible base URL support through the adapter path.
|
||||
- Added adapter unit tests for model/base URL handling, retries, context cancellation, optional API key behavior, and error redaction.
|
||||
|
||||
Explicitly deferred from Phase 9 into later phases:
|
||||
- Runtime wiring in runner/module infrastructure (without introducing real modules yet).
|
||||
- Wiring prompt/response diagnostics primitives into future module/validator call sites.
|
||||
- Wiring effective primary/validation LLM config resolution into runtime LLM call sites.
|
||||
|
||||
### Purpose
|
||||
|
||||
Implement the provider-neutral LLM infrastructure needed by both proposal generation and LLM-backed validators, without yet implementing real modules.
|
||||
|
||||
### Scope completed in this phase
|
||||
|
||||
Implemented:
|
||||
- Provider-neutral internal structured LLM contract with caller-provided typed output decoding.
|
||||
- OpenAI-compatible structured-output adapter (`instructor-go`) with:
|
||||
- configurable base URL and model
|
||||
- optional API key behavior
|
||||
- retry budget
|
||||
- timeout-aware HTTP client handling
|
||||
- context cancellation propagation
|
||||
- structured response decoding into caller-provided typed targets
|
||||
- redacted error surfaces.
|
||||
- Scheduler/semaphore infrastructure for bounded backend concurrency with context-aware acquisition and reliable permit release.
|
||||
- Effective LLM config resolution helpers for:
|
||||
- primary LLM settings
|
||||
- validation inheritance from primary when unset
|
||||
- validation overrides when set.
|
||||
- Generic JSON diagnostics primitives for request metadata, request payload, response payload, and optional error payload, with secret redaction.
|
||||
- Unit tests covering adapter behavior, scheduler behavior, effective config resolution, and diagnostics redaction/JSON validity.
|
||||
|
||||
Do not implement:
|
||||
- Real correction modules.
|
||||
- LLM-backed validators.
|
||||
- Prompt text for domain modules.
|
||||
- End-to-end transcript polishing.
|
||||
|
||||
### Expected behavior at end of phase
|
||||
|
||||
At the end of Phase 9, the codebase had tested LLM infrastructure primitives, while default CLI runtime behavior remained deterministic preprocessing/reporting because real modules were not implemented yet.
|
||||
|
||||
### Definition of done status
|
||||
|
||||
Met:
|
||||
- Structured LLM client infrastructure is implemented and tested.
|
||||
- OpenAI-compatible structured-output client exists and is tested.
|
||||
- Scheduler enforces configured concurrency and is tested.
|
||||
- Primary and validation effective config resolution exists and is tested.
|
||||
- Prompt/response diagnostics primitives exist and are tested.
|
||||
- API keys are redacted in LLM adapter errors and diagnostics artifacts; config/report diagnostics redaction remains in place.
|
||||
- No real module behavior was introduced.
|
||||
- `go test ./...` passes.
|
||||
|
||||
## Phase 10: LLM-backed validators
|
||||
|
||||
Completed.
|
||||
|
||||
Implemented:
|
||||
- LLM-backed validator request/response models in `internal/framework/validators`.
|
||||
- Prompt builders for:
|
||||
- spoken-form plausibility
|
||||
- meaning reversal detection
|
||||
- editorial review
|
||||
- grammar review
|
||||
- spoken-word review
|
||||
- Deterministic batching by `validation_max_prompt_tokens` with stable ordering and no drop/dup behavior.
|
||||
- LLM validator execution through the internal structured client abstraction (no direct provider calls in validator code).
|
||||
- Scheduler/concurrency hooks for LLM validator calls.
|
||||
- Prompt/response diagnostics artifact writing for LLM validator batches using Phase 9 diagnostics primitives.
|
||||
- Secret redaction in validator LLM diagnostics artifacts.
|
||||
- Strict structured-response safety and cardinality checks (missing/duplicate/unknown indexes fail closed).
|
||||
- Runner/report integration so LLM validator decisions and rejections appear in module reports.
|
||||
- Fake-module and fake-client tests for approval/rejection, malformed output, cardinality errors, batching, scheduler usage, and diagnostics redaction.
|
||||
|
||||
Not implemented in Phase 10 (by design):
|
||||
- Real correction modules (`glossary`, `homophones`, `spoken_word`, `grammar`).
|
||||
- Real module implementation and full runtime wiring (Phase 12+).
|
||||
- Domain proposal prompts.
|
||||
- Default CLI end-to-end transcript polishing behavior.
|
||||
|
||||
## Phase 11: Shared LLM proposal generation framework and module registry
|
||||
|
||||
Completed.
|
||||
|
||||
Implemented:
|
||||
- Shared proposal-generation package `internal/framework/proposal_generation`.
|
||||
- Reusable request model for proposal generation including:
|
||||
- module key/instance
|
||||
- replacement policy
|
||||
- working transcript context
|
||||
- optional section metadata
|
||||
- glossary/config context
|
||||
- diagnostics context
|
||||
- injected structured LLM client/scheduler dependencies.
|
||||
- Structured correction-set response model and parsing into existing proposal models:
|
||||
- `proposals.CorrectionProposal`
|
||||
- `proposals.EnrichedCorrectionProposal`.
|
||||
- Deterministic proposal-index assignment via caller-provided start index.
|
||||
- Proposal-generation diagnostics artifact writing using generic LLM diagnostics primitives with secret redaction.
|
||||
- Scheduler-aware proposal generation through the internal LLM scheduler interface.
|
||||
- Production module-registry scaffolding in `internal/framework/modules` with:
|
||||
- known module-key recognition for `glossary`, `homophones`, `spoken_word`, `grammar`
|
||||
- constructor registration and dependency-injection path
|
||||
- explicit unsupported and recognized-but-unimplemented module errors.
|
||||
- Runner/CLI injection-path tests showing shared proposal generation can flow through runner validation/application semantics using fake modules/clients.
|
||||
|
||||
Not implemented in Phase 11 (by design):
|
||||
- Real production `glossary`, `homophones`, and `spoken_word` modules.
|
||||
- Domain proposal prompts for production modules.
|
||||
- Default CLI end-to-end transcript polishing behavior.
|
||||
|
||||
## Phase 12: Grammar module
|
||||
|
||||
Completed.
|
||||
|
||||
Implemented:
|
||||
- Production grammar module package in `internal/modules/grammar`.
|
||||
- Grammar prompt builder aligned to Python intent and constrained to punctuation/capitalization/spacing/article cleanup.
|
||||
- Grammar proposal generation through shared `internal/framework/proposal_generation` using `contracts.StructuredLLMClient`.
|
||||
- Scheduler-aware grammar proposal generation through existing scheduler hooks.
|
||||
- Grammar replacement policy `require_unique` (matching Python behavior).
|
||||
- Grammar validator chain using existing deterministic and LLM-backed validator infrastructure.
|
||||
- Grammar confidence threshold enforcement through existing config + confidence-threshold validator behavior.
|
||||
- Explicit runtime support for `--modules grammar` through normalization, chunking, runner, proposal generation, validation, application, and reporting.
|
||||
- Prompt/response diagnostics artifacts for grammar proposal + validator interactions with secret redaction.
|
||||
- Module-level reports for grammar including validator decisions/rejections, applied changes, and skipped changes.
|
||||
- CLI/runtime fake-client tests for approved proposals, validator rejection, application skips, diagnostics, failure/error.log behavior, and report outputs (`--report-json` and run-dir `report.json`).
|
||||
|
||||
Not implemented in Phase 12 (by design):
|
||||
- Production `glossary`, `homophones`, and `spoken_word` modules.
|
||||
- Full default module sequence execution as a feature-complete claim.
|
||||
|
||||
## Phase 13: Glossary module and protected-term behavior
|
||||
|
||||
Completed.
|
||||
|
||||
Implemented:
|
||||
- Production glossary module package in `internal/modules/glossary`.
|
||||
- Glossary prompt builder aligned to Python intent and constrained to glossary-supported domain/acoustic corrections.
|
||||
- Prompt context using glossary names, aliases, categories, summaries, and plural forms where available.
|
||||
- Glossary proposal generation through shared `internal/framework/proposal_generation` using `contracts.StructuredLLMClient`.
|
||||
- Scheduler-aware glossary proposal generation through existing scheduler hooks.
|
||||
- Glossary replacement policy `replace_all` (matching Python behavior).
|
||||
- Glossary validator chain using existing deterministic and LLM-backed validators.
|
||||
- Glossary confidence threshold enforcement through existing config + confidence-threshold validator behavior.
|
||||
- Deterministic glossary-derived protected-term extraction (`internal/framework/validators/protected_terms.go`) from names, aliases, and plural forms, with stable deduplicated ordering.
|
||||
- Protected-term validator behavior remaining available to non-glossary modules via existing deterministic validators.
|
||||
- Explicit runtime support for `--modules glossary` through normalization, chunking, runner, proposal generation, validation, application, and reporting.
|
||||
- Repeated glossary-stage support (`--modules glossary,glossary`) with deterministic instance naming and mutable working-transcript handoff across stages.
|
||||
- Prompt/response diagnostics artifacts for glossary proposal + validator interactions with secret redaction.
|
||||
- Module-level reports for glossary including generated proposals, validator decisions/rejections, applied changes, and application skips.
|
||||
- CLI/runtime fake-client tests for approved proposals, validator rejection, application skips, repeated stages, diagnostics, failure/error.log behavior, and report outputs (`--report-json` and run-dir `report.json`).
|
||||
|
||||
Not implemented in Phase 13 (by design):
|
||||
- Production `homophones` and `spoken_word` modules.
|
||||
- Full default module sequence execution as a feature-complete claim.
|
||||
|
||||
## Phase 14: Homophones module
|
||||
|
||||
Completed.
|
||||
|
||||
Implemented:
|
||||
- Production homophones module package in `internal/modules/homophones`.
|
||||
- Homophones prompt builder aligned to Python intent and constrained to conservative homophone/near-homophone/mistranscription corrections.
|
||||
- Prompt context using glossary/protected-term information (names, aliases, plurals where present) to avoid damaging known domain terms.
|
||||
- Homophones proposal generation through shared `internal/framework/proposal_generation` using `contracts.StructuredLLMClient`.
|
||||
- Scheduler-aware homophones proposal generation through existing scheduler hooks.
|
||||
- Homophones replacement policy `require_unique` (matching Python behavior).
|
||||
- Homophones validator chain using existing deterministic and LLM-backed validators.
|
||||
- Homophones confidence threshold enforcement through existing config + confidence-threshold validator behavior.
|
||||
- Protected-term guardrails remaining active for homophones via existing deterministic validators.
|
||||
- Explicit runtime support for `--modules homophones` through normalization, chunking, runner, proposal generation, validation, application, and reporting.
|
||||
- Prompt/response diagnostics artifacts for homophones proposal + validator interactions with secret redaction.
|
||||
- Module-level reports for homophones including generated proposals, validator decisions/rejections, applied changes, and application skips.
|
||||
- CLI/runtime fake-client tests for approved proposals, validator rejection, application skips, diagnostics, protected-term rejection behavior, failure/error.log behavior, and report outputs (`--report-json` and run-dir `report.json`).
|
||||
- Focused interoperability tests for already-implemented module combinations (for example `glossary,homophones`) to verify working-transcript handoff and guardrails without claiming full default-sequence parity.
|
||||
|
||||
Not implemented in Phase 14 (by design):
|
||||
- Production `spoken_word` module.
|
||||
- Full default module sequence execution as a feature-complete claim.
|
||||
|
||||
## Phase 15: Spoken-word module
|
||||
|
||||
Completed.
|
||||
|
||||
Implemented:
|
||||
- Production spoken_word module package in `internal/modules/spoken_word`.
|
||||
- Spoken_word prompt builder aligned to Python intent and constrained to conservative dysfluency cleanup.
|
||||
- Prompt context using glossary/protected-term information (names, aliases, plurals where present) to avoid damaging known domain terms.
|
||||
- Strong prompt guardrails preserving meaning, intent, speaker voice, named entities, game/domain terms, and substantive content.
|
||||
- Explicit prompt guardrails against summarization, style rewriting, grammar-only cleanup, punctuation-only cleanup, invention, event reordering, and certainty inflation.
|
||||
- Spoken_word proposal generation through shared `internal/framework/proposal_generation` using `contracts.StructuredLLMClient`.
|
||||
- Scheduler-aware spoken_word proposal generation through existing scheduler hooks.
|
||||
- Spoken_word replacement policy `require_unique` (matching Python behavior).
|
||||
- Spoken_word validator chain using existing deterministic and LLM-backed validators.
|
||||
- Strong semantic guardrails in runtime validator chain through existing LLM-backed validators (`spoken_word_review`, `meaning_reversal_review`).
|
||||
- Spoken-word confidence threshold enforcement through existing config + confidence-threshold validator behavior.
|
||||
- Protected-term guardrails remaining active for spoken_word via existing deterministic validators.
|
||||
- Explicit runtime support for `--modules spoken_word` through normalization, chunking, runner, proposal generation, validation, application, and reporting.
|
||||
- Prompt/response diagnostics artifacts for spoken_word proposal + validator interactions with secret redaction.
|
||||
- Module-level reports for spoken_word including generated proposals, validator decisions/rejections, applied changes, and application skips.
|
||||
- CLI/runtime fake-client tests for approved cleanup, validator rejection, meaning-changing rejection, application skips, diagnostics, protected-term rejection behavior, failure/error.log behavior, and report outputs (`--report-json` and run-dir `report.json`).
|
||||
- Focused interoperability tests for already-implemented module combinations (for example `spoken_word,grammar`) to verify working-transcript handoff and guardrails without claiming full default-sequence parity.
|
||||
|
||||
Not implemented in Phase 15 (by design):
|
||||
- Full default module sequence execution as a feature-complete claim.
|
||||
|
||||
## Phase 16: Default full pipeline integration
|
||||
|
||||
Completed.
|
||||
|
||||
### Purpose
|
||||
|
||||
Enable and harden the full default module sequence in the Go runtime path.
|
||||
|
||||
### Scope
|
||||
|
||||
Implement:
|
||||
- Default runtime sequence:
|
||||
- `glossary`
|
||||
- `homophones`
|
||||
- `glossary`
|
||||
- `spoken_word`
|
||||
- `grammar`
|
||||
- End-to-end execution through all real modules.
|
||||
- Accurate resolved module instance names.
|
||||
- Module-level report aggregation.
|
||||
- Full run-level applied/skipped summaries.
|
||||
- Skip-aware retention behavior using actual skipped correction data.
|
||||
- Failure behavior with partial module progress.
|
||||
- PipelineRunError or equivalent partial-progress error type if not already implemented.
|
||||
- Diagnostics for each module instance.
|
||||
- Tests for successful full pipeline using fake LLM.
|
||||
- Tests for mid-pipeline failure preserving partial report and diagnostics.
|
||||
|
||||
Do not implement:
|
||||
- Python archive removal.
|
||||
- Side-by-side rollout tooling beyond what is needed for tests.
|
||||
|
||||
### Expected behavior at end of phase
|
||||
|
||||
Running `audita process transcript.json --glossary glossary.yaml --output corrected.json` should execute the full Go module pipeline and produce a corrected transcript.
|
||||
|
||||
### Definition of done
|
||||
|
||||
- Default module sequence runs end-to-end.
|
||||
- All real modules participate in runtime path.
|
||||
- Reports include all module instances.
|
||||
- Applied/skipped changes are aggregated at run level.
|
||||
- Failed runs retain useful partial reports and diagnostics.
|
||||
- `auto` retention keeps successful runs with skipped corrections.
|
||||
- CLI stdout/stderr behavior remains subprocess-safe.
|
||||
- `go test ./...` passes without requiring external LLM credentials.
|
||||
|
||||
### Phase 16 completion status
|
||||
|
||||
Implemented:
|
||||
- Normal `audita process` runs without `--modules` now execute the full sequence:
|
||||
- `glossary`
|
||||
- `homophones`
|
||||
- `glossary`
|
||||
- `spoken_word`
|
||||
- `grammar`
|
||||
- Explicit `--modules` still overrides the default sequence.
|
||||
- Repeated glossary stages are deterministic (`glossary_1`, `glossary_2`) and reported distinctly.
|
||||
- Full-pipeline module reports aggregate applied changes, application skips, validator rejections, and failed-module metadata.
|
||||
- Full-pipeline diagnostics include prompt/response artifacts for module proposal generation and validator LLM interactions with secret redaction.
|
||||
- Mid-pipeline failure preserves partial module progress in reports and retains diagnostics + `error.log`.
|
||||
- Auto-retention keeps successful runs with actual skipped/rejected corrections and always retains failed runs.
|
||||
|
||||
Intentionally deferred:
|
||||
- Phase 17 parity fixture suite.
|
||||
- Phase 18 operational hardening.
|
||||
- Phase 19 rollout/Python retirement work.
|
||||
|
||||
## Phase 17: Python parity fixture suite
|
||||
|
||||
Completed.
|
||||
|
||||
### Purpose
|
||||
|
||||
Establish confidence that the Go implementation matches the behavior and safety posture of the initial Python implementation.
|
||||
|
||||
### Scope
|
||||
|
||||
Implement:
|
||||
- Parity fixtures based on representative Python-era inputs and expected behaviors.
|
||||
- Fake LLM response fixtures where exact deterministic behavior is required.
|
||||
- Golden tests for:
|
||||
- transcript schema handling
|
||||
- glossary schema handling
|
||||
- normalization
|
||||
- chunking
|
||||
- proposal application
|
||||
- validator behavior
|
||||
- module reports
|
||||
- diagnostics artifacts
|
||||
- default pipeline shape
|
||||
- Comparison tests or scripts that can run Python and Go side by side where practical.
|
||||
- Documentation of intentional differences between Python and Go.
|
||||
|
||||
Do not require:
|
||||
- Real LLM credentials for normal automated tests.
|
||||
- Exact nondeterministic natural-language output equivalence across Python and Go.
|
||||
|
||||
### Expected behavior at end of phase
|
||||
|
||||
The repository has a durable test suite demonstrating that the Go implementation preserves the functional contract of the Python implementation.
|
||||
|
||||
### Definition of done
|
||||
|
||||
- Representative parity fixtures exist.
|
||||
- Golden tests cover deterministic behavior.
|
||||
- Fake LLM tests cover full pipeline behavior.
|
||||
- Intentional differences from Python are documented.
|
||||
- Unintentional compatibility breaks are fixed.
|
||||
- `go test ./...` passes.
|
||||
|
||||
### Phase 17 completion status
|
||||
|
||||
Implemented:
|
||||
- Durable parity fixture harness in Go test path (`internal/cli/parity_test.go`).
|
||||
- Representative parity fixture corpus under `internal/cli/testdata/parity`.
|
||||
- Fake LLM proposal/validator fixtures driving deterministic parity tests.
|
||||
- Default full-pipeline parity coverage including sequence/order and repeated glossary naming:
|
||||
- `glossary_1`
|
||||
- `homophones`
|
||||
- `glossary_2`
|
||||
- `spoken_word`
|
||||
- `grammar`
|
||||
- Parity checks that are strict for deterministic contract fields:
|
||||
- transcript content
|
||||
- module order and instance names
|
||||
- applied/skipped/rejected counts
|
||||
- report status and failed-module metadata
|
||||
- Parity checks that ignore nondeterministic metadata fields:
|
||||
- timestamps
|
||||
- run IDs
|
||||
- temp/absolute paths
|
||||
- provider token usage details
|
||||
- Documentation of intentional Python-vs-Go differences and honest open parity gaps in `docs/python-parity.md`.
|
||||
- Normal `go test ./...` path remains independent of real LLM credentials and Python dependencies.
|
||||
|
||||
Intentionally deferred:
|
||||
- Phase 18 operational hardening and subprocess integration expansion.
|
||||
- Phase 19 rollout and Python retirement work.
|
||||
|
||||
## Phase 18: Operational hardening and subprocess integration
|
||||
|
||||
### Purpose
|
||||
|
||||
Harden the Go binary for use as the production Audita implementation in the surrounding application suite.
|
||||
|
||||
### Scope
|
||||
|
||||
Implement:
|
||||
- Additional subprocess tests for large transcripts.
|
||||
- Timeout/cancellation tests.
|
||||
- Failure-mode tests for unreadable inputs, unwritable outputs, malformed LLM responses, and backend failures.
|
||||
- Clear stderr summaries pointing to diagnostics.
|
||||
- Review of all output paths and file-close behavior.
|
||||
- Review of all secret redaction paths.
|
||||
- Review of LLM retry and timeout behavior.
|
||||
- Documentation for production use from orchestrators such as Narratio.
|
||||
|
||||
### Expected behavior at end of phase
|
||||
|
||||
The Go binary should be safe to call from other Go applications and should not reproduce the Python subprocess/stdio integration problems.
|
||||
|
||||
### Definition of done
|
||||
|
||||
- Subprocess tests cover success, failure, large inputs, and cancellation.
|
||||
- stdout contains machine output only.
|
||||
- stderr contains human-readable logs/errors only.
|
||||
- API keys do not appear in reports, diagnostics, logs, or tests.
|
||||
- Failed runs always preserve diagnostics.
|
||||
- Operational docs are accurate.
|
||||
- `go test ./...` passes.
|
||||
|
||||
### Phase 18 completion status
|
||||
|
||||
Completed.
|
||||
|
||||
Implemented:
|
||||
- Expanded subprocess integration coverage in `cmd/audita/main_integration_test.go` for:
|
||||
- successful default full-pipeline runs with `--output` and without `--output`;
|
||||
- successful `--report-json` writes;
|
||||
- large-transcript subprocess behavior;
|
||||
- missing/unreadable transcript and missing/malformed glossary failures (portable handling where required);
|
||||
- unwritable output and unwritable report-json failure behavior (portable handling where required);
|
||||
- malformed structured LLM response failure behavior;
|
||||
- synthetic backend LLM failure behavior;
|
||||
- timeout/cancellation behavior with deterministic context cancellation hooks;
|
||||
- mid-pipeline failure with partial module progress preserved in reports.
|
||||
- Hardened subprocess failure stderr output to include diagnostics path when available.
|
||||
- Added deterministic test-only subprocess LLM/runtime hooks used only in helper-process tests:
|
||||
- backend failure mode;
|
||||
- malformed structured-output mode;
|
||||
- block-until-cancel mode;
|
||||
- mid-pipeline fail mode.
|
||||
- Added regression coverage for secret redaction across subprocess stdout/stderr/report/diagnostics artifacts.
|
||||
- Confirmed existing LLM adapter and scheduler tests continue to cover:
|
||||
- timeout and context cancellation propagation;
|
||||
- retry behavior;
|
||||
- malformed output safety;
|
||||
- permit release on error/cancellation.
|
||||
- Added focused subprocess-caller operational guidance in `docs/subprocess-operations.md`.
|
||||
|
||||
Historical note:
|
||||
- Phase 19 documentation/rollout/Python-retirement transitions were deferred at the end of Phase 18 and completed in Phase 19.
|
||||
|
||||
## Phase 19: Documentation, rollout, and Python retirement
|
||||
|
||||
### Purpose
|
||||
|
||||
Make the Go implementation the documented active implementation and preserve the Python implementation only as historical reference if desired.
|
||||
|
||||
### Scope
|
||||
|
||||
Implement:
|
||||
- README updates.
|
||||
- Architecture updates.
|
||||
- Rewrite notes updates.
|
||||
- Installation/build instructions for Go binary.
|
||||
- Migration notes from Python to Go.
|
||||
- Documentation of any intentionally changed behavior.
|
||||
- Removal or archival of Python-specific operational instructions from the primary path.
|
||||
- Clear statement that the Go implementation is now feature-complete.
|
||||
|
||||
### Expected behavior at end of phase
|
||||
|
||||
The repository clearly presents the Go implementation as the active Audita implementation.
|
||||
|
||||
### Definition of done
|
||||
|
||||
- Documentation no longer describes Go as only a deterministic foundation.
|
||||
- Documentation accurately describes full transcript polishing behavior.
|
||||
- Python implementation is marked archived/prototype/reference, or removed if that is the chosen repository policy.
|
||||
- Users can build, test, and run the Go implementation from docs alone.
|
||||
- `go test ./...` passes.
|
||||
|
||||
### Phase 19 completion status
|
||||
|
||||
Completed.
|
||||
|
||||
Implemented:
|
||||
- README is Go-first and documents active full-pipeline runtime behavior.
|
||||
- Build/install/test usage guidance is Go-primary and operationally oriented.
|
||||
- Subprocess/orchestrator behavior is documented in `docs/subprocess-operations.md`.
|
||||
- Migration guidance from Python to Go is documented in `docs/migration-from-python.md`.
|
||||
- Python parity notes and open parity gaps remain documented in `docs/python-parity.md`.
|
||||
- Python implementation status is explicitly legacy/reference in primary docs and `python/README.md`.
|
||||
- Documentation no longer routes normal operations to Python.
|
||||
- Normal `go test ./...` remains independent of real LLM credentials and Python dependencies.
|
||||
|
||||
## Cross-phase compatibility requirements
|
||||
|
||||
These constraints were used across implementation phases and remain useful maintenance checks:
|
||||
|
||||
- Preserve subprocess-safe stdout/stderr behavior.
|
||||
- Preserve CLI-over-env precedence.
|
||||
- Preserve existing accepted transcript input forms.
|
||||
- Preserve glossary YAML compatibility unless a deliberate migration is documented.
|
||||
- Preserve deterministic safety-first proposal application semantics.
|
||||
- Preserve run-dir diagnostics and report writing.
|
||||
- Preserve API key redaction.
|
||||
- Do not require real LLM credentials for normal `go test ./...`.
|
||||
- Do not claim a feature is implemented until it is in the runtime path.
|
||||
- Keep default behavior honest in docs and CLI help.
|
||||
- Prefer fake LLMs and golden fixtures for automated tests.
|
||||
- Keep real LLM smoke tests opt-in.
|
||||
|
||||
## Guidance for Codex-style prompts
|
||||
|
||||
When requesting implementation work, use one phase at a time.
|
||||
|
||||
Good prompt shape:
|
||||
- Name the exact phase.
|
||||
- State what is in scope.
|
||||
- State what is explicitly out of scope.
|
||||
- Require tests.
|
||||
- Require `go test ./...`.
|
||||
- Require documentation updates only when the phase changes user-visible or architectural status.
|
||||
- Require that no later-phase features be implemented opportunistically.
|
||||
|
||||
Avoid broad prompts such as:
|
||||
- “finish the rewrite”
|
||||
- “make Audita work”
|
||||
- “port the Python app”
|
||||
- “implement all modules”
|
||||
|
||||
Those prompts blur phase boundaries and make review difficult.
|
||||
|
||||
## Suggested review checklist for every phase
|
||||
|
||||
Before accepting a phase implementation, verify:
|
||||
|
||||
- Does the implementation match the phase scope?
|
||||
- Did it avoid implementing unrelated later-phase behavior?
|
||||
- Does `go test ./...` pass?
|
||||
- Are stdout and stderr still clean for subprocess callers?
|
||||
- Are API keys redacted everywhere?
|
||||
- Are reports machine-readable?
|
||||
- Are diagnostics sufficient for debugging?
|
||||
- Are fake LLM tests used instead of requiring real credentials?
|
||||
- Are new public behaviors documented?
|
||||
- Does the code follow the architecture in `docs/architecture.md`?
|
||||
- Does the implementation move Audita closer to Python feature parity?
|
||||
1059
docs/roadmap.md
1059
docs/roadmap.md
File diff suppressed because it is too large
Load Diff
@@ -1,688 +0,0 @@
|
||||
# Validator Refactor Plan
|
||||
|
||||
## Purpose
|
||||
|
||||
This document describes a concrete package-ownership refactor for Audita's built-in validators.
|
||||
|
||||
Target outcome:
|
||||
- each built-in validator has its own package under `internal/validators/<validator_key>/`;
|
||||
- the rest of the application depends on the existing common validator runtime contract;
|
||||
- validator-specific construction and configuration are owned by validator-specific packages;
|
||||
- shared validator runtime machinery remains centralized where reuse is high;
|
||||
- runner and module code do not need to know concrete validator implementation types.
|
||||
|
||||
This is not a runtime redesign. Audita already has the important validator abstraction: the runner receives validators through a common interface, calls `Validate`, and consumes validator results without needing to understand validator internals. This plan preserves that model and improves package ownership around it.
|
||||
|
||||
## Current state
|
||||
|
||||
The current codebase already has the core runtime pieces:
|
||||
|
||||
- `contracts.Validator` in `internal/framework/contracts`;
|
||||
- `Validate(ctx, req)` semantics;
|
||||
- validator request/result/decision models;
|
||||
- runner orchestration that treats validators as decision producers;
|
||||
- stable built-in validator keys;
|
||||
- a built-in validator registry and built-in module chain definitions;
|
||||
- deterministic and LLM-backed validator implementations;
|
||||
- embedded prompt assets and prompt metadata for LLM-backed validators.
|
||||
|
||||
Current validator ownership is split across two layers:
|
||||
|
||||
- `internal/validators/`
|
||||
- built-in key constants;
|
||||
- built-in registry;
|
||||
- built-in module chain definitions.
|
||||
|
||||
- `internal/framework/validators/`
|
||||
- request/result/decision models;
|
||||
- deterministic validator implementation details;
|
||||
- generic LLM-backed validator runtime;
|
||||
- batching;
|
||||
- prompt/response and diagnostics helpers;
|
||||
- cardinality enforcement and shared validation helpers.
|
||||
|
||||
This refactor should make `internal/validators/<validator_key>/` the visible home of each built-in validator, while keeping shared runtime machinery in `internal/framework/validators`.
|
||||
|
||||
## Goals
|
||||
|
||||
Required goals:
|
||||
|
||||
- Mirror the module package pattern by giving each built-in validator its own package under `internal/validators`.
|
||||
- Preserve the existing `contracts.Validator` runtime contract.
|
||||
- Keep built-in validator keys stable.
|
||||
- Keep built-in module validator chains stable.
|
||||
- Preserve current runtime behavior and report shape unless explicitly noted.
|
||||
- Remove concrete validator type knowledge from the runner.
|
||||
- Make production registry construction route through validator-owned packages.
|
||||
- Localize the `protected_terms` glossary-stage special case inside the `protected_terms` validator package.
|
||||
- Keep LLM batching, diagnostics, response parsing, cardinality checks, and shared helper behavior centralized.
|
||||
|
||||
Secondary goals:
|
||||
|
||||
- Make validator construction read similarly to module construction.
|
||||
- Keep validator-specific tests close to validator-specific packages.
|
||||
- Reduce the impression that `internal/framework/validators` owns the built-in validator catalog.
|
||||
- Make future built-in validators easier to add without expanding a central framework file.
|
||||
|
||||
## Non-goals
|
||||
|
||||
Do not implement any of the following as part of this refactor:
|
||||
|
||||
- User-defined validator plugins.
|
||||
- User-configurable validator chains.
|
||||
- New validator types.
|
||||
- Changes to validator decision cardinality rules.
|
||||
- Changes to prompt assets or prompt text.
|
||||
- Changes to structured response schemas.
|
||||
- Changes to scheduler behavior.
|
||||
- Changes to output schemas, reports, diagnostics shape, or correction ledger shape except where a stable validator key already appears.
|
||||
- A new competing validator interface.
|
||||
|
||||
## Architectural principle
|
||||
|
||||
This should be a package-ownership cleanup over an already sound runtime abstraction.
|
||||
|
||||
The correct bias is:
|
||||
|
||||
- keep runtime semantics stable;
|
||||
- move validator identity and construction into validator-owned packages;
|
||||
- keep shared machinery centralized;
|
||||
- remove concrete implementation leakage from the runner;
|
||||
- prefer wrappers first and deeper cleanup second.
|
||||
|
||||
## Target package layout
|
||||
|
||||
Target end state:
|
||||
|
||||
```text
|
||||
internal/framework/validators/
|
||||
models.go
|
||||
runtime.go
|
||||
llm_runtime.go
|
||||
llm_batching.go
|
||||
diagnostics.go
|
||||
cardinality.go
|
||||
prompt_helpers.go
|
||||
protected_vocabulary.go
|
||||
|
||||
internal/validators/
|
||||
registry.go
|
||||
chains.go
|
||||
metadata.go
|
||||
interfaces.go
|
||||
|
||||
internal/validators/confidence_threshold/
|
||||
validator.go
|
||||
validator_test.go
|
||||
|
||||
internal/validators/original_text_presence/
|
||||
validator.go
|
||||
validator_test.go
|
||||
|
||||
internal/validators/non_empty_corrected_text/
|
||||
validator.go
|
||||
validator_test.go
|
||||
|
||||
internal/validators/no_effect/
|
||||
validator.go
|
||||
validator_test.go
|
||||
|
||||
internal/validators/protected_terms/
|
||||
validator.go
|
||||
validator_test.go
|
||||
|
||||
internal/validators/spoken_form_plausibility/
|
||||
validator.go
|
||||
validator_test.go
|
||||
|
||||
internal/validators/meaning_reversal_review/
|
||||
validator.go
|
||||
validator_test.go
|
||||
|
||||
internal/validators/editorial_review/
|
||||
validator.go
|
||||
validator_test.go
|
||||
|
||||
internal/validators/grammar_review/
|
||||
validator.go
|
||||
validator_test.go
|
||||
|
||||
internal/validators/spoken_word_review/
|
||||
validator.go
|
||||
validator_test.go
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- `internal/framework/validators` remains the shared runtime layer.
|
||||
- `internal/validators/<name>` owns construction and validator-specific configuration.
|
||||
- `internal/validators/registry.go` remains the built-in production registry entrypoint.
|
||||
- `internal/validators/chains.go` remains the built-in module chain resolver.
|
||||
- It is acceptable for `internal/framework/validators` to retain a generic shared LLM validator runtime type if only validator-owned packages construct it.
|
||||
|
||||
## Runtime contract
|
||||
|
||||
Keep `contracts.Validator` as the framework-facing contract.
|
||||
|
||||
Do not introduce a second competing validator interface. If helper interfaces are needed, they should supplement the existing contract rather than replace it.
|
||||
|
||||
The runner should continue to operate on validators as opaque components:
|
||||
|
||||
- receive `[]contracts.Validator`;
|
||||
- call `Validate(ctx, req)`;
|
||||
- consume returned validator results;
|
||||
- remove rejected proposals from the eligible set;
|
||||
- preserve deterministic-before-LLM execution ordering;
|
||||
- preserve existing decision cardinality rules.
|
||||
|
||||
## Validator metadata
|
||||
|
||||
Add a small metadata surface to avoid concrete implementation checks in the runner.
|
||||
|
||||
Recommended location:
|
||||
|
||||
- `internal/validators/interfaces.go` or `internal/validators/metadata.go`
|
||||
|
||||
This package must stay low-level:
|
||||
|
||||
- it may import `internal/framework/contracts`;
|
||||
- it must not import validator-specific packages;
|
||||
- it must not import the registry.
|
||||
|
||||
Suggested API:
|
||||
|
||||
```go
|
||||
type ExecutionClass string
|
||||
|
||||
const (
|
||||
ExecutionClassDeterministic ExecutionClass = "deterministic"
|
||||
ExecutionClassLLMBacked ExecutionClass = "llm_backed"
|
||||
)
|
||||
|
||||
type ClassifiedValidator interface {
|
||||
contracts.Validator
|
||||
ExecutionClass() ExecutionClass
|
||||
}
|
||||
```
|
||||
|
||||
Runner behavior:
|
||||
|
||||
- if a validator implements `ClassifiedValidator`, use `ExecutionClass()`;
|
||||
- otherwise treat it as deterministic by default;
|
||||
- never type-assert against concrete framework validator types such as `*frameworkvalidators.LLMBackedValidator`.
|
||||
|
||||
This removes concrete implementation knowledge from the runner while preserving ordering semantics.
|
||||
|
||||
## Validator package constructors
|
||||
|
||||
Each validator package should expose a constructor that returns `contracts.Validator`.
|
||||
|
||||
Use the simplest constructor that honestly reflects the validator's dependencies.
|
||||
|
||||
Acceptable patterns:
|
||||
|
||||
```go
|
||||
func New() (contracts.Validator, error)
|
||||
```
|
||||
|
||||
```go
|
||||
func New(opts Options) (contracts.Validator, error)
|
||||
```
|
||||
|
||||
```go
|
||||
func NewGlossaryStage(opts Options) (contracts.Validator, error)
|
||||
```
|
||||
|
||||
Guidance:
|
||||
|
||||
- Use `New()` only when construction truly requires no runtime dependencies.
|
||||
- Use `New(opts Options)` when the existing registry already supplies dependencies such as config, glossary, LLM client, scheduler, diagnostics context, or prompt metadata.
|
||||
- Keep `Options` package-local unless several validators genuinely share the same option structure.
|
||||
- Do not force zero-argument constructors if doing so would hide dependencies in globals or cause construction-time behavior to become implicit.
|
||||
- Constructor names should make module-sensitive behavior explicit, especially for `protected_terms`.
|
||||
|
||||
## Built-in validator registry
|
||||
|
||||
Keep `internal/validators/registry.go` as the production wiring layer.
|
||||
|
||||
After this refactor, registry entries should call validator-package constructors rather than framework concrete implementations.
|
||||
|
||||
Example target shape:
|
||||
|
||||
```go
|
||||
{Key: KeyConfidenceThreshold, Build: confidencethreshold.New}
|
||||
{Key: KeyGrammarReview, Build: grammarreview.New}
|
||||
```
|
||||
|
||||
The registry should continue to provide:
|
||||
|
||||
- stable validator keys;
|
||||
- built-in validator metadata;
|
||||
- clear lookup/build failures for unknown keys;
|
||||
- stable production construction behavior.
|
||||
|
||||
The registry should not become a user plugin system.
|
||||
|
||||
## Built-in validator keys
|
||||
|
||||
Preserve these keys exactly:
|
||||
|
||||
- `confidence_threshold`
|
||||
- `original_text_presence`
|
||||
- `non_empty_corrected_text`
|
||||
- `no_effect`
|
||||
- `protected_terms`
|
||||
- `spoken_form_plausibility`
|
||||
- `meaning_reversal_review`
|
||||
- `editorial_review`
|
||||
- `grammar_review`
|
||||
- `spoken_word_review`
|
||||
|
||||
These keys must continue to appear consistently in:
|
||||
|
||||
- registry entries;
|
||||
- built-in chain definitions;
|
||||
- reports;
|
||||
- diagnostics metadata or paths where validator identity appears;
|
||||
- correction ledger entries;
|
||||
- utilization diagnostics;
|
||||
- tests;
|
||||
- documentation.
|
||||
|
||||
## Built-in module chains
|
||||
|
||||
Preserve the current effective built-in module chains exactly unless an existing test proves the documented chain differs from the runtime and the runtime behavior is clearly the intended source of truth.
|
||||
|
||||
Do not use this refactor to revisit validator policy.
|
||||
|
||||
The chain resolver may continue to live in `internal/validators/chains.go`, but it should construct validators through package-owned constructors.
|
||||
|
||||
## Treatment of deterministic validators
|
||||
|
||||
Move ownership into per-validator packages:
|
||||
|
||||
- `internal/validators/confidence_threshold`
|
||||
- `internal/validators/original_text_presence`
|
||||
- `internal/validators/non_empty_corrected_text`
|
||||
- `internal/validators/no_effect`
|
||||
- `internal/validators/protected_terms`
|
||||
|
||||
Implementation guidance:
|
||||
|
||||
- If the implementation is short and self-contained, it may live directly in the validator package.
|
||||
- If logic is shared, keep shared helpers in `internal/framework/validators`.
|
||||
- Do not duplicate cardinality helpers or vocabulary helpers.
|
||||
- Prefer behavior-preserving wrappers first, then move implementation details only where it clearly improves ownership.
|
||||
|
||||
The goal is not to empty `internal/framework/validators`; the goal is to stop making it look like the built-in validator catalog.
|
||||
|
||||
## Protected terms special case
|
||||
|
||||
`protected_terms` has module-sensitive behavior and must be handled carefully.
|
||||
|
||||
Target ownership:
|
||||
|
||||
- `internal/validators/protected_terms` owns both general and glossary-stage construction.
|
||||
|
||||
Suggested constructors:
|
||||
|
||||
```go
|
||||
func New(opts Options) (contracts.Validator, error)
|
||||
func NewGlossaryStage(opts Options) (contracts.Validator, error)
|
||||
```
|
||||
|
||||
or, if no options are needed:
|
||||
|
||||
```go
|
||||
func New() (contracts.Validator, error)
|
||||
func NewGlossaryStage() (contracts.Validator, error)
|
||||
```
|
||||
|
||||
Behavior requirements:
|
||||
|
||||
- non-glossary modules keep existing protected-term behavior;
|
||||
- glossary-stage behavior remains stricter if that is the current runtime behavior;
|
||||
- both variants continue to report the stable key `protected_terms`;
|
||||
- chain resolution should not directly reference framework concrete protected-term validator types.
|
||||
|
||||
Add or preserve explicit tests for:
|
||||
|
||||
- glossary-stage protected-term behavior;
|
||||
- non-glossary protected-term behavior;
|
||||
- stable `protected_terms` identity in reports and correction ledger entries.
|
||||
|
||||
## Treatment of LLM-backed validators
|
||||
|
||||
Move ownership into per-validator packages:
|
||||
|
||||
- `internal/validators/spoken_form_plausibility`
|
||||
- `internal/validators/meaning_reversal_review`
|
||||
- `internal/validators/editorial_review`
|
||||
- `internal/validators/grammar_review`
|
||||
- `internal/validators/spoken_word_review`
|
||||
|
||||
Recommended implementation:
|
||||
|
||||
- each package provides a thin wrapper over shared LLM runtime machinery;
|
||||
- each package owns the stable validator key and constructor;
|
||||
- each package selects or configures the appropriate prompt ID/type through existing prompt registry surfaces;
|
||||
- each package exposes `ExecutionClass() == llm_backed` either directly or through a wrapper;
|
||||
- shared framework code continues to own batching, diagnostics, structured response parsing, model resolution, and decision mapping.
|
||||
|
||||
Avoid this anti-pattern:
|
||||
|
||||
- five copied versions of the generic LLM-backed validator runtime.
|
||||
|
||||
Acceptable shared runtime:
|
||||
|
||||
- a generic shared `LLMBackedValidator`;
|
||||
- a shared `NewLLMBackedValidator(...)` factory;
|
||||
- a shared LLM validator executor configured by validator packages.
|
||||
|
||||
The important boundary is that validator-specific ownership is visible under `internal/validators/<name>`, even if shared execution remains centralized.
|
||||
|
||||
## Prompt ownership
|
||||
|
||||
Do not change prompt assets in this refactor.
|
||||
|
||||
Prompt assets already live under `internal/prompts`. Validator packages may own prompt selection/configuration by referring to existing prompt IDs, but they should not duplicate prompt text or move Markdown assets.
|
||||
|
||||
For example, a package may configure the shared LLM runtime with:
|
||||
|
||||
- validator key: `grammar_review`;
|
||||
- prompt ID: `validators.grammar_review`;
|
||||
- structured response schema: `validator_decision_set`;
|
||||
- execution class: `llm_backed`.
|
||||
|
||||
But this refactor should not alter prompt text, prompt metadata, prompt IDs, prompt hashes, or prompt rendering behavior except where imports must be adjusted.
|
||||
|
||||
## Import-cycle rules
|
||||
|
||||
Avoid import cycles by keeping dependencies acyclic.
|
||||
|
||||
Preferred dependency direction:
|
||||
|
||||
```text
|
||||
internal/framework/contracts
|
||||
internal/framework/validators
|
||||
internal/prompts
|
||||
↑
|
||||
internal/validators/<validator_key>
|
||||
↑
|
||||
internal/validators/registry and chains
|
||||
↑
|
||||
module construction / runner wiring
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- validator-specific packages must not import `internal/validators/registry`;
|
||||
- the registry may import validator-specific packages;
|
||||
- runner may import low-level validator metadata but should not import validator-specific packages;
|
||||
- metadata interfaces should live in a low-level package that does not import registry or validator-specific packages;
|
||||
- shared framework runtime must not import the built-in registry.
|
||||
|
||||
## Recommended implementation passes
|
||||
|
||||
Use three implementation passes unless blocked.
|
||||
|
||||
### Pass 1: Metadata, wrappers, and registry switch
|
||||
|
||||
Purpose:
|
||||
- introduce execution metadata;
|
||||
- remove runner dependency on concrete framework validator types;
|
||||
- create per-validator packages as thin wrappers;
|
||||
- switch the registry to construct validators through those packages.
|
||||
|
||||
Tasks:
|
||||
- add validator execution classification metadata;
|
||||
- update runner ordering logic to use metadata;
|
||||
- create one package per built-in validator under `internal/validators`;
|
||||
- keep wrappers behavior-preserving;
|
||||
- preserve stable keys;
|
||||
- preserve built-in chains;
|
||||
- switch registry build closures to per-package constructors;
|
||||
- add tests for registry completeness, metadata classification, and ordering behavior.
|
||||
|
||||
Acceptance criteria:
|
||||
- runner no longer type-asserts against framework concrete validator types;
|
||||
- every built-in validator has a package;
|
||||
- registry builds all validators through package constructors;
|
||||
- deterministic-before-LLM behavior is unchanged;
|
||||
- all existing tests pass.
|
||||
|
||||
### Pass 2: Protected terms localization, framework cleanup, and test locality
|
||||
|
||||
Purpose:
|
||||
- localize the `protected_terms` special case;
|
||||
- reduce framework built-in ownership;
|
||||
- move or add tests near validator packages.
|
||||
|
||||
Tasks:
|
||||
- add explicit protected-terms constructors for general and glossary-stage behavior;
|
||||
- update chain resolution to call those constructors;
|
||||
- remove direct chain/registry references to protected-term framework concrete types;
|
||||
- reduce exported framework validator types where package wrappers fully own construction;
|
||||
- keep shared runtime helpers in `internal/framework/validators`;
|
||||
- move or add validator-specific tests under the validator packages where practical;
|
||||
- retain shared runtime tests in the framework package.
|
||||
|
||||
Acceptance criteria:
|
||||
- protected-term behavior is unchanged;
|
||||
- glossary-stage behavior is explicitly tested;
|
||||
- framework package no longer appears to own the built-in validator catalog;
|
||||
- validator-specific behavior is tested near validator packages where practical;
|
||||
- all existing tests pass.
|
||||
|
||||
### Pass 3: Documentation and review
|
||||
|
||||
Purpose:
|
||||
- align docs with the final structure;
|
||||
- verify no accidental behavior drift or scope creep.
|
||||
|
||||
Tasks:
|
||||
- update `docs/architecture.md`;
|
||||
- update `docs/validators.md`;
|
||||
- update any public-contract or diagnostics docs only if validator package ownership needs mention there;
|
||||
- document the distinction between validator-owned packages and shared validator runtime;
|
||||
- document the 1.0 boundary that validator chains remain built-in and not user-configurable;
|
||||
- run `go test ./...`;
|
||||
- review imports and package boundaries for accidental cycles or leakage.
|
||||
|
||||
Acceptance criteria:
|
||||
- docs describe the new package layout accurately;
|
||||
- docs do not imply plugin support or user-configurable chains;
|
||||
- tests pass;
|
||||
- git diff shows package-ownership refactor only.
|
||||
|
||||
## File-by-file change guide
|
||||
|
||||
Expected direct edits:
|
||||
|
||||
- `internal/framework/runner/runner.go`
|
||||
- replace concrete LLM validator type check with execution metadata interface check.
|
||||
|
||||
- `internal/validators/interfaces.go` or `internal/validators/metadata.go`
|
||||
- define execution classification types and optional interface.
|
||||
|
||||
- `internal/validators/registry.go`
|
||||
- import per-validator packages;
|
||||
- route construction through package constructors;
|
||||
- preserve stable keys.
|
||||
|
||||
- `internal/validators/chains.go`
|
||||
- preserve chain key lists and order;
|
||||
- use protected-terms glossary-stage constructor where appropriate;
|
||||
- avoid concrete framework validator types.
|
||||
|
||||
- `internal/framework/validators/deterministic.go`
|
||||
- reduce or remove exported built-in validator concrete types if wrappers fully own construction;
|
||||
- keep shared helpers where useful.
|
||||
|
||||
- `internal/framework/validators/llm_validators.go`
|
||||
- keep shared LLM runtime;
|
||||
- reduce exported surface only if safe;
|
||||
- do not duplicate LLM runtime across validator packages.
|
||||
|
||||
Expected new files:
|
||||
|
||||
- `internal/validators/confidence_threshold/validator.go`
|
||||
- `internal/validators/original_text_presence/validator.go`
|
||||
- `internal/validators/non_empty_corrected_text/validator.go`
|
||||
- `internal/validators/no_effect/validator.go`
|
||||
- `internal/validators/protected_terms/validator.go`
|
||||
- `internal/validators/spoken_form_plausibility/validator.go`
|
||||
- `internal/validators/meaning_reversal_review/validator.go`
|
||||
- `internal/validators/editorial_review/validator.go`
|
||||
- `internal/validators/grammar_review/validator.go`
|
||||
- `internal/validators/spoken_word_review/validator.go`
|
||||
- corresponding package-local tests where practical.
|
||||
|
||||
Expected documentation edits:
|
||||
|
||||
- `docs/architecture.md`
|
||||
- `docs/validators.md`
|
||||
|
||||
Possibly update:
|
||||
|
||||
- `docs/public-contract.md`
|
||||
- `docs/diagnostics.md`
|
||||
|
||||
Only update README if the high-level project description would otherwise be inaccurate.
|
||||
|
||||
## Test strategy
|
||||
|
||||
Recommended test distribution:
|
||||
|
||||
- `internal/validators/<name>/validator_test.go`
|
||||
- package-specific constructor and behavior tests.
|
||||
|
||||
- `internal/validators/registry_test.go`
|
||||
- registry wiring and key coverage.
|
||||
|
||||
- `internal/validators/module_chains_test.go`
|
||||
- built-in module chain coverage.
|
||||
|
||||
- `internal/framework/runner/runner_test.go`
|
||||
- orchestration semantics only, including deterministic-before-LLM ordering.
|
||||
|
||||
- `internal/framework/validators/*_test.go`
|
||||
- shared runtime behavior only.
|
||||
|
||||
Required test coverage:
|
||||
|
||||
- every built-in validator package exists and constructs successfully;
|
||||
- every built-in key resolves through the registry;
|
||||
- every built-in module chain resolves;
|
||||
- unknown validator keys fail deterministically;
|
||||
- runner ordering uses execution classification, not concrete type checks;
|
||||
- unclassified validators default to deterministic;
|
||||
- protected-terms glossary-stage behavior is preserved;
|
||||
- stable validator keys remain present in reports, diagnostics, utilization diagnostics, and correction ledger entries where currently applicable;
|
||||
- no live LLM credentials are needed for `go test ./...`.
|
||||
|
||||
## Ordering and safety rules
|
||||
|
||||
Implementation rules:
|
||||
|
||||
- preserve validator keys exactly;
|
||||
- preserve built-in module chain order exactly;
|
||||
- preserve `protected_terms` glossary-stage behavior exactly;
|
||||
- preserve deterministic-before-LLM ordering;
|
||||
- preserve decision cardinality enforcement;
|
||||
- preserve report field names and validator identity strings;
|
||||
- preserve prompt assets and prompt metadata;
|
||||
- preserve structured response schemas;
|
||||
- preserve scheduler semantics;
|
||||
- preserve output schemas;
|
||||
- avoid duplicating shared LLM runtime code;
|
||||
- keep tests passing after each implementation pass.
|
||||
|
||||
## Risks and mitigations
|
||||
|
||||
### Risk: accidental behavior drift in protected terms
|
||||
|
||||
Why it matters:
|
||||
- `protected_terms` has module-sensitive behavior.
|
||||
|
||||
Mitigation:
|
||||
- add or preserve explicit tests for glossary-stage and non-glossary-stage behavior;
|
||||
- use named constructors rather than hidden request-based branching where the stage variant is selected at chain build time.
|
||||
|
||||
### Risk: runner ordering regressions
|
||||
|
||||
Why it matters:
|
||||
- deterministic validators must still run before LLM-backed validators.
|
||||
|
||||
Mitigation:
|
||||
- add or retain tests proving ordering with a mixed validator list;
|
||||
- treat unknown or unclassified validators as deterministic.
|
||||
|
||||
### Risk: over-duplicating LLM validator code
|
||||
|
||||
Why it matters:
|
||||
- LLM-backed validators share batching, diagnostics, parsing, and structured-output handling.
|
||||
|
||||
Mitigation:
|
||||
- keep one shared LLM runtime implementation;
|
||||
- use thin validator-package wrappers.
|
||||
|
||||
### Risk: import cycles
|
||||
|
||||
Why it matters:
|
||||
- registry imports validator packages;
|
||||
- validator packages import contracts and shared framework runtime;
|
||||
- runner imports contracts and metadata.
|
||||
|
||||
Mitigation:
|
||||
- keep metadata low-level;
|
||||
- keep registry out of validator-specific packages;
|
||||
- keep shared framework runtime independent of the built-in registry.
|
||||
|
||||
### Risk: excessive cleanup during refactor
|
||||
|
||||
Why it matters:
|
||||
- broader cleanup can obscure behavior changes and increase review risk.
|
||||
|
||||
Mitigation:
|
||||
- wrappers first;
|
||||
- deeper cleanup only after registry construction is package-owned;
|
||||
- do not combine with prompt, scheduler, report, config, or output-schema changes.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
The refactor is complete when all of the following are true:
|
||||
|
||||
- every built-in validator has its own package under `internal/validators`;
|
||||
- production validator registry constructs validators via those packages;
|
||||
- module packages still work with `[]contracts.Validator` and require no validator implementation knowledge;
|
||||
- runner has no concrete-type dependency on framework validator implementations;
|
||||
- `protected_terms` glossary-stage handling is owned by the validator package;
|
||||
- built-in validator keys are unchanged;
|
||||
- built-in module chain order and behavior are unchanged;
|
||||
- LLM-backed validators still use shared runtime machinery;
|
||||
- prompt assets and structured response schemas are unchanged;
|
||||
- tests pass with `go test ./...`;
|
||||
- docs describe the new package ownership model.
|
||||
|
||||
## Suggested follow-up cleanup after this refactor
|
||||
|
||||
Not required for this refactor, but worth considering later:
|
||||
|
||||
- introduce a validator factory in `internal/framework/validators` only if future dynamic construction becomes necessary;
|
||||
- consider whether `contracts.ValidationRequest = validators.Request` should remain an alias or become a contract-owned type;
|
||||
- consider whether chain definitions should eventually move closer to module registry or module packages if module-specific validator policy becomes more configurable;
|
||||
- consider whether `editorial_review` remains useful if no production module uses it;
|
||||
- consider a later user-configurable chain system only if there is a clear product need.
|
||||
|
||||
## Summary
|
||||
|
||||
This refactor should make validators mirror modules at the package ownership level without changing Audita's runtime model.
|
||||
|
||||
The desired end state is:
|
||||
|
||||
- validator packages own built-in validator construction;
|
||||
- the registry owns production wiring;
|
||||
- the runner owns orchestration only;
|
||||
- shared framework code owns reusable runtime mechanics;
|
||||
- tests and docs reflect those boundaries.
|
||||
Reference in New Issue
Block a user