eric/audita
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Documentation cleanup and addition of pre-1.0 roadmap punchlist

Browse Source
This commit is contained in:
Eric Rakestraw
2026-05-12 20:40:35 -05:00
parent 3d45571bb0
commit 20f612215f
6 changed files with 936 additions and 261 deletions
Download Patch File Download Diff File Expand all files Collapse all files

770
docs/python/rewrite-notes.md Normal file
View File

@@ -0,0 +1,770 @@
# Audita Rewrite Notes (Historical)
This document is a historical record of the Python-to-Go rewrite project.
It is retained for engineering context, not as primary runtime guidance.
## Definition of done for the Go rewrite
The Go rewrite is complete when both of the following are true:
1. Feature parity with the initial Python implementation:
- `audita process` performs end-to-end transcript polishing, not only deterministic preprocessing.
- The default module sequence is implemented and active in the runtime path:
- `glossary`
- `homophones`
- `glossary`
- `spoken_word`
- `grammar`
- Repeated module instances are resolved deterministically, for example `glossary_1` and `glossary_2`.
- Real LLM-backed proposal generation is implemented through an OpenAI-compatible structured-output client.
- Deterministic validators and LLM-backed validators are implemented and enforced in the runtime path.
- Proposal application preserves the safety-first semantics of the Python implementation.
- Structured run reports and diagnostics are sufficient for debugging successful runs, skipped corrections, and failures.
- The Go CLI can replace the Python CLI in the surrounding orchestration pipeline without downstream contract surprises.
2. Adherence to the intended application architecture in `docs/architecture.md`:
- Sequential module pipeline over a mutable working transcript.
- Bounded intra-module concurrency only.
- Provider-neutral LLM abstraction.
- Separate proposal and validation LLM settings.
- Prompt/response diagnostics for LLM stages.
- Module-level run reports with applied and skipped changes.
- Strict stdout/stderr discipline for subprocess callers.
- No hidden service dependency; the CLI remains the primary integration surface.
## Final status (Phase 19 complete)
The Go rewrite definition of done is met, with documented parity caveats tracked in `docs/python-parity.md`.
- Phase 19 is complete.
- Go Audita is the active implementation.
- The default full module pipeline is implemented (`glossary,homophones,glossary,spoken_word,grammar`).
- Parity fixtures and operational hardening coverage exist in normal `go test ./...`.
- Python is retained as a legacy/reference implementation and is not the primary operational path.
## Current implementation status
The Go rewrite is complete and feature-complete for the intended CLI runtime architecture.
Implemented:
- CLI command surface for `audita process`.
- Config/env/flag loading and validation.
- Transcript/glossary schema parsing and validation.
- Deterministic normalization with summary stats.
- Deterministic chunking with summary stats.
- Per-run diagnostics directory plus source/normalization/chunking artifacts.
- Redacted invocation/effective-config diagnostics metadata artifacts.
- Process report output through `--report-json` and run-dir `report.json`.
- Report-level diagnostics artifact references.
- Framework foundation packages for contracts and proposal preview/apply semantics.
- Production runner orchestration over a mutable working transcript.
- Module-level report structures and run-level module summaries.
- CLI runner integration point via injectable module factory/registry (used by deterministic tests).
- Runtime validator models and deterministic validator implementations.
- Validator cardinality enforcement (missing/duplicate/unknown proposal index errors).
- Deterministic validator-chain execution in the production runner.
- Module reports including validator decisions and validator rejections.
- LLM-backed validator request/response models and prompt builders.
- LLM validator batching by validation prompt-token budget.
- LLM validator runtime integration through structured LLM client abstraction and scheduler hooks.
- LLM validator prompt/response diagnostics artifact wiring with secret redaction.
- Shared LLM proposal-generation helper with structured correction-set parsing.
- Deterministic proposal-index assignment for shared proposal generation.
- Proposal-generation prompt/response diagnostics artifact wiring with secret redaction.
- Production module-registry scaffolding with known key recognition and explicit unsupported/unimplemented errors.
- Production grammar module package with Python-aligned prompt intent and guardrails.
- Explicit `--modules grammar` runtime path through runner, shared proposal generation, validators, application, reporting, and diagnostics.
- Production glossary module package with Python-aligned prompt intent and guardrails.
- Glossary-derived deterministic protected-term extraction and validator integration.
- Explicit `--modules glossary` runtime path through runner, shared proposal generation, validators, application, reporting, and diagnostics.
- Repeated glossary stage support with deterministic instance names (`glossary_1`, `glossary_2`), including mutable working-transcript handoff.
- Production homophones module package with Python-aligned prompt intent and guardrails.
- Explicit `--modules homophones` runtime path through runner, shared proposal generation, validators, application, reporting, and diagnostics.
- Focused multi-module runtime tests for already-implemented interoperability (for example `glossary,homophones`) without claiming full default-pipeline completion.
- Production spoken_word module package with Python-aligned prompt intent and guardrails.
- Explicit `--modules spoken_word` runtime path through runner, shared proposal generation, validators, application, reporting, and diagnostics.
- Focused multi-module runtime tests for already-implemented interoperability (for example `spoken_word,grammar`) without claiming full default-pipeline completion.
- All production modules now exist (`glossary`, `homophones`, `spoken_word`, `grammar`) and are integrated into the default full-sequence runtime path.
- Default runtime sequence is now active and ordered as:
- `glossary`
- `homophones`
- `glossary`
- `spoken_word`
- `grammar`
- Repeated glossary stages resolve and report deterministically as `glossary_1` and `glossary_2`.
- Full-pipeline module reports and run-level summaries aggregate applied/skipped/failed metadata across all module instances.
- Mid-pipeline failure reporting preserves partial progress and failed-module metadata.
- Full-pipeline diagnostics include proposal/validator prompt-response artifacts with redaction.
- Skip-aware retention uses actual module skipped/rejected correction data.
- Broad deterministic and CLI/subprocess test coverage for implemented phases through `go test ./...`.
- Internal typed structured LLM contract (`StructuredLLMClient.CompleteStructured(ctx, req, out)`).
- `internal/framework/llm` instructor-go-backed adapter with:
- configurable base URL/model/retries/mode/timeout
- optional API key support for local-compatible endpoints
- API-key redaction in returned errors
- typed structured decode into caller-provided outputs.
- LLM scheduler/semaphore infrastructure for bounded concurrency with context-aware acquisition and reliable release.
- LLM effective-config resolution helpers:
- primary config resolution
- validation config inheritance from primary when validation fields are unset
- validation override behavior when validation fields are set.
- Generic JSON diagnostics primitives for LLM interactions (request metadata, request payload, response payload, optional error payload) with secret redaction.
Phase 19 documentation and rollout items are implemented:
- Go-first README/build/test/usage guidance.
- subprocess/orchestrator operational guidance.
- Python-to-Go migration guidance.
- explicit legacy/reference Python status documentation.
## Completed phases
### Phase 1: Go CLI skeleton
Completed.
Implemented:
- Go module and `cmd/audita` entrypoint.
- `audita process` command surface.
- Core flags and config wiring.
- Subprocess-safe command behavior foundation.
### Phase 2: Schemas and file I/O
Completed.
Implemented:
- Transcript parsing for bare arrays and `{ "segments": [...] }` input.
- Glossary YAML parsing.
- Transcript and glossary validation.
- Canonical transcript output serialization.
### Phase 3: Deterministic normalization
Completed.
Implemented:
- Chronological sorting.
- Same-speaker segment merging.
- Gap-sensitive join behavior.
- Duration and token-budget merge constraints.
- Sequential normalized segment IDs.
- Normalization summary stats.
### Phase 4: Chunking and token estimation
Completed.
Implemented:
- Deterministic heuristic token estimation.
- Contiguous transcript sectioning.
- Min/max section token behavior.
- Optional target section handling.
- Chunking summaries and diagnostics artifacts.
### Phase 5: Proposal model and application semantics
Completed.
Implemented:
- Correction proposal and enriched proposal models.
- Replacement policies:
- `require_unique`
- `replace_all`
- Safe preview logic.
- Deterministic proposal application.
- Applied/skipped change records.
- Stable skip reasons.
### Phase 6: Reports and diagnostics
Completed for the current deterministic runtime scope.
Implemented:
- Per-run diagnostics directory creation.
- Source transcript artifacts.
- Parsed source transcript artifacts.
- Normalized transcript artifact.
- Normalization summary artifact.
- Chunking summary artifact.
- Redacted invocation metadata artifact.
- Redacted effective-config artifact.
- Run-dir `report.json`.
- Optional external `--report-json`.
- Failure `error.log`.
- Report-level diagnostics artifact references.
- Retention decision model with future skipped-correction hook.
Intentionally deferred:
- Module prompt/response diagnostics artifacts are not produced yet because module execution and LLM calls are not implemented in the runtime path.
### Phase 7: Pipeline runner with deterministic test modules
Completed.
Implemented:
- `internal/framework/runner` production package with sequential module orchestration.
- Deterministic module run-spec resolution and repeated instance naming (`glossary_1`, `glossary_2`, etc.).
- Mutable working transcript handoff across module instances.
- Proposal application through `internal/framework/proposals`.
- Per-module applied/skipped change capture and module status/timing metadata.
- Partial-progress return on module failure, with pipeline stop on first failure.
- Process report support for module-level results and run-level module summaries.
- CLI runtime integration point via injectable module factory/registry, exercised by deterministic fake-module tests.
Not implemented in Phase 7 (by design):
- Real `glossary`, `homophones`, `spoken_word`, `grammar` production modules.
- Validators (Phase 8).
- Structured LLM calls or scheduler behavior.
- Prompt/response diagnostics.
- End-to-end transcript polishing.
Current runtime behavior note:
- Default user-facing CLI behavior now executes the full production module sequence unless `--modules` explicitly overrides it.
## Phase 8: Runtime validator framework and deterministic validators
Completed.
Implemented:
- Runtime validator request/result models in `internal/framework/validators`.
- Deterministic validator reason codes for stable reporting.
- Validator cardinality enforcement:
- one decision per candidate proposal index
- missing indexes are errors
- duplicate indexes are errors
- unknown indexes are errors
- Deterministic validators:
- confidence threshold
- original-text presence against working transcript
- non-empty correction
- identical/no-effect rejection
- conservative protected glossary-term guard
- Ordered validator-chain execution in the production runner.
- Runner behavior where only validator-approved proposals proceed to proposal application.
- Module-level reporting of validator decisions and validator rejections, distinct from application-level skips.
- Deterministic fake-module tests covering approvals, rejections, validator order/filtering, and cardinality failure pipeline-stop behavior.
Not implemented in Phase 8 (by design):
- LLM-backed validators (Phase 10).
- Structured LLM runtime wiring (Phase 9 follow-up).
- Real correction modules.
- Prompt/response diagnostics runtime wiring.
- End-to-end transcript polishing.
## Phase 9: Structured LLM client and scheduler infrastructure
### Status
Completed for Phase 9 infrastructure scope.
Implemented in this phase so far:
- Added internal structured LLM contract support for caller-provided typed outputs.
- Added `internal/framework/llm` adapter backed by `github.com/jxnl/instructor-go`.
- Confirmed OpenAI-compatible base URL support through the adapter path.
- Added adapter unit tests for model/base URL handling, retries, context cancellation, optional API key behavior, and error redaction.
Explicitly deferred from Phase 9 into later phases:
- Runtime wiring in runner/module infrastructure (without introducing real modules yet).
- Wiring prompt/response diagnostics primitives into future module/validator call sites.
- Wiring effective primary/validation LLM config resolution into runtime LLM call sites.
### Purpose
Implement the provider-neutral LLM infrastructure needed by both proposal generation and LLM-backed validators, without yet implementing real modules.
### Scope completed in this phase
Implemented:
- Provider-neutral internal structured LLM contract with caller-provided typed output decoding.
- OpenAI-compatible structured-output adapter (`instructor-go`) with:
- configurable base URL and model
- optional API key behavior
- retry budget
- timeout-aware HTTP client handling
- context cancellation propagation
- structured response decoding into caller-provided typed targets
- redacted error surfaces.
- Scheduler/semaphore infrastructure for bounded backend concurrency with context-aware acquisition and reliable permit release.
- Effective LLM config resolution helpers for:
- primary LLM settings
- validation inheritance from primary when unset
- validation overrides when set.
- Generic JSON diagnostics primitives for request metadata, request payload, response payload, and optional error payload, with secret redaction.
- Unit tests covering adapter behavior, scheduler behavior, effective config resolution, and diagnostics redaction/JSON validity.
Do not implement:
- Real correction modules.
- LLM-backed validators.
- Prompt text for domain modules.
- End-to-end transcript polishing.
### Expected behavior at end of phase
At the end of Phase 9, the codebase had tested LLM infrastructure primitives, while default CLI runtime behavior remained deterministic preprocessing/reporting because real modules were not implemented yet.
### Definition of done status
Met:
- Structured LLM client infrastructure is implemented and tested.
- OpenAI-compatible structured-output client exists and is tested.
- Scheduler enforces configured concurrency and is tested.
- Primary and validation effective config resolution exists and is tested.
- Prompt/response diagnostics primitives exist and are tested.
- API keys are redacted in LLM adapter errors and diagnostics artifacts; config/report diagnostics redaction remains in place.
- No real module behavior was introduced.
- `go test ./...` passes.
## Phase 10: LLM-backed validators
Completed.
Implemented:
- LLM-backed validator request/response models in `internal/framework/validators`.
- Prompt builders for:
- spoken-form plausibility
- meaning reversal detection
- editorial review
- grammar review
- spoken-word review
- Deterministic batching by `validation_max_prompt_tokens` with stable ordering and no drop/dup behavior.
- LLM validator execution through the internal structured client abstraction (no direct provider calls in validator code).
- Scheduler/concurrency hooks for LLM validator calls.
- Prompt/response diagnostics artifact writing for LLM validator batches using Phase 9 diagnostics primitives.
- Secret redaction in validator LLM diagnostics artifacts.
- Strict structured-response safety and cardinality checks (missing/duplicate/unknown indexes fail closed).
- Runner/report integration so LLM validator decisions and rejections appear in module reports.
- Fake-module and fake-client tests for approval/rejection, malformed output, cardinality errors, batching, scheduler usage, and diagnostics redaction.
Not implemented in Phase 10 (by design):
- Real correction modules (`glossary`, `homophones`, `spoken_word`, `grammar`).
- Real module implementation and full runtime wiring (Phase 12+).
- Domain proposal prompts.
- Default CLI end-to-end transcript polishing behavior.
## Phase 11: Shared LLM proposal generation framework and module registry
Completed.
Implemented:
- Shared proposal-generation package `internal/framework/proposal_generation`.
- Reusable request model for proposal generation including:
- module key/instance
- replacement policy
- working transcript context
- optional section metadata
- glossary/config context
- diagnostics context
- injected structured LLM client/scheduler dependencies.
- Structured correction-set response model and parsing into existing proposal models:
- `proposals.CorrectionProposal`
- `proposals.EnrichedCorrectionProposal`.
- Deterministic proposal-index assignment via caller-provided start index.
- Proposal-generation diagnostics artifact writing using generic LLM diagnostics primitives with secret redaction.
- Scheduler-aware proposal generation through the internal LLM scheduler interface.
- Production module-registry scaffolding in `internal/framework/modules` with:
- known module-key recognition for `glossary`, `homophones`, `spoken_word`, `grammar`
- constructor registration and dependency-injection path
- explicit unsupported and recognized-but-unimplemented module errors.
- Runner/CLI injection-path tests showing shared proposal generation can flow through runner validation/application semantics using fake modules/clients.
Not implemented in Phase 11 (by design):
- Real production `glossary`, `homophones`, and `spoken_word` modules.
- Domain proposal prompts for production modules.
- Default CLI end-to-end transcript polishing behavior.
## Phase 12: Grammar module
Completed.
Implemented:
- Production grammar module package in `internal/modules/grammar`.
- Grammar prompt builder aligned to Python intent and constrained to punctuation/capitalization/spacing/article cleanup.
- Grammar proposal generation through shared `internal/framework/proposal_generation` using `contracts.StructuredLLMClient`.
- Scheduler-aware grammar proposal generation through existing scheduler hooks.
- Grammar replacement policy `require_unique` (matching Python behavior).
- Grammar validator chain using existing deterministic and LLM-backed validator infrastructure.
- Grammar confidence threshold enforcement through existing config + confidence-threshold validator behavior.
- Explicit runtime support for `--modules grammar` through normalization, chunking, runner, proposal generation, validation, application, and reporting.
- Prompt/response diagnostics artifacts for grammar proposal + validator interactions with secret redaction.
- Module-level reports for grammar including validator decisions/rejections, applied changes, and skipped changes.
- CLI/runtime fake-client tests for approved proposals, validator rejection, application skips, diagnostics, failure/error.log behavior, and report outputs (`--report-json` and run-dir `report.json`).
Not implemented in Phase 12 (by design):
- Production `glossary`, `homophones`, and `spoken_word` modules.
- Full default module sequence execution as a feature-complete claim.
## Phase 13: Glossary module and protected-term behavior
Completed.
Implemented:
- Production glossary module package in `internal/modules/glossary`.
- Glossary prompt builder aligned to Python intent and constrained to glossary-supported domain/acoustic corrections.
- Prompt context using glossary names, aliases, categories, summaries, and plural forms where available.
- Glossary proposal generation through shared `internal/framework/proposal_generation` using `contracts.StructuredLLMClient`.
- Scheduler-aware glossary proposal generation through existing scheduler hooks.
- Glossary replacement policy `replace_all` (matching Python behavior).
- Glossary validator chain using existing deterministic and LLM-backed validators.
- Glossary confidence threshold enforcement through existing config + confidence-threshold validator behavior.
- Deterministic glossary-derived protected-term extraction (`internal/framework/validators/protected_terms.go`) from names, aliases, and plural forms, with stable deduplicated ordering.
- Protected-term validator behavior remaining available to non-glossary modules via existing deterministic validators.
- Explicit runtime support for `--modules glossary` through normalization, chunking, runner, proposal generation, validation, application, and reporting.
- Repeated glossary-stage support (`--modules glossary,glossary`) with deterministic instance naming and mutable working-transcript handoff across stages.
- Prompt/response diagnostics artifacts for glossary proposal + validator interactions with secret redaction.
- Module-level reports for glossary including generated proposals, validator decisions/rejections, applied changes, and application skips.
- CLI/runtime fake-client tests for approved proposals, validator rejection, application skips, repeated stages, diagnostics, failure/error.log behavior, and report outputs (`--report-json` and run-dir `report.json`).
Not implemented in Phase 13 (by design):
- Production `homophones` and `spoken_word` modules.
- Full default module sequence execution as a feature-complete claim.
## Phase 14: Homophones module
Completed.
Implemented:
- Production homophones module package in `internal/modules/homophones`.
- Homophones prompt builder aligned to Python intent and constrained to conservative homophone/near-homophone/mistranscription corrections.
- Prompt context using glossary/protected-term information (names, aliases, plurals where present) to avoid damaging known domain terms.
- Homophones proposal generation through shared `internal/framework/proposal_generation` using `contracts.StructuredLLMClient`.
- Scheduler-aware homophones proposal generation through existing scheduler hooks.
- Homophones replacement policy `require_unique` (matching Python behavior).
- Homophones validator chain using existing deterministic and LLM-backed validators.
- Homophones confidence threshold enforcement through existing config + confidence-threshold validator behavior.
- Protected-term guardrails remaining active for homophones via existing deterministic validators.
- Explicit runtime support for `--modules homophones` through normalization, chunking, runner, proposal generation, validation, application, and reporting.
- Prompt/response diagnostics artifacts for homophones proposal + validator interactions with secret redaction.
- Module-level reports for homophones including generated proposals, validator decisions/rejections, applied changes, and application skips.
- CLI/runtime fake-client tests for approved proposals, validator rejection, application skips, diagnostics, protected-term rejection behavior, failure/error.log behavior, and report outputs (`--report-json` and run-dir `report.json`).
- Focused interoperability tests for already-implemented module combinations (for example `glossary,homophones`) to verify working-transcript handoff and guardrails without claiming full default-sequence parity.
Not implemented in Phase 14 (by design):
- Production `spoken_word` module.
- Full default module sequence execution as a feature-complete claim.
## Phase 15: Spoken-word module
Completed.
Implemented:
- Production spoken_word module package in `internal/modules/spoken_word`.
- Spoken_word prompt builder aligned to Python intent and constrained to conservative dysfluency cleanup.
- Prompt context using glossary/protected-term information (names, aliases, plurals where present) to avoid damaging known domain terms.
- Strong prompt guardrails preserving meaning, intent, speaker voice, named entities, game/domain terms, and substantive content.
- Explicit prompt guardrails against summarization, style rewriting, grammar-only cleanup, punctuation-only cleanup, invention, event reordering, and certainty inflation.
- Spoken_word proposal generation through shared `internal/framework/proposal_generation` using `contracts.StructuredLLMClient`.
- Scheduler-aware spoken_word proposal generation through existing scheduler hooks.
- Spoken_word replacement policy `require_unique` (matching Python behavior).
- Spoken_word validator chain using existing deterministic and LLM-backed validators.
- Strong semantic guardrails in runtime validator chain through existing LLM-backed validators (`spoken_word_review`, `meaning_reversal_review`).
- Spoken-word confidence threshold enforcement through existing config + confidence-threshold validator behavior.
- Protected-term guardrails remaining active for spoken_word via existing deterministic validators.
- Explicit runtime support for `--modules spoken_word` through normalization, chunking, runner, proposal generation, validation, application, and reporting.
- Prompt/response diagnostics artifacts for spoken_word proposal + validator interactions with secret redaction.
- Module-level reports for spoken_word including generated proposals, validator decisions/rejections, applied changes, and application skips.
- CLI/runtime fake-client tests for approved cleanup, validator rejection, meaning-changing rejection, application skips, diagnostics, protected-term rejection behavior, failure/error.log behavior, and report outputs (`--report-json` and run-dir `report.json`).
- Focused interoperability tests for already-implemented module combinations (for example `spoken_word,grammar`) to verify working-transcript handoff and guardrails without claiming full default-sequence parity.
Not implemented in Phase 15 (by design):
- Full default module sequence execution as a feature-complete claim.
## Phase 16: Default full pipeline integration
Completed.
### Purpose
Enable and harden the full default module sequence in the Go runtime path.
### Scope
Implement:
- Default runtime sequence:
- `glossary`
- `homophones`
- `glossary`
- `spoken_word`
- `grammar`
- End-to-end execution through all real modules.
- Accurate resolved module instance names.
- Module-level report aggregation.
- Full run-level applied/skipped summaries.
- Skip-aware retention behavior using actual skipped correction data.
- Failure behavior with partial module progress.
- PipelineRunError or equivalent partial-progress error type if not already implemented.
- Diagnostics for each module instance.
- Tests for successful full pipeline using fake LLM.
- Tests for mid-pipeline failure preserving partial report and diagnostics.
Do not implement:
- Python archive removal.
- Side-by-side rollout tooling beyond what is needed for tests.
### Expected behavior at end of phase
Running `audita process transcript.json --glossary glossary.yaml --output corrected.json` should execute the full Go module pipeline and produce a corrected transcript.
### Definition of done
- Default module sequence runs end-to-end.
- All real modules participate in runtime path.
- Reports include all module instances.
- Applied/skipped changes are aggregated at run level.
- Failed runs retain useful partial reports and diagnostics.
- `auto` retention keeps successful runs with skipped corrections.
- CLI stdout/stderr behavior remains subprocess-safe.
- `go test ./...` passes without requiring external LLM credentials.
### Phase 16 completion status
Implemented:
- Normal `audita process` runs without `--modules` now execute the full sequence:
- `glossary`
- `homophones`
- `glossary`
- `spoken_word`
- `grammar`
- Explicit `--modules` still overrides the default sequence.
- Repeated glossary stages are deterministic (`glossary_1`, `glossary_2`) and reported distinctly.
- Full-pipeline module reports aggregate applied changes, application skips, validator rejections, and failed-module metadata.
- Full-pipeline diagnostics include prompt/response artifacts for module proposal generation and validator LLM interactions with secret redaction.
- Mid-pipeline failure preserves partial module progress in reports and retains diagnostics + `error.log`.
- Auto-retention keeps successful runs with actual skipped/rejected corrections and always retains failed runs.
Intentionally deferred:
- Phase 17 parity fixture suite.
- Phase 18 operational hardening.
- Phase 19 rollout/Python retirement work.
## Phase 17: Python parity fixture suite
Completed.
### Purpose
Establish confidence that the Go implementation matches the behavior and safety posture of the initial Python implementation.
### Scope
Implement:
- Parity fixtures based on representative Python-era inputs and expected behaviors.
- Fake LLM response fixtures where exact deterministic behavior is required.
- Golden tests for:
- transcript schema handling
- glossary schema handling
- normalization
- chunking
- proposal application
- validator behavior
- module reports
- diagnostics artifacts
- default pipeline shape
- Comparison tests or scripts that can run Python and Go side by side where practical.
- Documentation of intentional differences between Python and Go.
Do not require:
- Real LLM credentials for normal automated tests.
- Exact nondeterministic natural-language output equivalence across Python and Go.
### Expected behavior at end of phase
The repository has a durable test suite demonstrating that the Go implementation preserves the functional contract of the Python implementation.
### Definition of done
- Representative parity fixtures exist.
- Golden tests cover deterministic behavior.
- Fake LLM tests cover full pipeline behavior.
- Intentional differences from Python are documented.
- Unintentional compatibility breaks are fixed.
- `go test ./...` passes.
### Phase 17 completion status
Implemented:
- Durable parity fixture harness in Go test path (`internal/cli/parity_test.go`).
- Representative parity fixture corpus under `internal/cli/testdata/parity`.
- Fake LLM proposal/validator fixtures driving deterministic parity tests.
- Default full-pipeline parity coverage including sequence/order and repeated glossary naming:
- `glossary_1`
- `homophones`
- `glossary_2`
- `spoken_word`
- `grammar`
- Parity checks that are strict for deterministic contract fields:
- transcript content
- module order and instance names
- applied/skipped/rejected counts
- report status and failed-module metadata
- Parity checks that ignore nondeterministic metadata fields:
- timestamps
- run IDs
- temp/absolute paths
- provider token usage details
- Documentation of intentional Python-vs-Go differences and honest open parity gaps in `docs/python-parity.md`.
- Normal `go test ./...` path remains independent of real LLM credentials and Python dependencies.
Intentionally deferred:
- Phase 18 operational hardening and subprocess integration expansion.
- Phase 19 rollout and Python retirement work.
## Phase 18: Operational hardening and subprocess integration
### Purpose
Harden the Go binary for use as the production Audita implementation in the surrounding application suite.
### Scope
Implement:
- Additional subprocess tests for large transcripts.
- Timeout/cancellation tests.
- Failure-mode tests for unreadable inputs, unwritable outputs, malformed LLM responses, and backend failures.
- Clear stderr summaries pointing to diagnostics.
- Review of all output paths and file-close behavior.
- Review of all secret redaction paths.
- Review of LLM retry and timeout behavior.
- Documentation for production use from orchestrators such as Narratio.
### Expected behavior at end of phase
The Go binary should be safe to call from other Go applications and should not reproduce the Python subprocess/stdio integration problems.
### Definition of done
- Subprocess tests cover success, failure, large inputs, and cancellation.
- stdout contains machine output only.
- stderr contains human-readable logs/errors only.
- API keys do not appear in reports, diagnostics, logs, or tests.
- Failed runs always preserve diagnostics.
- Operational docs are accurate.
- `go test ./...` passes.
### Phase 18 completion status
Completed.
Implemented:
- Expanded subprocess integration coverage in `cmd/audita/main_integration_test.go` for:
- successful default full-pipeline runs with `--output` and without `--output`;
- successful `--report-json` writes;
- large-transcript subprocess behavior;
- missing/unreadable transcript and missing/malformed glossary failures (portable handling where required);
- unwritable output and unwritable report-json failure behavior (portable handling where required);
- malformed structured LLM response failure behavior;
- synthetic backend LLM failure behavior;
- timeout/cancellation behavior with deterministic context cancellation hooks;
- mid-pipeline failure with partial module progress preserved in reports.
- Hardened subprocess failure stderr output to include diagnostics path when available.
- Added deterministic test-only subprocess LLM/runtime hooks used only in helper-process tests:
- backend failure mode;
- malformed structured-output mode;
- block-until-cancel mode;
- mid-pipeline fail mode.
- Added regression coverage for secret redaction across subprocess stdout/stderr/report/diagnostics artifacts.
- Confirmed existing LLM adapter and scheduler tests continue to cover:
- timeout and context cancellation propagation;
- retry behavior;
- malformed output safety;
- permit release on error/cancellation.
- Added focused subprocess-caller operational guidance in `docs/subprocess-operations.md`.
Historical note:
- Phase 19 documentation/rollout/Python-retirement transitions were deferred at the end of Phase 18 and completed in Phase 19.
## Phase 19: Documentation, rollout, and Python retirement
### Purpose
Make the Go implementation the documented active implementation and preserve the Python implementation only as historical reference if desired.
### Scope
Implement:
- README updates.
- Architecture updates.
- Rewrite notes updates.
- Installation/build instructions for Go binary.
- Migration notes from Python to Go.
- Documentation of any intentionally changed behavior.
- Removal or archival of Python-specific operational instructions from the primary path.
- Clear statement that the Go implementation is now feature-complete.
### Expected behavior at end of phase
The repository clearly presents the Go implementation as the active Audita implementation.
### Definition of done
- Documentation no longer describes Go as only a deterministic foundation.
- Documentation accurately describes full transcript polishing behavior.
- Python implementation is marked archived/prototype/reference, or removed if that is the chosen repository policy.
- Users can build, test, and run the Go implementation from docs alone.
- `go test ./...` passes.
### Phase 19 completion status
Completed.
Implemented:
- README is Go-first and documents active full-pipeline runtime behavior.
- Build/install/test usage guidance is Go-primary and operationally oriented.
- Subprocess/orchestrator behavior is documented in `docs/subprocess-operations.md`.
- Migration guidance from Python to Go is documented in `docs/migration-from-python.md`.
- Python parity notes and open parity gaps remain documented in `docs/python-parity.md`.
- Python implementation status is explicitly legacy/reference in primary docs and `python/README.md`.
- Documentation no longer routes normal operations to Python.
- Normal `go test ./...` remains independent of real LLM credentials and Python dependencies.
## Cross-phase compatibility requirements
These constraints were used across implementation phases and remain useful maintenance checks:
- Preserve subprocess-safe stdout/stderr behavior.
- Preserve CLI-over-env precedence.
- Preserve existing accepted transcript input forms.
- Preserve glossary YAML compatibility unless a deliberate migration is documented.
- Preserve deterministic safety-first proposal application semantics.
- Preserve run-dir diagnostics and report writing.
- Preserve API key redaction.
- Do not require real LLM credentials for normal `go test ./...`.
- Do not claim a feature is implemented until it is in the runtime path.
- Keep default behavior honest in docs and CLI help.
- Prefer fake LLMs and golden fixtures for automated tests.
- Keep real LLM smoke tests opt-in.
## Guidance for Codex-style prompts
When requesting implementation work, use one phase at a time.
Good prompt shape:
- Name the exact phase.
- State what is in scope.
- State what is explicitly out of scope.
- Require tests.
- Require `go test ./...`.
- Require documentation updates only when the phase changes user-visible or architectural status.
- Require that no later-phase features be implemented opportunistically.
Avoid broad prompts such as:
- “finish the rewrite”
- “make Audita work”
- “port the Python app”
- “implement all modules”
Those prompts blur phase boundaries and make review difficult.
## Suggested review checklist for every phase
Before accepting a phase implementation, verify:
- Does the implementation match the phase scope?
- Did it avoid implementing unrelated later-phase behavior?
- Does `go test ./...` pass?
- Are stdout and stderr still clean for subprocess callers?
- Are API keys redacted everywhere?
- Are reports machine-readable?
- Are diagnostics sufficient for debugging?
- Are fake LLM tests used instead of requiring real credentials?
- Are new public behaviors documented?
- Does the code follow the architecture in `docs/architecture.md`?
- Does the implementation move Audita closer to Python feature parity?