Cleanup pass to remove refactoring-related artifacts and references

This commit is contained in:
2026-05-12 15:19:50 -05:00
parent af84249da0
commit 390daa8b84
24 changed files with 138 additions and 173 deletions

12
.gitignore vendored
View File

@@ -45,6 +45,17 @@ go.work.sum
narratio narratio
local-test local-test
pipeline.yml pipeline.yml
bin/
# Local run artifacts
.audita-runs/
report.json
corrected.json
normalized.json
# Coverage artifacts
coverage.out
coverage.txt
# ---> VisualStudioCode # ---> VisualStudioCode
.vscode/* .vscode/*
@@ -59,4 +70,3 @@ pipeline.yml
# Built Visual Studio Code Extensions # Built Visual Studio Code Extensions
*.vsix *.vsix

143
README.md
View File

@@ -1,28 +1,27 @@
# Audita (Go) # Audita
Audita is a transcript polishing CLI. Audita is a transcript polishing CLI.
The Go implementation in this repository is the active implementation. It runs a full default correction pipeline over transcript JSON using glossary context, LLM-backed proposal generation, validator chains, deterministic proposal application, and structured reports/diagnostics. `audita process` validates transcript/glossary input, normalizes and chunks transcript segments, runs the default correction pipeline, and emits corrected transcript output plus machine-readable diagnostics and reports.
## What Audita does ## What Audita Does
`audita process` performs: Default module sequence:
- transcript/glossary schema validation; - `glossary`
- deterministic normalization and chunking; - `homophones`
- default module sequence: - `glossary`
- `glossary` - `spoken_word`
- `homophones` - `grammar`
- `glossary`
- `spoken_word`
- `grammar`
- glossary-backed domain/acoustic corrections;
- conservative homophone and likely mistranscription corrections;
- conservative spoken-word cleanup (dysfluencies/fillers) with semantic guardrails;
- grammar/punctuation/capitalization/formatting cleanup;
- machine-readable process and module reports;
- per-run diagnostics artifacts with secret redaction.
## Build and install Pipeline behavior includes:
- glossary-backed domain/acoustic corrections
- conservative homophone and mistranscription corrections
- conservative spoken-word dysfluency cleanup with semantic guardrails
- grammar/punctuation/capitalization/formatting cleanup
- validator-chain enforcement before application
- run reports and diagnostics artifacts with secret redaction
## Build and Install
Build a local binary: Build a local binary:
@@ -36,7 +35,7 @@ Install into your Go bin directory:
go install ./cmd/audita go install ./cmd/audita
``` ```
Run help: CLI help:
```sh ```sh
audita --help audita --help
@@ -45,21 +44,19 @@ audita process --help
## Test ## Test
Run the full test suite: Run all tests:
```sh ```sh
go test ./... go test ./...
``` ```
Normal tests are deterministic and do not require real LLM credentials or Python dependencies. ## Basic Usage
## Basic usage
Required inputs: Required inputs:
- transcript JSON path (positional argument) - transcript JSON path (positional argument)
- `--glossary <glossary.yaml>` - `--glossary <glossary.yaml>`
Default full pipeline (recommended local example): Recommended run:
```sh ```sh
audita process transcript.json \ audita process transcript.json \
@@ -78,13 +75,13 @@ audita process transcript.json \
--report-json report.json --report-json report.json
``` ```
Emit transcript JSON to stdout (no `--output`): Write transcript JSON to stdout (no `--output`):
```sh ```sh
audita process transcript.json --glossary glossary.yaml audita process transcript.json --glossary glossary.yaml
``` ```
Diagnostics/work-dir control: Control diagnostics location/retention:
```sh ```sh
audita process transcript.json \ audita process transcript.json \
@@ -95,17 +92,14 @@ audita process transcript.json \
--report-json report.json --report-json report.json
``` ```
## Stdout/stderr and orchestration behavior ## Stdout/Stderr Contract
- With `--output`, stdout should be empty on success. - With `--output`, stdout is expected to be empty on success.
- Without `--output`, stdout contains transcript JSON only on success. - Without `--output`, stdout contains transcript JSON only on success.
- `--report-json` writes report JSON to file; report JSON is never printed to stdout. - `--report-json` writes a file and is never printed to stdout.
- stderr is for human-readable warnings/errors. - stderr is human-readable diagnostics/errors.
For parent-process integration guidance, see: For subprocess orchestration guidance, see [`docs/subprocess-operations.md`](docs/subprocess-operations.md).
- [`docs/subprocess-operations.md`](docs/subprocess-operations.md)
For orchestrated runs, use both `--output` and `--report-json`.
## Configuration ## Configuration
@@ -114,13 +108,14 @@ Precedence:
2. environment (`AUDITA_*`) 2. environment (`AUDITA_*`)
3. CLI flags 3. CLI flags
### Module sequence ### Modules
- `AUDITA_MODULES` (CSV, e.g. `glossary,homophones,glossary,spoken_word,grammar`) - `AUDITA_MODULES` (CSV)
- CLI override: `--modules` - CLI: `--modules`
### Primary LLM settings ### Primary LLM
Environment:
- `AUDITA_LLM_API_KEY` (or `OPENROUTER_API_KEY` fallback) - `AUDITA_LLM_API_KEY` (or `OPENROUTER_API_KEY` fallback)
- `AUDITA_MODEL` - `AUDITA_MODEL`
- `AUDITA_BASE_URL` - `AUDITA_BASE_URL`
@@ -128,15 +123,17 @@ Precedence:
- `AUDITA_MAX_RETRIES` - `AUDITA_MAX_RETRIES`
- `AUDITA_LLM_CONCURRENCY` - `AUDITA_LLM_CONCURRENCY`
CLI overrides: CLI:
- `--llm-api-key` - `--llm-api-key`
- `--model` - `--model`
- `--base-url` - `--base-url`
- `--llm-timeout-seconds` - `--llm-timeout-seconds`
- `--max-retries` - `--max-retries`
- `--llm-concurrency`
### Validation LLM settings ### Validation LLM
Environment:
- `AUDITA_VALIDATION_LLM_API_KEY` - `AUDITA_VALIDATION_LLM_API_KEY`
- `AUDITA_VALIDATION_MODEL` - `AUDITA_VALIDATION_MODEL`
- `AUDITA_VALIDATION_BASE_URL` - `AUDITA_VALIDATION_BASE_URL`
@@ -145,7 +142,7 @@ CLI overrides:
- `AUDITA_VALIDATION_LLM_CONCURRENCY` - `AUDITA_VALIDATION_LLM_CONCURRENCY`
- `AUDITA_VALIDATION_MAX_PROMPT_TOKENS` - `AUDITA_VALIDATION_MAX_PROMPT_TOKENS`
CLI overrides: CLI:
- `--validation-llm-api-key` - `--validation-llm-api-key`
- `--validation-model` - `--validation-model`
- `--validation-base-url` - `--validation-base-url`
@@ -154,25 +151,27 @@ CLI overrides:
- `--validation-llm-concurrency` - `--validation-llm-concurrency`
- `--validation-max-prompt-tokens` - `--validation-max-prompt-tokens`
Validation LLM inheritance behavior: Validation concurrency behavior:
- unset validation fields inherit from primary LLM config; - when validation concurrency is unset, it inherits primary `llm-concurrency`
- set validation fields override primary values for validation calls only. - when explicitly set, validation concurrency must be `<= llm-concurrency`
### Confidence thresholds ### Confidence Thresholds
Environment:
- `AUDITA_GLOSSARY_CONFIDENCE_THRESHOLD` - `AUDITA_GLOSSARY_CONFIDENCE_THRESHOLD`
- `AUDITA_HOMOPHONES_CONFIDENCE_THRESHOLD` - `AUDITA_HOMOPHONES_CONFIDENCE_THRESHOLD`
- `AUDITA_SPOKEN_WORD_CONFIDENCE_THRESHOLD` - `AUDITA_SPOKEN_WORD_CONFIDENCE_THRESHOLD`
- `AUDITA_GRAMMAR_CONFIDENCE_THRESHOLD` - `AUDITA_GRAMMAR_CONFIDENCE_THRESHOLD`
CLI overrides: CLI:
- `--glossary-confidence-threshold` - `--glossary-confidence-threshold`
- `--homophones-confidence-threshold` - `--homophones-confidence-threshold`
- `--spoken-word-confidence-threshold` - `--spoken-word-confidence-threshold`
- `--grammar-confidence-threshold` - `--grammar-confidence-threshold`
### Normalization and chunking ### Normalization and Chunking
Environment:
- `AUDITA_NORMALIZE_MAX_SEGMENT_GAP` - `AUDITA_NORMALIZE_MAX_SEGMENT_GAP`
- `AUDITA_NORMALIZE_ELLIPSIS_GAP` - `AUDITA_NORMALIZE_ELLIPSIS_GAP`
- `AUDITA_NORMALIZE_MAX_SEGMENT_DURATION` - `AUDITA_NORMALIZE_MAX_SEGMENT_DURATION`
@@ -181,7 +180,7 @@ CLI overrides:
- `AUDITA_MIN_SECTION_TOKENS` - `AUDITA_MIN_SECTION_TOKENS`
- `AUDITA_TARGET_SECTIONS` - `AUDITA_TARGET_SECTIONS`
CLI overrides: CLI:
- `--normalize-max-segment-gap` - `--normalize-max-segment-gap`
- `--normalize-ellipsis-gap` - `--normalize-ellipsis-gap`
- `--normalize-max-segment-duration` - `--normalize-max-segment-duration`
@@ -190,46 +189,38 @@ CLI overrides:
- `--min-section-tokens` - `--min-section-tokens`
- `--target-sections` - `--target-sections`
### Work-dir and retention ### Work Directory
Environment:
- `AUDITA_WORK_DIR` - `AUDITA_WORK_DIR`
- `AUDITA_WORK_DIR_RETENTION` (`auto`, `always`, `never`) - `AUDITA_WORK_DIR_RETENTION` (`auto`, `always`, `never`)
CLI overrides: CLI:
- `--work-dir` - `--work-dir`
- `--work-dir-retention` - `--work-dir-retention`
Retention summary: Retention behavior:
- `always`: keep all run directories. - `always`: keep all run directories
- `never`: keep successful run directories. - `never`: keep successful run directories
- `auto`: keep failed runs and successful runs with skipped/rejected corrections. - `auto`: keep failed runs and successful runs with skipped/rejected corrections
## Report and diagnostics ## Reports and Diagnostics
Per-run diagnostics include: Per-run diagnostics include:
- source transcript artifacts; - source transcript artifacts
- normalized transcript artifact; - normalized transcript artifact
- normalization summary; - normalization summary
- chunking summary; - chunking summary
- invocation metadata; - invocation metadata
- redacted effective config; - redacted effective config
- prompt/response diagnostics for module and validator LLM interactions; - module/validator prompt-response diagnostics
- `report.json`; - `report.json`
- `error.log` on failure. - `error.log` on failure
Optional external report output: Optional external report output:
- `--report-json <path>` - `--report-json <path>`
## Legacy Python reference ## Documentation
The original Python implementation is preserved in [`python/`](python/) as a legacy/reference implementation for parity history and migration context.
For migration guidance, see [`docs/migration-from-python.md`](docs/migration-from-python.md).
Parity fixture notes and intentional differences:
- [`docs/python-parity.md`](docs/python-parity.md)
## Additional docs
- Architecture: [`docs/architecture.md`](docs/architecture.md) - Architecture: [`docs/architecture.md`](docs/architecture.md)
- Rewrite history and phase notes: [`docs/rewrite-notes.md`](docs/rewrite-notes.md) - Subprocess operations: [`docs/subprocess-operations.md`](docs/subprocess-operations.md)
- Migration from Python to Go: [`docs/migration-from-python.md`](docs/migration-from-python.md)

View File

@@ -312,7 +312,7 @@ func TestProcessSuccessReportJSONSubprocess(t *testing.T) {
t.Fatalf("expected valid report JSON, got %q", string(report)) t.Fatalf("expected valid report JSON, got %q", string(report))
} }
// Ensure report JSON is not printed to stdout. // Ensure report JSON is not printed to stdout.
if strings.Contains(result.stdout, `"phase16-default-pipeline-integration"`) { if strings.Contains(result.stdout, `"default_pipeline"`) {
t.Fatalf("report JSON leaked to stdout: %q", result.stdout) t.Fatalf("report JSON leaked to stdout: %q", result.stdout)
} }
} }
@@ -493,7 +493,7 @@ func TestProcessCancellationViaSubprocessTimeoutHook(t *testing.T) {
} }
func TestProcessSubprocessNoSecretLeakInOutputsAndDiagnostics(t *testing.T) { func TestProcessSubprocessNoSecretLeakInOutputsAndDiagnostics(t *testing.T) {
secret := "phase18-subprocess-secret" secret := "subprocess-secret"
workDir := t.TempDir() workDir := t.TempDir()
reportPath := filepath.Join(t.TempDir(), "report.json") reportPath := filepath.Join(t.TempDir(), "report.json")
outputPath := filepath.Join(t.TempDir(), "out.json") outputPath := filepath.Join(t.TempDir(), "out.json")

View File

@@ -1,11 +1,10 @@
# Audita Go Architecture # Audita Architecture
## Scope and intent ## Scope and intent
This document describes: This document describes:
- the implemented Go architecture used in production today; and - the architecture used in production today.
- historical rewrite-phase context that explains how the architecture was delivered.
Status labels are explicit so future engineers and LLM agents do not infer missing behavior that is not actually missing. Historical rewrite details live in `docs/rewrite-notes.md`.
## Current implementation status ## Current implementation status
Implemented today: Implemented today:
@@ -14,7 +13,7 @@ Implemented today:
- Transcript and glossary parsing/validation. - Transcript and glossary parsing/validation.
- Deterministic transcript normalization. - Deterministic transcript normalization.
- Deterministic token estimation and transcript chunking. - Deterministic token estimation and transcript chunking.
- Per-run diagnostics directory creation plus Phase 6 process-level artifacts. - Per-run diagnostics directory creation plus process-level artifacts.
- Process report JSON output with diagnostics artifact references. - Process report JSON output with diagnostics artifact references.
- Framework foundation packages for contracts and proposal application. - Framework foundation packages for contracts and proposal application.
- Production runner orchestration package with deterministic sequential module execution. - Production runner orchestration package with deterministic sequential module execution.
@@ -32,7 +31,7 @@ Implemented today:
- Shared LLM proposal-generation helper with structured correction-set parsing. - Shared LLM proposal-generation helper with structured correction-set parsing.
- Deterministic proposal-index assignment and enriched proposal mapping for shared generation. - Deterministic proposal-index assignment and enriched proposal mapping for shared generation.
- Proposal-generation diagnostics artifacts with secret redaction. - Proposal-generation diagnostics artifacts with secret redaction.
- Production module registry scaffolding with known-key recognition and explicit unsupported/unimplemented errors. - Production module registry with known-key recognition and explicit unsupported-module errors.
- Production `grammar` module implementation in `internal/modules/grammar`. - Production `grammar` module implementation in `internal/modules/grammar`.
- Production `glossary` module implementation in `internal/modules/glossary`. - Production `glossary` module implementation in `internal/modules/glossary`.
- Production `homophones` module implementation in `internal/modules/homophones`. - Production `homophones` module implementation in `internal/modules/homophones`.
@@ -52,19 +51,6 @@ Current reality:
- `grammar` - `grammar`
- repeated glossary stages are deterministic and reported distinctly as `glossary_1` and `glossary_2`. - repeated glossary stages are deterministic and reported distinctly as `glossary_1` and `glossary_2`.
Phase sequencing note:
- Phase 9 LLM infrastructure is complete (structured client, scheduler, effective config resolution, diagnostics primitives);
- Phase 10 LLM-backed validator runtime integration is complete;
- Phase 11 shared proposal-generation framework and module-registry scaffolding are complete;
- Phase 12 grammar module implementation and explicit runtime wiring are complete;
- Phase 13 glossary module and protected-term behavior are complete;
- Phase 14 homophones module implementation and explicit runtime wiring are complete;
- Phase 15 spoken-word module implementation and explicit runtime wiring are complete;
- Phase 16 default full pipeline integration is complete;
- Phase 17 parity fixture suite is complete;
- Phase 18 operational hardening and subprocess integration are complete;
- Phase 19 documentation/rollout completion is complete.
## Actual Go package layout ## Actual Go package layout
```text ```text
@@ -177,7 +163,7 @@ Current runtime flow (`internal/cli/run.go`):
- explicit `--modules` overrides the default sequence; - explicit `--modules` overrides the default sequence;
- test/injected module factory path remains available for deterministic runtime tests. - test/injected module factory path remains available for deterministic runtime tests.
13. Output working transcript to `--output` file or stdout. 13. Output working transcript to `--output` file or stdout.
14. Build process report (`phase` currently set to `phase16-default-pipeline-integration`). 14. Build process report (`phase` currently set to `default_pipeline`).
15. Optionally write `--report-json`; always write run-dir `report.json`. 15. Optionally write `--report-json`; always write run-dir `report.json`.
16. Apply work-dir retention. 16. Apply work-dir retention.
@@ -290,7 +276,7 @@ Current behavior details:
- section balancing is deterministic but heuristic. - section balancing is deterministic but heuristic.
## Implemented proposal/replacement infrastructure ## Implemented proposal/replacement infrastructure
`internal/framework/proposals` provides deterministic foundation logic: `internal/framework/proposals` provides deterministic proposal composition logic:
- `CorrectionProposal` and `EnrichedCorrectionProposal` models; - `CorrectionProposal` and `EnrichedCorrectionProposal` models;
- replacement policies: `require_unique`, `replace_all`; - replacement policies: `require_unique`, `replace_all`;
- safe preview (`PreviewProposalForSegment`) with stable skip reasons; - safe preview (`PreviewProposalForSegment`) with stable skip reasons;
@@ -362,7 +348,7 @@ This helper only produces candidate proposals; validator-chain execution and pro
- proposal/validation structured LLM clients - proposal/validation structured LLM clients
- proposal/validation schedulers - proposal/validation schedulers
- diagnostics directory context - diagnostics directory context
- returns explicit errors for unknown keys (`unsupported_module`) and recognized-but-unimplemented keys (`unimplemented_module`). - returns explicit errors for unknown keys (`unsupported_module`).
The `grammar`, `glossary`, `homophones`, and `spoken_word` module keys are now registered and constructible. The `grammar`, `glossary`, `homophones`, and `spoken_word` module keys are now registered and constructible.
@@ -372,7 +358,7 @@ The `grammar`, `glossary`, `homophones`, and `spoken_word` module keys are now r
- explicit guardrails against meaning-changing rewrites, style rewrites, summarization, and invention; - explicit guardrails against meaning-changing rewrites, style rewrites, summarization, and invention;
- proposal generation through `internal/framework/proposal_generation` and `contracts.StructuredLLMClient`; - proposal generation through `internal/framework/proposal_generation` and `contracts.StructuredLLMClient`;
- scheduler-aware proposal calls through existing `contracts.LLMScheduler` hooks; - scheduler-aware proposal calls through existing `contracts.LLMScheduler` hooks;
- replacement policy `require_unique` (matching Python implementation); - replacement policy `require_unique` (current runtime policy);
- validator chain integration using existing deterministic + LLM-backed validators; - validator chain integration using existing deterministic + LLM-backed validators;
- grammar confidence threshold enforcement through existing validator/config infrastructure; - grammar confidence threshold enforcement through existing validator/config infrastructure;
- module-level reporting and diagnostics capture through existing runner/reporting paths. - module-level reporting and diagnostics capture through existing runner/reporting paths.
@@ -465,9 +451,6 @@ Current runtime note:
- default non-explicit runs usually have no module-level skipped corrections, so `auto` commonly removes clean successful run directories. - default non-explicit runs usually have no module-level skipped corrections, so `auto` commonly removes clean successful run directories.
- explicit grammar/glossary/homophones/spoken_word runs can produce validator rejections and application skips, which are reflected in reports and retention input. - explicit grammar/glossary/homophones/spoken_word runs can produce validator rejections and application skips, which are reflected in reports and retention input.
Phase note:
- Earlier-phase documentation deferred module and LLM prompt work; those deferred items were completed by Phases 12-16.
## Current tests and quality posture ## Current tests and quality posture
Implemented tests currently cover: Implemented tests currently cover:
- CLI argument handling and behavior (`internal/cli/run_test.go`) - CLI argument handling and behavior (`internal/cli/run_test.go`)
@@ -483,7 +466,7 @@ Implemented tests currently cover:
- validator models, cardinality enforcement, and deterministic validators (`internal/framework/validators/*_test.go`) - validator models, cardinality enforcement, and deterministic validators (`internal/framework/validators/*_test.go`)
- LLM-backed validator batching, prompt builders, structured-response safety, scheduler hooks, and diagnostics redaction (`internal/framework/validators/*_test.go`, `internal/framework/runner/*_test.go`) - LLM-backed validator batching, prompt builders, structured-response safety, scheduler hooks, and diagnostics redaction (`internal/framework/validators/*_test.go`, `internal/framework/runner/*_test.go`)
- shared proposal-generation request/response parsing, deterministic indexing, scheduler hooks, and diagnostics redaction (`internal/framework/proposal_generation/*_test.go`, `internal/framework/runner/*_test.go`) - shared proposal-generation request/response parsing, deterministic indexing, scheduler hooks, and diagnostics redaction (`internal/framework/proposal_generation/*_test.go`, `internal/framework/runner/*_test.go`)
- production module-registry known-key recognition and unsupported/unimplemented error behavior (`internal/framework/modules/*_test.go`, `internal/cli/run_test.go`) - production module-registry known-key recognition and unsupported/internal-registry error behavior (`internal/framework/modules/*_test.go`, `internal/cli/run_test.go`)
- production grammar module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, and explicit CLI/runtime integration (`internal/modules/grammar/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`) - production grammar module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, and explicit CLI/runtime integration (`internal/modules/grammar/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
- production glossary module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, repeated-stage behavior, and explicit CLI/runtime integration (`internal/modules/glossary/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`) - production glossary module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, repeated-stage behavior, and explicit CLI/runtime integration (`internal/modules/glossary/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
- production homophones module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, protected-term behavior, and explicit CLI/runtime integration (`internal/modules/homophones/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`) - production homophones module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, protected-term behavior, and explicit CLI/runtime integration (`internal/modules/homophones/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
@@ -493,7 +476,7 @@ Implemented tests currently cover:
- subprocess operational hardening behavior including large-input, failure-mode, timeout/cancellation, backend-failure, and partial-progress paths (`cmd/audita/main_integration_test.go`) - subprocess operational hardening behavior including large-input, failure-mode, timeout/cancellation, backend-failure, and partial-progress paths (`cmd/audita/main_integration_test.go`)
- report/diagnostics redaction and artifact-shape behavior across success and failure paths (`internal/cli/run_test.go`, `cmd/audita/main_integration_test.go`) - report/diagnostics redaction and artifact-shape behavior across success and failure paths (`internal/cli/run_test.go`, `cmd/audita/main_integration_test.go`)
## Operational hardening status (Phase 18) ## Operational hardening status
The runtime now includes hardened subprocess behavior for parent-process callers: The runtime now includes hardened subprocess behavior for parent-process callers:
- deterministic success/failure exit codes; - deterministic success/failure exit codes;
- strict stdout/stderr separation suitable for machine orchestration; - strict stdout/stderr separation suitable for machine orchestration;
@@ -504,9 +487,7 @@ The runtime now includes hardened subprocess behavior for parent-process callers
Operational caller guidance is documented in [`docs/subprocess-operations.md`](docs/subprocess-operations.md). Operational caller guidance is documented in [`docs/subprocess-operations.md`](docs/subprocess-operations.md).
## Final status (Phase 19 complete) ## Final status
- Go Audita is the active implementation. - Audita's default full module-sequence runtime is implemented and tested.
- Default full module-sequence runtime is implemented and tested.
- Parity fixtures and operational hardening coverage are in place. - Parity fixtures and operational hardening coverage are in place.
- Python is preserved as a legacy/reference implementation in `python/` and is no longer the primary operational path. - Historical migration context is documented in [`docs/migration-from-python.md`](docs/migration-from-python.md).
- Migration and rollout guidance is documented in [`docs/migration-from-python.md`](docs/migration-from-python.md).

View File

@@ -1,6 +1,6 @@
# Python vs Go Parity Notes # Python vs Go Parity Notes
This document tracks Phase 17 parity-fixture coverage and differences between the original Python implementation and the Go rewrite. This document tracks parity-fixture coverage and intentional differences between historical Python behavior and the current Audita runtime.
## Scope ## Scope
@@ -54,7 +54,7 @@ Current parity fixtures cover:
## Open Parity Gaps (Not Intentional) ## Open Parity Gaps (Not Intentional)
These are known Phase 17 expansion opportunities and should not be labeled as intentional compatibility differences: These are known parity expansion opportunities and should not be labeled as intentional compatibility differences:
1. Broader Python fixture import 1. Broader Python fixture import
- The current Go parity fixtures are native fixture cases; they do not yet ingest all existing Python test fixtures directly. - The current Go parity fixtures are native fixture cases; they do not yet ingest all existing Python test fixtures directly.

View File

@@ -1,4 +1,7 @@
# Audita Go Rewrite Notes # Audita Rewrite Notes (Historical)
This document is a historical record of the Python-to-Go rewrite project.
It is retained for engineering context, not as primary runtime guidance.
## Definition of done for the Go rewrite ## Definition of done for the Go rewrite

View File

@@ -1,4 +1,4 @@
# Audita Go Subprocess Operations # Audita Subprocess Operations
This document describes how parent processes should invoke `audita process` safely in production orchestration. This document describes how parent processes should invoke `audita process` safely in production orchestration.

View File

@@ -542,7 +542,7 @@ func extractErrorPhase(err error) (phase string, message string) {
func buildProcessReport(status string, inv processInvocation, runDir *diagnostics.RunDirectory, startedAt, completedAt time.Time, errorMessage string, errorPhase string, normalizationSummary *normalization.NormalizationSummary, chunkingSummary *chunking.Summary, runOutput *runner.RunOutput) reporting.ProcessReport { func buildProcessReport(status string, inv processInvocation, runDir *diagnostics.RunDirectory, startedAt, completedAt time.Time, errorMessage string, errorPhase string, normalizationSummary *normalization.NormalizationSummary, chunkingSummary *chunking.Summary, runOutput *runner.RunOutput) reporting.ProcessReport {
report := reporting.ProcessReport{ report := reporting.ProcessReport{
Phase: "phase16-default-pipeline-integration", Phase: "default_pipeline",
Status: status, Status: status,
Operation: "process", Operation: "process",
TranscriptPath: inv.TranscriptPath, TranscriptPath: inv.TranscriptPath,

View File

@@ -750,8 +750,8 @@ func TestRunProcessReportJSONIncludesChunkingSummary(t *testing.T) {
if report.Chunking.MaxSectionTokens == 0 { if report.Chunking.MaxSectionTokens == 0 {
t.Errorf("expected max_section_tokens in report") t.Errorf("expected max_section_tokens in report")
} }
if report.Phase != "phase16-default-pipeline-integration" { if report.Phase != "default_pipeline" {
t.Errorf("expected phase 'phase16-default-pipeline-integration', got %q", report.Phase) t.Errorf("expected phase 'default_pipeline', got %q", report.Phase)
} }
} }
@@ -1114,7 +1114,7 @@ func TestRunProcessExplicitUnsupportedModulesFailClearly(t *testing.T) {
} }
func TestRunProcessExplicitGrammarAppliesCorrectionAndReportsDiagnostics(t *testing.T) { func TestRunProcessExplicitGrammarAppliesCorrectionAndReportsDiagnostics(t *testing.T) {
secret := "phase12-secret" secret := "grammar-secret"
processProposalLLMClient = &fakeStructuredLLMClient{ processProposalLLMClient = &fakeStructuredLLMClient{
proposalResponses: []proposal_generation.StructuredCorrectionSet{ proposalResponses: []proposal_generation.StructuredCorrectionSet{
{ {
@@ -1309,7 +1309,7 @@ func TestRunProcessExplicitGrammarMalformedLLMOutputFailsWithErrorLog(t *testing
} }
func TestRunProcessDefaultRunExecutesFullModuleSequence(t *testing.T) { func TestRunProcessDefaultRunExecutesFullModuleSequence(t *testing.T) {
secret := "phase16-secret" secret := "pipeline-secret"
proposalClient := &fakeStructuredLLMClient{ proposalClient := &fakeStructuredLLMClient{
proposalResponses: []proposal_generation.StructuredCorrectionSet{ proposalResponses: []proposal_generation.StructuredCorrectionSet{
{Corrections: []proposal_generation.StructuredCorrectionProposal{ {Corrections: []proposal_generation.StructuredCorrectionProposal{
@@ -1531,7 +1531,7 @@ func TestRunProcessDefaultFullPipelineFailurePreservesPartialProgressAndRetentio
} }
func TestRunProcessDefaultFullPipelineAggregatesSkipsAndAutoRetentionKeepsRunDir(t *testing.T) { func TestRunProcessDefaultFullPipelineAggregatesSkipsAndAutoRetentionKeepsRunDir(t *testing.T) {
secret := "phase16-retention-secret" secret := "retention-secret"
proposalClient := &fakeStructuredLLMClient{ proposalClient := &fakeStructuredLLMClient{
proposalResponses: []proposal_generation.StructuredCorrectionSet{ proposalResponses: []proposal_generation.StructuredCorrectionSet{
{Corrections: []proposal_generation.StructuredCorrectionProposal{ {Corrections: []proposal_generation.StructuredCorrectionProposal{
@@ -1648,7 +1648,7 @@ func TestRunProcessDefaultFullPipelineAggregatesSkipsAndAutoRetentionKeepsRunDir
} }
func TestRunProcessExplicitGlossaryAppliesCorrectionAndReportsDiagnostics(t *testing.T) { func TestRunProcessExplicitGlossaryAppliesCorrectionAndReportsDiagnostics(t *testing.T) {
secret := "phase13-secret" secret := "glossary-secret"
proposalClient := &fakeStructuredLLMClient{ proposalClient := &fakeStructuredLLMClient{
proposalResponses: []proposal_generation.StructuredCorrectionSet{ proposalResponses: []proposal_generation.StructuredCorrectionSet{
{ {
@@ -1923,7 +1923,7 @@ func TestRunProcessExplicitGlossaryMalformedLLMOutputFailsWithErrorLog(t *testin
} }
func TestRunProcessExplicitHomophonesAppliesCorrectionAndReportsDiagnostics(t *testing.T) { func TestRunProcessExplicitHomophonesAppliesCorrectionAndReportsDiagnostics(t *testing.T) {
secret := "phase14-secret" secret := "homophones-secret"
proposalClient := &fakeStructuredLLMClient{ proposalClient := &fakeStructuredLLMClient{
proposalResponses: []proposal_generation.StructuredCorrectionSet{ proposalResponses: []proposal_generation.StructuredCorrectionSet{
{ {
@@ -2241,7 +2241,7 @@ func TestRunProcessExplicitGlossaryThenHomophonesSeesWorkingTranscriptChanges(t
} }
func TestRunProcessExplicitSpokenWordAppliesCleanupAndReportsDiagnostics(t *testing.T) { func TestRunProcessExplicitSpokenWordAppliesCleanupAndReportsDiagnostics(t *testing.T) {
secret := "phase15-secret" secret := "spoken-word-secret"
proposalClient := &fakeStructuredLLMClient{ proposalClient := &fakeStructuredLLMClient{
proposalResponses: []proposal_generation.StructuredCorrectionSet{ proposalResponses: []proposal_generation.StructuredCorrectionSet{
{ {

View File

@@ -5,8 +5,8 @@
"proposal_responses_file": "default-full-pipeline.proposals.json", "proposal_responses_file": "default-full-pipeline.proposals.json",
"validation_responses_file": "default-full-pipeline.validations.json", "validation_responses_file": "default-full-pipeline.validations.json",
"env": { "env": {
"AUDITA_LLM_API_KEY": "phase17-secret", "AUDITA_LLM_API_KEY": "parity-secret",
"AUDITA_VALIDATION_LLM_API_KEY": "phase17-secret" "AUDITA_VALIDATION_LLM_API_KEY": "parity-secret"
}, },
"expect": { "expect": {
"exit_code": 0, "exit_code": 0,
@@ -16,7 +16,7 @@
"module_count": 5, "module_count": 5,
"total_applied_changes": 3, "total_applied_changes": 3,
"total_skipped_changes": 3, "total_skipped_changes": 3,
"secret_markers": ["phase17-secret"], "secret_markers": ["parity-secret"],
"expected_proposal_calls": [ "expected_proposal_calls": [
"glossary_1:proposal", "glossary_1:proposal",
"homophones:proposal", "homophones:proposal",

View File

@@ -7,6 +7,6 @@
{"validations": [{"correction_index": 0, "approved": true, "confidence": 0.99, "reason": "ok"}]}, {"validations": [{"correction_index": 0, "approved": true, "confidence": 0.99, "reason": "ok"}]},
{"validations": [{"correction_index": 0, "approved": true, "confidence": 0.99, "reason": "ok"}]}, {"validations": [{"correction_index": 0, "approved": true, "confidence": 0.99, "reason": "ok"}]},
{"validations": [{"correction_index": 0, "approved": false, "confidence": 0.99, "reason": "reject cleanup"}]}, {"validations": [{"correction_index": 0, "approved": false, "confidence": 0.99, "reason": "reject cleanup"}]},
{"validations": [{"correction_index": 0, "approved": true, "confidence": 0.99, "reason": "phase17-secret"}]}, {"validations": [{"correction_index": 0, "approved": true, "confidence": 0.99, "reason": "parity-secret"}]},
{"validations": [{"correction_index": 0, "approved": true, "confidence": 0.99, "reason": "ok"}]} {"validations": [{"correction_index": 0, "approved": true, "confidence": 0.99, "reason": "ok"}]}
] ]

View File

@@ -11,7 +11,7 @@ import (
func TestProcessReportModuleResultsJSONSuccessAndSkipped(t *testing.T) { func TestProcessReportModuleResultsJSONSuccessAndSkipped(t *testing.T) {
now := time.Now().UTC() now := time.Now().UTC()
report := ProcessReport{ report := ProcessReport{
Phase: "phase15-spoken-word-module", Phase: "default_pipeline",
Status: "success", Status: "success",
ModuleResults: []ModuleReport{ ModuleResults: []ModuleReport{
{ {
@@ -80,7 +80,7 @@ func TestProcessReportModuleResultsJSONSuccessAndSkipped(t *testing.T) {
func TestProcessReportModuleResultsJSONFailedModule(t *testing.T) { func TestProcessReportModuleResultsJSONFailedModule(t *testing.T) {
now := time.Now().UTC() now := time.Now().UTC()
report := ProcessReport{ report := ProcessReport{
Phase: "phase15-spoken-word-module", Phase: "default_pipeline",
Status: "failed", Status: "failed",
ModuleResults: []ModuleReport{ ModuleResults: []ModuleReport{
{ {

View File

@@ -13,7 +13,7 @@ import (
// testChunkProposalHarness is a test-only helper that composes existing // testChunkProposalHarness is a test-only helper that composes existing
// deterministic chunking and proposal-application primitives. // deterministic chunking and proposal-application primitives.
// Production runner orchestration is implemented in later phases. // It isolates proposal/composition behavior from full runner orchestration.
type testChunkProposalHarness struct { type testChunkProposalHarness struct {
module TranscriptModule module TranscriptModule
} }

View File

@@ -22,8 +22,7 @@ const (
) )
const ( const (
ReasonUnsupportedModule = "unsupported_module" ReasonUnsupportedModule = "unsupported_module"
ReasonUnimplementedModule = "unimplemented_module"
) )
var knownModuleKeys = map[string]struct{}{ var knownModuleKeys = map[string]struct{}{
@@ -66,8 +65,7 @@ type Factory struct {
constructors map[string]Constructor constructors map[string]Constructor
} }
// NewFactory creates a production registry scaffold with known module keys but // NewFactory creates a production module registry.
// no real module constructors registered yet.
func NewFactory(deps Dependencies) *Factory { func NewFactory(deps Dependencies) *Factory {
factory := &Factory{ factory := &Factory{
deps: deps, deps: deps,
@@ -133,7 +131,7 @@ func (f *Factory) ModuleForSpec(spec contracts.ModuleRunSpec) (contracts.Transcr
constructor, ok := f.constructors[key] constructor, ok := f.constructors[key]
if !ok || constructor == nil { if !ok || constructor == nil {
return nil, &UnimplementedModuleError{ModuleKey: key} return nil, fmt.Errorf("internal module registry error: constructor for module %q is not configured", key)
} }
module, err := constructor(context.Background(), ConstructRequest{ module, err := constructor(context.Background(), ConstructRequest{
@@ -170,17 +168,3 @@ func (e *UnsupportedModuleError) Error() string {
func (e *UnsupportedModuleError) ReasonCode() string { func (e *UnsupportedModuleError) ReasonCode() string {
return ReasonUnsupportedModule return ReasonUnsupportedModule
} }
// UnimplementedModuleError indicates a known module key without constructor.
type UnimplementedModuleError struct {
ModuleKey string
}
func (e *UnimplementedModuleError) Error() string {
return fmt.Sprintf("module %q is recognized but not implemented", strings.TrimSpace(e.ModuleKey))
}
// ReasonCode returns a stable reason code suitable for reporting.
func (e *UnimplementedModuleError) ReasonCode() string {
return ReasonUnimplementedModule
}

View File

@@ -3,6 +3,7 @@ package modules
import ( import (
"context" "context"
"errors" "errors"
"strings"
"testing" "testing"
"gitea.maximumdirect.net/eric/audita/internal/framework/contracts" "gitea.maximumdirect.net/eric/audita/internal/framework/contracts"
@@ -64,22 +65,17 @@ func TestUnsupportedUnknownModuleKeyFailsCleanly(t *testing.T) {
} }
} }
func TestRecognizedButUnimplementedModuleKeyFailsCleanly(t *testing.T) { func TestRecognizedButMissingConstructorFailsWithInternalRegistryError(t *testing.T) {
factory := NewFactory(Dependencies{}) factory := NewFactory(Dependencies{})
// Force an unimplemented state for a known key to keep reason-code behavior tested. // Force a missing constructor state for a known key.
factory.constructors[ModuleKeySpokenWord] = nil factory.constructors[ModuleKeySpokenWord] = nil
_, err := factory.ModuleForSpec(contracts.ModuleRunSpec{ModuleKey: ModuleKeySpokenWord, InstanceName: ModuleKeySpokenWord}) _, err := factory.ModuleForSpec(contracts.ModuleRunSpec{ModuleKey: ModuleKeySpokenWord, InstanceName: ModuleKeySpokenWord})
if err == nil { if err == nil {
t.Fatal("expected unimplemented-module error") t.Fatal("expected constructor-missing error")
} }
if !strings.Contains(err.Error(), "internal module registry error") {
var unimplemented *UnimplementedModuleError t.Fatalf("expected internal registry error, got %v", err)
if !errors.As(err, &unimplemented) {
t.Fatalf("expected UnimplementedModuleError, got %T (%v)", err, err)
}
if unimplemented.ReasonCode() != ReasonUnimplementedModule {
t.Fatalf("unexpected reason code: %q", unimplemented.ReasonCode())
} }
} }

View File

@@ -185,7 +185,7 @@ func TestGenerateCandidatesMultipleSectionsStableMetadata(t *testing.T) {
} }
func TestGenerateCandidatesDiagnosticsWrittenAndRedacted(t *testing.T) { func TestGenerateCandidatesDiagnosticsWrittenAndRedacted(t *testing.T) {
secret := "phase11-secret" secret := "proposal-secret"
client := &fakeStructuredClient{ client := &fakeStructuredClient{
responses: []StructuredCorrectionSet{ responses: []StructuredCorrectionSet{
{Corrections: []StructuredCorrectionProposal{{TargetSegmentID: 1, OriginalText: secret, CorrectedText: "safe", Confidence: 0.9}}}, {Corrections: []StructuredCorrectionProposal{{TargetSegmentID: 1, OriginalText: secret, CorrectedText: "safe", Confidence: 0.9}}},

View File

@@ -30,7 +30,7 @@ type EnrichedCorrectionProposal struct {
// Validate checks only basic structural integrity of the proposal. // Validate checks only basic structural integrity of the proposal.
// Semantic checks (staleness, span presence, replacement behavior) are handled // Semantic checks (staleness, span presence, replacement behavior) are handled
// by later proposal preview/application and validator phases. // by proposal preview/application and validator execution.
func (p CorrectionProposal) Validate() error { func (p CorrectionProposal) Validate() error {
if p.TargetSegmentID <= 0 { if p.TargetSegmentID <= 0 {
return fmt.Errorf("proposal id must be positive") return fmt.Errorf("proposal id must be positive")

View File

@@ -41,7 +41,7 @@ func New() (*Module, error) {
func (m *Module) Key() string { return "glossary" } func (m *Module) Key() string { return "glossary" }
func (m *Module) ReplacementPolicy() proposals.ReplacementPolicy { func (m *Module) ReplacementPolicy() proposals.ReplacementPolicy {
// Python glossary module uses replace_all to update repeated term occurrences. // Update repeated term occurrences.
return proposals.ReplacementPolicyReplaceAll return proposals.ReplacementPolicyReplaceAll
} }

View File

@@ -41,7 +41,7 @@ func New() (*Module, error) {
func (m *Module) Key() string { return "grammar" } func (m *Module) Key() string { return "grammar" }
func (m *Module) ReplacementPolicy() proposals.ReplacementPolicy { func (m *Module) ReplacementPolicy() proposals.ReplacementPolicy {
// Python grammar module uses require_unique for conservative single-span replacement. // Conservative single-span replacement.
return proposals.ReplacementPolicyRequireUnique return proposals.ReplacementPolicyRequireUnique
} }

View File

@@ -22,8 +22,8 @@ type promptTranscriptSection struct {
Segments []promptSegment `json:"segments"` Segments []promptSegment `json:"segments"`
} }
// BuildProposalMessages mirrors the Python grammar-module prompt intent: // BuildProposalMessages constrains corrections to punctuation/capitalization/
// punctuation/capitalization/spacing cleanup only, with strict meaning guards. // spacing cleanup with strict meaning guards.
func BuildProposalMessages(transcript *schema.Transcript, glossary *schema.Glossary, sectionIndex int) ([]contracts.LLMMessage, error) { func BuildProposalMessages(transcript *schema.Transcript, glossary *schema.Glossary, sectionIndex int) ([]contracts.LLMMessage, error) {
glossaryJSON, err := json.MarshalIndent(glossary, "", " ") glossaryJSON, err := json.MarshalIndent(glossary, "", " ")
if err != nil { if err != nil {

View File

@@ -41,7 +41,7 @@ func New() (*Module, error) {
func (m *Module) Key() string { return "homophones" } func (m *Module) Key() string { return "homophones" }
func (m *Module) ReplacementPolicy() proposals.ReplacementPolicy { func (m *Module) ReplacementPolicy() proposals.ReplacementPolicy {
// Python homophones module uses require_unique for conservative single-span replacement. // Conservative single-span replacement.
return proposals.ReplacementPolicyRequireUnique return proposals.ReplacementPolicyRequireUnique
} }

View File

@@ -22,8 +22,8 @@ type promptTranscriptSection struct {
Segments []promptSegment `json:"segments"` Segments []promptSegment `json:"segments"`
} }
// BuildProposalMessages mirrors the Python homophones-module prompt intent: // BuildProposalMessages constrains corrections to conservative homophone and
// conservative homophone and mistranscription correction only. // mistranscription updates.
func BuildProposalMessages(transcript *schema.Transcript, glossary *schema.Glossary, sectionIndex int) ([]contracts.LLMMessage, error) { func BuildProposalMessages(transcript *schema.Transcript, glossary *schema.Glossary, sectionIndex int) ([]contracts.LLMMessage, error) {
glossaryJSON, err := json.MarshalIndent(glossary, "", " ") glossaryJSON, err := json.MarshalIndent(glossary, "", " ")
if err != nil { if err != nil {

View File

@@ -41,7 +41,7 @@ func New() (*Module, error) {
func (m *Module) Key() string { return "spoken_word" } func (m *Module) Key() string { return "spoken_word" }
func (m *Module) ReplacementPolicy() proposals.ReplacementPolicy { func (m *Module) ReplacementPolicy() proposals.ReplacementPolicy {
// Python spoken_word module uses require_unique for conservative single-span replacement. // Conservative single-span replacement.
return proposals.ReplacementPolicyRequireUnique return proposals.ReplacementPolicyRequireUnique
} }

View File

@@ -22,8 +22,8 @@ type promptTranscriptSection struct {
Segments []promptSegment `json:"segments"` Segments []promptSegment `json:"segments"`
} }
// BuildProposalMessages mirrors the Python spoken_word-module prompt intent: // BuildProposalMessages constrains corrections to conservative dysfluency
// conservative dysfluency cleanup with strict semantic preservation. // cleanup with strict semantic preservation.
func BuildProposalMessages(transcript *schema.Transcript, glossary *schema.Glossary, sectionIndex int) ([]contracts.LLMMessage, error) { func BuildProposalMessages(transcript *schema.Transcript, glossary *schema.Glossary, sectionIndex int) ([]contracts.LLMMessage, error) {
glossaryJSON, err := json.MarshalIndent(glossary, "", " ") glossaryJSON, err := json.MarshalIndent(glossary, "", " ")
if err != nil { if err != nil {