Complete Phase 19 Go rollout documentation

2026-05-12 13:46:14 +00:00
parent 68e2d9b549
commit fb59cb21b9
5 changed files with 345 additions and 65 deletions
--- a/README.md
+++ b/README.md
@@ -1,61 +1,235 @@
-# Audita (Go Rewrite)
+# Audita (Go)

-Audita is a transcript polishing CLI. The Go implementation at the repository root is canonical going forward, and is being delivered compatibility-first against the frozen Python reference in `python/`.
+Audita is a transcript polishing CLI.

-## Current Status: Phase 3 Deterministic Foundation
+The Go implementation in this repository is the active implementation. It runs a full default correction pipeline over transcript JSON using glossary context, LLM-backed proposal generation, validator chains, deterministic proposal application, and structured reports/diagnostics.

-The current `audita process` implementation includes deterministic input handling and foundation behavior:
+## What Audita does

- typed transcript parsing for both accepted top-level forms (`[]` and `{ "segments": [...] }`);
- typed glossary YAML parsing;
- transcript and glossary schema validation with actionable errors;
- deterministic same-speaker normalization;
- sequential normalized segment IDs starting at `1`;
- deterministic token estimation;
- contiguous transcript chunking and section summaries;
- normalized transcript JSON output;
- minimal structured `--report-json` output including normalization and chunking summary fields;
- minimal normalization diagnostics artifacts in per-run work directories;
- chunking diagnostics artifacts in per-run work directories;
- subprocess-safe stdout/stderr behavior.
+`audita process` performs:
+- transcript/glossary schema validation;
+- deterministic normalization and chunking;
+- default module sequence:
+  - `glossary`
+  - `homophones`
+  - `glossary`
+  - `spoken_word`
+  - `grammar`
+- glossary-backed domain/acoustic corrections;
+- conservative homophone and likely mistranscription corrections;
+- conservative spoken-word cleanup (dysfluencies/fillers) with semantic guardrails;
+- grammar/punctuation/capitalization/formatting cleanup;
+- machine-readable process and module reports;
+- per-run diagnostics artifacts with secret redaction.

-The repository also now contains deterministic proposal foundation packages (models, preview, application, skip reporting, and framework contracts) with fixture/golden tests.
+## Build and install

-Still not implemented in the Go CLI runtime path:
-
- real module execution pipeline;
- real `glossary`, `homophones`, `spoken_word`, or `grammar` module behavior;
- LLM proposal generation;
- validator-chain execution;
- concrete structured LLM client implementation;
- final end-to-end transcript polishing behavior.
-
-## Quick Start (Go)
-
-Run tests:
+Build a local binary:

 ```sh
-go test ./...
+go build -o ./bin/audita ./cmd/audita
+```
+
+Install into your Go bin directory:
+
+```sh
+go install ./cmd/audita
 ```

 Run help:

 ```sh
-go run ./cmd/audita --help
-go run ./cmd/audita process --help
+audita --help
+audita process --help
 ```

-Run current deterministic flow:
+## Test
+
+Run the full test suite:

 ```sh
-go run ./cmd/audita process transcript.json --glossary glossary.yaml --output corrected.json
+go test ./...
 ```

-Without `--output`, transcript JSON is written to stdout.
+Normal tests are deterministic and do not require real LLM credentials or Python dependencies.

-## Repository Notes
+## Basic usage

- `python/` contains the frozen Python implementation used as behavioral reference during the Go port.
- Architecture and rewrite guidance:
-  - `docs/architecture.md`
-  - `docs/rewrite-notes.md`
+Required inputs:
+- transcript JSON path (positional argument)
+- `--glossary <glossary.yaml>`
+
+Default full pipeline (recommended local example):
+
+```sh
+audita process transcript.json \
+  --glossary glossary.yaml \
+  --output corrected.json \
+  --report-json report.json
+```
+
+Explicit module override:
+
+```sh
+audita process transcript.json \
+  --glossary glossary.yaml \
+  --modules glossary,homophones,grammar \
+  --output corrected.json \
+  --report-json report.json
+```
+
+Emit transcript JSON to stdout (no `--output`):
+
+```sh
+audita process transcript.json --glossary glossary.yaml
+```
+
+Diagnostics/work-dir control:
+
+```sh
+audita process transcript.json \
+  --glossary glossary.yaml \
+  --work-dir /tmp/audita \
+  --work-dir-retention auto \
+  --output corrected.json \
+  --report-json report.json
+```
+
+## Stdout/stderr and orchestration behavior
+
+- With `--output`, stdout should be empty on success.
+- Without `--output`, stdout contains transcript JSON only on success.
+- `--report-json` writes report JSON to file; report JSON is never printed to stdout.
+- stderr is for human-readable warnings/errors.
+
+For parent-process integration guidance, see:
+- [`docs/subprocess-operations.md`](docs/subprocess-operations.md)
+
+For orchestrated runs, use both `--output` and `--report-json`.
+
+## Configuration
+
+Precedence:
+1. defaults
+2. environment (`AUDITA_*`)
+3. CLI flags
+
+### Module sequence
+
+- `AUDITA_MODULES` (CSV, e.g. `glossary,homophones,glossary,spoken_word,grammar`)
+- CLI override: `--modules`
+
+### Primary LLM settings
+
+- `AUDITA_LLM_API_KEY` (or `OPENROUTER_API_KEY` fallback)
+- `AUDITA_MODEL`
+- `AUDITA_BASE_URL`
+- `AUDITA_LLM_TIMEOUT_SECONDS`
+- `AUDITA_MAX_RETRIES`
+- `AUDITA_LLM_CONCURRENCY`
+
+CLI overrides:
+- `--llm-api-key`
+- `--model`
+- `--base-url`
+- `--llm-timeout-seconds`
+- `--max-retries`
+
+### Validation LLM settings
+
+- `AUDITA_VALIDATION_LLM_API_KEY`
+- `AUDITA_VALIDATION_MODEL`
+- `AUDITA_VALIDATION_BASE_URL`
+- `AUDITA_VALIDATION_LLM_TIMEOUT_SECONDS`
+- `AUDITA_VALIDATION_MAX_RETRIES`
+- `AUDITA_VALIDATION_LLM_CONCURRENCY`
+- `AUDITA_VALIDATION_MAX_PROMPT_TOKENS`
+
+CLI overrides:
+- `--validation-llm-api-key`
+- `--validation-model`
+- `--validation-base-url`
+- `--validation-llm-timeout-seconds`
+- `--validation-max-retries`
+- `--validation-llm-concurrency`
+- `--validation-max-prompt-tokens`
+
+Validation LLM inheritance behavior:
+- unset validation fields inherit from primary LLM config;
+- set validation fields override primary values for validation calls only.
+
+### Confidence thresholds
+
+- `AUDITA_GLOSSARY_CONFIDENCE_THRESHOLD`
+- `AUDITA_HOMOPHONES_CONFIDENCE_THRESHOLD`
+- `AUDITA_SPOKEN_WORD_CONFIDENCE_THRESHOLD`
+- `AUDITA_GRAMMAR_CONFIDENCE_THRESHOLD`
+
+CLI overrides:
+- `--glossary-confidence-threshold`
+- `--homophones-confidence-threshold`
+- `--spoken-word-confidence-threshold`
+- `--grammar-confidence-threshold`
+
+### Normalization and chunking
+
+- `AUDITA_NORMALIZE_MAX_SEGMENT_GAP`
+- `AUDITA_NORMALIZE_ELLIPSIS_GAP`
+- `AUDITA_NORMALIZE_MAX_SEGMENT_DURATION`
+- `AUDITA_NORMALIZE_MAX_SEGMENT_TOKENS`
+- `AUDITA_MAX_SECTION_TOKENS`
+- `AUDITA_MIN_SECTION_TOKENS`
+- `AUDITA_TARGET_SECTIONS`
+
+CLI overrides:
+- `--normalize-max-segment-gap`
+- `--normalize-ellipsis-gap`
+- `--normalize-max-segment-duration`
+- `--normalize-max-segment-tokens`
+- `--max-section-tokens`
+- `--min-section-tokens`
+- `--target-sections`
+
+### Work-dir and retention
+
+- `AUDITA_WORK_DIR`
+- `AUDITA_WORK_DIR_RETENTION` (`auto`, `always`, `never`)
+
+CLI overrides:
+- `--work-dir`
+- `--work-dir-retention`
+
+Retention summary:
+- `always`: keep all run directories.
+- `never`: keep successful run directories.
+- `auto`: keep failed runs and successful runs with skipped/rejected corrections.
+
+## Report and diagnostics
+
+Per-run diagnostics include:
+- source transcript artifacts;
+- normalized transcript artifact;
+- normalization summary;
+- chunking summary;
+- invocation metadata;
+- redacted effective config;
+- prompt/response diagnostics for module and validator LLM interactions;
+- `report.json`;
+- `error.log` on failure.
+
+Optional external report output:
+- `--report-json <path>`
+
+## Legacy Python reference
+
+The original Python implementation is preserved in [`python/`](python/) as a legacy/reference implementation for parity history and migration context.
+For migration guidance, see [`docs/migration-from-python.md`](docs/migration-from-python.md).
+
+Parity fixture notes and intentional differences:
+- [`docs/python-parity.md`](docs/python-parity.md)
+
+## Additional docs
+
+- Architecture: [`docs/architecture.md`](docs/architecture.md)
+- Rewrite history and phase notes: [`docs/rewrite-notes.md`](docs/rewrite-notes.md)
+- Migration from Python to Go: [`docs/migration-from-python.md`](docs/migration-from-python.md)