eric/audita
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Improved organization of the documentation

Browse Source
This commit is contained in:
Eric Rakestraw
2026-05-14 09:24:02 -05:00
parent 2b2a3fc024
commit b103bb2e7d
8 changed files with 0 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

849
docs/architecture.md
View File

@@ -1,849 +0,0 @@
# Audita Architecture
## Scope and intent
This document describes:
- the architecture used in production today.
Historical rewrite details live in `docs/rewrite-notes.md`.
## Current implementation status
Implemented today:
- Go CLI entrypoint and `audita process` wiring.
- Config defaults, env loading, CLI override precedence, and validation.
- Transcript and glossary parsing/validation.
- Deterministic transcript normalization.
- Deterministic token estimation and transcript chunking.
- Per-run diagnostics directory creation plus process-level artifacts.
- Process report JSON output with diagnostics artifact references.
- Framework foundation packages for contracts and proposal application.
- Production runner orchestration package with deterministic sequential module execution.
- Module-level report structures with applied/skipped change records.
- Runtime validator models and deterministic validators.
- Deterministic validator-chain execution in the runner with cardinality enforcement.
- Module-level validator decision/rejection reporting.
- Internal structured LLM client contract plus an Audita-owned OpenAI-compatible structured LLM adapter package.
- Bounded FIFO LLM scheduler infrastructure with context-aware permit handling.
- Runtime primary/validation LLM effective-config resolution helpers with validation inheritance.
- Generic JSON prompt/response diagnostics writer primitives with secret redaction.
- LLM-backed validator models, prompt builders, batching, and runtime execution.
- Runner wiring for LLM validators via the internal structured LLM abstraction and scheduler hooks.
- LLM validator diagnostics artifacts and report-level decision metadata paths.
- Shared LLM proposal-generation helper with structured correction-set parsing.
- Deterministic proposal-index assignment and enriched proposal mapping for shared generation.
- Proposal-generation diagnostics artifacts with secret redaction.
- Production module registry with known-key recognition and explicit unsupported-module errors.
- Production `grammar` module implementation in `internal/modules/grammar`.
- Production `glossary` module implementation in `internal/modules/glossary`.
- Production `homophones` module implementation in `internal/modules/homophones`.
- Production `spoken_word` module implementation in `internal/modules/spoken_word`.
- Explicit runtime support for `--modules grammar` through the production runner path.
- Explicit runtime support for `--modules glossary`, including repeated stages such as `--modules glossary,glossary`.
- Explicit runtime support for `--modules homophones` through the production runner path.
- Explicit runtime support for `--modules spoken_word` through the production runner path.
Current reality:
- all production modules exist and are wired into the default runtime path.
- a normal `audita process` run without `--modules` now executes the full sequence:
- `glossary`
- `homophones`
- `glossary`
- `spoken_word`
- `grammar`
- repeated glossary stages are deterministic and reported distinctly as `glossary_1` and `glossary_2`.
## Actual Go package layout
```text
cmd/audita/
main.go
internal/cli/
run.go
internal/core/config/
config.go
env.go
flags.go
redaction.go
validation.go
internal/core/schema/
transcript.go
glossary.go
errors.go
internal/core/io/
files.go
internal/core/normalization/
normalize.go
tokens.go
internal/core/chunking/
sections.go
summary.go
tokens.go
internal/core/diagnostics/
run_dir.go
internal/core/reporting/
report.go
internal/framework/contracts/
contracts.go
internal/framework/proposals/
proposal.go
policy.go
preview.go
apply.go
internal/framework/runner/
observability.go
runner.go
internal/framework/proposal_generation/
generate.go
internal/framework/modules/
registry.go
internal/modules/grammar/
module.go
prompt.go
internal/modules/glossary/
module.go
prompt.go
internal/modules/homophones/
module.go
prompt.go
internal/modules/spoken_word/
module.go
prompt.go
internal/framework/validators/
models.go
deterministic.go
llm_models.go
llm_prompt_builders.go
llm_batching.go
llm_validators.go
internal/validators/
metadata/
metadata.go
registry.go
chains.go
confidence_threshold/
validator.go
original_text_presence/
validator.go
non_empty_corrected_text/
validator.go
no_effect/
validator.go
protected_terms/
validator.go
spoken_form_plausibility/
validator.go
meaning_reversal_review/
validator.go
editorial_review/
validator.go
grammar_review/
validator.go
spoken_word_review/
validator.go
internal/prompts/
registry.go
render.go
assets/
shared/
modules/
validators/
internal/framework/llm/
openai_compatible_client.go
scheduler.go
effective_config.go
diagnostics.go
internal/framework/responseschema/
registry.go
registry_test.go
internal/cli/
review_artifacts.go
parity_test.go
release_fixtures_test.go
testdata/
parity/
release/
```
## Current CLI behavior
Primary commands:
```sh
audita process <transcript.json> --glossary <glossary.yaml> [flags]
audita config validate --config <config.yml>
audita config print-effective [--config <config.yml>]
```
Current runtime flow (`internal/cli/run.go`):
1. Build runtime config from:
- defaults;
- file config source (`--config`, `AUDITA_CONFIG`, or `/etc/audita/config.yml` when present);
- environment overrides;
- CLI overrides.
2. Parse flags and apply CLI overrides.
3. Validate transcript positional argument and required `--glossary`.
4. Create per-run diagnostics directory.
5. Read transcript and glossary files.
6. Parse/validate transcript and glossary.
7. Write source transcript artifacts.
8. Normalize transcript.
9. Write normalized transcript and normalization summary artifacts.
10. Chunk normalized transcript and compute chunk summaries.
11. Write chunking summary artifact.
12. Execute runner modules sequentially:
- default run path uses configured default sequence (`glossary,homophones,glossary,spoken_word,grammar`);
- explicit `--modules` overrides the default sequence;
- test/injected module factory path remains available for deterministic runtime tests.
- each module recomputes chunks from the current working transcript, runs chunk proposal work concurrently, aggregates deterministically, validates, and applies approved proposals once.
13. Output working transcript to `--output` file or stdout.
14. Build process report metadata.
15. Optionally write `--report-json`; always write run-dir `report.json`.
16. Apply work-dir retention.
Config command behavior (`internal/cli/run.go`):
- `audita config validate --config <path>`:
- loads and validates a versioned YAML config file;
- does not require transcript or glossary inputs.
- `audita config print-effective [--config <path>]`:
- builds effective config from defaults + file config + env overrides;
- prints redacted JSON to stdout;
- does not require transcript or glossary inputs.
Parity fixture status:
- representative Python-parity fixture coverage exists under `internal/cli/testdata/parity`;
- parity tests use fake structured LLM responses for deterministic behavior, including default full-pipeline shape assertions;
- parity comparisons intentionally ignore nondeterministic metadata (timestamps, run IDs, temp paths, token usage) and remain strict for deterministic contract fields (transcript content, module order/instance naming, applied/skipped/rejected counts, and status).
- intentional Python-vs-Go differences and open parity gaps are documented in `docs/python-parity.md`.
Important behavior details:
- Glossary is validated and is used for explicit glossary/grammar/homophones/spoken_word module correction paths.
- Default production CLI behavior now executes the full production module sequence unless `--modules` override is supplied.
- Explicit `--modules grammar`, `--modules glossary`, `--modules homophones`, and `--modules spoken_word` continue to run production module paths with LLM-backed proposal generation and validator-chain execution.
- Default runs (without explicit module selection) perform LLM calls through production module and validator paths.
- Success path is generally quiet on stderr.
- Source IDs are preserved into a canonical transcript before normalization; normalization then reassigns output IDs sequentially from `1`.
## Implemented data contracts
### Transcript input
Accepted top-level forms:
- bare JSON array of segments
- object with `segments` array
Source segment contract:
- `id` optional integer
- `speaker` non-empty string
- `start` finite non-negative number
- `end` finite non-negative number with `end >= start`
- `text` non-empty string
- `categories` optional array of non-empty strings
Additional checks:
- duplicate explicit source IDs are rejected.
### Transcript output
Transcript output is selected through an output schema registry (`internal/core/outputschema`).
Supported output schemas:
- `bare-segments` (default):
- top-level JSON array of normalized segments;
- each segment includes `id`, `speaker`, `start`, `end`, `text`, optional `categories`.
- `audita-v1`:
- top-level object with:
- `schema: "audita-v1"`
- `version: "v1"`
- `segments: [...]` (same normalized segment payload).
Current status:
- `seriatim-intermediate` is not implemented yet; selecting it fails clearly as an unsupported output schema.
Selection behavior:
- CLI: `--output-schema <name>`
- file config: `output.schema: <name>`
- precedence remains runtime-wide defaults -> file config -> env -> CLI.
Both stdout transcript output and `--output` file output use the same selected output encoder.
### Glossary input
YAML with `glossary` entries. Required fields per entry:
- `name`, `category`, `summary`
Optional:
- `aliases`, `plural`
## Implemented config/env/flag behavior
Precedence for `audita process`:
1. defaults (`config.Default()`)
2. config file (if resolved from `--config`, `AUDITA_CONFIG`, or default path)
3. environment overrides
4. CLI flags (`ApplyCLIOverrides`)
File-config source behavior:
- explicit `--config <path>`:
- required to exist, otherwise process fails clearly.
- `AUDITA_CONFIG` (when `--config` is not provided):
- required to exist, otherwise process fails clearly.
- default path `/etc/audita/config.yml` (when neither explicit source is provided):
- used only when present;
- silently ignored when missing.
Versioned file-config behavior (`internal/core/config/file_config.go`):
- supported version: `version: 1`;
- missing version fails;
- unsupported version fails;
- strict unknown-field rejection is enabled.
`api_key_env` behavior:
- file config can declare API key environment variable names for proposal/validation LLM settings;
- runtime resolves those names from the process environment during config application;
- no direct API-key value field is supported in file config.
Redaction behavior:
- effective config artifacts and `audita config print-effective` both use the same redaction path (`Config.Redacted()`), so API keys are not emitted in plaintext.
Implemented config surfaces include:
- module list
- primary and validation LLM settings
- total/proposal/validation LLM concurrency controls
- transcript description context (`--transcript-description`)
- section token controls and target sections
- confidence thresholds
- normalization controls
- work-dir and retention mode
Current caveat:
- LLM/module-related settings are active for default and explicit module-run paths.
- compatibility environment variables and lower-level CLI tuning flags remain available while the preferred config-driven surface is adopted.
Transcript description behavior:
- `--transcript-description` is a process-flag input for optional user-supplied background context.
- runtime config stores this value in `Config.TranscriptDescription` after CLI trimming and length validation.
- default value is empty; empty values produce no prompt context section.
- this value is intentionally non-secret and appears in effective config and invocation metadata artifacts.
## Implemented transcript description prompt context
Transcript description context is wired through production prompt paths:
- proposal prompts for `glossary`, `homophones`, `spoken_word`, and `grammar`;
- LLM-backed validator prompts for spoken-form plausibility, meaning reversal, editorial review, grammar review, and spoken-word review.
Prompt guardrail semantics are consistent across modules and validators:
- transcript description is labeled as "background context only";
- it may help interpret ambiguous terms;
- it must not override transcript content;
- the model must not invent corrections, facts, names, events, motivations, or speaker intent from this description.
Generated transcript descriptions remain deferred and are not implemented in the current runtime.
## Implemented embedded prompt assets
Prompt assets are now built-in embedded Markdown files under `internal/prompts/assets`:
- `assets/modules/*` for production module proposal prompts;
- `assets/validators/*` for LLM-backed validator prompts;
- `assets/shared/prompt_hardening.md` for shared prompt-injection hardening text.
Prompt source behavior:
- built-in embedded prompts are the only supported source in current runtime;
- filesystem prompt overrides and prompt-source selection flags are not implemented.
`internal/prompts` registry responsibilities:
- register stable prompt IDs;
- register prompt version and source metadata;
- load embedded assets;
- compute deterministic SHA-256 prompt source hashes;
- render system/user prompts with `text/template` using missing-key errors.
Prompt metadata fields:
- `prompt_id`
- `prompt_version`
- `prompt_source` (`builtin`)
- `embedded_path`
- `sha256`
Prompt rendering flow:
- module proposal builders construct typed template data (section JSON, glossary JSON, transcript-description block) and render via `internal/prompts`;
- validator prompt builders construct typed template data (validation payload JSON, transcript-description block) and render via `internal/prompts`.
Shared prompt hardening:
- the same centralized hardening fragment is included in every proposal and LLM-validator prompt;
- hardening text enforces untrusted transcript handling, no instruction-following from transcript content, and no invented facts/corrections.
Prompt metadata diagnostics flow:
- proposal-generation diagnostics request metadata includes prompt metadata;
- LLM-validator diagnostics request metadata includes prompt metadata;
- detailed prompt metadata is diagnostics-scoped today and is not yet expanded into broad report-level prompt registries.
## Implemented structured LLM infrastructure
`internal/framework/contracts` now defines a typed structured-completion contract:
- `StructuredLLMClient.CompleteStructured(ctx, req, out)`
- caller-owned typed decode target via `out` pointer.
- caller-selected structured response schema metadata via `StructuredCompletionRequest.ResponseSchema`.
`internal/framework/llm` provides `OpenAICompatibleClient`, a direct `net/http` adapter over OpenAI-compatible chat completions:
- configurable `base_url`, model, optional API key, retries, HTTP client, and request timeout;
- OpenAI-compatible endpoint behavior (for example OpenAI/OpenRouter/local-compatible base URLs);
- request message translation from `contracts.LLMMessage` to chat-completions messages;
- strict `response_format.type = json_schema` with registered structured response schemas (`strict: true`, schema name, and schema body);
- response metadata mapping (provider/model/token usage) into Audita-owned response types;
- API-key redaction in adapter-returned errors;
- context cancellation and timeout propagation through request contexts and HTTP client timeouts;
- bounded retry behavior for transient request failures and malformed retryable structured responses.
Structured response schemas are owned by Audita in `internal/framework/responseschema` and currently include:
- key `correction_set`:
- id `audita.correction_set`
- version `v1`
- name `audita_correction_set_v1`
- sha256 `05f8ff3fa04f68115c0cb1859d2656f51aa5c0bae8ff2470b2d4f6f531953195`
- key `validator_decision_set`:
- id `audita.validator_decision_set`
- version `v1`
- name `audita_validator_decision_set_v1`
- sha256 `b73f4790b98fbb955f0aec5496dd8ce9a8fe14aa2f35c700b4b4e5634f106fd5`
Provider-level structured output is treated as a guardrail, not a trust boundary:
- the adapter decodes assistant message content into caller-owned structs;
- proposal-generation and validator layers continue local validation (shape, cardinality, confidence bounds, and proposal-index semantics) before changes can be applied.
Current runtime boundary:
- the default CLI runtime path (without explicit module selection) instantiates the full production module sequence.
- LLM calls are exercised in production in both default full-pipeline runs and explicit `--modules` runs, and in tests when fake/injected clients are used.
- normal `go test ./...` does not require real LLM credentials or Python dependencies.
`internal/framework/llm` also provides:
- a bounded FIFO `Scheduler` for controlled concurrent LLM calls with reliable permit release on success, error, and cancellation;
- primary/validation effective-config resolution helpers, including validation inheritance fallback to total LLM concurrency settings;
- generic interaction diagnostics primitives that write machine-readable JSON artifacts for request metadata, request payload, response payload, and optional error payload with secret redaction.
Structured LLM diagnostics behavior:
- proposal-generation and validator diagnostics include structured response schema metadata (`id`, `version`, `name`, `sha256`) when schema-driven calls are made;
- API keys and bearer tokens are redacted from request/response/error diagnostics artifacts and surfaced errors.
Dependency posture:
- the runtime no longer depends on `instructor-go`;
- structured LLM behavior is implemented through Audita-owned code paths behind `StructuredLLMClient`.
LLM concurrency runtime behavior:
- `total` concurrency bounds all proposal and validation LLM calls.
- `proposal` concurrency adds a proposal-only sub-cap, composed with total.
- `validation` concurrency adds a validation-only sub-cap, composed with total.
- legacy `llm-concurrency` inputs remain compatibility aliases for total concurrency.
- modules execute serially, chunk proposals run concurrently within each module, and approved proposals are applied once per module in deterministic order.
## Implemented normalization behavior
Normalization (`internal/core/normalization`) currently:
- sorts by segment start time;
- merges adjacent same-speaker segments when constraints pass;
- uses gap-based joiners:
- gap `< ellipsis_gap` -> single space join
- gap `>= ellipsis_gap` -> `... ` join
- enforces merged duration and token-limit constraints;
- reassigns output IDs sequentially from `1`;
- returns `NormalizationSummary` with merge and skip counters.
Note: merged categories are concatenated (not deduplicated).
## Implemented chunking behavior
Chunking (`internal/core/chunking`) currently provides:
- deterministic heuristic token estimation;
- contiguous sectioning with section metadata;
- max/min section token validation;
- optional `target_sections` override for section-count planning;
- summary and detailed summary generation.
Current behavior details:
- if a single segment exceeds max tokens, it is emitted as its own section (not hard-failed);
- default section count is planned from `ceil(total_tokens / max_section_tokens)`;
- section sizing targets `ceil(total_tokens / section_count)` with a deterministic forward pass;
- sections remain contiguous and ordered, and segments are never split.
## Implemented proposal/replacement infrastructure
`internal/framework/proposals` provides deterministic proposal composition logic:
- `CorrectionProposal` and `EnrichedCorrectionProposal` models;
- replacement policies: `require_unique`, `replace_all`;
- safe preview (`PreviewProposalForSegment`) with stable skip reasons;
- deterministic apply (`ApplyProposals`) in ascending `proposal_index` order;
- applied/skipped change records suitable for reporting.
`internal/framework/contracts` provides interfaces and run-spec metadata scaffolding, including deterministic repeated module instance naming (`ResolveModuleRunSpecs`).
These primitives are wired into the production runner and report model. The grammar, glossary, homophones, and spoken_word modules are implemented.
## Implemented validator runtime infrastructure
`internal/framework/validators` provides deterministic validator infrastructure:
- runtime validation request/result models;
- stable validator reason codes;
- cardinality enforcement for validator decisions:
- missing proposal indexes fail
- duplicate proposal indexes fail
- unknown proposal indexes fail
- deterministic validators:
- confidence threshold by module key/config threshold
- original-text presence against current working transcript
- non-empty corrected text
- identical/no-effect rejection
- conservative protected glossary-term guard for non-glossary modules
`internal/framework/runner` executes module pipelines with deterministic boundaries:
- modules still execute serially over the working transcript;
- section proposal work is launched promptly and can run concurrently;
- section-level validator-chain work starts as section proposals become available (deterministic validators before LLM-backed validators);
- proposal-generation and LLM-validator calls can overlap under composed scheduler limits;
- approved proposals are still applied once per module after section work settles.
Validator rejections are reported distinctly from proposal-application skips.
Validator composition is now explicit and registry-backed through `internal/validators`:
- built-in validator registry with stable keys and lookup/build failure for unknown keys;
- built-in chain definitions per production module key;
- production modules resolve validator chains from those built-in definitions.
Package ownership boundary:
- `internal/validators/<validator_key>` owns built-in validator construction and stable key identity.
- `internal/framework/validators` remains shared runtime machinery:
- request/result models;
- decision cardinality enforcement;
- protected-vocabulary helpers;
- generic LLM-backed validator runtime, batching, and diagnostics glue.
Validator execution classification metadata:
- `internal/validators/metadata` defines execution class markers:
- `deterministic`
- `llm_backed`
- runner ordering uses this metadata interface rather than concrete framework validator type assertions.
- validators without classification metadata default to deterministic ordering.
`protected_terms` construction ownership:
- `internal/validators/protected_terms.New()` builds the general (non-glossary-stage) variant.
- `internal/validators/protected_terms.NewGlossaryStage()` builds the glossary-stage variant used by glossary chains.
- both variants preserve the stable key `protected_terms`.
Stable built-in validator keys:
- deterministic:
- `confidence_threshold`
- `original_text_presence`
- `non_empty_corrected_text`
- `no_effect`
- `protected_terms`
- LLM-backed:
- `spoken_form_plausibility`
- `meaning_reversal_review`
- `editorial_review`
- `grammar_review`
- `spoken_word_review`
Built-in module chains:
- `glossary`:
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
- `protected_terms`
- `non_empty_corrected_text`
- `spoken_form_plausibility`
- `meaning_reversal_review`
- `homophones`:
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
- `protected_terms`
- `non_empty_corrected_text`
- `spoken_form_plausibility`
- `meaning_reversal_review`
- `spoken_word`:
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
- `protected_terms`
- `non_empty_corrected_text`
- `spoken_word_review`
- `meaning_reversal_review`
- `grammar`:
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
- `protected_terms`
- `non_empty_corrected_text`
- `grammar_review`
- `meaning_reversal_review`
1.0 boundary:
- validator chains are built-in and not user-configurable from config/CLI.
- existing threshold and batching knobs remain configurable.
## Implemented LLM-backed validator infrastructure
`internal/framework/validators` now includes LLM-backed validator support:
- typed request/response models for structured LLM validation;
- prompt builders for:
- spoken-form plausibility
- meaning reversal detection
- editorial review
- grammar review
- spoken-word review
- deterministic batching by `validation_max_prompt_tokens`;
- strict cardinality validation of structured LLM decisions (missing/duplicate/unknown indexes fail);
- safe failure behavior for malformed/invalid structured responses.
`internal/framework/runner` wires LLM validators into existing validator chains using:
- the internal structured LLM client abstraction (`contracts.StructuredLLMClient`);
- bounded scheduler hooks for validator call execution;
- diagnostics writer hooks for machine-readable prompt/response artifacts with secret redaction.
## Implemented shared proposal-generation infrastructure
`internal/framework/proposal_generation` provides a reusable, prompt-agnostic helper for future real modules:
- structured request model including module key/instance, replacement policy, working transcript context, optional section metadata, glossary, config, and diagnostics context;
- structured correction-set response model (`corrections`) mapped into existing `proposals.CorrectionProposal` and `proposals.EnrichedCorrectionProposal` models;
- deterministic proposal-index assignment through a caller-provided `start_index`;
- structured LLM calls through `contracts.StructuredLLMClient` only (no direct provider calls);
- optional bounded execution through scheduler hooks (`contracts.LLMScheduler`);
- prompt/response diagnostics artifact writing via the generic `internal/framework/llm` diagnostics primitives with redaction of API keys/secrets.
This helper only produces candidate proposals; validator-chain execution and proposal application remain runner responsibilities.
## Implemented production module-registry scaffolding
`internal/framework/modules` now provides a production registry scaffold:
- recognizes intended module keys:
- `glossary`
- `homophones`
- `spoken_word`
- `grammar`
- supports explicit constructor registration with dependency injection for:
- run spec
- config
- glossary
- proposal/validation structured LLM clients
- proposal/validation schedulers
- diagnostics directory context
- returns explicit errors for unknown keys (`unsupported_module`).
The `grammar`, `glossary`, `homophones`, and `spoken_word` module keys are now registered and constructible.
## Implemented grammar production module
`internal/modules/grammar` now provides the first production module:
- prompt builder faithfully constrained to punctuation/capitalization/spacing/article cleanup;
- explicit guardrails against meaning-changing rewrites, style rewrites, summarization, and invention;
- proposal generation through `internal/framework/proposal_generation` and `contracts.StructuredLLMClient`;
- scheduler-aware proposal calls through existing `contracts.LLMScheduler` hooks;
- replacement policy `require_unique` (current runtime policy);
- validator chain integration using existing deterministic + LLM-backed validators;
- grammar confidence threshold enforcement through existing validator/config infrastructure;
- module-level reporting and diagnostics capture through existing runner/reporting paths.
## Implemented glossary production module
`internal/modules/glossary` now provides the second production module:
- prompt builder aligned to Python glossary-module intent, constrained to glossary-backed domain/acoustic corrections;
- prompt context includes glossary names, aliases, categories, summaries, and plural forms where available;
- guardrails against broad style rewriting and against replacing unrelated terms simply because they appear in glossary entries;
- proposal generation through `internal/framework/proposal_generation` and `contracts.StructuredLLMClient`;
- scheduler-aware proposal calls through existing `contracts.LLMScheduler` hooks;
- replacement policy `replace_all` (matching Python glossary behavior);
- validator chain integration using existing deterministic + LLM-backed validators;
- glossary confidence threshold enforcement through existing validator/config infrastructure;
- module-level reporting and diagnostics capture through existing runner/reporting paths;
- explicit support for repeated glossary stages with deterministic instance names (`glossary_1`, `glossary_2`, ...), where later stages see prior-stage working transcript changes.
## Implemented protected-term behavior
`internal/framework/validators/protected_terms.go` provides deterministic glossary-derived protected vocabulary:
- extracts protected terms from glossary names and aliases;
- includes explicit plural fields and synthetic plural forms where safe;
- deduplicates and returns stable ordering for repeatable behavior/tests.
This vocabulary is used by deterministic validators for both glossary-stage and non-glossary-stage protection checks, keeping protected-term guardrails active across modules.
## Implemented homophones production module
`internal/modules/homophones` now provides the third production module:
- prompt builder aligned to Python homophones-module intent, constrained to conservative homophone/near-homophone/mistranscription corrections;
- prompt context includes protected glossary names/aliases/plurals to avoid damaging known terms;
- explicit guardrails against punctuation cleanup, grammar cleanup, style rewriting, summarization, and content invention;
- proposal generation through `internal/framework/proposal_generation` and `contracts.StructuredLLMClient`;
- scheduler-aware proposal calls through existing `contracts.LLMScheduler` hooks;
- replacement policy `require_unique` (matching Python homophones behavior);
- validator chain integration using existing deterministic + LLM-backed validators;
- homophones confidence threshold enforcement through existing validator/config infrastructure;
- protected-term guardrails for non-glossary modules remain active and are exercised through the homophones path;
- module-level reporting and diagnostics capture through existing runner/reporting paths.
## Implemented spoken_word production module
`internal/modules/spoken_word` now provides the fourth production module:
- prompt builder aligned to Python spoken_word-module intent, constrained to conservative dysfluency cleanup;
- strong prompt guardrails preserving meaning/intent/voice/named entities/domain terms and substantive content;
- explicit guardrails against summarization, style rewriting, grammar-only cleanup, punctuation-only cleanup, invention, and meaning-changing rewrites;
- proposal generation through `internal/framework/proposal_generation` and `contracts.StructuredLLMClient`;
- scheduler-aware proposal calls through existing `contracts.LLMScheduler` hooks;
- replacement policy `require_unique` (matching Python spoken_word behavior);
- validator chain integration using existing deterministic + LLM-backed validators, including strong semantic guardrails (`spoken_word_review`, `meaning_reversal_review`);
- spoken_word confidence threshold enforcement through existing validator/config infrastructure;
- protected-term guardrails for non-glossary modules remain active and are exercised through the spoken_word path;
- module-level reporting and diagnostics capture through existing runner/reporting paths.
## Reports and diagnostics (implemented)
Current per-run artifacts include:
- `source-transcript.json`
- `source-transcript-parsed.json`
- `normalized-transcript.json`
- `normalization-summary.json`
- `chunking-summary.json`
- `utilization-diagnostics.json`
- `correction-ledger.json`
- `invocation.json`
- `effective-config.json` (redacted credentials)
- `report.json`
- `error.log` on failure
`--report-json` writes a separate report file when requested.
Current process reports include diagnostics metadata references for:
- diagnostics directory path;
- source transcript artifact path;
- parsed source transcript artifact path;
- normalized transcript artifact path;
- normalization summary artifact path;
- chunking summary artifact path;
- utilization diagnostics artifact path;
- correction ledger artifact path;
- invocation metadata artifact path;
- redacted effective-config artifact path;
- error-log artifact path on failure.
Current process reports also include:
- module-level results (when runner modules execute), including applied/skipped proposal changes;
- run-level module summary totals and failed module instance metadata.
- module-level validator decisions and validator rejections.
- optional decision-level diagnostic artifact paths for validator LLM interactions when available.
- stable validator keys in `validator_name` fields for validator decisions/rejections.
- explicit report metadata:
- report schema name;
- report schema version;
- selected output schema;
- config file version when config file input is used.
- review/observability artifacts:
- run-level and module-level utilization/timing summaries;
- flattened correction ledger entries for applied/rejected/skipped/failed correction dispositions.
Utilization diagnostics collection:
- collection is performed in the runner path via lightweight instrumentation around LLM scheduler and structured-client execution (`internal/framework/runner`);
- instrumentation is observational only and does not change scheduler acquisition/release semantics or module execution order;
- serialized artifact: `utilization-diagnostics.json`.
Utilization diagnostics high-level shape:
- `effective_concurrency`:
- `total_llm`, `proposal_llm`, `validation_llm`;
- `run_timing`:
- `run_wall_time_ms`;
- `scheduler_queue_wait_ms`;
- `llm_execution_time_ms`;
- `deterministic_validation_time_ms`;
- `max_in_flight_llm_calls`;
- `average_in_flight_llm_calls`;
- `llm_calls`:
- `total_proposal_calls`;
- `total_validation_calls`;
- `modules`:
- per-module key/instance timing summaries including module wall time and per-module call counts;
- `validators`:
- per-validator summaries keyed by stable validator key with elapsed time and LLM-backed marker.
Correction ledger construction:
- ledger entries are built from runner module results in the CLI report/diagnostics path (`internal/cli/review_artifacts.go`);
- serialized artifact: `correction-ledger.json`;
- one flattened record per applied/validator-rejected/application-skipped outcome where data is available, plus module-failed records for failed module instances.
Correction ledger high-level shape:
- run/module/proposal identity:
- `run_id`, `module_key`, `module_instance`, `proposal_index`, `segment_id`;
- correction payload:
- `original_text`, `proposed_corrected_text`, `applied_corrected_text` (when applied), `replacement_policy`;
- disposition:
- `disposition` in `{applied,rejected,skipped,failed}`;
- `disposition_reason_code`, `disposition_message`;
- validator decision snapshots:
- `deterministic_validator_decisions[]`;
- `llm_validator_decisions[]`;
- each decision uses stable validator keys and reason codes.
Identity and metadata boundaries:
- stable module keys/instance names and stable validator keys are included directly in ledger records;
- prompt metadata and structured response schema metadata remain in LLM interaction diagnostics payloads and are not duplicated into every ledger row;
- reports reference artifact paths for utilization and ledger files through diagnostics metadata.
Redaction and retention:
- secret redaction guarantees continue to apply to diagnostics/report artifacts;
- utilization and ledger artifacts are emitted within the existing run-directory retention model (`auto|always|never`) and are retained/removed with the run directory.
Current report schema metadata values:
- `report_metadata.report_schema_name = "audita-process-report"`
- `report_metadata.report_schema_version = "v1"`
Retention modes implemented in `ApplyRetention`:
- `always`: keep all run directories.
- `never`: keep successful run directories.
- `auto`: keep failed runs and successful runs with skipped corrections.
- failed runs are always retained.
Current runtime note:
- default non-explicit runs usually have no module-level skipped corrections, so `auto` commonly removes clean successful run directories.
- explicit grammar/glossary/homophones/spoken_word runs can produce validator rejections and application skips, which are reflected in reports and retention input.
## Current tests and quality posture
Implemented tests currently cover:
- CLI argument handling and behavior (`internal/cli/run_test.go`)
- subprocess stdout/stderr and exit-code behavior (`cmd/audita/main_integration_test.go`)
- config/env/override validation (`internal/core/config/*_test.go`)
- transcript and glossary schema validation (`internal/core/schema/*_test.go`)
- deterministic normalization (`internal/core/normalization/*_test.go`)
- deterministic chunking and summaries (`internal/core/chunking/*_test.go`)
- proposal preview/apply semantics (`internal/framework/proposals/*_test.go`)
- contracts/foundation composition tests (`internal/framework/contracts/*_test.go`)
- runner sequencing and failure behavior with deterministic fake modules (`internal/framework/runner/*_test.go`)
- CLI runner integration through injected fake module factories (`internal/cli/run_test.go`)
- validator models, cardinality enforcement, and deterministic validators (`internal/framework/validators/*_test.go`)
- LLM-backed validator batching, prompt builders, structured-response safety, scheduler hooks, and diagnostics redaction (`internal/framework/validators/*_test.go`, `internal/framework/runner/*_test.go`)
- shared proposal-generation request/response parsing, deterministic indexing, scheduler hooks, and diagnostics redaction (`internal/framework/proposal_generation/*_test.go`, `internal/framework/runner/*_test.go`)
- production module-registry known-key recognition and unsupported/internal-registry error behavior (`internal/framework/modules/*_test.go`, `internal/cli/run_test.go`)
- production grammar module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, and explicit CLI/runtime integration (`internal/modules/grammar/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
- production glossary module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, repeated-stage behavior, and explicit CLI/runtime integration (`internal/modules/glossary/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
- production homophones module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, protected-term behavior, and explicit CLI/runtime integration (`internal/modules/homophones/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
- production spoken_word module prompt constraints, proposal mapping, validator-chain behavior, semantic guardrail behavior, confidence-threshold enforcement, diagnostics redaction, protected-term behavior, and explicit CLI/runtime integration (`internal/modules/spoken_word/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
- glossary-derived protected-term extraction and stable behavior (`internal/framework/validators/protected_terms_test.go`)
- default full-pipeline runtime shape and ordering (`internal/cli/run_test.go`, `cmd/audita/main_integration_test.go`, `internal/cli/parity_test.go`)
- subprocess operational hardening behavior including large-input, failure-mode, timeout/cancellation, backend-failure, and partial-progress paths (`cmd/audita/main_integration_test.go`)
- report/diagnostics redaction and artifact-shape behavior across success and failure paths (`internal/cli/run_test.go`, `cmd/audita/main_integration_test.go`)
- curated release-fixture and idempotence-oriented readiness checks using fake structured LLM responses (`internal/cli/release_fixtures_test.go`, `internal/cli/testdata/release`)
## Operational hardening status
The runtime now includes hardened subprocess behavior for parent-process callers:
- deterministic success/failure exit codes;
- strict stdout/stderr separation suitable for machine orchestration;
- failure stderr summaries that include diagnostics location when available;
- retained failure diagnostics (`report.json`, `error.log`, and artifacts written before failure);
- deterministic timeout/cancellation behavior in tests;
- redaction coverage for API keys/secrets across reports, diagnostics artifacts, and surfaced errors.
- stable output routing behavior:
- with `--output`, stdout remains empty on success;
- without `--output`, stdout contains only transcript JSON in the selected output schema;
- `--report-json` writes report data to file only (never stdout).
Operational caller guidance is documented in [`docs/subprocess-operations.md`](docs/subprocess-operations.md).
## Final status
- Audita's default full module-sequence runtime is implemented and tested.
- Parity fixtures and operational hardening coverage are in place.
- Historical migration context is documented in [`docs/migration-from-python.md`](docs/migration-from-python.md).