Files

Eric Rakestraw 87e560dd3d Added architecture reference documentation for the upcoming Go rewrite

2026-05-10 23:04:29 +00:00

19 KiB

Raw Blame History

Audita Go Architecture

Purpose

Audita is a framework-first transcript polishing application. It takes a source transcript, a glossary, and runtime configuration; runs an ordered sequence of correction modules; and writes a corrected transcript, a structured run report, and diagnostics artifacts.

The Go implementation should preserve the public behavior and safety posture of the existing Python implementation while using idiomatic Go internals. Treat the existing Python implementation, README, architecture notes, tests, and representative outputs as the behavioral specification during the rewrite.

Design goals

Preserve the existing batch CLI contract.
Preserve the existing module pipeline semantics.
Preserve deterministic safety checks and skipped-change reporting.
Preserve diagnostics and report artifacts as first-class outputs.
Make subprocess execution reliable from other Go applications.
Keep LLM provider integration small, explicit, and OpenAI-compatible.
Keep the design modular enough to add new transcript correction modules and validators.
Prefer simple, auditable Go code over agent frameworks or heavy runtime abstractions.

Non-goals for the initial Go rewrite

The initial Go implementation should not attempt to redesign Audita. In particular, it should not initially:

replace the CLI-first batch model with an HTTP service;
redesign the transcript or report schema;
improve prompt wording during the port;
change the default module order;
chase exact nondeterministic LLM output parity with the Python implementation;
introduce a general-purpose workflow or agent framework;
require downstream applications to change their integration model.

Later versions may add an HTTP API, new module types, richer report formats, or prompt improvements after the Go CLI is behaviorally stable.

Public CLI contract

The primary command is:

audita process transcript.json --glossary glossary.yaml --output corrected.json

Optional report output:

audita process transcript.json --glossary glossary.yaml --output corrected.json --report-json report.json

Custom module sequence:

audita process transcript.json --glossary glossary.yaml --modules grammar --output corrected.json

Expected stream behavior:

If --output is provided, corrected transcript JSON is written to that file.
If --output is omitted, corrected transcript JSON is written to stdout.
Progress logs and human-readable errors are written to stderr.
--report-json writes a machine-readable report to the specified file.
Report JSON is never mixed into stdout.
LLM prompt/response diagnostics are written under the per-run work directory, not to stdout.
Exit code 0 means the run completed successfully.
Nonzero exit codes mean the run failed; failed runs preserve diagnostics.

This stdout/stderr discipline is a core requirement because Audita is expected to be called as a subprocess by other Go applications.

Configuration model

Configuration sources should be applied in this order:

built-in defaults;
environment variables;
CLI flags.

CLI flags override environment variables. Environment variables override defaults.

The initial Go implementation should preserve the existing configuration surface where practical, including:

module sequence;
primary LLM API key, base URL, model, timeout, retry count, and concurrency;
validation LLM API key, base URL, model, timeout, retry count, and concurrency;
proposal-stage section token bounds;
validation-stage prompt token bounds;
module-specific confidence thresholds;
normalization parameters;
work directory location and retention policy.

Recommended Go shape:

internal/core/config
  Config
  LLMConfig
  NormalizationConfig
  ModuleConfig
  WorkDirRetention
  LoadFromEnv
  ApplyCLIOverrides
  Validate
  EffectiveValidationLLMConfig

API credentials must be redacted from reports, logs, diagnostics metadata, and error details.

High-level runtime flow

audita process should execute the following flow:

Parse CLI arguments.
Load and validate configuration.
Load and validate transcript input.
Load and validate glossary input.
Create a per-run diagnostics directory.
Persist redacted invocation/config metadata.
Persist the source transcript artifact.
Normalize transcript deterministically.
Persist normalized transcript and normalization summary.
Resolve configured module sequence into module run instances.
Execute modules sequentially over a mutable working transcript.
For each module instance:
- chunk the current working transcript into contiguous token-bounded sections;
- generate structured correction proposals using the module prompt;
- enrich proposals with module/run metadata;
- run validator chain;
- apply approved proposals according to replacement policy;
- report applied changes and skipped changes.
Sort final transcript chronologically.
Write corrected transcript to output file or stdout.
Write report.json if requested and always write authoritative report into retained diagnostics.
Apply work-dir retention policy.
On failure, preserve diagnostics, write failed report, write error.log, print concise stderr summary, and exit nonzero.

Suggested Go package layout

cmd/audita/
  main.go

internal/core/config/
  config.go
  env.go
  flags.go
  validation.go

internal/core/schema/
  transcript.go
  glossary.go
  report.go

internal/core/io/
  transcript.go
  glossary.go
  report.go
  json.go
  yaml.go

internal/core/normalization/
  normalize.go
  summary.go

internal/core/chunking/
  tokens.go
  sections.go
  batches.go

internal/core/diagnostics/
  run_dir.go
  artifacts.go
  redaction.go
  retention.go

internal/core/errors/
  errors.go
  exit_codes.go

internal/framework/runner/
  runner.go
  context.go
  result.go

internal/framework/proposals/
  proposal.go
  apply.go
  policy.go

internal/framework/llm/
  client.go
  openai_compatible.go
  structured.go
  retry.go
  scheduler.go

internal/framework/modules/
  module.go
  registry.go
  sequence.go

internal/modules/glossary/
  module.go
  prompt.go
  response.go

internal/modules/homophones/
  module.go
  prompt.go
  response.go

internal/modules/spokenword/
  module.go
  prompt.go
  response.go

internal/modules/grammar/
  module.go
  prompt.go
  response.go

internal/validators/
  validator.go
  result.go

internal/validators/deterministic/
  confidence.go
  original_text.go
  protected_terms.go
  non_empty.go

internal/validators/llm/
  validators.go
  prompts.go
  responses.go

internal/validators/protection/
  glossary_terms.go
  matching.go

internal/testutil/
  fixtures.go
  fakellm.go
  golden.go

This package layout is a starting point, not a rule. Prefer fewer packages if the implementation becomes fragmented. Keep package boundaries aligned with stable domain concepts: config, schema, normalization, chunking, diagnostics, LLM, proposals, modules, validators, and runner.

Core data contracts

Transcript

Audita should accept either:

a bare JSON array of segments; or
an object containing a segments array.

Segment fields:

id          integer
speaker     string
start       number
end         number
text        string
categories  optional array of strings

Input may contain source IDs. The normalized internal transcript should use strict sequential IDs starting at 1.

Final output should use the same normalized segment shape and should be sorted chronologically.

Glossary

The glossary model should support the existing glossary semantics used by the Python implementation, including domain-specific terms and protected vocabulary used by validators. The Go implementation should preserve the accepted YAML shape rather than inventing a new one during the rewrite.

Correction proposal

Modules emit structured proposals with the following logical fields:

id              target segment ID
original_text   span expected to exist in the target segment
corrected_text  replacement text
confidence      number between 0.0 and 1.0

The framework enriches proposals with execution metadata:

proposal_index
module_key
module_instance
section_id or batch_id
validator decisions
application status
skip reason, if any

Validation decision

Each validator must return exactly one decision for each candidate proposal index it was asked to evaluate.

Missing, duplicate, or unknown proposal indexes are framework errors. The framework should not silently ignore malformed validator output.

Run report

The run report should be machine-readable JSON. It should describe:

run status;
start/end timestamps or elapsed duration;
effective module sequence;
normalization summary;
module reports;
applied changes;
skipped changes;
errors, when present;
diagnostics directory, when retained.

The report should be stable enough for downstream tooling to consume.

Pipeline model

Audita uses a staged transform model over a mutable working transcript.

source transcript
  -> deterministic normalization
  -> working transcript
  -> module: glossary_1
  -> module: homophones
  -> module: glossary_2
  -> module: spoken_word
  -> module: grammar
  -> final transcript

Module order is architecturally significant. Each module sees the transcript produced by all previous modules.

Default logical module sequence:

glossary, homophones, glossary, spoken_word, grammar

Default resolved module instance names:

glossary_1, homophones, glossary_2, spoken_word, grammar

Repeated logical module keys should be auto-numbered in reports and diagnostics.

Module interface

A module should be a small object with stable metadata and proposal behavior.

Conceptual interface:

type TranscriptModule interface {
    Key() string
    ReplacementPolicy() proposals.ReplacementPolicy
    Validators() []validators.Validator
    Propose(ctx context.Context, req ProposalRequest) ([]proposals.CorrectionProposal, error)
}

A module is responsible for:

building its proposal prompt;
requesting structured proposal output from the LLM client;
mapping structured output into framework proposals;
attaching its validator chain.

A module should not:

own global process state;
write final outputs;
decide work-dir retention;
apply proposals directly;
bypass framework validators;
perform unbounded concurrency.

Module responsibilities

`glossary`

Proposes glossary-supported acoustic corrections and domain-specific term corrections. This module should be conservative and should rely heavily on glossary evidence and protected-vocabulary validation.

`homophones`

Proposes conservative homophone and mistranscription corrections. This module should avoid stylistic editing and should focus on likely transcription errors.

`spoken_word`

Proposes conservative dysfluency cleanup that does not affect substantive meaning. This module should be strongly guarded by semantic validators because it is easy to over-edit spoken language.

`grammar`

Proposes punctuation, capitalization, and spacing cleanup only. This module should not rewrite content for style or clarity beyond formatting and readability corrections.

Replacement policies

Proposal application should be centralized in the framework.

Supported policies:

require_unique  original_text must match exactly once in the target segment
replace_all     replace every occurrence of original_text in the target segment

If a proposal cannot be applied safely, it should be skipped and reported instead of crashing the run. Examples:

target segment no longer exists;
original_text is missing;
require_unique found zero matches;
require_unique found multiple matches;
corrected text is empty after validation;
proposal became stale after earlier module edits.

Validator model

Validators filter candidate proposals before application.

Validator categories:

deterministic validators;
LLM-backed validators.

Deterministic validators should run before LLM validators whenever possible because they are cheaper, faster, and more predictable.

Examples of deterministic validators:

confidence threshold;
original text presence;
non-empty correction;
identical text rejection;
protected glossary term checks.

Examples of LLM-backed validators:

spoken-form plausibility;
meaning reversal detection;
editorial review;
grammar review;
spoken-word cleanup review.

A validator should return structured results only. Human-readable explanations may be included in reports, but validation logic should consume typed decisions.

LLM integration

Audita should use a small OpenAI-compatible structured-output client.

The client should support:

configurable base URL;
configurable model;
optional API key;
per-request timeout;
retry budget;
structured JSON response schema;
raw prompt/response diagnostic capture;
context cancellation;
clear error wrapping.

Conceptual interface:

type StructuredLLMClient interface {
    CompleteStructured(ctx context.Context, req StructuredRequest, out any) error
}

The LLM client should not know about Audita modules, transcript schemas, or validators. It should only know how to submit an OpenAI-compatible request and decode a structured response.

Proposal and validation phases may use different effective LLM settings. Validation settings inherit from proposal settings unless explicitly overridden.

Structured output policy

For every LLM call, Audita should prefer strict structured JSON output over free-form text parsing.

Each module and LLM validator should define:

request payload type;
response payload type;
JSON schema, where the backend supports schema-constrained output;
response validation rules;
retry behavior for malformed or incomplete responses.

Malformed structured responses should become typed errors that include diagnostic references but do not leak API keys or excessive prompt text to stderr.

Concurrency model

Module stages remain sequential.

Within a module, bounded concurrency is allowed for:

proposal generation across transcript sections;
LLM validation batches;
adjacent independent LLM validators that evaluate the same candidate set;
prompt/response artifact writes.

All backend LLM calls should pass through a scheduler or semaphore that enforces configured concurrency limits.

Use context.Context for cancellation and timeout propagation.

Avoid unbounded goroutine creation. Every concurrent unit should be attached to the current run context and should return errors through a controlled mechanism such as errgroup.

Diagnostics model

A retained run directory should use a predictable structure similar to:

runs/
  2026-05-10T210000Z-<short-id>/
    invocation.json
    source-transcript.json
    normalized-transcript.json
    normalization-summary.json
    modules/
      glossary_1/
        section-001.prompt.json
        section-001.response.json
      homophones/
      glossary_2/
      spoken_word/
      grammar/
    report.json
    error.log

Retention modes:

always  keep every run directory
never   remove successful run directories
auto    keep failed runs and successful runs with final skipped corrections

Failed runs are always retained.

Diagnostics should be useful for debugging LLM behavior, proposal generation, validation decisions, and replacement failures.

Error handling model

Use typed errors for expected failure classes:

configuration errors;
input validation errors;
normalization errors;
module resolution errors;
LLM request errors;
structured response errors;
validator contract errors;
proposal application errors;
output write errors.

A pipeline failure should preserve partial progress when possible:

partial working transcript;
completed module reports;
applied changes so far;
skipped changes so far;
failed module information;
error details;
diagnostics references.

The CLI should print concise stderr output and point to retained diagnostics rather than dumping large prompts or stack traces directly to the terminal.

Testing strategy

The Go implementation should use three categories of tests.

Deterministic unit tests

Cover:

config loading and precedence;
transcript parsing;
glossary parsing;
normalization;
chunking;
proposal preview/application;
replacement policies;
validator contract enforcement;
report serialization;
diagnostics retention decisions.

Fake-LLM integration tests

Use a fake structured LLM client to test:

module proposal flow;
validator flow;
full pipeline execution;
repeated module instance names;
skipped-change reporting;
malformed LLM responses;
retry behavior;
partial failure reporting.

Subprocess tests

Invoke the compiled audita binary from tests and verify:

stdout contains only corrected transcript JSON when --output is omitted;
stdout is clean when --output is provided;
stderr contains logs/errors only;
--report-json writes a valid report;
failed runs exit nonzero;
failed runs retain diagnostics;
large inputs do not deadlock stdout/stderr pipes;
cancellation and timeout behavior are reliable.

Extension points

Add a module

Create a package under internal/modules/<name>.
Implement the module interface.
Define prompt builder and structured response type.
Define or reuse validators.
Register the logical module key in the module registry.
Add fake-LLM tests.
Add a fixture-level integration test.

Add a validator

Implement the validator interface.
Define typed result/decision structures.
For LLM validators, define prompt and structured response schema.
Enforce one decision per candidate proposal index.
Add tests for approved, rejected, malformed, and missing-decision cases.
Attach the validator to module chains intentionally.

Add an LLM backend

Implement StructuredLLMClient.
Preserve request timeout and context cancellation behavior.
Preserve raw prompt/response diagnostic capture.
Preserve structured response validation and retry semantics.
Add fake or local integration tests.

Implementation posture

The Go version should be boring infrastructure:

explicit structs;
explicit validation;
small interfaces;
clear package boundaries;
no hidden global state;
deterministic file outputs;
stable subprocess behavior;
diagnostics-first failures;
structured LLM calls rather than text scraping.

The primary measure of success is not that the Go code resembles the Python code. The measure of success is that the Go binary can replace the Python CLI in the surrounding transcript pipeline with fewer operational surprises.

19 KiB Raw Blame History