eric/audita
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Complete Phase 13 glossary module

Browse Source
This commit is contained in:
Eric Rakestraw
2026-05-12 11:20:02 +00:00
parent fc3a7b7a67
commit 543a7ff8ef
14 changed files with 1049 additions and 97 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
docs/architecture.md
View File

@@ -34,19 +34,22 @@ Implemented today:
- Proposal-generation diagnostics artifacts with secret redaction.
- Production module registry scaffolding with known-key recognition and explicit unsupported/unimplemented errors.
- Production `grammar` module implementation in `internal/modules/grammar`.
- Production `glossary` module implementation in `internal/modules/glossary`.
- Explicit runtime support for `--modules grammar` through the production runner path.
- Explicit runtime support for `--modules glossary`, including repeated stages such as `--modules glossary,glossary`.
Not implemented in CLI runtime path today:
- Real module execution pipeline for `glossary`, `homophones`, and `spoken_word`.
- Real domain proposal prompts for production modules.
- End-to-end transcript polishing with real module behavior.
- Real module execution pipeline for `homophones` and `spoken_word`.
- Real domain proposal prompts for remaining production modules.
- End-to-end transcript polishing with the full default module sequence.
Phase sequencing note:
- Phase 9 LLM infrastructure is complete (structured client, scheduler, effective config resolution, diagnostics primitives);
- Phase 10 LLM-backed validator runtime integration is complete;
- Phase 11 shared proposal-generation framework and module-registry scaffolding are complete;
- Phase 12 grammar module implementation and explicit runtime wiring are complete;
- next recommended phase is Phase 13 (glossary module and protected-term behavior).
- Phase 13 glossary module and protected-term behavior are complete;
- next recommended phase is Phase 14 (homophones module).
## Actual Go package layout
@@ -109,6 +112,10 @@ internal/modules/grammar/
module.go
prompt.go
internal/modules/glossary/
module.go
prompt.go
internal/framework/validators/
models.go
deterministic.go
@@ -144,17 +151,17 @@ Current runtime flow (`internal/cli/run.go`):
10. Chunk normalized transcript and compute chunk summaries.
11. Write chunking summary artifact.
12. Execute runner modules sequentially when:
- `--modules` is explicitly provided (production grammar path); or
- `--modules` is explicitly provided (production grammar/glossary paths); or
- a test/injected module factory is provided.
13. Output working transcript to `--output` file or stdout.
14. Build process report (`phase` currently set to `phase12-grammar-module`).
14. Build process report (`phase` currently set to `phase13-glossary-module`).
15. Optionally write `--report-json`; always write run-dir `report.json`.
16. Apply work-dir retention.
Important behavior details:
- Glossary is validated but not yet used for real correction module logic.
- Glossary is validated and is used for explicit glossary/grammar module correction paths.
- Default production CLI behavior remains deterministic normalization/chunking/reporting unless modules are explicitly selected with `--modules`.
- Explicit `--modules grammar` runs the production grammar module path with LLM-backed proposal generation and validator-chain execution.
- Explicit `--modules grammar` and `--modules glossary` run production module paths with LLM-backed proposal generation and validator-chain execution.
- Default runs (without explicit module selection) do not perform LLM calls.
- Success path is generally quiet on stderr.
- Source IDs are preserved into a canonical transcript before normalization; normalization then reassigns output IDs sequentially from `1`.
@@ -203,7 +210,7 @@ Implemented config surfaces include:
- work-dir and retention mode
Current caveat:
- LLM/module-related settings are active for explicit grammar runs; the default non-explicit path remains deterministic.
- LLM/module-related settings are active for explicit grammar/glossary runs; the default non-explicit path remains deterministic.
## Implemented structured LLM infrastructure
`internal/framework/contracts` now defines a typed structured-completion contract:
@@ -220,7 +227,7 @@ Current caveat:
Current runtime boundary:
- the default CLI runtime path (without explicit module selection) still does not instantiate the full production module sequence.
- LLM calls are exercised in production when `--modules grammar` is explicitly requested and in tests when fake/injected clients are used.
- LLM calls are exercised in production when `--modules grammar` or `--modules glossary` is explicitly requested and in tests when fake/injected clients are used.
`internal/framework/llm` also provides:
- a bounded `Scheduler` for controlled concurrent LLM calls with reliable permit release;
@@ -262,7 +269,7 @@ Current behavior details:
`internal/framework/contracts` provides interfaces and run-spec metadata scaffolding, including deterministic repeated module instance naming (`ResolveModuleRunSpecs`).
These primitives are wired into the production runner and report model. The grammar module is implemented; other production modules remain pending.
These primitives are wired into the production runner and report model. The grammar and glossary modules are implemented; other production modules remain pending.
## Implemented validator runtime infrastructure
`internal/framework/validators` provides deterministic validator infrastructure:
@@ -327,7 +334,7 @@ This helper only produces candidate proposals; validator-chain execution and pro
- diagnostics directory context
- returns explicit errors for unknown keys (`unsupported_module`) and recognized-but-unimplemented keys (`unimplemented_module`).
The `grammar` module key is now registered and constructible. `glossary`, `homophones`, and `spoken_word` remain recognized-but-unimplemented.
The `grammar` and `glossary` module keys are now registered and constructible. `homophones` and `spoken_word` remain recognized-but-unimplemented.
## Implemented grammar production module
`internal/modules/grammar` now provides the first production module:
@@ -340,6 +347,27 @@ The `grammar` module key is now registered and constructible. `glossary`, `homop
- grammar confidence threshold enforcement through existing validator/config infrastructure;
- module-level reporting and diagnostics capture through existing runner/reporting paths.
## Implemented glossary production module
`internal/modules/glossary` now provides the second production module:
- prompt builder aligned to Python glossary-module intent, constrained to glossary-backed domain/acoustic corrections;
- prompt context includes glossary names, aliases, categories, summaries, and plural forms where available;
- guardrails against broad style rewriting and against replacing unrelated terms simply because they appear in glossary entries;
- proposal generation through `internal/framework/proposal_generation` and `contracts.StructuredLLMClient`;
- scheduler-aware proposal calls through existing `contracts.LLMScheduler` hooks;
- replacement policy `replace_all` (matching Python glossary behavior);
- validator chain integration using existing deterministic + LLM-backed validators;
- glossary confidence threshold enforcement through existing validator/config infrastructure;
- module-level reporting and diagnostics capture through existing runner/reporting paths;
- explicit support for repeated glossary stages with deterministic instance names (`glossary_1`, `glossary_2`, ...), where later stages see prior-stage working transcript changes.
## Implemented protected-term behavior
`internal/framework/validators/protected_terms.go` provides deterministic glossary-derived protected vocabulary:
- extracts protected terms from glossary names and aliases;
- includes explicit plural fields and synthetic plural forms where safe;
- deduplicates and returns stable ordering for repeatable behavior/tests.
This vocabulary is used by deterministic validators for both glossary-stage and non-glossary-stage protection checks, keeping protected-term guardrails active across modules.
## Reports and diagnostics (implemented)
Current per-run artifacts include:
- `source-transcript.json`
@@ -379,7 +407,7 @@ Retention modes implemented in `ApplyRetention`:
Current runtime note:
- default non-explicit runs usually have no module-level skipped corrections, so `auto` commonly removes clean successful run directories.
- explicit grammar runs can produce validator rejections and application skips, which are reflected in reports and retention input.
- explicit grammar/glossary runs can produce validator rejections and application skips, which are reflected in reports and retention input.
Intentionally deferred to module/LLM phases:
- real domain proposal prompts and production module implementations remain tied to later module phases.
@@ -401,13 +429,15 @@ Implemented tests currently cover:
- shared proposal-generation request/response parsing, deterministic indexing, scheduler hooks, and diagnostics redaction (`internal/framework/proposal_generation/*_test.go`, `internal/framework/runner/*_test.go`)
- production module-registry known-key recognition and unsupported/unimplemented error behavior (`internal/framework/modules/*_test.go`, `internal/cli/run_test.go`)
- production grammar module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, and explicit CLI/runtime integration (`internal/modules/grammar/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
- production glossary module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, repeated-stage behavior, and explicit CLI/runtime integration (`internal/modules/glossary/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
- glossary-derived protected-term extraction and stable behavior (`internal/framework/validators/protected_terms_test.go`)
Not covered yet (because not implemented): production `glossary`, `homophones`, and `spoken_word` modules plus full default-sequence transcript-polishing runtime behavior.
Not covered yet (because not implemented): production `homophones` and `spoken_word` modules plus full default-sequence transcript-polishing runtime behavior.
## Intended final architecture (not yet implemented)
The intended end-state still matches the rewrite plan:
- sequential module pipeline over a mutable working transcript
- real module implementations (`glossary`, `homophones`, `spoken_word`, `grammar`)
- real module implementations (`homophones`, `spoken_word`)
- structured LLM proposal generation
- deterministic and LLM validators
- validator cardinality enforcement in pipeline execution