Complete Phase 14 homophones module

This commit is contained in:
2026-05-12 11:37:35 +00:00
parent 543a7ff8ef
commit dbf3605712
10 changed files with 858 additions and 57 deletions

View File

@@ -35,12 +35,13 @@ Implemented today:
- Production module registry scaffolding with known-key recognition and explicit unsupported/unimplemented errors. - Production module registry scaffolding with known-key recognition and explicit unsupported/unimplemented errors.
- Production `grammar` module implementation in `internal/modules/grammar`. - Production `grammar` module implementation in `internal/modules/grammar`.
- Production `glossary` module implementation in `internal/modules/glossary`. - Production `glossary` module implementation in `internal/modules/glossary`.
- Production `homophones` module implementation in `internal/modules/homophones`.
- Explicit runtime support for `--modules grammar` through the production runner path. - Explicit runtime support for `--modules grammar` through the production runner path.
- Explicit runtime support for `--modules glossary`, including repeated stages such as `--modules glossary,glossary`. - Explicit runtime support for `--modules glossary`, including repeated stages such as `--modules glossary,glossary`.
- Explicit runtime support for `--modules homophones` through the production runner path.
Not implemented in CLI runtime path today: Not implemented in CLI runtime path today:
- Real module execution pipeline for `homophones` and `spoken_word`. - Real module execution pipeline for `spoken_word`.
- Real domain proposal prompts for remaining production modules.
- End-to-end transcript polishing with the full default module sequence. - End-to-end transcript polishing with the full default module sequence.
Phase sequencing note: Phase sequencing note:
@@ -49,7 +50,8 @@ Phase sequencing note:
- Phase 11 shared proposal-generation framework and module-registry scaffolding are complete; - Phase 11 shared proposal-generation framework and module-registry scaffolding are complete;
- Phase 12 grammar module implementation and explicit runtime wiring are complete; - Phase 12 grammar module implementation and explicit runtime wiring are complete;
- Phase 13 glossary module and protected-term behavior are complete; - Phase 13 glossary module and protected-term behavior are complete;
- next recommended phase is Phase 14 (homophones module). - Phase 14 homophones module implementation and explicit runtime wiring are complete;
- next recommended phase is Phase 15 (spoken-word module).
## Actual Go package layout ## Actual Go package layout
@@ -116,6 +118,10 @@ internal/modules/glossary/
module.go module.go
prompt.go prompt.go
internal/modules/homophones/
module.go
prompt.go
internal/framework/validators/ internal/framework/validators/
models.go models.go
deterministic.go deterministic.go
@@ -151,17 +157,17 @@ Current runtime flow (`internal/cli/run.go`):
10. Chunk normalized transcript and compute chunk summaries. 10. Chunk normalized transcript and compute chunk summaries.
11. Write chunking summary artifact. 11. Write chunking summary artifact.
12. Execute runner modules sequentially when: 12. Execute runner modules sequentially when:
- `--modules` is explicitly provided (production grammar/glossary paths); or - `--modules` is explicitly provided (production grammar/glossary/homophones paths); or
- a test/injected module factory is provided. - a test/injected module factory is provided.
13. Output working transcript to `--output` file or stdout. 13. Output working transcript to `--output` file or stdout.
14. Build process report (`phase` currently set to `phase13-glossary-module`). 14. Build process report (`phase` currently set to `phase14-homophones-module`).
15. Optionally write `--report-json`; always write run-dir `report.json`. 15. Optionally write `--report-json`; always write run-dir `report.json`.
16. Apply work-dir retention. 16. Apply work-dir retention.
Important behavior details: Important behavior details:
- Glossary is validated and is used for explicit glossary/grammar module correction paths. - Glossary is validated and is used for explicit glossary/grammar/homophones module correction paths.
- Default production CLI behavior remains deterministic normalization/chunking/reporting unless modules are explicitly selected with `--modules`. - Default production CLI behavior remains deterministic normalization/chunking/reporting unless modules are explicitly selected with `--modules`.
- Explicit `--modules grammar` and `--modules glossary` run production module paths with LLM-backed proposal generation and validator-chain execution. - Explicit `--modules grammar`, `--modules glossary`, and `--modules homophones` run production module paths with LLM-backed proposal generation and validator-chain execution.
- Default runs (without explicit module selection) do not perform LLM calls. - Default runs (without explicit module selection) do not perform LLM calls.
- Success path is generally quiet on stderr. - Success path is generally quiet on stderr.
- Source IDs are preserved into a canonical transcript before normalization; normalization then reassigns output IDs sequentially from `1`. - Source IDs are preserved into a canonical transcript before normalization; normalization then reassigns output IDs sequentially from `1`.
@@ -210,7 +216,7 @@ Implemented config surfaces include:
- work-dir and retention mode - work-dir and retention mode
Current caveat: Current caveat:
- LLM/module-related settings are active for explicit grammar/glossary runs; the default non-explicit path remains deterministic. - LLM/module-related settings are active for explicit grammar/glossary/homophones runs; the default non-explicit path remains deterministic.
## Implemented structured LLM infrastructure ## Implemented structured LLM infrastructure
`internal/framework/contracts` now defines a typed structured-completion contract: `internal/framework/contracts` now defines a typed structured-completion contract:
@@ -227,7 +233,7 @@ Current caveat:
Current runtime boundary: Current runtime boundary:
- the default CLI runtime path (without explicit module selection) still does not instantiate the full production module sequence. - the default CLI runtime path (without explicit module selection) still does not instantiate the full production module sequence.
- LLM calls are exercised in production when `--modules grammar` or `--modules glossary` is explicitly requested and in tests when fake/injected clients are used. - LLM calls are exercised in production when `--modules grammar`, `--modules glossary`, or `--modules homophones` is explicitly requested and in tests when fake/injected clients are used.
`internal/framework/llm` also provides: `internal/framework/llm` also provides:
- a bounded `Scheduler` for controlled concurrent LLM calls with reliable permit release; - a bounded `Scheduler` for controlled concurrent LLM calls with reliable permit release;
@@ -334,7 +340,7 @@ This helper only produces candidate proposals; validator-chain execution and pro
- diagnostics directory context - diagnostics directory context
- returns explicit errors for unknown keys (`unsupported_module`) and recognized-but-unimplemented keys (`unimplemented_module`). - returns explicit errors for unknown keys (`unsupported_module`) and recognized-but-unimplemented keys (`unimplemented_module`).
The `grammar` and `glossary` module keys are now registered and constructible. `homophones` and `spoken_word` remain recognized-but-unimplemented. The `grammar`, `glossary`, and `homophones` module keys are now registered and constructible. `spoken_word` remains recognized-but-unimplemented.
## Implemented grammar production module ## Implemented grammar production module
`internal/modules/grammar` now provides the first production module: `internal/modules/grammar` now provides the first production module:
@@ -368,6 +374,19 @@ The `grammar` and `glossary` module keys are now registered and constructible. `
This vocabulary is used by deterministic validators for both glossary-stage and non-glossary-stage protection checks, keeping protected-term guardrails active across modules. This vocabulary is used by deterministic validators for both glossary-stage and non-glossary-stage protection checks, keeping protected-term guardrails active across modules.
## Implemented homophones production module
`internal/modules/homophones` now provides the third production module:
- prompt builder aligned to Python homophones-module intent, constrained to conservative homophone/near-homophone/mistranscription corrections;
- prompt context includes protected glossary names/aliases/plurals to avoid damaging known terms;
- explicit guardrails against punctuation cleanup, grammar cleanup, style rewriting, summarization, and content invention;
- proposal generation through `internal/framework/proposal_generation` and `contracts.StructuredLLMClient`;
- scheduler-aware proposal calls through existing `contracts.LLMScheduler` hooks;
- replacement policy `require_unique` (matching Python homophones behavior);
- validator chain integration using existing deterministic + LLM-backed validators;
- homophones confidence threshold enforcement through existing validator/config infrastructure;
- protected-term guardrails for non-glossary modules remain active and are exercised through the homophones path;
- module-level reporting and diagnostics capture through existing runner/reporting paths.
## Reports and diagnostics (implemented) ## Reports and diagnostics (implemented)
Current per-run artifacts include: Current per-run artifacts include:
- `source-transcript.json` - `source-transcript.json`
@@ -407,7 +426,7 @@ Retention modes implemented in `ApplyRetention`:
Current runtime note: Current runtime note:
- default non-explicit runs usually have no module-level skipped corrections, so `auto` commonly removes clean successful run directories. - default non-explicit runs usually have no module-level skipped corrections, so `auto` commonly removes clean successful run directories.
- explicit grammar/glossary runs can produce validator rejections and application skips, which are reflected in reports and retention input. - explicit grammar/glossary/homophones runs can produce validator rejections and application skips, which are reflected in reports and retention input.
Intentionally deferred to module/LLM phases: Intentionally deferred to module/LLM phases:
- real domain proposal prompts and production module implementations remain tied to later module phases. - real domain proposal prompts and production module implementations remain tied to later module phases.
@@ -430,14 +449,15 @@ Implemented tests currently cover:
- production module-registry known-key recognition and unsupported/unimplemented error behavior (`internal/framework/modules/*_test.go`, `internal/cli/run_test.go`) - production module-registry known-key recognition and unsupported/unimplemented error behavior (`internal/framework/modules/*_test.go`, `internal/cli/run_test.go`)
- production grammar module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, and explicit CLI/runtime integration (`internal/modules/grammar/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`) - production grammar module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, and explicit CLI/runtime integration (`internal/modules/grammar/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
- production glossary module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, repeated-stage behavior, and explicit CLI/runtime integration (`internal/modules/glossary/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`) - production glossary module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, repeated-stage behavior, and explicit CLI/runtime integration (`internal/modules/glossary/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
- production homophones module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, protected-term behavior, and explicit CLI/runtime integration (`internal/modules/homophones/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
- glossary-derived protected-term extraction and stable behavior (`internal/framework/validators/protected_terms_test.go`) - glossary-derived protected-term extraction and stable behavior (`internal/framework/validators/protected_terms_test.go`)
Not covered yet (because not implemented): production `homophones` and `spoken_word` modules plus full default-sequence transcript-polishing runtime behavior. Not covered yet (because not implemented): production `spoken_word` module plus full default-sequence transcript-polishing runtime behavior.
## Intended final architecture (not yet implemented) ## Intended final architecture (not yet implemented)
The intended end-state still matches the rewrite plan: The intended end-state still matches the rewrite plan:
- sequential module pipeline over a mutable working transcript - sequential module pipeline over a mutable working transcript
- real module implementations (`homophones`, `spoken_word`) - real module implementation (`spoken_word`)
- structured LLM proposal generation - structured LLM proposal generation
- deterministic and LLM validators - deterministic and LLM validators
- validator cardinality enforcement in pipeline execution - validator cardinality enforcement in pipeline execution

View File

@@ -65,6 +65,9 @@ Implemented:
- Glossary-derived deterministic protected-term extraction and validator integration. - Glossary-derived deterministic protected-term extraction and validator integration.
- Explicit `--modules glossary` runtime path through runner, shared proposal generation, validators, application, reporting, and diagnostics. - Explicit `--modules glossary` runtime path through runner, shared proposal generation, validators, application, reporting, and diagnostics.
- Repeated glossary stage support with deterministic instance names (`glossary_1`, `glossary_2`), including mutable working-transcript handoff. - Repeated glossary stage support with deterministic instance names (`glossary_1`, `glossary_2`), including mutable working-transcript handoff.
- Production homophones module package with Python-aligned prompt intent and guardrails.
- Explicit `--modules homophones` runtime path through runner, shared proposal generation, validators, application, reporting, and diagnostics.
- Focused multi-module runtime tests for already-implemented interoperability (for example `glossary,homophones`) without claiming full default-pipeline completion.
- Broad deterministic and CLI/subprocess test coverage for implemented phases through `go test ./...`. - Broad deterministic and CLI/subprocess test coverage for implemented phases through `go test ./...`.
- Internal typed structured LLM contract (`StructuredLLMClient.CompleteStructured(ctx, req, out)`). - Internal typed structured LLM contract (`StructuredLLMClient.CompleteStructured(ctx, req, out)`).
- `internal/framework/llm` instructor-go-backed adapter with: - `internal/framework/llm` instructor-go-backed adapter with:
@@ -80,8 +83,8 @@ Implemented:
- Generic JSON diagnostics primitives for LLM interactions (request metadata, request payload, response payload, optional error payload) with secret redaction. - Generic JSON diagnostics primitives for LLM interactions (request metadata, request payload, response payload, optional error payload) with secret redaction.
Not yet implemented in runtime pipeline: Not yet implemented in runtime pipeline:
- Real correction modules for `homophones` and `spoken_word`. - Real correction module for `spoken_word`.
- Domain proposal prompts for remaining real modules. - Domain proposal prompts for remaining real module.
- End-to-end transcript polishing behavior with the full default module sequence. - End-to-end transcript polishing behavior with the full default module sequence.
## Completed phases ## Completed phases
@@ -220,7 +223,7 @@ Not implemented in Phase 8 (by design):
## Remaining work plan ## Remaining work plan
Next recommended phase: **Phase 14 (homophones module)**. Next recommended phase: **Phase 15 (spoken-word module)**.
## Phase 9: Structured LLM client and scheduler infrastructure ## Phase 9: Structured LLM client and scheduler infrastructure
@@ -391,41 +394,27 @@ Not implemented in Phase 13 (by design):
## Phase 14: Homophones module ## Phase 14: Homophones module
### Purpose Completed.
Implement conservative homophone and mistranscription correction behavior. Implemented:
- Production homophones module package in `internal/modules/homophones`.
- Homophones prompt builder aligned to Python intent and constrained to conservative homophone/near-homophone/mistranscription corrections.
- Prompt context using glossary/protected-term information (names, aliases, plurals where present) to avoid damaging known domain terms.
- Homophones proposal generation through shared `internal/framework/proposal_generation` using `contracts.StructuredLLMClient`.
- Scheduler-aware homophones proposal generation through existing scheduler hooks.
- Homophones replacement policy `require_unique` (matching Python behavior).
- Homophones validator chain using existing deterministic and LLM-backed validators.
- Homophones confidence threshold enforcement through existing config + confidence-threshold validator behavior.
- Protected-term guardrails remaining active for homophones via existing deterministic validators.
- Explicit runtime support for `--modules homophones` through normalization, chunking, runner, proposal generation, validation, application, and reporting.
- Prompt/response diagnostics artifacts for homophones proposal + validator interactions with secret redaction.
- Module-level reports for homophones including generated proposals, validator decisions/rejections, applied changes, and application skips.
- CLI/runtime fake-client tests for approved proposals, validator rejection, application skips, diagnostics, protected-term rejection behavior, failure/error.log behavior, and report outputs (`--report-json` and run-dir `report.json`).
- Focused interoperability tests for already-implemented module combinations (for example `glossary,homophones`) to verify working-transcript handoff and guardrails without claiming full default-sequence parity.
### Scope Not implemented in Phase 14 (by design):
- Production `spoken_word` module.
Implement: - Full default module sequence execution as a feature-complete claim.
- `homophones` module package.
- Homophones prompt builder ported from Python.
- Homophones structured response model.
- Homophones replacement policy.
- Homophones confidence threshold handling.
- Homophones validator chain.
- Prompt/response diagnostics.
- CLI support for `--modules homophones`.
- Fake LLM tests.
- Tests for interaction with glossary/protected terms where relevant.
Do not implement:
- Spoken-word module.
- Default full pipeline parity claim unless spoken-word is already complete.
### Expected behavior at end of phase
Running `audita process ... --modules homophones` should perform real conservative homophone/mistranscription corrections using the configured LLM endpoint.
### Definition of done
- Homophones module runs in the production runner.
- Homophones proposals are generated through structured LLM calls.
- Validator chain is enforced.
- Protected-term behavior is respected where applicable.
- Applied/skipped homophone changes appear in reports.
- Prompt/response diagnostics are written.
- `go test ./...` passes without requiring external LLM credentials.
## Phase 15: Spoken-word module ## Phase 15: Spoken-word module

View File

@@ -460,7 +460,7 @@ func extractErrorPhase(err error) (phase string, message string) {
func buildProcessReport(status string, inv processInvocation, runDir *diagnostics.RunDirectory, startedAt, completedAt time.Time, errorMessage string, errorPhase string, normalizationSummary *normalization.NormalizationSummary, chunkingSummary *chunking.Summary, runOutput *runner.RunOutput) reporting.ProcessReport { func buildProcessReport(status string, inv processInvocation, runDir *diagnostics.RunDirectory, startedAt, completedAt time.Time, errorMessage string, errorPhase string, normalizationSummary *normalization.NormalizationSummary, chunkingSummary *chunking.Summary, runOutput *runner.RunOutput) reporting.ProcessReport {
report := reporting.ProcessReport{ report := reporting.ProcessReport{
Phase: "phase13-glossary-module", Phase: "phase14-homophones-module",
Status: status, Status: status,
Operation: "process", Operation: "process",
TranscriptPath: inv.TranscriptPath, TranscriptPath: inv.TranscriptPath,

View File

@@ -616,8 +616,8 @@ func TestRunProcessReportJSONIncludesChunkingSummary(t *testing.T) {
if report.Chunking.MaxSectionTokens == 0 { if report.Chunking.MaxSectionTokens == 0 {
t.Errorf("expected max_section_tokens in report") t.Errorf("expected max_section_tokens in report")
} }
if report.Phase != "phase13-glossary-module" { if report.Phase != "phase14-homophones-module" {
t.Errorf("expected phase 'phase13-glossary-module', got %q", report.Phase) t.Errorf("expected phase 'phase14-homophones-module', got %q", report.Phase)
} }
} }
@@ -924,7 +924,7 @@ func TestRunProcessProductionRegistryUnimplementedModuleFailsCleanly(t *testing.
exitCode := Run([]string{ exitCode := Run([]string{
"process", transcriptPath, "process", transcriptPath,
"--glossary", fixturePath("tiny_glossary.yaml"), "--glossary", fixturePath("tiny_glossary.yaml"),
"--modules", "homophones", "--modules", "spoken_word",
"--work-dir", workDir, "--work-dir", workDir,
"--work-dir-retention", "always", "--work-dir-retention", "always",
"--report-json", reportPath, "--report-json", reportPath,
@@ -955,7 +955,7 @@ func TestRunProcessProductionRegistryUnimplementedModuleFailsCleanly(t *testing.
} }
func TestRunProcessExplicitUnimplementedModulesFailClearly(t *testing.T) { func TestRunProcessExplicitUnimplementedModulesFailClearly(t *testing.T) {
for _, moduleKey := range []string{"homophones", "spoken_word"} { for _, moduleKey := range []string{"spoken_word"} {
t.Run(moduleKey, func(t *testing.T) { t.Run(moduleKey, func(t *testing.T) {
var stdout, stderr bytes.Buffer var stdout, stderr bytes.Buffer
transcriptPath := writeFile(t, "transcript.json", `[ transcriptPath := writeFile(t, "transcript.json", `[
@@ -1472,6 +1472,324 @@ func TestRunProcessExplicitGlossaryMalformedLLMOutputFailsWithErrorLog(t *testin
} }
} }
func TestRunProcessExplicitHomophonesAppliesCorrectionAndReportsDiagnostics(t *testing.T) {
secret := "phase14-secret"
proposalClient := &fakeStructuredLLMClient{
proposalResponses: []proposal_generation.StructuredCorrectionSet{
{
Corrections: []proposal_generation.StructuredCorrectionProposal{
{TargetSegmentID: 1, OriginalText: "gestures", CorrectedText: "Jesters", Confidence: 0.95},
},
},
},
}
validationClient := &fakeStructuredLLMClient{
validationResponses: []validators.LLMValidationResponse{
{Validations: []validators.LLMValidationDecision{{CorrectionIndex: 0, Approved: true, Confidence: 0.99, Reason: "spoken plausible"}}},
{Validations: []validators.LLMValidationDecision{{CorrectionIndex: 0, Approved: true, Confidence: 0.99, Reason: secret}}},
},
}
processProposalLLMClient = proposalClient
processValidationLLMClient = validationClient
t.Setenv("AUDITA_LLM_API_KEY", secret)
t.Setenv("AUDITA_VALIDATION_LLM_API_KEY", secret)
t.Cleanup(func() {
processProposalLLMClient = nil
processValidationLLMClient = nil
})
var stdout, stderr bytes.Buffer
workDir := t.TempDir()
reportPath := filepath.Join(t.TempDir(), "report.json")
outputPath := filepath.Join(t.TempDir(), "out.json")
transcriptPath := writeFile(t, "transcript.json", `[
{"id":1,"speaker":"Alice","start":0.0,"end":1.0,"text":"There were gestures in the hall."}
]`)
exitCode := Run([]string{
"process", transcriptPath,
"--glossary", fixturePath("tiny_glossary.yaml"),
"--modules", "homophones",
"--output", outputPath,
"--report-json", reportPath,
"--work-dir", workDir,
"--work-dir-retention", "always",
}, &stdout, &stderr)
if exitCode != 0 {
t.Fatalf("expected success, got %d stderr=%q", exitCode, stderr.String())
}
if stdout.Len() != 0 {
t.Fatalf("expected empty stdout with --output, got %q", stdout.String())
}
parsed, err := schema.ParseTranscriptJSON(readFile(t, outputPath))
if err != nil {
t.Fatalf("parse output: %v", err)
}
if parsed.Segments[0].Text != "There were Jesters in the hall." {
t.Fatalf("expected homophones correction applied, got %q", parsed.Segments[0].Text)
}
report := readProcessReport(t, reportPath)
if len(report.ModuleResults) != 1 || report.ModuleResults[0].ModuleKey != "homophones" {
t.Fatalf("expected one homophones module result, got %+v", report.ModuleResults)
}
if len(report.ModuleResults[0].AppliedChanges) != 1 {
t.Fatalf("expected one applied homophones change, got %+v", report.ModuleResults[0].AppliedChanges)
}
if len(report.ModuleResults[0].ValidatorDecisions) == 0 {
t.Fatalf("expected validator decisions in report")
}
runDir := onlyRunDir(t, workDir)
runReport := readProcessReport(t, filepath.Join(runDir, "report.json"))
if len(runReport.ModuleResults) != 1 || runReport.ModuleResults[0].ModuleKey != "homophones" {
t.Fatalf("expected homophones module results in run-dir report, got %+v", runReport.ModuleResults)
}
diagFiles, globErr := filepath.Glob(filepath.Join(runDir, "homophones", "*response-payload.json"))
if globErr != nil {
t.Fatalf("glob diagnostics: %v", globErr)
}
if len(diagFiles) == 0 {
t.Fatalf("expected homophones diagnostics payload files in %s", filepath.Join(runDir, "homophones"))
}
for _, f := range diagFiles {
raw := string(readFile(t, f))
if strings.Contains(raw, secret) {
t.Fatalf("secret leaked in diagnostics %q: %s", f, raw)
}
}
}
func TestRunProcessExplicitHomophonesRejectedAndApplicationSkipAreDistinct(t *testing.T) {
processProposalLLMClient = &fakeStructuredLLMClient{
proposalResponses: []proposal_generation.StructuredCorrectionSet{
{
Corrections: []proposal_generation.StructuredCorrectionProposal{
{TargetSegmentID: 1, OriginalText: "hall", CorrectedText: "temple", Confidence: 0.99},
{TargetSegmentID: 1, OriginalText: "there were gestures in the hall", CorrectedText: "there were gestures in the temple", Confidence: 0.99},
{TargetSegmentID: 1, OriginalText: "and there were gestures.", CorrectedText: "and there were jesters.", Confidence: 0.99},
},
},
},
}
processValidationLLMClient = &fakeStructuredLLMClient{
validationResponses: []validators.LLMValidationResponse{
{
Validations: []validators.LLMValidationDecision{
{CorrectionIndex: 0, Approved: true, Confidence: 0.9, Reason: "ok"},
{CorrectionIndex: 1, Approved: true, Confidence: 0.9, Reason: "ok"},
{CorrectionIndex: 2, Approved: false, Confidence: 0.9, Reason: "reject"},
},
},
{
Validations: []validators.LLMValidationDecision{
{CorrectionIndex: 0, Approved: true, Confidence: 0.9, Reason: "ok"},
{CorrectionIndex: 1, Approved: true, Confidence: 0.9, Reason: "ok"},
},
},
},
}
t.Cleanup(func() {
processProposalLLMClient = nil
processValidationLLMClient = nil
})
var stdout, stderr bytes.Buffer
reportPath := filepath.Join(t.TempDir(), "report.json")
outputPath := filepath.Join(t.TempDir(), "out.json")
transcriptPath := writeFile(t, "transcript.json", `[
{"id":1,"speaker":"Alice","start":0.0,"end":1.0,"text":"there were gestures in the hall and there were gestures."}
]`)
exitCode := Run([]string{
"process", transcriptPath,
"--glossary", fixturePath("tiny_glossary.yaml"),
"--modules", "homophones",
"--output", outputPath,
"--report-json", reportPath,
}, &stdout, &stderr)
if exitCode != 0 {
t.Fatalf("expected success, got %d stderr=%q", exitCode, stderr.String())
}
report := readProcessReport(t, reportPath)
if len(report.ModuleResults) != 1 {
t.Fatalf("expected one module result")
}
module := report.ModuleResults[0]
if len(module.ValidatorRejected) != 1 {
t.Fatalf("expected one validator rejection, got %+v", module.ValidatorRejected)
}
if len(module.SkippedChanges) != 1 {
t.Fatalf("expected one application skip, got %+v", module.SkippedChanges)
}
if module.ValidatorRejected[0].ReasonCode == string(module.SkippedChanges[0].SkipReason) {
t.Fatalf("validator rejection and application skip should remain distinct")
}
}
func TestRunProcessExplicitHomophonesProtectedGlossaryTermRejected(t *testing.T) {
processProposalLLMClient = &fakeStructuredLLMClient{
proposalResponses: []proposal_generation.StructuredCorrectionSet{
{
Corrections: []proposal_generation.StructuredCorrectionProposal{
{TargetSegmentID: 1, OriginalText: "Audita", CorrectedText: "audita", Confidence: 0.99},
},
},
},
}
t.Cleanup(func() {
processProposalLLMClient = nil
processValidationLLMClient = nil
})
var stdout, stderr bytes.Buffer
reportPath := filepath.Join(t.TempDir(), "report.json")
outputPath := filepath.Join(t.TempDir(), "out.json")
transcriptPath := writeFile(t, "transcript.json", `[
{"id":1,"speaker":"Alice","start":0.0,"end":1.0,"text":"Audita held the line."}
]`)
exitCode := Run([]string{
"process", transcriptPath,
"--glossary", fixturePath("tiny_glossary.yaml"),
"--modules", "homophones",
"--output", outputPath,
"--report-json", reportPath,
}, &stdout, &stderr)
if exitCode != 0 {
t.Fatalf("expected success, got %d stderr=%q", exitCode, stderr.String())
}
parsed, err := schema.ParseTranscriptJSON(readFile(t, outputPath))
if err != nil {
t.Fatalf("parse output: %v", err)
}
if parsed.Segments[0].Text != "Audita held the line." {
t.Fatalf("expected protected glossary term to remain unchanged, got %q", parsed.Segments[0].Text)
}
report := readProcessReport(t, reportPath)
if len(report.ModuleResults) != 1 {
t.Fatalf("expected one module result, got %+v", report.ModuleResults)
}
module := report.ModuleResults[0]
if len(module.AppliedChanges) != 0 {
t.Fatalf("expected no applied changes, got %+v", module.AppliedChanges)
}
if len(module.ValidatorRejected) != 1 {
t.Fatalf("expected one validator rejection, got %+v", module.ValidatorRejected)
}
if module.ValidatorRejected[0].ReasonCode != validators.ReasonProtectedGlossaryTerm {
t.Fatalf("expected protected glossary term rejection, got %+v", module.ValidatorRejected[0])
}
}
func TestRunProcessExplicitHomophonesMalformedLLMOutputFailsWithErrorLog(t *testing.T) {
processProposalLLMClient = &fakeStructuredLLMClient{err: errors.New("malformed structured output")}
t.Cleanup(func() { processProposalLLMClient = nil })
var stdout, stderr bytes.Buffer
workDir := t.TempDir()
reportPath := filepath.Join(t.TempDir(), "report.json")
transcriptPath := writeFile(t, "transcript.json", `[
{"id":1,"speaker":"Alice","start":0.0,"end":1.0,"text":"hello"}
]`)
exitCode := Run([]string{
"process", transcriptPath,
"--glossary", fixturePath("tiny_glossary.yaml"),
"--modules", "homophones",
"--work-dir", workDir,
"--work-dir-retention", "always",
"--report-json", reportPath,
}, &stdout, &stderr)
if exitCode == 0 {
t.Fatal("expected failure")
}
if stdout.Len() != 0 {
t.Fatalf("expected empty stdout on failure, got %q", stdout.String())
}
if !strings.Contains(stderr.String(), "runner_execution") {
t.Fatalf("expected runner_execution error, got %q", stderr.String())
}
runDir := onlyRunDir(t, workDir)
if _, err := os.Stat(filepath.Join(runDir, "error.log")); err != nil {
t.Fatalf("expected error.log on failed homophones run: %v", err)
}
report := readProcessReport(t, reportPath)
if report.Status != "failed" || report.ErrorPhase != "runner_execution" {
t.Fatalf("expected failed runner_execution report, got %+v", report)
}
}
func TestRunProcessExplicitGlossaryThenHomophonesSeesWorkingTranscriptChanges(t *testing.T) {
proposalClient := &fakeStructuredLLMClient{
proposalResponses: []proposal_generation.StructuredCorrectionSet{
{
Corrections: []proposal_generation.StructuredCorrectionProposal{
{TargetSegmentID: 1, OriginalText: "gestures", CorrectedText: "Jesters", Confidence: 0.99},
},
},
{
Corrections: []proposal_generation.StructuredCorrectionProposal{
{TargetSegmentID: 1, OriginalText: "Jesters", CorrectedText: "Jester", Confidence: 0.99},
},
},
},
}
validationClient := &fakeStructuredLLMClient{
validationResponses: []validators.LLMValidationResponse{
{Validations: []validators.LLMValidationDecision{{CorrectionIndex: 0, Approved: true, Confidence: 0.99, Reason: "ok"}}},
{Validations: []validators.LLMValidationDecision{{CorrectionIndex: 0, Approved: true, Confidence: 0.99, Reason: "ok"}}},
{Validations: []validators.LLMValidationDecision{{CorrectionIndex: 0, Approved: true, Confidence: 0.99, Reason: "ok"}}},
{Validations: []validators.LLMValidationDecision{{CorrectionIndex: 0, Approved: true, Confidence: 0.99, Reason: "ok"}}},
},
}
processProposalLLMClient = proposalClient
processValidationLLMClient = validationClient
t.Cleanup(func() {
processProposalLLMClient = nil
processValidationLLMClient = nil
})
var stdout, stderr bytes.Buffer
reportPath := filepath.Join(t.TempDir(), "report.json")
outputPath := filepath.Join(t.TempDir(), "out.json")
transcriptPath := writeFile(t, "transcript.json", `[
{"id":1,"speaker":"Alice","start":0.0,"end":1.0,"text":"There were gestures in the hall."}
]`)
exitCode := Run([]string{
"process", transcriptPath,
"--glossary", fixturePath("tiny_glossary.yaml"),
"--modules", "glossary,homophones",
"--output", outputPath,
"--report-json", reportPath,
}, &stdout, &stderr)
if exitCode != 0 {
t.Fatalf("expected success, got %d stderr=%q", exitCode, stderr.String())
}
parsed, err := schema.ParseTranscriptJSON(readFile(t, outputPath))
if err != nil {
t.Fatalf("parse output: %v", err)
}
if parsed.Segments[0].Text != "There were Jester in the hall." {
t.Fatalf("expected homophones stage to see glossary output, got %q", parsed.Segments[0].Text)
}
report := readProcessReport(t, reportPath)
if len(report.ModuleResults) != 2 {
t.Fatalf("expected two module results, got %+v", report.ModuleResults)
}
if report.ModuleResults[0].ModuleKey != "glossary" || report.ModuleResults[1].ModuleKey != "homophones" {
t.Fatalf("expected glossary then homophones results, got %+v", report.ModuleResults)
}
if len(report.ModuleResults[0].AppliedChanges) != 1 || len(report.ModuleResults[1].AppliedChanges) != 1 {
t.Fatalf("expected one applied change per module, got %+v", report.ModuleResults)
}
}
func TestRunProcessChunkingSummaryArtifactWritten(t *testing.T) { func TestRunProcessChunkingSummaryArtifactWritten(t *testing.T) {
var stdout bytes.Buffer var stdout bytes.Buffer
var stderr bytes.Buffer var stderr bytes.Buffer

View File

@@ -11,7 +11,7 @@ import (
func TestProcessReportModuleResultsJSONSuccessAndSkipped(t *testing.T) { func TestProcessReportModuleResultsJSONSuccessAndSkipped(t *testing.T) {
now := time.Now().UTC() now := time.Now().UTC()
report := ProcessReport{ report := ProcessReport{
Phase: "phase13-glossary-module", Phase: "phase14-homophones-module",
Status: "success", Status: "success",
ModuleResults: []ModuleReport{ ModuleResults: []ModuleReport{
{ {
@@ -80,7 +80,7 @@ func TestProcessReportModuleResultsJSONSuccessAndSkipped(t *testing.T) {
func TestProcessReportModuleResultsJSONFailedModule(t *testing.T) { func TestProcessReportModuleResultsJSONFailedModule(t *testing.T) {
now := time.Now().UTC() now := time.Now().UTC()
report := ProcessReport{ report := ProcessReport{
Phase: "phase13-glossary-module", Phase: "phase14-homophones-module",
Status: "failed", Status: "failed",
ModuleResults: []ModuleReport{ ModuleResults: []ModuleReport{
{ {

View File

@@ -10,6 +10,7 @@ import (
"gitea.maximumdirect.net/eric/audita/internal/framework/contracts" "gitea.maximumdirect.net/eric/audita/internal/framework/contracts"
glossarymodule "gitea.maximumdirect.net/eric/audita/internal/modules/glossary" glossarymodule "gitea.maximumdirect.net/eric/audita/internal/modules/glossary"
grammarmodule "gitea.maximumdirect.net/eric/audita/internal/modules/grammar" grammarmodule "gitea.maximumdirect.net/eric/audita/internal/modules/grammar"
homophonesmodule "gitea.maximumdirect.net/eric/audita/internal/modules/homophones"
) )
const ( const (
@@ -72,6 +73,7 @@ func NewFactory(deps Dependencies) *Factory {
constructors: make(map[string]Constructor, len(knownModuleKeys)), constructors: make(map[string]Constructor, len(knownModuleKeys)),
} }
_ = factory.RegisterConstructor(ModuleKeyGlossary, constructGlossaryModule) _ = factory.RegisterConstructor(ModuleKeyGlossary, constructGlossaryModule)
_ = factory.RegisterConstructor(ModuleKeyHomophones, constructHomophonesModule)
_ = factory.RegisterConstructor(ModuleKeyGrammar, constructGrammarModule) _ = factory.RegisterConstructor(ModuleKeyGrammar, constructGrammarModule)
return factory return factory
} }
@@ -104,6 +106,12 @@ func constructGlossaryModule(ctx context.Context, req ConstructRequest) (contrac
return glossarymodule.New() return glossarymodule.New()
} }
func constructHomophonesModule(ctx context.Context, req ConstructRequest) (contracts.TranscriptModule, error) {
_ = ctx
_ = req
return homophonesmodule.New()
}
// ModuleForSpec resolves one configured run spec into a module instance. // ModuleForSpec resolves one configured run spec into a module instance.
func (f *Factory) ModuleForSpec(spec contracts.ModuleRunSpec) (contracts.TranscriptModule, error) { func (f *Factory) ModuleForSpec(spec contracts.ModuleRunSpec) (contracts.TranscriptModule, error) {
if f == nil { if f == nil {

View File

@@ -66,7 +66,7 @@ func TestUnsupportedUnknownModuleKeyFailsCleanly(t *testing.T) {
func TestRecognizedButUnimplementedModuleKeyFailsCleanly(t *testing.T) { func TestRecognizedButUnimplementedModuleKeyFailsCleanly(t *testing.T) {
factory := NewFactory(Dependencies{}) factory := NewFactory(Dependencies{})
for _, key := range []string{ModuleKeyHomophones, ModuleKeySpokenWord} { for _, key := range []string{ModuleKeySpokenWord} {
t.Run(key, func(t *testing.T) { t.Run(key, func(t *testing.T) {
_, err := factory.ModuleForSpec(contracts.ModuleRunSpec{ModuleKey: key, InstanceName: key}) _, err := factory.ModuleForSpec(contracts.ModuleRunSpec{ModuleKey: key, InstanceName: key})
if err == nil { if err == nil {
@@ -106,6 +106,17 @@ func TestGlossaryIsRegisteredAndConstructibleByDefault(t *testing.T) {
} }
} }
func TestHomophonesIsRegisteredAndConstructibleByDefault(t *testing.T) {
factory := NewFactory(Dependencies{})
module, err := factory.ModuleForSpec(contracts.ModuleRunSpec{ModuleKey: ModuleKeyHomophones, InstanceName: ModuleKeyHomophones})
if err != nil {
t.Fatalf("ModuleForSpec error: %v", err)
}
if module.Key() != ModuleKeyHomophones {
t.Fatalf("expected homophones module key, got %q", module.Key())
}
}
func TestRegisterConstructorAndConstruct(t *testing.T) { func TestRegisterConstructorAndConstruct(t *testing.T) {
factory := NewFactory(Dependencies{}) factory := NewFactory(Dependencies{})
if err := factory.RegisterConstructor(ModuleKeyGlossary, func(ctx context.Context, req ConstructRequest) (contracts.TranscriptModule, error) { if err := factory.RegisterConstructor(ModuleKeyGlossary, func(ctx context.Context, req ConstructRequest) (contracts.TranscriptModule, error) {

View File

@@ -0,0 +1,76 @@
package homophones
import (
"context"
"fmt"
"gitea.maximumdirect.net/eric/audita/internal/framework/contracts"
"gitea.maximumdirect.net/eric/audita/internal/framework/proposal_generation"
"gitea.maximumdirect.net/eric/audita/internal/framework/proposals"
"gitea.maximumdirect.net/eric/audita/internal/framework/validators"
)
type Module struct {
validators []contracts.Validator
}
func New() (*Module, error) {
spokenForm, err := validators.NewLLMBackedValidator("spoken_form_plausibility_review", validators.LLMValidatorTypeSpokenFormPlausibility, "")
if err != nil {
return nil, err
}
meaningReversal, err := validators.NewLLMBackedValidator("meaning_reversal_review", validators.LLMValidatorTypeMeaningReversal, "")
if err != nil {
return nil, err
}
return &Module{
validators: []contracts.Validator{
validators.NoEffectValidator{},
validators.OriginalTextPresenceValidator{},
validators.ConfidenceThresholdValidator{},
validators.ProtectedGlossaryTermValidator{},
validators.NonEmptyCorrectionValidator{},
spokenForm,
meaningReversal,
},
}, nil
}
func (m *Module) Key() string { return "homophones" }
func (m *Module) ReplacementPolicy() proposals.ReplacementPolicy {
// Python homophones module uses require_unique for conservative single-span replacement.
return proposals.ReplacementPolicyRequireUnique
}
func (m *Module) Validators() []contracts.Validator {
return append([]contracts.Validator(nil), m.validators...)
}
func (m *Module) Propose(ctx context.Context, req contracts.ProposalRequest) ([]proposals.CorrectionProposal, error) {
messages, err := BuildProposalMessages(req.WorkingTranscript, req.Glossary)
if err != nil {
return nil, err
}
generated, err := proposal_generation.GenerateCandidates(ctx, proposal_generation.Request{
ModuleKey: req.RunSpec.ModuleKey,
ModuleInstance: req.RunSpec.InstanceName,
ReplacementPolicy: req.RunSpec.ReplacementPolicy,
WorkingTranscript: req.WorkingTranscript,
Section: req.Section,
Glossary: req.Glossary,
Config: req.Config,
Messages: messages,
StageName: fmt.Sprintf("%s:proposal", req.RunSpec.InstanceName),
StartIndex: 0,
LLMClient: req.LLMClient,
Scheduler: req.LLMScheduler,
DiagnosticsDir: req.DiagnosticsDir,
})
if err != nil {
return nil, err
}
return generated.Corrections, nil
}

View File

@@ -0,0 +1,296 @@
package homophones
import (
"context"
"errors"
"os"
"path/filepath"
"strings"
"testing"
"gitea.maximumdirect.net/eric/audita/internal/core/config"
"gitea.maximumdirect.net/eric/audita/internal/core/schema"
"gitea.maximumdirect.net/eric/audita/internal/framework/contracts"
"gitea.maximumdirect.net/eric/audita/internal/framework/proposal_generation"
"gitea.maximumdirect.net/eric/audita/internal/framework/proposals"
"gitea.maximumdirect.net/eric/audita/internal/framework/validators"
)
type fakeLLMClient struct {
responses []proposal_generation.StructuredCorrectionSet
err error
calls []contracts.StructuredCompletionRequest
}
func (f *fakeLLMClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
_ = ctx
f.calls = append(f.calls, req)
if f.err != nil {
return contracts.StructuredCompletionResponse{}, f.err
}
target, ok := out.(*proposal_generation.StructuredCorrectionSet)
if !ok {
return contracts.StructuredCompletionResponse{}, errors.New("unexpected output type")
}
if len(f.responses) == 0 {
return contracts.StructuredCompletionResponse{}, errors.New("unexpected call")
}
*target = f.responses[0]
f.responses = f.responses[1:]
return contracts.StructuredCompletionResponse{}, nil
}
type countingScheduler struct {
runs int
}
func (s *countingScheduler) Run(ctx context.Context, fn func(context.Context) error) error {
s.runs++
return fn(ctx)
}
func tinyTranscript() *schema.Transcript {
return &schema.Transcript{Segments: []schema.Segment{
{ID: 1, Speaker: "A", Start: 0, End: 1, Text: "There were gestures in the hall.", Categories: []string{"session"}},
}}
}
func tinyGlossary() *schema.Glossary {
return &schema.Glossary{Entries: []schema.GlossaryEntry{
{Name: "Jesters", Aliases: []string{"Jester"}, Category: "faction", Summary: "Guild members", Plural: "Jesters"},
}}
}
func TestBuildProposalMessagesContainsContextAndConservativeConstraints(t *testing.T) {
msgs, err := BuildProposalMessages(tinyTranscript(), tinyGlossary())
if err != nil {
t.Fatalf("BuildProposalMessages error: %v", err)
}
if len(msgs) != 2 {
t.Fatalf("expected 2 messages, got %d", len(msgs))
}
combined := msgs[0].Content + "\n" + msgs[1].Content
for _, want := range []string{
"Protected glossary/context:",
"Transcript section:",
`"Aliases":`,
`"Category": "faction"`,
`"Summary": "Guild members"`,
`"Plural": "Jesters"`,
"homophone correction assistant",
"mistranscriptions of spoken English",
"Do not add or remove punctuation",
"Do not return speaker, start, or end fields",
`"id": 1`,
} {
if !strings.Contains(combined, want) {
t.Fatalf("expected prompt to contain %q", want)
}
}
for _, forbidden := range []string{
"summarize the transcript",
"style rewrite",
"grammar cleanup",
"punctuation cleanup",
} {
if strings.Contains(strings.ToLower(combined), forbidden) {
t.Fatalf("prompt should not invite %q", forbidden)
}
}
}
func TestHomophonesModuleReplacementPolicy(t *testing.T) {
m, err := New()
if err != nil {
t.Fatalf("New error: %v", err)
}
if m.ReplacementPolicy() != proposals.ReplacementPolicyRequireUnique {
t.Fatalf("unexpected replacement policy: %q", m.ReplacementPolicy())
}
}
func TestHomophonesModuleValidatorChain(t *testing.T) {
m, err := New()
if err != nil {
t.Fatalf("New error: %v", err)
}
got := make([]string, 0, len(m.Validators()))
for _, v := range m.Validators() {
got = append(got, v.Name())
}
want := []string{
"no_effect",
"original_text_presence",
"confidence_threshold",
"protected_glossary_terms",
"non_empty_correction",
"spoken_form_plausibility_review",
"meaning_reversal_review",
}
if strings.Join(got, ",") != strings.Join(want, ",") {
t.Fatalf("unexpected validator chain\n got: %v\nwant: %v", got, want)
}
}
func TestHomophonesModuleProposeMapsCorrectionsAndWritesDiagnostics(t *testing.T) {
secret := "homophones-secret"
client := &fakeLLMClient{
responses: []proposal_generation.StructuredCorrectionSet{
{
Corrections: []proposal_generation.StructuredCorrectionProposal{
{TargetSegmentID: 1, OriginalText: "gestures", CorrectedText: "Jesters", Confidence: 0.94},
},
},
},
}
scheduler := &countingScheduler{}
cfg := config.Default()
cfg.PrimaryLLM.APIKey = secret
m, err := New()
if err != nil {
t.Fatalf("New error: %v", err)
}
diagDir := t.TempDir()
out, err := m.Propose(context.Background(), contracts.ProposalRequest{
ExecutionContext: contracts.ExecutionContext{
Config: &cfg,
WorkingTranscript: tinyTranscript(),
Glossary: tinyGlossary(),
DiagnosticsDir: diagDir,
},
RunSpec: contracts.ModuleRunSpec{
ModuleKey: "homophones",
InstanceName: "homophones",
ReplacementPolicy: proposals.ReplacementPolicyRequireUnique,
},
LLMClient: client,
LLMScheduler: scheduler,
})
if err != nil {
t.Fatalf("Propose error: %v", err)
}
if scheduler.runs != 1 {
t.Fatalf("expected scheduler run count 1, got %d", scheduler.runs)
}
if len(client.calls) != 1 || client.calls[0].StageName != "homophones:proposal" {
t.Fatalf("expected one homophones:proposal call, got %+v", client.calls)
}
if len(out) != 1 || out[0].CorrectedText != "Jesters" {
t.Fatalf("unexpected proposals: %+v", out)
}
diagFiles, globErr := filepath.Glob(filepath.Join(diagDir, "homophones", "*proposal*response-payload.json"))
if globErr != nil {
t.Fatalf("glob error: %v", globErr)
}
if len(diagFiles) == 0 {
t.Fatalf("expected diagnostics response payload under %s", filepath.Join(diagDir, "homophones"))
}
for _, f := range diagFiles {
raw, readErr := os.ReadFile(f)
if readErr != nil {
t.Fatalf("read diag %q: %v", f, readErr)
}
if strings.Contains(string(raw), secret) {
t.Fatalf("secret leaked in diagnostics: %s", string(raw))
}
}
}
func TestHomophonesConfidenceThresholdIsUsed(t *testing.T) {
module, err := New()
if err != nil {
t.Fatalf("New error: %v", err)
}
var thresholdValidator contracts.Validator
for _, v := range module.Validators() {
if v.Name() == "confidence_threshold" {
thresholdValidator = v
break
}
}
if thresholdValidator == nil {
t.Fatal("expected confidence_threshold validator")
}
cfg := config.Default()
cfg.Thresholds.Homophones = 0.95
result, err := thresholdValidator.Validate(context.Background(), validators.Request{
Config: &cfg,
ModuleKey: "homophones",
CandidateProposal: []proposals.EnrichedCorrectionProposal{
{
CorrectionProposal: proposals.CorrectionProposal{
TargetSegmentID: 1,
OriginalText: "gestures",
CorrectedText: "Jesters",
Confidence: 0.94,
},
ProposalMetadata: proposals.ProposalMetadata{ProposalIndex: 0, ModuleKey: "homophones", ModuleInstance: "homophones"},
},
},
})
if err != nil {
t.Fatalf("validate error: %v", err)
}
if len(result.Decisions) != 1 {
t.Fatalf("expected one decision, got %+v", result.Decisions)
}
if result.Decisions[0].Approved {
t.Fatalf("expected rejection under homophones threshold, got %+v", result.Decisions[0])
}
if result.Decisions[0].ReasonCode != validators.ReasonLowConfidence {
t.Fatalf("expected low confidence reason, got %+v", result.Decisions[0])
}
}
func TestHomophonesProtectedGlossaryTermBehaviorRejectsUnsafeCorrection(t *testing.T) {
module, err := New()
if err != nil {
t.Fatalf("New error: %v", err)
}
var protectedValidator contracts.Validator
for _, v := range module.Validators() {
if v.Name() == "protected_glossary_terms" {
protectedValidator = v
break
}
}
if protectedValidator == nil {
t.Fatal("expected protected_glossary_terms validator")
}
working := &schema.Transcript{Segments: []schema.Segment{
{ID: 1, Speaker: "A", Start: 0, End: 1, Text: "Jesters are here."},
}}
glossary := &schema.Glossary{Entries: []schema.GlossaryEntry{
{Name: "Jesters", Category: "faction", Summary: "Faction"},
}}
result, err := protectedValidator.Validate(context.Background(), validators.Request{
WorkingTranscript: working,
Glossary: glossary,
ModuleKey: "homophones",
CandidateProposal: []proposals.EnrichedCorrectionProposal{
{
CorrectionProposal: proposals.CorrectionProposal{
TargetSegmentID: 1,
OriginalText: "Jesters",
CorrectedText: "gestures",
Confidence: 0.99,
},
ProposalMetadata: proposals.ProposalMetadata{ProposalIndex: 0, ModuleKey: "homophones", ModuleInstance: "homophones"},
},
},
})
if err != nil {
t.Fatalf("validate error: %v", err)
}
if len(result.Decisions) != 1 {
t.Fatalf("expected one decision, got %+v", result.Decisions)
}
if result.Decisions[0].Approved {
t.Fatalf("expected protected-term rejection, got %+v", result.Decisions[0])
}
if result.Decisions[0].ReasonCode != validators.ReasonProtectedGlossaryTerm {
t.Fatalf("expected protected glossary-term reason, got %+v", result.Decisions[0])
}
}

View File

@@ -0,0 +1,83 @@
package homophones
import (
"encoding/json"
"fmt"
"gitea.maximumdirect.net/eric/audita/internal/core/schema"
"gitea.maximumdirect.net/eric/audita/internal/framework/contracts"
)
type promptSegment struct {
ID int `json:"id"`
Speaker string `json:"speaker"`
Start float64 `json:"start"`
End float64 `json:"end"`
Text string `json:"text"`
Categories []string `json:"categories,omitempty"`
}
type promptTranscriptSection struct {
SectionIndex int `json:"section_index"`
Segments []promptSegment `json:"segments"`
}
// BuildProposalMessages mirrors the Python homophones-module prompt intent:
// conservative homophone and mistranscription correction only.
func BuildProposalMessages(transcript *schema.Transcript, glossary *schema.Glossary) ([]contracts.LLMMessage, error) {
glossaryJSON, err := json.MarshalIndent(glossary, "", " ")
if err != nil {
return nil, fmt.Errorf("marshal glossary prompt context: %w", err)
}
sectionPayload := promptTranscriptSection{
SectionIndex: 0,
Segments: make([]promptSegment, 0),
}
if transcript != nil {
for _, s := range transcript.Segments {
sectionPayload.Segments = append(sectionPayload.Segments, promptSegment{
ID: s.ID,
Speaker: s.Speaker,
Start: s.Start,
End: s.End,
Text: s.Text,
Categories: append([]string(nil), s.Categories...),
})
}
}
sectionJSON, err := json.MarshalIndent(sectionPayload, "", " ")
if err != nil {
return nil, fmt.Errorf("marshal transcript prompt context: %w", err)
}
system := "You are Audita, a conservative homophone correction assistant. Identify only transcript changes that plausibly reflect homophones, phonetic similarity, or common mistranscriptions of spoken English. Do not make punctuation, capitalization, spacing, filler-word, repetition, style, or grammar edits. Do not paraphrase, summarize, or rewrite content."
user := "Review this transcript section and return only homophone or spoken-form corrections that should be applied.\n\n" +
"Rules:\n" +
"- Approve only corrections where the original text is plausibly a mistaken homophone, phonetic rendering, or mistranscription of what was likely spoken.\n" +
"- Allow examples such as changing \"dam\" to \"damn\", \"rank\" to \"Hrank\", or \"gestures\" to \"Jesters\" when local context supports the correction.\n" +
"- Reject unrelated substitutions like changing \"Lyra\" to \"Jesters\".\n" +
"- Reject antonyms or reversals such as changing \"visible\" to \"invisible\".\n" +
"- Do not add or remove punctuation, alter capitalization only, normalize spacing, remove filler words, collapse repetitions, or make general readability edits.\n" +
"- Treat glossary names and aliases as protected spellings and context.\n" +
"- You may correct toward glossary names, aliases, or their plural forms when the correction is acoustically plausible and supported by local context.\n" +
"- Do not replace, Anglicize, normalize, lowercase, or otherwise alter protected glossary names or aliases that already appear correctly in the transcript.\n" +
"- Preserve canonical glossary capitalization for protected names and aliases, even if they look unusual.\n" +
"- If a segment includes categories, treat them as additional transcript context.\n" +
"- Use the exact id from the input segment.\n" +
"- For returned corrections, original_text must be only the exact text span that needs replacement, not the full segment text unless the whole segment is the replacement span.\n" +
"- Choose an original_text span that appears exactly once in the current segment text.\n" +
"- corrected_text must be only the replacement text for that span, not the full corrected segment text unless the whole segment is the replacement span.\n" +
"- Each returned correction must contain only id, original_text, corrected_text, and confidence.\n" +
"- Do not return corrections where original_text and corrected_text are identical.\n" +
"- Do not return speaker, start, or end fields.\n" +
"- Return only changed segments; do not return entries for unchanged segments.\n" +
"- confidence must be between 0.0 and 1.0.\n" +
"- If no corrections are needed, return an empty corrections list.\n\n" +
fmt.Sprintf("Protected glossary/context:\n%s\n\nTranscript section:\n%s", string(glossaryJSON), string(sectionJSON))
return []contracts.LLMMessage{
{Role: "system", Content: system},
{Role: "user", Content: user},
}, nil
}