Complete Phase 15 spoken-word module
This commit is contained in:
@@ -36,14 +36,18 @@ Implemented today:
|
||||
- Production `grammar` module implementation in `internal/modules/grammar`.
|
||||
- Production `glossary` module implementation in `internal/modules/glossary`.
|
||||
- Production `homophones` module implementation in `internal/modules/homophones`.
|
||||
- Production `spoken_word` module implementation in `internal/modules/spoken_word`.
|
||||
- Explicit runtime support for `--modules grammar` through the production runner path.
|
||||
- Explicit runtime support for `--modules glossary`, including repeated stages such as `--modules glossary,glossary`.
|
||||
- Explicit runtime support for `--modules homophones` through the production runner path.
|
||||
- Explicit runtime support for `--modules spoken_word` through the production runner path.
|
||||
|
||||
Not implemented in CLI runtime path today:
|
||||
- Real module execution pipeline for `spoken_word`.
|
||||
- End-to-end transcript polishing with the full default module sequence.
|
||||
|
||||
Current reality:
|
||||
- all production modules exist and are explicitly runnable by `--modules`, but default full-sequence integration remains a later phase.
|
||||
|
||||
Phase sequencing note:
|
||||
- Phase 9 LLM infrastructure is complete (structured client, scheduler, effective config resolution, diagnostics primitives);
|
||||
- Phase 10 LLM-backed validator runtime integration is complete;
|
||||
@@ -51,7 +55,8 @@ Phase sequencing note:
|
||||
- Phase 12 grammar module implementation and explicit runtime wiring are complete;
|
||||
- Phase 13 glossary module and protected-term behavior are complete;
|
||||
- Phase 14 homophones module implementation and explicit runtime wiring are complete;
|
||||
- next recommended phase is Phase 15 (spoken-word module).
|
||||
- Phase 15 spoken-word module implementation and explicit runtime wiring are complete;
|
||||
- next recommended phase is Phase 16 (default full pipeline integration).
|
||||
|
||||
## Actual Go package layout
|
||||
|
||||
@@ -122,6 +127,10 @@ internal/modules/homophones/
|
||||
module.go
|
||||
prompt.go
|
||||
|
||||
internal/modules/spoken_word/
|
||||
module.go
|
||||
prompt.go
|
||||
|
||||
internal/framework/validators/
|
||||
models.go
|
||||
deterministic.go
|
||||
@@ -157,17 +166,17 @@ Current runtime flow (`internal/cli/run.go`):
|
||||
10. Chunk normalized transcript and compute chunk summaries.
|
||||
11. Write chunking summary artifact.
|
||||
12. Execute runner modules sequentially when:
|
||||
- `--modules` is explicitly provided (production grammar/glossary/homophones paths); or
|
||||
- `--modules` is explicitly provided (production grammar/glossary/homophones/spoken_word paths); or
|
||||
- a test/injected module factory is provided.
|
||||
13. Output working transcript to `--output` file or stdout.
|
||||
14. Build process report (`phase` currently set to `phase14-homophones-module`).
|
||||
14. Build process report (`phase` currently set to `phase15-spoken-word-module`).
|
||||
15. Optionally write `--report-json`; always write run-dir `report.json`.
|
||||
16. Apply work-dir retention.
|
||||
|
||||
Important behavior details:
|
||||
- Glossary is validated and is used for explicit glossary/grammar/homophones module correction paths.
|
||||
- Glossary is validated and is used for explicit glossary/grammar/homophones/spoken_word module correction paths.
|
||||
- Default production CLI behavior remains deterministic normalization/chunking/reporting unless modules are explicitly selected with `--modules`.
|
||||
- Explicit `--modules grammar`, `--modules glossary`, and `--modules homophones` run production module paths with LLM-backed proposal generation and validator-chain execution.
|
||||
- Explicit `--modules grammar`, `--modules glossary`, `--modules homophones`, and `--modules spoken_word` run production module paths with LLM-backed proposal generation and validator-chain execution.
|
||||
- Default runs (without explicit module selection) do not perform LLM calls.
|
||||
- Success path is generally quiet on stderr.
|
||||
- Source IDs are preserved into a canonical transcript before normalization; normalization then reassigns output IDs sequentially from `1`.
|
||||
@@ -216,7 +225,7 @@ Implemented config surfaces include:
|
||||
- work-dir and retention mode
|
||||
|
||||
Current caveat:
|
||||
- LLM/module-related settings are active for explicit grammar/glossary/homophones runs; the default non-explicit path remains deterministic.
|
||||
- LLM/module-related settings are active for explicit grammar/glossary/homophones/spoken_word runs; the default non-explicit path remains deterministic.
|
||||
|
||||
## Implemented structured LLM infrastructure
|
||||
`internal/framework/contracts` now defines a typed structured-completion contract:
|
||||
@@ -233,7 +242,7 @@ Current caveat:
|
||||
|
||||
Current runtime boundary:
|
||||
- the default CLI runtime path (without explicit module selection) still does not instantiate the full production module sequence.
|
||||
- LLM calls are exercised in production when `--modules grammar`, `--modules glossary`, or `--modules homophones` is explicitly requested and in tests when fake/injected clients are used.
|
||||
- LLM calls are exercised in production when `--modules grammar`, `--modules glossary`, `--modules homophones`, or `--modules spoken_word` is explicitly requested and in tests when fake/injected clients are used.
|
||||
|
||||
`internal/framework/llm` also provides:
|
||||
- a bounded `Scheduler` for controlled concurrent LLM calls with reliable permit release;
|
||||
@@ -275,7 +284,7 @@ Current behavior details:
|
||||
|
||||
`internal/framework/contracts` provides interfaces and run-spec metadata scaffolding, including deterministic repeated module instance naming (`ResolveModuleRunSpecs`).
|
||||
|
||||
These primitives are wired into the production runner and report model. The grammar and glossary modules are implemented; other production modules remain pending.
|
||||
These primitives are wired into the production runner and report model. The grammar, glossary, homophones, and spoken_word modules are implemented.
|
||||
|
||||
## Implemented validator runtime infrastructure
|
||||
`internal/framework/validators` provides deterministic validator infrastructure:
|
||||
@@ -340,7 +349,7 @@ This helper only produces candidate proposals; validator-chain execution and pro
|
||||
- diagnostics directory context
|
||||
- returns explicit errors for unknown keys (`unsupported_module`) and recognized-but-unimplemented keys (`unimplemented_module`).
|
||||
|
||||
The `grammar`, `glossary`, and `homophones` module keys are now registered and constructible. `spoken_word` remains recognized-but-unimplemented.
|
||||
The `grammar`, `glossary`, `homophones`, and `spoken_word` module keys are now registered and constructible.
|
||||
|
||||
## Implemented grammar production module
|
||||
`internal/modules/grammar` now provides the first production module:
|
||||
@@ -387,6 +396,19 @@ This vocabulary is used by deterministic validators for both glossary-stage and
|
||||
- protected-term guardrails for non-glossary modules remain active and are exercised through the homophones path;
|
||||
- module-level reporting and diagnostics capture through existing runner/reporting paths.
|
||||
|
||||
## Implemented spoken_word production module
|
||||
`internal/modules/spoken_word` now provides the fourth production module:
|
||||
- prompt builder aligned to Python spoken_word-module intent, constrained to conservative dysfluency cleanup;
|
||||
- strong prompt guardrails preserving meaning/intent/voice/named entities/domain terms and substantive content;
|
||||
- explicit guardrails against summarization, style rewriting, grammar-only cleanup, punctuation-only cleanup, invention, and meaning-changing rewrites;
|
||||
- proposal generation through `internal/framework/proposal_generation` and `contracts.StructuredLLMClient`;
|
||||
- scheduler-aware proposal calls through existing `contracts.LLMScheduler` hooks;
|
||||
- replacement policy `require_unique` (matching Python spoken_word behavior);
|
||||
- validator chain integration using existing deterministic + LLM-backed validators, including strong semantic guardrails (`spoken_word_review`, `meaning_reversal_review`);
|
||||
- spoken_word confidence threshold enforcement through existing validator/config infrastructure;
|
||||
- protected-term guardrails for non-glossary modules remain active and are exercised through the spoken_word path;
|
||||
- module-level reporting and diagnostics capture through existing runner/reporting paths.
|
||||
|
||||
## Reports and diagnostics (implemented)
|
||||
Current per-run artifacts include:
|
||||
- `source-transcript.json`
|
||||
@@ -426,7 +448,7 @@ Retention modes implemented in `ApplyRetention`:
|
||||
|
||||
Current runtime note:
|
||||
- default non-explicit runs usually have no module-level skipped corrections, so `auto` commonly removes clean successful run directories.
|
||||
- explicit grammar/glossary/homophones runs can produce validator rejections and application skips, which are reflected in reports and retention input.
|
||||
- explicit grammar/glossary/homophones/spoken_word runs can produce validator rejections and application skips, which are reflected in reports and retention input.
|
||||
|
||||
Intentionally deferred to module/LLM phases:
|
||||
- real domain proposal prompts and production module implementations remain tied to later module phases.
|
||||
@@ -450,14 +472,15 @@ Implemented tests currently cover:
|
||||
- production grammar module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, and explicit CLI/runtime integration (`internal/modules/grammar/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
|
||||
- production glossary module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, repeated-stage behavior, and explicit CLI/runtime integration (`internal/modules/glossary/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
|
||||
- production homophones module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, protected-term behavior, and explicit CLI/runtime integration (`internal/modules/homophones/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
|
||||
- production spoken_word module prompt constraints, proposal mapping, validator-chain behavior, semantic guardrail behavior, confidence-threshold enforcement, diagnostics redaction, protected-term behavior, and explicit CLI/runtime integration (`internal/modules/spoken_word/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
|
||||
- glossary-derived protected-term extraction and stable behavior (`internal/framework/validators/protected_terms_test.go`)
|
||||
|
||||
Not covered yet (because not implemented): production `spoken_word` module plus full default-sequence transcript-polishing runtime behavior.
|
||||
Not covered yet (because not implemented): full default-sequence transcript-polishing runtime behavior as a single default path.
|
||||
|
||||
## Intended final architecture (not yet implemented)
|
||||
The intended end-state still matches the rewrite plan:
|
||||
- sequential module pipeline over a mutable working transcript
|
||||
- real module implementation (`spoken_word`)
|
||||
- default full module-sequence integration in the standard runtime path
|
||||
- structured LLM proposal generation
|
||||
- deterministic and LLM validators
|
||||
- validator cardinality enforcement in pipeline execution
|
||||
|
||||
@@ -68,6 +68,10 @@ Implemented:
|
||||
- Production homophones module package with Python-aligned prompt intent and guardrails.
|
||||
- Explicit `--modules homophones` runtime path through runner, shared proposal generation, validators, application, reporting, and diagnostics.
|
||||
- Focused multi-module runtime tests for already-implemented interoperability (for example `glossary,homophones`) without claiming full default-pipeline completion.
|
||||
- Production spoken_word module package with Python-aligned prompt intent and guardrails.
|
||||
- Explicit `--modules spoken_word` runtime path through runner, shared proposal generation, validators, application, reporting, and diagnostics.
|
||||
- Focused multi-module runtime tests for already-implemented interoperability (for example `spoken_word,grammar`) without claiming full default-pipeline completion.
|
||||
- All production modules now exist (`glossary`, `homophones`, `spoken_word`, `grammar`), but default full-sequence integration remains Phase 16 work.
|
||||
- Broad deterministic and CLI/subprocess test coverage for implemented phases through `go test ./...`.
|
||||
- Internal typed structured LLM contract (`StructuredLLMClient.CompleteStructured(ctx, req, out)`).
|
||||
- `internal/framework/llm` instructor-go-backed adapter with:
|
||||
@@ -83,8 +87,6 @@ Implemented:
|
||||
- Generic JSON diagnostics primitives for LLM interactions (request metadata, request payload, response payload, optional error payload) with secret redaction.
|
||||
|
||||
Not yet implemented in runtime pipeline:
|
||||
- Real correction module for `spoken_word`.
|
||||
- Domain proposal prompts for remaining real module.
|
||||
- End-to-end transcript polishing behavior with the full default module sequence.
|
||||
|
||||
## Completed phases
|
||||
@@ -223,7 +225,7 @@ Not implemented in Phase 8 (by design):
|
||||
|
||||
## Remaining work plan
|
||||
|
||||
Next recommended phase: **Phase 15 (spoken-word module)**.
|
||||
Next recommended phase: **Phase 16 (default full pipeline integration)**.
|
||||
|
||||
## Phase 9: Structured LLM client and scheduler infrastructure
|
||||
|
||||
@@ -418,41 +420,29 @@ Not implemented in Phase 14 (by design):
|
||||
|
||||
## Phase 15: Spoken-word module
|
||||
|
||||
### Purpose
|
||||
Completed.
|
||||
|
||||
Implement conservative dysfluency cleanup while preserving substantive meaning.
|
||||
Implemented:
|
||||
- Production spoken_word module package in `internal/modules/spoken_word`.
|
||||
- Spoken_word prompt builder aligned to Python intent and constrained to conservative dysfluency cleanup.
|
||||
- Prompt context using glossary/protected-term information (names, aliases, plurals where present) to avoid damaging known domain terms.
|
||||
- Strong prompt guardrails preserving meaning, intent, speaker voice, named entities, game/domain terms, and substantive content.
|
||||
- Explicit prompt guardrails against summarization, style rewriting, grammar-only cleanup, punctuation-only cleanup, invention, event reordering, and certainty inflation.
|
||||
- Spoken_word proposal generation through shared `internal/framework/proposal_generation` using `contracts.StructuredLLMClient`.
|
||||
- Scheduler-aware spoken_word proposal generation through existing scheduler hooks.
|
||||
- Spoken_word replacement policy `require_unique` (matching Python behavior).
|
||||
- Spoken_word validator chain using existing deterministic and LLM-backed validators.
|
||||
- Strong semantic guardrails in runtime validator chain through existing LLM-backed validators (`spoken_word_review`, `meaning_reversal_review`).
|
||||
- Spoken-word confidence threshold enforcement through existing config + confidence-threshold validator behavior.
|
||||
- Protected-term guardrails remaining active for spoken_word via existing deterministic validators.
|
||||
- Explicit runtime support for `--modules spoken_word` through normalization, chunking, runner, proposal generation, validation, application, and reporting.
|
||||
- Prompt/response diagnostics artifacts for spoken_word proposal + validator interactions with secret redaction.
|
||||
- Module-level reports for spoken_word including generated proposals, validator decisions/rejections, applied changes, and application skips.
|
||||
- CLI/runtime fake-client tests for approved cleanup, validator rejection, meaning-changing rejection, application skips, diagnostics, protected-term rejection behavior, failure/error.log behavior, and report outputs (`--report-json` and run-dir `report.json`).
|
||||
- Focused interoperability tests for already-implemented module combinations (for example `spoken_word,grammar`) to verify working-transcript handoff and guardrails without claiming full default-sequence parity.
|
||||
|
||||
### Scope
|
||||
|
||||
Implement:
|
||||
- `spoken_word` module package.
|
||||
- Spoken-word prompt builder ported from Python.
|
||||
- Spoken-word structured response model.
|
||||
- Spoken-word replacement policy.
|
||||
- Spoken-word confidence threshold handling.
|
||||
- Spoken-word validator chain.
|
||||
- Strong semantic guardrails using LLM-backed validators.
|
||||
- Prompt/response diagnostics.
|
||||
- CLI support for `--modules spoken_word`.
|
||||
- Fake LLM tests.
|
||||
- Tests for rejection of meaning-changing cleanup.
|
||||
|
||||
Do not implement:
|
||||
- Prompt redesign beyond faithful porting.
|
||||
- New stylistic rewriting behavior not present in the Python implementation.
|
||||
|
||||
### Expected behavior at end of phase
|
||||
|
||||
Running `audita process ... --modules spoken_word` should perform real conservative dysfluency cleanup, with guardrails against semantic changes.
|
||||
|
||||
### Definition of done
|
||||
|
||||
- Spoken-word module runs in the production runner.
|
||||
- Spoken-word proposals are generated through structured LLM calls.
|
||||
- Semantic validators reject meaning-changing proposals.
|
||||
- Applied/skipped spoken-word changes appear in reports.
|
||||
- Prompt/response diagnostics are written.
|
||||
- `go test ./...` passes without requiring external LLM credentials.
|
||||
Not implemented in Phase 15 (by design):
|
||||
- Full default module sequence execution as a feature-complete claim.
|
||||
|
||||
## Phase 16: Default full pipeline integration
|
||||
|
||||
|
||||
Reference in New Issue
Block a user