Complete Phase 12 grammar module
This commit is contained in:
@@ -33,9 +33,11 @@ Implemented today:
|
||||
- Deterministic proposal-index assignment and enriched proposal mapping for shared generation.
|
||||
- Proposal-generation diagnostics artifacts with secret redaction.
|
||||
- Production module registry scaffolding with known-key recognition and explicit unsupported/unimplemented errors.
|
||||
- Production `grammar` module implementation in `internal/modules/grammar`.
|
||||
- Explicit runtime support for `--modules grammar` through the production runner path.
|
||||
|
||||
Not implemented in CLI runtime path today:
|
||||
- Real module execution pipeline (`glossary`, `homophones`, `spoken_word`, `grammar`).
|
||||
- Real module execution pipeline for `glossary`, `homophones`, and `spoken_word`.
|
||||
- Real domain proposal prompts for production modules.
|
||||
- End-to-end transcript polishing with real module behavior.
|
||||
|
||||
@@ -43,7 +45,8 @@ Phase sequencing note:
|
||||
- Phase 9 LLM infrastructure is complete (structured client, scheduler, effective config resolution, diagnostics primitives);
|
||||
- Phase 10 LLM-backed validator runtime integration is complete;
|
||||
- Phase 11 shared proposal-generation framework and module-registry scaffolding are complete;
|
||||
- next recommended phase is Phase 12 (grammar module).
|
||||
- Phase 12 grammar module implementation and explicit runtime wiring are complete;
|
||||
- next recommended phase is Phase 13 (glossary module and protected-term behavior).
|
||||
|
||||
## Actual Go package layout
|
||||
|
||||
@@ -102,6 +105,10 @@ internal/framework/proposal_generation/
|
||||
internal/framework/modules/
|
||||
registry.go
|
||||
|
||||
internal/modules/grammar/
|
||||
module.go
|
||||
prompt.go
|
||||
|
||||
internal/framework/validators/
|
||||
models.go
|
||||
deterministic.go
|
||||
@@ -136,16 +143,19 @@ Current runtime flow (`internal/cli/run.go`):
|
||||
9. Write normalized transcript and normalization summary artifacts.
|
||||
10. Chunk normalized transcript and compute chunk summaries.
|
||||
11. Write chunking summary artifact.
|
||||
12. Optionally execute runner modules sequentially when a module factory is injected (tests currently use this path; production defaults still avoid real module execution).
|
||||
12. Execute runner modules sequentially when:
|
||||
- `--modules` is explicitly provided (production grammar path); or
|
||||
- a test/injected module factory is provided.
|
||||
13. Output working transcript to `--output` file or stdout.
|
||||
14. Build process report (`phase` currently set to `phase11-proposal-generation-framework`).
|
||||
14. Build process report (`phase` currently set to `phase12-grammar-module`).
|
||||
15. Optionally write `--report-json`; always write run-dir `report.json`.
|
||||
16. Apply work-dir retention.
|
||||
|
||||
Important behavior details:
|
||||
- Glossary is validated but not yet used for real correction module logic.
|
||||
- Default production CLI behavior remains deterministic normalization/chunking output because no real module implementations are registered yet.
|
||||
- No real LLM calls occur in the default production runtime path because no real modules are registered yet.
|
||||
- Default production CLI behavior remains deterministic normalization/chunking/reporting unless modules are explicitly selected with `--modules`.
|
||||
- Explicit `--modules grammar` runs the production grammar module path with LLM-backed proposal generation and validator-chain execution.
|
||||
- Default runs (without explicit module selection) do not perform LLM calls.
|
||||
- Success path is generally quiet on stderr.
|
||||
- Source IDs are preserved into a canonical transcript before normalization; normalization then reassigns output IDs sequentially from `1`.
|
||||
|
||||
@@ -193,7 +203,7 @@ Implemented config surfaces include:
|
||||
- work-dir and retention mode
|
||||
|
||||
Current caveat:
|
||||
- LLM/module-related settings are mostly infrastructure-only today; default runtime path does not execute real modules.
|
||||
- LLM/module-related settings are active for explicit grammar runs; the default non-explicit path remains deterministic.
|
||||
|
||||
## Implemented structured LLM infrastructure
|
||||
`internal/framework/contracts` now defines a typed structured-completion contract:
|
||||
@@ -209,8 +219,8 @@ Current caveat:
|
||||
- API-key redaction in adapter-returned errors.
|
||||
|
||||
Current runtime boundary:
|
||||
- the default CLI runtime path still does not instantiate real production modules, so no default end-to-end LLM polishing occurs.
|
||||
- LLM calls are exercised only when test/injected modules and validators are provided.
|
||||
- the default CLI runtime path (without explicit module selection) still does not instantiate the full production module sequence.
|
||||
- LLM calls are exercised in production when `--modules grammar` is explicitly requested and in tests when fake/injected clients are used.
|
||||
|
||||
`internal/framework/llm` also provides:
|
||||
- a bounded `Scheduler` for controlled concurrent LLM calls with reliable permit release;
|
||||
@@ -252,7 +262,7 @@ Current behavior details:
|
||||
|
||||
`internal/framework/contracts` provides interfaces and run-spec metadata scaffolding, including deterministic repeated module instance naming (`ResolveModuleRunSpecs`).
|
||||
|
||||
These primitives are wired into the production runner and report model. Real module implementations are still pending.
|
||||
These primitives are wired into the production runner and report model. The grammar module is implemented; other production modules remain pending.
|
||||
|
||||
## Implemented validator runtime infrastructure
|
||||
`internal/framework/validators` provides deterministic validator infrastructure:
|
||||
@@ -317,7 +327,18 @@ This helper only produces candidate proposals; validator-chain execution and pro
|
||||
- diagnostics directory context
|
||||
- returns explicit errors for unknown keys (`unsupported_module`) and recognized-but-unimplemented keys (`unimplemented_module`).
|
||||
|
||||
No real production correction modules are registered yet.
|
||||
The `grammar` module key is now registered and constructible. `glossary`, `homophones`, and `spoken_word` remain recognized-but-unimplemented.
|
||||
|
||||
## Implemented grammar production module
|
||||
`internal/modules/grammar` now provides the first production module:
|
||||
- prompt builder faithfully constrained to punctuation/capitalization/spacing/article cleanup;
|
||||
- explicit guardrails against meaning-changing rewrites, style rewrites, summarization, and invention;
|
||||
- proposal generation through `internal/framework/proposal_generation` and `contracts.StructuredLLMClient`;
|
||||
- scheduler-aware proposal calls through existing `contracts.LLMScheduler` hooks;
|
||||
- replacement policy `require_unique` (matching Python implementation);
|
||||
- validator chain integration using existing deterministic + LLM-backed validators;
|
||||
- grammar confidence threshold enforcement through existing validator/config infrastructure;
|
||||
- module-level reporting and diagnostics capture through existing runner/reporting paths.
|
||||
|
||||
## Reports and diagnostics (implemented)
|
||||
Current per-run artifacts include:
|
||||
@@ -357,7 +378,8 @@ Retention modes implemented in `ApplyRetention`:
|
||||
- failed runs are always retained.
|
||||
|
||||
Current runtime note:
|
||||
- real module execution is not implemented yet, so normal successful runs generally have no skipped corrections and `auto` typically removes clean successful run directories.
|
||||
- default non-explicit runs usually have no module-level skipped corrections, so `auto` commonly removes clean successful run directories.
|
||||
- explicit grammar runs can produce validator rejections and application skips, which are reflected in reports and retention input.
|
||||
|
||||
Intentionally deferred to module/LLM phases:
|
||||
- real domain proposal prompts and production module implementations remain tied to later module phases.
|
||||
@@ -378,8 +400,9 @@ Implemented tests currently cover:
|
||||
- LLM-backed validator batching, prompt builders, structured-response safety, scheduler hooks, and diagnostics redaction (`internal/framework/validators/*_test.go`, `internal/framework/runner/*_test.go`)
|
||||
- shared proposal-generation request/response parsing, deterministic indexing, scheduler hooks, and diagnostics redaction (`internal/framework/proposal_generation/*_test.go`, `internal/framework/runner/*_test.go`)
|
||||
- production module-registry known-key recognition and unsupported/unimplemented error behavior (`internal/framework/modules/*_test.go`, `internal/cli/run_test.go`)
|
||||
- production grammar module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, and explicit CLI/runtime integration (`internal/modules/grammar/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
|
||||
|
||||
Not covered yet (because not implemented): real production module implementations and full transcript-polishing runtime behavior.
|
||||
Not covered yet (because not implemented): production `glossary`, `homophones`, and `spoken_word` modules plus full default-sequence transcript-polishing runtime behavior.
|
||||
|
||||
## Intended final architecture (not yet implemented)
|
||||
The intended end-state still matches the rewrite plan:
|
||||
|
||||
@@ -59,6 +59,8 @@ Implemented:
|
||||
- Deterministic proposal-index assignment for shared proposal generation.
|
||||
- Proposal-generation prompt/response diagnostics artifact wiring with secret redaction.
|
||||
- Production module-registry scaffolding with known key recognition and explicit unsupported/unimplemented errors.
|
||||
- Production grammar module package with Python-aligned prompt intent and guardrails.
|
||||
- Explicit `--modules grammar` runtime path through runner, shared proposal generation, validators, application, reporting, and diagnostics.
|
||||
- Broad deterministic and CLI/subprocess test coverage for implemented phases through `go test ./...`.
|
||||
- Internal typed structured LLM contract (`StructuredLLMClient.CompleteStructured(ctx, req, out)`).
|
||||
- `internal/framework/llm` instructor-go-backed adapter with:
|
||||
@@ -74,8 +76,8 @@ Implemented:
|
||||
- Generic JSON diagnostics primitives for LLM interactions (request metadata, request payload, response payload, optional error payload) with secret redaction.
|
||||
|
||||
Not yet implemented in runtime pipeline:
|
||||
- Real correction modules.
|
||||
- Domain proposal prompts for real modules.
|
||||
- Real correction modules for `glossary`, `homophones`, and `spoken_word`.
|
||||
- Domain proposal prompts for remaining real modules.
|
||||
- End-to-end transcript polishing behavior.
|
||||
|
||||
## Completed phases
|
||||
@@ -214,7 +216,7 @@ Not implemented in Phase 8 (by design):
|
||||
|
||||
## Remaining work plan
|
||||
|
||||
Next recommended phase: **Phase 12 (grammar module)**.
|
||||
Next recommended phase: **Phase 13 (glossary module and protected-term behavior)**.
|
||||
|
||||
## Phase 9: Structured LLM client and scheduler infrastructure
|
||||
|
||||
@@ -333,52 +335,30 @@ Implemented:
|
||||
- Runner/CLI injection-path tests showing shared proposal generation can flow through runner validation/application semantics using fake modules/clients.
|
||||
|
||||
Not implemented in Phase 11 (by design):
|
||||
- Real production `glossary`, `homophones`, `spoken_word`, and `grammar` modules.
|
||||
- Real production `glossary`, `homophones`, and `spoken_word` modules.
|
||||
- Domain proposal prompts for production modules.
|
||||
- Default CLI end-to-end transcript polishing behavior.
|
||||
|
||||
## Phase 12: Grammar module
|
||||
|
||||
### Purpose
|
||||
Completed.
|
||||
|
||||
Implement the first production module in the Go runtime path. Grammar is a good first real module because it exercises LLM proposal generation and validator chains while remaining constrained to punctuation, capitalization, and spacing cleanup.
|
||||
Implemented:
|
||||
- Production grammar module package in `internal/modules/grammar`.
|
||||
- Grammar prompt builder aligned to Python intent and constrained to punctuation/capitalization/spacing/article cleanup.
|
||||
- Grammar proposal generation through shared `internal/framework/proposal_generation` using `contracts.StructuredLLMClient`.
|
||||
- Scheduler-aware grammar proposal generation through existing scheduler hooks.
|
||||
- Grammar replacement policy `require_unique` (matching Python behavior).
|
||||
- Grammar validator chain using existing deterministic and LLM-backed validator infrastructure.
|
||||
- Grammar confidence threshold enforcement through existing config + confidence-threshold validator behavior.
|
||||
- Explicit runtime support for `--modules grammar` through normalization, chunking, runner, proposal generation, validation, application, and reporting.
|
||||
- Prompt/response diagnostics artifacts for grammar proposal + validator interactions with secret redaction.
|
||||
- Module-level reports for grammar including validator decisions/rejections, applied changes, and skipped changes.
|
||||
- CLI/runtime fake-client tests for approved proposals, validator rejection, application skips, diagnostics, failure/error.log behavior, and report outputs (`--report-json` and run-dir `report.json`).
|
||||
|
||||
### Scope
|
||||
|
||||
Implement:
|
||||
- `grammar` module package.
|
||||
- Grammar prompt builder ported from the Python implementation.
|
||||
- Grammar structured response model.
|
||||
- Grammar replacement policy.
|
||||
- Grammar validator chain.
|
||||
- Grammar confidence threshold handling.
|
||||
- Prompt/response diagnostics.
|
||||
- Module-level report integration.
|
||||
- CLI support for `--modules grammar`.
|
||||
- Fake LLM tests.
|
||||
- Optional real LLM smoke test gated so normal `go test ./...` does not require credentials.
|
||||
|
||||
Do not implement:
|
||||
- Glossary module.
|
||||
- Homophones module.
|
||||
- Spoken-word module.
|
||||
- Default full module sequence as active parity claim.
|
||||
|
||||
### Expected behavior at end of phase
|
||||
|
||||
Running `audita process ... --modules grammar` should perform real grammar-stage transcript polishing using the configured LLM endpoint.
|
||||
|
||||
### Definition of done
|
||||
|
||||
- Grammar module runs in the production runner.
|
||||
- Grammar module generates structured proposals through the LLM client.
|
||||
- Grammar validator chain runs.
|
||||
- Approved grammar proposals are applied.
|
||||
- Applied/skipped grammar changes appear in reports.
|
||||
- Prompt/response diagnostics are written.
|
||||
- `--modules grammar` works end-to-end.
|
||||
- Default pipeline is not yet claimed complete.
|
||||
- `go test ./...` passes without requiring external LLM credentials.
|
||||
Not implemented in Phase 12 (by design):
|
||||
- Production `glossary`, `homophones`, and `spoken_word` modules.
|
||||
- Full default module sequence execution as a feature-complete claim.
|
||||
|
||||
## Phase 13: Glossary module and protected-term behavior
|
||||
|
||||
|
||||
Reference in New Issue
Block a user