Complete Phase 12 grammar module

docs/architecture.md

@@ -33,9 +33,11 @@ Implemented today:
 - Deterministic proposal-index assignment and enriched proposal mapping for shared generation.
 - Proposal-generation diagnostics artifacts with secret redaction.
 - Production module registry scaffolding with known-key recognition and explicit unsupported/unimplemented errors.
+- Production `grammar` module implementation in `internal/modules/grammar`.
+- Explicit runtime support for `--modules grammar` through the production runner path.
 
 Not implemented in CLI runtime path today:
-- Real module execution pipeline (`glossary`, `homophones`, `spoken_word`, `grammar`).
+- Real module execution pipeline for `glossary`, `homophones`, and `spoken_word`.
 - Real domain proposal prompts for production modules.
 - End-to-end transcript polishing with real module behavior.
 
@@ -43,7 +45,8 @@ Phase sequencing note:
 - Phase 9 LLM infrastructure is complete (structured client, scheduler, effective config resolution, diagnostics primitives);
 - Phase 10 LLM-backed validator runtime integration is complete;
 - Phase 11 shared proposal-generation framework and module-registry scaffolding are complete;
-- next recommended phase is Phase 12 (grammar module).
+- Phase 12 grammar module implementation and explicit runtime wiring are complete;
+- next recommended phase is Phase 13 (glossary module and protected-term behavior).
 
 ## Actual Go package layout
 
@@ -102,6 +105,10 @@ internal/framework/proposal_generation/
 internal/framework/modules/
   registry.go
 
+internal/modules/grammar/
+  module.go
+  prompt.go
+
 internal/framework/validators/
   models.go
   deterministic.go
@@ -136,16 +143,19 @@ Current runtime flow (`internal/cli/run.go`):
 9. Write normalized transcript and normalization summary artifacts.
 10. Chunk normalized transcript and compute chunk summaries.
 11. Write chunking summary artifact.
-12. Optionally execute runner modules sequentially when a module factory is injected (tests currently use this path; production defaults still avoid real module execution).
+12. Execute runner modules sequentially when:
+    - `--modules` is explicitly provided (production grammar path); or
+    - a test/injected module factory is provided.
 13. Output working transcript to `--output` file or stdout.
-14. Build process report (`phase` currently set to `phase11-proposal-generation-framework`).
+14. Build process report (`phase` currently set to `phase12-grammar-module`).
 15. Optionally write `--report-json`; always write run-dir `report.json`.
 16. Apply work-dir retention.
 
 Important behavior details:
 - Glossary is validated but not yet used for real correction module logic.
-- Default production CLI behavior remains deterministic normalization/chunking output because no real module implementations are registered yet.
-- No real LLM calls occur in the default production runtime path because no real modules are registered yet.
+- Default production CLI behavior remains deterministic normalization/chunking/reporting unless modules are explicitly selected with `--modules`.
+- Explicit `--modules grammar` runs the production grammar module path with LLM-backed proposal generation and validator-chain execution.
+- Default runs (without explicit module selection) do not perform LLM calls.
 - Success path is generally quiet on stderr.
 - Source IDs are preserved into a canonical transcript before normalization; normalization then reassigns output IDs sequentially from `1`.
 
@@ -193,7 +203,7 @@ Implemented config surfaces include:
 - work-dir and retention mode
 
 Current caveat:
-- LLM/module-related settings are mostly infrastructure-only today; default runtime path does not execute real modules.
+- LLM/module-related settings are active for explicit grammar runs; the default non-explicit path remains deterministic.
 
 ## Implemented structured LLM infrastructure
 `internal/framework/contracts` now defines a typed structured-completion contract:
@@ -209,8 +219,8 @@ Current caveat:
 - API-key redaction in adapter-returned errors.
 
 Current runtime boundary:
-- the default CLI runtime path still does not instantiate real production modules, so no default end-to-end LLM polishing occurs.
-- LLM calls are exercised only when test/injected modules and validators are provided.
+- the default CLI runtime path (without explicit module selection) still does not instantiate the full production module sequence.
+- LLM calls are exercised in production when `--modules grammar` is explicitly requested and in tests when fake/injected clients are used.
 
 `internal/framework/llm` also provides:
 - a bounded `Scheduler` for controlled concurrent LLM calls with reliable permit release;
@@ -252,7 +262,7 @@ Current behavior details:
 
 `internal/framework/contracts` provides interfaces and run-spec metadata scaffolding, including deterministic repeated module instance naming (`ResolveModuleRunSpecs`).
 
-These primitives are wired into the production runner and report model. Real module implementations are still pending.
+These primitives are wired into the production runner and report model. The grammar module is implemented; other production modules remain pending.
 
 ## Implemented validator runtime infrastructure
 `internal/framework/validators` provides deterministic validator infrastructure:
@@ -317,7 +327,18 @@ This helper only produces candidate proposals; validator-chain execution and pro
   - diagnostics directory context
 - returns explicit errors for unknown keys (`unsupported_module`) and recognized-but-unimplemented keys (`unimplemented_module`).
 
-No real production correction modules are registered yet.
+The `grammar` module key is now registered and constructible. `glossary`, `homophones`, and `spoken_word` remain recognized-but-unimplemented.
+
+## Implemented grammar production module
+`internal/modules/grammar` now provides the first production module:
+- prompt builder faithfully constrained to punctuation/capitalization/spacing/article cleanup;
+- explicit guardrails against meaning-changing rewrites, style rewrites, summarization, and invention;
+- proposal generation through `internal/framework/proposal_generation` and `contracts.StructuredLLMClient`;
+- scheduler-aware proposal calls through existing `contracts.LLMScheduler` hooks;
+- replacement policy `require_unique` (matching Python implementation);
+- validator chain integration using existing deterministic + LLM-backed validators;
+- grammar confidence threshold enforcement through existing validator/config infrastructure;
+- module-level reporting and diagnostics capture through existing runner/reporting paths.
 
 ## Reports and diagnostics (implemented)
 Current per-run artifacts include:
@@ -357,7 +378,8 @@ Retention modes implemented in `ApplyRetention`:
 - failed runs are always retained.
 
 Current runtime note:
-- real module execution is not implemented yet, so normal successful runs generally have no skipped corrections and `auto` typically removes clean successful run directories.
+- default non-explicit runs usually have no module-level skipped corrections, so `auto` commonly removes clean successful run directories.
+- explicit grammar runs can produce validator rejections and application skips, which are reflected in reports and retention input.
 
 Intentionally deferred to module/LLM phases:
 - real domain proposal prompts and production module implementations remain tied to later module phases.
@@ -378,8 +400,9 @@ Implemented tests currently cover:
 - LLM-backed validator batching, prompt builders, structured-response safety, scheduler hooks, and diagnostics redaction (`internal/framework/validators/*_test.go`, `internal/framework/runner/*_test.go`)
 - shared proposal-generation request/response parsing, deterministic indexing, scheduler hooks, and diagnostics redaction (`internal/framework/proposal_generation/*_test.go`, `internal/framework/runner/*_test.go`)
 - production module-registry known-key recognition and unsupported/unimplemented error behavior (`internal/framework/modules/*_test.go`, `internal/cli/run_test.go`)
+- production grammar module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, and explicit CLI/runtime integration (`internal/modules/grammar/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
 
-Not covered yet (because not implemented): real production module implementations and full transcript-polishing runtime behavior.
+Not covered yet (because not implemented): production `glossary`, `homophones`, and `spoken_word` modules plus full default-sequence transcript-polishing runtime behavior.
 
 ## Intended final architecture (not yet implemented)
 The intended end-state still matches the rewrite plan: