From 6d9a4bd017799d0e94a2369595afbec5d4644b6d Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 11 May 2026 20:10:56 -0500 Subject: [PATCH] Complete Phase 9 LLM infrastructure --- docs/architecture.md | 6 +-- docs/rewrite-notes.md | 45 ++++++++++++++------- internal/framework/llm/instructor_client.go | 2 - 3 files changed, 34 insertions(+), 19 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 11a6bfb..537a7fa 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -31,12 +31,12 @@ Not implemented in CLI runtime path today: - Real module execution pipeline (`glossary`, `homophones`, `spoken_word`, `grammar`). - Structured LLM proposal generation. - LLM-backed validators. -- Production LLM scheduler behavior. +- Runtime module/validator usage of the LLM scheduler infrastructure. - End-to-end transcript polishing with real module behavior. Phase sequencing note: -- structured LLM client, scheduler, config-resolution helpers, and diagnostics primitives are implemented; -- runtime wiring from modules/validators/runner into this LLM infrastructure remains Phase 9 follow-up work; +- Phase 9 LLM infrastructure is complete (structured client, scheduler, effective config resolution, diagnostics primitives); +- runtime wiring from modules/validators/runner into this LLM infrastructure remains future module/validator phase work; - LLM-backed validators remain Phase 10 work. ## Actual Go package layout diff --git a/docs/rewrite-notes.md b/docs/rewrite-notes.md index f364250..18049e9 100644 --- a/docs/rewrite-notes.md +++ b/docs/rewrite-notes.md @@ -208,13 +208,13 @@ Not implemented in Phase 8 (by design): ## Remaining work plan -Next recommended phase: **Phase 9 follow-up (runtime LLM wiring, still no real modules)**. +Next recommended phase: **Phase 10 (LLM-backed validators)**. ## Phase 9: Structured LLM client and scheduler infrastructure ### Status -Partially completed. +Completed for Phase 9 infrastructure scope. Implemented in this phase so far: - Added internal structured LLM contract support for caller-provided typed outputs. @@ -222,7 +222,7 @@ Implemented in this phase so far: - Confirmed OpenAI-compatible base URL support through the adapter path. - Added adapter unit tests for model/base URL handling, retries, context cancellation, optional API key behavior, and error redaction. -Still pending in Phase 9: +Explicitly deferred from Phase 9 into later phases: - Runtime wiring in runner/module infrastructure (without introducing real modules yet). - Wiring prompt/response diagnostics primitives into future module/validator call sites. - Wiring effective primary/validation LLM config resolution into runtime LLM call sites. @@ -231,11 +231,25 @@ Still pending in Phase 9: Implement the provider-neutral LLM infrastructure needed by both proposal generation and LLM-backed validators, without yet implementing real modules. -### Scope +### Scope completed in this phase -Implement (remaining): -- Runtime wiring for primary/validation effective LLM config resolution. -- Runtime usage of diagnostics primitives with credential redaction. +Implemented: +- Provider-neutral internal structured LLM contract with caller-provided typed output decoding. +- OpenAI-compatible structured-output adapter (`instructor-go`) with: + - configurable base URL and model + - optional API key behavior + - retry budget + - timeout-aware HTTP client handling + - context cancellation propagation + - structured response decoding into caller-provided typed targets + - redacted error surfaces. +- Scheduler/semaphore infrastructure for bounded backend concurrency with context-aware acquisition and reliable permit release. +- Effective LLM config resolution helpers for: + - primary LLM settings + - validation inheritance from primary when unset + - validation overrides when set. +- Generic JSON diagnostics primitives for request metadata, request payload, response payload, and optional error payload, with secret redaction. +- Unit tests covering adapter behavior, scheduler behavior, effective config resolution, and diagnostics redaction/JSON validity. Do not implement: - Real correction modules. @@ -245,15 +259,18 @@ Do not implement: ### Expected behavior at end of phase -The codebase has a tested OpenAI-compatible structured-output client adapter, but scheduler and runtime wiring remain before this phase is fully complete. The CLI still does not perform real LLM polishing. +The codebase has tested Phase 9 LLM infrastructure, but default CLI runtime behavior remains deterministic preprocessing/reporting because real modules and LLM-backed validators are not implemented. -### Definition of done +### Definition of done status -Remaining checklist to close Phase 9: -- Primary and validation LLM settings resolve correctly in runtime wiring. -- Prompt/response diagnostic primitives exist. -- API keys are not leaked. -- No real module behavior is introduced. +Met: +- Structured LLM client infrastructure is implemented and tested. +- OpenAI-compatible structured-output client exists and is tested. +- Scheduler enforces configured concurrency and is tested. +- Primary and validation effective config resolution exists and is tested. +- Prompt/response diagnostics primitives exist and are tested. +- API keys are redacted in LLM adapter errors and diagnostics artifacts; config/report diagnostics redaction remains in place. +- No real module behavior was introduced. - `go test ./...` passes. ## Phase 10: LLM-backed validators diff --git a/internal/framework/llm/instructor_client.go b/internal/framework/llm/instructor_client.go index 033c1f6..77e3e83 100644 --- a/internal/framework/llm/instructor_client.go +++ b/internal/framework/llm/instructor_client.go @@ -125,8 +125,6 @@ func (c *InstructorClient) CompleteStructured( }, nil } -// TODO(phase9): integrate bounded scheduler/semaphore in the next Phase 9 prompt. - func normalizeConfig(cfg InstructorClientConfig) (InstructorClientConfig, error) { cfg.BaseURL = strings.TrimSpace(cfg.BaseURL) cfg.Model = strings.TrimSpace(cfg.Model)