Implement Phase 9 structured LLM adapter spike

This commit is contained in:
2026-05-11 19:54:57 -05:00
parent aeb31f1c0d
commit 0b17a6fbeb
8 changed files with 730 additions and 25 deletions

View File

@@ -52,10 +52,17 @@ Implemented:
- Deterministic validator-chain execution in the production runner.
- Module reports including validator decisions and validator rejections.
- Broad deterministic and CLI/subprocess test coverage for implemented phases through `go test ./...`.
- Internal typed structured LLM contract (`StructuredLLMClient.CompleteStructured(ctx, req, out)`).
- `internal/framework/llm` instructor-go-backed adapter with:
- configurable base URL/model/retries/mode/timeout
- optional API key support for local-compatible endpoints
- API-key redaction in returned errors
- typed structured decode into caller-provided outputs.
Not yet implemented in runtime pipeline:
- Real correction modules.
- Structured LLM client integration.
- Structured LLM scheduler/concurrency orchestration.
- Runtime wiring from production runner/modules into the structured LLM adapter.
- LLM-backed validators.
- Prompt/response diagnostics for LLM calls.
- End-to-end transcript polishing behavior.
@@ -189,37 +196,44 @@ Implemented:
Not implemented in Phase 8 (by design):
- LLM-backed validators (Phase 10).
- Structured LLM client implementation or scheduler behavior (Phase 9).
- Structured LLM scheduler behavior or runtime wiring (Phase 9 follow-up).
- Real correction modules.
- Prompt/response diagnostics.
- End-to-end transcript polishing.
## Remaining work plan
Next recommended phase: **Phase 9 (structured LLM client and scheduler infrastructure)**.
Next recommended phase: **Phase 9 follow-up (scheduler + runtime LLM wiring, still no real modules)**.
## Phase 9: Structured LLM client and scheduler infrastructure
### Status
Partially completed.
Implemented in this phase so far:
- Added internal structured LLM contract support for caller-provided typed outputs.
- Added `internal/framework/llm` adapter backed by `github.com/jxnl/instructor-go`.
- Confirmed OpenAI-compatible base URL support through the adapter path.
- Added adapter unit tests for model/base URL handling, retries, context cancellation, optional API key behavior, and error redaction.
Still pending in Phase 9:
- Scheduler/semaphore behavior for bounded concurrency.
- Runtime wiring in runner/module infrastructure (without introducing real modules yet).
- Prompt/response diagnostics writer primitives for LLM call artifacts.
- Full primary vs validation LLM config-resolution plumbing into runtime LLM call sites.
### Purpose
Implement the provider-neutral LLM infrastructure needed by both proposal generation and LLM-backed validators, without yet implementing real modules.
### Scope
Implement:
- `StructuredLLMClient` interface refinement if needed.
- OpenAI-compatible chat completions client.
- Structured JSON response support.
- Request/response types for structured calls.
- Retry handling for malformed structured output.
- Per-request timeout behavior.
- Context cancellation.
Implement (remaining):
- Primary LLM config resolution.
- Validation LLM config resolution and inheritance from primary settings.
- Optional API key behavior for self-hosted endpoints.
- Redaction of credentials in all diagnostics and reports.
- LLM scheduler/semaphore for bounded backend concurrency.
- Unit tests using fake HTTP servers or fake client implementations.
- Prompt/response diagnostics writer primitives that can later be used by modules and validators.
Do not implement:
@@ -230,15 +244,13 @@ Do not implement:
### Expected behavior at end of phase
The codebase has a tested OpenAI-compatible structured-output client and scheduler, but the CLI still does not perform real LLM polishing unless later phases wire modules into the runner.
The codebase has a tested OpenAI-compatible structured-output client adapter, but scheduler and runtime wiring remain before this phase is fully complete. The CLI still does not perform real LLM polishing.
### Definition of done
- Structured LLM client is implemented and tested.
Remaining checklist to close Phase 9:
- Scheduler enforces configured concurrency.
- Primary and validation LLM settings resolve correctly.
- Timeout and retry behavior are tested.
- Malformed structured responses fail cleanly or retry according to config.
- Primary and validation LLM settings resolve correctly in runtime wiring.
- Prompt/response diagnostic primitives exist.
- API keys are not leaked.
- No real module behavior is introduced.