Implement Phase 9 structured LLM adapter spike
This commit is contained in:
@@ -52,10 +52,17 @@ Implemented:
|
||||
- Deterministic validator-chain execution in the production runner.
|
||||
- Module reports including validator decisions and validator rejections.
|
||||
- Broad deterministic and CLI/subprocess test coverage for implemented phases through `go test ./...`.
|
||||
- Internal typed structured LLM contract (`StructuredLLMClient.CompleteStructured(ctx, req, out)`).
|
||||
- `internal/framework/llm` instructor-go-backed adapter with:
|
||||
- configurable base URL/model/retries/mode/timeout
|
||||
- optional API key support for local-compatible endpoints
|
||||
- API-key redaction in returned errors
|
||||
- typed structured decode into caller-provided outputs.
|
||||
|
||||
Not yet implemented in runtime pipeline:
|
||||
- Real correction modules.
|
||||
- Structured LLM client integration.
|
||||
- Structured LLM scheduler/concurrency orchestration.
|
||||
- Runtime wiring from production runner/modules into the structured LLM adapter.
|
||||
- LLM-backed validators.
|
||||
- Prompt/response diagnostics for LLM calls.
|
||||
- End-to-end transcript polishing behavior.
|
||||
@@ -189,37 +196,44 @@ Implemented:
|
||||
|
||||
Not implemented in Phase 8 (by design):
|
||||
- LLM-backed validators (Phase 10).
|
||||
- Structured LLM client implementation or scheduler behavior (Phase 9).
|
||||
- Structured LLM scheduler behavior or runtime wiring (Phase 9 follow-up).
|
||||
- Real correction modules.
|
||||
- Prompt/response diagnostics.
|
||||
- End-to-end transcript polishing.
|
||||
|
||||
## Remaining work plan
|
||||
|
||||
Next recommended phase: **Phase 9 (structured LLM client and scheduler infrastructure)**.
|
||||
Next recommended phase: **Phase 9 follow-up (scheduler + runtime LLM wiring, still no real modules)**.
|
||||
|
||||
## Phase 9: Structured LLM client and scheduler infrastructure
|
||||
|
||||
### Status
|
||||
|
||||
Partially completed.
|
||||
|
||||
Implemented in this phase so far:
|
||||
- Added internal structured LLM contract support for caller-provided typed outputs.
|
||||
- Added `internal/framework/llm` adapter backed by `github.com/jxnl/instructor-go`.
|
||||
- Confirmed OpenAI-compatible base URL support through the adapter path.
|
||||
- Added adapter unit tests for model/base URL handling, retries, context cancellation, optional API key behavior, and error redaction.
|
||||
|
||||
Still pending in Phase 9:
|
||||
- Scheduler/semaphore behavior for bounded concurrency.
|
||||
- Runtime wiring in runner/module infrastructure (without introducing real modules yet).
|
||||
- Prompt/response diagnostics writer primitives for LLM call artifacts.
|
||||
- Full primary vs validation LLM config-resolution plumbing into runtime LLM call sites.
|
||||
|
||||
### Purpose
|
||||
|
||||
Implement the provider-neutral LLM infrastructure needed by both proposal generation and LLM-backed validators, without yet implementing real modules.
|
||||
|
||||
### Scope
|
||||
|
||||
Implement:
|
||||
- `StructuredLLMClient` interface refinement if needed.
|
||||
- OpenAI-compatible chat completions client.
|
||||
- Structured JSON response support.
|
||||
- Request/response types for structured calls.
|
||||
- Retry handling for malformed structured output.
|
||||
- Per-request timeout behavior.
|
||||
- Context cancellation.
|
||||
Implement (remaining):
|
||||
- Primary LLM config resolution.
|
||||
- Validation LLM config resolution and inheritance from primary settings.
|
||||
- Optional API key behavior for self-hosted endpoints.
|
||||
- Redaction of credentials in all diagnostics and reports.
|
||||
- LLM scheduler/semaphore for bounded backend concurrency.
|
||||
- Unit tests using fake HTTP servers or fake client implementations.
|
||||
- Prompt/response diagnostics writer primitives that can later be used by modules and validators.
|
||||
|
||||
Do not implement:
|
||||
@@ -230,15 +244,13 @@ Do not implement:
|
||||
|
||||
### Expected behavior at end of phase
|
||||
|
||||
The codebase has a tested OpenAI-compatible structured-output client and scheduler, but the CLI still does not perform real LLM polishing unless later phases wire modules into the runner.
|
||||
The codebase has a tested OpenAI-compatible structured-output client adapter, but scheduler and runtime wiring remain before this phase is fully complete. The CLI still does not perform real LLM polishing.
|
||||
|
||||
### Definition of done
|
||||
|
||||
- Structured LLM client is implemented and tested.
|
||||
Remaining checklist to close Phase 9:
|
||||
- Scheduler enforces configured concurrency.
|
||||
- Primary and validation LLM settings resolve correctly.
|
||||
- Timeout and retry behavior are tested.
|
||||
- Malformed structured responses fail cleanly or retry according to config.
|
||||
- Primary and validation LLM settings resolve correctly in runtime wiring.
|
||||
- Prompt/response diagnostic primitives exist.
|
||||
- API keys are not leaked.
|
||||
- No real module behavior is introduced.
|
||||
|
||||
Reference in New Issue
Block a user