Complete Phase 9 LLM infrastructure
This commit is contained in:
@@ -31,12 +31,12 @@ Not implemented in CLI runtime path today:
|
||||
- Real module execution pipeline (`glossary`, `homophones`, `spoken_word`, `grammar`).
|
||||
- Structured LLM proposal generation.
|
||||
- LLM-backed validators.
|
||||
- Production LLM scheduler behavior.
|
||||
- Runtime module/validator usage of the LLM scheduler infrastructure.
|
||||
- End-to-end transcript polishing with real module behavior.
|
||||
|
||||
Phase sequencing note:
|
||||
- structured LLM client, scheduler, config-resolution helpers, and diagnostics primitives are implemented;
|
||||
- runtime wiring from modules/validators/runner into this LLM infrastructure remains Phase 9 follow-up work;
|
||||
- Phase 9 LLM infrastructure is complete (structured client, scheduler, effective config resolution, diagnostics primitives);
|
||||
- runtime wiring from modules/validators/runner into this LLM infrastructure remains future module/validator phase work;
|
||||
- LLM-backed validators remain Phase 10 work.
|
||||
|
||||
## Actual Go package layout
|
||||
|
||||
@@ -208,13 +208,13 @@ Not implemented in Phase 8 (by design):
|
||||
|
||||
## Remaining work plan
|
||||
|
||||
Next recommended phase: **Phase 9 follow-up (runtime LLM wiring, still no real modules)**.
|
||||
Next recommended phase: **Phase 10 (LLM-backed validators)**.
|
||||
|
||||
## Phase 9: Structured LLM client and scheduler infrastructure
|
||||
|
||||
### Status
|
||||
|
||||
Partially completed.
|
||||
Completed for Phase 9 infrastructure scope.
|
||||
|
||||
Implemented in this phase so far:
|
||||
- Added internal structured LLM contract support for caller-provided typed outputs.
|
||||
@@ -222,7 +222,7 @@ Implemented in this phase so far:
|
||||
- Confirmed OpenAI-compatible base URL support through the adapter path.
|
||||
- Added adapter unit tests for model/base URL handling, retries, context cancellation, optional API key behavior, and error redaction.
|
||||
|
||||
Still pending in Phase 9:
|
||||
Explicitly deferred from Phase 9 into later phases:
|
||||
- Runtime wiring in runner/module infrastructure (without introducing real modules yet).
|
||||
- Wiring prompt/response diagnostics primitives into future module/validator call sites.
|
||||
- Wiring effective primary/validation LLM config resolution into runtime LLM call sites.
|
||||
@@ -231,11 +231,25 @@ Still pending in Phase 9:
|
||||
|
||||
Implement the provider-neutral LLM infrastructure needed by both proposal generation and LLM-backed validators, without yet implementing real modules.
|
||||
|
||||
### Scope
|
||||
### Scope completed in this phase
|
||||
|
||||
Implement (remaining):
|
||||
- Runtime wiring for primary/validation effective LLM config resolution.
|
||||
- Runtime usage of diagnostics primitives with credential redaction.
|
||||
Implemented:
|
||||
- Provider-neutral internal structured LLM contract with caller-provided typed output decoding.
|
||||
- OpenAI-compatible structured-output adapter (`instructor-go`) with:
|
||||
- configurable base URL and model
|
||||
- optional API key behavior
|
||||
- retry budget
|
||||
- timeout-aware HTTP client handling
|
||||
- context cancellation propagation
|
||||
- structured response decoding into caller-provided typed targets
|
||||
- redacted error surfaces.
|
||||
- Scheduler/semaphore infrastructure for bounded backend concurrency with context-aware acquisition and reliable permit release.
|
||||
- Effective LLM config resolution helpers for:
|
||||
- primary LLM settings
|
||||
- validation inheritance from primary when unset
|
||||
- validation overrides when set.
|
||||
- Generic JSON diagnostics primitives for request metadata, request payload, response payload, and optional error payload, with secret redaction.
|
||||
- Unit tests covering adapter behavior, scheduler behavior, effective config resolution, and diagnostics redaction/JSON validity.
|
||||
|
||||
Do not implement:
|
||||
- Real correction modules.
|
||||
@@ -245,15 +259,18 @@ Do not implement:
|
||||
|
||||
### Expected behavior at end of phase
|
||||
|
||||
The codebase has a tested OpenAI-compatible structured-output client adapter, but scheduler and runtime wiring remain before this phase is fully complete. The CLI still does not perform real LLM polishing.
|
||||
The codebase has tested Phase 9 LLM infrastructure, but default CLI runtime behavior remains deterministic preprocessing/reporting because real modules and LLM-backed validators are not implemented.
|
||||
|
||||
### Definition of done
|
||||
### Definition of done status
|
||||
|
||||
Remaining checklist to close Phase 9:
|
||||
- Primary and validation LLM settings resolve correctly in runtime wiring.
|
||||
- Prompt/response diagnostic primitives exist.
|
||||
- API keys are not leaked.
|
||||
- No real module behavior is introduced.
|
||||
Met:
|
||||
- Structured LLM client infrastructure is implemented and tested.
|
||||
- OpenAI-compatible structured-output client exists and is tested.
|
||||
- Scheduler enforces configured concurrency and is tested.
|
||||
- Primary and validation effective config resolution exists and is tested.
|
||||
- Prompt/response diagnostics primitives exist and are tested.
|
||||
- API keys are redacted in LLM adapter errors and diagnostics artifacts; config/report diagnostics redaction remains in place.
|
||||
- No real module behavior was introduced.
|
||||
- `go test ./...` passes.
|
||||
|
||||
## Phase 10: LLM-backed validators
|
||||
|
||||
@@ -125,8 +125,6 @@ func (c *InstructorClient) CompleteStructured(
|
||||
}, nil
|
||||
}
|
||||
|
||||
// TODO(phase9): integrate bounded scheduler/semaphore in the next Phase 9 prompt.
|
||||
|
||||
func normalizeConfig(cfg InstructorClientConfig) (InstructorClientConfig, error) {
|
||||
cfg.BaseURL = strings.TrimSpace(cfg.BaseURL)
|
||||
cfg.Model = strings.TrimSpace(cfg.Model)
|
||||
|
||||
Reference in New Issue
Block a user