Complete Phase 9 LLM infrastructure

This commit is contained in:
2026-05-11 20:10:56 -05:00
parent 426864eedb
commit 6d9a4bd017
3 changed files with 34 additions and 19 deletions

View File

@@ -31,12 +31,12 @@ Not implemented in CLI runtime path today:
- Real module execution pipeline (`glossary`, `homophones`, `spoken_word`, `grammar`).
- Structured LLM proposal generation.
- LLM-backed validators.
- Production LLM scheduler behavior.
- Runtime module/validator usage of the LLM scheduler infrastructure.
- End-to-end transcript polishing with real module behavior.
Phase sequencing note:
- structured LLM client, scheduler, config-resolution helpers, and diagnostics primitives are implemented;
- runtime wiring from modules/validators/runner into this LLM infrastructure remains Phase 9 follow-up work;
- Phase 9 LLM infrastructure is complete (structured client, scheduler, effective config resolution, diagnostics primitives);
- runtime wiring from modules/validators/runner into this LLM infrastructure remains future module/validator phase work;
- LLM-backed validators remain Phase 10 work.
## Actual Go package layout

View File

@@ -208,13 +208,13 @@ Not implemented in Phase 8 (by design):
## Remaining work plan
Next recommended phase: **Phase 9 follow-up (runtime LLM wiring, still no real modules)**.
Next recommended phase: **Phase 10 (LLM-backed validators)**.
## Phase 9: Structured LLM client and scheduler infrastructure
### Status
Partially completed.
Completed for Phase 9 infrastructure scope.
Implemented in this phase so far:
- Added internal structured LLM contract support for caller-provided typed outputs.
@@ -222,7 +222,7 @@ Implemented in this phase so far:
- Confirmed OpenAI-compatible base URL support through the adapter path.
- Added adapter unit tests for model/base URL handling, retries, context cancellation, optional API key behavior, and error redaction.
Still pending in Phase 9:
Explicitly deferred from Phase 9 into later phases:
- Runtime wiring in runner/module infrastructure (without introducing real modules yet).
- Wiring prompt/response diagnostics primitives into future module/validator call sites.
- Wiring effective primary/validation LLM config resolution into runtime LLM call sites.
@@ -231,11 +231,25 @@ Still pending in Phase 9:
Implement the provider-neutral LLM infrastructure needed by both proposal generation and LLM-backed validators, without yet implementing real modules.
### Scope
### Scope completed in this phase
Implement (remaining):
- Runtime wiring for primary/validation effective LLM config resolution.
- Runtime usage of diagnostics primitives with credential redaction.
Implemented:
- Provider-neutral internal structured LLM contract with caller-provided typed output decoding.
- OpenAI-compatible structured-output adapter (`instructor-go`) with:
- configurable base URL and model
- optional API key behavior
- retry budget
- timeout-aware HTTP client handling
- context cancellation propagation
- structured response decoding into caller-provided typed targets
- redacted error surfaces.
- Scheduler/semaphore infrastructure for bounded backend concurrency with context-aware acquisition and reliable permit release.
- Effective LLM config resolution helpers for:
- primary LLM settings
- validation inheritance from primary when unset
- validation overrides when set.
- Generic JSON diagnostics primitives for request metadata, request payload, response payload, and optional error payload, with secret redaction.
- Unit tests covering adapter behavior, scheduler behavior, effective config resolution, and diagnostics redaction/JSON validity.
Do not implement:
- Real correction modules.
@@ -245,15 +259,18 @@ Do not implement:
### Expected behavior at end of phase
The codebase has a tested OpenAI-compatible structured-output client adapter, but scheduler and runtime wiring remain before this phase is fully complete. The CLI still does not perform real LLM polishing.
The codebase has tested Phase 9 LLM infrastructure, but default CLI runtime behavior remains deterministic preprocessing/reporting because real modules and LLM-backed validators are not implemented.
### Definition of done
### Definition of done status
Remaining checklist to close Phase 9:
- Primary and validation LLM settings resolve correctly in runtime wiring.
- Prompt/response diagnostic primitives exist.
- API keys are not leaked.
- No real module behavior is introduced.
Met:
- Structured LLM client infrastructure is implemented and tested.
- OpenAI-compatible structured-output client exists and is tested.
- Scheduler enforces configured concurrency and is tested.
- Primary and validation effective config resolution exists and is tested.
- Prompt/response diagnostics primitives exist and are tested.
- API keys are redacted in LLM adapter errors and diagnostics artifacts; config/report diagnostics redaction remains in place.
- No real module behavior was introduced.
- `go test ./...` passes.
## Phase 10: LLM-backed validators

View File

@@ -125,8 +125,6 @@ func (c *InstructorClient) CompleteStructured(
}, nil
}
// TODO(phase9): integrate bounded scheduler/semaphore in the next Phase 9 prompt.
func normalizeConfig(cfg InstructorClientConfig) (InstructorClientConfig, error) {
cfg.BaseURL = strings.TrimSpace(cfg.BaseURL)
cfg.Model = strings.TrimSpace(cfg.Model)