86 lines
3.5 KiB
Markdown
86 lines
3.5 KiB
Markdown
# LLM Internals
|
|
|
|
## Purpose
|
|
|
|
`internal/llm` defines the provider-neutral `Client` interface and the
|
|
OpenAI-compatible client implementation. The [OpenAI-compatible integration
|
|
contract](../integrations/openai-compatible-chat.md) owns the outbound HTTP wire
|
|
format and protocol behavior.
|
|
|
|
## Construction
|
|
|
|
`NewOpenAICompatibleClient` validates a non-empty configured base URL, records
|
|
an optional default model, and establishes the default timeout. A non-positive
|
|
configured timeout uses the internal default.
|
|
|
|
When callers supply an `http.Client`, construction clones it rather than
|
|
mutating the caller's instance. A supplied client with no timeout receives the
|
|
resolved default in the clone; a supplied non-zero timeout is retained. The
|
|
client stores the trimmed base URL, default model, timeout, and cloned client.
|
|
|
|
## Generate Flow
|
|
|
|
`Generate` receives a `domain.GenerateRequest` from the runner:
|
|
|
|
1. validate the effective timeout and choose the request endpoint;
|
|
2. map the domain request to the internal wire-request representation;
|
|
3. validate and flatten extra parameters, encode JSON, and create the HTTP
|
|
request;
|
|
4. prefer a direct API key, otherwise resolve the configured key environment
|
|
variable;
|
|
5. derive a request HTTP client when an explicit timeout changes the configured
|
|
client;
|
|
6. execute the request, reject non-success status responses without returning
|
|
provider response bodies; and
|
|
7. decode the response subset into `domain.GenerateResponse`.
|
|
|
|
`openAIChatRequestFromGenerateRequest` is the conversion boundary for effective
|
|
model defaults, explicit numeric-presence state, rendered messages, structured
|
|
output, and session-ID validation. `openAIChatRequestPayload` protects reserved
|
|
fields and JSON encoding before an HTTP call. The external payload shape is
|
|
defined only in the [integration contract](../integrations/openai-compatible-chat.md).
|
|
|
|
## Error Categories
|
|
|
|
The package uses these internal sentinels:
|
|
|
|
- `ErrInvalidConfig` for invalid client construction;
|
|
- `ErrInvalidRequest` for invalid effective generation input;
|
|
- `ErrRequestFailed` for request construction or transport failures;
|
|
- `ErrUnexpectedStatus` for non-success HTTP responses; and
|
|
- `ErrMalformedResponse` for invalid or incomplete successful-response data.
|
|
|
|
The runner maps an invalid LLM request to its invalid-request category and
|
|
other LLM failures to its generation category. Adapters then apply their public
|
|
error contracts.
|
|
|
|
## Package-Local Guarantees
|
|
|
|
- The default-model fallback happens before wire encoding.
|
|
- Per-request timeout handling clones a configured HTTP client when needed; it
|
|
does not mutate shared client state.
|
|
- Direct API keys take precedence over environment lookup within this client.
|
|
- Provider response bodies are discarded for non-success status responses.
|
|
- The client does not implement retries, tool calls, or a stateful session
|
|
store.
|
|
|
|
## Verification And Change Recipe
|
|
|
|
Inspect:
|
|
|
|
- `internal/llm/openai_compatible_client_test.go`
|
|
- `internal/usecase/runner_test.go`
|
|
- `internal/adapter/http/handler_test.go`
|
|
|
|
When changing the client:
|
|
|
|
1. keep domain-to-wire mapping inside `internal/llm` and preserve the `Client`
|
|
interface;
|
|
2. test construction, timeout selection, mapping, and error categorization;
|
|
3. update the [OpenAI-compatible integration contract](../integrations/openai-compatible-chat.md)
|
|
for any observable wire or protocol change; and
|
|
4. update [runner internals](runner.md) if the client boundary or structured
|
|
output handoff changes.
|
|
|
|
The [testing policy](../policy/testing.md) owns global test sufficiency.
|