Refocus internal component documentation
This commit is contained in:
85
docs/internal/llm.md
Normal file
85
docs/internal/llm.md
Normal file
@@ -0,0 +1,85 @@
|
||||
# LLM Internals
|
||||
|
||||
## Purpose
|
||||
|
||||
`internal/llm` defines the provider-neutral `Client` interface and the
|
||||
OpenAI-compatible client implementation. The [OpenAI-compatible integration
|
||||
contract](../integrations/openai-compatible-chat.md) owns the outbound HTTP wire
|
||||
format and protocol behavior.
|
||||
|
||||
## Construction
|
||||
|
||||
`NewOpenAICompatibleClient` validates a non-empty configured base URL, records
|
||||
an optional default model, and establishes the default timeout. A non-positive
|
||||
configured timeout uses the internal default.
|
||||
|
||||
When callers supply an `http.Client`, construction clones it rather than
|
||||
mutating the caller's instance. A supplied client with no timeout receives the
|
||||
resolved default in the clone; a supplied non-zero timeout is retained. The
|
||||
client stores the trimmed base URL, default model, timeout, and cloned client.
|
||||
|
||||
## Generate Flow
|
||||
|
||||
`Generate` receives a `domain.GenerateRequest` from the runner:
|
||||
|
||||
1. validate the effective timeout and choose the request endpoint;
|
||||
2. map the domain request to the internal wire-request representation;
|
||||
3. validate and flatten extra parameters, encode JSON, and create the HTTP
|
||||
request;
|
||||
4. prefer a direct API key, otherwise resolve the configured key environment
|
||||
variable;
|
||||
5. derive a request HTTP client when an explicit timeout changes the configured
|
||||
client;
|
||||
6. execute the request, reject non-success status responses without returning
|
||||
provider response bodies; and
|
||||
7. decode the response subset into `domain.GenerateResponse`.
|
||||
|
||||
`openAIChatRequestFromGenerateRequest` is the conversion boundary for effective
|
||||
model defaults, explicit numeric-presence state, rendered messages, structured
|
||||
output, and session-ID validation. `openAIChatRequestPayload` protects reserved
|
||||
fields and JSON encoding before an HTTP call. The external payload shape is
|
||||
defined only in the [integration contract](../integrations/openai-compatible-chat.md).
|
||||
|
||||
## Error Categories
|
||||
|
||||
The package uses these internal sentinels:
|
||||
|
||||
- `ErrInvalidConfig` for invalid client construction;
|
||||
- `ErrInvalidRequest` for invalid effective generation input;
|
||||
- `ErrRequestFailed` for request construction or transport failures;
|
||||
- `ErrUnexpectedStatus` for non-success HTTP responses; and
|
||||
- `ErrMalformedResponse` for invalid or incomplete successful-response data.
|
||||
|
||||
The runner maps an invalid LLM request to its invalid-request category and
|
||||
other LLM failures to its generation category. Adapters then apply their public
|
||||
error contracts.
|
||||
|
||||
## Package-Local Guarantees
|
||||
|
||||
- The default-model fallback happens before wire encoding.
|
||||
- Per-request timeout handling clones a configured HTTP client when needed; it
|
||||
does not mutate shared client state.
|
||||
- Direct API keys take precedence over environment lookup within this client.
|
||||
- Provider response bodies are discarded for non-success status responses.
|
||||
- The client does not implement retries, tool calls, or a stateful session
|
||||
store.
|
||||
|
||||
## Verification And Change Recipe
|
||||
|
||||
Inspect:
|
||||
|
||||
- `internal/llm/openai_compatible_client_test.go`
|
||||
- `internal/usecase/runner_test.go`
|
||||
- `internal/adapter/http/handler_test.go`
|
||||
|
||||
When changing the client:
|
||||
|
||||
1. keep domain-to-wire mapping inside `internal/llm` and preserve the `Client`
|
||||
interface;
|
||||
2. test construction, timeout selection, mapping, and error categorization;
|
||||
3. update the [OpenAI-compatible integration contract](../integrations/openai-compatible-chat.md)
|
||||
for any observable wire or protocol change; and
|
||||
4. update [runner internals](runner.md) if the client boundary or structured
|
||||
output handoff changes.
|
||||
|
||||
The [testing policy](../policy/testing.md) owns global test sufficiency.
|
||||
Reference in New Issue
Block a user