Add OpenAI-compatible model client
This commit is contained in:
53
docs/internal/llm.md
Normal file
53
docs/internal/llm.md
Normal file
@@ -0,0 +1,53 @@
|
||||
# Internal Model Client
|
||||
|
||||
## Purpose
|
||||
|
||||
This document describes Promptkit's internal model-client implementation. The
|
||||
[architecture policy](../policy/architecture.md) owns the library boundary,
|
||||
and the
|
||||
[OpenAI-compatible chat integration](../integrations/openai-compatible-chat.md)
|
||||
owns the observable outbound HTTP contract.
|
||||
|
||||
The client is implemented only under `internal/llm`. The root package does not
|
||||
yet assemble it into a usable public engine.
|
||||
|
||||
## Components And Flow
|
||||
|
||||
`Client` is the provider-neutral generation boundary consumed by later
|
||||
orchestration. `OpenAICompatibleClient` is the built-in implementation. It
|
||||
uses internal domain values for rendered prompts, execution targets,
|
||||
structured output, responses, and token usage.
|
||||
|
||||
Construction validates the configured base URL and clones any supplied
|
||||
`http.Client` so Promptkit can apply its timeout default without mutating the
|
||||
caller's client. Generation then:
|
||||
|
||||
1. validates request-level timeout and endpoint requirements;
|
||||
2. maps the internal request into the OpenAI-compatible chat payload;
|
||||
3. validates and merges extra parameters;
|
||||
4. resolves authentication;
|
||||
5. performs the outbound request under the applicable deadlines; and
|
||||
6. decodes the first response choice and token usage.
|
||||
|
||||
The implementation has no retry loop, tool-call support, provider catalog,
|
||||
inbound HTTP behavior, or durable session store.
|
||||
|
||||
## Failure Categories
|
||||
|
||||
The package preserves distinct error identities for invalid client
|
||||
configuration, invalid generation requests, request execution failures,
|
||||
non-success provider statuses, and malformed successful responses. Provider
|
||||
response bodies are not included in non-success errors.
|
||||
|
||||
Caller cancellation and deadline failures during the outbound request are
|
||||
reported as request execution failures. The future runner can classify these
|
||||
identities without depending on HTTP status mapping.
|
||||
|
||||
## Test Ownership
|
||||
|
||||
The
|
||||
[OpenAI-compatible client tests](../../internal/llm/openai_compatible_client_test.go)
|
||||
own configuration, client cloning, deterministic deadline precedence,
|
||||
authentication, request and response mapping, malformed data, error identity,
|
||||
cancellation, and response-body suppression. They use local test servers and
|
||||
test transports; the default suite makes no live or paid provider requests.
|
||||
Reference in New Issue
Block a user