67 lines
3.0 KiB
Markdown
67 lines
3.0 KiB
Markdown
# Internal Model Client
|
|
|
|
## Purpose
|
|
|
|
This document describes Promptkit's internal model-client implementation. The
|
|
[architecture policy](../policy/architecture.md) owns the library boundary,
|
|
and the
|
|
[OpenAI-compatible chat integration](../integrations/openai-compatible-chat.md)
|
|
owns the observable outbound HTTP contract. The
|
|
[framework format reference](../formats.md) owns the profile and prompt
|
|
settings consumed by the client.
|
|
|
|
The concrete client remains under `internal/llm`. The root engine assembles it
|
|
as the default implementation behind Promptkit's public client boundary.
|
|
|
|
## Components And Flow
|
|
|
|
`Client` is the provider-neutral generation boundary consumed by later
|
|
orchestration. `OpenAICompatibleClient` is the built-in implementation. It
|
|
uses internal domain values for rendered prompts, execution targets,
|
|
structured output, responses, and token usage.
|
|
|
|
The runner supplies a fully resolved target after applying backend, profile,
|
|
and request precedence. The client uses its endpoint, credential metadata,
|
|
generation fields, and extra parameters. `BackendID` remains routing metadata
|
|
for the generation boundary and is not mapped into the provider payload.
|
|
|
|
Construction validates the configured base URL and clones any supplied
|
|
`http.Client` so Promptkit can apply its timeout default without mutating the
|
|
caller's client. Generation then:
|
|
|
|
1. validates request-level timeout and endpoint requirements;
|
|
2. maps the internal request into the OpenAI-compatible chat payload;
|
|
3. validates and merges extra parameters;
|
|
4. resolves authentication;
|
|
5. performs the outbound request under the applicable deadlines; and
|
|
6. decodes the first response choice and token usage.
|
|
|
|
`internal/llm` owns the set of reserved OpenAI-compatible request fields used
|
|
when validating extra parameters. Backend registration consumes the same rule
|
|
without making the model client depend on registry configuration.
|
|
|
|
The implementation has no retry loop, tool-call support, provider catalog,
|
|
inbound HTTP behavior, or durable session store.
|
|
|
|
## Failure Categories
|
|
|
|
The package preserves distinct error identities for invalid client
|
|
configuration, invalid generation requests, request execution failures,
|
|
non-success provider statuses, and malformed successful responses. Provider
|
|
response bodies are not included in non-success errors.
|
|
|
|
Caller cancellation and deadline failures during the outbound request are
|
|
reported as request execution failures. The runner classifies these identities
|
|
without depending on HTTP status mapping.
|
|
|
|
## Test Ownership
|
|
|
|
The
|
|
[OpenAI-compatible client tests](../../internal/llm/openai_compatible_client_test.go)
|
|
own configuration, client cloning, deterministic deadline precedence,
|
|
authentication, request and response mapping, malformed data, error identity,
|
|
cancellation, and response-body suppression. The root transport contract test
|
|
also verifies that resolved backend settings reach this client without
|
|
serializing backend identity. All use local test servers or test transports;
|
|
the default suite makes no live or paid provider requests.
|