89 lines
3.3 KiB
Markdown
89 lines
3.3 KiB
Markdown
# OpenAI-Compatible Chat Integration
|
|
|
|
## Purpose
|
|
|
|
This document defines the outbound HTTP behavior implemented by Promptkit's
|
|
internal OpenAI-compatible model client. The
|
|
[internal model-client document](../internal/llm.md) owns implementation flow,
|
|
errors, and test ownership. The root Promptkit engine uses this client by
|
|
default unless a consumer injects another implementation. The
|
|
[framework format reference](../formats.md) owns the profile and prompt values
|
|
that produce these outbound settings.
|
|
|
|
## Endpoint And Method
|
|
|
|
Generation sends an HTTP `POST` with `Content-Type: application/json`.
|
|
A non-empty endpoint from the execution target overrides the client's
|
|
configured base URL. After trailing slashes are removed,
|
|
`/chat/completions` is appended. Generation fails before sending when neither
|
|
source supplies an endpoint.
|
|
|
|
## Authentication
|
|
|
|
A non-empty API key supplied directly on the execution target takes
|
|
precedence. Otherwise, when an API-key environment-variable name is supplied,
|
|
the client reads that variable and requires a non-empty value. The selected
|
|
key is sent as `Authorization: Bearer <key>`. No authorization header is sent
|
|
when neither mechanism is configured.
|
|
|
|
## Request Body
|
|
|
|
The request body always contains `model` and `messages`. The execution
|
|
target's model takes precedence over the client's configured model, and one
|
|
must be available.
|
|
|
|
Each ordinary message contains its `role` and string `content`. A
|
|
cache-controlled message instead uses a text content block containing `type`,
|
|
`text`, and `cache_control`; an empty cache-control TTL is omitted.
|
|
|
|
A non-empty session ID is trimmed, checked against the internal domain limit,
|
|
and sent as top-level `session_id`. It is not sent as a session header.
|
|
|
|
The client conditionally includes:
|
|
|
|
- `temperature`, `max_tokens`, and `top_p` when non-zero or explicitly
|
|
present;
|
|
- non-empty `service_tier` and `reasoning_effort`; and
|
|
- `response_format` for JSON Schema structured output, including its name,
|
|
strict flag, and schema document.
|
|
|
|
Extra parameters are merged directly into the top-level body after JSON
|
|
serialization is verified. Empty keys and collisions with these reserved
|
|
fields are rejected before any provider call:
|
|
|
|
- `model`
|
|
- `session_id`
|
|
- `messages`
|
|
- `temperature`
|
|
- `max_tokens`
|
|
- `top_p`
|
|
- `service_tier`
|
|
- `reasoning_effort`
|
|
- `response_format`
|
|
|
|
## Response Handling
|
|
|
|
Any 2xx response is decoded as an OpenAI-compatible chat response. The client
|
|
returns the first choice's non-empty message content and maps prompt,
|
|
completion, total, cached, and cache-write token counts.
|
|
|
|
Invalid JSON, absent choices, and empty first-choice content are malformed
|
|
responses. For a non-2xx status, the error includes the status code but never
|
|
the provider response body.
|
|
|
|
## Timeout And Cancellation
|
|
|
|
Timeouts are layered:
|
|
|
|
- the caller context remains the outer cancellation boundary;
|
|
- a positive generation timeout adds a request context deadline;
|
|
- zero adds no generation-specific deadline;
|
|
- a negative generation timeout is invalid; and
|
|
- the cloned `http.Client` supplies the whole-request transport cap, retaining
|
|
a positive supplied-client timeout or applying the configured/default
|
|
timeout when the supplied value is not positive.
|
|
|
|
The earliest applicable caller, generation, or transport deadline controls the
|
|
request. Constructing the internal client does not mutate a supplied
|
|
`http.Client`.
|