promptkit/docs/integrations/openai-compatible-chat.md

OpenAI-Compatible Chat Integration

Purpose

This document defines the outbound HTTP behavior implemented by Promptkit's internal OpenAI-compatible model client. The internal model-client document owns implementation flow, errors, and test ownership. The root Promptkit engine uses this client by default unless a consumer injects another implementation. The framework format reference owns the profile and prompt values that produce these outbound settings.

Endpoint And Method

Generation sends an HTTP POST with Content-Type: application/json. Before the client is called, the engine resolves framework, backend, profile, and request values into one execution target. A non-empty endpoint from that target overrides the client's configured base URL. After trailing slashes are removed, /chat/completions is appended. Generation fails before sending when neither source supplies an endpoint.

The target's backend ID is routing metadata for prepared values, results, and injected clients. The built-in client does not derive the URL from that ID and does not serialize it in the provider request.

Authentication

A non-empty API key supplied directly on the execution target takes precedence. Otherwise, when an API-key environment-variable name is supplied, the client reads that variable and requires a non-empty value. The selected key is sent as Authorization: Bearer <key>. No authorization header is sent when neither mechanism is configured.

The target contains the already resolved environment-variable name: an explicit request override takes precedence over profile metadata, which takes precedence over the backend default. Only the name reaches prepared metadata; the environment value is read just before the provider call and is never added to the JSON body.

Request Body

The request body always contains model and messages. The execution target's model takes precedence over the client's configured model, and one must be available.

Each ordinary message contains its role and string content. A cache-controlled message instead uses a text content block containing type, text, and cache_control; an empty cache-control TTL is omitted.

A non-empty session ID is trimmed, checked against the internal domain limit, and sent as top-level session_id. It is not sent as a session header.

The client conditionally includes:

• temperature, max_tokens, and top_p when non-zero or explicitly present;
• non-empty service_tier and reasoning_effort; and
• response_format for JSON Schema structured output, including its name, strict flag, and schema document.

The engine resolves backend, profile, and request extra-parameter maps by whole-map replacement rather than key merging. The resulting effective map is then merged directly into the top-level body after JSON serialization is verified. Empty keys and collisions with these reserved fields are rejected before any provider call:

• model
• session_id
• messages
• temperature
• max_tokens
• top_p
• service_tier
• reasoning_effort
• response_format

backend_id, api_key_env, and resolved credential values are not provider request fields.

Response Handling

Any 2xx response is decoded as an OpenAI-compatible chat response. The client returns the first choice's non-empty message content and maps prompt, completion, total, cached, and cache-write token counts.

Invalid JSON, absent choices, and empty first-choice content are malformed responses. For a non-2xx status, the error includes the status code but never the provider response body.

Timeout And Cancellation

Timeouts are layered:

• the caller context remains the outer cancellation boundary;
• a positive generation timeout adds a request context deadline;
• zero adds no generation-specific deadline;
• a negative generation timeout is invalid; and
• the cloned http.Client supplies the whole-request transport cap, retaining a positive supplied-client timeout or applying the configured/default timeout when the supplied value is not positive.

The earliest applicable caller, generation, or transport deadline controls the request. Constructing the internal client does not mutate a supplied http.Client.