promptkit/docs/integrations/openai-compatible-chat.md

OpenAI-Compatible Chat Integration

Purpose

This document defines the outbound HTTP behavior implemented by Promptkit's internal OpenAI-compatible model client. The internal model-client document owns implementation flow, errors, and test ownership. The root Promptkit engine uses this client by default unless a consumer injects another implementation. The framework format reference owns the profile and prompt values that produce these outbound settings.

Endpoint And Method

Generation sends an HTTP POST with Content-Type: application/json. A non-empty endpoint from the execution target overrides the client's configured base URL. After trailing slashes are removed, /chat/completions is appended. Generation fails before sending when neither source supplies an endpoint.

Authentication

A non-empty API key supplied directly on the execution target takes precedence. Otherwise, when an API-key environment-variable name is supplied, the client reads that variable and requires a non-empty value. The selected key is sent as Authorization: Bearer <key>. No authorization header is sent when neither mechanism is configured.

Request Body

The request body always contains model and messages. The execution target's model takes precedence over the client's configured model, and one must be available.

Each ordinary message contains its role and string content. A cache-controlled message instead uses a text content block containing type, text, and cache_control; an empty cache-control TTL is omitted.

A non-empty session ID is trimmed, checked against the internal domain limit, and sent as top-level session_id. It is not sent as a session header.

The client conditionally includes:

• temperature, max_tokens, and top_p when non-zero or explicitly present;
• non-empty service_tier and reasoning_effort; and
• response_format for JSON Schema structured output, including its name, strict flag, and schema document.

Extra parameters are merged directly into the top-level body after JSON serialization is verified. Empty keys and collisions with these reserved fields are rejected before any provider call:

• model
• session_id
• messages
• temperature
• max_tokens
• top_p
• service_tier
• reasoning_effort
• response_format

Response Handling

Any 2xx response is decoded as an OpenAI-compatible chat response. The client returns the first choice's non-empty message content and maps prompt, completion, total, cached, and cache-write token counts.

Invalid JSON, absent choices, and empty first-choice content are malformed responses. For a non-2xx status, the error includes the status code but never the provider response body.

Timeout And Cancellation

Timeouts are layered:

• the caller context remains the outer cancellation boundary;
• a positive generation timeout adds a request context deadline;
• zero adds no generation-specific deadline;
• a negative generation timeout is invalid; and
• the cloned http.Client supplies the whole-request transport cap, retaining a positive supplied-client timeout or applying the configured/default timeout when the supplied value is not positive.

The earliest applicable caller, generation, or transport deadline controls the request. Constructing the internal client does not mutate a supplied http.Client.