# OpenAI-Compatible Chat Integration ## Purpose This document defines the outbound HTTP behavior implemented by Promptkit's internal OpenAI-compatible model client. The [internal model-client document](../internal/llm.md) owns implementation flow, errors, and test ownership. The root Promptkit engine uses this client by default unless a consumer injects another implementation. The [framework format reference](../formats.md) owns the profile and prompt values that produce these outbound settings. ## Endpoint And Method Generation sends an HTTP `POST` with `Content-Type: application/json`. Before the client is called, the engine resolves framework, backend, profile, and request values into one execution target. Endpoint configuration is trimmed and must be an absolute HTTP or HTTPS URL with a host and without user information, a query, or a fragment. A non-empty endpoint from the target overrides the client's configured base URL. The final selected endpoint is validated again before transport. The completion URL is composed through parsed URL path operations. Nested base paths are retained, repeated trailing slashes are normalized, and the result has exactly one appended `/chat/completions` suffix. Generation fails before sending when neither source supplies a valid endpoint. The target's backend ID is routing metadata for prepared values, results, and injected clients. The built-in client does not derive the URL from that ID and does not serialize it in the provider request. ## Authentication A usable API key supplied directly on the execution target takes precedence. Otherwise, when an API-key environment-variable name is supplied, the client reads and trims that variable. A bearer header is sent only when the resolved direct or environment credential is non-empty. When neither source is usable, the client omits `Authorization` and handles the provider response normally. An explicitly required target with no usable source is rejected before transport. The target contains the already resolved environment-variable name: an explicit request override takes precedence over profile metadata, which takes precedence over the backend default. Only the name reaches prepared metadata; the environment value is read just before the provider call and is never added to the JSON body. ## Explicit Smoke Diagnostics Promptkit's explicit smoke diagnostics are separate consumer operations; they are not part of ordinary `Run`, the offline examples, or release validation. With the built-in client, inference diagnostics use the same configured client and `/chat/completions` request as generation. Their content cases are private to Promptkit and are not application prompt, schema, or provider protocol extensions. The public [`Engine.SmokeTest` GoDoc](../../smoke_diagnostics.go) owns request selection, report facts, and injected-client behavior. HTTP-only diagnostics use the configured built-in HTTP client. Reachability is one unauthenticated `GET` to the selected diagnostic URL; an observed response counts as reachable even when its status is non-2xx. Authentication behavior is selected from backend metadata, not inferred from the host. The `openrouter_key` adapter makes one `GET` to `/key` relative to the effective backend base, unless configuration supplies a full diagnostic endpoint (see the [backend format reference](../formats.md#built-in-backends)). It may send the available bearer credential, but can dispatch without one. A 401 is recognized as rejection. A 200 is recognized as acceptance only when its JSON object has a `data` object containing string `label`, Boolean `is_free_tier`, and numeric `usage`; unrelated fields and account limits are ignored. This is only a minimal identifying subset of OpenRouter's documented [current-key response](https://github.com/OpenRouterTeam/docs/blob/main/api_reference/limits.mdx#checking-your-limits), not validation of the full account document. Other statuses and unrecognized 200 responses do not establish acceptance. Diagnostic HTTP evidence describes the last request observed at Promptkit's RoundTripper boundary and any response associated with it. It includes the selected destination without URL user information, response status, and credential presence when observed. Redirects can change the last observed destination or strip credentials. This evidence does not prove that bytes reached a peer or reveal dispatch hidden inside a custom transport. Diagnostic reports do not retain headers, credentials, provider bodies, or generated text. A consumer-injected `LLMClient` can support inference diagnostics, but does not establish HTTP status, destination, reachability, or upstream credential-supply facts; Promptkit does not fall back to a built-in transport for HTTP-only diagnostics. ## Request Body The request body always contains `model` and `messages`. The execution target's model takes precedence over the client's configured model, and one must be available. Each ordinary message contains its `role` and string `content`. A cache-controlled message instead uses a text content block containing `type`, `text`, and `cache_control`; an empty cache-control TTL is omitted. Promptkit sends only `developer`, `system`, `user`, and `assistant` roles and does so without provider-specific translation. Tool and deprecated function payloads are outside this text-message contract. A backend or model that rejects an otherwise supported role or context returns its ordinary provider error, which follows the normal generation-error path. The effective direct or prompt-rendered session ID is trimmed, limited to 256 Unicode code points, and sent when nonempty as top-level `session_id`. It is never also sent as a session header. The client conditionally includes: - `temperature`, `max_tokens`, and `top_p` only when selected by a profile or runtime override, including an explicit runtime zero; they are absent when unspecified; - non-empty `service_tier` and effective `reasoning_effort`; an explicitly disabled reasoning setting is empty and therefore omitted; and - `response_format` for JSON Schema structured output, including its name, strict flag, and schema document. Plain JSON validation does not add an object-only response constraint. The engine resolves backend, profile, and request extra-parameter maps by whole-map replacement rather than key merging. The resulting effective map is merged into the top-level body. Empty keys, values that cannot be encoded, and collisions with these reserved fields are rejected before any provider call: - `model` - `session_id` - `messages` - `temperature` - `max_tokens` - `top_p` - `service_tier` - `reasoning_effort` - `response_format` `backend_id`, `api_key_env`, and resolved credential values are not provider request fields. ## Response Handling Any 2xx response body is limited to 16 MiB (16,777,216 bytes). A larger declared `Content-Length` is rejected before the body is read, and streamed, chunked, or underreported bodies are read through the same bound with at most one additional byte used to detect overflow. A body exactly at the limit is allowed. The body is closed on every outcome and an oversized stream is not drained. The bounded body must contain exactly one OpenAI-compatible JSON response object followed only by JSON whitespace and EOF. The client returns the first choice's explicitly present string message content, including an empty or whitespace-only string, and maps prompt, completion, total, cached, and cache-write token counts. Invalid or truncated JSON, trailing non-whitespace data, a second JSON value, absent choices, missing content, `null` content, non-string content, and size overflow are malformed responses and return no partial result. Each of the five token fields retains whether the provider reported a non-null integer, including zero. Omitted and `null` fields are unknown for observations. A strictly decoded successful envelope can yield observed usage even when a later missing choice or content makes the attempt malformed. Invalid framing and non-2xx error bodies supply no observed usage; provider work may still have occurred. Reported counts are not inferred, reconciled, or proof of complete cost accounting. For a non-2xx status, Promptkit recognizes one JSON document with a top-level object-valued `error` member. Its optional `message` and `type` fields must be strings, and `code` may be a string or JSON number. Valid supported fields are handled independently, numeric codes retain their JSON number text, and unknown fields are ignored. Missing, invalid, malformed, or multiply framed envelopes contribute no provider detail. Non-success bodies have a 65,536-byte limit. A larger declared `Content-Length` is not read; otherwise the client reads at most one additional byte to detect streamed or underreported overflow. Empty, unreadable, oversized, malformed, and unrecognized bodies retain only the received status. The body is always closed and no oversized stream is drained beyond that probe. Extracted strings are made valid UTF-8, trimmed, and converted to one line by collapsing Unicode whitespace, control, and format-character runs. Blank values are omitted. Codes and types longer than 256 Unicode code points are omitted; messages longer than 4,096 code points are truncated at a code-point boundary with an ellipsis inside the limit. Promptkit never exposes raw bodies, headers, endpoints, credentials, request data, schemas, generated content, or unsupported provider metadata through this handling. An outbound `http.Client.Do` failure retains both Promptkit's request-failure identity and the exact transport error for `errors.Is` and `errors.As` checks. The rendered error does not include the selected endpoint, request headers, request content, credentials, or provider body. ## Transient Status Retries The built-in path retries only HTTP 429, 500, 502, 503, and 504. It makes two additional attempts by default; engine configuration or an individual run can choose a different nonnegative count, including zero for single-attempt behavior. The same budget applies separately to initial generation and each structural repair generation. An injected `LLMClient` is dispatched once by Promptkit regardless of this setting. A custom `http.Client` on the built-in path remains subject to these retries. Each retry uses equal jitter between half and all of an exponential local delay, starting at 500 milliseconds and capped at 8 seconds. The first `Retry-After` header value sets a minimum not-before time measured from response receipt if it is valid unsigned decimal seconds or an HTTP date. Invalid values use local delay; past dates impose no extra delay. The provider minimum is not capped at 8 seconds. Retry waiting is cancellable and holds no active backend permit. A caller without a deadline can wait a long time for provider timing. The request body, selected URL, and resolved authorization are fixed across attempts of one generation. Each attempt creates a fresh HTTP request and receives its own configured generation and HTTP-client timeout after capacity admission. The caller context covers every attempt and wait without a reset. Transport failures, malformed responses, noneligible statuses, and invalid generated content are not transient retries. A positive budget ending in eligible statuses reports retry exhaustion with the final provider cause. ## Timeout And Cancellation Timeouts are layered: - the caller context remains the outer cancellation boundary; - a positive generation timeout adds a request context deadline; - zero adds no generation-specific deadline; - a negative generation timeout is invalid; and - the cloned `http.Client` supplies the whole-request transport cap, retaining a positive supplied-client timeout or applying the configured/default timeout when the supplied value is not positive. The earliest applicable caller, generation, or transport deadline controls the request. Caller cancellation retains `context.Canceled`; caller, generation, and whole-request timeout failures retain `context.DeadlineExceeded`, together with the request-failure identity. Constructing the internal client does not mutate a supplied `http.Client`.