scriptorium/docs/integrations/openai-compatible-chat.md

OpenAI-Compatible Chat Integration

Scope

This document defines the outbound LLM contract implemented by internal/llm/openai_compatible_client.go.

It documents only fields and behaviors currently serialized by code.

Endpoint Construction

Request endpoint is built as:

1. choose base URL:
   • GenerateRequest.Target.Endpoint if set
   • otherwise client config BaseURL
2. trim trailing slash
3. append /chat/completions

Example:

• base URL: http://localhost:8000/v1
• final URL: http://localhost:8000/v1/chat/completions

Request Fields Sent

Serialized JSON fields:

• model (required after fallback resolution)
• session_id (only when the rendered prompt includes a non-empty session ID)
• messages (rendered prompt messages)
• temperature (when non-zero, or when explicitly overridden to zero)
• max_tokens (when non-zero, or when explicitly overridden to zero)
• top_p (when non-zero, or when explicitly overridden to zero)
• service_tier (only when non-empty)
• reasoning_effort (only when non-empty)
• response_format (only when structured output is provided)
• profile/request extra_params as additional provider-specific top-level fields

service_tier is provider-specific. OpenRouter currently documents request values such as flex and priority; Scriptorium forwards any non-empty configured value and lets the backend validate support.

reasoning_effort is provider-specific. Scriptorium forwards any non-empty configured value as top-level reasoning_effort and lets the backend validate support.

extra_params are flattened into the outbound JSON object. They are not wrapped in an extra_params object:

{
  "model": "gpt-4o-mini",
  "messages": [
    {
      "role": "user",
      "content": "rendered text"
    }
  ],
  "provider_route": "primary",
  "provider_options": {
    "retry_budget": 2
  }
}

extra_params values must be JSON-compatible. Supported value shapes include strings, numbers, booleans, objects, and arrays.

Reserved extra_params keys are rejected before the HTTP request is made:

• model
• session_id
• messages
• temperature
• max_tokens
• top_p
• service_tier
• reasoning_effort
• response_format

Empty extra_params keys and values that cannot be encoded as JSON are also rejected before the HTTP request is made.

session_id is rendered from prompt YAML using request variables and serialized as a top-level JSON request field. Scriptorium does not send an x-session-id header. Empty rendered session IDs are omitted, and values longer than 256 characters are rejected before the HTTP request.

Messages without prompt cache control serialize with string content:

{
  "role": "system",
  "content": "rendered text"
}

Messages with prompt cache control serialize as a single text content-block array:

{
  "role": "system",
  "content": [
    {
      "type": "text",
      "text": "rendered text",
      "cache_control": {
        "type": "ephemeral",
        "ttl": "1h"
      }
    }
  ]
}

When cache-control ttl is unset in the prompt definition, ttl is omitted from the outbound payload.

Structured output is currently json_schema only, serialized as:

{
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "...",
      "strict": true,
      "schema": {"type": "object"}
    }
  }
}

Authentication Header

If Target.APIKeyEnv is set:

• resolve environment variable value at request time
• set Authorization: Bearer <value>

If the environment variable is unset/empty:

• request fails before HTTP call (ErrInvalidRequest)

If Target.APIKeyEnv is empty:

• no Authorization header is sent

Timeout Behavior

Base timeout comes from client configuration.

Per-request override:

• if Target.TimeoutSeconds > 0, use that value for request timeout
• if Target.TimeoutSeconds == 0 and the value came from an explicit request override, disable the HTTP client timeout
• if Target.TimeoutSeconds < 0, request is rejected (ErrInvalidRequest)

Response Expectations

Expected successful response shape (subset used):

• choices[0].message.content
• usage.prompt_tokens
• usage.completion_tokens
• usage.total_tokens
• usage.prompt_tokens_details.cached_tokens (optional)
• usage.cache_write_tokens (optional)

Absent cache usage fields are treated as zero. Parsed cache usage is exposed through run results and adapter response surfaces as:

• cached_tokens
• cache_write_tokens

Malformed response conditions include:

• invalid JSON
• empty choices
• empty choices[0].message.content

Malformed responses return ErrMalformedResponse.

Error Handling

• network/request-construction failures: ErrRequestFailed
• non-2xx HTTP status: ErrUnexpectedStatus (includes status code and trimmed response body snippet)
• malformed response shape/content: ErrMalformedResponse

Unsupported Or Non-Serialized Fields

The client does not serialize top-level cache_control.

No built-in retries, tool-calls, or multi-request payload modes are implemented in this client.

Relationship To Runner

When prompt validation mode is json_schema, runner prepares a structured-output schema spec and passes it to the client as StructuredOutput.

The client only serializes the provider request payload; it does not load schema files itself.