scriptorium/docs/integrations/openai-compatible-chat.md

# OpenAI-Compatible Chat Integration

## Scope

This document defines the outbound LLM contract implemented by `internal/llm/openai_compatible_client.go`.

It documents only fields and behaviors currently serialized by code.

## Endpoint Construction

Request endpoint is built as:

1. choose base URL:
   - `GenerateRequest.Target.Endpoint` if set
   - otherwise client config `BaseURL`
2. trim trailing slash
3. append `/chat/completions`

Example:

- base URL: `http://localhost:8000/v1`
- final URL: `http://localhost:8000/v1/chat/completions`

## Request Fields Sent

Serialized JSON fields:

- `model` (required after fallback resolution)
- `session_id` (only when the rendered prompt includes a non-empty session ID)
- `messages` (rendered prompt messages)
- `temperature` (when non-zero, or when explicitly overridden to zero)
- `max_tokens` (when non-zero, or when explicitly overridden to zero)
- `top_p` (when non-zero, or when explicitly overridden to zero)
- `service_tier` (only when non-empty)
- `reasoning_effort` (only when non-empty)
- `response_format` (only when structured output is provided)
- profile/request `extra_params` as additional provider-specific top-level fields

`service_tier` is provider-specific. OpenRouter currently documents request values such as `flex` and `priority`; Scriptorium forwards any non-empty configured value and lets the backend validate support.

`reasoning_effort` is provider-specific. Scriptorium forwards any non-empty configured value as top-level `reasoning_effort` and lets the backend validate support.

`extra_params` are flattened into the outbound JSON object. They are not wrapped in an `extra_params` object:

```json
{
  "model": "gpt-4o-mini",
  "messages": [
    {
      "role": "user",
      "content": "rendered text"
    }
  ],
  "provider_route": "primary",
  "provider_options": {
    "retry_budget": 2
  }
}
```

`extra_params` values must be JSON-compatible. Supported value shapes include strings, numbers, booleans, objects, and arrays.

Reserved `extra_params` keys are rejected before the HTTP request is made:

- `model`
- `session_id`
- `messages`
- `temperature`
- `max_tokens`
- `top_p`
- `service_tier`
- `reasoning_effort`
- `response_format`

Empty `extra_params` keys and values that cannot be encoded as JSON are also rejected before the HTTP request is made.

`session_id` is rendered from prompt YAML using request variables and serialized as a top-level JSON request field. Scriptorium does not send an `x-session-id` header. Empty rendered session IDs are omitted, and values longer than 256 characters are rejected before the HTTP request.

Messages without prompt cache control serialize with string `content`:

```json
{
  "role": "system",
  "content": "rendered text"
}
```

Messages with prompt cache control serialize as a single text content-block array:

```json
{
  "role": "system",
  "content": [
    {
      "type": "text",
      "text": "rendered text",
      "cache_control": {
        "type": "ephemeral",
        "ttl": "1h"
      }
    }
  ]
}
```

When cache-control `ttl` is unset in the prompt definition, `ttl` is omitted from the outbound payload.

Structured output is currently `json_schema` only, serialized as:

```json
{
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "...",
      "strict": true,
      "schema": {"type": "object"}
    }
  }
}
```

## Authentication Header

If `Target.APIKeyEnv` is set:

- resolve environment variable value at request time
- set `Authorization: Bearer <value>`

If the environment variable is unset/empty:

- request fails before HTTP call (`ErrInvalidRequest`)

If `Target.APIKeyEnv` is empty:

- no `Authorization` header is sent

## Timeout Behavior

Base timeout comes from client configuration.

Per-request override:

- if `Target.TimeoutSeconds > 0`, use that value for request timeout
- if `Target.TimeoutSeconds == 0` and the value came from an explicit request override, disable the HTTP client timeout
- if `Target.TimeoutSeconds < 0`, request is rejected (`ErrInvalidRequest`)

## Response Expectations

Expected successful response shape (subset used):

- `choices[0].message.content`
- `usage.prompt_tokens`
- `usage.completion_tokens`
- `usage.total_tokens`
- `usage.prompt_tokens_details.cached_tokens` (optional)
- `usage.cache_write_tokens` (optional)

Absent cache usage fields are treated as zero. Parsed cache usage is exposed through run results and adapter response surfaces as:

- `cached_tokens`
- `cache_write_tokens`

Malformed response conditions include:

- invalid JSON
- empty `choices`
- empty `choices[0].message.content`

Malformed responses return `ErrMalformedResponse`.

## Error Handling

- network/request-construction failures: `ErrRequestFailed`
- non-2xx HTTP status: `ErrUnexpectedStatus` (includes status code and trimmed response body snippet)
- malformed response shape/content: `ErrMalformedResponse`

## Unsupported Or Non-Serialized Fields

The client does not serialize top-level `cache_control`.

No built-in retries, tool-calls, or multi-request payload modes are implemented in this client.

## Relationship To Runner

When prompt validation mode is `json_schema`, runner prepares a structured-output schema spec and passes it to the client as `StructuredOutput`.

The client only serializes the provider request payload; it does not load schema files itself.