scriptorium/docs/integrations/openai-compatible-chat.md

OpenAI-Compatible Chat Integration

Scope

This document defines the outbound LLM contract implemented by internal/llm/openai_compatible_client.go.

It documents only fields and behaviors currently serialized by code.

Endpoint Construction

Request endpoint is built as:

1. choose base URL:
   • GenerateRequest.Target.Endpoint if set
   • otherwise client config BaseURL
2. trim trailing slash
3. append /chat/completions

Example:

• base URL: http://localhost:8000/v1
• final URL: http://localhost:8000/v1/chat/completions

Request Fields Sent

Serialized JSON fields:

• model (required after fallback resolution)
• messages (role/content pairs from rendered prompt)
• temperature (only when non-zero)
• max_tokens (only when non-zero)
• top_p (only when non-zero)
• service_tier (only when non-empty)
• response_format (only when structured output is provided)

service_tier is provider-specific. OpenRouter currently documents request values such as flex and priority; Scriptorium forwards any non-empty configured value and lets the backend validate support.

Structured output is currently json_schema only, serialized as:

{
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "...",
      "strict": true,
      "schema": {"type": "object"}
    }
  }
}

Authentication Header

If Target.APIKeyEnv is set:

• resolve environment variable value at request time
• set Authorization: Bearer <value>

If the environment variable is unset/empty:

• request fails before HTTP call (ErrInvalidRequest)

If Target.APIKeyEnv is empty:

• no Authorization header is sent

Timeout Behavior

Base timeout comes from client configuration.

Per-request override:

• if Target.TimeoutSeconds > 0, use that value for request timeout
• if Target.TimeoutSeconds < 0, request is rejected (ErrInvalidRequest)

Response Expectations

Expected successful response shape (subset used):

• choices[0].message.content
• usage.prompt_tokens
• usage.completion_tokens
• usage.total_tokens

Malformed response conditions include:

• invalid JSON
• empty choices
• empty choices[0].message.content

Malformed responses return ErrMalformedResponse.

Error Handling

• network/request-construction failures: ErrRequestFailed
• non-2xx HTTP status: ErrUnexpectedStatus (includes status code and trimmed response body snippet)
• malformed response shape/content: ErrMalformedResponse

Unsupported Or Non-Serialized Fields

The following fields may exist in profile/effective settings but are not currently serialized into outbound chat-completions payloads:

• reasoning_effort
• extra_params

No built-in retries, tool-calls, or multi-request payload modes are implemented in this client.

Relationship To Runner

When prompt validation mode is json_schema, runner prepares a structured-output schema spec and passes it to the client as StructuredOutput.

The client only serializes the provider request payload; it does not load schema files itself.