155 lines
4.2 KiB
Markdown
155 lines
4.2 KiB
Markdown
# OpenAI-Compatible Chat Integration
|
|
|
|
## Scope
|
|
|
|
This document defines the outbound LLM contract implemented by `internal/llm/openai_compatible_client.go`.
|
|
|
|
It documents only fields and behaviors currently serialized by code.
|
|
|
|
## Endpoint Construction
|
|
|
|
Request endpoint is built as:
|
|
|
|
1. choose base URL:
|
|
- `GenerateRequest.Target.Endpoint` if set
|
|
- otherwise client config `BaseURL`
|
|
2. trim trailing slash
|
|
3. append `/chat/completions`
|
|
|
|
Example:
|
|
|
|
- base URL: `http://localhost:8000/v1`
|
|
- final URL: `http://localhost:8000/v1/chat/completions`
|
|
|
|
## Request Fields Sent
|
|
|
|
Serialized JSON fields:
|
|
|
|
- `model` (required after fallback resolution)
|
|
- `session_id` (only when the rendered prompt includes a non-empty session ID)
|
|
- `messages` (rendered prompt messages)
|
|
- `temperature` (only when non-zero)
|
|
- `max_tokens` (only when non-zero)
|
|
- `top_p` (only when non-zero)
|
|
- `service_tier` (only when non-empty)
|
|
- `response_format` (only when structured output is provided)
|
|
|
|
`service_tier` is provider-specific. OpenRouter currently documents request values such as `flex` and `priority`; Scriptorium forwards any non-empty configured value and lets the backend validate support.
|
|
|
|
`session_id` is rendered from prompt YAML using request variables and serialized as a top-level JSON request field. Scriptorium does not send an `x-session-id` header. Empty rendered session IDs are omitted, and values longer than 256 characters are rejected before the HTTP request.
|
|
|
|
Messages without prompt cache control serialize with string `content`:
|
|
|
|
```json
|
|
{
|
|
"role": "system",
|
|
"content": "rendered text"
|
|
}
|
|
```
|
|
|
|
Messages with prompt cache control serialize as a single text content-block array:
|
|
|
|
```json
|
|
{
|
|
"role": "system",
|
|
"content": [
|
|
{
|
|
"type": "text",
|
|
"text": "rendered text",
|
|
"cache_control": {
|
|
"type": "ephemeral",
|
|
"ttl": "1h"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
When cache-control `ttl` is unset in the prompt definition, `ttl` is omitted from the outbound payload.
|
|
|
|
Structured output is currently `json_schema` only, serialized as:
|
|
|
|
```json
|
|
{
|
|
"response_format": {
|
|
"type": "json_schema",
|
|
"json_schema": {
|
|
"name": "...",
|
|
"strict": true,
|
|
"schema": {"type": "object"}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
## Authentication Header
|
|
|
|
If `Target.APIKeyEnv` is set:
|
|
|
|
- resolve environment variable value at request time
|
|
- set `Authorization: Bearer <value>`
|
|
|
|
If the environment variable is unset/empty:
|
|
|
|
- request fails before HTTP call (`ErrInvalidRequest`)
|
|
|
|
If `Target.APIKeyEnv` is empty:
|
|
|
|
- no `Authorization` header is sent
|
|
|
|
## Timeout Behavior
|
|
|
|
Base timeout comes from client configuration.
|
|
|
|
Per-request override:
|
|
|
|
- if `Target.TimeoutSeconds > 0`, use that value for request timeout
|
|
- if `Target.TimeoutSeconds < 0`, request is rejected (`ErrInvalidRequest`)
|
|
|
|
## Response Expectations
|
|
|
|
Expected successful response shape (subset used):
|
|
|
|
- `choices[0].message.content`
|
|
- `usage.prompt_tokens`
|
|
- `usage.completion_tokens`
|
|
- `usage.total_tokens`
|
|
- `usage.prompt_tokens_details.cached_tokens` (optional)
|
|
- `usage.cache_write_tokens` (optional)
|
|
|
|
Absent cache usage fields are treated as zero. Parsed cache usage is exposed through run results and adapter response surfaces as:
|
|
|
|
- `cached_tokens`
|
|
- `cache_write_tokens`
|
|
|
|
Malformed response conditions include:
|
|
|
|
- invalid JSON
|
|
- empty `choices`
|
|
- empty `choices[0].message.content`
|
|
|
|
Malformed responses return `ErrMalformedResponse`.
|
|
|
|
## Error Handling
|
|
|
|
- network/request-construction failures: `ErrRequestFailed`
|
|
- non-2xx HTTP status: `ErrUnexpectedStatus` (includes status code and trimmed response body snippet)
|
|
- malformed response shape/content: `ErrMalformedResponse`
|
|
|
|
## Unsupported Or Non-Serialized Fields
|
|
|
|
The following fields may exist in profile/effective settings but are not currently serialized into outbound chat-completions payloads:
|
|
|
|
- `reasoning_effort`
|
|
- `extra_params`
|
|
|
|
The client also does not serialize top-level `cache_control`.
|
|
|
|
No built-in retries, tool-calls, or multi-request payload modes are implemented in this client.
|
|
|
|
## Relationship To Runner
|
|
|
|
When prompt validation mode is `json_schema`, runner prepares a structured-output schema spec and passes it to the client as `StructuredOutput`.
|
|
|
|
The client only serializes the provider request payload; it does not load schema files itself.
|