Consolidate external documentation contracts
This commit is contained in:
@@ -1,118 +1,67 @@
|
||||
# OpenAI-Compatible Chat Integration
|
||||
|
||||
## Scope
|
||||
This is the outbound wire contract for Scriptorium's OpenAI-compatible
|
||||
chat-completions client.
|
||||
|
||||
This document defines the outbound LLM contract implemented by `internal/llm/openai_compatible_client.go`.
|
||||
## Endpoint And Method
|
||||
|
||||
It documents only fields and behaviors currently serialized by code.
|
||||
Scriptorium uses the request endpoint override when present; otherwise it uses
|
||||
the configured client base URL. It removes a trailing slash and sends
|
||||
`POST /chat/completions`.
|
||||
|
||||
## Endpoint Construction
|
||||
For example, `http://localhost:8000/v1` becomes
|
||||
`http://localhost:8000/v1/chat/completions`.
|
||||
|
||||
Request endpoint is built as:
|
||||
## Request Payload
|
||||
|
||||
1. choose base URL:
|
||||
- `GenerateRequest.Target.Endpoint` if set
|
||||
- otherwise client config `BaseURL`
|
||||
2. trim trailing slash
|
||||
3. append `/chat/completions`
|
||||
The payload always contains `model` and rendered `messages`. It additionally
|
||||
contains these fields when applicable:
|
||||
|
||||
Example:
|
||||
| Field | Inclusion |
|
||||
| --- | --- |
|
||||
| `session_id` | Non-empty rendered prompt session ID. |
|
||||
| `temperature` | Non-zero effective value or an explicit zero override. |
|
||||
| `max_tokens` | Non-zero effective value or an explicit zero override. |
|
||||
| `top_p` | Non-zero effective value or an explicit zero override. |
|
||||
| `service_tier` | Any non-empty configured value. |
|
||||
| `reasoning_effort` | Any non-empty configured value. |
|
||||
| `response_format` | Structured output is requested. |
|
||||
| provider-specific fields | Flattened from `extra_params`. |
|
||||
|
||||
- base URL: `http://localhost:8000/v1`
|
||||
- final URL: `http://localhost:8000/v1/chat/completions`
|
||||
`service_tier` and `reasoning_effort` are forwarded without a provider value
|
||||
catalog; the selected backend decides which values it supports.
|
||||
|
||||
## Request Fields Sent
|
||||
`extra_params` are top-level JSON fields, not a nested object. Keys cannot be
|
||||
empty or collide with `model`, `session_id`, `messages`, `temperature`,
|
||||
`max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or
|
||||
`response_format`. Values must be JSON-serializable.
|
||||
|
||||
Serialized JSON fields:
|
||||
A rendered `session_id` is sent as a top-level JSON field, not as a header.
|
||||
Empty values are omitted. The maximum length is 256 Unicode code points.
|
||||
|
||||
- `model` (required after fallback resolution)
|
||||
- `session_id` (only when the rendered prompt includes a non-empty session ID)
|
||||
- `messages` (rendered prompt messages)
|
||||
- `temperature` (when non-zero, or when explicitly overridden to zero)
|
||||
- `max_tokens` (when non-zero, or when explicitly overridden to zero)
|
||||
- `top_p` (when non-zero, or when explicitly overridden to zero)
|
||||
- `service_tier` (only when non-empty)
|
||||
- `reasoning_effort` (only when non-empty)
|
||||
- `response_format` (only when structured output is provided)
|
||||
- profile/request `extra_params` as additional provider-specific top-level fields
|
||||
|
||||
`service_tier` is provider-specific. OpenRouter currently documents request values such as `flex` and `priority`; Scriptorium forwards any non-empty configured value and lets the backend validate support.
|
||||
|
||||
`reasoning_effort` is provider-specific. Scriptorium forwards any non-empty configured value as top-level `reasoning_effort` and lets the backend validate support.
|
||||
|
||||
`extra_params` are flattened into the outbound JSON object. They are not wrapped in an `extra_params` object:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "gpt-4o-mini",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "rendered text"
|
||||
}
|
||||
],
|
||||
"provider_route": "primary",
|
||||
"provider_options": {
|
||||
"retry_budget": 2
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`extra_params` values must be JSON-compatible. Supported value shapes include strings, numbers, booleans, objects, and arrays.
|
||||
|
||||
Reserved `extra_params` keys are rejected before the HTTP request is made:
|
||||
|
||||
- `model`
|
||||
- `session_id`
|
||||
- `messages`
|
||||
- `temperature`
|
||||
- `max_tokens`
|
||||
- `top_p`
|
||||
- `service_tier`
|
||||
- `reasoning_effort`
|
||||
- `response_format`
|
||||
|
||||
Empty `extra_params` keys and values that cannot be encoded as JSON are also rejected before the HTTP request is made.
|
||||
|
||||
`session_id` is rendered from prompt YAML using request variables and serialized as a top-level JSON request field. Scriptorium does not send an `x-session-id` header. Empty rendered session IDs are omitted, and values longer than 256 characters are rejected before the HTTP request.
|
||||
|
||||
Messages without prompt cache control serialize with string `content`:
|
||||
Messages without cache control use string `content`. A message with cache
|
||||
control uses one text block:
|
||||
|
||||
```json
|
||||
{
|
||||
"role": "system",
|
||||
"content": "rendered text"
|
||||
"content": [{
|
||||
"type": "text",
|
||||
"text": "rendered text",
|
||||
"cache_control": {"type": "ephemeral", "ttl": "1h"}
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
Messages with prompt cache control serialize as a single text content-block array:
|
||||
|
||||
```json
|
||||
{
|
||||
"role": "system",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "rendered text",
|
||||
"cache_control": {
|
||||
"type": "ephemeral",
|
||||
"ttl": "1h"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
When cache-control `ttl` is unset in the prompt definition, `ttl` is omitted from the outbound payload.
|
||||
|
||||
Structured output is currently `json_schema` only, serialized as:
|
||||
When the prompt omits cache-control `ttl`, the payload omits `ttl`.
|
||||
Structured JSON Schema output is sent as:
|
||||
|
||||
```json
|
||||
{
|
||||
"response_format": {
|
||||
"type": "json_schema",
|
||||
"json_schema": {
|
||||
"name": "...",
|
||||
"name": "schema name",
|
||||
"strict": true,
|
||||
"schema": {"type": "object"}
|
||||
}
|
||||
@@ -120,74 +69,40 @@ Structured output is currently `json_schema` only, serialized as:
|
||||
}
|
||||
```
|
||||
|
||||
## Authentication Header
|
||||
## Authentication And Timeout
|
||||
|
||||
If `Target.APIKey` is set:
|
||||
When a direct request API key is present, Scriptorium sends
|
||||
`Authorization: Bearer <key>` and does not read `api_key_env`. Otherwise, it
|
||||
resolves the configured non-empty `api_key_env` at request time and sends the
|
||||
same header. If neither mechanism supplies a key, it sends no
|
||||
`Authorization` header.
|
||||
|
||||
- set `Authorization: Bearer <value>`
|
||||
- do not read `Target.APIKeyEnv`
|
||||
The configured client timeout applies by default. A positive effective
|
||||
`timeout_seconds` replaces it. An explicit request override of zero disables
|
||||
the HTTP-client timeout; negative values are rejected before a request is sent.
|
||||
|
||||
If `Target.APIKey` is empty and `Target.APIKeyEnv` is set:
|
||||
## Response Subset And Failures
|
||||
|
||||
- resolve environment variable value at request time
|
||||
- set `Authorization: Bearer <value>`
|
||||
A successful provider response must supply non-empty
|
||||
`choices[0].message.content`. Scriptorium reads these optional or required
|
||||
usage fields when present:
|
||||
|
||||
If the environment variable is unset/empty:
|
||||
|
||||
- request fails before HTTP call (`ErrInvalidRequest`)
|
||||
|
||||
If both `Target.APIKey` and `Target.APIKeyEnv` are empty:
|
||||
|
||||
- no `Authorization` header is sent
|
||||
|
||||
## Timeout Behavior
|
||||
|
||||
Base timeout comes from client configuration.
|
||||
|
||||
Per-request override:
|
||||
|
||||
- if `Target.TimeoutSeconds > 0`, use that value for request timeout
|
||||
- if `Target.TimeoutSeconds == 0` and the value came from an explicit request override, disable the HTTP client timeout
|
||||
- if `Target.TimeoutSeconds < 0`, request is rejected (`ErrInvalidRequest`)
|
||||
|
||||
## Response Expectations
|
||||
|
||||
Expected successful response shape (subset used):
|
||||
|
||||
- `choices[0].message.content`
|
||||
- `usage.prompt_tokens`
|
||||
- `usage.completion_tokens`
|
||||
- `usage.total_tokens`
|
||||
- `usage.prompt_tokens_details.cached_tokens` (optional)
|
||||
- `usage.cache_write_tokens` (optional)
|
||||
- `usage.prompt_tokens_details.cached_tokens`
|
||||
- `usage.cache_write_tokens`
|
||||
|
||||
Absent cache usage fields are treated as zero. Parsed cache usage is exposed through run results and adapter response surfaces as:
|
||||
Missing cache usage is reported as zero. Invalid JSON, an empty choices array,
|
||||
or empty first-choice content is a malformed provider response. Network and
|
||||
request-construction failures, non-2xx responses, and malformed responses fail
|
||||
the outbound call. Provider response bodies are not exposed by this client.
|
||||
|
||||
- `cached_tokens`
|
||||
- `cache_write_tokens`
|
||||
The client does not implement built-in retries, tool calls, top-level
|
||||
`cache_control`, or multi-request payload modes.
|
||||
|
||||
Malformed response conditions include:
|
||||
## Related References
|
||||
|
||||
- invalid JSON
|
||||
- empty `choices`
|
||||
- empty `choices[0].message.content`
|
||||
|
||||
Malformed responses return `ErrMalformedResponse`.
|
||||
|
||||
## Error Handling
|
||||
|
||||
- network/request-construction failures: `ErrRequestFailed`
|
||||
- non-2xx HTTP status: `ErrUnexpectedStatus` (includes status code; provider response bodies are not included)
|
||||
- malformed response shape/content: `ErrMalformedResponse`
|
||||
|
||||
## Unsupported Or Non-Serialized Fields
|
||||
|
||||
The client does not serialize top-level `cache_control`.
|
||||
|
||||
No built-in retries, tool-calls, or multi-request payload modes are implemented in this client.
|
||||
|
||||
## Relationship To Runner
|
||||
|
||||
When prompt validation mode is `json_schema`, runner prepares a structured-output schema spec and passes it to the client as `StructuredOutput`.
|
||||
|
||||
The client only serializes the provider request payload; it does not load schema files itself.
|
||||
Prompt schema preparation and runner orchestration are described in
|
||||
[runner internals](../internal/runner.md). Prompt and profile configuration is
|
||||
defined by the [configuration reference](../config.md).
|
||||
|
||||
Reference in New Issue
Block a user