114 lines
3.0 KiB
Markdown
114 lines
3.0 KiB
Markdown
# OpenAI-Compatible Chat Integration
|
|
|
|
## Scope
|
|
|
|
This document defines the outbound LLM contract implemented by `internal/llm/openai_compatible_client.go`.
|
|
|
|
It documents only fields and behaviors currently serialized by code.
|
|
|
|
## Endpoint Construction
|
|
|
|
Request endpoint is built as:
|
|
|
|
1. choose base URL:
|
|
- `GenerateRequest.Target.Endpoint` if set
|
|
- otherwise client config `BaseURL`
|
|
2. trim trailing slash
|
|
3. append `/chat/completions`
|
|
|
|
Example:
|
|
|
|
- base URL: `http://localhost:8000/v1`
|
|
- final URL: `http://localhost:8000/v1/chat/completions`
|
|
|
|
## Request Fields Sent
|
|
|
|
Serialized JSON fields:
|
|
|
|
- `model` (required after fallback resolution)
|
|
- `messages` (role/content pairs from rendered prompt)
|
|
- `temperature` (only when non-zero)
|
|
- `max_tokens` (only when non-zero)
|
|
- `top_p` (only when non-zero)
|
|
- `service_tier` (only when non-empty)
|
|
- `response_format` (only when structured output is provided)
|
|
|
|
`service_tier` is provider-specific. OpenRouter currently documents request values such as `flex` and `priority`; Scriptorium forwards any non-empty configured value and lets the backend validate support.
|
|
|
|
Structured output is currently `json_schema` only, serialized as:
|
|
|
|
```json
|
|
{
|
|
"response_format": {
|
|
"type": "json_schema",
|
|
"json_schema": {
|
|
"name": "...",
|
|
"strict": true,
|
|
"schema": {"type": "object"}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
## Authentication Header
|
|
|
|
If `Target.APIKeyEnv` is set:
|
|
|
|
- resolve environment variable value at request time
|
|
- set `Authorization: Bearer <value>`
|
|
|
|
If the environment variable is unset/empty:
|
|
|
|
- request fails before HTTP call (`ErrInvalidRequest`)
|
|
|
|
If `Target.APIKeyEnv` is empty:
|
|
|
|
- no `Authorization` header is sent
|
|
|
|
## Timeout Behavior
|
|
|
|
Base timeout comes from client configuration.
|
|
|
|
Per-request override:
|
|
|
|
- if `Target.TimeoutSeconds > 0`, use that value for request timeout
|
|
- if `Target.TimeoutSeconds < 0`, request is rejected (`ErrInvalidRequest`)
|
|
|
|
## Response Expectations
|
|
|
|
Expected successful response shape (subset used):
|
|
|
|
- `choices[0].message.content`
|
|
- `usage.prompt_tokens`
|
|
- `usage.completion_tokens`
|
|
- `usage.total_tokens`
|
|
|
|
Malformed response conditions include:
|
|
|
|
- invalid JSON
|
|
- empty `choices`
|
|
- empty `choices[0].message.content`
|
|
|
|
Malformed responses return `ErrMalformedResponse`.
|
|
|
|
## Error Handling
|
|
|
|
- network/request-construction failures: `ErrRequestFailed`
|
|
- non-2xx HTTP status: `ErrUnexpectedStatus` (includes status code and trimmed response body snippet)
|
|
- malformed response shape/content: `ErrMalformedResponse`
|
|
|
|
## Unsupported Or Non-Serialized Fields
|
|
|
|
The following fields may exist in profile/effective settings but are not currently serialized into outbound chat-completions payloads:
|
|
|
|
- `reasoning_effort`
|
|
- `extra_params`
|
|
|
|
No built-in retries, tool-calls, or multi-request payload modes are implemented in this client.
|
|
|
|
## Relationship To Runner
|
|
|
|
When prompt validation mode is `json_schema`, runner prepares a structured-output schema spec and passes it to the client as `StructuredOutput`.
|
|
|
|
The client only serializes the provider request payload; it does not load schema files itself.
|