129 lines
3.2 KiB
Markdown
129 lines
3.2 KiB
Markdown
# OpenAI-Compatible Structured Output
|
|
|
|
This document describes the external LLM provider contract implemented by the
|
|
production Notarius LLM client.
|
|
|
|
## Provider
|
|
|
|
- Provider key: `openai-compatible`
|
|
- HTTP method: `POST`
|
|
- Endpoint: `<base_url>/chat/completions`
|
|
- Request body: JSON
|
|
- Response mode: chat completions with structured JSON schema output
|
|
|
|
`base_url` is trimmed of trailing slashes before `/chat/completions` is
|
|
appended. Configure provider settings in [Configuration](../config.md).
|
|
|
|
## Request
|
|
|
|
The client sends a JSON object with:
|
|
|
|
```json
|
|
{
|
|
"model": "configured-model",
|
|
"messages": [
|
|
{
|
|
"role": "system",
|
|
"content": "..."
|
|
},
|
|
{
|
|
"role": "user",
|
|
"content": "..."
|
|
}
|
|
],
|
|
"response_format": {
|
|
"type": "json_schema",
|
|
"json_schema": {
|
|
"name": "schema_name",
|
|
"strict": true,
|
|
"schema": {}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Implemented request behavior:
|
|
|
|
- `model` comes from the structured completion request when set, otherwise from
|
|
the configured LLM profile.
|
|
- `messages` must be non-empty; each role and content must be non-empty after
|
|
trimming.
|
|
- `response_format.type` is always `json_schema`.
|
|
- `response_format.json_schema.strict` is always `true`.
|
|
- `response_format.json_schema.name` and `schema` come from the extractor or
|
|
validator making the call.
|
|
|
|
If an API key is configured, the client sends:
|
|
|
|
```text
|
|
Authorization: Bearer <api-key>
|
|
```
|
|
|
|
The client always sends `Content-Type: application/json`.
|
|
|
|
## Response
|
|
|
|
The client expects a JSON response with at least one choice:
|
|
|
|
```json
|
|
{
|
|
"model": "provider-model",
|
|
"choices": [
|
|
{
|
|
"message": {
|
|
"content": "{\"field\":\"value\"}"
|
|
}
|
|
}
|
|
],
|
|
"usage": {
|
|
"prompt_tokens": 10,
|
|
"completion_tokens": 5,
|
|
"total_tokens": 15
|
|
}
|
|
}
|
|
```
|
|
|
|
`choices[0].message.content` may be either:
|
|
|
|
- a JSON string whose contents are valid JSON; or
|
|
- raw JSON.
|
|
|
|
The decoded content is unmarshaled into the caller-provided structured output
|
|
target. If `usage` is present, prompt, completion, and total token counts are
|
|
copied into the completion response.
|
|
|
|
## Errors And Retries
|
|
|
|
The client validates base URL, model, response schema name, response schema
|
|
JSON, messages, and output target before or during the call.
|
|
|
|
Retryable failures:
|
|
|
|
- HTTP request failure;
|
|
- response body read failure;
|
|
- HTTP `429`;
|
|
- HTTP `5xx`;
|
|
- malformed provider response envelope;
|
|
- missing choices;
|
|
- missing, empty, or invalid assistant JSON content;
|
|
- structured-output decode failure.
|
|
|
|
Non-retryable provider status codes include non-`429` `4xx` responses.
|
|
|
|
Provider error bodies are parsed for `error.message` or `message` when present.
|
|
Configured API key values and bearer-token values are redacted from returned
|
|
provider errors.
|
|
|
|
## Timeouts And Concurrency
|
|
|
|
The configured profile timeout is applied per provider request when greater
|
|
than zero. Context cancellation is respected.
|
|
|
|
The production CLI wraps the provider client with the LLM scheduler. Effective
|
|
concurrency is described in [LLM runtime internals](../internal/llm.md).
|
|
|
|
## Limits
|
|
|
|
This contract documents only the fields the implemented client sends and reads.
|
|
Provider-specific extensions are ignored unless they affect those fields.
|