OpenAI-Compatible Structured Output

This document describes the external LLM provider contract implemented by the production Notarius LLM client.

Provider

Provider key: openai-compatible
HTTP method: POST
Endpoint: <base_url>/chat/completions
Request body: JSON
Response mode: chat completions with structured JSON schema output

base_url is trimmed of trailing slashes before /chat/completions is appended. Configure provider settings in Configuration.

Request

The client sends a JSON object with:

{
  "model": "configured-model",
  "messages": [
    {
      "role": "system",
      "content": "..."
    },
    {
      "role": "user",
      "content": "..."
    }
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "schema_name",
      "strict": true,
      "schema": {}
    }
  }
}

Implemented request behavior:

model comes from the structured completion request when set, otherwise from the configured LLM profile.
messages must be non-empty; each role and content must be non-empty after trimming.
response_format.type is always json_schema.
response_format.json_schema.strict is always true.
response_format.json_schema.name and schema come from the extractor or validator making the call.

If an API key is configured, the client sends:

Authorization: Bearer <api-key>

The client always sends Content-Type: application/json.

Response

The client expects a JSON response with at least one choice:

{
  "model": "provider-model",
  "choices": [
    {
      "message": {
        "content": "{\"field\":\"value\"}"
      }
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 5,
    "total_tokens": 15
  }
}

choices[0].message.content may be either:

a JSON string whose contents are valid JSON; or
raw JSON.

The decoded content is unmarshaled into the caller-provided structured output target. If usage is present, prompt, completion, and total token counts are copied into the completion response.

Errors And Retries

The client validates base URL, model, response schema name, response schema JSON, messages, and output target before or during the call.

Retryable failures:

HTTP request failure;
response body read failure;
HTTP 429;
HTTP 5xx;
malformed provider response envelope;
missing choices;
missing, empty, or invalid assistant JSON content;
structured-output decode failure.

Non-retryable provider status codes include non-429 4xx responses.

Provider error bodies are parsed for error.message or message when present. Configured API key values and bearer-token values are redacted from returned provider errors.

Timeouts And Concurrency

The configured profile timeout is applied per provider request when greater than zero. Context cancellation is respected.

The production CLI wraps the provider client with the LLM scheduler. Effective concurrency is described in LLM runtime internals.

Limits

This contract documents only the fields the implemented client sends and reads. Provider-specific extensions are ignored unless they affect those fields.

3.2 KiB Raw Blame History