audita/docs/integrations/openai-compatible-llm.md

OpenAI-Compatible LLM Integration

Scope

This document defines the external LLM endpoint contract Audita currently uses.

It covers:

• endpoint and auth expectations;
• structured request and response shape;
• retry and timeout behavior;
• diagnostics and secret redaction.

For user-facing CLI flags and config keys, see docs/cli.md and docs/config.md.

Endpoint Contract

Audita sends HTTPS POST requests to:

• <base_url>/chat/completions

base_url comes from primary or validation LLM config and is required.

Authentication Contract

When an API key is configured, Audita sends:

• Authorization: Bearer <api_key>

When no API key is configured, the Authorization header is omitted.

Request Shape

Audita sends a chat-completions payload with:

• model;
• messages (role/content pairs);
• response_format using JSON Schema strict mode.

Representative shape:

{
  "model": "example-model",
  "messages": [
    {"role": "system", "content": "..."},
    {"role": "user", "content": "..."}
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "correction_set",
      "strict": true,
      "schema": {"type": "object"}
    }
  }
}

Behavioral requirements enforced by Audita:

• model must resolve to a non-empty value;
• each message must have non-empty role and content;
• response_format.type is always json_schema;
• response_format.json_schema.name and schema must be present;
• request schema JSON must be valid JSON.

Response Handling Contract

Audita expects a successful JSON response with at least one choice and assistant content that can be interpreted as JSON.

Supported assistant content forms:

• string containing JSON;
• raw JSON value.

Audita then decodes the JSON against the expected structured output type.

Current structured schema identities used by Audita runtime:

• correction_set
• validator_decision_set

Retries and Timeouts

Retry behavior:

• default max retries is 3 when unset;
• retries apply to retryable transport/decode/server-side errors;
• HTTP 429 and 5xx responses are retryable;
• retry stops immediately when context is canceled or deadline expires.

Timeout behavior:

• request timeout is derived from configured LLM timeout settings;
• timeout/cancellation propagate through HTTP requests and return nonzero process failures.

Error Behavior

Non-2xx responses fail the request.

Error message extraction behavior:

• if provider JSON includes error.message, Audita surfaces that message;
• else if provider JSON includes top-level message, Audita surfaces that;
• otherwise Audita surfaces status code plus response body text.

Malformed or incompatible structured responses fail safely and are surfaced as runtime errors or validator/proposal warnings depending on call site.

Secret Redaction

Configured LLM secrets are redacted from:

• surfaced adapter/runtime errors;
• LLM diagnostics request/response/error artifacts;
• effective config/report artifacts that include LLM configuration material.

Redaction marker:

• [REDACTED]

Compatibility Boundaries

This integration documentation applies only to the implemented OpenAI-compatible chat completions flow.

Not part of current behavior:

• provider SDK integration;
• non-OpenAI-compatible API contracts;
• server-side model routing features beyond explicitly configured model/base URL.