OpenAI-Compatible LLM Integration

This document owns the external protocol used by Audita's structured LLM adapter. Settings, inheritance, defaults, and credential supply belong to configuration; implementation and scheduling belong to LLM runtime internals.

Endpoint and Authentication

Audita sends an HTTP POST to the configured base URL with trailing slashes removed and /chat/completions appended. Supply an HTTP or HTTPS base URL that makes this the intended endpoint; the adapter does not require HTTPS in code. Requests have Content-Type: application/json. A non-empty configured API key adds Authorization: Bearer <api_key>; otherwise that header is omitted.

The endpoint must support the configured model and structured JSON-schema responses. Local configuration validation does not probe that support.

Request Shape

The payload contains model, an array of messages with string role/content, and response_format. The structured-output portion has this illustrative shape:

{
  "type": "json_schema",
  "json_schema": {
    "name": "correction_set",
    "strict": true,
    "schema": {"type": "object"}
  }
}

The actual schema is supplied for the call; the fragment above is not a complete request schema. Runtime calls use correction_set and validator_decision_set identities. The adapter requires a non-empty resolved model, non-empty message roles/content, a schema name, and valid schema JSON. It sends strict mode to the provider; local typed decoding and proposal/decision checks still apply afterward.

Response Handling Contract

A successful response must be a JSON envelope with at least one choices item. Audita reads the first choice's message.content, accepting a non-empty string containing JSON or a non-null raw JSON value that can decode into the expected output type. Empty content, missing choices, and incompatible payloads fail. The adapter also reads optional model and token-usage metadata.

There is no streaming protocol or provider SDK dependency in this integration. Audita uses this chat-completions contract, not other provider API shapes.

Retries and Timeouts

The effective adapter retry count is additional attempts after the initial request; see configuration for the current zero-value behavior. Transport/read errors, malformed response envelopes/content, typed decode failures, HTTP 429, and server-error responses are retryable. Other non-success HTTP responses and a response missing choices are not retried by the adapter. Cancellation or an expired context stops retrying.

The configured HTTP timeout applies per request; it is not a whole-process time budget. A parent process can impose its own overall timeout. See configuration for timeout and retry values and subprocess guidance for parent responsibilities.

Error Behavior

For non-success responses, error detail comes from provider error.message, then a top-level message, otherwise the HTTP status and response body. Configured keys are redacted from surfaced errors and LLM diagnostic payloads.

Malformed structured responses can become section/batch warnings and rejected proposals rather than fatal run errors, depending on the call site. Other adapter failures can terminate processing with partial results. Consult the report contract for those outcomes and operations for handling diagnostic contents.

3.5 KiB Raw Blame History