120 lines
3.4 KiB
Markdown
120 lines
3.4 KiB
Markdown
# OpenAI-Compatible LLM Integration
|
|
|
|
## Scope
|
|
|
|
This document defines the external LLM endpoint contract Audita currently uses.
|
|
|
|
It covers:
|
|
- endpoint and auth expectations;
|
|
- structured request and response shape;
|
|
- retry and timeout behavior;
|
|
- diagnostics and secret redaction.
|
|
|
|
For user-facing CLI flags and config keys, see [`docs/cli.md`](../cli.md) and [`docs/config.md`](../config.md).
|
|
|
|
## Endpoint Contract
|
|
|
|
Audita sends HTTPS `POST` requests to:
|
|
|
|
- `<base_url>/chat/completions`
|
|
|
|
`base_url` comes from primary or validation LLM config and is required.
|
|
|
|
## Authentication Contract
|
|
|
|
When an API key is configured, Audita sends:
|
|
|
|
- `Authorization: Bearer <api_key>`
|
|
|
|
When no API key is configured, the `Authorization` header is omitted.
|
|
|
|
## Request Shape
|
|
|
|
Audita sends a chat-completions payload with:
|
|
- `model`;
|
|
- `messages` (role/content pairs);
|
|
- `response_format` using JSON Schema strict mode.
|
|
|
|
Representative shape:
|
|
|
|
```json
|
|
{
|
|
"model": "example-model",
|
|
"messages": [
|
|
{"role": "system", "content": "..."},
|
|
{"role": "user", "content": "..."}
|
|
],
|
|
"response_format": {
|
|
"type": "json_schema",
|
|
"json_schema": {
|
|
"name": "correction_set",
|
|
"strict": true,
|
|
"schema": {"type": "object"}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Behavioral requirements enforced by Audita:
|
|
- `model` must resolve to a non-empty value;
|
|
- each message must have non-empty `role` and `content`;
|
|
- `response_format.type` is always `json_schema`;
|
|
- `response_format.json_schema.name` and `schema` must be present;
|
|
- request schema JSON must be valid JSON.
|
|
|
|
## Response Handling Contract
|
|
|
|
Audita expects a successful JSON response with at least one choice and assistant content that can be interpreted as JSON.
|
|
|
|
Supported assistant content forms:
|
|
- string containing JSON;
|
|
- raw JSON value.
|
|
|
|
Audita then decodes the JSON against the expected structured output type.
|
|
|
|
Current structured schema identities used by Audita runtime:
|
|
- `correction_set`
|
|
- `validator_decision_set`
|
|
|
|
## Retries and Timeouts
|
|
|
|
Retry behavior:
|
|
- default max retries is `3` when unset;
|
|
- retries apply to retryable transport/decode/server-side errors;
|
|
- HTTP `429` and `5xx` responses are retryable;
|
|
- retry stops immediately when context is canceled or deadline expires.
|
|
|
|
Timeout behavior:
|
|
- request timeout is derived from configured LLM timeout settings;
|
|
- timeout/cancellation propagate through HTTP requests and return nonzero process failures.
|
|
|
|
## Error Behavior
|
|
|
|
Non-2xx responses fail the request.
|
|
|
|
Error message extraction behavior:
|
|
- if provider JSON includes `error.message`, Audita surfaces that message;
|
|
- else if provider JSON includes top-level `message`, Audita surfaces that;
|
|
- otherwise Audita surfaces status code plus response body text.
|
|
|
|
Malformed or incompatible structured responses fail safely and are surfaced as runtime errors or validator/proposal warnings depending on call site.
|
|
|
|
## Secret Redaction
|
|
|
|
Configured LLM secrets are redacted from:
|
|
- surfaced adapter/runtime errors;
|
|
- LLM diagnostics request/response/error artifacts;
|
|
- effective config/report artifacts that include LLM configuration material.
|
|
|
|
Redaction marker:
|
|
- `[REDACTED]`
|
|
|
|
## Compatibility Boundaries
|
|
|
|
This integration documentation applies only to the implemented OpenAI-compatible chat completions flow.
|
|
|
|
Not part of current behavior:
|
|
- provider SDK integration;
|
|
- non-OpenAI-compatible API contracts;
|
|
- server-side model routing features beyond explicitly configured model/base URL.
|