77 lines
3.5 KiB
Markdown
77 lines
3.5 KiB
Markdown
# OpenAI-Compatible LLM Integration
|
|
|
|
This document owns the external protocol used by Audita's structured LLM adapter.
|
|
Settings, inheritance, defaults, and credential supply belong to
|
|
[configuration](../config.md#llm-settings); implementation and scheduling belong to
|
|
[LLM runtime internals](../internal/llm-runtime.md).
|
|
|
|
## Endpoint and Authentication
|
|
|
|
Audita sends an HTTP `POST` to the configured base URL with trailing slashes
|
|
removed and `/chat/completions` appended. Supply an HTTP or HTTPS base URL that
|
|
makes this the intended endpoint; the adapter does not require HTTPS in code.
|
|
Requests have `Content-Type: application/json`. A non-empty configured API key
|
|
adds `Authorization: Bearer <api_key>`; otherwise that header is omitted.
|
|
|
|
The endpoint must support the configured model and structured JSON-schema
|
|
responses. Local configuration validation does not probe that support.
|
|
|
|
## Request Shape
|
|
|
|
The payload contains `model`, an array of `messages` with string `role`/`content`,
|
|
and `response_format`. The structured-output portion has this illustrative shape:
|
|
|
|
```json
|
|
{
|
|
"type": "json_schema",
|
|
"json_schema": {
|
|
"name": "correction_set",
|
|
"strict": true,
|
|
"schema": {"type": "object"}
|
|
}
|
|
}
|
|
```
|
|
|
|
The actual schema is supplied for the call; the fragment above is not a complete
|
|
request schema. Runtime calls use `correction_set` and `validator_decision_set`
|
|
identities. The adapter requires a non-empty resolved model, non-empty message
|
|
roles/content, a schema name, and valid schema JSON. It sends strict mode to the
|
|
provider; local typed decoding and proposal/decision checks still apply afterward.
|
|
|
|
## Response Handling Contract
|
|
|
|
A successful response must be a JSON envelope with at least one `choices` item.
|
|
Audita reads the first choice's `message.content`, accepting a non-empty string
|
|
containing JSON or a non-null raw JSON value that can decode into the expected
|
|
output type. Empty content, missing choices, and incompatible payloads fail.
|
|
The adapter also reads optional `model` and token-usage metadata.
|
|
|
|
There is no streaming protocol or provider SDK dependency in this integration.
|
|
Audita uses this chat-completions contract, not other provider API shapes.
|
|
|
|
## Retries and Timeouts
|
|
|
|
The effective adapter retry count is additional attempts after the initial request;
|
|
see [configuration](../config.md#llm-settings) for the current zero-value behavior.
|
|
Transport/read errors, malformed response envelopes/content, typed decode failures,
|
|
HTTP `429`, and server-error responses are retryable. Other non-success HTTP
|
|
responses and a response missing choices are not retried by the adapter.
|
|
Cancellation or an expired context stops retrying.
|
|
|
|
The configured HTTP timeout applies per request; it is not a whole-process time
|
|
budget. A parent process can impose its own overall timeout. See
|
|
[configuration](../config.md#llm-settings) for timeout and retry values and
|
|
[subprocess guidance](subprocess.md) for parent responsibilities.
|
|
|
|
## Error Behavior
|
|
|
|
For non-success responses, error detail comes from provider `error.message`, then
|
|
a top-level `message`, otherwise the HTTP status and response body. Configured
|
|
keys are redacted from surfaced errors and LLM diagnostic payloads.
|
|
|
|
Malformed structured responses can become section/batch warnings and rejected
|
|
proposals rather than fatal run errors, depending on the call site. Other adapter
|
|
failures can terminate processing with partial results. Consult the
|
|
[report contract](output-files.md#module-outcomes) for those outcomes and
|
|
[operations](../operations.md#artifact-handling) for handling diagnostic contents.
|