eric/audita
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Use PromptKit prepared execution for structured model calls

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-24 01:57:25 +00:00
parent 96d3d03bee
commit c8a78bf024
17 changed files with 560 additions and 1366 deletions
Download Patch File Download Diff File Expand all files Collapse all files

97
docs/integrations/openai-compatible-llm.md
View File

@@ -1,76 +1,31 @@
# OpenAI-Compatible LLM Integration
This document owns the external protocol used by Audita's structured LLM adapter.
Settings, inheritance, defaults, and credential supply belong to
[configuration](../config.md#llm-settings); implementation and scheduling belong to
[LLM runtime internals](../internal/llm-runtime.md).
Audita uses the pinned [PromptKit](https://gitea.maximumdirect.net/eric/promptkit)
built-in OpenAI-compatible transport. PromptKit owns chat-completions wire framing,
provider envelope parsing, transient retries, per-generation timeouts, and JSON
Schema repair. A backend's configured base endpoint must support the selected
profile's model and structured JSON-schema responses; local configuration
inspection does not contact the provider.
## Endpoint and Authentication
Audita selects a role profile, supplies its registered backend credential, and
passes the embedded response schema as a complete per-call output contract. It
prepares each call once, checks that the prepared target still matches the
inspected backend and endpoint, then executes that same prepared request. A
profile cannot introduce a separate credential environment lookup. An absent
backend credential permits an unauthenticated local service.
Audita sends an HTTP `POST` to the configured base URL with trailing slashes
removed and `/chat/completions` appended. Supply an HTTP or HTTPS base URL that
makes this the intended endpoint; the adapter does not require HTTPS in code.
Requests have `Content-Type: application/json`. A non-empty configured API key
adds `Authorization: Bearer <api_key>`; otherwise that header is omitted.
`llm.transient_retries` counts additional attempts for eligible transient
failures; explicit zero makes one attempt. `llm.repair_attempts` counts
additional generations for schema-invalid model output. The operation timeout
covers preparation, capacity waiting, retries, generation, and repair. Caller
cancellation or an earlier deadline also applies. See [configuration](../config.md#llm-settings)
for defaults and [PromptKit's consumer contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/consumers/pkg-promptkit.md)
for the pinned protocol and retry details.
The endpoint must support the configured model and structured JSON-schema
responses. Local configuration validation does not probe that support.
## Request Shape
The payload contains `model`, an array of `messages` with string `role`/`content`,
and `response_format`. The structured-output portion has this illustrative shape:
```json
{
"type": "json_schema",
"json_schema": {
"name": "correction_set",
"strict": true,
"schema": {"type": "object"}
}
}
```
The actual schema is supplied for the call; the fragment above is not a complete
request schema. Runtime calls use `correction_set` and `validator_decision_set`
identities. The adapter requires a non-empty resolved model, non-empty message
roles/content, a schema name, and valid schema JSON. It sends strict mode to the
provider; local typed decoding and proposal/decision checks still apply afterward.
## Response Handling Contract
A successful response must be a JSON envelope with at least one `choices` item.
Audita reads the first choice's `message.content`, accepting a non-empty string
containing JSON or a non-null raw JSON value that can decode into the expected
output type. Empty content, missing choices, and incompatible payloads fail.
The adapter also reads optional `model` and token-usage metadata.
There is no streaming protocol or provider SDK dependency in this integration.
Audita uses this chat-completions contract, not other provider API shapes.
## Retries and Timeouts
The effective adapter retry count is additional attempts after the initial request;
see [configuration](../config.md#llm-settings) for the current zero-value behavior.
Transport/read errors, malformed response envelopes/content, typed decode failures,
HTTP `429`, and server-error responses are retryable. Other non-success HTTP
responses and a response missing choices are not retried by the adapter.
Cancellation or an expired context stops retrying.
The configured HTTP timeout applies per request; it is not a whole-process time
budget. A parent process can impose its own overall timeout. See
[configuration](../config.md#llm-settings) for timeout and retry values and
[subprocess guidance](subprocess.md) for parent responsibilities.
## Error Behavior
For non-success responses, error detail comes from provider `error.message`, then
a top-level `message`, otherwise the HTTP status and response body. Configured
keys are redacted from surfaced errors and LLM diagnostic payloads.
Malformed structured responses can become section/batch warnings and rejected
proposals rather than fatal run errors, depending on the call site. Other adapter
failures can terminate processing with partial results. Consult the
[report contract](output-files.md#module-outcomes) for those outcomes and
[operations](../operations.md#artifact-handling) for handling diagnostic contents.
A failed JSON Schema result after repair is classified as malformed generated
output; proposal and validator callers can turn it into a warning. Provider,
transport, malformed HTTP envelope, retry, admission, and deadline failures stay
runtime failures. Audita decodes into domain types only after PromptKit validates
the JSON. Errors exposed to users carry safe messages while retaining typed
causes for internal classification. Diagnostic request/response content is
sensitive and follows [operations guidance](../operations.md#artifact-handling).