91 lines
3.3 KiB
Markdown
91 lines
3.3 KiB
Markdown
# Structured LLM Architecture
|
|
|
|
## Purpose
|
|
|
|
This document describes Audita's structured LLM runtime boundary and adapter behavior.
|
|
|
|
## Why Audita owns the adapter
|
|
|
|
Audita owns a small structured LLM adapter so that core runtime behavior is controlled inside the repository:
|
|
- request construction and schema handling are explicit and testable;
|
|
- retries, timeouts, cancellation, and error redaction are consistent across modules and validators;
|
|
- provider SDK types are not exposed outside the adapter boundary;
|
|
- dependency weight and transitive provider-specific behavior are reduced.
|
|
|
|
At runtime, the rest of Audita depends only on the internal contract:
|
|
- `StructuredLLMClient`
|
|
- `CompleteStructured(ctx, req, out)`
|
|
|
|
## OpenAI-compatible request shape
|
|
|
|
At a conceptual level, Audita sends chat completion requests with:
|
|
- `model`
|
|
- `messages` (role/content pairs)
|
|
- `response_format`:
|
|
- `type = "json_schema"`
|
|
- `json_schema.name` (stable schema name)
|
|
- `json_schema.strict = true`
|
|
- `json_schema.schema` (registered JSON Schema payload)
|
|
|
|
The adapter uses OpenAI-compatible `POST {base_url}/chat/completions` over `net/http`.
|
|
|
|
## Structured response schema registry
|
|
|
|
Structured response schemas are registered in `internal/framework/responseschema` with stable metadata:
|
|
- schema key
|
|
- schema ID
|
|
- schema version
|
|
- schema name (OpenAI-compatible `response_format` name)
|
|
- raw JSON Schema payload
|
|
- SHA-256 hash
|
|
|
|
Current schemas:
|
|
- `correction_set`:
|
|
- id `audita.correction_set`
|
|
- version `v1`
|
|
- name `audita_correction_set_v1`
|
|
- `validator_decision_set`:
|
|
- id `audita.validator_decision_set`
|
|
- version `v1`
|
|
- name `audita_validator_decision_set_v1`
|
|
|
|
## Provider compatibility assumptions
|
|
|
|
Audita assumes an OpenAI-compatible chat-completions endpoint that:
|
|
- accepts message arrays with model selection;
|
|
- accepts `response_format.type = json_schema`;
|
|
- returns a completion with assistant message content and optional usage metadata.
|
|
|
|
Provider-specific differences are expected in strictness and error payload shapes, so the adapter treats provider output as untrusted until locally decoded.
|
|
|
|
## Local decode and validation remain mandatory
|
|
|
|
Provider-level structured output is a transport guardrail, not final validation.
|
|
|
|
After receiving a response, Audita still:
|
|
- decodes assistant content into typed request-specific structs;
|
|
- validates proposal and validator payload invariants locally;
|
|
- enforces deterministic validator/cardinality rules before any transcript application.
|
|
|
|
This protects runtime correctness even when provider responses are malformed, partial, or semantically inconsistent.
|
|
|
|
## Diagnostics and redaction
|
|
|
|
When structured schemas are used, diagnostics metadata records:
|
|
- schema ID
|
|
- schema version
|
|
- schema name
|
|
- schema hash
|
|
|
|
Diagnostics and surfaced errors preserve secret redaction:
|
|
- API keys and bearer tokens are redacted from request/response/error artifacts;
|
|
- redaction is applied before diagnostic files are written.
|
|
|
|
## Runtime behavior guarantees
|
|
|
|
The structured LLM path preserves existing runtime guarantees:
|
|
- bounded LLM call execution through schedulers;
|
|
- context-aware cancellation and timeout propagation;
|
|
- retry behavior for transient failures and retryable malformed structured responses;
|
|
- deterministic module/chunk/proposal/validator behavior outside provider nondeterminism.
|