73 lines
2.3 KiB
Markdown
73 lines
2.3 KiB
Markdown
# Structured LLM Architecture
|
|
|
|
## Scope
|
|
This document describes Audita's structured LLM runtime boundary and adapter behavior.
|
|
|
|
## Runtime boundary
|
|
Production LLM integration depends on the internal contract only:
|
|
- `contracts.StructuredLLMClient`
|
|
- `CompleteStructured(ctx, req, out)`
|
|
|
|
Provider SDK types do not leak past this boundary.
|
|
|
|
## Adapter ownership
|
|
`internal/framework/llm` owns the OpenAI-compatible HTTP adapter and shared LLM runtime utilities.
|
|
|
|
Key responsibilities:
|
|
- request assembly;
|
|
- timeout/cancellation propagation;
|
|
- bounded retry behavior;
|
|
- scheduler integration;
|
|
- provider response decoding;
|
|
- error redaction.
|
|
|
|
## Structured schema registry
|
|
Structured response schemas are registered in `internal/framework/responseschema` and include stable metadata:
|
|
- `id`
|
|
- `version`
|
|
- `name`
|
|
- `json_schema`
|
|
- `sha256`
|
|
|
|
Current schema keys:
|
|
- `correction_set`
|
|
- `validator_decision_set`
|
|
|
|
Schema metadata is attached to diagnostics through `Schema.DiagnosticsMap()`.
|
|
|
|
## Request shape assumptions
|
|
Audita targets OpenAI-compatible chat-completions endpoints and sends structured requests with:
|
|
- model;
|
|
- chat messages;
|
|
- `response_format.type = json_schema`;
|
|
- schema name and JSON schema payload.
|
|
|
|
## Local validation remains mandatory
|
|
Provider schema enforcement is treated as transport-level guardrails.
|
|
|
|
Audita still validates output locally before applying behavior changes:
|
|
- proposal decoding and proposal invariants;
|
|
- validator decision decoding and cardinality checks;
|
|
- deterministic validation and apply-time rules.
|
|
|
|
## Shared malformed-output policy
|
|
Malformed structured-output classification is centralized in `internal/framework/structuredoutput`.
|
|
|
|
Proposal generation and validator execution both use this shared classifier so downgrade behavior cannot drift between the two paths.
|
|
|
|
## Secrets and redaction
|
|
Secret extraction for LLM redaction is centralized in `llm.ConfiguredSecrets(cfg)` and reused by proposal and validator diagnostics writers.
|
|
|
|
Secrets are redacted from:
|
|
- diagnostics artifacts;
|
|
- report artifacts;
|
|
- surfaced adapter/runtime errors.
|
|
|
|
## Concurrency and scheduling
|
|
LLM execution is constrained by composed scheduler limits:
|
|
- total LLM concurrency;
|
|
- proposal LLM concurrency;
|
|
- validation LLM concurrency.
|
|
|
|
The scheduler is FIFO and context-aware so permits are released on success, failure, and cancellation.
|