40 lines
2.3 KiB
Markdown
40 lines
2.3 KiB
Markdown
# Audita LLM Runtime
|
|
|
|
## Client Boundary
|
|
|
|
`llm.NewRuntime` assembles one PromptKit engine for both roles. It loads embedded
|
|
prompt and schema assets, registers application backends, and inspects the
|
|
selected proposal and validation profiles offline. A role-bound `llm.RoleClient`
|
|
implements `structuredoutput.Client`; proposal and validator contracts alias that
|
|
interface. Both roles share engine-local backend capacity.
|
|
|
|
For each logical call, the adapter starts an operation context and a diagnostic
|
|
record before preparation. It supplies the reserved hardening text, the selected
|
|
profile and resolved backend credential, and the complete embedded schema/repair
|
|
contract. `PrepareExecution` produces the exact messages and hashes retained in
|
|
an Audita snapshot. The adapter checks the prepared backend and endpoint before
|
|
`RunPrepared`. It records available final content, usage, model, and profile
|
|
metadata and decodes only a validated result. Call hooks expose logical
|
|
start/finish records and PromptKit execution events for debug utilization.
|
|
|
|
The [integration contract](../integrations/openai-compatible-llm.md) describes
|
|
wire ownership and failure behavior. `structuredoutput.Failure` has a safe
|
|
message, fixed reason code, typed cause, and recoverable/fatal/canceled class.
|
|
Counts distinguish provider attempts, transient retries, and structural
|
|
repairs, including on errors. Secrets from all configured backends, including unselected
|
|
ones, are available to the diagnostics redaction boundary.
|
|
|
|
## Orchestration
|
|
|
|
CLI processing supplies two role clients from the same engine to the runner.
|
|
PromptKit's per-backend admission applies to both roles and every generation,
|
|
retry, and repair. The runner still processes module instances in order,
|
|
collects concurrent section work deterministically, and cancels remaining
|
|
section work only on fatal failures. `llm.DiagnosticsWriter` remains the protected
|
|
request/response artifact writer. `llm.ObservationCollector` captures metadata
|
|
events under a short mutex and freezes them after runner work joins. Its logical
|
|
records include preparation failures; summary usage comes from terminal events
|
|
once per call and retains knownness and completeness for each field. Runner wall
|
|
and validator timings remain separate from provider attempts. Ordinary reports
|
|
derive compact runner outcomes; full observations are debug-only.
|