35 lines
1.8 KiB
Markdown
35 lines
1.8 KiB
Markdown
# Audita LLM Runtime
|
|
|
|
## Client Boundary
|
|
|
|
`llm.NewRuntime` assembles one PromptKit engine for both roles. It loads embedded
|
|
prompt and schema assets, registers application backends, and inspects the
|
|
selected proposal and validation profiles offline. A role-bound `llm.RoleClient`
|
|
implements `structuredoutput.Client`; proposal and validator contracts alias that
|
|
interface during migration. Both roles share engine-local backend capacity.
|
|
|
|
For each logical call, the adapter starts an operation context and a diagnostic
|
|
record before preparation. It supplies the reserved hardening text, the selected
|
|
profile and resolved backend credential, and the complete embedded schema/repair
|
|
contract. `PrepareExecution` produces the exact messages and hashes retained in
|
|
an Audita snapshot. The adapter checks the prepared backend and endpoint before
|
|
`RunPrepared`. It records available final content, usage, model, and profile
|
|
metadata and decodes only a validated result. Call hooks expose logical
|
|
start/finish records and PromptKit execution events for the report collector.
|
|
|
|
The [integration contract](../integrations/openai-compatible-llm.md) describes
|
|
wire ownership and failure behavior. `structuredoutput.Failure` has a safe
|
|
message and a typed cause; only exhausted structural repair is a malformed
|
|
output failure. Secrets from all configured backends, including unselected
|
|
ones, are available to the diagnostics redaction boundary.
|
|
|
|
## Current Orchestration
|
|
|
|
The old `llm.Scheduler` and runner instrumentation still exist pending the
|
|
orchestration migration. The final runtime uses PromptKit backend admission,
|
|
retry, and repair rather than that scheduler. `llm.DiagnosticsWriter` remains
|
|
the current protected artifact writer.
|
|
|
|
Focused adapter tests use controlled local HTTP responses. Complete runner and
|
|
CLI integration checks follow the caller and orchestration migration.
|