61 lines
3.6 KiB
Markdown
61 lines
3.6 KiB
Markdown
# Audita LLM Runtime
|
|
|
|
## Client Boundary
|
|
|
|
`llm.NewRuntime` assembles one PromptKit engine for both roles. PromptKit's
|
|
catalogs and defaults are followed by Audita's embedded prompt and schema
|
|
inputs, operator documents from `profiles_dir`, and the highest-priority
|
|
application backend mapping. Audita owns backend endpoints, capacity settings,
|
|
and resolved credentials; native PromptKit layering owns profile inheritance,
|
|
prompt overrides, and provider-specific fields. Imports are disabled. The
|
|
runtime inspects both selected roles offline, requiring complete model and
|
|
endpoint values and rejecting competing profile credential lookups. A
|
|
role-bound `llm.RoleClient` implements `structuredoutput.Client`; proposal and
|
|
validator contracts alias that interface. Both roles share engine-local backend
|
|
capacity.
|
|
|
|
Successful construction captures the final definitions and referenced message
|
|
contents. The runtime does not reload operator documents or message files during
|
|
calls. If final engine construction fails, an embedded-only diagnostic
|
|
construction distinguishes shipped asset failures from invalid operator input.
|
|
An asset failure remains an asset error; when embedded construction succeeds,
|
|
the original failure remains a configuration error. The diagnostic engine
|
|
never replaces or executes the requested configuration, and caller cancellation
|
|
skips this path. The captured inspection also supplies an internal source
|
|
ledger: all loaded host YAML plus effective, source-relative host
|
|
`content_file` resources are tracked for output-collision checks.
|
|
|
|
For each logical call, the adapter starts an operation context and a diagnostic
|
|
record before preparation. It supplies the reserved hardening text, the selected
|
|
profile and resolved backend credential, and the complete embedded schema/repair
|
|
contract. `PrepareExecution` produces the exact messages and hashes retained in
|
|
an Audita snapshot. The adapter checks the prepared backend and endpoint before
|
|
`RunPrepared`. It records available final content, usage, model, and profile
|
|
metadata and decodes only a validated result. Call hooks expose logical
|
|
start/finish records and PromptKit execution events for debug utilization.
|
|
|
|
The [integration contract](../integrations/openai-compatible-llm.md) describes
|
|
wire ownership and failure behavior. `structuredoutput.Failure` has a safe
|
|
message, fixed reason code, typed cause, and recoverable/fatal/canceled class.
|
|
Counts distinguish provider attempts, transient retries, and structural
|
|
repairs, including on errors. Secrets from all configured backends, including unselected
|
|
ones, are available to the diagnostics redaction boundary.
|
|
|
|
CLI explain output and debug configuration origins share a value-free projection
|
|
of effective field origins. The [CLI contract](../cli.md) defines its optional
|
|
inheritance evidence fields.
|
|
|
|
## Orchestration
|
|
|
|
CLI processing supplies two role clients from the same engine to the runner.
|
|
PromptKit's per-backend admission applies to both roles and every generation,
|
|
retry, and repair. The runner still processes module instances in order,
|
|
collects concurrent section work deterministically, and cancels remaining
|
|
section work only on fatal failures. `llm.DiagnosticsWriter` remains the protected
|
|
request/response artifact writer. `llm.ObservationCollector` captures metadata
|
|
events under a short mutex and freezes them after runner work joins. Its logical
|
|
records include preparation failures; summary usage comes from terminal events
|
|
once per call and retains knownness and completeness for each field. Runner wall
|
|
and validator timings remain separate from provider attempts. Ordinary reports
|
|
derive compact runner outcomes; full observations are debug-only.
|