3.6 KiB
Audita LLM Runtime
Client Boundary
llm.NewRuntime assembles one PromptKit engine for both roles. PromptKit's
catalogs and defaults are followed by Audita's embedded prompt and schema
inputs, operator documents from profiles_dir, and the highest-priority
application backend mapping. Audita owns backend endpoints, capacity settings,
and resolved credentials; native PromptKit layering owns profile inheritance,
prompt overrides, and provider-specific fields. Imports are disabled. The
runtime inspects both selected roles offline, requiring complete model and
endpoint values and rejecting competing profile credential lookups. A
role-bound llm.RoleClient implements structuredoutput.Client; proposal and
validator contracts alias that interface. Both roles share engine-local backend
capacity.
Successful construction captures the final definitions and referenced message
contents. The runtime does not reload operator documents or message files during
calls. If final engine construction fails, an embedded-only diagnostic
construction distinguishes shipped asset failures from invalid operator input.
An asset failure remains an asset error; when embedded construction succeeds,
the original failure remains a configuration error. The diagnostic engine
never replaces or executes the requested configuration, and caller cancellation
skips this path. The captured inspection also supplies an internal source
ledger: all loaded host YAML plus effective, source-relative host
content_file resources are tracked for output-collision checks.
For each logical call, the adapter starts an operation context and a diagnostic
record before preparation. It supplies the reserved hardening text, the selected
profile and resolved backend credential, and the complete embedded schema/repair
contract. PrepareExecution produces the exact messages and hashes retained in
an Audita snapshot. The adapter checks the prepared backend and endpoint before
RunPrepared. It records available final content, usage, model, and profile
metadata and decodes only a validated result. Call hooks expose logical
start/finish records and PromptKit execution events for debug utilization.
The integration contract describes
wire ownership and failure behavior. structuredoutput.Failure has a safe
message, fixed reason code, typed cause, and recoverable/fatal/canceled class.
Counts distinguish provider attempts, transient retries, and structural
repairs, including on errors. Secrets from all configured backends, including unselected
ones, are available to the diagnostics redaction boundary.
CLI explain output and debug configuration origins share a value-free projection of effective field origins. The CLI contract defines its optional inheritance evidence fields.
Orchestration
CLI processing supplies two role clients from the same engine to the runner.
PromptKit's per-backend admission applies to both roles and every generation,
retry, and repair. The runner still processes module instances in order,
collects concurrent section work deterministically, and cancels remaining
section work only on fatal failures. llm.DiagnosticsWriter remains the protected
request/response artifact writer. llm.ObservationCollector captures metadata
events under a short mutex and freezes them after runner work joins. Its logical
records include preparation failures; summary usage comes from terminal events
once per call and retains knownness and completeness for each field. Runner wall
and validator timings remain separate from provider attempts. Ordinary reports
derive compact runner outcomes; full observations are debug-only.