audita/docs/internal/llm-runtime.md

Audita LLM Runtime

Scope

This document describes the structured LLM runtime and scheduler behavior.

Client Boundary

All runtime LLM calls go through contracts.StructuredLLMClient.

Primary adapter:

• internal/framework/llm/OpenAICompatibleClient

Adapter and Response Schema Registry

The adapter implements the LLM wire contract. It maps internal structured requests to HTTP messages and decodes provider content into the supplied Go output target. Endpoint, authentication, response shapes, and retry conditions are defined in that contract rather than repeated here.

internal/framework/responseschema stores the per-call response schema definitions and diagnostic metadata. Proposal generation and validation choose their schema before invoking the client. Provider strict mode does not replace local checks.

Error Classification

The adapter marks retryable errors internally and uses context state and attempt count to decide whether to retry. It sanitizes errors before returning them. internal/framework/structuredoutput classifies malformed output for proposal and validator call sites, which can turn those errors into warnings and rejected section/batch work. Other errors propagate through the runner.

Scheduling and Concurrency

internal/framework/llm/Scheduler provides FIFO, context-aware permit gating.

CLI orchestration composes global and stage schedulers; runner instrumentation measures them. Limit definitions and inheritance belong to configuration.

Scheduler release is guarded to avoid permit leaks on cancellation/error. Runner collects utilization observations; their serialized shape is defined by the output-file contract.

Diagnostics and Redaction

internal/framework/llm/DiagnosticsWriter writes request/response/error artifacts.

Configured secrets are derived from llm.ConfiguredSecrets(cfg) and redacted from:

• diagnostics payloads;
• surfaced runtime/adapter errors.

Changing LLM Integration or Scheduling

Review client, scheduler, redaction, and structured-output malformed-response coverage alongside runner orchestration and utilization tests. Use the testing policy to select sufficient checks; preserve the LLM integration contract and architecture invariants.

Key Tests

• internal/framework/llm/openai_compatible_client_test.go
• internal/framework/llm/scheduler_test.go
• internal/framework/llm/diagnostics_test.go
• internal/framework/responseschema/registry_test.go
• internal/framework/structuredoutput/malformed_test.go
• internal/framework/runner/*_test.go (orchestration and utilization)