# LLM Runtime

The implemented LLM runtime lives in `internal/framework/llm`. It provides
transport-neutral structured completion contracts, an OpenAI-compatible HTTP
adapter, concurrency scheduling, schema registry helpers, retry behavior, and
secret redaction.

## Contract

Modules depend on `contracts.StructuredLLMClient`:

```go
CompleteStructured(ctx, request, out) (response, error)
```

The request contains messages, optional model override, response schema name,
and response schema JSON. The caller supplies a pointer target for decoded
structured output.

Modules that call the LLM own their prompts and schemas. Provider adapters
should not contain domain-specific prompt logic.

## Production Client Construction

`internal/cli` builds the production LLM client from the effective config:

1. find the effective LLM profile;
2. build `OpenAICompatibleClientConfig`;
3. create an OpenAI-compatible client;
4. create a scheduler from profile or global concurrency;
5. wrap the client with `NewScheduledClient`;
6. return non-secret LLM profile manifest metadata.

The current run command requires exactly one distinct effective LLM profile for
the resolved pipeline.

## OpenAI-Compatible Adapter

`OpenAICompatibleClient` posts JSON to:

```text
<base_url>/chat/completions
```

It sends:

- `model`
- `messages`
- `response_format.type = "json_schema"`
- `response_format.json_schema.name`
- `response_format.json_schema.strict = true`
- `response_format.json_schema.schema`

If an API key is configured, the adapter sends an `Authorization: Bearer ...`
header.

The adapter accepts assistant content either as a JSON string containing JSON or
as raw JSON content. It then unmarshals that content into the caller-provided
target.

External wire-contract details belong in the
[OpenAI-compatible integration doc](../integrations/openai-compatible.md).

## Retries And Timeouts

The adapter retries:

- provider request failures;
- response read failures;
- HTTP `429`;
- HTTP `5xx`;
- malformed provider envelopes;
- malformed assistant JSON;
- structured-output decode failures.

Non-retryable `4xx` responses are returned without retry. Request timeout comes
from the effective LLM profile. Context cancellation is respected.

## Scheduler

`Scheduler` bounds concurrent provider calls. It tracks in-flight calls and a
FIFO queue of waiters. Cancellation removes queued waiters or releases granted
permits.

`NewScheduledClient` wraps any structured LLM client and runs each completion
inside the scheduler.

Effective concurrency is:

1. `llm_profiles.<id>.max_concurrency`, when greater than zero;
2. `concurrency.total_llm`, when greater than zero;
3. `1`.

## Schema Registry

The framework schema registry embeds generic test schemas. It also exposes
helpers for caller-owned schemas:

- `LoadResponseSchema`
- `LookupResponseSchema`
- `MustLookupResponseSchema`
- `ResponseSchema.DiagnosticsMap`

`DiagnosticsMap` omits raw schema content and includes metadata such as key,
ID, version, name, and SHA-256.

The D&D spell extractor owns and loads its own embedded response schema.

## Secret Redaction

Provider errors are passed through `ErrorWithSecretsRedacted` with the API key
and bearer-token value. Config diagnostics use redacted effective config
payloads.

Do not add raw provider request bodies, response bodies, API keys, or prompt
payloads to diagnostics by default.