Add Scriptorium-backed LLM runtime

This commit is contained in:
2026-07-05 18:21:37 +00:00
parent de6689bc1d
commit f6224dcbee
12 changed files with 742 additions and 39 deletions

View File

@@ -1,9 +1,10 @@
# LLM Runtime
The implemented LLM runtime lives in `internal/framework/llm`. It provides
transport-neutral structured completion contracts, an OpenAI-compatible HTTP
adapter, concurrency scheduling, schema registry helpers, retry behavior, and
secret redaction.
transport-neutral structured completion contracts, a Scriptorium-backed
production client, an OpenAI-compatible HTTP adapter retained for legacy tests
and helpers, concurrency scheduling, prompt/schema asset registration, schema
registry helpers, and secret redaction.
## Contract
@@ -13,26 +14,48 @@ Modules depend on `contracts.StructuredLLMClient`:
CompleteStructured(ctx, request, out) (response, error)
```
The request contains messages, optional model override, response schema name,
and response schema JSON. The caller supplies a pointer target for decoded
structured output.
The request contains prompt ID/version, profile ID, session ID, prompt input
materials, variables, and legacy rendered-message/schema fields used by modules
that have not yet moved to prompt-asset execution. The caller supplies a pointer
target for decoded structured output.
Modules that call the LLM own their prompts and schemas. Provider adapters
should not contain domain-specific prompt logic.
Modules that call the LLM own their prompts, schemas, prompt IDs, validators,
and domain-specific interpretation. Provider adapters should not contain
domain-specific prompt logic.
## Production Client Construction
`internal/cli` builds the production LLM client from the effective config:
1. find the effective LLM profile;
2. build `OpenAICompatibleClientConfig`;
3. create an OpenAI-compatible client;
4. create a scheduler from profile or global concurrency;
5. wrap the client with `NewScheduledClient`;
6. return non-secret LLM profile manifest metadata.
1. collect production Scriptorium prompt and schema assets from module packages;
2. create a Scriptorium-backed structured client using effective Scriptorium
profile source settings;
3. create a scheduler from global LLM concurrency;
4. wrap the client with `NewScheduledClient`;
5. let the runtime report non-secret profile manifest metadata after calls.
The current run command requires exactly one distinct effective LLM profile for
the resolved pipeline.
The runtime records the actual selected Scriptorium profile, provider, and model
used during execution. Manifest population does not rely on a precomputed
profile ID before pipeline execution.
## Scriptorium Adapter
`ScriptoriumClient` implements `contracts.StructuredLLMClient` by converting
Notarius prompt requests into Scriptorium `RunRequest` values. It:
- validates the caller output target and prompt ID;
- converts `LLMInputMaterial` values into inline Scriptorium artifacts;
- passes `session_id` through Scriptorium variables when present;
- sends explicit profile IDs only when the request supplies one;
- lets Scriptorium render prompts, call the configured provider, and validate
structured output;
- unmarshals successful JSON into the caller-provided target;
- maps token usage and selected profile/model metadata into the Notarius
response and manifest profile recorder.
Generated-output validation failures are returned as Notarius errors. Provider
and runtime errors are wrapped with prompt context and bearer tokens are
redacted from error strings.
## OpenAI-Compatible Adapter
@@ -73,8 +96,8 @@ The adapter retries:
- malformed assistant JSON;
- structured-output decode failures.
Non-retryable `4xx` responses are returned without retry. Request timeout comes
from the effective LLM profile. Context cancellation is respected.
Non-retryable `4xx` responses are returned without retry. Context cancellation
is respected.
## Scheduler
@@ -87,9 +110,8 @@ inside the scheduler.
Effective concurrency is:
1. `llm_profiles.<id>.max_concurrency`, when greater than zero;
2. `concurrency.total_llm`, when greater than zero;
3. `1`.
1. `concurrency.total_llm`, when greater than zero;
2. `1`.
## Schema Registry
@@ -104,13 +126,14 @@ helpers for caller-owned schemas:
`DiagnosticsMap` omits raw schema content and includes metadata such as key,
ID, version, name, and SHA-256.
The D&D spell extractor owns and loads its own embedded response schema.
Production modules own and register their Scriptorium prompt and schema assets.
Framework packages may collect those files but must not contain D&D-specific
prompt content.
## Secret Redaction
Provider errors are passed through `ErrorWithSecretsRedacted` with the API key
and bearer-token value. Config diagnostics use redacted effective config
payloads.
Provider errors are redacted before surfacing through the Scriptorium-backed
client. Config diagnostics use redacted effective config payloads.
Do not add raw provider request bodies, response bodies, API keys, or prompt
payloads to diagnostics by default.

View File

@@ -41,8 +41,9 @@ production modules.
- `internal/framework/pipeline`: module registries, module specs, profile
resolution, capability checks, run orchestration, warnings, validation, and
manifest population.
- `internal/framework/llm`: OpenAI-compatible structured-output client,
scheduler, schema registry, retries, and secret redaction.
- `internal/framework/llm`: Scriptorium-backed structured-output client,
prompt/schema asset registry, scheduler, schema registry, retries, and secret
redaction.
- `internal/framework/prompt`: embedded prompt registry and template rendering.
- `internal/framework/validate`: validator decision helpers and cardinality
enforcement.