notarius/docs/internal/llm.md

LLM Runtime Internals

internal/framework/llm implements Notarius's transport boundary for structured completion. It contains the Scriptorium adapter, concurrency scheduler, prompt/schema registries, selected-profile recording, and provider-error redaction.

Provider-neutral ownership rules are defined in Architecture. Profile sources, credentials, and concurrency settings are defined in Configuration.

Structured Contract

Modules and LLM-backed validators depend on contracts.StructuredLLMClient.CompleteStructured. A request identifies a prompt and optional profile/session, supplies named input materials and variables, and provides a caller-owned decoding target. A successful response contains the validated raw structured bytes plus non-secret provider, model, profile, and token metadata.

The caller owns prompt selection, response-schema selection, and interpretation of the decoded result. LLMInputMaterial keeps source and reference bytes with their origin metadata so the adapter can pass named artifacts to Scriptorium without exposing Scriptorium types through stage contracts.

Production Construction

internal/cli constructs the production runtime by:

1. allocating the asset registry populated by the generic, Seriatim, and D&D package-family registrars;
2. creating a ScriptoriumClient from the effective profile source;
3. attaching an LLMProfileRecorder;
4. creating a scheduler from the effective concurrency limit;
5. returning a ScheduledClient wrapper;
6. decorating that shared client before preparation when debug recording is enabled; and
7. injecting that one shared client into complete pipeline preparation before the source file is read or the runner is invoked.

The D&D scene chunker and spell, NPC, and combat extractors retain this injected client and use it for every structured completion. Operation requests do not carry an LLM client.

The CLI separately gathers explicit profile IDs from resolved LLM-capable stage and validator bindings. It prepares a small internal check prompt for each ID so missing or invalid profiles fail before pipeline execution. The runtime profile override syntax and scope are defined in the CLI reference; binding rules are defined in Configuration.

Scriptorium Adapter

ScriptoriumClient converts a Notarius request into a Scriptorium RunRequest. It validates the decoding target and prompt identity, maps named input materials to inline artifacts, forwards explicit profile and session context, delegates rendering/provider execution/structured validation, and unmarshals successful JSON into the caller target.

Empty optional input material is represented by a single space so Scriptorium retains the named input. The client returns Scriptorium's validated structured bytes rather than re-encoding the caller target, allowing modules to preserve the runtime result exactly.

Selected profile, provider, model, and token metadata are mapped into the Notarius response. The recorder deduplicates profiles by identity and supplies manifest-safe profile summaries after actual calls; manifest population does not guess the selected prompt default in advance.

Generated-output validation failures and provider failures are wrapped with prompt context. Error strings pass through bearer-token redaction before they cross the runtime boundary.

Scheduling

Scheduler uses a bounded permit count and a FIFO waiter queue. Immediate acquisition increments the in-flight count; queued acquisition waits for a permit or context cancellation. Cancellation removes a queued waiter, while a cancelled waiter that has already received a permit releases it.

ScheduledClient acquires a permit around each structured completion and defers release on every result path. The effective limit and default are configuration facts in Configuration.

This provider-call ceiling is independent of the pipeline's extract worker limit. Concurrent lanes, retries, and validators all use the same scheduled client, so increasing framework workers cannot exceed total_llm. Pipeline dispatch and cancellation mechanics are documented in Pipeline Internals.

Prompt And Schema Assets

AssetRegistry combines caller-owned prompt filesystems under stable prefixes and rejects invalid or conflicting registrations. Production module packages register their own prompt and schema assets; generic framework code contains no D&D prompt content. internal/framework/promptfs provides the domain-neutral filesystem composition helper used to combine module-owned files with shared domain prompt fragments.

The D&D scene chunker and spell, NPC, and combat-turn extractors each declare an ordered prompt asset manifest. The manifest lists the package-owned YAML and Markdown files, then the exact shared fragments rendered by that prompt; the same ordered list drives both filesystem mounting and the prompt fingerprint. The three extraction prompts share system, extraction evidence, in-world identity, transcript, and campaign-reference messages. Spell and combat prompts then render immediate resolution and the ephemeral NPC registry before their lane-specific material; the NPC prompt renders its task and instructions instead. The scene prompt uses system, transcript, and campaign-reference messages before its task and instructions. Identity, transcript, and campaign-reference messages are ephemeral in the extraction prompts; the scene prompt marks transcript and campaign references ephemeral, and the NPC registry is ephemeral where spell and combat prompts use it. Unused shared assets are neither mounted nor fingerprinted. Universal extraction-evidence and output policy lives only in the shared extraction assets; package-owned prompt files retain artifact-specific rules. The scene prompt keeps its separate output rule because it does not render the extraction-evidence asset.

Schema helpers load embedded JSON Schema with identity and digest metadata, return defensive copies, and expose a diagnostics map that omits schema bytes. The small framework registry contains only generic test schemas; production schemas remain package-owned.

The spell, NPC, and combat extractors' package-owned prompts declare their structured JSON inputs and private response schemas. Each private response schema remains separate from its durable artifact codec schema; this work does not use shared schema fragments or schema generation. The combat private schema owns the transport envelope—required fields, JSON types, nullability, and unknown-field rejection—while its deterministic validators own semantic constraints such as enum membership, non-empty values and collections, and positive numbers. The spell extractor's prompt declares a required application/json spell_catalog input and an optional application/json npcs input. The extractor generates the catalog input from its prepared effective catalog as {"spell_names":[...]} using sorted canonical names only. The shared D&D prompt assets include a generic NPC grounding fragment directly after the campaign reference message for spell and combat prompts. When an NPC registry is bound, the domain registry boundary strictly decodes and identity-validates one durable artifact, re-encodes canonical JSON, and generates a semantic digest over those bytes. The unbound input is exactly {"npcs":[]}. Input digests cover the generated bytes; manifests record catalog identity and optional NPC registry digest/count rather than names, aliases, overlay bytes, registry paths, or source metadata. Combat prompt, response-schema, mapping, normalization, identity, and bound-registry fingerprints remain separate semantic inputs to checkpoint identity.

Debug And Redaction Boundaries

The pipeline may wrap the client with a debug recorder that captures prepared prompt/response material for an explicitly requested debug run. Debug summaries and manifests receive identities, hashes, usage, and selected profile summaries rather than prompt, source, reference, schema, or response content.

The Scriptorium error wrapper removes bearer credential values from surfaced provider errors; RedactSecrets and ErrorWithSecretsRedacted support known secret values elsewhere in the runtime. Config summaries use a separate clone-and-redact path in internal/core/config. These mechanisms implement the security invariant in Architecture; operator handling of debug data is defined in Operations.

Failure Behavior

• Invalid targets, missing prompt IDs, malformed structured output, and Scriptorium failures return contextual errors to the calling module.
• Scheduler construction rejects non-positive limits; acquisition respects context cancellation.
• Asset registration rejects invalid roots, missing content, and path conflicts.
• Schema loading distinguishes missing assets, invalid JSON, and invalid metadata.
• Profile validation errors occur during CLI preparation when an explicit selected ID cannot be prepared.

Tests To Inspect

• internal/framework/llm/scriptorium_client_test.go: adapter mapping and local HTTP integration.
• internal/framework/llm/scheduler_test.go and scheduled_client_test.go: permits, FIFO behavior, cancellation, and wrapper release.
• internal/framework/llm/asset_registry_test.go and schema_registry_test.go: asset composition, validation, and defensive copies.
• internal/framework/llm/secrets_test.go: provider-error redaction.
• internal/cli/run_contract_test.go: profile validation, production client wiring, manifest recording, and debug integration.
• Module-local scriptorium_assets_test.go files: prompt inputs and package asset registration.