notarius/docs/internal/llm.md

# LLM Runtime Internals

`internal/framework/llm` implements Notarius's transport boundary for structured
completion. It contains the Scriptorium adapter, concurrency scheduler,
prompt/schema registries, selected-profile recording, and provider-error
redaction.

Provider-neutral ownership rules are defined in
[Architecture](../policy/architecture.md#llm-boundary). Profile sources,
credentials, and concurrency settings are defined in
[Configuration](../config.md).

## Structured Contract

Modules and LLM-backed validators depend on
`contracts.StructuredLLMClient.CompleteStructured`. A request identifies a
prompt and optional profile/session, supplies named input materials and
variables, and provides a caller-owned decoding target. A successful response
contains the validated raw structured bytes plus non-secret provider, model,
profile, and token metadata.

The caller owns prompt selection, response-schema selection, and interpretation
of the decoded result. `LLMInputMaterial` keeps source and reference bytes with
their origin metadata so the adapter can pass named artifacts to Scriptorium
without exposing Scriptorium types through stage contracts.

## Production Construction

`internal/cli` constructs the production runtime by:

1. allocating the asset registry populated by the generic, Seriatim, and D&D
   package-family registrars;
2. creating a `ScriptoriumClient` from the effective profile source;
3. attaching an `LLMProfileRecorder`;
4. creating a scheduler from the effective concurrency limit;
5. returning a `ScheduledClient` wrapper;
6. decorating that shared client before preparation when debug recording is
   enabled; and
7. injecting that one shared client into complete pipeline preparation before
   the source file is read or the runner is invoked.

The D&D scene chunker and spell, NPC, combat-turn, item-event, NPC-interaction, and
scene-description extractors retain this
injected client and use it for every structured completion. Operation requests
do not carry an LLM client.

The CLI separately gathers explicit profile IDs from resolved LLM-capable stage
and validator bindings. It prepares a small internal check prompt for each ID so
missing or invalid profiles fail before pipeline execution. The runtime profile
override syntax and scope are defined in the
[CLI reference](../cli.md#run); binding rules are defined in
[Configuration](../config.md#module-bindings).

## Scriptorium Adapter

`ScriptoriumClient` converts a Notarius request into a Scriptorium `RunRequest`.
It validates the decoding target and prompt identity, maps named input materials
to inline artifacts, forwards explicit profile and session context, delegates
rendering/provider execution/structured validation, and unmarshals successful
JSON into the caller target.

Empty optional input material is represented by a single space so Scriptorium
retains the named input. The client returns Scriptorium's validated structured
bytes rather than re-encoding the caller target, allowing modules to preserve
the runtime result exactly.

Selected profile, provider, model, and token metadata are mapped into the
Notarius response. The recorder deduplicates profiles by identity and supplies
manifest-safe profile summaries after actual calls; manifest population does
not guess the selected prompt default in advance.

Generated-output validation failures and provider failures are wrapped with
prompt context. Error strings pass through bearer-token redaction before they
cross the runtime boundary.

## Scheduling

`Scheduler` uses a bounded permit count and a FIFO waiter queue. Immediate
acquisition increments the in-flight count; queued acquisition waits for a
permit or context cancellation. Cancellation removes a queued waiter, while a
cancelled waiter that has already received a permit releases it.

`ScheduledClient` acquires a permit around each structured completion and
defers release on every result path. The effective limit and default are
configuration facts in [Configuration](../config.md#defaults).

This provider-call ceiling is independent of the pipeline's extract worker
limit. Concurrent lanes, retries, and validators all use the same scheduled
client, so increasing framework workers cannot exceed `total_llm`. Pipeline
dispatch and cancellation mechanics are documented in
[Pipeline Internals](pipeline.md#execution-flow).

## Prompt And Schema Assets

`AssetRegistry` combines caller-owned prompt filesystems under stable prefixes
and rejects invalid or conflicting registrations. Production module packages
register their own prompt and schema assets; generic framework code contains no
D&D prompt content. `internal/framework/promptfs` provides the domain-neutral
filesystem composition helper used to combine module-owned files with shared
domain prompt fragments.

The D&D scene chunker and spell, NPC, combat-turn, item-event, NPC-interaction, and
scene-description extractors each declare an
ordered prompt asset manifest. The manifest lists the package-owned YAML and
Markdown files, then the exact shared fragments rendered by that prompt; the
same ordered list drives both filesystem mounting and the prompt fingerprint.
Unused shared assets are neither mounted nor fingerprinted. Universal
extraction-evidence and output policy lives only in the shared extraction
assets; package-owned prompt files retain artifact-specific rules. The scene
prompt keeps its separate output rule because it does not render the
extraction-evidence asset.

### D&D Extraction Prompt Ordering And Cache Boundaries

D&D extraction prompts order messages from the most reusable content to the
most variable content. New extraction lanes use these tiers in order:

1. universal shared content, including the system, extraction-evidence, and
   in-world identity messages;
2. stable campaign or run context shared across lanes, including campaign
   references;
3. stable subset- and lane-specific context and instructions, including an NPC
   registry, catalog, task, or extraction instructions when applicable;
4. the chunk transcript as the final user message.

This ordering lets requests reuse the longest identical prefix before the
per-chunk transcript changes. Cache reuse requires the preceding message
sequence and content to be exactly identical; semantic similarity is not
sufficient. Cache boundaries belong at the ends of reusable stable tiers,
subject to the provider's cache-boundary limit. The shared identity and
campaign-reference messages form the first two extraction boundaries. Spell,
combat, and interaction prompts add a boundary at the shared NPC registry. Each extraction
prompt places its final boundary on its lane-specific instructions, immediately
before the transcript. The transcript does not carry cache control because no
reusable content follows it.

Accordingly, the common prefix of the spell, NPC, combat, item-event, and interaction
extraction prompts is system,
extraction evidence, identity, and campaign references. The NPC prompt then
renders task, instructions, and transcript. Spell renders the NPC registry,
catalog, task, instructions, and transcript. Combat renders the NPC registry,
task, instructions, and transcript. Item-event renders task, instructions, and
transcript without a generated-artifact input. NPC interaction renders the names-only NPC
registry, task, instructions, and transcript. The
scene chunker is not an extraction lane: it retains its separate system,
transcript, campaign-reference, task, and instruction order and marks its
transcript and campaign-reference messages ephemeral.

The scene-description extractor deliberately omits the citation-oriented
`common-dnd-extraction-evidence.md` asset because Notarius attaches the whole
accepted chunk range itself. Its manifest is system, shared identity, shared
campaign references, lane task, lane instructions, then the transcript. The
identity, campaign-reference, and instruction messages are ephemeral cache
boundaries; the transcript is last and has no cache control. Compatible shared
messages remain canonical shared assets rather than copied package text.

### D&D NPC Normalization Prompt Ordering And Cache Boundaries

NPC normalization has a distinct prompt and response-schema identity from NPC
extraction. Its stable message tiers are the common D&D system asset, followed
by package-owned task and normalization instructions. Cache boundaries follow
the shared system tier and the package instructions. The variable tail contains
the private candidate-name-and-range input and a windowed transcript input
whose cited units provide local context; neither has a cache boundary because
it changes with the document.

This prompt intentionally omits shared identity guidance,
extraction-evidence, and campaign-reference assets: it reconciles existing
records rather than extracting events or adding evidence. Its package-owned
manifest and schema identity are fingerprinted separately, so a normalization
prompt or schema change cannot reuse a prior normalization checkpoint.

Shared wording belongs in the canonical assets under
`internal/modules/dnd/shared`; extraction packages reference those assets in
their manifests instead of copying similar text into package-local files.
Package-local assets contain only lane-specific content. An extraction lane may
depart from the tier order only when prompt-quality evidence or a provider
constraint makes the exception necessary; document the exception and rationale
here when it becomes implemented behavior.

Schema helpers load embedded JSON Schema with identity and digest metadata,
return defensive copies, and expose a diagnostics map that omits schema bytes.
The small framework registry contains only generic test schemas; production
schemas remain package-owned.

The spell, NPC, combat, item-event, NPC-interaction, and scene-description extractors'
package-owned prompts declare their
structured JSON inputs and private response schemas. Each private response
schema remains separate from its durable artifact codec schema; this work does
not use shared schema fragments or schema generation. Those private schemas own
the transport envelope—required fields, JSON types, nullability, and
unknown-field rejection—while deterministic validators own semantic constraints
such as enum membership, non-empty values and collections, and positive
numbers. The spell extractor's prompt declares a required
`application/json` `spell_catalog` input and an optional `application/json`
`npcs` input. The extractor generates
the catalog input from its prepared
effective catalog as `{"spell_names":[...]}` using sorted canonical names only.
The shared D&D prompt assets include a generic NPC grounding fragment directly
after the campaign reference message for spell, combat, and interaction prompts. When an NPC
registry is bound, the
domain registry boundary strictly decodes and identity-validates one durable
artifact, re-encodes canonical JSON for provenance, and separately generates a
names-only prompt projection. The unbound projection is exactly `{"npcs":[]}`.
Prompt input and component-local checkpoint digests cover the projected bytes;
manifests retain the optional full registry digest/count rather than names,
overlay bytes, registry paths, or source metadata. Combat and interaction prompt,
response-schema, mapping, normalization, identity, and registry-projection
fingerprints remain separate semantic inputs to checkpoint identity.

## Debug And Redaction Boundaries

The pipeline may wrap the client with a debug recorder that captures prepared
prompt/response material for an explicitly requested debug run. Debug summaries
and manifests receive identities, hashes, usage, and selected profile summaries
rather than prompt, source, reference, schema, or response content.

The Scriptorium error wrapper removes bearer credential values from surfaced
provider errors; `RedactSecrets` and `ErrorWithSecretsRedacted` support known
secret values elsewhere in the runtime. Config summaries use a separate
clone-and-redact path in `internal/core/config`. These mechanisms implement the
security invariant in
[Architecture](../policy/architecture.md#state-output-and-safety); operator
handling of debug data is defined in [Operations](../operations.md#debug).

## Failure Behavior

- Invalid targets, missing prompt IDs, malformed structured output, and
  Scriptorium failures return contextual errors to the calling module.
- Scheduler construction rejects non-positive limits; acquisition respects
  context cancellation.
- Asset registration rejects invalid roots, missing content, and path conflicts.
- Schema loading distinguishes missing assets, invalid JSON, and invalid
  metadata.
- Profile validation errors occur during CLI preparation when an explicit
  selected ID cannot be prepared.

## Tests To Inspect

- `internal/framework/llm/scriptorium_client_test.go`: adapter mapping and local
  HTTP integration.
- `internal/framework/llm/scheduler_test.go` and
  `scheduled_client_test.go`: permits, FIFO behavior, cancellation, and wrapper
  release.
- `internal/framework/llm/asset_registry_test.go` and
  `schema_registry_test.go`: asset composition, validation, and defensive
  copies.
- `internal/framework/llm/secrets_test.go`: provider-error redaction.
- `internal/cli/run_contract_test.go`: profile validation, production client
  wiring, manifest recording, and debug integration.
- Module-local `scriptorium_assets_test.go` files: prompt inputs and package
  asset registration.