notarius/docs/roadmap/scriptorium.md

Scriptorium LLM Runtime Roadmap

This roadmap describes the target state for replacing Notarius' local OpenAI-compatible LLM adapter with an integration based on gitea.maximumdirect.net/eric/scriptorium.

The upgrade is worthwhile only if it preserves Notarius' architectural boundaries:

• modules own prompt intent, prompt variables, response schemas, validation, and domain interpretation;
• provider plumbing stays behind framework LLM contracts;
• diagnostics and manifests remain Notarius-owned and secret-free;
• raw API keys are resolved operationally and must not be stored in config, diagnostics, manifests, examples, or prompt/profile assets;
• Scriptorium types do not leak into chunk, extract, or normalize module contracts unless explicitly chosen as a future public contract.

Motivation

Notarius currently has a local structured-output OpenAI-compatible adapter. That adapter is intentionally narrow, but it makes Notarius responsible for provider request shape, profile support, structured-output retries, model-specific extensions, and future cache-related request fields.

Scriptorium adds capabilities that are directly useful for Notarius:

• built-in profiles for many providers and model families, including OpenRouter and local OpenAI-compatible inference endpoints;
• prompt definitions with multiple system and user messages;
• per-prompt session_id support for sticky provider routing and input cache affinity;
• per-message cache_control support for providers that understand it;
• a simple structured-output path where a prompt supplies a schema and Scriptorium sends it upstream and retries non-compliant responses.

The most important near-term motivation is input-cache reuse. Existing D&D recap prompts already use a cacheable user message shaped as:

A transcript of a Dungeons & Dragons gameplay session is provided below.

{{ input "transcript" }}

The transcript input is the same Seriatim JSON transcript that Notarius already accepts as MVP input, including stable numbered segments. Notarius should align chunk, extract, and normalize LLM calls so this transcript message can be byte-identical across recap generation and structured extraction work. For large transcripts, this can materially reduce provider cost when cached input reads are available.

Target State

Notarius uses Scriptorium as the production LLM execution engine behind the existing module-facing LLM boundary.

Chunk, extract, and normalize modules continue to receive a Notarius StructuredLLMClient through their existing request contracts. They do not construct provider clients, read secrets, or depend directly on Scriptorium request/result types.

The production CLI constructs a Scriptorium-backed client from the effective Notarius configuration, wraps it in Notarius scheduling and diagnostics policy where needed, and returns Notarius-owned non-secret LLM profile manifest metadata.

Module-owned prompts are rendered as ordered chat messages, not only as one system message and one user message. Prompt rendering must support:

• repeated roles;
• shared input messages;
• module-specific task and instruction messages;
• optional cache-control metadata;
• a stable session ID for calls that should share provider routing/cache affinity;
• structured-output schema metadata owned by the module.

The shared D&D transcript message should be rendered from the original Seriatim input bytes, not reconstructed from normalized source units. The parsed source document remains the canonical source-reference model for validation and artifact grounding, but the cacheable transcript prompt message should preserve the exact transcript payload supplied to the run.

Other large text inputs, including references such as campaign glossaries, rosters, previous recaps, and future reference producers, should follow the same input-material model. They should be available to prompt rendering as stable input artifacts that can be placed in cacheable messages without being copied into manifests or diagnostics.

Boundary Requirements

Provider Plumbing

Provider-specific request fields, profile expansion, and OpenAI-compatible wire details should be isolated in the framework LLM runtime or in the Scriptorium-backed adapter. Stage modules should request structured completion through Notarius contracts and should not know whether the backing client is the legacy local adapter or Scriptorium.

Prompt Ownership

Modules remain responsible for prompt intent, prompt versioning, response schema selection, and semantic validation. Scriptorium provides the prompt definition, rendering, execution, and structured-output workflow, but it should not become the owner of D&D-specific semantics.

Shared prompt assets are allowed when they express generic reusable context, such as the D&D transcript input message. Module-specific task prompts should remain owned by the relevant module.

Diagnostics And Manifests

Notarius should continue to decide what appears in diagnostics and run manifests. Scriptorium prepared/run metadata may be useful input, but Notarius must filter it through existing policy:

• no raw prompt payloads by default;
• no raw response schema content in manifests;
• no source transcript payloads in manifests;
• no API keys or bearer tokens;
• enough prompt/profile/schema hashes and IDs to audit a run later.

Secret Handling

Raw API keys should stay out of durable config and prompt/profile assets. Notarius may continue resolving API keys from configured environment-variable names, then pass the secret request-scope to Scriptorium. Redacted diagnostics must continue to prove that resolved secrets are not serialized.

Prompt And Cache Strategy

The desired D&D LLM prompt shape is:

1. stable shared system message;
2. stable shared transcript user message containing the original Seriatim JSON;
3. optional stable cacheable context messages, such as previous recap, glossary, roster, or other references;
4. module-specific task message;
5. module-specific instruction message;
6. module-owned structured-output schema, when the module expects JSON.

The transcript message should be byte-identical whenever the same transcript input bytes are used. Avoid reconstructing JSON from parsed source units because formatting, key order, whitespace, or escaping changes would defeat cache reuse.

The transcript and large reference messages should be cacheable when the active provider path supports cache-control metadata. Providers that ignore cache controls should still receive a valid prompt.

The session ID should be stable for all LLM calls that operate on the same session transcript and should be explicitly visible in diagnostics or manifest metadata only as a non-secret identifier.

Notarius should provide an easy CLI UX for setting the session ID. This lets an external D&D pipeline orchestrator pass the same session ID used by recap or other LLM steps, maximizing provider routing affinity and cached input reuse.

Configuration Intent

Notarius configuration should remain the user-facing source of pipeline composition and operational settings. For LLM execution profiles, Notarius should cut over to Scriptorium profile configuration instead of maintaining a separate Notarius-specific profile schema.

Notarius should allow configuration to select Scriptorium built-in profiles and to point at Scriptorium profile files or directories. This avoids evolving two nearly identical profile systems and gives users immediate access to the provider and model catalog that motivated the migration.

Notarius still owns validation, redaction, and manifest provenance for the effective pipeline. The integration should wrap Scriptorium profile loading so that errors are actionable, secrets remain environment-based, and emitted metadata stays non-secret.

Compatibility Policy

This feature should be a hard cutover to Scriptorium-backed prompt execution and profile loading. Backward compatibility with the local Notarius prompt renderer, local LLM profile schema, or local OpenAI-compatible adapter is not a product requirement for this migration.

The final production runtime should have one documented LLM execution path. Module behavior may still be tested with fake Notarius StructuredLLMClient implementations, but production runtime behavior should be Scriptorium-backed.

Documentation Outcomes

When this feature is implemented, current-behavior docs should be updated to describe:

• the implemented production LLM runtime;
• supported LLM profile fields and provider/profile selection behavior;
• cacheable transcript prompt behavior, if exposed to users;
• session ID behavior;
• diagnostics and manifest provenance;
• troubleshooting for Scriptorium profile, prompt, schema, and validation failures.

The OpenAI-compatible integration doc should either be retired, narrowed to the legacy fallback, or reframed as an upstream provider contract delegated through Scriptorium, depending on the final runtime shape.

Deferred Work

The Scriptorium migration does not itself require:

• a general workflow language;
• arbitrary per-stage prompt authoring by end users;
• token budgeting or context-window planning;
• non-file reference producers;
• semantic retrieval over transcript or reference content;
• multiple effective LLM profiles in one Notarius run.

Those remain separate future features.

Resolved Design Choices

Prompt Asset Ownership And Format

Production LLM prompts should cut over to Scriptorium-compatible prompt definitions without preserving backward compatibility for the current Notarius system/user prompt renderer.

Module-owned prompt definitions should keep task prompts and schemas near the module that owns the semantic behavior. Shared prompt messages, such as the D&D transcript input message and shared reference/context messages, should live in a shared prompt asset area and be referenced by module-owned prompt definitions.

This uses Scriptorium's native strengths: ordered messages, cache-control metadata, session IDs, input helpers, profile selection, and schema-backed execution. It also gives Notarius a direct way to reuse the exact transcript and reference messages across D&D recap, chunking, extraction, and normalization work without each module hand-rolling message assembly.

Raw Transcript Preservation Location

Original input bytes should be preserved as run-scoped input material and made available to prompt rendering without storing them in run manifests. The parsed SourceDocument should continue to carry normalized source units for framework logic and source-reference validation.

The same model should apply to all large text inputs, including references. Glossaries, campaign rosters, previous recaps, and other future reference documents may be large enough to deserve their own cacheable messages. They should be treated as prompt input artifacts rather than being copied into source document metadata.

This keeps the cacheable transcript message byte-identical to the original Seriatim JSON and avoids source-format leakage into chunk, extract, and normalize contracts. It also reduces the risk that raw inputs or references are accidentally serialized into diagnostics or manifests.

Session ID Source

Notarius should add an explicit session ID concept for LLM calls, with a deterministic default derived from the source document identity when no operator-provided value is configured.

The CLI should provide an easy way to supply this session ID. This lets an external D&D pipeline orchestrator call Notarius with the same session ID used by recap generation or other LLM steps, maximizing provider routing affinity and cached input savings.

OpenRouter sticky routing and provider cache behavior are most useful when every related call for the same session shares a stable identifier. An explicit concept makes the behavior auditable and avoids each module inventing its own ID. A deterministic default keeps simple local runs ergonomic.

Scriptorium Profile Exposure

Notarius should cut over to Scriptorium profile configuration instead of maintaining a separate Notarius LLM profile schema. Configuration should be able to select Scriptorium built-in profiles and point at Scriptorium profile files or directories.

The Scriptorium profile format is already close to the desired Notarius target. Maintaining a separate Notarius profile schema would likely create duplicate configuration that eventually converges back toward Scriptorium's model. A hard cutover avoids that churn and gives users immediate access to Scriptorium's provider and model catalog.

Notarius still needs to wrap this profile loading with Notarius-owned validation, diagnostics, manifest provenance, and secret-redaction policy.

Structured-Output Retry Ownership

Notarius should use Scriptorium's structured-output execution and retry behavior for production calls, while translating results and errors back into Notarius' StructuredLLMClient response and error expectations.

This avoids duplicating structured-output enforcement in Notarius and lets Scriptorium own provider-specific request and repair mechanics. Notarius still retains module-level validation and source-reference checks after decoded structured output is returned.