notarius/docs/roadmap/scriptorium.md

# Scriptorium LLM Runtime Roadmap

This roadmap describes the target state for replacing Notarius' local
OpenAI-compatible LLM adapter with an integration based on
`gitea.maximumdirect.net/eric/scriptorium`.

The upgrade is worthwhile only if it preserves Notarius' architectural
boundaries:

- modules own prompt intent, prompt variables, response schemas, validation, and
  domain interpretation;
- provider plumbing stays behind framework LLM contracts;
- diagnostics and manifests remain Notarius-owned and secret-free;
- raw API keys are resolved operationally and must not be stored in config,
  diagnostics, manifests, examples, or prompt/profile assets;
- Scriptorium types do not leak into chunk, extract, or normalize module
  contracts unless explicitly chosen as a future public contract.

## Motivation

Notarius currently has a local structured-output OpenAI-compatible adapter. That
adapter is intentionally narrow, but it makes Notarius responsible for provider
request shape, profile support, structured-output retries, model-specific
extensions, and future cache-related request fields.

Scriptorium adds capabilities that are directly useful for Notarius:

- built-in profiles for many providers and model families, including OpenRouter
  and local OpenAI-compatible inference endpoints;
- prompt definitions with multiple system and user messages;
- per-prompt `session_id` support for sticky provider routing and input cache
  affinity;
- per-message `cache_control` support for providers that understand it;
- a simple structured-output path where a prompt supplies a schema and
  Scriptorium sends it upstream and retries non-compliant responses.

The most important near-term motivation is input-cache reuse. Existing D&D
recap prompts already use a cacheable user message shaped as:

```text
A transcript of a Dungeons & Dragons gameplay session is provided below.

{{ input "transcript" }}
```

The `transcript` input is the same Seriatim JSON transcript that Notarius
already accepts as MVP input, including stable numbered segments. Notarius
should align chunk, extract, and normalize LLM calls so this transcript message
can be byte-identical across recap generation and structured extraction work.
For large transcripts, this can materially reduce provider cost when cached
input reads are available.

## Target State

Notarius uses Scriptorium as the production LLM execution engine behind the
existing module-facing LLM boundary.

Chunk, extract, and normalize modules continue to receive a Notarius
`StructuredLLMClient` through their existing request contracts. They do not
construct provider clients, read secrets, or depend directly on Scriptorium
request/result types.

The production CLI constructs a Scriptorium-backed client from the effective
Notarius configuration, wraps it in Notarius scheduling and diagnostics policy
where needed, and returns Notarius-owned non-secret LLM profile manifest
metadata.

Module-owned prompts are rendered as ordered chat messages, not only as one
system message and one user message. Prompt rendering must support:

- repeated roles;
- shared input messages;
- module-specific task and instruction messages;
- optional cache-control metadata;
- a stable session ID for calls that should share provider routing/cache
  affinity;
- structured-output schema metadata owned by the module.

The shared D&D transcript message should be rendered from the original Seriatim
input bytes, not reconstructed from normalized source units. The parsed source
document remains the canonical source-reference model for validation and
artifact grounding, but the cacheable transcript prompt message should preserve
the exact transcript payload supplied to the run.

Other large text inputs, including references such as campaign glossaries,
rosters, previous recaps, and future reference producers, should follow the same
input-material model. They should be available to prompt rendering as stable
input artifacts that can be placed in cacheable messages without being copied
into manifests or diagnostics.

## Boundary Requirements

### Provider Plumbing

Provider-specific request fields, profile expansion, and OpenAI-compatible wire
details should be isolated in the framework LLM runtime or in the
Scriptorium-backed adapter. Stage modules should request structured completion
through Notarius contracts and should not know whether the backing client is the
legacy local adapter or Scriptorium.

### Prompt Ownership

Modules remain responsible for prompt intent, prompt versioning, response schema
selection, and semantic validation. Scriptorium provides the prompt definition,
rendering, execution, and structured-output workflow, but it should not become
the owner of D&D-specific semantics.

Shared prompt assets are allowed when they express generic reusable context,
such as the D&D transcript input message. Module-specific task prompts should
remain owned by the relevant module.

### Diagnostics And Manifests

Notarius should continue to decide what appears in diagnostics and run
manifests. Scriptorium prepared/run metadata may be useful input, but Notarius
must filter it through existing policy:

- no raw prompt payloads by default;
- no raw response schema content in manifests;
- no source transcript payloads in manifests;
- no API keys or bearer tokens;
- enough prompt/profile/schema hashes and IDs to audit a run later.

### Secret Handling

Raw API keys should stay out of durable config and prompt/profile assets.
Notarius may continue resolving API keys from configured environment-variable
names, then pass the secret request-scope to Scriptorium. Redacted diagnostics
must continue to prove that resolved secrets are not serialized.

## Prompt And Cache Strategy

The desired D&D LLM prompt shape is:

1. stable shared system message;
2. stable shared transcript user message containing the original Seriatim JSON;
3. optional stable cacheable context messages, such as previous recap,
   glossary, roster, or other references;
4. module-specific task message;
5. module-specific instruction message;
6. module-owned structured-output schema, when the module expects JSON.

The transcript message should be byte-identical whenever the same transcript
input bytes are used. Avoid reconstructing JSON from parsed source units because
formatting, key order, whitespace, or escaping changes would defeat cache reuse.

The transcript and large reference messages should be cacheable when the active
provider path supports cache-control metadata. Providers that ignore cache
controls should still receive a valid prompt.

The session ID should be stable for all LLM calls that operate on the same
session transcript and should be explicitly visible in diagnostics or manifest
metadata only as a non-secret identifier.

Notarius should provide an easy CLI UX for setting the session ID. This lets an
external D&D pipeline orchestrator pass the same session ID used by recap or
other LLM steps, maximizing provider routing affinity and cached input reuse.

## Configuration Intent

Notarius configuration should remain the user-facing source of pipeline
composition and operational settings. For LLM execution profiles, Notarius
should cut over to Scriptorium profile configuration instead of maintaining a
separate Notarius-specific profile schema.

Notarius should allow configuration to select Scriptorium built-in profiles and
to point at Scriptorium profile files or directories. This avoids evolving two
nearly identical profile systems and gives users immediate access to the
provider and model catalog that motivated the migration.

Notarius still owns validation, redaction, and manifest provenance for the
effective pipeline. The integration should wrap Scriptorium profile loading so
that errors are actionable, secrets remain environment-based, and emitted
metadata stays non-secret.

## Compatibility Policy

This feature should be a hard cutover to Scriptorium-backed prompt execution and
profile loading. Backward compatibility with the local Notarius prompt renderer,
local LLM profile schema, or local OpenAI-compatible adapter is not a product
requirement for this migration.

The final production runtime should have one documented LLM execution path.
Module behavior may still be tested with fake Notarius `StructuredLLMClient`
implementations, but production runtime behavior should be Scriptorium-backed.

## Documentation Outcomes

When this feature is implemented, current-behavior docs should be updated to
describe:

- the implemented production LLM runtime;
- supported LLM profile fields and provider/profile selection behavior;
- cacheable transcript prompt behavior, if exposed to users;
- session ID behavior;
- diagnostics and manifest provenance;
- troubleshooting for Scriptorium profile, prompt, schema, and validation
  failures.

The OpenAI-compatible integration doc should either be retired, narrowed to the
legacy fallback, or reframed as an upstream provider contract delegated through
Scriptorium, depending on the final runtime shape.

## Deferred Work

The Scriptorium migration does not itself require:

- a general workflow language;
- arbitrary per-stage prompt authoring by end users;
- token budgeting or context-window planning;
- non-file reference producers;
- semantic retrieval over transcript or reference content;
- multiple effective LLM profiles in one Notarius run.

Those remain separate future features.

## Resolved Design Choices

### Prompt Asset Ownership And Format

Production LLM prompts should cut over to Scriptorium-compatible prompt
definitions without preserving backward compatibility for the current Notarius
system/user prompt renderer.

Module-owned prompt definitions should keep task prompts and schemas near the
module that owns the semantic behavior. Shared prompt messages, such as the D&D
transcript input message and shared reference/context messages, should live in a
shared prompt asset area and be referenced by module-owned prompt definitions.

This uses Scriptorium's native strengths: ordered messages, cache-control
metadata, session IDs, input helpers, profile selection, and schema-backed
execution. It also gives Notarius a direct way to reuse the exact transcript and
reference messages across D&D recap, chunking, extraction, and normalization
work without each module hand-rolling message assembly.

### Raw Transcript Preservation Location

Original input bytes should be preserved as run-scoped input material and made
available to prompt rendering without storing them in run manifests. The parsed
`SourceDocument` should continue to carry normalized source units for framework
logic and source-reference validation.

The same model should apply to all large text inputs, including references.
Glossaries, campaign rosters, previous recaps, and other future reference
documents may be large enough to deserve their own cacheable messages. They
should be treated as prompt input artifacts rather than being copied into source
document metadata.

This keeps the cacheable transcript message byte-identical to the original
Seriatim JSON and avoids source-format leakage into chunk, extract, and
normalize contracts. It also reduces the risk that raw inputs or references are
accidentally serialized into diagnostics or manifests.

### Session ID Source

Notarius should add an explicit session ID concept for LLM calls, with a
deterministic default derived from the source document identity when no
operator-provided value is configured.

The CLI should provide an easy way to supply this session ID. This lets an
external D&D pipeline orchestrator call Notarius with the same session ID used
by recap generation or other LLM steps, maximizing provider routing affinity and
cached input savings.

OpenRouter sticky routing and provider cache behavior are most useful when
every related call for the same session shares a stable identifier. An explicit
concept makes the behavior auditable and avoids each module inventing its own
ID. A deterministic default keeps simple local runs ergonomic.

### Scriptorium Profile Exposure

Notarius should cut over to Scriptorium profile configuration instead of
maintaining a separate Notarius LLM profile schema. Configuration should be able
to select Scriptorium built-in profiles and point at Scriptorium profile files
or directories.

The Scriptorium profile format is already close to the desired Notarius target.
Maintaining a separate Notarius profile schema would likely create duplicate
configuration that eventually converges back toward Scriptorium's model. A hard
cutover avoids that churn and gives users immediate access to Scriptorium's
provider and model catalog.

Notarius still needs to wrap this profile loading with Notarius-owned
validation, diagnostics, manifest provenance, and secret-redaction policy.

### Structured-Output Retry Ownership

Notarius should use Scriptorium's structured-output execution and retry behavior
for production calls, while translating results and errors back into Notarius'
`StructuredLLMClient` response and error expectations.

This avoids duplicating structured-output enforcement in Notarius and lets
Scriptorium own provider-specific request and repair mechanics. Notarius still
retains module-level validation and source-reference checks after decoded
structured output is returned.