From 9532ae8121c031bf5d4e8e29b3ef9893ccacdd24 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 5 Jul 2026 12:42:15 -0500 Subject: [PATCH] Add a feature roadmap and implementation plan to import the scriptorium LLM library --- docs/roadmap/implementation.md | 474 +++++++++++++++++++++++++++++++++ docs/roadmap/scriptorium.md | 295 ++++++++++++++++++++ 2 files changed, 769 insertions(+) create mode 100644 docs/roadmap/implementation.md create mode 100644 docs/roadmap/scriptorium.md diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 00000000..96a6e77e --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,474 @@ +# Scriptorium Cutover Implementation Plan + +This plan implements the target state in +[scriptorium.md](scriptorium.md): a hard cutover from Notarius' local +OpenAI-compatible adapter, local LLM profile schema, and local system/user +prompt renderer to Scriptorium-backed prompt execution. + +The target implementer is an LLM coding agent. Complete each stage in order. +Do not preserve backward compatibility for removed local LLM-profile or +prompt-rendering behavior unless a stage explicitly says to keep a temporary +test seam. + +## Global Constraints + +- Keep Scriptorium types out of chunk, extract, and normalize module contracts. + Module-facing requests should use Notarius-owned contract types. +- Keep provider plumbing behind `internal/framework/llm` and CLI construction. +- Keep module-owned prompt intent, prompt IDs, response schemas, validators, and + source-reference validation with the relevant modules. +- Keep raw input material, raw references, raw prompts, raw schemas, and API + keys out of manifests and default diagnostics. +- Treat the original source input and large references as prompt input + materials. They may be passed to Scriptorium as inline/file inputs, but should + not be copied into durable provenance. +- Use Scriptorium profile configuration directly. Do not keep Notarius' + `llm_profiles` schema as a second profile system. +- Use Scriptorium structured-output execution and validation behavior for + production calls. Notarius should still unmarshal successful structured JSON + into module-owned response structs and run module-owned deterministic + validation afterward. + +## Stage 1: Dependency And API Grounding + +Goal: add the Scriptorium dependency and establish the exact public API surface +used by the rest of the implementation. + +Work: + +- Add `gitea.maximumdirect.net/eric/scriptorium` to `go.mod` with `go get`. +- Inspect the installed package with `go doc` or source reads before coding the + adapter. Confirm names and fields for: + - `NewEngine`; + - `Config`; + - `WithPromptFS`, `WithPromptFile`, or equivalent prompt source options; + - `WithProfileFile`, `WithProfileFS`, profile directory config, and built-in + profile behavior; + - `WithSchemaFS` or equivalent schema source options; + - `RunRequest`, including prompt ID, prompt version, profile ID, inputs, + variables, metadata, direct API key, and validation override fields; + - `RunResult`, including raw output, output artifact, validation status, + prompt/profile/model metadata, usage, and timings; + - public sentinel errors. +- Add a short internal implementation note as a code comment only where needed; + do not add user-facing docs in this stage except if a package-level test + helper needs explanation. + +Acceptance checks: + +- `go test ./internal/framework/llm` +- `go test ./internal/cli` +- `go test ./...` + +## Stage 2: Prompt Input Materials And Session IDs + +Goal: make raw source input and references available to module prompt execution +without storing large content in manifests or default diagnostics. + +Work: + +- Add Notarius-owned contract types under `internal/framework/contracts`: + - `LLMInputMaterial` with at least `Name`, `MediaType`, `Content`, `Digest`, + `OriginURI`, and `SizeBytes`; + - a helper-friendly collection type, such as `LLMInputSet`, if useful. +- Add prompt-execution fields to `StructuredCompletionRequest`: + - `PromptID`; + - `PromptVersion`; + - `ProfileID` or reuse the request `LLMProfile` value when called from stage + requests; + - `SessionID`; + - `Inputs map[string]LLMInputMaterial`; + - `Vars map[string]any`. +- Keep or remove the old `Messages`, `Model`, `ResponseSchemaName`, and + `ResponseSchema` fields according to what makes the cutover cleanest. The + final production path must not require modules to pass rendered messages or + raw schema JSON directly to the provider adapter. +- Add `SourceInput contracts.LLMInputMaterial` and `SessionID string` to + `ChunkRequest`, `ExtractionRequest`, and `NormalizeRequest`. +- Add `SessionID string` to `pipeline.RunInput`. +- In `pipeline.Run`, build a source input material from `RunInput.RawInput` and + `RunInput.Path`: + - preserve the bytes exactly; + - infer media type from the input path extension, using `application/json` + for `.json` and a deterministic fallback for unknown extensions; + - compute a SHA-256 digest; + - use a file URI or path-derived origin URI; + - do not put the bytes in manifest metadata. +- Pass the same source input material and session ID to chunk, extract, and + normalize requests. +- Reuse existing materialized `ReferenceSet` content for reference prompt + inputs. Do not introduce a second file-reading path for references. +- Add CLI support for `--session-id ` on `notarius run`. + - Empty means use a deterministic default derived from the parsed source + document ID. + - The deterministic default must be stable across runs over the same parsed + source document. + - The explicit value should be trimmed and rejected if empty after trimming. +- Because the default depends on the parsed source document, compute and attach + the final session ID inside the runner after input parsing, or return enough + information from parsing for the CLI to resolve it before LLM calls. +- Record the non-secret session ID in run metadata or manifest metadata, but + never record raw prompt content. + +Acceptance tests: + +- Contract tests proving `LLMInputMaterial.Content` is defensively copied where + relevant and omitted from JSON. +- Pipeline tests proving chunk, extract, and normalize requests receive the same + source input bytes and session ID. +- CLI tests for `--session-id`, including explicit value, missing value, + trimming, and default behavior. +- Redaction/diagnostics tests proving raw source bytes and reference content do + not appear in manifests or default diagnostics. + +Focused checks: + +- `go test ./internal/framework/contracts` +- `go test ./internal/framework/pipeline` +- `go test ./internal/cli` + +## Stage 3: Configuration Hard Cutover To Scriptorium Profiles + +Goal: replace Notarius' local LLM profile schema with Scriptorium profile +selection and profile sources. + +Work: + +- Bump the Notarius file config version because this is an incompatible config + schema change. +- Remove the top-level `llm_profiles` file-config schema and the local + `config.LLMProfile` model. +- Add a top-level Scriptorium config block. Use this shape unless Scriptorium's + installed API requires a small naming adjustment: + + ```yaml + scriptorium: + profile_dir: ./profiles + profile_file: ./profiles.yml + ``` + +- Treat `profile_dir` and `profile_file` as mutually exclusive in Notarius + config validation. Built-in Scriptorium profiles remain available when neither + is set. +- Keep existing module binding field name `llm_profile`; it now names a + Scriptorium profile ID. +- Stop forcing empty module `llm_profile` bindings to + `pipeline.DefaultLLMProfile`. Empty means "use the Scriptorium prompt's + `default_profile`." +- Keep `--llm-profile` as a run-level operational override. It should set the + same explicit Scriptorium profile ID for every LLM-eligible selected target. +- Remove `OpenAICompatibleClientConfig` construction from `internal/core/config`. +- Remove local OpenAI-compatible provider validation from config validation. +- Add config validation for: + - config version; + - mutually exclusive `scriptorium.profile_dir` and + `scriptorium.profile_file`; + - non-empty profile source paths when fields are present; + - non-empty explicit `llm_profile` strings after trimming. +- Add CLI/config validation that uses Scriptorium profile loading to reject + unknown explicit profile IDs when possible. Prompt-default profile failures + may surface during Scriptorium prepare/run if Scriptorium owns that lookup. +- Remove or replace environment override behavior tied to + `NOTARIUS_LLM_DEFAULT_*`. Secrets should come from Scriptorium profile + `api_key_env` values or direct request-scoped Scriptorium behavior. +- Update redacted effective config behavior for the new Scriptorium config + shape. + +Acceptance tests: + +- Config parsing accepts `scriptorium.profile_dir`. +- Config parsing accepts `scriptorium.profile_file`. +- Config validation rejects both fields set at once. +- Config validation rejects stale `llm_profiles`. +- Existing pipeline bindings with explicit `llm_profile` resolve to trimmed + Scriptorium profile IDs. +- Empty `llm_profile` remains empty through resolution unless `--llm-profile` + is supplied. +- CLI config validation fails cleanly for an unknown explicit profile ID. +- Redacted config diagnostics do not contain raw API keys. + +Focused checks: + +- `go test ./internal/core/config` +- `go test ./internal/cli` + +## Stage 4: Scriptorium Prompt And Schema Assets + +Goal: move production LLM prompts and schemas to Scriptorium-compatible assets +while preserving module ownership. + +Work: + +- Create a prompt/schema asset registration mechanism that does not put + D&D-specific prompt content in framework packages. + - Recommended shape: add a small prompt-asset registry in + `internal/framework/llm` or a sibling framework package that can collect + `fs.FS` roots for Scriptorium prompt and schema sources. + - Production module packages should register their own Scriptorium prompt and + schema assets through production catalog/registry wiring in `internal/cli`. + - Shared D&D prompt assets may live in a D&D-specific module package such as + `internal/modules/dnd/promptassets`; they must not live in `internal/core` + or source-agnostic framework packages. +- Add shared D&D prompt assets: + - stable shared system message, if needed; + - stable cacheable transcript user message: + + ```text + A transcript of a Dungeons & Dragons gameplay session is provided below. + + {{ input "transcript" }} + ``` + + - stable cacheable reference/context message templates for roster, glossary, + previous recap, or other large references used by current D&D modules. +- Convert `dnd/scenes` to a Scriptorium prompt definition: + - prompt ID: `dnd.scenes`; + - prompt version: current module prompt version; + - input `transcript`, required, `application/json`; + - messages ordered for cache reuse: shared system, shared transcript user + message with cache control, scene task, scene instructions; + - output JSON schema path pointing at the existing scene schema asset; + - schema IDs/names/versions remain module-owned and manifest-safe. +- Convert `dnd/spells` to a Scriptorium prompt definition: + - prompt ID: `dnd.spells`; + - prompt version: current module prompt version; + - input `transcript`, required, `application/json`; + - optional inputs for `roster` and `glossary`; + - messages ordered for cache reuse: shared system, shared transcript user + message with cache control, optional/reference context message with cache + control, spell task, spell instructions; + - output JSON schema path pointing at the existing spell schema asset. +- For optional references, pass empty inline input material when the slot is + unbound unless Scriptorium's template/input semantics support missing + optional inputs cleanly. Do not let optional missing references make prompt + rendering fail. +- For multiple reference items in one slot, concatenate deterministically with + stable headings that include only non-secret provenance, then pass the result + as that prompt input. Existing single-item slots should keep their current + behavior. +- Replace current prompt hash metadata with hashes derived from the + Scriptorium prompt definition plus message assets, or with Scriptorium + prepared-run prompt metadata if it is available without raw prompt content. +- Preserve existing manifest metadata keys where practical: + - `prompt_id`; + - `prompt_version`; + - `prompt_sha256`; + - `response_schema_key`; + - `response_schema_id`; + - `response_schema_name`; + - `response_schema_version`; + - `response_schema_sha256`. + +Acceptance tests: + +- Prompt asset loading fails fast for missing prompt files or schemas. +- `dnd/scenes` prepared prompt contains separate transcript and task messages. +- `dnd/spells` prepared prompt contains separate transcript, reference, and + task messages. +- The transcript message body is byte-identical to the expected shared template + plus original Seriatim JSON bytes. +- Reference prompt input rendering is deterministic. +- Prompt/schema diagnostics omit raw prompt text, raw transcript bytes, raw + reference content, and raw schema JSON. + +Focused checks: + +- `go test ./internal/framework/llm` +- `go test ./internal/modules/chunk/dnd/scenes` +- `go test ./internal/modules/extract/dnd/spells` + +## Stage 5: Scriptorium-Backed LLM Runtime + +Goal: implement the production `StructuredLLMClient` using Scriptorium. + +Work: + +- Add a Scriptorium-backed client under `internal/framework/llm`. +- Its constructor should accept: + - Scriptorium profile source settings from effective Notarius config; + - registered prompt/schema assets; + - request timeout or HTTP client settings only if still owned by Notarius + after the profile cutover; + - optional Scriptorium engine options for tests. +- Implement `CompleteStructured(ctx, req, out)` by: + - validating `out` is a non-nil pointer; + - validating `req.PromptID` is non-empty; + - converting Notarius `LLMInputMaterial` values to Scriptorium artifact refs; + - adding `session_id` to Scriptorium request vars when non-empty; + - passing explicit profile ID only when the module binding or CLI override + supplied one; + - passing no raw API key unless a deliberate request-scoped secret path is + implemented; + - calling Scriptorium `Run`; + - converting final validation failure into a Notarius error; + - unmarshaling successful structured JSON into `out`; + - returning `StructuredCompletionResponse` with raw JSON content, provider, + model, profile ID when available, token usage, and non-secret metadata. +- Add a profile recorder or response accumulator so `RunManifest.LLMProfiles` + records the actual Scriptorium profile/provider/model values used during the + run. + - Do not rely on one precomputed profile ID before pipeline execution. + - Deduplicate profile manifest entries deterministically. +- Keep the existing `Scheduler` and scheduled client wrapper unless Scriptorium + provides an equivalent Notarius-approved concurrency mechanism. +- Ensure all Scriptorium errors are wrapped with context and converted to + concise CLI-facing errors. Preserve `errors.Is` checks internally when + practical. +- Apply Notarius secret redaction to errors before writing diagnostics. +- Remove production construction of `OpenAICompatibleClient`. + +Acceptance tests: + +- Scriptorium adapter maps Notarius prompt request fields into the expected + Scriptorium run request using an injected fake Scriptorium LLM client. +- Successful structured output unmarshals into the caller target. +- Scriptorium validation failure returns an error. +- Provider/runtime failure returns an error with operation context. +- Context cancellation is respected. +- Token usage maps into `StructuredCompletionResponse`. +- Used Scriptorium profile/provider/model metadata appears in the run manifest + without secrets. +- API keys or bearer tokens in synthetic errors are redacted. +- Scheduler still bounds concurrent Scriptorium-backed calls. + +Focused checks: + +- `go test ./internal/framework/llm` +- `go test ./internal/framework/pipeline` +- `go test ./internal/cli` + +## Stage 6: Module Cutover And Legacy Runtime Removal + +Goal: update production modules to call the prompt-based contract and remove +obsolete local prompt/runtime code. + +Work: + +- Update `dnd/scenes`: + - stop rendering local system/user prompt strings; + - call `CompleteStructured` with `PromptID`, `PromptVersion`, `SessionID`, + `ProfileID` or request LLM profile, `transcript` input material, and any + required vars; + - keep existing response validation, chunk canonicalization, caveat handling, + and manifest metadata policy. +- Update `dnd/spells`: + - stop rendering local system/user prompt strings; + - call `CompleteStructured` with `transcript`, optional `roster`, optional + `glossary`, session ID, profile ID, and vars; + - keep existing spell response validation, source-reference validation, and + manifest metadata policy. +- Update any LLM-backed normalize modules if present. If only noop normalize is + present, ensure the contract and tests prove normalizers can receive the same + Scriptorium-capable client and prompt inputs. +- Remove the old `internal/framework/prompt` renderer if no remaining code uses + it. If generic tests still need prompt rendering, replace them with + Scriptorium prompt asset tests or delete obsolete tests. +- Remove `internal/framework/llm/openai_compatible_client.go` and its tests + after the Scriptorium adapter tests cover replacement behavior. +- Remove local OpenAI-compatible integration docs after current-behavior docs + are updated in Stage 7. +- Remove stale schema registry helpers only if they are no longer needed for + module-owned schema metadata. Keep lightweight schema hashing/loading helpers + if modules still use them for manifest metadata. + +Acceptance tests: + +- `dnd/scenes` fake-client tests assert the module sends prompt ID, + transcript input, session ID, and schema/prompt metadata rather than rendered + message text. +- `dnd/spells` fake-client tests assert roster/glossary inputs are passed as + inputs and not interpolated locally. +- Existing malformed LLM response tests still fail as malformed structured + output. +- Existing source-reference validation tests still pass. +- No production code imports the old prompt renderer or local + OpenAI-compatible client. + +Focused checks: + +- `go test ./internal/modules/chunk/dnd/scenes` +- `go test ./internal/modules/extract/dnd/spells` +- If `internal/framework/prompt` is deleted, do not run a package-specific test + for it; instead verify with `rg -n "internal/framework/prompt|RenderUserSystem" internal` + that no production code still depends on it. + +## Stage 7: CLI, Examples, Docs, And Full Validation + +Goal: finish user-facing behavior, examples, and canonical docs for the new +runtime. + +Work: + +- Update `examples/dnd-spells.config.yml` and any maintained test configs to + the new config version and Scriptorium profile source behavior. +- Update `docs/config.md`: + - new config version; + - `scriptorium.profile_dir` and `scriptorium.profile_file`; + - `llm_profile` now means Scriptorium profile ID; + - empty `llm_profile` behavior; + - removed `llm_profiles`; + - removed `NOTARIUS_LLM_DEFAULT_*` env behavior if removed; + - secret-handling policy through Scriptorium profile `api_key_env`. +- Update `docs/cli.md`: + - add `--session-id`; + - update `--llm-profile` wording to Scriptorium profile override; + - update run examples if needed. +- Update `docs/internal/llm.md`: + - Scriptorium-backed runtime; + - prompt asset loading; + - input material handling; + - scheduling; + - structured-output validation; + - profile manifest recording; + - secret redaction. +- Update `docs/internal/modules.md`: + - module prompt ownership through Scriptorium prompt definitions; + - chunk/extract/normalize request input materials and session ID; + - D&D scenes/spells prompt metadata. +- Update `docs/internal/pipeline.md`: + - raw input material lifecycle; + - reference material lifecycle into prompt inputs; + - session ID lifecycle; + - actual LLM profile provenance. +- Update `docs/integrations/json-output.md` for any manifest changes. +- Remove or rewrite `docs/integrations/openai-compatible.md`. If no local + OpenAI-compatible adapter remains, do not document it as current Notarius + behavior. +- Update `docs/troubleshooting.md` for: + - Scriptorium profile-not-found; + - prompt-not-found; + - schema/validation failures; + - missing API key env values; + - session ID usage if relevant. +- Update `docs/operations.md` if diagnostics or retention behavior changes. +- Remove the Scriptorium migration item from `docs/roadmap/future.md` once the + feature is implemented. +- Replace `docs/roadmap/implementation.md` with a completed note or remove it + after implementation is complete, according to the repository's current + roadmap cleanup pattern. + +Acceptance tests and inspections: + +- `rg -n "llm_profiles|OpenAICompatibleClient|openai-compatible|RenderUserSystem|NOTARIUS_LLM_DEFAULT" internal docs examples` + should return only intentional historical/deferred references, if any. +- `rg -n "scriptorium|session-id|profile_dir|profile_file" docs examples` + should show current-behavior docs and examples are updated. +- `go test ./...` +- `go vet ./...` +- `go build ./cmd/notarius` + +## Cross-Stage Review Checklist + +Before considering the implementation complete, verify: + +- No Scriptorium public types appear in chunk, extract, or normalize contracts. +- No raw source input, reference content, prompt text, schema JSON, API key, or + bearer token appears in manifests or default diagnostics. +- The shared transcript message uses original Seriatim JSON bytes exactly. +- Large reference content can be supplied as Scriptorium prompt inputs without + changing module-facing reference slot contracts. +- `--session-id` is easy for an external orchestrator to pass. +- Empty `llm_profile` lets prompt defaults work; explicit `llm_profile` and + `--llm-profile` select Scriptorium profile IDs. +- Production Notarius LLM execution goes through Scriptorium. +- Current-behavior documentation describes only implemented behavior. diff --git a/docs/roadmap/scriptorium.md b/docs/roadmap/scriptorium.md new file mode 100644 index 00000000..8d841c8d --- /dev/null +++ b/docs/roadmap/scriptorium.md @@ -0,0 +1,295 @@ +# Scriptorium LLM Runtime Roadmap + +This roadmap describes the target state for replacing Notarius' local +OpenAI-compatible LLM adapter with an integration based on +`gitea.maximumdirect.net/eric/scriptorium`. + +The upgrade is worthwhile only if it preserves Notarius' architectural +boundaries: + +- modules own prompt intent, prompt variables, response schemas, validation, and + domain interpretation; +- provider plumbing stays behind framework LLM contracts; +- diagnostics and manifests remain Notarius-owned and secret-free; +- raw API keys are resolved operationally and must not be stored in config, + diagnostics, manifests, examples, or prompt/profile assets; +- Scriptorium types do not leak into chunk, extract, or normalize module + contracts unless explicitly chosen as a future public contract. + +## Motivation + +Notarius currently has a local structured-output OpenAI-compatible adapter. That +adapter is intentionally narrow, but it makes Notarius responsible for provider +request shape, profile support, structured-output retries, model-specific +extensions, and future cache-related request fields. + +Scriptorium adds capabilities that are directly useful for Notarius: + +- built-in profiles for many providers and model families, including OpenRouter + and local OpenAI-compatible inference endpoints; +- prompt definitions with multiple system and user messages; +- per-prompt `session_id` support for sticky provider routing and input cache + affinity; +- per-message `cache_control` support for providers that understand it; +- a simple structured-output path where a prompt supplies a schema and + Scriptorium sends it upstream and retries non-compliant responses. + +The most important near-term motivation is input-cache reuse. Existing D&D +recap prompts already use a cacheable user message shaped as: + +```text +A transcript of a Dungeons & Dragons gameplay session is provided below. + +{{ input "transcript" }} +``` + +The `transcript` input is the same Seriatim JSON transcript that Notarius +already accepts as MVP input, including stable numbered segments. Notarius +should align chunk, extract, and normalize LLM calls so this transcript message +can be byte-identical across recap generation and structured extraction work. +For large transcripts, this can materially reduce provider cost when cached +input reads are available. + +## Target State + +Notarius uses Scriptorium as the production LLM execution engine behind the +existing module-facing LLM boundary. + +Chunk, extract, and normalize modules continue to receive a Notarius +`StructuredLLMClient` through their existing request contracts. They do not +construct provider clients, read secrets, or depend directly on Scriptorium +request/result types. + +The production CLI constructs a Scriptorium-backed client from the effective +Notarius configuration, wraps it in Notarius scheduling and diagnostics policy +where needed, and returns Notarius-owned non-secret LLM profile manifest +metadata. + +Module-owned prompts are rendered as ordered chat messages, not only as one +system message and one user message. Prompt rendering must support: + +- repeated roles; +- shared input messages; +- module-specific task and instruction messages; +- optional cache-control metadata; +- a stable session ID for calls that should share provider routing/cache + affinity; +- structured-output schema metadata owned by the module. + +The shared D&D transcript message should be rendered from the original Seriatim +input bytes, not reconstructed from normalized source units. The parsed source +document remains the canonical source-reference model for validation and +artifact grounding, but the cacheable transcript prompt message should preserve +the exact transcript payload supplied to the run. + +Other large text inputs, including references such as campaign glossaries, +rosters, previous recaps, and future reference producers, should follow the same +input-material model. They should be available to prompt rendering as stable +input artifacts that can be placed in cacheable messages without being copied +into manifests or diagnostics. + +## Boundary Requirements + +### Provider Plumbing + +Provider-specific request fields, profile expansion, and OpenAI-compatible wire +details should be isolated in the framework LLM runtime or in the +Scriptorium-backed adapter. Stage modules should request structured completion +through Notarius contracts and should not know whether the backing client is the +legacy local adapter or Scriptorium. + +### Prompt Ownership + +Modules remain responsible for prompt intent, prompt versioning, response schema +selection, and semantic validation. Scriptorium provides the prompt definition, +rendering, execution, and structured-output workflow, but it should not become +the owner of D&D-specific semantics. + +Shared prompt assets are allowed when they express generic reusable context, +such as the D&D transcript input message. Module-specific task prompts should +remain owned by the relevant module. + +### Diagnostics And Manifests + +Notarius should continue to decide what appears in diagnostics and run +manifests. Scriptorium prepared/run metadata may be useful input, but Notarius +must filter it through existing policy: + +- no raw prompt payloads by default; +- no raw response schema content in manifests; +- no source transcript payloads in manifests; +- no API keys or bearer tokens; +- enough prompt/profile/schema hashes and IDs to audit a run later. + +### Secret Handling + +Raw API keys should stay out of durable config and prompt/profile assets. +Notarius may continue resolving API keys from configured environment-variable +names, then pass the secret request-scope to Scriptorium. Redacted diagnostics +must continue to prove that resolved secrets are not serialized. + +## Prompt And Cache Strategy + +The desired D&D LLM prompt shape is: + +1. stable shared system message; +2. stable shared transcript user message containing the original Seriatim JSON; +3. optional stable cacheable context messages, such as previous recap, + glossary, roster, or other references; +4. module-specific task message; +5. module-specific instruction message; +6. module-owned structured-output schema, when the module expects JSON. + +The transcript message should be byte-identical whenever the same transcript +input bytes are used. Avoid reconstructing JSON from parsed source units because +formatting, key order, whitespace, or escaping changes would defeat cache reuse. + +The transcript and large reference messages should be cacheable when the active +provider path supports cache-control metadata. Providers that ignore cache +controls should still receive a valid prompt. + +The session ID should be stable for all LLM calls that operate on the same +session transcript and should be explicitly visible in diagnostics or manifest +metadata only as a non-secret identifier. + +Notarius should provide an easy CLI UX for setting the session ID. This lets an +external D&D pipeline orchestrator pass the same session ID used by recap or +other LLM steps, maximizing provider routing affinity and cached input reuse. + +## Configuration Intent + +Notarius configuration should remain the user-facing source of pipeline +composition and operational settings. For LLM execution profiles, Notarius +should cut over to Scriptorium profile configuration instead of maintaining a +separate Notarius-specific profile schema. + +Notarius should allow configuration to select Scriptorium built-in profiles and +to point at Scriptorium profile files or directories. This avoids evolving two +nearly identical profile systems and gives users immediate access to the +provider and model catalog that motivated the migration. + +Notarius still owns validation, redaction, and manifest provenance for the +effective pipeline. The integration should wrap Scriptorium profile loading so +that errors are actionable, secrets remain environment-based, and emitted +metadata stays non-secret. + +## Compatibility Policy + +This feature should be a hard cutover to Scriptorium-backed prompt execution and +profile loading. Backward compatibility with the local Notarius prompt renderer, +local LLM profile schema, or local OpenAI-compatible adapter is not a product +requirement for this migration. + +The final production runtime should have one documented LLM execution path. +Module behavior may still be tested with fake Notarius `StructuredLLMClient` +implementations, but production runtime behavior should be Scriptorium-backed. + +## Documentation Outcomes + +When this feature is implemented, current-behavior docs should be updated to +describe: + +- the implemented production LLM runtime; +- supported LLM profile fields and provider/profile selection behavior; +- cacheable transcript prompt behavior, if exposed to users; +- session ID behavior; +- diagnostics and manifest provenance; +- troubleshooting for Scriptorium profile, prompt, schema, and validation + failures. + +The OpenAI-compatible integration doc should either be retired, narrowed to the +legacy fallback, or reframed as an upstream provider contract delegated through +Scriptorium, depending on the final runtime shape. + +## Deferred Work + +The Scriptorium migration does not itself require: + +- a general workflow language; +- arbitrary per-stage prompt authoring by end users; +- token budgeting or context-window planning; +- non-file reference producers; +- semantic retrieval over transcript or reference content; +- multiple effective LLM profiles in one Notarius run. + +Those remain separate future features. + +## Resolved Design Choices + +### Prompt Asset Ownership And Format + +Production LLM prompts should cut over to Scriptorium-compatible prompt +definitions without preserving backward compatibility for the current Notarius +system/user prompt renderer. + +Module-owned prompt definitions should keep task prompts and schemas near the +module that owns the semantic behavior. Shared prompt messages, such as the D&D +transcript input message and shared reference/context messages, should live in a +shared prompt asset area and be referenced by module-owned prompt definitions. + +This uses Scriptorium's native strengths: ordered messages, cache-control +metadata, session IDs, input helpers, profile selection, and schema-backed +execution. It also gives Notarius a direct way to reuse the exact transcript and +reference messages across D&D recap, chunking, extraction, and normalization +work without each module hand-rolling message assembly. + +### Raw Transcript Preservation Location + +Original input bytes should be preserved as run-scoped input material and made +available to prompt rendering without storing them in run manifests. The parsed +`SourceDocument` should continue to carry normalized source units for framework +logic and source-reference validation. + +The same model should apply to all large text inputs, including references. +Glossaries, campaign rosters, previous recaps, and other future reference +documents may be large enough to deserve their own cacheable messages. They +should be treated as prompt input artifacts rather than being copied into source +document metadata. + +This keeps the cacheable transcript message byte-identical to the original +Seriatim JSON and avoids source-format leakage into chunk, extract, and +normalize contracts. It also reduces the risk that raw inputs or references are +accidentally serialized into diagnostics or manifests. + +### Session ID Source + +Notarius should add an explicit session ID concept for LLM calls, with a +deterministic default derived from the source document identity when no +operator-provided value is configured. + +The CLI should provide an easy way to supply this session ID. This lets an +external D&D pipeline orchestrator call Notarius with the same session ID used +by recap generation or other LLM steps, maximizing provider routing affinity and +cached input savings. + +OpenRouter sticky routing and provider cache behavior are most useful when +every related call for the same session shares a stable identifier. An explicit +concept makes the behavior auditable and avoids each module inventing its own +ID. A deterministic default keeps simple local runs ergonomic. + +### Scriptorium Profile Exposure + +Notarius should cut over to Scriptorium profile configuration instead of +maintaining a separate Notarius LLM profile schema. Configuration should be able +to select Scriptorium built-in profiles and point at Scriptorium profile files +or directories. + +The Scriptorium profile format is already close to the desired Notarius target. +Maintaining a separate Notarius profile schema would likely create duplicate +configuration that eventually converges back toward Scriptorium's model. A hard +cutover avoids that churn and gives users immediate access to Scriptorium's +provider and model catalog. + +Notarius still needs to wrap this profile loading with Notarius-owned +validation, diagnostics, manifest provenance, and secret-redaction policy. + +### Structured-Output Retry Ownership + +Notarius should use Scriptorium's structured-output execution and retry behavior +for production calls, while translating results and errors back into Notarius' +`StructuredLLMClient` response and error expectations. + +This avoids duplicating structured-output enforcement in Notarius and lets +Scriptorium own provider-specific request and repair mechanics. Notarius still +retains module-level validation and source-reference checks after decoded +structured output is returned.