notarius/docs/roadmap/implementation.md

# Scriptorium Cutover Implementation Plan

This plan implements the target state in
[scriptorium.md](scriptorium.md): a hard cutover from Notarius' local
OpenAI-compatible adapter, local LLM profile schema, and local system/user
prompt renderer to Scriptorium-backed prompt execution.

The target implementer is an LLM coding agent. Complete each stage in order.
Do not preserve backward compatibility for removed local LLM-profile or
prompt-rendering behavior unless a stage explicitly says to keep a temporary
test seam.

## Global Constraints

- Keep Scriptorium types out of chunk, extract, and normalize module contracts.
  Module-facing requests should use Notarius-owned contract types.
- Keep provider plumbing behind `internal/framework/llm` and CLI construction.
- Keep module-owned prompt intent, prompt IDs, response schemas, validators, and
  source-reference validation with the relevant modules.
- Keep raw input material, raw references, raw prompts, raw schemas, and API
  keys out of manifests and default diagnostics.
- Treat the original source input and large references as prompt input
  materials. They may be passed to Scriptorium as inline/file inputs, but should
  not be copied into durable provenance.
- Use Scriptorium profile configuration directly. Do not keep Notarius'
  `llm_profiles` schema as a second profile system.
- Use Scriptorium structured-output execution and validation behavior for
  production calls. Notarius should still unmarshal successful structured JSON
  into module-owned response structs and run module-owned deterministic
  validation afterward.

## Stage 1: Dependency And API Grounding

Goal: add the Scriptorium dependency and establish the exact public API surface
used by the rest of the implementation.

Work:

- Add `gitea.maximumdirect.net/eric/scriptorium` to `go.mod` with `go get`.
- Inspect the installed package with `go doc` or source reads before coding the
  adapter. Confirm names and fields for:
  - `NewEngine`;
  - `Config`;
  - `WithPromptFS`, `WithPromptFile`, or equivalent prompt source options;
  - `WithProfileFile`, `WithProfileFS`, profile directory config, and built-in
    profile behavior;
  - `WithSchemaFS` or equivalent schema source options;
  - `RunRequest`, including prompt ID, prompt version, profile ID, inputs,
    variables, metadata, direct API key, and validation override fields;
  - `RunResult`, including raw output, output artifact, validation status,
    prompt/profile/model metadata, usage, and timings;
  - public sentinel errors.
- Add a short internal implementation note as a code comment only where needed;
  do not add user-facing docs in this stage except if a package-level test
  helper needs explanation.

Acceptance checks:

- `go test ./internal/framework/llm`
- `go test ./internal/cli`
- `go test ./...`

## Stage 2: Prompt Input Materials And Session IDs

Goal: make raw source input and references available to module prompt execution
without storing large content in manifests or default diagnostics.

Work:

- Add Notarius-owned contract types under `internal/framework/contracts`:
  - `LLMInputMaterial` with at least `Name`, `MediaType`, `Content`, `Digest`,
    `OriginURI`, and `SizeBytes`;
  - a helper-friendly collection type, such as `LLMInputSet`, if useful.
- Add prompt-execution fields to `StructuredCompletionRequest`:
  - `PromptID`;
  - `PromptVersion`;
  - `ProfileID` or reuse the request `LLMProfile` value when called from stage
    requests;
  - `SessionID`;
  - `Inputs map[string]LLMInputMaterial`;
  - `Vars map[string]any`.
- Keep or remove the old `Messages`, `Model`, `ResponseSchemaName`, and
  `ResponseSchema` fields according to what makes the cutover cleanest. The
  final production path must not require modules to pass rendered messages or
  raw schema JSON directly to the provider adapter.
- Add `SourceInput contracts.LLMInputMaterial` and `SessionID string` to
  `ChunkRequest`, `ExtractionRequest`, and `NormalizeRequest`.
- Add `SessionID string` to `pipeline.RunInput`.
- In `pipeline.Run`, build a source input material from `RunInput.RawInput` and
  `RunInput.Path`:
  - preserve the bytes exactly;
  - infer media type from the input path extension, using `application/json`
    for `.json` and a deterministic fallback for unknown extensions;
  - compute a SHA-256 digest;
  - use a file URI or path-derived origin URI;
  - do not put the bytes in manifest metadata.
- Pass the same source input material and session ID to chunk, extract, and
  normalize requests.
- Reuse existing materialized `ReferenceSet` content for reference prompt
  inputs. Do not introduce a second file-reading path for references.
- Add CLI support for `--session-id <id>` on `notarius run`.
  - Empty means use a deterministic default derived from the parsed source
    document ID.
  - The deterministic default must be stable across runs over the same parsed
    source document.
  - The explicit value should be trimmed and rejected if empty after trimming.
- Because the default depends on the parsed source document, compute and attach
  the final session ID inside the runner after input parsing, or return enough
  information from parsing for the CLI to resolve it before LLM calls.
- Record the non-secret session ID in run metadata or manifest metadata, but
  never record raw prompt content.

Acceptance tests:

- Contract tests proving `LLMInputMaterial.Content` is defensively copied where
  relevant and omitted from JSON.
- Pipeline tests proving chunk, extract, and normalize requests receive the same
  source input bytes and session ID.
- CLI tests for `--session-id`, including explicit value, missing value,
  trimming, and default behavior.
- Redaction/diagnostics tests proving raw source bytes and reference content do
  not appear in manifests or default diagnostics.

Focused checks:

- `go test ./internal/framework/contracts`
- `go test ./internal/framework/pipeline`
- `go test ./internal/cli`

## Stage 3: Configuration Hard Cutover To Scriptorium Profiles

Goal: replace Notarius' local LLM profile schema with Scriptorium profile
selection and profile sources.

Work:

- Bump the Notarius file config version because this is an incompatible config
  schema change.
- Remove the top-level `llm_profiles` file-config schema and the local
  `config.LLMProfile` model.
- Add a top-level Scriptorium config block. Use this shape unless Scriptorium's
  installed API requires a small naming adjustment:

  ```yaml
  scriptorium:
    profile_dir: ./profiles
    profile_file: ./profiles.yml
  ```

- Treat `profile_dir` and `profile_file` as mutually exclusive in Notarius
  config validation. Built-in Scriptorium profiles remain available when neither
  is set.
- Keep existing module binding field name `llm_profile`; it now names a
  Scriptorium profile ID.
- Stop forcing empty module `llm_profile` bindings to
  `pipeline.DefaultLLMProfile`. Empty means "use the Scriptorium prompt's
  `default_profile`."
- Keep `--llm-profile` as a run-level operational override. It should set the
  same explicit Scriptorium profile ID for every LLM-eligible selected target.
- Remove `OpenAICompatibleClientConfig` construction from `internal/core/config`.
- Remove local OpenAI-compatible provider validation from config validation.
- Add config validation for:
  - config version;
  - mutually exclusive `scriptorium.profile_dir` and
    `scriptorium.profile_file`;
  - non-empty profile source paths when fields are present;
  - non-empty explicit `llm_profile` strings after trimming.
- Add CLI/config validation that uses Scriptorium profile loading to reject
  unknown explicit profile IDs when possible. Prompt-default profile failures
  may surface during Scriptorium prepare/run if Scriptorium owns that lookup.
- Remove or replace environment override behavior tied to
  `NOTARIUS_LLM_DEFAULT_*`. Secrets should come from Scriptorium profile
  `api_key_env` values or direct request-scoped Scriptorium behavior.
- Update redacted effective config behavior for the new Scriptorium config
  shape.

Acceptance tests:

- Config parsing accepts `scriptorium.profile_dir`.
- Config parsing accepts `scriptorium.profile_file`.
- Config validation rejects both fields set at once.
- Config validation rejects stale `llm_profiles`.
- Existing pipeline bindings with explicit `llm_profile` resolve to trimmed
  Scriptorium profile IDs.
- Empty `llm_profile` remains empty through resolution unless `--llm-profile`
  is supplied.
- CLI config validation fails cleanly for an unknown explicit profile ID.
- Redacted config diagnostics do not contain raw API keys.

Focused checks:

- `go test ./internal/core/config`
- `go test ./internal/cli`

## Stage 4: Scriptorium Prompt And Schema Assets

Goal: move production LLM prompts and schemas to Scriptorium-compatible assets
while preserving module ownership.

Work:

- Create a prompt/schema asset registration mechanism that does not put
  D&D-specific prompt content in framework packages.
  - Recommended shape: add a small prompt-asset registry in
    `internal/framework/llm` or a sibling framework package that can collect
    `fs.FS` roots for Scriptorium prompt and schema sources.
  - Production module packages should register their own Scriptorium prompt and
    schema assets through production catalog/registry wiring in `internal/cli`.
  - Shared D&D prompt assets may live in a D&D-specific module package such as
    `internal/modules/dnd/promptassets`; they must not live in `internal/core`
    or source-agnostic framework packages.
- Add shared D&D prompt assets:
  - stable shared system message, if needed;
  - stable cacheable transcript user message:

    ```text
    A transcript of a Dungeons & Dragons gameplay session is provided below.

    {{ input "transcript" }}
    ```

  - stable cacheable reference/context message templates for roster, glossary,
    previous recap, or other large references used by current D&D modules.
- Convert `dnd/scenes` to a Scriptorium prompt definition:
  - prompt ID: `dnd.scenes`;
  - prompt version: current module prompt version;
  - input `transcript`, required, `application/json`;
  - messages ordered for cache reuse: shared system, shared transcript user
    message with cache control, scene task, scene instructions;
  - output JSON schema path pointing at the existing scene schema asset;
  - schema IDs/names/versions remain module-owned and manifest-safe.
- Convert `dnd/spells` to a Scriptorium prompt definition:
  - prompt ID: `dnd.spells`;
  - prompt version: current module prompt version;
  - input `transcript`, required, `application/json`;
  - optional inputs for `roster` and `glossary`;
  - messages ordered for cache reuse: shared system, shared transcript user
    message with cache control, optional/reference context message with cache
    control, spell task, spell instructions;
  - output JSON schema path pointing at the existing spell schema asset.
- For optional references, pass empty inline input material when the slot is
  unbound unless Scriptorium's template/input semantics support missing
  optional inputs cleanly. Do not let optional missing references make prompt
  rendering fail.
- For multiple reference items in one slot, concatenate deterministically with
  stable headings that include only non-secret provenance, then pass the result
  as that prompt input. Existing single-item slots should keep their current
  behavior.
- Replace current prompt hash metadata with hashes derived from the
  Scriptorium prompt definition plus message assets, or with Scriptorium
  prepared-run prompt metadata if it is available without raw prompt content.
- Preserve existing manifest metadata keys where practical:
  - `prompt_id`;
  - `prompt_version`;
  - `prompt_sha256`;
  - `response_schema_key`;
  - `response_schema_id`;
  - `response_schema_name`;
  - `response_schema_version`;
  - `response_schema_sha256`.

Acceptance tests:

- Prompt asset loading fails fast for missing prompt files or schemas.
- `dnd/scenes` prepared prompt contains separate transcript and task messages.
- `dnd/spells` prepared prompt contains separate transcript, reference, and
  task messages.
- The transcript message body is byte-identical to the expected shared template
  plus original Seriatim JSON bytes.
- Reference prompt input rendering is deterministic.
- Prompt/schema diagnostics omit raw prompt text, raw transcript bytes, raw
  reference content, and raw schema JSON.

Focused checks:

- `go test ./internal/framework/llm`
- `go test ./internal/modules/chunk/dnd/scenes`
- `go test ./internal/modules/extract/dnd/spells`

## Stage 5: Scriptorium-Backed LLM Runtime

Goal: implement the production `StructuredLLMClient` using Scriptorium.

Work:

- Add a Scriptorium-backed client under `internal/framework/llm`.
- Its constructor should accept:
  - Scriptorium profile source settings from effective Notarius config;
  - registered prompt/schema assets;
  - request timeout or HTTP client settings only if still owned by Notarius
    after the profile cutover;
  - optional Scriptorium engine options for tests.
- Implement `CompleteStructured(ctx, req, out)` by:
  - validating `out` is a non-nil pointer;
  - validating `req.PromptID` is non-empty;
  - converting Notarius `LLMInputMaterial` values to Scriptorium artifact refs;
  - adding `session_id` to Scriptorium request vars when non-empty;
  - passing explicit profile ID only when the module binding or CLI override
    supplied one;
  - passing no raw API key unless a deliberate request-scoped secret path is
    implemented;
  - calling Scriptorium `Run`;
  - converting final validation failure into a Notarius error;
  - unmarshaling successful structured JSON into `out`;
  - returning `StructuredCompletionResponse` with raw JSON content, provider,
    model, profile ID when available, token usage, and non-secret metadata.
- Add a profile recorder or response accumulator so `RunManifest.LLMProfiles`
  records the actual Scriptorium profile/provider/model values used during the
  run.
  - Do not rely on one precomputed profile ID before pipeline execution.
  - Deduplicate profile manifest entries deterministically.
- Keep the existing `Scheduler` and scheduled client wrapper unless Scriptorium
  provides an equivalent Notarius-approved concurrency mechanism.
- Ensure all Scriptorium errors are wrapped with context and converted to
  concise CLI-facing errors. Preserve `errors.Is` checks internally when
  practical.
- Apply Notarius secret redaction to errors before writing diagnostics.
- Remove production construction of `OpenAICompatibleClient`.

Acceptance tests:

- Scriptorium adapter maps Notarius prompt request fields into the expected
  Scriptorium run request using an injected fake Scriptorium LLM client.
- Successful structured output unmarshals into the caller target.
- Scriptorium validation failure returns an error.
- Provider/runtime failure returns an error with operation context.
- Context cancellation is respected.
- Token usage maps into `StructuredCompletionResponse`.
- Used Scriptorium profile/provider/model metadata appears in the run manifest
  without secrets.
- API keys or bearer tokens in synthetic errors are redacted.
- Scheduler still bounds concurrent Scriptorium-backed calls.

Focused checks:

- `go test ./internal/framework/llm`
- `go test ./internal/framework/pipeline`
- `go test ./internal/cli`

## Stage 6: Module Cutover And Legacy Runtime Removal

Goal: update production modules to call the prompt-based contract and remove
obsolete local prompt/runtime code.

Work:

- Update `dnd/scenes`:
  - stop rendering local system/user prompt strings;
  - call `CompleteStructured` with `PromptID`, `PromptVersion`, `SessionID`,
    `ProfileID` or request LLM profile, `transcript` input material, and any
    required vars;
  - keep existing response validation, chunk canonicalization, caveat handling,
    and manifest metadata policy.
- Update `dnd/spells`:
  - stop rendering local system/user prompt strings;
  - call `CompleteStructured` with `transcript`, optional `roster`, optional
    `glossary`, session ID, profile ID, and vars;
  - keep existing spell response validation, source-reference validation, and
    manifest metadata policy.
- Update any LLM-backed normalize modules if present. If only noop normalize is
  present, ensure the contract and tests prove normalizers can receive the same
  Scriptorium-capable client and prompt inputs.
- Remove the old `internal/framework/prompt` renderer if no remaining code uses
  it. If generic tests still need prompt rendering, replace them with
  Scriptorium prompt asset tests or delete obsolete tests.
- Remove `internal/framework/llm/openai_compatible_client.go` and its tests
  after the Scriptorium adapter tests cover replacement behavior.
- Remove local OpenAI-compatible integration docs after current-behavior docs
  are updated in Stage 7.
- Remove stale schema registry helpers only if they are no longer needed for
  module-owned schema metadata. Keep lightweight schema hashing/loading helpers
  if modules still use them for manifest metadata.

Acceptance tests:

- `dnd/scenes` fake-client tests assert the module sends prompt ID,
  transcript input, session ID, and schema/prompt metadata rather than rendered
  message text.
- `dnd/spells` fake-client tests assert roster/glossary inputs are passed as
  inputs and not interpolated locally.
- Existing malformed LLM response tests still fail as malformed structured
  output.
- Existing source-reference validation tests still pass.
- No production code imports the old prompt renderer or local
  OpenAI-compatible client.

Focused checks:

- `go test ./internal/modules/chunk/dnd/scenes`
- `go test ./internal/modules/extract/dnd/spells`
- If `internal/framework/prompt` is deleted, do not run a package-specific test
  for it; instead verify with `rg -n "internal/framework/prompt|RenderUserSystem" internal`
  that no production code still depends on it.

## Stage 7: CLI, Examples, Docs, And Full Validation

Goal: finish user-facing behavior, examples, and canonical docs for the new
runtime.

Work:

- Update `examples/dnd-spells.config.yml` and any maintained test configs to
  the new config version and Scriptorium profile source behavior.
- Update `docs/config.md`:
  - new config version;
  - `scriptorium.profile_dir` and `scriptorium.profile_file`;
  - `llm_profile` now means Scriptorium profile ID;
  - empty `llm_profile` behavior;
  - removed `llm_profiles`;
  - removed `NOTARIUS_LLM_DEFAULT_*` env behavior if removed;
  - secret-handling policy through Scriptorium profile `api_key_env`.
- Update `docs/cli.md`:
  - add `--session-id`;
  - update `--llm-profile` wording to Scriptorium profile override;
  - update run examples if needed.
- Update `docs/internal/llm.md`:
  - Scriptorium-backed runtime;
  - prompt asset loading;
  - input material handling;
  - scheduling;
  - structured-output validation;
  - profile manifest recording;
  - secret redaction.
- Update `docs/internal/modules.md`:
  - module prompt ownership through Scriptorium prompt definitions;
  - chunk/extract/normalize request input materials and session ID;
  - D&D scenes/spells prompt metadata.
- Update `docs/internal/pipeline.md`:
  - raw input material lifecycle;
  - reference material lifecycle into prompt inputs;
  - session ID lifecycle;
  - actual LLM profile provenance.
- Update `docs/integrations/json-output.md` for any manifest changes.
- Remove or rewrite `docs/integrations/openai-compatible.md`. If no local
  OpenAI-compatible adapter remains, do not document it as current Notarius
  behavior.
- Update `docs/troubleshooting.md` for:
  - Scriptorium profile-not-found;
  - prompt-not-found;
  - schema/validation failures;
  - missing API key env values;
  - session ID usage if relevant.
- Update `docs/operations.md` if diagnostics or retention behavior changes.
- Remove the Scriptorium migration item from `docs/roadmap/future.md` once the
  feature is implemented.
- Replace `docs/roadmap/implementation.md` with a completed note or remove it
  after implementation is complete, according to the repository's current
  roadmap cleanup pattern.

Acceptance tests and inspections:

- `rg -n "llm_profiles|OpenAICompatibleClient|openai-compatible|RenderUserSystem|NOTARIUS_LLM_DEFAULT" internal docs examples`
  should return only intentional historical/deferred references, if any.
- `rg -n "scriptorium|session-id|profile_dir|profile_file" docs examples`
  should show current-behavior docs and examples are updated.
- `go test ./...`
- `go vet ./...`
- `go build ./cmd/notarius`

## Cross-Stage Review Checklist

Before considering the implementation complete, verify:

- No Scriptorium public types appear in chunk, extract, or normalize contracts.
- No raw source input, reference content, prompt text, schema JSON, API key, or
  bearer token appears in manifests or default diagnostics.
- The shared transcript message uses original Seriatim JSON bytes exactly.
- Large reference content can be supplied as Scriptorium prompt inputs without
  changing module-facing reference slot contracts.
- `--session-id` is easy for an external orchestrator to pass.
- Empty `llm_profile` lets prompt defaults work; explicit `llm_profile` and
  `--llm-profile` select Scriptorium profile IDs.
- Production Notarius LLM execution goes through Scriptorium.
- Current-behavior documentation describes only implemented behavior.