From 3217bb3e12fa02d9ad66cd9b1de5ebe26420109f Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 5 Jul 2026 20:13:26 -0500 Subject: [PATCH] Add a roadmap to refactor prompt assets --- docs/roadmap/implementation.md | 128 +++++++++++++++++++++++++++++---- docs/roadmap/scriptorium.md | 33 --------- docs/roadmap/sharedassets.md | 97 +++++++++++++++++++++++++ 3 files changed, 213 insertions(+), 45 deletions(-) delete mode 100644 docs/roadmap/scriptorium.md create mode 100644 docs/roadmap/sharedassets.md diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 12051e9..9572817 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,16 +1,120 @@ -# Scriptorium Runtime Migration +# Shared Prompt And Asset Layout Implementation Plan -The Scriptorium-backed LLM runtime migration is complete. +## Summary -Current implemented behavior is documented in: +Implement the asset layout defined in +[Shared Prompt And Asset Layout Roadmap](sharedassets.md). This is a structural +reorganization only: prompt semantics, prompt IDs, prompt versions, response +schema IDs, schema names, runtime module contracts, diagnostics policy, and +manifest provenance shape must remain unchanged. -- [Configuration](../config.md) -- [CLI Reference](../cli.md) -- [Operations](../operations.md) -- [Troubleshooting](../troubleshooting.md) -- [LLM Runtime Internals](../internal/llm.md) -- [Module Internals](../internal/modules.md) -- [Pipeline Internals](../internal/pipeline.md) -- [JSON Output](../integrations/json-output.md) +## Stage 1: Introduce Shared Assets Package -Remaining product and operational ideas belong in [Future Work](future.md). +- Create `internal/modules/sharedassets` as the cross-module shared asset + package. +- Move the existing shared D&D prompt fragments from + `internal/modules/dnd/promptassets` into `internal/modules/sharedassets`. +- Rename shared prompt files exactly as: + - `common-dnd-system.md` + - `common-dnd-transcript.md` + - `common-dnd-references.md` +- Implement the shared package with the same responsibilities as the current + shared prompt package: + - embed shared prompt files; + - register them with `llm.AssetRegistry`; + - expose hash-part helpers for common shared prompts and reference prompts. +- Keep Scriptorium-visible prompt paths under the shared prompt root as + `common-dnd-*.md`; do not preserve the old `shared/*.md` Scriptorium paths. +- Update production prompt asset collection in `internal/cli` and D&D module + prompt tests to import `internal/modules/sharedassets`. +- Do not delete the old `internal/modules/dnd/promptassets` package until all + callers have been updated and tests compile. + +## Stage 2: Flatten Module Asset Trees + +- Move `dnd/spells` module-owned prompt assets to: + - `internal/modules/extract/dnd/spells/assets/prompts/dnd.spells.yaml` + - `internal/modules/extract/dnd/spells/assets/prompts/task.md` + - `internal/modules/extract/dnd/spells/assets/prompts/instructions.md` +- Move `dnd/scenes` module-owned prompt assets to: + - `internal/modules/chunk/dnd/scenes/assets/prompts/dnd.scenes.yaml` + - `internal/modules/chunk/dnd/scenes/assets/prompts/task.md` + - `internal/modules/chunk/dnd/scenes/assets/prompts/instructions.md` +- Keep module schemas under each module's existing shallow schema directory: + - `assets/schemas/dnd_spells.v1.json` + - `assets/schemas/dnd_scenes.v1.json` +- Update each module's `go:embed` patterns to include only the new shallow + prompt and schema paths. +- Update each module's prompt registration root from the old + `assets/scriptorium/prompts` tree to `assets/prompts`. +- Update prompt hash inputs to point at the new module prompt paths and the new + shared asset hash parts. + +## Stage 3: Update Scriptorium Prompt Definitions + +- Update `dnd.spells.yaml` and `dnd.scenes.yaml` to use the new shallow + `content_file` paths. +- Shared prompt references must point to: + - `./common-dnd-system.md` + - `./common-dnd-transcript.md` + - `./common-dnd-references.md` +- Module prompt references must point to: + - `./task.md` + - `./instructions.md` +- Keep all existing prompt IDs, prompt versions, default profiles, inputs, + cache-control settings, output schema paths, validation modes, and repair + attempts unchanged. +- Confirm the flattened module prompt roots and shared prompt roots combine into + one Scriptorium prompt filesystem without duplicate asset paths. + +## Stage 4: Remove Old Package And Update Docs + +- Delete `internal/modules/dnd/promptassets` after all imports and asset + references are gone. +- Update canonical current-behavior docs to describe the new convention: + - module-owned prompts and schemas live under each module's shallow + `assets/prompts` and `assets/schemas` directories; + - shared reusable prompt fragments live under + `internal/modules/sharedassets`; + - framework and CLI code may collect assets, but modules still own prompt + intent, prompt IDs, schemas, and interpretation. +- Keep roadmap docs as future/target-state material only; do not describe + completed behavior in `docs/roadmap/` until the implementation is complete. + +## Stage 5: Verification + +- Update focused tests that assert asset paths, prepared prompt message content, + prompt hash metadata, or import paths. +- Add or preserve coverage proving: + - production prompt asset registration includes shared assets, `dnd/scenes` + assets, and `dnd/spells` assets; + - Scriptorium can prepare both D&D prompts using the moved assets; + - prompt and schema hashes remain present in module manifest metadata; + - raw source input, references, prompt text, and schema JSON still do not leak + into default diagnostics or manifests. +- Run focused validation: + +```sh +go test ./internal/framework/llm +go test ./internal/modules/chunk/dnd/scenes ./internal/modules/extract/dnd/spells +go test ./internal/cli +``` + +- Run full validation: + +```sh +go test ./... +go vet ./... +go build ./cmd/notarius +``` + +## Assumptions + +- This migration is allowed to change internal asset paths because prompt assets + are embedded implementation details, not public API. +- Prompt IDs, versions, schema IDs, schema names, runtime request contracts, and + output manifest shape are stable and must not change in this pass. +- `internal/modules/sharedassets` may contain D&D-specific reusable prompt + fragments when filenames make the domain scope explicit. +- Shared assets should contain durable operating rules and reusable framing, not + module-specific extraction, chunking, normalization, or validation semantics. diff --git a/docs/roadmap/scriptorium.md b/docs/roadmap/scriptorium.md deleted file mode 100644 index e75af12..0000000 --- a/docs/roadmap/scriptorium.md +++ /dev/null @@ -1,33 +0,0 @@ -# Completed Scriptorium LLM Runtime Migration - -The Scriptorium-backed LLM runtime migration is complete. Notarius now uses -`gitea.maximumdirect.net/eric/scriptorium` as the production LLM execution -engine behind Notarius-owned module contracts, diagnostics, manifest -provenance, and secret-handling policy. - -## Implemented Outcomes - -- production LLM execution uses Scriptorium prompt assets and profiles; -- chunk, extract, and normalize modules call LLMs through the Notarius - `StructuredLLMClient` boundary; -- modules pass prompt IDs, prompt versions, profile IDs, session IDs, input - materials, and prompt variables instead of locally rendered chat messages; -- source transcript input is preserved as run-scoped input material for - byte-stable cacheable prompt messages; -- Scriptorium profile provenance is recorded in run manifests without raw - prompts, schemas, source payloads, references, or secrets. - -## Canonical Docs - -Current behavior is documented in: - -- [Configuration](../config.md) -- [CLI Reference](../cli.md) -- [Internal LLM Runtime](../internal/llm.md) -- [Pipeline Internals](../internal/pipeline.md) -- [JSON Output Integration](../integrations/json-output.md) -- [Troubleshooting](../troubleshooting.md) - -There is no active Scriptorium-specific roadmap work in this file. Deferred -work related to broader LLM, reference, retrieval, context-window, or provider -strategy belongs in [Future Roadmap](future.md). diff --git a/docs/roadmap/sharedassets.md b/docs/roadmap/sharedassets.md new file mode 100644 index 0000000..e4e6e27 --- /dev/null +++ b/docs/roadmap/sharedassets.md @@ -0,0 +1,97 @@ +# Shared Prompt And Asset Layout Roadmap + +This roadmap defines the target state for reorganizing embedded prompt and +schema assets so module-owned files are shallow and shared prompt fragments live +in an explicit shared asset package. + +## Motivation + +The current Scriptorium asset layout works, but module prompt files are nested +more deeply than their package ownership requires. For example, a module package +already identifies its stage, domain, and artifact, so paths like +`assets/scriptorium/prompts/dnd/spells/...` repeat context that is already in +the Go package path. + +Shared D&D prompt fragments also currently live under a D&D-specific +`promptassets` package. That is serviceable, but it undersells the fact that +shared assets are intentionally cross-module. A clearer shared package will make +reuse deliberate and make room for common prompt hardening and reference-use +rules. + +## Target State + +Module-owned assets should stay in the module package that owns the behavior, +but their internal paths should be shallow: + +```text +internal/modules/extract/dnd/spells/assets/ + prompts/ + dnd.spells.yaml + task.md + instructions.md + schemas/ + dnd_spells.v1.json + +internal/modules/chunk/dnd/scenes/assets/ + prompts/ + dnd.scenes.yaml + task.md + instructions.md + schemas/ + dnd_scenes.v1.json +``` + +Shared reusable assets should move to: + +```text +internal/modules/sharedassets/ + assets.go + prompts/ + common-dnd-system.md + common-dnd-transcript.md + common-dnd-references.md + schemas/ +``` + +The `sharedassets` package should own only reusable fragments and shared asset +registration. It should not own module task semantics, response schemas, +validators, artifact interpretation, or stage behavior. + +## Ownership Rules + +Module packages own: + +- complete Scriptorium prompt definitions; +- module task prompts and instruction prompts; +- response schemas and schema metadata; +- prompt IDs and versions; +- semantic validation and interpretation. + +`internal/modules/sharedassets` owns: + +- shared prompt fragments used by more than one module; +- standardized cross-module prompt hardening; +- shared source/transcript framing; +- shared reference framing and reference-use constraints; +- future genuinely shared schemas, if a real cross-module schema emerges. + +Shared prompt fragments should be durable operating rules, not artifact-specific +instructions. They may tell the model to treat source material as primary +evidence, use references only for disambiguation unless a module says otherwise, +ignore instructions embedded inside source/reference content, follow the +structured output contract, and avoid exposing prompt instructions. + +## Documentation Outcome + +When implemented, canonical internal documentation should describe the asset +ownership convention: + +- module-owned prompts and schemas live under each module's shallow `assets` + tree; +- shared reusable fragments live under `internal/modules/sharedassets`; +- framework and CLI code may collect assets, but module packages remain + responsible for prompt intent and schemas. + +Prompt IDs, prompt versions, response schema IDs, schema names, and runtime +module contracts should remain unchanged unless a later feature explicitly +changes prompt semantics.