From 671ff6d13215f3211f822a69d4471f234ad28086 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 6 Jul 2026 17:26:38 +0000 Subject: [PATCH] Document completed DnD shared helper refactor --- docs/internal/modules.md | 20 +-- docs/internal/overview.md | 10 +- docs/roadmap/dnd.md | 171 +++----------------- docs/roadmap/implementation.md | 286 ++------------------------------- 4 files changed, 50 insertions(+), 437 deletions(-) diff --git a/docs/internal/modules.md b/docs/internal/modules.md index b3202b0..d28b26c 100644 --- a/docs/internal/modules.md +++ b/docs/internal/modules.md @@ -42,11 +42,13 @@ rendered reference bytes. LLM-backed modules own Scriptorium prompt definitions and response schemas in their embedded assets. Module-owned prompts live under each module's shallow -`assets/prompts` tree and schemas live under `assets/schemas`. Shared reusable -prompt fragments live under `internal/modules/sharedassets`. Module contracts -should expose prompt IDs, versions, input material names, and non-secret -prompt/schema hashes through manifest metadata; they should not expose -Scriptorium public types through chunk, extract, or normalize contracts. +`assets/prompts` tree and schemas live under `assets/schemas`. Generic shared +prompt filesystem composition lives under `internal/modules/sharedassets`. +Common D&D prompt fragments, reference slot helpers, prompt input assembly, and +reference rendering live under `internal/modules/sharedassets/dnd`. Module +contracts should expose prompt IDs, versions, input material names, and +non-secret prompt/schema hashes through manifest metadata; they should not +expose Scriptorium public types through chunk, extract, or normalize contracts. Chunk modules receive the structured LLM client, configured Scriptorium profile ID, prompt session ID, and raw source input material through @@ -107,8 +109,8 @@ session ID to the runtime; validates model-authored source-unit boundaries; and converts each scene into a deterministic source chunk. Its prompt definition lives under `assets/prompts` and its schema under -`assets/schemas`. Shared reusable prompt fragments are provided by -`internal/modules/sharedassets` and referenced from prompt definitions under +`assets/schemas`. Shared reusable D&D prompt fragments are provided by +`internal/modules/sharedassets/dnd` and referenced from prompt definitions under `./sharedassets/`. Requires: @@ -146,8 +148,8 @@ spell-cast responses into artifact candidates; and supplies deterministic validators. Its prompt definition lives under `assets/prompts` and its schema under -`assets/schemas`. Shared reusable prompt fragments are provided by -`internal/modules/sharedassets` and referenced from prompt definitions under +`assets/schemas`. Shared reusable D&D prompt fragments are provided by +`internal/modules/sharedassets/dnd` and referenced from prompt definitions under `./sharedassets/`. Requires: diff --git a/docs/internal/overview.md b/docs/internal/overview.md index fdf2371..a86fd37 100644 --- a/docs/internal/overview.md +++ b/docs/internal/overview.md @@ -65,10 +65,12 @@ Each module package owns its contract implementation, module spec, registration, options, focused tests, and module-specific errors. Module-owned prompts and schemas live in each module's shallow `assets/prompts` -and `assets/schemas` directories. Shared reusable D&D prompt fragments live in -`internal/modules/sharedassets`. +and `assets/schemas` directories. Generic shared prompt filesystem composition +lives in `internal/modules/sharedassets`; shared D&D prompt fragments and +reference helpers live in `internal/modules/sharedassets/dnd`. Shared asset package: `internal/modules/sharedassets` +Shared D&D helper package: `internal/modules/sharedassets/dnd` ## Fixtures And Tests @@ -88,7 +90,9 @@ servers. - Source-format details stay in input modules and integration docs. - Extraction-domain details stay in extract modules and artifact docs. -- Shared prompt fragments stay in `internal/modules/sharedassets`. +- Generic shared prompt plumbing stays in `internal/modules/sharedassets`; + domain-specific shared prompt behavior stays with the relevant module helper + package. - Provider wire details stay in the LLM runtime and provider integration docs. - Durable output contracts belong in integration docs. - Operator procedures belong in `docs/operations.md`, not internal docs. diff --git a/docs/roadmap/dnd.md b/docs/roadmap/dnd.md index 5557fb4..785d88e 100644 --- a/docs/roadmap/dnd.md +++ b/docs/roadmap/dnd.md @@ -1,157 +1,22 @@ -# D&D Shared Module Helper Roadmap +# D&D Shared Module Helper Refactor -This roadmap defines the target state for consolidating Dungeons & Dragons -helper code that is currently duplicated across the implemented D&D modules. -The refactor should preserve runtime behavior: module keys, prompt IDs, prompt -versions, Scriptorium-visible prompt files, message order, response schemas, -reference slot compatibility, manifests, diagnostics policy, and CLI/config -semantics should not change. +The D&D shared module helper refactor is complete. -## Motivation +The implemented package split is: -The `dnd/scenes` chunker and `dnd/spells` extractor now share several D&D prompt -and reference conventions: +- `internal/modules/sharedassets`: generic shared prompt filesystem composition + and domain-neutral shared asset plumbing. +- `internal/modules/sharedassets/dnd`: D&D shared prompt fragments, accepted + reference media types, `players`/`party`/`glossary` reference slot helpers, + deprecated `roster` alias handling, prompt input assembly, deterministic + reference rendering, and D&D shared prompt hash parts. +- Concrete D&D modules: stage contracts and registration, module keys, + provided/required capabilities, prompt IDs and versions, module-local prompt + definitions, response schemas, validators, artifact or scene interpretation, + and manifest metadata. -- prompt inputs named `transcript`, `players`, `party`, and `glossary`; -- deprecated `roster` reference bindings mapped to the `party` prompt input; -- D&D reference media-type declarations; -- deterministic rendering of one or more reference files into prompt input - materials; -- shared D&D prompt fragments currently under `internal/modules/sharedassets`; -- prompt hash inclusion for shared D&D prompt fragments. - -Keeping this logic duplicated makes future D&D modules more likely to drift. It -also makes small prompt-contract changes expensive because each module must -update the same reference-slot, prompt-input, and test helper behavior. - -## Target Package Boundary - -D&D-specific shared helpers should live under: - -```text -internal/modules/sharedassets/dnd -``` - -The package name should be `dnd`. This keeps generic shared asset plumbing in -`internal/modules/sharedassets` while giving D&D-specific prompt and reference -policy a clear home. - -`internal/modules/sharedassets` should continue to own: - -- generic shared prompt asset registration; -- `ModulePromptFS` and related prompt filesystem layout helpers; -- non-domain-specific embedded asset plumbing; -- future prompt assets only when they are truly domain-neutral. - -`internal/modules/sharedassets/dnd` should own D&D-specific reusable helpers: - -- D&D shared prompt fragments; -- accepted reference media types for D&D reference slots; -- reference slot definitions for `players`, `party`, `glossary`, and the - deprecated `roster` alias; -- prompt input assembly for D&D modules that use the shared prompt fragments; -- deterministic reference material rendering; -- shared D&D prompt hash parts. - -Concrete D&D modules should continue to own: - -- stage contracts and module registration; -- module keys and provided/required capabilities; -- prompt IDs and prompt versions; -- module-local prompt definitions, task prompts, and instruction prompts; -- response schema definitions and schema metadata; -- artifact interpretation, scene interpretation, validators, and output - conversion. - -Framework, core, and CLI packages must not import -`internal/modules/sharedassets/dnd`. - -The intended asset layout is: - -```text -internal/modules/sharedassets/ - assets.go - prompt_fs.go - -internal/modules/sharedassets/dnd/ - assets.go - prompt_inputs.go - references.go - assets/ - prompts/ - common-dnd-system.md - common-dnd-transcript.md - common-dnd-references.md -``` - -The parent `sharedassets` package should have no hard-coded knowledge of -`common-dnd-*` files. It should provide generic composition primitives that a -domain package such as `sharedassets/dnd` can use. - -## Desired End State - -The `dnd/scenes` and `dnd/spells` modules should remain small owners of their -stage-specific behavior. Shared D&D helpers should remove duplicated mechanics -without hiding module semantics. - -After the refactor: - -- both modules should call one shared helper to build the D&D prompt input set - from source input and resolved references; -- both modules should use one shared source of truth for D&D reference slots and - accepted media types; -- D&D shared prompt files should live in `internal/modules/sharedassets/dnd` - beside the code that owns the D&D prompt contract; -- generic shared asset plumbing should compose caller-provided shared prompt - files without knowing which domain owns them; -- deprecated `roster` bindings should continue to work as an alias for `party`; -- reference rendering behavior should remain deterministic and covered by tests - in the shared D&D helper package; -- real prompt asset tests should remain decoupled from exact embedded prompt - prose; -- D&D module tests should focus on module contracts, request construction, - response interpretation, validators, diagnostics, and manifest metadata rather - than duplicate reference rendering details. - -## Documentation Outcome - -Current-behavior docs should describe the D&D shared helper package only where -it helps future maintainers understand ownership: - -- [Internal Modules](../internal/modules.md) should mention that common D&D - prompt/reference helper behavior lives under - `internal/modules/sharedassets/dnd`. -- [Internal Overview](../internal/overview.md) may mention the package if it - lists shared module-support packages. - -User-facing docs should not describe internal helper placement unless behavior -visible to users changes. This refactor should not change visible behavior. - -## Resolved Decisions - -### Should generic reference-slot cloning move to `contracts`? - -Decision: move generic cloning to `internal/framework/contracts`, for example -as `CloneReferenceSlots`. The operation is not D&D-specific and exists because -`contracts.ReferenceSlot` contains mutable slices. Keeping the clone helper with -the contract type makes defensive copying easier to reuse in future modules -without introducing domain imports. - -### Should shared D&D prompt hash parts move into `sharedassets/dnd`? - -Decision: move D&D shared prompt hash helpers into -`internal/modules/sharedassets/dnd`. The hash parts are D&D-specific prompt -provenance, and placing them beside the D&D prompt contract reduces drift. - -### Should D&D shared prompt assets move into `sharedassets/dnd`? - -Decision: move D&D shared prompt assets into -`internal/modules/sharedassets/dnd/assets/prompts`. The current -`common-dnd-system.md`, `common-dnd-transcript.md`, and -`common-dnd-references.md` files are not truly generic; they are part of the -D&D prompt contract. Keeping the files beside D&D reference slots, prompt input -assembly, reference rendering, and prompt hash helpers gives the package clear -ownership of reusable D&D prompt behavior. - -The parent `internal/modules/sharedassets` package may still contain prompt -assets in the future, but only when they are genuinely domain-neutral. +The `dnd/scenes` chunker and `dnd/spells` extractor now use the shared D&D +helper package for common prompt and reference behavior while keeping +stage-specific semantics in their concrete module packages. D&D shared prompt +fragments are exposed to Scriptorium through each module's local +`./sharedassets/` prompt paths. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 6c7b083..7ccf168 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,277 +1,19 @@ -# D&D Shared Helper Refactor Implementation Plan +# D&D Shared Helper Refactor Completed -## Summary +The D&D shared helper refactor is implemented. -Implement the target state defined in -[D&D Shared Module Helper Roadmap](dnd.md). This is an internal refactor: do not -change module keys, prompt IDs, prompt versions, Scriptorium-visible message -order, response schema IDs or names, CLI/config semantics, manifest shape, -diagnostics redaction policy, or D&D scene/spell interpretation behavior. - -The intended package split is: +Implemented package ownership: - `internal/modules/sharedassets`: generic prompt filesystem composition and - non-domain-specific asset plumbing only; -- `internal/modules/sharedassets/dnd`: reusable D&D prompt assets, prompt input - helpers, reference slot helpers, reference rendering, and D&D shared prompt - hash parts; -- concrete D&D modules: stage contracts, module registration, module-local - prompt definitions, response schemas, validators, and response - interpretation. + domain-neutral shared asset plumbing. +- `internal/modules/sharedassets/dnd`: reusable D&D prompt fragments, prompt + filesystem composition for D&D modules, D&D reference slot helpers, prompt + input assembly, reference rendering, and D&D shared prompt hash parts. +- Concrete D&D modules: stage contracts, module registration, module-local + prompt definitions, response schemas, validators, request validation, + response conversion, and manifest metadata ownership. -## Stage 1: Add Generic Defensive-Copy Helper - -- Add `CloneReferenceSlots` to `internal/framework/contracts`. -- The helper must: - - return `nil` for an empty input slice; - - allocate a new slot slice; - - deep-copy each `AcceptedMediaTypes` slice; - - preserve all other `ReferenceSlot` fields exactly. -- Add focused tests in `internal/framework/contracts` proving nil/empty - behavior, deep-copy behavior, and full field preservation. -- Do not import any concrete module package from `contracts`. - -## Stage 2: Make Shared Prompt FS Composition Domain-Neutral - -- Refactor `internal/modules/sharedassets.ModulePromptFS` so the parent package - no longer knows about D&D prompt filenames or embedded D&D assets. -- Add a generic shared prompt file descriptor: - -```go -type SharedPromptFile struct { - Name string - FS fs.FS - Path string -} -``` - -- Change `ModulePromptFS` to this shape: - -```go -func ModulePromptFS( - moduleDir string, - moduleFS fs.FS, - files []ModulePromptFile, - sharedFiles ...SharedPromptFile, -) (fs.FS, error) -``` - -- Keep existing `ModulePromptFile` semantics: - - `Name` is mounted directly under `assets/prompts//`; - - `Path` is read from the module embedded filesystem. -- Mount each `SharedPromptFile` under - `assets/prompts//sharedassets/`. -- Validate all names and paths with contextual errors: - - `moduleDir` must be a valid non-root fs path; - - module prompt `Name` must be a single filename with no path separators; - - shared prompt `Name` must be a single filename with no path separators; - - `moduleFS` and each shared file `FS` must be non-nil; - - missing module or shared files must return errors naming the source path. -- Preserve the in-memory FS behavior currently covered by tests, including - readable intermediate directories and root `.` support. -- Update `internal/modules/sharedassets` tests so they use test-owned shared - prompt files from `fstest.MapFS`, not production D&D prompt assets. -- Remove D&D-specific helpers, constants, and embedded asset dependencies from - the parent `sharedassets` package. - -## Stage 3: Create `sharedassets/dnd` - -- Create `internal/modules/sharedassets/dnd` with package name `dnd`. -- Move the current D&D shared prompt files into: - -```text -internal/modules/sharedassets/dnd/assets/prompts/common-dnd-system.md -internal/modules/sharedassets/dnd/assets/prompts/common-dnd-transcript.md -internal/modules/sharedassets/dnd/assets/prompts/common-dnd-references.md -``` - -- Add `assets.go` in the D&D package with `go:embed assets/prompts/*.md`. -- Expose D&D shared asset helpers: - -```go -func SharedPromptFiles() []sharedassets.SharedPromptFile -func CommonHashParts() []llm.AssetHashPart -func ReferenceHashParts() []llm.AssetHashPart -func ModulePromptFS(moduleDir string, moduleFS fs.FS, files []sharedassets.ModulePromptFile) (fs.FS, error) -``` - -- `SharedPromptFiles` must return a new slice each call. -- `CommonHashParts` must include `common-dnd-system.md` and - `common-dnd-transcript.md`. -- `ReferenceHashParts` must include `common-dnd-references.md`. -- `ModulePromptFS` must call `sharedassets.ModulePromptFS` with - `SharedPromptFiles()...`. -- Do not expose a D&D package registration function unless a concrete caller - needs top-level D&D shared prompt files outside module-local prompt - composition. The D&D prompt definitions should continue to load shared files - through `./sharedassets/...` inside each module prompt directory. - -## Stage 4: Add D&D Reference And Prompt Input Helpers - -- In `internal/modules/sharedassets/dnd`, add D&D reference helpers: - -```go -type ReferenceSlotDescriptions struct { - Glossary string - Party string - Players string - Roster string -} - -func ReferenceMediaTypes() []string -func ReferenceSlots(descriptions ReferenceSlotDescriptions) []contracts.ReferenceSlot -``` - -- `ReferenceMediaTypes` must return a defensive copy of: - - `application/json` - - `application/x-yaml` - - `application/yaml` - - `text/markdown` - - `text/plain` -- `ReferenceSlots` must return slots in the same order currently exposed by - both D&D modules: `glossary`, `party`, `players`, `roster`. -- Slot names and alias semantics must remain unchanged. -- Slot descriptions should be supplied by each concrete module through - `ReferenceSlotDescriptions` so current module metadata text can be preserved. -- `ReferenceSlots` must use `contracts.CloneReferenceSlots` or equivalent - defensive-copy behavior before returning. - -- Add prompt input helpers: - -```go -func PromptInputs(sourceInput contracts.LLMInputMaterial, references contracts.ReferenceSet) contracts.LLMInputSet -func TranscriptPromptMaterial(material contracts.LLMInputMaterial) contracts.LLMInputMaterial -func ReferencePromptMaterial(name string, slot contracts.ResolvedReferenceSlot) contracts.LLMInputMaterial -func ReferencePromptInput(slot contracts.ResolvedReferenceSlot) []byte -``` - -- `PromptInputs` must return inputs named `transcript`, `players`, `party`, and - `glossary`. -- If `party` has no items and `roster` has items, `PromptInputs` must use the - `roster` slot content for the `party` input. -- `PromptInputs` must not include a `roster` prompt input. -- `TranscriptPromptMaterial` must clone the source input and set `Name` to - `transcript`. -- `ReferencePromptMaterial` must use media type `text/plain`; for a single - reference item it must propagate that item's digest and origin URI. -- `ReferencePromptInput` behavior must match current D&D module behavior: - - empty slot renders as a single space; - - one item renders as raw item content; - - multiple items are copied, sorted deterministically by origin URI, digest, - then content, and rendered with the same heading/metadata format currently - used by `dnd/scenes` and `dnd/spells`. - -- Add focused tests under `internal/modules/sharedassets/dnd` for all helper - behavior. These tests may assert rendering of helper-owned fixture inputs, but - must not assert exact production embedded prompt prose. - -## Stage 5: Update D&D Modules To Use Shared Helpers - -- Update `internal/modules/chunk/dnd/scenes`: - - import `internal/modules/sharedassets/dnd`; - - replace local accepted media type and slot cloning logic with - `dnd.ReferenceSlots`; - - preserve existing scene-specific slot descriptions; - - replace local prompt input/reference rendering helpers with - `dnd.PromptInputs`; - - replace `sharedassets.ModulePromptFS` with `dnd.ModulePromptFS`; - - replace `sharedassets.CommonHashParts` and `ReferenceHashParts` with the - D&D package equivalents. -- Update `internal/modules/extract/dnd/spells` the same way, preserving current - extractor-specific slot descriptions. -- Delete local duplicated helper functions that become unused: - - `acceptedReferenceMediaTypes`; - - `cloneReferenceSlots`; - - `transcriptPromptInput`; - - `referencePromptMaterial`; - - `referencePromptInput`. -- Keep module-local schema loading, prompt IDs, prompt versions, validators, - request validation, response conversion, and manifest metadata ownership in - each concrete module. - -## Stage 6: Update Asset Registration And Tests - -- Update production prompt asset registration in `internal/cli` as needed: - - remove any call that registers D&D shared prompt files from the generic - `sharedassets` package; - - ensure `dnd/scenes` and `dnd/spells` prompt registration still makes their - module-local `./sharedassets/common-dnd-*.md` files available. -- Update production prompt asset tests so they assert module-local shared asset - paths such as: - - `dnd.scenes/sharedassets/common-dnd-system.md` - - `dnd.spells/sharedassets/common-dnd-system.md` -- Remove expectations for top-level `common-dnd-*.md` prompt files unless a - genuine top-level registration remains necessary. -- Move reference rendering tests from concrete D&D modules into - `internal/modules/sharedassets/dnd`. -- Keep concrete D&D module tests focused on: - - module specs and registry behavior; - - LLM request prompt ID/version/profile/session fields; - - transcript and reference input propagation; - - Scriptorium prompt preparation; - - diagnostics redaction; - - manifest metadata; - - scene/spell response interpretation. -- Follow the prompt asset testing policy in `docs/policy/development.md`: do - not assert exact production embedded prompt prose. - -## Stage 7: Update Documentation - -- Update canonical internal docs for implemented behavior: - - `docs/internal/modules.md`: mention that common D&D prompt/reference helper - behavior lives under `internal/modules/sharedassets/dnd`; - - `docs/internal/overview.md`: mention the package only if the overview lists - shared module-support packages. -- Do not add user-facing documentation unless visible behavior changes. -- Update `docs/roadmap/dnd.md` after implementation so it no longer presents - this refactor as future work. -- Replace this file with a concise completed-note document once the work is - implemented. - -## Validation - -Run focused tests after each meaningful stage: - -```sh -go test ./internal/framework/contracts -go test ./internal/modules/sharedassets -go test ./internal/modules/sharedassets/dnd -go test ./internal/modules/chunk/dnd/scenes -go test ./internal/modules/extract/dnd/spells -go test ./internal/cli -``` - -Run full validation before completion: - -```sh -go test ./... -go vet ./... -go build ./cmd/notarius -``` - -Run inspection searches: - -```sh -rg -n "common-dnd|CommonHashParts|ReferenceHashParts|referencePromptInput|cloneReferenceSlots|acceptedReferenceMediaTypes" internal/modules internal/framework -rg -n "Divide the provided transcript|Extract Dungeons & Dragons spell-cast artifacts|A transcript of a Dungeons & Dragons gameplay session|Roster reference|Glossary reference" internal/**/*_test.go -``` - -Expected inspection outcomes: - -- D&D shared prompt files and hash helpers are owned by - `internal/modules/sharedassets/dnd`; -- parent `sharedassets` has no hard-coded `common-dnd-*` knowledge; -- concrete D&D modules no longer carry duplicated reference rendering logic; -- tests do not assert exact production embedded prompt prose. - -## Non-Goals - -- Do not change D&D prompt wording except for file moves required by this - refactor. -- Do not change prompt IDs, prompt versions, profile defaults, Scriptorium - message order, schema files, schema metadata, module keys, capabilities, - manifest shapes, or diagnostics policy. -- Do not move scene boundary logic, spell artifact logic, validators, or schema - loading into the shared D&D package. -- Do not make framework, core, or CLI packages depend on - `internal/modules/sharedassets/dnd`. +The refactor preserved module keys, prompt IDs, prompt versions, +Scriptorium-visible message order, response schema IDs and names, CLI/config +semantics, manifest shape, diagnostics redaction policy, and D&D scene/spell +interpretation behavior.