Add a feature roadmap and staged implementation plan to refactor shared D&D module assets

This commit is contained in:
2026-07-06 12:03:49 -05:00
parent 47cf7e76ec
commit 7861d040df
4 changed files with 445 additions and 74 deletions

View File

@@ -51,6 +51,23 @@ go test ./internal/modules/extract/dnd/spells
go test ./internal/modules/output/json
```
## Prompt Asset Tests
Tests should not assert the exact text of embedded prompt assets for production
modules. Prompt wording is expected to change frequently during development, and
tests should not fail solely because prompt prose was edited.
Prefer assertions against prompt structure, prompt IDs and versions, declared
inputs, schema wiring, input propagation, diagnostics redaction, and successful
prompt preparation.
Narrow exceptions:
- generic or test-only modules may use fixed prompt text assertions when the
text is part of the test surface;
- test fixtures may supply their own prompt text and assert against that fixture
text.
## Go Conventions
- Prefer the standard library unless a dependency is justified by correctness,

157
docs/roadmap/dnd.md Normal file
View File

@@ -0,0 +1,157 @@
# D&D Shared Module Helper Roadmap
This roadmap defines the target state for consolidating Dungeons & Dragons
helper code that is currently duplicated across the implemented D&D modules.
The refactor should preserve runtime behavior: module keys, prompt IDs, prompt
versions, Scriptorium-visible prompt files, message order, response schemas,
reference slot compatibility, manifests, diagnostics policy, and CLI/config
semantics should not change.
## Motivation
The `dnd/scenes` chunker and `dnd/spells` extractor now share several D&D prompt
and reference conventions:
- prompt inputs named `transcript`, `players`, `party`, and `glossary`;
- deprecated `roster` reference bindings mapped to the `party` prompt input;
- D&D reference media-type declarations;
- deterministic rendering of one or more reference files into prompt input
materials;
- shared D&D prompt fragments currently under `internal/modules/sharedassets`;
- prompt hash inclusion for shared D&D prompt fragments.
Keeping this logic duplicated makes future D&D modules more likely to drift. It
also makes small prompt-contract changes expensive because each module must
update the same reference-slot, prompt-input, and test helper behavior.
## Target Package Boundary
D&D-specific shared helpers should live under:
```text
internal/modules/sharedassets/dnd
```
The package name should be `dnd`. This keeps generic shared asset plumbing in
`internal/modules/sharedassets` while giving D&D-specific prompt and reference
policy a clear home.
`internal/modules/sharedassets` should continue to own:
- generic shared prompt asset registration;
- `ModulePromptFS` and related prompt filesystem layout helpers;
- non-domain-specific embedded asset plumbing;
- future prompt assets only when they are truly domain-neutral.
`internal/modules/sharedassets/dnd` should own D&D-specific reusable helpers:
- D&D shared prompt fragments;
- accepted reference media types for D&D reference slots;
- reference slot definitions for `players`, `party`, `glossary`, and the
deprecated `roster` alias;
- prompt input assembly for D&D modules that use the shared prompt fragments;
- deterministic reference material rendering;
- shared D&D prompt hash parts.
Concrete D&D modules should continue to own:
- stage contracts and module registration;
- module keys and provided/required capabilities;
- prompt IDs and prompt versions;
- module-local prompt definitions, task prompts, and instruction prompts;
- response schema definitions and schema metadata;
- artifact interpretation, scene interpretation, validators, and output
conversion.
Framework, core, and CLI packages must not import
`internal/modules/sharedassets/dnd`.
The intended asset layout is:
```text
internal/modules/sharedassets/
assets.go
prompt_fs.go
internal/modules/sharedassets/dnd/
assets.go
prompt_inputs.go
references.go
assets/
prompts/
common-dnd-system.md
common-dnd-transcript.md
common-dnd-references.md
```
The parent `sharedassets` package should have no hard-coded knowledge of
`common-dnd-*` files. It should provide generic composition primitives that a
domain package such as `sharedassets/dnd` can use.
## Desired End State
The `dnd/scenes` and `dnd/spells` modules should remain small owners of their
stage-specific behavior. Shared D&D helpers should remove duplicated mechanics
without hiding module semantics.
After the refactor:
- both modules should call one shared helper to build the D&D prompt input set
from source input and resolved references;
- both modules should use one shared source of truth for D&D reference slots and
accepted media types;
- D&D shared prompt files should live in `internal/modules/sharedassets/dnd`
beside the code that owns the D&D prompt contract;
- generic shared asset plumbing should compose caller-provided shared prompt
files without knowing which domain owns them;
- deprecated `roster` bindings should continue to work as an alias for `party`;
- reference rendering behavior should remain deterministic and covered by tests
in the shared D&D helper package;
- real prompt asset tests should remain decoupled from exact embedded prompt
prose;
- D&D module tests should focus on module contracts, request construction,
response interpretation, validators, diagnostics, and manifest metadata rather
than duplicate reference rendering details.
## Documentation Outcome
Current-behavior docs should describe the D&D shared helper package only where
it helps future maintainers understand ownership:
- [Internal Modules](../internal/modules.md) should mention that common D&D
prompt/reference helper behavior lives under
`internal/modules/sharedassets/dnd`.
- [Internal Overview](../internal/overview.md) may mention the package if it
lists shared module-support packages.
User-facing docs should not describe internal helper placement unless behavior
visible to users changes. This refactor should not change visible behavior.
## Resolved Decisions
### Should generic reference-slot cloning move to `contracts`?
Decision: move generic cloning to `internal/framework/contracts`, for example
as `CloneReferenceSlots`. The operation is not D&D-specific and exists because
`contracts.ReferenceSlot` contains mutable slices. Keeping the clone helper with
the contract type makes defensive copying easier to reuse in future modules
without introducing domain imports.
### Should shared D&D prompt hash parts move into `sharedassets/dnd`?
Decision: move D&D shared prompt hash helpers into
`internal/modules/sharedassets/dnd`. The hash parts are D&D-specific prompt
provenance, and placing them beside the D&D prompt contract reduces drift.
### Should D&D shared prompt assets move into `sharedassets/dnd`?
Decision: move D&D shared prompt assets into
`internal/modules/sharedassets/dnd/assets/prompts`. The current
`common-dnd-system.md`, `common-dnd-transcript.md`, and
`common-dnd-references.md` files are not truly generic; they are part of the
D&D prompt contract. Keeping the files beside D&D reference slots, prompt input
assembly, reference rendering, and prompt hash helpers gives the package clear
ownership of reusable D&D prompt behavior.
The parent `internal/modules/sharedassets` package may still contain prompt
assets in the future, but only when they are genuinely domain-neutral.

View File

@@ -1,19 +1,277 @@
# Completed Shared Asset Layout Follow-Up
# D&D Shared Helper Refactor Implementation Plan
The shared prompt and asset layout reorganization is complete.
## Summary
Implemented behavior is documented in canonical current-behavior docs:
Implement the target state defined in
[D&D Shared Module Helper Roadmap](dnd.md). This is an internal refactor: do not
change module keys, prompt IDs, prompt versions, Scriptorium-visible message
order, response schema IDs or names, CLI/config semantics, manifest shape,
diagnostics redaction policy, or D&D scene/spell interpretation behavior.
- [Internal Overview](../internal/overview.md)
- [Internal LLM Runtime](../internal/llm.md)
- [Internal Modules](../internal/modules.md)
The intended package split is:
## Implemented Outcomes
- `internal/modules/sharedassets`: generic prompt filesystem composition and
non-domain-specific asset plumbing only;
- `internal/modules/sharedassets/dnd`: reusable D&D prompt assets, prompt input
helpers, reference slot helpers, reference rendering, and D&D shared prompt
hash parts;
- concrete D&D modules: stage contracts, module registration, module-local
prompt definitions, response schemas, validators, and response
interpretation.
- module-owned prompts and schemas use shallow module-local asset trees;
- reusable D&D prompt fragments live under `internal/modules/sharedassets`;
- shared prompt filesystem composition is centralized in `sharedassets`;
- D&D scene and spell modules no longer carry duplicated prompt filesystem
helper implementations.
## Stage 1: Add Generic Defensive-Copy Helper
Remaining future work belongs in [Future Work](future.md).
- Add `CloneReferenceSlots` to `internal/framework/contracts`.
- The helper must:
- return `nil` for an empty input slice;
- allocate a new slot slice;
- deep-copy each `AcceptedMediaTypes` slice;
- preserve all other `ReferenceSlot` fields exactly.
- Add focused tests in `internal/framework/contracts` proving nil/empty
behavior, deep-copy behavior, and full field preservation.
- Do not import any concrete module package from `contracts`.
## Stage 2: Make Shared Prompt FS Composition Domain-Neutral
- Refactor `internal/modules/sharedassets.ModulePromptFS` so the parent package
no longer knows about D&D prompt filenames or embedded D&D assets.
- Add a generic shared prompt file descriptor:
```go
type SharedPromptFile struct {
Name string
FS fs.FS
Path string
}
```
- Change `ModulePromptFS` to this shape:
```go
func ModulePromptFS(
moduleDir string,
moduleFS fs.FS,
files []ModulePromptFile,
sharedFiles ...SharedPromptFile,
) (fs.FS, error)
```
- Keep existing `ModulePromptFile` semantics:
- `Name` is mounted directly under `assets/prompts/<moduleDir>/`;
- `Path` is read from the module embedded filesystem.
- Mount each `SharedPromptFile` under
`assets/prompts/<moduleDir>/sharedassets/<Name>`.
- Validate all names and paths with contextual errors:
- `moduleDir` must be a valid non-root fs path;
- module prompt `Name` must be a single filename with no path separators;
- shared prompt `Name` must be a single filename with no path separators;
- `moduleFS` and each shared file `FS` must be non-nil;
- missing module or shared files must return errors naming the source path.
- Preserve the in-memory FS behavior currently covered by tests, including
readable intermediate directories and root `.` support.
- Update `internal/modules/sharedassets` tests so they use test-owned shared
prompt files from `fstest.MapFS`, not production D&D prompt assets.
- Remove D&D-specific helpers, constants, and embedded asset dependencies from
the parent `sharedassets` package.
## Stage 3: Create `sharedassets/dnd`
- Create `internal/modules/sharedassets/dnd` with package name `dnd`.
- Move the current D&D shared prompt files into:
```text
internal/modules/sharedassets/dnd/assets/prompts/common-dnd-system.md
internal/modules/sharedassets/dnd/assets/prompts/common-dnd-transcript.md
internal/modules/sharedassets/dnd/assets/prompts/common-dnd-references.md
```
- Add `assets.go` in the D&D package with `go:embed assets/prompts/*.md`.
- Expose D&D shared asset helpers:
```go
func SharedPromptFiles() []sharedassets.SharedPromptFile
func CommonHashParts() []llm.AssetHashPart
func ReferenceHashParts() []llm.AssetHashPart
func ModulePromptFS(moduleDir string, moduleFS fs.FS, files []sharedassets.ModulePromptFile) (fs.FS, error)
```
- `SharedPromptFiles` must return a new slice each call.
- `CommonHashParts` must include `common-dnd-system.md` and
`common-dnd-transcript.md`.
- `ReferenceHashParts` must include `common-dnd-references.md`.
- `ModulePromptFS` must call `sharedassets.ModulePromptFS` with
`SharedPromptFiles()...`.
- Do not expose a D&D package registration function unless a concrete caller
needs top-level D&D shared prompt files outside module-local prompt
composition. The D&D prompt definitions should continue to load shared files
through `./sharedassets/...` inside each module prompt directory.
## Stage 4: Add D&D Reference And Prompt Input Helpers
- In `internal/modules/sharedassets/dnd`, add D&D reference helpers:
```go
type ReferenceSlotDescriptions struct {
Glossary string
Party string
Players string
Roster string
}
func ReferenceMediaTypes() []string
func ReferenceSlots(descriptions ReferenceSlotDescriptions) []contracts.ReferenceSlot
```
- `ReferenceMediaTypes` must return a defensive copy of:
- `application/json`
- `application/x-yaml`
- `application/yaml`
- `text/markdown`
- `text/plain`
- `ReferenceSlots` must return slots in the same order currently exposed by
both D&D modules: `glossary`, `party`, `players`, `roster`.
- Slot names and alias semantics must remain unchanged.
- Slot descriptions should be supplied by each concrete module through
`ReferenceSlotDescriptions` so current module metadata text can be preserved.
- `ReferenceSlots` must use `contracts.CloneReferenceSlots` or equivalent
defensive-copy behavior before returning.
- Add prompt input helpers:
```go
func PromptInputs(sourceInput contracts.LLMInputMaterial, references contracts.ReferenceSet) contracts.LLMInputSet
func TranscriptPromptMaterial(material contracts.LLMInputMaterial) contracts.LLMInputMaterial
func ReferencePromptMaterial(name string, slot contracts.ResolvedReferenceSlot) contracts.LLMInputMaterial
func ReferencePromptInput(slot contracts.ResolvedReferenceSlot) []byte
```
- `PromptInputs` must return inputs named `transcript`, `players`, `party`, and
`glossary`.
- If `party` has no items and `roster` has items, `PromptInputs` must use the
`roster` slot content for the `party` input.
- `PromptInputs` must not include a `roster` prompt input.
- `TranscriptPromptMaterial` must clone the source input and set `Name` to
`transcript`.
- `ReferencePromptMaterial` must use media type `text/plain`; for a single
reference item it must propagate that item's digest and origin URI.
- `ReferencePromptInput` behavior must match current D&D module behavior:
- empty slot renders as a single space;
- one item renders as raw item content;
- multiple items are copied, sorted deterministically by origin URI, digest,
then content, and rendered with the same heading/metadata format currently
used by `dnd/scenes` and `dnd/spells`.
- Add focused tests under `internal/modules/sharedassets/dnd` for all helper
behavior. These tests may assert rendering of helper-owned fixture inputs, but
must not assert exact production embedded prompt prose.
## Stage 5: Update D&D Modules To Use Shared Helpers
- Update `internal/modules/chunk/dnd/scenes`:
- import `internal/modules/sharedassets/dnd`;
- replace local accepted media type and slot cloning logic with
`dnd.ReferenceSlots`;
- preserve existing scene-specific slot descriptions;
- replace local prompt input/reference rendering helpers with
`dnd.PromptInputs`;
- replace `sharedassets.ModulePromptFS` with `dnd.ModulePromptFS`;
- replace `sharedassets.CommonHashParts` and `ReferenceHashParts` with the
D&D package equivalents.
- Update `internal/modules/extract/dnd/spells` the same way, preserving current
extractor-specific slot descriptions.
- Delete local duplicated helper functions that become unused:
- `acceptedReferenceMediaTypes`;
- `cloneReferenceSlots`;
- `transcriptPromptInput`;
- `referencePromptMaterial`;
- `referencePromptInput`.
- Keep module-local schema loading, prompt IDs, prompt versions, validators,
request validation, response conversion, and manifest metadata ownership in
each concrete module.
## Stage 6: Update Asset Registration And Tests
- Update production prompt asset registration in `internal/cli` as needed:
- remove any call that registers D&D shared prompt files from the generic
`sharedassets` package;
- ensure `dnd/scenes` and `dnd/spells` prompt registration still makes their
module-local `./sharedassets/common-dnd-*.md` files available.
- Update production prompt asset tests so they assert module-local shared asset
paths such as:
- `dnd.scenes/sharedassets/common-dnd-system.md`
- `dnd.spells/sharedassets/common-dnd-system.md`
- Remove expectations for top-level `common-dnd-*.md` prompt files unless a
genuine top-level registration remains necessary.
- Move reference rendering tests from concrete D&D modules into
`internal/modules/sharedassets/dnd`.
- Keep concrete D&D module tests focused on:
- module specs and registry behavior;
- LLM request prompt ID/version/profile/session fields;
- transcript and reference input propagation;
- Scriptorium prompt preparation;
- diagnostics redaction;
- manifest metadata;
- scene/spell response interpretation.
- Follow the prompt asset testing policy in `docs/policy/development.md`: do
not assert exact production embedded prompt prose.
## Stage 7: Update Documentation
- Update canonical internal docs for implemented behavior:
- `docs/internal/modules.md`: mention that common D&D prompt/reference helper
behavior lives under `internal/modules/sharedassets/dnd`;
- `docs/internal/overview.md`: mention the package only if the overview lists
shared module-support packages.
- Do not add user-facing documentation unless visible behavior changes.
- Update `docs/roadmap/dnd.md` after implementation so it no longer presents
this refactor as future work.
- Replace this file with a concise completed-note document once the work is
implemented.
## Validation
Run focused tests after each meaningful stage:
```sh
go test ./internal/framework/contracts
go test ./internal/modules/sharedassets
go test ./internal/modules/sharedassets/dnd
go test ./internal/modules/chunk/dnd/scenes
go test ./internal/modules/extract/dnd/spells
go test ./internal/cli
```
Run full validation before completion:
```sh
go test ./...
go vet ./...
go build ./cmd/notarius
```
Run inspection searches:
```sh
rg -n "common-dnd|CommonHashParts|ReferenceHashParts|referencePromptInput|cloneReferenceSlots|acceptedReferenceMediaTypes" internal/modules internal/framework
rg -n "Divide the provided transcript|Extract Dungeons & Dragons spell-cast artifacts|A transcript of a Dungeons & Dragons gameplay session|Roster reference|Glossary reference" internal/**/*_test.go
```
Expected inspection outcomes:
- D&D shared prompt files and hash helpers are owned by
`internal/modules/sharedassets/dnd`;
- parent `sharedassets` has no hard-coded `common-dnd-*` knowledge;
- concrete D&D modules no longer carry duplicated reference rendering logic;
- tests do not assert exact production embedded prompt prose.
## Non-Goals
- Do not change D&D prompt wording except for file moves required by this
refactor.
- Do not change prompt IDs, prompt versions, profile defaults, Scriptorium
message order, schema files, schema metadata, module keys, capabilities,
manifest shapes, or diagnostics policy.
- Do not move scene boundary logic, spell artifact logic, validators, or schema
loading into the shared D&D package.
- Do not make framework, core, or CLI packages depend on
`internal/modules/sharedassets/dnd`.

View File

@@ -1,61 +0,0 @@
# Completed Shared Prompt And Asset Layout
The embedded prompt and schema asset layout has been reorganized so module-owned
files are shallow and shared prompt fragments live in an explicit shared asset
package.
## Outcome
Module-owned assets stay in the module package that owns the behavior, with
shallow internal paths:
Shared reusable assets live under:
```text
internal/modules/sharedassets/
assets.go
prompts/
common-dnd-system.md
common-dnd-transcript.md
common-dnd-references.md
schemas/
```
The `sharedassets` package should own only reusable fragments and shared asset
registration. It should not own module task semantics, response schemas,
validators, artifact interpretation, or stage behavior.
## Ownership Rules
Module packages own:
- complete Scriptorium prompt definitions;
- module task prompts and instruction prompts;
- response schemas and schema metadata;
- prompt IDs and versions;
- semantic validation and interpretation.
`internal/modules/sharedassets` owns:
- shared prompt fragments used by more than one module;
- standardized cross-module prompt hardening;
- shared source/transcript framing;
- shared reference framing and reference-use constraints;
- future genuinely shared schemas, if a real cross-module schema emerges.
Shared prompt fragments should be durable operating rules, not artifact-specific
instructions. They may tell the model to treat source material as primary
evidence, use references only for disambiguation unless a module says otherwise,
ignore instructions embedded inside source/reference content, follow the
structured output contract, and avoid exposing prompt instructions.
## Canonical Docs
Current behavior is documented in:
- [Internal Overview](../internal/overview.md)
- [Internal LLM Runtime](../internal/llm.md)
- [Internal Modules](../internal/modules.md)
Prompt IDs, prompt versions, response schema IDs, schema names, and runtime
module contracts remain unchanged by this reorganization.