Plan the D&D prompt asset cleanup

This commit is contained in:
2026-08-05 14:28:05 +00:00
parent 406ad1d362
commit bad5db3ef3
3 changed files with 1123 additions and 0 deletions

View File

@@ -22,6 +22,48 @@ not as committed release dates.
spell, combat, interaction, and scene-description lanes after real-world use.
Add more complex chunking only in response to demonstrated failures.
### Canonical Entity Registries And Occurrences
- Establish a consistent module and artifact naming convention that makes the
two entity-extraction roles explicit. Prefer `dnd/<entity>-registry` for a
module that identifies canonical nouns and `dnd/<entity>-occurrences` for a
downstream module that consumes that registry and extracts transcript-backed
evidence about those nouns. Apply the convention coherently to NPCs,
locations, and items rather than preserving the current mixture of bare
plurals, `interactions`, `occurrences`, and `events` names.
- Treat this as a contract-level change rather than a prompt-asset rename. A
full feature roadmap should inventory and update module IDs, artifact kinds
and schema identities where applicable, pipeline dependencies, generated
references, checkpoints, configuration examples, tests, and documentation.
Because Notarius remains pre-release, prefer a clean migration over aliases
or compatibility shims unless the repository has acquired a concrete need
for them before implementation.
- Define the registry contract around stable, canonicalizable entity names.
In particular, tighten location extraction to include explicitly named
in-world places, such as `Waterdeep` or `The Yawning Portal`, while excluding
generic, temporary, relative, or purely descriptive references such as `the
room`, `the bar`, `the hallway`, or `upstairs`. Determine eligibility by
whether the transcript establishes a stable name or unique designation, not
by capitalization alone, because transcript capitalization is unreliable.
- Keep registry identification separate from occurrence evidence. A downstream
occurrence extractor may resolve a generic reference such as `the bar` to an
existing named registry entry when transcript context supports that
coreference, but it must not create a new canonical entity merely from the
generic reference. The occurrence itself must remain independently supported
by transcript ranges; registry membership is grounding, not event evidence.
- Define equivalent eligibility and coreference rules for NPC and item
registries so future entity types follow one architectural pattern while
retaining their domain-specific occurrence kinds. The full roadmap should
decide how unnamed but uniquely identifiable NPCs and items are represented,
and whether an item registry is introduced before or alongside renaming the
existing item-event lane.
- Evaluate the renamed and tightened contracts against representative
transcripts. Review false-positive registry entries, missed named entities,
coreference accuracy, unsupported occurrence links, and the behavior of
cost-effective smaller models. Keep these semantic changes separate from the
prompt-asset consolidation so regressions can be attributed to the correct
work set.
## Shared Normalization And Quality Work
### Generic LLM-Assisted Deduplication

View File

@@ -0,0 +1,618 @@
# D&D Prompt Asset Cleanup Implementation Plan
## Objective
Implement the target state defined in
[D&D Prompt Asset Cleanup](prompts.md): standardize D&D prompt asset names,
clarify transcript scopes, consolidate fixed module instructions, remove prompt
prose duplication, normalize the two outlying private response-schema
identities, and document the resulting authoring conventions.
This plan changes internal LLM-facing assets and their owning registration code.
It does not change durable artifact contracts, public prompt IDs, module IDs,
pipeline configuration, reference bindings, profile selection, or extraction
policy. In particular, the proposed registry/occurrence module renaming and the
proper-name-only location policy in [Future Work](future.md) are not part of
this implementation.
The stages are ordered dependencies. Complete and validate each stage before
starting the next. Each stage is sized for one gpt-5.6-terra implementation
prompt.
## Implementation Rules
Apply these rules throughout all stages:
- Read `docs/policy/architecture.md`, `docs/policy/documentation.md`, and
`docs/policy/testing.md` before making changes.
- Keep the root `assets` Go package content-only. Do not add prompt composition,
schema selection, registration, or D&D policy to that package.
- Preserve every public PromptKit prompt `id`, prompt version, default profile,
input name, input requirement, output format, validation mode, and repair
setting unless a stage below expressly changes private schema identity.
- Preserve the message orders and cache boundaries specified in `prompts.md`.
Consolidating fixed prose is not authorization to move variable material or
family-specific policy into the system message.
- Treat prompt prose as an owned contract, not as text to concatenate
mechanically. Merge `task.md` and `instructions.md` by editing the result into
one coherent `instructions.md`, removing duplicated transitions and rules.
- Begin with minimal, non-repetitive prose. Do not retain or reintroduce a
duplicated instruction merely because repetition might help a model. That
decision requires later observational evidence.
- Preserve all lane-specific domain semantics identified in `prompts.md`, even
when removing generic output, evidence, reference, or schema restatements.
- Keep all shared filenames prefixed with `common-dnd-`.
- Do not add compatibility aliases for retired private asset names or private
schema identities. Notarius is pre-release, and one-time prompt fingerprint,
checkpoint, and provider-cache invalidation is accepted.
- Keep default tests deterministic, offline, and credential-free. Do not call a
live LLM during implementation or make live model quality an automated gate.
- Do not add exact prompt snapshots, exact shared-prefix lengths, or tests that
enumerate every private asset merely to detect change. Retain tests of actual
preparation, input placement, cache boundaries, schema registration, and the
byte-identical extraction prefix.
## Stage 1: Establish Shared System And Transcript-Scope Assets
### Goal
Give universal instructions and each form of transcript material one correct
shared owner, then migrate every current D&D prompt to the appropriate scope in
one coherent change.
### Required changes
1. Revise `assets/dnd/shared/prompts/common-dnd-system.md` so it contains all
and only behavior applicable to scene chunking, every extraction lane, and
both entity normalizers:
- the D&D transcript-processing role;
- reliance only on supplied inputs;
- awareness of transcription errors, repetition, incomplete sentences, and
misheard proper nouns; and
- exactly one schema-conforming JSON object with no explanatory prose.
Refer to supplied inputs generally. Do not limit the statement to transcript
and reference inputs, because grounded extractors and normalizers receive
other material. Do not place rendered inputs, transcript-evidence mechanics,
identity policy, reference policy, registry policy, or reconciliation policy
in the system message.
2. Revise
`assets/dnd/shared/prompts/common-dnd-extraction-evidence.md` to own only
extraction evidence and source-range policy. Retain transcript-only evidence,
integer range endpoints, omission of `source_id`, and narrow non-contiguous
ranges. Remove the universal JSON-only/output-schema sentences now owned by
the system message. Avoid repeating detailed campaign-reference or registry
semantics owned by their input fragments.
3. Replace `assets/dnd/shared/prompts/common-dnd-transcript.md` with:
- `common-dnd-transcript-full.md`, which renders the complete ordered session
for scene chunking and does not imply that the complete session is one
scene;
- `common-dnd-transcript-chunk.md`, which renders one extraction chunk,
limits reporting and inference to that chunk, and states that unit IDs
retain source-wide meaning; and
- `common-dnd-transcript-windows.md`, which renders selected normalization
evidence windows and states that they may be incomplete, non-contiguous,
or overlapping and that absence outside them is not evidence.
Each file must render `{{ input "transcript" }}` exactly once. Scope files
describe material completeness and permissible use; do not move scene,
extraction-evidence, or reconciliation policy into them.
4. Update `internal/modules/dnd/shared/assets.go` so `sharedPromptPaths` exposes
the three new names and no longer exposes `common-dnd-transcript.md`.
5. Update every owning `promptAssetManifest` and current prompt YAML definition
to select exactly one transcript-scope asset:
| Scope asset | Prompt IDs |
| --- | --- |
| `common-dnd-transcript-full.md` | `dnd.scenes` |
| `common-dnd-transcript-chunk.md` | `dnd.spells`, `dnd.npcs`, `dnd.combat_turns`, `dnd.enemy_events`, `dnd.item_events`, `dnd.npc_interactions`, `dnd.scene_descriptions`, `dnd.locations`, `dnd.location_occurrences` |
| `common-dnd-transcript-windows.md` | `dnd.npcs.normalize`, `dnd.locations.normalize` |
Preserve the existing message position and cache control while changing the
referenced scope filename. Extraction prompts must continue to share the
same rendered prefix through the chunk transcript.
6. Update focused prompt-asset and shared-asset tests. A focused assertion that
an owning prompt family selects its required transcript scope is warranted;
do not replace it with exact prose snapshots or a full shared-file inventory.
Expand `internal/modules/dnd/register/prompt_cache_test.go` to include the
location and location-occurrence extraction prompts in its byte-identical
prefix check while retaining the existing other extraction lanes.
### Acceptance criteria
- The old `common-dnd-transcript.md` is absent and unreferenced.
- Every D&D prompt prepares with exactly the transcript semantics supplied by
its caller.
- The shared system message is byte-identical across all D&D prompts and
contains no variable input.
- The extraction-evidence fragment no longer repeats the universal output rule.
- Extraction prompts retain a byte-identical rendered prefix through campaign
references and the chunk transcript when given identical inputs.
- No public prompt, module, durable artifact, or pipeline contract changes.
### Validation
```sh
go test ./internal/modules/dnd/shared/... ./internal/modules/dnd/...
git diff --check
```
## Stage 2: Standardize Prompt And Context Filenames
### Goal
Make prompt definitions and variable context assets predictable without changing
their rendered content or message order.
### Required changes
1. Rename every prompt-ID-derived YAML file below to `prompt.yaml` and update its
owning `promptAssetManifest` entry to mount `prompt.yaml` from
`prompts/prompt.yaml`:
- `assets/dnd/scenes/prompts/`
- `assets/dnd/spells/prompts/`
- `assets/dnd/npcs/extract/prompts/`
- `assets/dnd/npcs/normalize/prompts/`
- `assets/dnd/combat-turns/prompts/`
- `assets/dnd/enemy-events/prompts/`
- `assets/dnd/item-events/prompts/`
- `assets/dnd/npc-interactions/prompts/`
- `assets/dnd/scene-descriptions/prompts/`
- `assets/dnd/locations/extract/prompts/`
- `assets/dnd/locations/normalize/prompts/`
- `assets/dnd/location-occurrences/prompts/`
Leave the YAML `id` fields unchanged. PromptKit continues to distinguish the
definitions through each manifest's existing `ModuleDir`.
2. Apply these context-fragment renames and update both YAML `content_file`
references and manifest `ModuleFiles` entries:
- `assets/dnd/spells/prompts/catalog.md` to `spell-catalog.md`;
- `assets/dnd/enemy-events/prompts/grounding.md` to
`combat-grounding.md`; and
- `assets/dnd/location-occurrences/prompts/locations.md` to
`location-registry.md`.
Do not edit their prompt prose in this mechanical stage.
3. Update tests that intentionally exercise the production asset layout:
- In `internal/modules/dnd/register/register_test.go`, retain a registrar-
boundary check for one canonical `prompt.yaml` per maintained prompt, but
remove lists of every task, instruction, context, and shared fragment.
- In `internal/cli/production_contract_test.go`, remove private prompt-file
inventory assertions already owned by the D&D registrar and module prompt-
preparation tests. Retain its CLI composition and public catalog contract
checks.
- Update module-local tests only where a renamed file is intentionally part
of the manifest/preparation behavior.
- Do not rewrite generic `internal/framework/llm` or
`internal/framework/promptfs` fixtures merely because they use arbitrary
noncanonical filenames; those tests prove generic filesystem behavior, not
the D&D authoring convention.
4. Search for and remove all production references to the retired filenames.
Do not preserve copies or aliases.
### Acceptance criteria
- Every D&D prompt subtree contains `prompt.yaml` and no prompt-ID-derived YAML
definition.
- The three renamed context files have descriptive names and render in their
original message positions.
- Every module manifest mounts the canonical names and still produces a prompt
fingerprint.
- Production-composition tests protect registration behavior without
enumerating every private content file.
- All prompts prepare through PromptKit with unchanged IDs, versions, inputs,
profiles, output modes, and cache controls.
### Validation
```sh
go test ./internal/modules/dnd/... ./internal/cli/...
git diff --check
```
## Stage 3: Consolidate Scene And Entity-Normalization Instructions
### Goal
Adopt the single `instructions.md` convention for scene chunking, scene
description, NPC normalization, and location normalization while preserving
their distinct message-order contracts.
### Required changes
1. For scene chunking, merge `task.md` and `instructions.md` into one coherent
`instructions.md` and delete `task.md`. Retain:
- the definition of a coherent scene and meaningful boundary conditions;
- reasons to split and reasons not to split;
- complete ordered transcript coverage with no gaps or overlaps; and
- inclusive, positive source-unit endpoints.
Remove the generic JSON-only rule now owned by the system prompt and field-
exclusivity prose already enforced by the private response schema. In
`prompt.yaml`, place the combined instructions after campaign references and
before the full transcript. It has no cache control; references and the final
full transcript retain ephemeral cache control.
2. For scene descriptions, merge `task.md` and `instructions.md` into one
`instructions.md` and delete `task.md`. Retain the one-accepted-chunk scope,
all `kind` definitions and precedence, and title and summary semantics.
Remove schema field-list exclusions and universal JSON-only language. Keep
the combined final instructions message ephemeral.
3. For both NPC and location normalizers, rename `task.md` to
`instructions.md`, update `prompt.yaml` and the manifest, and edit the prose
against `common-dnd-entity-reconciliation.md`:
- NPC instructions own same-individual alias judgment and canonical display-
name preferences.
- Location instructions own same-place identity judgment, separation of
nested or uncertain places, and canonical display-name selection.
- The shared reconciliation fragment alone owns opaque keys, valid group
membership, canonical-member constraints, invention prohibitions, and
omission of uncertain groups.
Preserve the normalizer order: system, module instructions, cacheable shared
reconciliation policy, candidates, cacheable transcript windows.
4. Update module-local prompt preparation tests to assert meaningful relative
ordering, input isolation, and cache boundaries. Remove exact message-count
assertions whose only purpose was to preserve the former task/instruction
split.
### Acceptance criteria
- These four prompt directories each contain one module-owned
`instructions.md` and no `task.md`.
- Scene and normalizer domain policy remains complete but does not repeat
universal system, response-schema, or shared reconciliation rules.
- Scene chunking still receives the full transcript last; normalizers still
receive candidates before transcript windows.
- Prompt preparation and existing scene/normalization behavior tests pass.
### Validation
```sh
go test ./internal/modules/dnd/chunk/scenes ./internal/modules/dnd/extract/scenedescriptions ./internal/modules/dnd/normalize/npcs ./internal/modules/dnd/normalize/locations
git diff --check
```
## Stage 4: Consolidate NPC, Location, And Item-Event Instructions
### Goal
Consolidate three comparatively self-contained extraction prompts and remove
their shared or schema-enforced repetition without changing lane semantics.
### Required changes
1. Merge each prompt's `task.md` and `instructions.md` into its existing
`instructions.md`, delete `task.md`, and update `prompt.yaml` and the owning
manifest for:
- NPC extraction;
- location extraction; and
- item-event extraction.
2. NPC instructions must retain individually identifiable non-PC inclusion,
proper-name/stable-title/alias eligibility, all existing exclusion
categories, observed display spelling, and the prohibition on invented
anonymous labels. Remove generic narrow-range, reference-not-evidence,
schema-field, and lore-output restatements already owned elsewhere.
3. Location instructions must retain the current implemented location policy,
including the current allowance for a generic label used for a specific
place, supported place categories, alias/nesting handling, and domain
exclusions. Do **not** implement the future proper-name-only registry policy
in this stage. Remove only generic evidence, reference, source-range, and
schema-shape repetition.
4. Item-event instructions must retain event eligibility, all five kind
definitions, currency treatment, quantity semantics, holder transitions,
collective `party` behavior, transfer restrictions, and the distinction
between discovery and acquisition. Remove only generic evidence-range,
output-envelope, and schema-field restatements.
5. Keep each consolidated extraction instruction as the final ephemeral
message. Update focused tests to protect preparation, inputs, output schema,
and cache placement without asserting the retired two-message structure.
### Acceptance criteria
- The three prompt directories contain no `task.md`.
- No domain inclusion, exclusion, event-kind, currency, holder, or location
behavior changes.
- Generic evidence and response-shape rules have one shared/schema owner.
- All three prompts prepare and their extractor tests remain green.
### Validation
```sh
go test ./internal/modules/dnd/extract/npcs ./internal/modules/dnd/extract/locations ./internal/modules/dnd/extract/itemevents
git diff --check
```
## Stage 5: Consolidate Spell And Combat-Turn Instructions
### Goal
Consolidate the spell and combat-turn prompts while keeping their identity,
catalog, grounding, and event semantics in the correct assets.
### Required changes
1. Merge `task.md` and `instructions.md` into `instructions.md`, delete
`task.md`, and update `prompt.yaml` and manifests for spells and combat turns.
2. Spell module instructions own:
- inclusion of actual casts and unambiguous declared casting attempts;
- exclusion of mentions, plans, rules discussion, and unsupported catalog
matches;
- in-world caster attribution, including player-character and GM-narrated
casts; and
- the lane-specific requirement that evidence collectively support caster,
spell, and casting event.
`spell-catalog.md` alone owns the rendered catalog, exact canonical spelling,
and the fact that the catalog is recognition context rather than evidence.
Shared identity and evidence fragments own their generic rules.
3. Combat-turn instructions own combat-event eligibility and exclusions,
chronology, unmatched actors, turn-level evidence grouping, and the
`turn_kind` meanings. Remove repeated player/NPC identity grounding,
transcript-only evidence, source-range mechanics, and schema-field lists
owned by shared assets or the response schema.
4. Keep both consolidated instruction messages final and ephemeral. Preserve
the spell catalog and optional NPC grounding as separate variable-context
messages after evidence policy and before module instructions.
### Acceptance criteria
- Both prompt directories contain no `task.md`.
- Spell recognition remains catalog-constrained and caster attribution remains
evidence-grounded.
- Combat-turn categories, event boundaries, actor policy, and evidence grouping
are unchanged.
- Neither prompt repeats generic shared identity, evidence, or output-format
policy.
- Prompt preparation, spell schema/catalog tests, combat tests, and the shared
extraction-prefix test pass.
### Validation
```sh
go test ./internal/modules/dnd/extract/spells ./internal/modules/dnd/extract/combatturns ./internal/modules/dnd/register
git diff --check
```
## Stage 6: Consolidate Grounded Occurrence And Enemy-Event Instructions
### Goal
Finish module prompt consolidation for the three registry- or artifact-grounded
event extractors.
### Required changes
1. Merge `task.md` and `instructions.md` into `instructions.md`, delete
`task.md`, and update `prompt.yaml` and manifests for:
- NPC interactions;
- location occurrences; and
- enemy events.
2. NPC-interaction instructions own registry membership as an eligibility
constraint, occurrence boundaries, all interaction kinds and precedence,
combat-alignment splitting, and domain exclusions. The shared NPC fragment
owns registry rendering, canonical grounding, and its non-evidence status;
shared identity and evidence fragments own their generic rules.
3. Location-occurrence instructions own occurrence eligibility, all occurrence
kinds and precedence, handling of speculation, independent overlapping
facts, and domain exclusions. `location-registry.md` owns exact ID/name
rendering and registry-not-evidence semantics. Remove repeated schema fields,
`source_id`, narrow-range, and registry-context language from module prose.
4. Enemy-event instructions own event eligibility, the five event kinds and
outcome definitions, one-engagement-per-scene policy, exclusions, and naming
of registered or grounded unnamed opponents. `combat-grounding.md` and the
shared NPC fragment own rendering and the non-evidence status of auxiliary
inputs. The shared evidence fragment owns source-range mechanics. Remove the
repeated response-field list and generic evidence statements.
5. Keep each consolidated instruction message final and ephemeral, with all
registry and grounding messages before it and after the chunk transcript.
### Acceptance criteria
- No D&D prompt directory contains `task.md`.
- All occurrence kinds, precedence rules, enemy outcomes, splitting rules, and
registry constraints remain intact.
- Registry and auxiliary artifacts remain grounding only and never become
transcript evidence.
- All three prompts prepare, and focused extractor, registry, invariant, and
cache-prefix tests pass.
### Validation
```sh
go test ./internal/modules/dnd/extract/npcinteractions ./internal/modules/dnd/extract/locationoccurrences ./internal/modules/dnd/extract/enemyevents ./internal/modules/dnd/register
git diff --check
```
## Stage 7: Normalize Private Scene And Spell Schema Identities
### Goal
Bring the two outlying private LLM response schemas into the naming convention
already used by the other D&D prompt schemas, without touching durable artifact
schemas.
### Required changes
1. Normalize the scene private response schema:
- rename `assets/dnd/scenes/schemas/dnd_scenes.v1.json` to
`dnd_scenes_llm.v1.json`;
- change its `$id` from `notarius.dnd.scenes` to
`notarius.dnd.scenes.llm`;
- change the scene `prompt.yaml` `schema_path` accordingly; and
- update `internal/modules/dnd/chunk/scenes/schema.go` to use key
`dnd_scenes_llm`, ID `notarius.dnd.scenes.llm`, name
`notarius_dnd_scenes_llm_v1`, and the renamed asset path.
2. Normalize the already correctly named spell private response schema:
- keep the filename `dnd_spells_llm.v1.json`;
- change its `$id` from `notarius.dnd.spells` to
`notarius.dnd.spells.llm`; and
- update `internal/modules/dnd/extract/spells/schema.go` to use key
`dnd_spells_llm`, ID `notarius.dnd.spells.llm`, and name
`notarius_dnd_spells_llm_v1`.
3. Keep both versions at `v1`; these are private pre-release response contracts,
not a compatibility migration. Do not alter required fields, property types,
enum values, `additionalProperties`, or any owning Go response type.
4. Update focused schema-loading, prompt-preparation, registrar, and production
composition expectations. Carefully distinguish the private spell response
schema from the durable spell artifact schema under
`internal/modules/dnd/codec/spells/assets/schemas/`; the durable schema and
`docs/integrations/dnd-spell-artifacts.md` must not change.
5. Remove every reference to the retired private scene filename and old private
IDs/names/keys. Do not add aliases.
### Acceptance criteria
- Scene and spell private schemas use `_llm`/`.llm` consistently in filenames
where applicable, registry keys, schema IDs, and response-schema names.
- Both schemas still load and validate the same private Go response shapes.
- Durable scene chunk-map and spell artifact contracts are unchanged.
- Registrar and module prompt preparation tests pass without schema collisions.
### Validation
```sh
go test ./internal/modules/dnd/chunk/scenes ./internal/modules/dnd/extract/spells ./internal/modules/dnd/register ./internal/cli
git diff --check
```
## Stage 8: Document Conventions And Perform Final Verification
### Goal
Make the implemented conventions discoverable in their canonical documentation,
remove stale references, and verify the complete repository.
### Required changes
1. Rewrite the prompt-construction portion of `docs/internal/dnd.md` to describe
current implemented behavior:
- the canonical `prompt.yaml` and `instructions.md` local names;
- module manifests as the owner of selection, order, and fingerprinting;
- the `common-dnd-` visual provenance prefix;
- the single-owner hierarchy for universal, family, input, module, and schema
rules;
- the eligibility test for system-prompt content;
- the full transcript, chunk transcript, and transcript-window assets and
their consumers;
- the final extraction, scene-chunking, and entity-normalization message
orders and cache controls;
- the rule against default prompt repetition and the observational-evidence
threshold for reintroducing it; and
- the content-only `assets` boundary, with a link to ADR-0011 rather than a
duplicate rationale.
Remove the current descriptions of separate task and instructions messages
and the generic transcript fragment.
2. Do not add an assets-package README or a new ADR. Do not modify ADR-0011.
`docs/internal/llm.md` should remain provider-neutral and should not duplicate
D&D filenames or ordering. Update it only if an earlier stage changed a
provider-neutral PromptKit preparation or cache-control fact; no such change
is expected by this plan.
3. Review all changed prompt definitions and manifests as a set. Confirm that:
- every prompt has one system message using the shared system asset;
- every prompt selects exactly one correct transcript-scope asset;
- every extraction prompt retains the stable shared prefix and final
ephemeral module instructions;
- scene chunking and normalizers retain their intentional distinct orders;
- no module-specific prompt repeats a rule already owned by its selected
shared assets or response schema; and
- the root `assets` Go package remains a single minimal filesystem file with
no business logic or internal/PromptKit dependency.
4. Search for stale production references to:
- `common-dnd-transcript.md`;
- prompt-ID-derived YAML filenames;
- D&D `task.md` prompt files;
- `catalog.md`, `grounding.md`, and the location-occurrence `locations.md`;
- `dnd_scenes.v1.json` and the old scene private schema identity; and
- the old spell private schema key, ID, and response-schema name.
Exclude historical roadmap text only when the reference intentionally
explains a retired name. There should be no stale reference in current code,
assets, tests, or implemented documentation.
5. Review tests for policy compliance. Retain behavior-level assertions for
preparation, schema compatibility, input isolation, cache placement, and the
identical extraction prefix. Remove any leftover assertion whose only value
is freezing exact prompt prose, message count, shared-prefix length, or a
complete private asset inventory.
6. Run formatting where applicable and the complete offline repository checks.
Do not perform live model evaluation. Report that human evaluation with
representative transcripts and cost-effective target models remains the
next observational step after implementation; do not reintroduce prompt
repetition before that evidence exists.
### Acceptance criteria
- `docs/internal/dnd.md` accurately describes the implemented prompt layout,
ownership, transcript scopes, and cache ordering without duplicating external
contracts.
- No new ADR or assets README exists, and ADR-0011 remains unchanged.
- No stale production references or retired files remain.
- All maintained prompts prepare offline, all schemas register, all repository
tests and static checks pass, and the CLI builds.
- The final diff contains no durable artifact, public module/configuration, or
pipeline behavior change.
### Validation
```sh
go test ./...
go vet ./...
go build ./cmd/notarius
git diff --check
```
Run `gofmt` on the Go files changed by these stages before the commands above.
Inspect the diff and ensure formatting did not rewrite unrelated user changes;
never format an entire directory when the worktree contains unrelated edits.
## Open Questions
None. The feature roadmap and this plan make the prompt ownership, transcript
scope, naming, schema identity, documentation, compatibility, and testing
decisions required for implementation.

463
docs/roadmap/prompts.md Normal file
View File

@@ -0,0 +1,463 @@
# D&D Prompt Asset Cleanup
## Purpose
This roadmap defines the target state for the embedded D&D prompt and private
schema assets. The work is intended to make prompts easier to inspect, compare,
and revise; eliminate unnecessary repetition; preserve useful prompt-cache
boundaries; and establish conventions that future D&D modules can follow.
The central design principle is that each instruction should have one clear
owner. Shared behavior belongs in a shared prompt asset, module-specific
behavior belongs in the module's instruction asset, variable grounding belongs
in a dedicated input asset, and purely structural requirements belong in the
JSON schema.
## Current State
The root `assets` package correctly centralizes embedded prompt and schema
content without owning D&D business logic. The D&D prompt assets nevertheless
retain several conventions inherited from their former module-local layouts:
- prompt definitions use prompt-ID-derived filenames rather than a consistent
local filename;
- most prompt directories divide fixed module instructions between `task.md`
and `instructions.md`, although that division generally does not represent a
meaningful cache, trust, or input boundary;
- module-specific instructions repeat some rules already supplied by shared D&D
messages or enforced by their response schemas;
- one generic transcript fragment currently describes full-session input,
chunk-scoped input, and selected normalization windows even though those
materials have different completeness and inference boundaries;
- variable context assets use several naming styles, including generic names
such as `catalog.md` and `grounding.md`;
- the private scene and spell response-schema identifiers do not follow the
`_llm`/`.llm` convention used by the other private D&D schemas.
These are internal authoring and maintenance inconsistencies. The durable
artifact contracts and the separation between content ownership in `assets`
and behavior ownership in `internal/modules` remain sound.
## Goals
The completed cleanup should:
- give every D&D prompt directory the same predictable entry-point names;
- consolidate fixed module-specific instructions into one message unless a
distinct message boundary has a concrete purpose;
- remove module-specific restatements of rules already owned by shared assets;
- retain separate messages for shared policy, stable references, variable
transcript content, and other inputs where the boundary communicates trust,
provenance, reuse, or cache behavior;
- distinguish full-session transcripts, extraction chunks, and normalization
evidence windows through separate shared assets with explicit scope rules;
- keep all shared D&D filenames visibly identifiable in prompt definitions;
- make variable context filenames self-describing;
- normalize private LLM response-schema identities without changing durable
output schemas;
- document the conventions future prompt authors must follow; and
- preserve the rule that the root `assets` Go package contains no business
logic.
## Decisions
### Canonical prompt directory layout
Every D&D prompt directory should use the following local layout:
```text
prompts/
prompt.yaml
instructions.md
[optional variable or context fragments]
```
The prompt definition filename is always `prompt.yaml`. The prompt's `id` field,
not its filename, remains the canonical global identifier. This is safe because
each module registers a distinct asset subtree with PromptKit; the local
`prompt.yaml` filenames do not share a virtual directory.
Use the `.yaml` extension consistently. It matches the existing embedded prompt
and profile assets.
For prompts that currently have both `task.md` and `instructions.md`, combine
their non-duplicative content into `instructions.md`. For prompts that currently
have only `task.md`, rename that asset to `instructions.md`. Do not preserve an
otherwise artificial two-message split merely to distinguish a short task
statement from its rules.
### Retain the `common-dnd-` prefix
Shared D&D assets must retain the `common-dnd-` filename prefix, including:
- `common-dnd-system.md`
- `common-dnd-identity.md`
- `common-dnd-references.md`
- `common-dnd-transcript-full.md`
- `common-dnd-transcript-chunk.md`
- `common-dnd-transcript-windows.md`
- `common-dnd-extraction-evidence.md`
- `common-dnd-npcs.md`
- `common-dnd-entity-reconciliation.md`
The prefix is a deliberate visual provenance marker. When reviewing a prompt
definition, an author should be able to distinguish shared messages from
module-owned messages immediately. Because asset references are explicit and
machine-resolved, retaining the prefix has no meaningful mechanical cost.
### One owner per instruction
Prompt prose should follow this ownership hierarchy:
1. Universal D&D prompting behavior belongs in a shared D&D asset.
2. Rules shared by a prompt family belong in the narrowest applicable shared
asset.
3. Semantics of a variable input belong in the fragment that renders that
input.
4. Lane-specific extraction, classification, or normalization policy belongs in
that lane's `instructions.md`.
5. Object shape, required fields, enum values, and rejection of unknown fields
belong in the JSON schema unless prose is needed to explain their domain
meaning.
Module-specific prompt assets should omit instructions already provided by the
shared prompt set. In particular, they should not routinely repeat generic rules
about transcript-only evidence, references not being evidence, omission of
`source_id`, narrow transcript ranges, schema-only output, or explanatory text.
This cleanup should begin with the minimal instruction set that is both correct
and non-repetitive. Repetition may be reintroduced only when evaluation produces
repeatable observational evidence that a target model needs reinforcement. Any
such repetition should be narrow, placed close to the rule's canonical owner,
and documented as a deliberate model-reliability tradeoff rather than allowed to
become accidental drift.
This rule does not authorize removal of domain semantics. Similar wording is not
duplication when it defines a distinct lane contract, such as event precedence,
holder transitions, scene-kind definitions, identity inclusion criteria, or the
relationship between a combat actor and an extracted event.
### System-prompt eligibility
`common-dnd-system.md` should own every fixed instruction that genuinely applies
to every maintained D&D LLM prompt. Consolidating those universal rules into the
system message gives them one authoritative owner, applies the appropriate
instruction priority, and preserves one byte-identical initial message across
chunking, extraction, and normalization prompts.
An instruction belongs in the shared system message only when it is:
- applicable to scene chunking, every extraction lane, and both entity
normalizers;
- independent of prompt-specific or variable inputs;
- static across calls; and
- appropriate to treat as a high-priority behavioral constraint.
The system message should therefore own the D&D transcript-processing role,
reliance only on supplied inputs, treatment of transcription uncertainty, and
the requirement to return exactly one schema-conforming JSON object without
explanatory prose.
Rules shared by only a subset of prompts must remain in the narrowest applicable
shared user-message fragment. Transcript-evidence and citation rules, identity
rules, campaign-reference semantics, registry grounding, and reconciliation
policy are not universal system instructions. No rendered transcript,
reference, registry, catalog, candidate, or other variable material belongs in
the system message.
### Shared prompt ownership
The shared D&D files should have clear, non-overlapping responsibilities:
| Asset | Canonical responsibility |
| --- | --- |
| `common-dnd-system.md` | D&D domain role, reliance on supplied inputs, treatment of transcription uncertainty, and the universal requirement to emit exactly one schema-conforming JSON object without commentary. |
| `common-dnd-identity.md` | Rules for resolving supported in-world identities and the distinction between disambiguation and participation evidence. |
| `common-dnd-references.md` | Rendering and interpretation of campaign references, including that references provide context rather than establishing transcript events. |
| `common-dnd-transcript-full.md` | Rendering the complete ordered session transcript and identifying it as complete session input. |
| `common-dnd-transcript-chunk.md` | Rendering one extraction chunk, limiting the task to that chunk, and explaining that its unit IDs retain their source-wide meaning. |
| `common-dnd-transcript-windows.md` | Rendering selected reconciliation evidence windows and explaining that they may be incomplete, non-contiguous, or overlapping. |
| `common-dnd-extraction-evidence.md` | Transcript-evidence policy, source-range construction, omission of `source_id`, and preference for narrow supporting ranges. |
| `common-dnd-npcs.md` | Rendering and interpreting an upstream NPC registry as grounding rather than evidence. |
| `common-dnd-entity-reconciliation.md` | Safe duplicate-group proposal mechanics shared by entity normalizers. |
As part of normalization, universal output-format prose should move to
`common-dnd-system.md`, while evidence-specific prose should remain in
`common-dnd-extraction-evidence.md`. The system message should refer generally
to supplied inputs rather than only transcripts and references, because
normalizers and grounded extractors receive additional input types.
### Transcript-scope ownership
The current `common-dnd-transcript.md` should be replaced rather than retained
as an alias. All three replacements may render the same prompt input named
`transcript`, but each must accurately describe the material supplied by its
caller:
- `common-dnd-transcript-full.md` is used by scene chunking. It identifies the
material as the complete ordered session and permits whole-session coverage
decisions without implying that the entire session is one scene.
- `common-dnd-transcript-chunk.md` is used by every extraction lane. It states
that only the current chunk is in scope, that source-wide unit IDs remain
authoritative, and that the model must not infer or report events outside the
supplied chunk.
- `common-dnd-transcript-windows.md` is used by NPC and location normalization.
It states that the material consists of selected evidence windows that may be
incomplete, non-contiguous, or overlapping. The model may use those windows
to evaluate candidate identity but must not treat absence outside them as
evidence.
The scope fragments should describe input completeness and permissible use;
they should not absorb scene policy, extraction evidence mechanics, or entity
reconciliation rules owned by other assets. Future prompt families must select
the fragment matching the material actually passed by code rather than choosing
one based merely on an input variable named `transcript`.
### Module-specific ownership
After shared duplication is removed, each module's `instructions.md` should
retain only its substantive lane policy:
| Prompt | Module-owned policy |
| --- | --- |
| Scene chunking | Scene-boundary rules, ordered coverage, and valid treatment of gaps or overlaps. |
| Scene descriptions | `kind` classification definitions and precedence, plus title and summary semantics. |
| NPC extraction | Inclusion and exclusion rules for individually identifiable NPCs. |
| NPC interactions | Interaction kinds, precedence, occurrence splitting, and NPC-specific event semantics. |
| Location extraction | Inclusion and exclusion rules for identifiable in-world places. |
| Location occurrences | Occurrence kinds, precedence, splitting, and location-specific event semantics. |
| Spell extraction | What constitutes a cast, catalog-name policy, and caster attribution. |
| Combat turns | Combat event boundaries, event kinds, actor semantics, and combat-specific inclusion rules. |
| Item events | Event kinds, possession transitions, holder semantics, quantities, and currency treatment. |
| Enemy events | Enemy-event kinds, outcome semantics, encounter identity, and combat-grounding rules. |
| NPC normalization | NPC-specific duplicate criteria and canonical-name selection. |
| Location normalization | Location-specific duplicate criteria and canonical-name selection. |
Schema field lists should not be copied into prose solely to restate the schema.
Prose may still explain how a field is interpreted or how two fields relate when
the schema cannot express that semantic rule.
### Message boundaries and cache ordering
Combining `task.md` and `instructions.md` must not lead to indiscriminate message
consolidation. Separate messages remain appropriate when they preserve a shared
byte-identical prefix, distinguish fixed instructions from variable material,
or identify the provenance and trust level of an input.
Extraction prompts should retain this general order:
```text
system
identity
campaign references
chunk transcript
evidence policy
optional grounding or variable context
module instructions
```
The scene chunker should retain this general order:
```text
system
campaign references
module instructions
full transcript
```
Entity normalizers should retain this general order:
```text
system
module instructions
shared entity-reconciliation policy
candidates
transcript windows
```
Existing cache controls should remain attached to the same semantic boundaries
unless PromptKit's contract requires a mechanical adjustment. The cleanup must
not move variable transcript or candidate content into an earlier stable prefix
merely to reduce message count.
### Variable context asset names
Optional non-shared fragments should describe the input they render rather than
use generic names. Normalize the current assets as follows where applicable:
- `catalog.md` to `spell-catalog.md`;
- `grounding.md` to `combat-grounding.md`;
- `locations.md` to `location-registry.md`.
The name `candidates.md` may remain in normalizer-local directories because its
meaning is clear in that context. Future context assets should use similarly
specific nouns and should remain separate from fixed `instructions.md` content.
### Private response-schema identity
Private schemas used only to constrain LLM responses should consistently use
the `_llm` filename/key/name convention and the `.llm` schema-ID suffix already
used by most D&D modules.
Normalize the scene schema to:
- file: `dnd_scenes_llm.v1.json`;
- schema ID: `notarius.dnd.scenes.llm`;
- registry key: `dnd_scenes_llm`;
- response-schema name: `notarius_dnd_scenes_llm_v1`.
Normalize the spell schema metadata to:
- file: `dnd_spells_llm.v1.json` (already named correctly);
- schema ID: `notarius.dnd.spells.llm`;
- registry key: `dnd_spells_llm`;
- response-schema name: `notarius_dnd_spells_llm_v1`.
Private schema filenames must remain globally unique. They should not be renamed
to a generic local name because the schema registry exposes a shared namespace.
Durable artifact schema identifiers and shapes are out of scope and must not
change.
### Explicit prompt definitions remain explicit
The repeated `version`, `default_profile`, input, message, output, and schema
declarations in `prompt.yaml` files should remain explicit. They are declarative
configuration, not prompt-prose duplication. Introducing a generator, template
layer, or Go builder for these small definitions would make inspection harder
and create another synchronization boundary.
Similarly, module manifests should continue to declare the assets their prompt
uses. Shared asset selection is module behavior and does not belong in the root
`assets` package.
## Compatibility and Cache Effects
This is an intentional internal prompt revision. Renaming prompt definition
files, consolidating messages, editing shared prose, and normalizing private
schema identifiers will change prompt and component fingerprints. Existing
checkpoints may no longer be reusable, and backend prompt caches will experience
a one-time cold start. Those fingerprint, checkpoint, and cache effects are
accepted consequences of reaching the target prompt layout.
The following contracts must remain stable:
- public prompt IDs;
- module IDs and configuration fields;
- default profile selection;
- required and optional prompt inputs;
- durable artifact schema IDs and artifact shapes;
- extraction and normalization semantics, except for removing unintended prompt
repetition; and
- pipeline dependency behavior.
No backward-compatible copies of old private prompt assets or schema aliases are
required while the application remains pre-release.
## Documentation and Maintenance Rules
The implemented convention should be documented in the canonical internal D&D
and LLM documentation rather than in an assets-package README that could drift.
The documentation should explain:
- the canonical `prompt.yaml` and `instructions.md` names;
- why shared assets retain the `common-dnd-` prefix;
- the instruction-ownership hierarchy;
- the eligibility test for moving a rule into `common-dnd-system.md`;
- the full, chunk, and evidence-window transcript contracts and which prompt
families select each one;
- when a separate message boundary is warranted;
- the cache-oriented message ordering;
- the prohibition on business logic in `assets`; and
- the evidence threshold for deliberately repeating an instruction.
The Go file in the root `assets` package should remain a minimal embedding and
filesystem-access layer. Prompt composition, schema selection, module policy,
and validation logic must remain in their existing owning packages.
`docs/internal/dnd.md` should be the canonical owner of these D&D prompt
authoring conventions and the concrete message-order patterns. Its existing
prompt-construction section should be revised when the assets change rather
than creating an `assets` README or duplicating the rules in contributor
orientation. `docs/internal/llm.md` should receive only any short update needed
to describe provider-neutral PromptKit preparation or cache-control mechanics;
it should link to the D&D guide instead of repeating domain-specific asset names
or ordering.
No new ADR is warranted for this cleanup. ADR-0011 already records the
architecturally significant decisions: physical centralization, content-only
package ownership, and retention of prompt semantics and ordering by modules.
The filename, instruction-ownership, transcript-scope, and system-message rules
are internal authoring conventions that should be allowed to evolve with model
evaluation. They do not create a new dependency boundary or reverse an accepted
architectural decision. If a later change promotes these conventions into a
cross-domain runtime-enforced prompt composition contract, that broader
decision should receive its own ADR at that time; ADR-0011 must not be edited to
retrofit the new rationale.
## Validation and Evaluation Policy
Automated validation should establish that the refactor preserves executable
contracts without freezing prompt prose. It should cover:
- preparation of every embedded prompt through PromptKit;
- prompt input declarations and required-input behavior;
- private response-schema registration and compatibility with the owning Go
response types;
- expected cache controls and meaningful stable-prefix boundaries;
- selection of the transcript-scope fragment appropriate to each prompt
family, verified through prompt preparation and owning-module behavior rather
than exact prose snapshots;
- manifest references after asset renames;
- absence of references to retired filenames and private schema identifiers;
and
- the full repository test suite.
Tests must not assert exact prompt wording, exact message counts, shared-prefix
lengths, or asset inventories solely as change detectors. Such tests would make
intentional prompt iteration unnecessarily expensive and conflict with the
project's testing policy.
Model quality should be evaluated by running representative transcripts through
all affected lanes, with particular attention to cost-effective smaller models.
Review should compare extraction correctness, unsupported claims, missing
events, source-range quality, and schema reliability. The initial evaluation
should use the minimal non-repetitive prompts. A repeated instruction should be
restored only if failures recur and there is a credible causal reason that
reinforcement will help.
## Non-Goals
This roadmap does not include:
- changes to durable D&D artifact schemas;
- new extraction lanes or pipeline behavior;
- changes to profile selection or model settings;
- a prompt-definition generator or templating framework;
- moving business logic into `assets`;
- collapsing all prompt material into one message;
- placing family-specific rules or rendered variable content in the system
message;
- preserving old prompt fingerprints or warm backend cache entries; or
- introducing brittle prompt snapshots or change-detector tests.
## Target End State
Every D&D prompt is discoverable through a local `prompt.yaml`, has one fixed
module-owned `instructions.md`, and uses separately named context fragments only
where inputs or cache boundaries justify them. Shared D&D assets remain visibly
marked with `common-dnd-` and have documented, non-overlapping responsibilities.
Module instructions contain no routine repetition of shared or schema-enforced
rules, while retaining all lane-specific domain semantics.
The shared system message contains all and only universal, static D&D behavior.
Scene chunking renders the explicitly full transcript fragment, extraction
lanes render the explicitly chunk-scoped fragment, and entity normalizers render
the explicitly incomplete evidence-window fragment. No prompt presents one
scope as another merely because all three inputs use the name `transcript`.
Private scene and spell response schemas follow the established LLM-only naming
convention, durable contracts are unchanged, all manifests and PromptKit
definitions prepare successfully, and human evaluation begins from a minimal
non-repetitive prompt set. Future modules can follow the same conventions
without copying an existing extractor and inheriting accidental prose or naming
drift.