Plan the D&D prompt asset cleanup
This commit is contained in:
@@ -22,6 +22,48 @@ not as committed release dates.
|
||||
spell, combat, interaction, and scene-description lanes after real-world use.
|
||||
Add more complex chunking only in response to demonstrated failures.
|
||||
|
||||
### Canonical Entity Registries And Occurrences
|
||||
|
||||
- Establish a consistent module and artifact naming convention that makes the
|
||||
two entity-extraction roles explicit. Prefer `dnd/<entity>-registry` for a
|
||||
module that identifies canonical nouns and `dnd/<entity>-occurrences` for a
|
||||
downstream module that consumes that registry and extracts transcript-backed
|
||||
evidence about those nouns. Apply the convention coherently to NPCs,
|
||||
locations, and items rather than preserving the current mixture of bare
|
||||
plurals, `interactions`, `occurrences`, and `events` names.
|
||||
- Treat this as a contract-level change rather than a prompt-asset rename. A
|
||||
full feature roadmap should inventory and update module IDs, artifact kinds
|
||||
and schema identities where applicable, pipeline dependencies, generated
|
||||
references, checkpoints, configuration examples, tests, and documentation.
|
||||
Because Notarius remains pre-release, prefer a clean migration over aliases
|
||||
or compatibility shims unless the repository has acquired a concrete need
|
||||
for them before implementation.
|
||||
- Define the registry contract around stable, canonicalizable entity names.
|
||||
In particular, tighten location extraction to include explicitly named
|
||||
in-world places, such as `Waterdeep` or `The Yawning Portal`, while excluding
|
||||
generic, temporary, relative, or purely descriptive references such as `the
|
||||
room`, `the bar`, `the hallway`, or `upstairs`. Determine eligibility by
|
||||
whether the transcript establishes a stable name or unique designation, not
|
||||
by capitalization alone, because transcript capitalization is unreliable.
|
||||
- Keep registry identification separate from occurrence evidence. A downstream
|
||||
occurrence extractor may resolve a generic reference such as `the bar` to an
|
||||
existing named registry entry when transcript context supports that
|
||||
coreference, but it must not create a new canonical entity merely from the
|
||||
generic reference. The occurrence itself must remain independently supported
|
||||
by transcript ranges; registry membership is grounding, not event evidence.
|
||||
- Define equivalent eligibility and coreference rules for NPC and item
|
||||
registries so future entity types follow one architectural pattern while
|
||||
retaining their domain-specific occurrence kinds. The full roadmap should
|
||||
decide how unnamed but uniquely identifiable NPCs and items are represented,
|
||||
and whether an item registry is introduced before or alongside renaming the
|
||||
existing item-event lane.
|
||||
- Evaluate the renamed and tightened contracts against representative
|
||||
transcripts. Review false-positive registry entries, missed named entities,
|
||||
coreference accuracy, unsupported occurrence links, and the behavior of
|
||||
cost-effective smaller models. Keep these semantic changes separate from the
|
||||
prompt-asset consolidation so regressions can be attributed to the correct
|
||||
work set.
|
||||
|
||||
## Shared Normalization And Quality Work
|
||||
|
||||
### Generic LLM-Assisted Deduplication
|
||||
|
||||
618
docs/roadmap/implementation.md
Normal file
618
docs/roadmap/implementation.md
Normal file
@@ -0,0 +1,618 @@
|
||||
# D&D Prompt Asset Cleanup Implementation Plan
|
||||
|
||||
## Objective
|
||||
|
||||
Implement the target state defined in
|
||||
[D&D Prompt Asset Cleanup](prompts.md): standardize D&D prompt asset names,
|
||||
clarify transcript scopes, consolidate fixed module instructions, remove prompt
|
||||
prose duplication, normalize the two outlying private response-schema
|
||||
identities, and document the resulting authoring conventions.
|
||||
|
||||
This plan changes internal LLM-facing assets and their owning registration code.
|
||||
It does not change durable artifact contracts, public prompt IDs, module IDs,
|
||||
pipeline configuration, reference bindings, profile selection, or extraction
|
||||
policy. In particular, the proposed registry/occurrence module renaming and the
|
||||
proper-name-only location policy in [Future Work](future.md) are not part of
|
||||
this implementation.
|
||||
|
||||
The stages are ordered dependencies. Complete and validate each stage before
|
||||
starting the next. Each stage is sized for one gpt-5.6-terra implementation
|
||||
prompt.
|
||||
|
||||
## Implementation Rules
|
||||
|
||||
Apply these rules throughout all stages:
|
||||
|
||||
- Read `docs/policy/architecture.md`, `docs/policy/documentation.md`, and
|
||||
`docs/policy/testing.md` before making changes.
|
||||
- Keep the root `assets` Go package content-only. Do not add prompt composition,
|
||||
schema selection, registration, or D&D policy to that package.
|
||||
- Preserve every public PromptKit prompt `id`, prompt version, default profile,
|
||||
input name, input requirement, output format, validation mode, and repair
|
||||
setting unless a stage below expressly changes private schema identity.
|
||||
- Preserve the message orders and cache boundaries specified in `prompts.md`.
|
||||
Consolidating fixed prose is not authorization to move variable material or
|
||||
family-specific policy into the system message.
|
||||
- Treat prompt prose as an owned contract, not as text to concatenate
|
||||
mechanically. Merge `task.md` and `instructions.md` by editing the result into
|
||||
one coherent `instructions.md`, removing duplicated transitions and rules.
|
||||
- Begin with minimal, non-repetitive prose. Do not retain or reintroduce a
|
||||
duplicated instruction merely because repetition might help a model. That
|
||||
decision requires later observational evidence.
|
||||
- Preserve all lane-specific domain semantics identified in `prompts.md`, even
|
||||
when removing generic output, evidence, reference, or schema restatements.
|
||||
- Keep all shared filenames prefixed with `common-dnd-`.
|
||||
- Do not add compatibility aliases for retired private asset names or private
|
||||
schema identities. Notarius is pre-release, and one-time prompt fingerprint,
|
||||
checkpoint, and provider-cache invalidation is accepted.
|
||||
- Keep default tests deterministic, offline, and credential-free. Do not call a
|
||||
live LLM during implementation or make live model quality an automated gate.
|
||||
- Do not add exact prompt snapshots, exact shared-prefix lengths, or tests that
|
||||
enumerate every private asset merely to detect change. Retain tests of actual
|
||||
preparation, input placement, cache boundaries, schema registration, and the
|
||||
byte-identical extraction prefix.
|
||||
|
||||
## Stage 1: Establish Shared System And Transcript-Scope Assets
|
||||
|
||||
### Goal
|
||||
|
||||
Give universal instructions and each form of transcript material one correct
|
||||
shared owner, then migrate every current D&D prompt to the appropriate scope in
|
||||
one coherent change.
|
||||
|
||||
### Required changes
|
||||
|
||||
1. Revise `assets/dnd/shared/prompts/common-dnd-system.md` so it contains all
|
||||
and only behavior applicable to scene chunking, every extraction lane, and
|
||||
both entity normalizers:
|
||||
|
||||
- the D&D transcript-processing role;
|
||||
- reliance only on supplied inputs;
|
||||
- awareness of transcription errors, repetition, incomplete sentences, and
|
||||
misheard proper nouns; and
|
||||
- exactly one schema-conforming JSON object with no explanatory prose.
|
||||
|
||||
Refer to supplied inputs generally. Do not limit the statement to transcript
|
||||
and reference inputs, because grounded extractors and normalizers receive
|
||||
other material. Do not place rendered inputs, transcript-evidence mechanics,
|
||||
identity policy, reference policy, registry policy, or reconciliation policy
|
||||
in the system message.
|
||||
|
||||
2. Revise
|
||||
`assets/dnd/shared/prompts/common-dnd-extraction-evidence.md` to own only
|
||||
extraction evidence and source-range policy. Retain transcript-only evidence,
|
||||
integer range endpoints, omission of `source_id`, and narrow non-contiguous
|
||||
ranges. Remove the universal JSON-only/output-schema sentences now owned by
|
||||
the system message. Avoid repeating detailed campaign-reference or registry
|
||||
semantics owned by their input fragments.
|
||||
|
||||
3. Replace `assets/dnd/shared/prompts/common-dnd-transcript.md` with:
|
||||
|
||||
- `common-dnd-transcript-full.md`, which renders the complete ordered session
|
||||
for scene chunking and does not imply that the complete session is one
|
||||
scene;
|
||||
- `common-dnd-transcript-chunk.md`, which renders one extraction chunk,
|
||||
limits reporting and inference to that chunk, and states that unit IDs
|
||||
retain source-wide meaning; and
|
||||
- `common-dnd-transcript-windows.md`, which renders selected normalization
|
||||
evidence windows and states that they may be incomplete, non-contiguous,
|
||||
or overlapping and that absence outside them is not evidence.
|
||||
|
||||
Each file must render `{{ input "transcript" }}` exactly once. Scope files
|
||||
describe material completeness and permissible use; do not move scene,
|
||||
extraction-evidence, or reconciliation policy into them.
|
||||
|
||||
4. Update `internal/modules/dnd/shared/assets.go` so `sharedPromptPaths` exposes
|
||||
the three new names and no longer exposes `common-dnd-transcript.md`.
|
||||
|
||||
5. Update every owning `promptAssetManifest` and current prompt YAML definition
|
||||
to select exactly one transcript-scope asset:
|
||||
|
||||
| Scope asset | Prompt IDs |
|
||||
| --- | --- |
|
||||
| `common-dnd-transcript-full.md` | `dnd.scenes` |
|
||||
| `common-dnd-transcript-chunk.md` | `dnd.spells`, `dnd.npcs`, `dnd.combat_turns`, `dnd.enemy_events`, `dnd.item_events`, `dnd.npc_interactions`, `dnd.scene_descriptions`, `dnd.locations`, `dnd.location_occurrences` |
|
||||
| `common-dnd-transcript-windows.md` | `dnd.npcs.normalize`, `dnd.locations.normalize` |
|
||||
|
||||
Preserve the existing message position and cache control while changing the
|
||||
referenced scope filename. Extraction prompts must continue to share the
|
||||
same rendered prefix through the chunk transcript.
|
||||
|
||||
6. Update focused prompt-asset and shared-asset tests. A focused assertion that
|
||||
an owning prompt family selects its required transcript scope is warranted;
|
||||
do not replace it with exact prose snapshots or a full shared-file inventory.
|
||||
Expand `internal/modules/dnd/register/prompt_cache_test.go` to include the
|
||||
location and location-occurrence extraction prompts in its byte-identical
|
||||
prefix check while retaining the existing other extraction lanes.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- The old `common-dnd-transcript.md` is absent and unreferenced.
|
||||
- Every D&D prompt prepares with exactly the transcript semantics supplied by
|
||||
its caller.
|
||||
- The shared system message is byte-identical across all D&D prompts and
|
||||
contains no variable input.
|
||||
- The extraction-evidence fragment no longer repeats the universal output rule.
|
||||
- Extraction prompts retain a byte-identical rendered prefix through campaign
|
||||
references and the chunk transcript when given identical inputs.
|
||||
- No public prompt, module, durable artifact, or pipeline contract changes.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/modules/dnd/shared/... ./internal/modules/dnd/...
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 2: Standardize Prompt And Context Filenames
|
||||
|
||||
### Goal
|
||||
|
||||
Make prompt definitions and variable context assets predictable without changing
|
||||
their rendered content or message order.
|
||||
|
||||
### Required changes
|
||||
|
||||
1. Rename every prompt-ID-derived YAML file below to `prompt.yaml` and update its
|
||||
owning `promptAssetManifest` entry to mount `prompt.yaml` from
|
||||
`prompts/prompt.yaml`:
|
||||
|
||||
- `assets/dnd/scenes/prompts/`
|
||||
- `assets/dnd/spells/prompts/`
|
||||
- `assets/dnd/npcs/extract/prompts/`
|
||||
- `assets/dnd/npcs/normalize/prompts/`
|
||||
- `assets/dnd/combat-turns/prompts/`
|
||||
- `assets/dnd/enemy-events/prompts/`
|
||||
- `assets/dnd/item-events/prompts/`
|
||||
- `assets/dnd/npc-interactions/prompts/`
|
||||
- `assets/dnd/scene-descriptions/prompts/`
|
||||
- `assets/dnd/locations/extract/prompts/`
|
||||
- `assets/dnd/locations/normalize/prompts/`
|
||||
- `assets/dnd/location-occurrences/prompts/`
|
||||
|
||||
Leave the YAML `id` fields unchanged. PromptKit continues to distinguish the
|
||||
definitions through each manifest's existing `ModuleDir`.
|
||||
|
||||
2. Apply these context-fragment renames and update both YAML `content_file`
|
||||
references and manifest `ModuleFiles` entries:
|
||||
|
||||
- `assets/dnd/spells/prompts/catalog.md` to `spell-catalog.md`;
|
||||
- `assets/dnd/enemy-events/prompts/grounding.md` to
|
||||
`combat-grounding.md`; and
|
||||
- `assets/dnd/location-occurrences/prompts/locations.md` to
|
||||
`location-registry.md`.
|
||||
|
||||
Do not edit their prompt prose in this mechanical stage.
|
||||
|
||||
3. Update tests that intentionally exercise the production asset layout:
|
||||
|
||||
- In `internal/modules/dnd/register/register_test.go`, retain a registrar-
|
||||
boundary check for one canonical `prompt.yaml` per maintained prompt, but
|
||||
remove lists of every task, instruction, context, and shared fragment.
|
||||
- In `internal/cli/production_contract_test.go`, remove private prompt-file
|
||||
inventory assertions already owned by the D&D registrar and module prompt-
|
||||
preparation tests. Retain its CLI composition and public catalog contract
|
||||
checks.
|
||||
- Update module-local tests only where a renamed file is intentionally part
|
||||
of the manifest/preparation behavior.
|
||||
- Do not rewrite generic `internal/framework/llm` or
|
||||
`internal/framework/promptfs` fixtures merely because they use arbitrary
|
||||
noncanonical filenames; those tests prove generic filesystem behavior, not
|
||||
the D&D authoring convention.
|
||||
|
||||
4. Search for and remove all production references to the retired filenames.
|
||||
Do not preserve copies or aliases.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- Every D&D prompt subtree contains `prompt.yaml` and no prompt-ID-derived YAML
|
||||
definition.
|
||||
- The three renamed context files have descriptive names and render in their
|
||||
original message positions.
|
||||
- Every module manifest mounts the canonical names and still produces a prompt
|
||||
fingerprint.
|
||||
- Production-composition tests protect registration behavior without
|
||||
enumerating every private content file.
|
||||
- All prompts prepare through PromptKit with unchanged IDs, versions, inputs,
|
||||
profiles, output modes, and cache controls.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/modules/dnd/... ./internal/cli/...
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 3: Consolidate Scene And Entity-Normalization Instructions
|
||||
|
||||
### Goal
|
||||
|
||||
Adopt the single `instructions.md` convention for scene chunking, scene
|
||||
description, NPC normalization, and location normalization while preserving
|
||||
their distinct message-order contracts.
|
||||
|
||||
### Required changes
|
||||
|
||||
1. For scene chunking, merge `task.md` and `instructions.md` into one coherent
|
||||
`instructions.md` and delete `task.md`. Retain:
|
||||
|
||||
- the definition of a coherent scene and meaningful boundary conditions;
|
||||
- reasons to split and reasons not to split;
|
||||
- complete ordered transcript coverage with no gaps or overlaps; and
|
||||
- inclusive, positive source-unit endpoints.
|
||||
|
||||
Remove the generic JSON-only rule now owned by the system prompt and field-
|
||||
exclusivity prose already enforced by the private response schema. In
|
||||
`prompt.yaml`, place the combined instructions after campaign references and
|
||||
before the full transcript. It has no cache control; references and the final
|
||||
full transcript retain ephemeral cache control.
|
||||
|
||||
2. For scene descriptions, merge `task.md` and `instructions.md` into one
|
||||
`instructions.md` and delete `task.md`. Retain the one-accepted-chunk scope,
|
||||
all `kind` definitions and precedence, and title and summary semantics.
|
||||
Remove schema field-list exclusions and universal JSON-only language. Keep
|
||||
the combined final instructions message ephemeral.
|
||||
|
||||
3. For both NPC and location normalizers, rename `task.md` to
|
||||
`instructions.md`, update `prompt.yaml` and the manifest, and edit the prose
|
||||
against `common-dnd-entity-reconciliation.md`:
|
||||
|
||||
- NPC instructions own same-individual alias judgment and canonical display-
|
||||
name preferences.
|
||||
- Location instructions own same-place identity judgment, separation of
|
||||
nested or uncertain places, and canonical display-name selection.
|
||||
- The shared reconciliation fragment alone owns opaque keys, valid group
|
||||
membership, canonical-member constraints, invention prohibitions, and
|
||||
omission of uncertain groups.
|
||||
|
||||
Preserve the normalizer order: system, module instructions, cacheable shared
|
||||
reconciliation policy, candidates, cacheable transcript windows.
|
||||
|
||||
4. Update module-local prompt preparation tests to assert meaningful relative
|
||||
ordering, input isolation, and cache boundaries. Remove exact message-count
|
||||
assertions whose only purpose was to preserve the former task/instruction
|
||||
split.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- These four prompt directories each contain one module-owned
|
||||
`instructions.md` and no `task.md`.
|
||||
- Scene and normalizer domain policy remains complete but does not repeat
|
||||
universal system, response-schema, or shared reconciliation rules.
|
||||
- Scene chunking still receives the full transcript last; normalizers still
|
||||
receive candidates before transcript windows.
|
||||
- Prompt preparation and existing scene/normalization behavior tests pass.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/modules/dnd/chunk/scenes ./internal/modules/dnd/extract/scenedescriptions ./internal/modules/dnd/normalize/npcs ./internal/modules/dnd/normalize/locations
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 4: Consolidate NPC, Location, And Item-Event Instructions
|
||||
|
||||
### Goal
|
||||
|
||||
Consolidate three comparatively self-contained extraction prompts and remove
|
||||
their shared or schema-enforced repetition without changing lane semantics.
|
||||
|
||||
### Required changes
|
||||
|
||||
1. Merge each prompt's `task.md` and `instructions.md` into its existing
|
||||
`instructions.md`, delete `task.md`, and update `prompt.yaml` and the owning
|
||||
manifest for:
|
||||
|
||||
- NPC extraction;
|
||||
- location extraction; and
|
||||
- item-event extraction.
|
||||
|
||||
2. NPC instructions must retain individually identifiable non-PC inclusion,
|
||||
proper-name/stable-title/alias eligibility, all existing exclusion
|
||||
categories, observed display spelling, and the prohibition on invented
|
||||
anonymous labels. Remove generic narrow-range, reference-not-evidence,
|
||||
schema-field, and lore-output restatements already owned elsewhere.
|
||||
|
||||
3. Location instructions must retain the current implemented location policy,
|
||||
including the current allowance for a generic label used for a specific
|
||||
place, supported place categories, alias/nesting handling, and domain
|
||||
exclusions. Do **not** implement the future proper-name-only registry policy
|
||||
in this stage. Remove only generic evidence, reference, source-range, and
|
||||
schema-shape repetition.
|
||||
|
||||
4. Item-event instructions must retain event eligibility, all five kind
|
||||
definitions, currency treatment, quantity semantics, holder transitions,
|
||||
collective `party` behavior, transfer restrictions, and the distinction
|
||||
between discovery and acquisition. Remove only generic evidence-range,
|
||||
output-envelope, and schema-field restatements.
|
||||
|
||||
5. Keep each consolidated extraction instruction as the final ephemeral
|
||||
message. Update focused tests to protect preparation, inputs, output schema,
|
||||
and cache placement without asserting the retired two-message structure.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- The three prompt directories contain no `task.md`.
|
||||
- No domain inclusion, exclusion, event-kind, currency, holder, or location
|
||||
behavior changes.
|
||||
- Generic evidence and response-shape rules have one shared/schema owner.
|
||||
- All three prompts prepare and their extractor tests remain green.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/modules/dnd/extract/npcs ./internal/modules/dnd/extract/locations ./internal/modules/dnd/extract/itemevents
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 5: Consolidate Spell And Combat-Turn Instructions
|
||||
|
||||
### Goal
|
||||
|
||||
Consolidate the spell and combat-turn prompts while keeping their identity,
|
||||
catalog, grounding, and event semantics in the correct assets.
|
||||
|
||||
### Required changes
|
||||
|
||||
1. Merge `task.md` and `instructions.md` into `instructions.md`, delete
|
||||
`task.md`, and update `prompt.yaml` and manifests for spells and combat turns.
|
||||
|
||||
2. Spell module instructions own:
|
||||
|
||||
- inclusion of actual casts and unambiguous declared casting attempts;
|
||||
- exclusion of mentions, plans, rules discussion, and unsupported catalog
|
||||
matches;
|
||||
- in-world caster attribution, including player-character and GM-narrated
|
||||
casts; and
|
||||
- the lane-specific requirement that evidence collectively support caster,
|
||||
spell, and casting event.
|
||||
|
||||
`spell-catalog.md` alone owns the rendered catalog, exact canonical spelling,
|
||||
and the fact that the catalog is recognition context rather than evidence.
|
||||
Shared identity and evidence fragments own their generic rules.
|
||||
|
||||
3. Combat-turn instructions own combat-event eligibility and exclusions,
|
||||
chronology, unmatched actors, turn-level evidence grouping, and the
|
||||
`turn_kind` meanings. Remove repeated player/NPC identity grounding,
|
||||
transcript-only evidence, source-range mechanics, and schema-field lists
|
||||
owned by shared assets or the response schema.
|
||||
|
||||
4. Keep both consolidated instruction messages final and ephemeral. Preserve
|
||||
the spell catalog and optional NPC grounding as separate variable-context
|
||||
messages after evidence policy and before module instructions.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- Both prompt directories contain no `task.md`.
|
||||
- Spell recognition remains catalog-constrained and caster attribution remains
|
||||
evidence-grounded.
|
||||
- Combat-turn categories, event boundaries, actor policy, and evidence grouping
|
||||
are unchanged.
|
||||
- Neither prompt repeats generic shared identity, evidence, or output-format
|
||||
policy.
|
||||
- Prompt preparation, spell schema/catalog tests, combat tests, and the shared
|
||||
extraction-prefix test pass.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/modules/dnd/extract/spells ./internal/modules/dnd/extract/combatturns ./internal/modules/dnd/register
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 6: Consolidate Grounded Occurrence And Enemy-Event Instructions
|
||||
|
||||
### Goal
|
||||
|
||||
Finish module prompt consolidation for the three registry- or artifact-grounded
|
||||
event extractors.
|
||||
|
||||
### Required changes
|
||||
|
||||
1. Merge `task.md` and `instructions.md` into `instructions.md`, delete
|
||||
`task.md`, and update `prompt.yaml` and manifests for:
|
||||
|
||||
- NPC interactions;
|
||||
- location occurrences; and
|
||||
- enemy events.
|
||||
|
||||
2. NPC-interaction instructions own registry membership as an eligibility
|
||||
constraint, occurrence boundaries, all interaction kinds and precedence,
|
||||
combat-alignment splitting, and domain exclusions. The shared NPC fragment
|
||||
owns registry rendering, canonical grounding, and its non-evidence status;
|
||||
shared identity and evidence fragments own their generic rules.
|
||||
|
||||
3. Location-occurrence instructions own occurrence eligibility, all occurrence
|
||||
kinds and precedence, handling of speculation, independent overlapping
|
||||
facts, and domain exclusions. `location-registry.md` owns exact ID/name
|
||||
rendering and registry-not-evidence semantics. Remove repeated schema fields,
|
||||
`source_id`, narrow-range, and registry-context language from module prose.
|
||||
|
||||
4. Enemy-event instructions own event eligibility, the five event kinds and
|
||||
outcome definitions, one-engagement-per-scene policy, exclusions, and naming
|
||||
of registered or grounded unnamed opponents. `combat-grounding.md` and the
|
||||
shared NPC fragment own rendering and the non-evidence status of auxiliary
|
||||
inputs. The shared evidence fragment owns source-range mechanics. Remove the
|
||||
repeated response-field list and generic evidence statements.
|
||||
|
||||
5. Keep each consolidated instruction message final and ephemeral, with all
|
||||
registry and grounding messages before it and after the chunk transcript.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- No D&D prompt directory contains `task.md`.
|
||||
- All occurrence kinds, precedence rules, enemy outcomes, splitting rules, and
|
||||
registry constraints remain intact.
|
||||
- Registry and auxiliary artifacts remain grounding only and never become
|
||||
transcript evidence.
|
||||
- All three prompts prepare, and focused extractor, registry, invariant, and
|
||||
cache-prefix tests pass.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/modules/dnd/extract/npcinteractions ./internal/modules/dnd/extract/locationoccurrences ./internal/modules/dnd/extract/enemyevents ./internal/modules/dnd/register
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 7: Normalize Private Scene And Spell Schema Identities
|
||||
|
||||
### Goal
|
||||
|
||||
Bring the two outlying private LLM response schemas into the naming convention
|
||||
already used by the other D&D prompt schemas, without touching durable artifact
|
||||
schemas.
|
||||
|
||||
### Required changes
|
||||
|
||||
1. Normalize the scene private response schema:
|
||||
|
||||
- rename `assets/dnd/scenes/schemas/dnd_scenes.v1.json` to
|
||||
`dnd_scenes_llm.v1.json`;
|
||||
- change its `$id` from `notarius.dnd.scenes` to
|
||||
`notarius.dnd.scenes.llm`;
|
||||
- change the scene `prompt.yaml` `schema_path` accordingly; and
|
||||
- update `internal/modules/dnd/chunk/scenes/schema.go` to use key
|
||||
`dnd_scenes_llm`, ID `notarius.dnd.scenes.llm`, name
|
||||
`notarius_dnd_scenes_llm_v1`, and the renamed asset path.
|
||||
|
||||
2. Normalize the already correctly named spell private response schema:
|
||||
|
||||
- keep the filename `dnd_spells_llm.v1.json`;
|
||||
- change its `$id` from `notarius.dnd.spells` to
|
||||
`notarius.dnd.spells.llm`; and
|
||||
- update `internal/modules/dnd/extract/spells/schema.go` to use key
|
||||
`dnd_spells_llm`, ID `notarius.dnd.spells.llm`, and name
|
||||
`notarius_dnd_spells_llm_v1`.
|
||||
|
||||
3. Keep both versions at `v1`; these are private pre-release response contracts,
|
||||
not a compatibility migration. Do not alter required fields, property types,
|
||||
enum values, `additionalProperties`, or any owning Go response type.
|
||||
|
||||
4. Update focused schema-loading, prompt-preparation, registrar, and production
|
||||
composition expectations. Carefully distinguish the private spell response
|
||||
schema from the durable spell artifact schema under
|
||||
`internal/modules/dnd/codec/spells/assets/schemas/`; the durable schema and
|
||||
`docs/integrations/dnd-spell-artifacts.md` must not change.
|
||||
|
||||
5. Remove every reference to the retired private scene filename and old private
|
||||
IDs/names/keys. Do not add aliases.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- Scene and spell private schemas use `_llm`/`.llm` consistently in filenames
|
||||
where applicable, registry keys, schema IDs, and response-schema names.
|
||||
- Both schemas still load and validate the same private Go response shapes.
|
||||
- Durable scene chunk-map and spell artifact contracts are unchanged.
|
||||
- Registrar and module prompt preparation tests pass without schema collisions.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/modules/dnd/chunk/scenes ./internal/modules/dnd/extract/spells ./internal/modules/dnd/register ./internal/cli
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 8: Document Conventions And Perform Final Verification
|
||||
|
||||
### Goal
|
||||
|
||||
Make the implemented conventions discoverable in their canonical documentation,
|
||||
remove stale references, and verify the complete repository.
|
||||
|
||||
### Required changes
|
||||
|
||||
1. Rewrite the prompt-construction portion of `docs/internal/dnd.md` to describe
|
||||
current implemented behavior:
|
||||
|
||||
- the canonical `prompt.yaml` and `instructions.md` local names;
|
||||
- module manifests as the owner of selection, order, and fingerprinting;
|
||||
- the `common-dnd-` visual provenance prefix;
|
||||
- the single-owner hierarchy for universal, family, input, module, and schema
|
||||
rules;
|
||||
- the eligibility test for system-prompt content;
|
||||
- the full transcript, chunk transcript, and transcript-window assets and
|
||||
their consumers;
|
||||
- the final extraction, scene-chunking, and entity-normalization message
|
||||
orders and cache controls;
|
||||
- the rule against default prompt repetition and the observational-evidence
|
||||
threshold for reintroducing it; and
|
||||
- the content-only `assets` boundary, with a link to ADR-0011 rather than a
|
||||
duplicate rationale.
|
||||
|
||||
Remove the current descriptions of separate task and instructions messages
|
||||
and the generic transcript fragment.
|
||||
|
||||
2. Do not add an assets-package README or a new ADR. Do not modify ADR-0011.
|
||||
`docs/internal/llm.md` should remain provider-neutral and should not duplicate
|
||||
D&D filenames or ordering. Update it only if an earlier stage changed a
|
||||
provider-neutral PromptKit preparation or cache-control fact; no such change
|
||||
is expected by this plan.
|
||||
|
||||
3. Review all changed prompt definitions and manifests as a set. Confirm that:
|
||||
|
||||
- every prompt has one system message using the shared system asset;
|
||||
- every prompt selects exactly one correct transcript-scope asset;
|
||||
- every extraction prompt retains the stable shared prefix and final
|
||||
ephemeral module instructions;
|
||||
- scene chunking and normalizers retain their intentional distinct orders;
|
||||
- no module-specific prompt repeats a rule already owned by its selected
|
||||
shared assets or response schema; and
|
||||
- the root `assets` Go package remains a single minimal filesystem file with
|
||||
no business logic or internal/PromptKit dependency.
|
||||
|
||||
4. Search for stale production references to:
|
||||
|
||||
- `common-dnd-transcript.md`;
|
||||
- prompt-ID-derived YAML filenames;
|
||||
- D&D `task.md` prompt files;
|
||||
- `catalog.md`, `grounding.md`, and the location-occurrence `locations.md`;
|
||||
- `dnd_scenes.v1.json` and the old scene private schema identity; and
|
||||
- the old spell private schema key, ID, and response-schema name.
|
||||
|
||||
Exclude historical roadmap text only when the reference intentionally
|
||||
explains a retired name. There should be no stale reference in current code,
|
||||
assets, tests, or implemented documentation.
|
||||
|
||||
5. Review tests for policy compliance. Retain behavior-level assertions for
|
||||
preparation, schema compatibility, input isolation, cache placement, and the
|
||||
identical extraction prefix. Remove any leftover assertion whose only value
|
||||
is freezing exact prompt prose, message count, shared-prefix length, or a
|
||||
complete private asset inventory.
|
||||
|
||||
6. Run formatting where applicable and the complete offline repository checks.
|
||||
Do not perform live model evaluation. Report that human evaluation with
|
||||
representative transcripts and cost-effective target models remains the
|
||||
next observational step after implementation; do not reintroduce prompt
|
||||
repetition before that evidence exists.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- `docs/internal/dnd.md` accurately describes the implemented prompt layout,
|
||||
ownership, transcript scopes, and cache ordering without duplicating external
|
||||
contracts.
|
||||
- No new ADR or assets README exists, and ADR-0011 remains unchanged.
|
||||
- No stale production references or retired files remain.
|
||||
- All maintained prompts prepare offline, all schemas register, all repository
|
||||
tests and static checks pass, and the CLI builds.
|
||||
- The final diff contains no durable artifact, public module/configuration, or
|
||||
pipeline behavior change.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./...
|
||||
go vet ./...
|
||||
go build ./cmd/notarius
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Run `gofmt` on the Go files changed by these stages before the commands above.
|
||||
Inspect the diff and ensure formatting did not rewrite unrelated user changes;
|
||||
never format an entire directory when the worktree contains unrelated edits.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None. The feature roadmap and this plan make the prompt ownership, transcript
|
||||
scope, naming, schema identity, documentation, compatibility, and testing
|
||||
decisions required for implementation.
|
||||
463
docs/roadmap/prompts.md
Normal file
463
docs/roadmap/prompts.md
Normal file
@@ -0,0 +1,463 @@
|
||||
# D&D Prompt Asset Cleanup
|
||||
|
||||
## Purpose
|
||||
|
||||
This roadmap defines the target state for the embedded D&D prompt and private
|
||||
schema assets. The work is intended to make prompts easier to inspect, compare,
|
||||
and revise; eliminate unnecessary repetition; preserve useful prompt-cache
|
||||
boundaries; and establish conventions that future D&D modules can follow.
|
||||
|
||||
The central design principle is that each instruction should have one clear
|
||||
owner. Shared behavior belongs in a shared prompt asset, module-specific
|
||||
behavior belongs in the module's instruction asset, variable grounding belongs
|
||||
in a dedicated input asset, and purely structural requirements belong in the
|
||||
JSON schema.
|
||||
|
||||
## Current State
|
||||
|
||||
The root `assets` package correctly centralizes embedded prompt and schema
|
||||
content without owning D&D business logic. The D&D prompt assets nevertheless
|
||||
retain several conventions inherited from their former module-local layouts:
|
||||
|
||||
- prompt definitions use prompt-ID-derived filenames rather than a consistent
|
||||
local filename;
|
||||
- most prompt directories divide fixed module instructions between `task.md`
|
||||
and `instructions.md`, although that division generally does not represent a
|
||||
meaningful cache, trust, or input boundary;
|
||||
- module-specific instructions repeat some rules already supplied by shared D&D
|
||||
messages or enforced by their response schemas;
|
||||
- one generic transcript fragment currently describes full-session input,
|
||||
chunk-scoped input, and selected normalization windows even though those
|
||||
materials have different completeness and inference boundaries;
|
||||
- variable context assets use several naming styles, including generic names
|
||||
such as `catalog.md` and `grounding.md`;
|
||||
- the private scene and spell response-schema identifiers do not follow the
|
||||
`_llm`/`.llm` convention used by the other private D&D schemas.
|
||||
|
||||
These are internal authoring and maintenance inconsistencies. The durable
|
||||
artifact contracts and the separation between content ownership in `assets`
|
||||
and behavior ownership in `internal/modules` remain sound.
|
||||
|
||||
## Goals
|
||||
|
||||
The completed cleanup should:
|
||||
|
||||
- give every D&D prompt directory the same predictable entry-point names;
|
||||
- consolidate fixed module-specific instructions into one message unless a
|
||||
distinct message boundary has a concrete purpose;
|
||||
- remove module-specific restatements of rules already owned by shared assets;
|
||||
- retain separate messages for shared policy, stable references, variable
|
||||
transcript content, and other inputs where the boundary communicates trust,
|
||||
provenance, reuse, or cache behavior;
|
||||
- distinguish full-session transcripts, extraction chunks, and normalization
|
||||
evidence windows through separate shared assets with explicit scope rules;
|
||||
- keep all shared D&D filenames visibly identifiable in prompt definitions;
|
||||
- make variable context filenames self-describing;
|
||||
- normalize private LLM response-schema identities without changing durable
|
||||
output schemas;
|
||||
- document the conventions future prompt authors must follow; and
|
||||
- preserve the rule that the root `assets` Go package contains no business
|
||||
logic.
|
||||
|
||||
## Decisions
|
||||
|
||||
### Canonical prompt directory layout
|
||||
|
||||
Every D&D prompt directory should use the following local layout:
|
||||
|
||||
```text
|
||||
prompts/
|
||||
prompt.yaml
|
||||
instructions.md
|
||||
[optional variable or context fragments]
|
||||
```
|
||||
|
||||
The prompt definition filename is always `prompt.yaml`. The prompt's `id` field,
|
||||
not its filename, remains the canonical global identifier. This is safe because
|
||||
each module registers a distinct asset subtree with PromptKit; the local
|
||||
`prompt.yaml` filenames do not share a virtual directory.
|
||||
|
||||
Use the `.yaml` extension consistently. It matches the existing embedded prompt
|
||||
and profile assets.
|
||||
|
||||
For prompts that currently have both `task.md` and `instructions.md`, combine
|
||||
their non-duplicative content into `instructions.md`. For prompts that currently
|
||||
have only `task.md`, rename that asset to `instructions.md`. Do not preserve an
|
||||
otherwise artificial two-message split merely to distinguish a short task
|
||||
statement from its rules.
|
||||
|
||||
### Retain the `common-dnd-` prefix
|
||||
|
||||
Shared D&D assets must retain the `common-dnd-` filename prefix, including:
|
||||
|
||||
- `common-dnd-system.md`
|
||||
- `common-dnd-identity.md`
|
||||
- `common-dnd-references.md`
|
||||
- `common-dnd-transcript-full.md`
|
||||
- `common-dnd-transcript-chunk.md`
|
||||
- `common-dnd-transcript-windows.md`
|
||||
- `common-dnd-extraction-evidence.md`
|
||||
- `common-dnd-npcs.md`
|
||||
- `common-dnd-entity-reconciliation.md`
|
||||
|
||||
The prefix is a deliberate visual provenance marker. When reviewing a prompt
|
||||
definition, an author should be able to distinguish shared messages from
|
||||
module-owned messages immediately. Because asset references are explicit and
|
||||
machine-resolved, retaining the prefix has no meaningful mechanical cost.
|
||||
|
||||
### One owner per instruction
|
||||
|
||||
Prompt prose should follow this ownership hierarchy:
|
||||
|
||||
1. Universal D&D prompting behavior belongs in a shared D&D asset.
|
||||
2. Rules shared by a prompt family belong in the narrowest applicable shared
|
||||
asset.
|
||||
3. Semantics of a variable input belong in the fragment that renders that
|
||||
input.
|
||||
4. Lane-specific extraction, classification, or normalization policy belongs in
|
||||
that lane's `instructions.md`.
|
||||
5. Object shape, required fields, enum values, and rejection of unknown fields
|
||||
belong in the JSON schema unless prose is needed to explain their domain
|
||||
meaning.
|
||||
|
||||
Module-specific prompt assets should omit instructions already provided by the
|
||||
shared prompt set. In particular, they should not routinely repeat generic rules
|
||||
about transcript-only evidence, references not being evidence, omission of
|
||||
`source_id`, narrow transcript ranges, schema-only output, or explanatory text.
|
||||
|
||||
This cleanup should begin with the minimal instruction set that is both correct
|
||||
and non-repetitive. Repetition may be reintroduced only when evaluation produces
|
||||
repeatable observational evidence that a target model needs reinforcement. Any
|
||||
such repetition should be narrow, placed close to the rule's canonical owner,
|
||||
and documented as a deliberate model-reliability tradeoff rather than allowed to
|
||||
become accidental drift.
|
||||
|
||||
This rule does not authorize removal of domain semantics. Similar wording is not
|
||||
duplication when it defines a distinct lane contract, such as event precedence,
|
||||
holder transitions, scene-kind definitions, identity inclusion criteria, or the
|
||||
relationship between a combat actor and an extracted event.
|
||||
|
||||
### System-prompt eligibility
|
||||
|
||||
`common-dnd-system.md` should own every fixed instruction that genuinely applies
|
||||
to every maintained D&D LLM prompt. Consolidating those universal rules into the
|
||||
system message gives them one authoritative owner, applies the appropriate
|
||||
instruction priority, and preserves one byte-identical initial message across
|
||||
chunking, extraction, and normalization prompts.
|
||||
|
||||
An instruction belongs in the shared system message only when it is:
|
||||
|
||||
- applicable to scene chunking, every extraction lane, and both entity
|
||||
normalizers;
|
||||
- independent of prompt-specific or variable inputs;
|
||||
- static across calls; and
|
||||
- appropriate to treat as a high-priority behavioral constraint.
|
||||
|
||||
The system message should therefore own the D&D transcript-processing role,
|
||||
reliance only on supplied inputs, treatment of transcription uncertainty, and
|
||||
the requirement to return exactly one schema-conforming JSON object without
|
||||
explanatory prose.
|
||||
|
||||
Rules shared by only a subset of prompts must remain in the narrowest applicable
|
||||
shared user-message fragment. Transcript-evidence and citation rules, identity
|
||||
rules, campaign-reference semantics, registry grounding, and reconciliation
|
||||
policy are not universal system instructions. No rendered transcript,
|
||||
reference, registry, catalog, candidate, or other variable material belongs in
|
||||
the system message.
|
||||
|
||||
### Shared prompt ownership
|
||||
|
||||
The shared D&D files should have clear, non-overlapping responsibilities:
|
||||
|
||||
| Asset | Canonical responsibility |
|
||||
| --- | --- |
|
||||
| `common-dnd-system.md` | D&D domain role, reliance on supplied inputs, treatment of transcription uncertainty, and the universal requirement to emit exactly one schema-conforming JSON object without commentary. |
|
||||
| `common-dnd-identity.md` | Rules for resolving supported in-world identities and the distinction between disambiguation and participation evidence. |
|
||||
| `common-dnd-references.md` | Rendering and interpretation of campaign references, including that references provide context rather than establishing transcript events. |
|
||||
| `common-dnd-transcript-full.md` | Rendering the complete ordered session transcript and identifying it as complete session input. |
|
||||
| `common-dnd-transcript-chunk.md` | Rendering one extraction chunk, limiting the task to that chunk, and explaining that its unit IDs retain their source-wide meaning. |
|
||||
| `common-dnd-transcript-windows.md` | Rendering selected reconciliation evidence windows and explaining that they may be incomplete, non-contiguous, or overlapping. |
|
||||
| `common-dnd-extraction-evidence.md` | Transcript-evidence policy, source-range construction, omission of `source_id`, and preference for narrow supporting ranges. |
|
||||
| `common-dnd-npcs.md` | Rendering and interpreting an upstream NPC registry as grounding rather than evidence. |
|
||||
| `common-dnd-entity-reconciliation.md` | Safe duplicate-group proposal mechanics shared by entity normalizers. |
|
||||
|
||||
As part of normalization, universal output-format prose should move to
|
||||
`common-dnd-system.md`, while evidence-specific prose should remain in
|
||||
`common-dnd-extraction-evidence.md`. The system message should refer generally
|
||||
to supplied inputs rather than only transcripts and references, because
|
||||
normalizers and grounded extractors receive additional input types.
|
||||
|
||||
### Transcript-scope ownership
|
||||
|
||||
The current `common-dnd-transcript.md` should be replaced rather than retained
|
||||
as an alias. All three replacements may render the same prompt input named
|
||||
`transcript`, but each must accurately describe the material supplied by its
|
||||
caller:
|
||||
|
||||
- `common-dnd-transcript-full.md` is used by scene chunking. It identifies the
|
||||
material as the complete ordered session and permits whole-session coverage
|
||||
decisions without implying that the entire session is one scene.
|
||||
- `common-dnd-transcript-chunk.md` is used by every extraction lane. It states
|
||||
that only the current chunk is in scope, that source-wide unit IDs remain
|
||||
authoritative, and that the model must not infer or report events outside the
|
||||
supplied chunk.
|
||||
- `common-dnd-transcript-windows.md` is used by NPC and location normalization.
|
||||
It states that the material consists of selected evidence windows that may be
|
||||
incomplete, non-contiguous, or overlapping. The model may use those windows
|
||||
to evaluate candidate identity but must not treat absence outside them as
|
||||
evidence.
|
||||
|
||||
The scope fragments should describe input completeness and permissible use;
|
||||
they should not absorb scene policy, extraction evidence mechanics, or entity
|
||||
reconciliation rules owned by other assets. Future prompt families must select
|
||||
the fragment matching the material actually passed by code rather than choosing
|
||||
one based merely on an input variable named `transcript`.
|
||||
|
||||
### Module-specific ownership
|
||||
|
||||
After shared duplication is removed, each module's `instructions.md` should
|
||||
retain only its substantive lane policy:
|
||||
|
||||
| Prompt | Module-owned policy |
|
||||
| --- | --- |
|
||||
| Scene chunking | Scene-boundary rules, ordered coverage, and valid treatment of gaps or overlaps. |
|
||||
| Scene descriptions | `kind` classification definitions and precedence, plus title and summary semantics. |
|
||||
| NPC extraction | Inclusion and exclusion rules for individually identifiable NPCs. |
|
||||
| NPC interactions | Interaction kinds, precedence, occurrence splitting, and NPC-specific event semantics. |
|
||||
| Location extraction | Inclusion and exclusion rules for identifiable in-world places. |
|
||||
| Location occurrences | Occurrence kinds, precedence, splitting, and location-specific event semantics. |
|
||||
| Spell extraction | What constitutes a cast, catalog-name policy, and caster attribution. |
|
||||
| Combat turns | Combat event boundaries, event kinds, actor semantics, and combat-specific inclusion rules. |
|
||||
| Item events | Event kinds, possession transitions, holder semantics, quantities, and currency treatment. |
|
||||
| Enemy events | Enemy-event kinds, outcome semantics, encounter identity, and combat-grounding rules. |
|
||||
| NPC normalization | NPC-specific duplicate criteria and canonical-name selection. |
|
||||
| Location normalization | Location-specific duplicate criteria and canonical-name selection. |
|
||||
|
||||
Schema field lists should not be copied into prose solely to restate the schema.
|
||||
Prose may still explain how a field is interpreted or how two fields relate when
|
||||
the schema cannot express that semantic rule.
|
||||
|
||||
### Message boundaries and cache ordering
|
||||
|
||||
Combining `task.md` and `instructions.md` must not lead to indiscriminate message
|
||||
consolidation. Separate messages remain appropriate when they preserve a shared
|
||||
byte-identical prefix, distinguish fixed instructions from variable material,
|
||||
or identify the provenance and trust level of an input.
|
||||
|
||||
Extraction prompts should retain this general order:
|
||||
|
||||
```text
|
||||
system
|
||||
identity
|
||||
campaign references
|
||||
chunk transcript
|
||||
evidence policy
|
||||
optional grounding or variable context
|
||||
module instructions
|
||||
```
|
||||
|
||||
The scene chunker should retain this general order:
|
||||
|
||||
```text
|
||||
system
|
||||
campaign references
|
||||
module instructions
|
||||
full transcript
|
||||
```
|
||||
|
||||
Entity normalizers should retain this general order:
|
||||
|
||||
```text
|
||||
system
|
||||
module instructions
|
||||
shared entity-reconciliation policy
|
||||
candidates
|
||||
transcript windows
|
||||
```
|
||||
|
||||
Existing cache controls should remain attached to the same semantic boundaries
|
||||
unless PromptKit's contract requires a mechanical adjustment. The cleanup must
|
||||
not move variable transcript or candidate content into an earlier stable prefix
|
||||
merely to reduce message count.
|
||||
|
||||
### Variable context asset names
|
||||
|
||||
Optional non-shared fragments should describe the input they render rather than
|
||||
use generic names. Normalize the current assets as follows where applicable:
|
||||
|
||||
- `catalog.md` to `spell-catalog.md`;
|
||||
- `grounding.md` to `combat-grounding.md`;
|
||||
- `locations.md` to `location-registry.md`.
|
||||
|
||||
The name `candidates.md` may remain in normalizer-local directories because its
|
||||
meaning is clear in that context. Future context assets should use similarly
|
||||
specific nouns and should remain separate from fixed `instructions.md` content.
|
||||
|
||||
### Private response-schema identity
|
||||
|
||||
Private schemas used only to constrain LLM responses should consistently use
|
||||
the `_llm` filename/key/name convention and the `.llm` schema-ID suffix already
|
||||
used by most D&D modules.
|
||||
|
||||
Normalize the scene schema to:
|
||||
|
||||
- file: `dnd_scenes_llm.v1.json`;
|
||||
- schema ID: `notarius.dnd.scenes.llm`;
|
||||
- registry key: `dnd_scenes_llm`;
|
||||
- response-schema name: `notarius_dnd_scenes_llm_v1`.
|
||||
|
||||
Normalize the spell schema metadata to:
|
||||
|
||||
- file: `dnd_spells_llm.v1.json` (already named correctly);
|
||||
- schema ID: `notarius.dnd.spells.llm`;
|
||||
- registry key: `dnd_spells_llm`;
|
||||
- response-schema name: `notarius_dnd_spells_llm_v1`.
|
||||
|
||||
Private schema filenames must remain globally unique. They should not be renamed
|
||||
to a generic local name because the schema registry exposes a shared namespace.
|
||||
Durable artifact schema identifiers and shapes are out of scope and must not
|
||||
change.
|
||||
|
||||
### Explicit prompt definitions remain explicit
|
||||
|
||||
The repeated `version`, `default_profile`, input, message, output, and schema
|
||||
declarations in `prompt.yaml` files should remain explicit. They are declarative
|
||||
configuration, not prompt-prose duplication. Introducing a generator, template
|
||||
layer, or Go builder for these small definitions would make inspection harder
|
||||
and create another synchronization boundary.
|
||||
|
||||
Similarly, module manifests should continue to declare the assets their prompt
|
||||
uses. Shared asset selection is module behavior and does not belong in the root
|
||||
`assets` package.
|
||||
|
||||
## Compatibility and Cache Effects
|
||||
|
||||
This is an intentional internal prompt revision. Renaming prompt definition
|
||||
files, consolidating messages, editing shared prose, and normalizing private
|
||||
schema identifiers will change prompt and component fingerprints. Existing
|
||||
checkpoints may no longer be reusable, and backend prompt caches will experience
|
||||
a one-time cold start. Those fingerprint, checkpoint, and cache effects are
|
||||
accepted consequences of reaching the target prompt layout.
|
||||
|
||||
The following contracts must remain stable:
|
||||
|
||||
- public prompt IDs;
|
||||
- module IDs and configuration fields;
|
||||
- default profile selection;
|
||||
- required and optional prompt inputs;
|
||||
- durable artifact schema IDs and artifact shapes;
|
||||
- extraction and normalization semantics, except for removing unintended prompt
|
||||
repetition; and
|
||||
- pipeline dependency behavior.
|
||||
|
||||
No backward-compatible copies of old private prompt assets or schema aliases are
|
||||
required while the application remains pre-release.
|
||||
|
||||
## Documentation and Maintenance Rules
|
||||
|
||||
The implemented convention should be documented in the canonical internal D&D
|
||||
and LLM documentation rather than in an assets-package README that could drift.
|
||||
The documentation should explain:
|
||||
|
||||
- the canonical `prompt.yaml` and `instructions.md` names;
|
||||
- why shared assets retain the `common-dnd-` prefix;
|
||||
- the instruction-ownership hierarchy;
|
||||
- the eligibility test for moving a rule into `common-dnd-system.md`;
|
||||
- the full, chunk, and evidence-window transcript contracts and which prompt
|
||||
families select each one;
|
||||
- when a separate message boundary is warranted;
|
||||
- the cache-oriented message ordering;
|
||||
- the prohibition on business logic in `assets`; and
|
||||
- the evidence threshold for deliberately repeating an instruction.
|
||||
|
||||
The Go file in the root `assets` package should remain a minimal embedding and
|
||||
filesystem-access layer. Prompt composition, schema selection, module policy,
|
||||
and validation logic must remain in their existing owning packages.
|
||||
|
||||
`docs/internal/dnd.md` should be the canonical owner of these D&D prompt
|
||||
authoring conventions and the concrete message-order patterns. Its existing
|
||||
prompt-construction section should be revised when the assets change rather
|
||||
than creating an `assets` README or duplicating the rules in contributor
|
||||
orientation. `docs/internal/llm.md` should receive only any short update needed
|
||||
to describe provider-neutral PromptKit preparation or cache-control mechanics;
|
||||
it should link to the D&D guide instead of repeating domain-specific asset names
|
||||
or ordering.
|
||||
|
||||
No new ADR is warranted for this cleanup. ADR-0011 already records the
|
||||
architecturally significant decisions: physical centralization, content-only
|
||||
package ownership, and retention of prompt semantics and ordering by modules.
|
||||
The filename, instruction-ownership, transcript-scope, and system-message rules
|
||||
are internal authoring conventions that should be allowed to evolve with model
|
||||
evaluation. They do not create a new dependency boundary or reverse an accepted
|
||||
architectural decision. If a later change promotes these conventions into a
|
||||
cross-domain runtime-enforced prompt composition contract, that broader
|
||||
decision should receive its own ADR at that time; ADR-0011 must not be edited to
|
||||
retrofit the new rationale.
|
||||
|
||||
## Validation and Evaluation Policy
|
||||
|
||||
Automated validation should establish that the refactor preserves executable
|
||||
contracts without freezing prompt prose. It should cover:
|
||||
|
||||
- preparation of every embedded prompt through PromptKit;
|
||||
- prompt input declarations and required-input behavior;
|
||||
- private response-schema registration and compatibility with the owning Go
|
||||
response types;
|
||||
- expected cache controls and meaningful stable-prefix boundaries;
|
||||
- selection of the transcript-scope fragment appropriate to each prompt
|
||||
family, verified through prompt preparation and owning-module behavior rather
|
||||
than exact prose snapshots;
|
||||
- manifest references after asset renames;
|
||||
- absence of references to retired filenames and private schema identifiers;
|
||||
and
|
||||
- the full repository test suite.
|
||||
|
||||
Tests must not assert exact prompt wording, exact message counts, shared-prefix
|
||||
lengths, or asset inventories solely as change detectors. Such tests would make
|
||||
intentional prompt iteration unnecessarily expensive and conflict with the
|
||||
project's testing policy.
|
||||
|
||||
Model quality should be evaluated by running representative transcripts through
|
||||
all affected lanes, with particular attention to cost-effective smaller models.
|
||||
Review should compare extraction correctness, unsupported claims, missing
|
||||
events, source-range quality, and schema reliability. The initial evaluation
|
||||
should use the minimal non-repetitive prompts. A repeated instruction should be
|
||||
restored only if failures recur and there is a credible causal reason that
|
||||
reinforcement will help.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This roadmap does not include:
|
||||
|
||||
- changes to durable D&D artifact schemas;
|
||||
- new extraction lanes or pipeline behavior;
|
||||
- changes to profile selection or model settings;
|
||||
- a prompt-definition generator or templating framework;
|
||||
- moving business logic into `assets`;
|
||||
- collapsing all prompt material into one message;
|
||||
- placing family-specific rules or rendered variable content in the system
|
||||
message;
|
||||
- preserving old prompt fingerprints or warm backend cache entries; or
|
||||
- introducing brittle prompt snapshots or change-detector tests.
|
||||
|
||||
## Target End State
|
||||
|
||||
Every D&D prompt is discoverable through a local `prompt.yaml`, has one fixed
|
||||
module-owned `instructions.md`, and uses separately named context fragments only
|
||||
where inputs or cache boundaries justify them. Shared D&D assets remain visibly
|
||||
marked with `common-dnd-` and have documented, non-overlapping responsibilities.
|
||||
Module instructions contain no routine repetition of shared or schema-enforced
|
||||
rules, while retaining all lane-specific domain semantics.
|
||||
|
||||
The shared system message contains all and only universal, static D&D behavior.
|
||||
Scene chunking renders the explicitly full transcript fragment, extraction
|
||||
lanes render the explicitly chunk-scoped fragment, and entity normalizers render
|
||||
the explicitly incomplete evidence-window fragment. No prompt presents one
|
||||
scope as another merely because all three inputs use the name `transcript`.
|
||||
|
||||
Private scene and spell response schemas follow the established LLM-only naming
|
||||
convention, durable contracts are unchanged, all manifests and PromptKit
|
||||
definitions prepare successfully, and human evaluation begins from a minimal
|
||||
non-repetitive prompt set. Future modules can follow the same conventions
|
||||
without copying an existing extractor and inheriting accidental prose or naming
|
||||
drift.
|
||||
Reference in New Issue
Block a user