258 lines
12 KiB
Markdown
258 lines
12 KiB
Markdown
# D&D Prompt Cache Ordering Implementation Plan
|
|
|
|
## Summary
|
|
|
|
Implement the target state in
|
|
[D&D Prompt Cache Ordering](dnd-prompt-cache-ordering.md): give all six D&D
|
|
extraction prompts a byte-identical rendered prefix through the transcript
|
|
chunk, retain purpose-specific ordering for scene chunking and NPC
|
|
normalization, consolidate tests around durable prompt-order invariants, and
|
|
replace the current transcript-last documentation.
|
|
|
|
This is a prompt-asset and documentation change. Do not change Go module APIs,
|
|
LLM request contracts, prompt IDs or versions, response schemas, artifact
|
|
schemas, reference declarations, extraction behavior, retry behavior, or
|
|
durable output. Existing prompt fingerprints must continue to provide
|
|
checkpoint invalidation; no checkpoint migration or cleanup is required.
|
|
|
|
## Stage 1: Reorder The Six Extraction Prompts
|
|
|
|
Update these manifests:
|
|
|
|
- `internal/modules/dnd/extract/npcs/assets/prompts/dnd.npcs.yaml`
|
|
- `internal/modules/dnd/extract/itemevents/assets/prompts/dnd.item_events.yaml`
|
|
- `internal/modules/dnd/extract/scenedescriptions/assets/prompts/dnd.scene_descriptions.yaml`
|
|
- `internal/modules/dnd/extract/combatturns/assets/prompts/dnd.combat_turns.yaml`
|
|
- `internal/modules/dnd/extract/npcinteractions/assets/prompts/dnd.npc_interactions.yaml`
|
|
- `internal/modules/dnd/extract/spells/assets/prompts/dnd.spells.yaml`
|
|
|
|
Give every manifest this exact leading message sequence:
|
|
|
|
1. `common-dnd-system.md`, with role `system` and no cache control;
|
|
2. `common-dnd-identity.md`, with role `user` and no cache control;
|
|
3. `common-dnd-references.md`, with role `user` and ephemeral cache control;
|
|
4. `common-dnd-transcript.md`, with role `user` and ephemeral cache control.
|
|
|
|
The common prefix must contain no extraction-evidence policy, NPC registry,
|
|
spell catalog, task, or module instructions. Those messages vary across lanes
|
|
and would prevent the transcript from participating in a shared cache prefix.
|
|
|
|
Append the following lane-specific sequences after the transcript:
|
|
|
|
| Prompt | Post-transcript sequence |
|
|
| --- | --- |
|
|
| `dnd.npcs` | `common-dnd-extraction-evidence.md`, `task.md`, `instructions.md` |
|
|
| `dnd.item_events` | `common-dnd-extraction-evidence.md`, `task.md`, `instructions.md` |
|
|
| `dnd.scene_descriptions` | `task.md`, `instructions.md` |
|
|
| `dnd.combat_turns` | `common-dnd-extraction-evidence.md`, `common-dnd-npcs.md`, `task.md`, `instructions.md` |
|
|
| `dnd.npc_interactions` | `common-dnd-extraction-evidence.md`, `common-dnd-npcs.md`, `task.md`, `instructions.md` |
|
|
| `dnd.spells` | `common-dnd-extraction-evidence.md`, `common-dnd-npcs.md`, `catalog.md`, `task.md`, `instructions.md` |
|
|
|
|
For every extraction prompt:
|
|
|
|
- apply ephemeral cache control to the final `instructions.md` message;
|
|
- do not apply cache control to the extraction-evidence, NPC-registry, catalog,
|
|
or task messages;
|
|
- remove the existing cache control from `common-dnd-identity.md` and
|
|
`common-dnd-npcs.md`; and
|
|
- retain the existing inputs, requiredness, media types, output contract,
|
|
profile, repair setting, prompt identity, and schema path.
|
|
|
|
Do not add the extraction-evidence asset to the scene-description prompt merely
|
|
to make it resemble the other lanes. It is not universal shared context and
|
|
therefore belongs outside the common-prefix contract.
|
|
|
|
### Stage 1 completion criteria
|
|
|
|
- Equivalent transcript, player, party, and glossary inputs render the same
|
|
first four messages for all six extraction prompts, including identical
|
|
roles, content, and cache metadata.
|
|
- Each transcript input is rendered exactly once.
|
|
- Every lane-specific message follows the transcript.
|
|
- Prompt preparation and schema wiring remain successful for every lane.
|
|
|
|
## Stage 2: Apply Purpose-Specific Ordering To Other D&D LLM Prompts
|
|
|
|
Update
|
|
`internal/modules/dnd/chunk/scenes/assets/prompts/dnd.scenes.yaml` to use:
|
|
|
|
1. `common-dnd-system.md`, without cache control;
|
|
2. `common-dnd-references.md`, with ephemeral cache control;
|
|
3. `task.md`, without cache control;
|
|
4. `instructions.md`, without cache control; and
|
|
5. `common-dnd-transcript.md`, with ephemeral cache control.
|
|
|
|
The scene chunker consumes the full transcript once and has no sibling lanes
|
|
that can share a transcript prefix. Stable task instructions therefore remain
|
|
before the changing full transcript.
|
|
|
|
Update
|
|
`internal/modules/dnd/normalize/npcs/assets/prompts/dnd.npcs.normalize.yaml` to
|
|
use:
|
|
|
|
1. `common-dnd-system.md`, without cache control;
|
|
2. `task.md`, without cache control;
|
|
3. `instructions.md`, with ephemeral cache control;
|
|
4. `candidates.md`, without cache control; and
|
|
5. `common-dnd-transcript.md`, with ephemeral cache control.
|
|
|
|
Remove the normalizer system message's current cache control. Preserve the
|
|
candidate-before-evidence ordering: the transcript windows support evaluation
|
|
of the candidate collection and are not a cross-lane transcript prefix.
|
|
|
|
Do not change input material construction, transcript windowing, prompt
|
|
fragments, module implementations, or response handling in either package.
|
|
|
|
### Stage 2 completion criteria
|
|
|
|
- Scene chunking renders references separately before the task and the full
|
|
transcript exactly once at the end.
|
|
- NPC normalization renders candidates and transcript windows in separate
|
|
messages, in that order, with the agreed cache boundaries.
|
|
- Both prompts retain their current identities, schemas, inputs, and runtime
|
|
behavior.
|
|
|
|
## Stage 3: Consolidate Prompt-Ordering Verification
|
|
|
|
Read and follow `docs/policy/testing.md` before modifying tests. The meaningful
|
|
risk is loss of the shared rendered prefix or incorrect placement of
|
|
request-specific material, not edits to prompt prose.
|
|
|
|
Add one family-level extraction invariant test in
|
|
`internal/modules/dnd/register/prompt_cache_test.go` (or an equivalently scoped
|
|
new register-package test file):
|
|
|
|
- register the complete D&D prompt asset family in a real in-memory
|
|
`llm.AssetRegistry`;
|
|
- construct a Scriptorium engine with a local, non-networked test profile;
|
|
- prepare all six extraction prompts using their exported prompt IDs and
|
|
versions;
|
|
- supply the same unique transcript, players, party, and glossary sentinels to
|
|
every prompt, plus only the lane-specific required inputs such as the NPC
|
|
registry and spell catalog;
|
|
- locate the one rendered message containing the transcript sentinel rather
|
|
than locating it through prompt prose or a hard-coded phrase;
|
|
- assert that the transcript sentinel occurs exactly once in each prepared
|
|
prompt;
|
|
- compare the rendered messages from the start through the transcript message
|
|
across all six prompts, including role, full rendered content, and complete
|
|
cache-control metadata;
|
|
- assert that the shared prefix contains exactly the four agreed messages and
|
|
that at least one lane-specific message follows it in every extraction
|
|
prompt; and
|
|
- assert that NPC-registry and spell-catalog sentinels, where supplied, occur
|
|
only after the transcript.
|
|
|
|
Use this family-level test as the single owner of the cross-lane prefix policy.
|
|
Do not duplicate the same prefix assertion in every extractor package.
|
|
|
|
Revise the existing prompt-asset tests in:
|
|
|
|
- `internal/modules/dnd/extract/npcs/scriptorium_assets_test.go`
|
|
- `internal/modules/dnd/extract/itemevents/scriptorium_assets_test.go`
|
|
- `internal/modules/dnd/extract/scenedescriptions/scriptorium_assets_test.go`
|
|
- `internal/modules/dnd/extract/combatturns/scriptorium_assets_test.go`
|
|
- `internal/modules/dnd/extract/npcinteractions/scriptorium_assets_test.go`
|
|
- `internal/modules/dnd/extract/spells/scriptorium_assets_test.go`
|
|
- `internal/modules/dnd/chunk/scenes/scriptorium_assets_test.go`
|
|
- `internal/modules/dnd/normalize/npcs/scriptorium_assets_test.go`
|
|
|
|
Remove transcript-last expectations that contradict the new extraction policy.
|
|
Keep package-local coverage only where it protects package-owned behavior, such
|
|
as prompt/schema registration, declared shared assets, isolation and placement
|
|
of package-specific input material, cache boundaries unique to the scene
|
|
chunker or NPC normalizer, and non-disclosure properties. Consolidate or remove
|
|
positional and prose-marker checks already covered by the family-level
|
|
invariant.
|
|
|
|
Tests may search for unique input sentinels created by the test. They must not
|
|
require the presence of specific words, phrases, sentences, or other prose in a
|
|
prompt asset. Do not introduce golden snapshots of complete prompts or a test
|
|
that treats harmless wording changes as failures.
|
|
|
|
### Stage 3 completion criteria
|
|
|
|
- One durable family-level test fails if any extraction lane moves
|
|
lane-specific material ahead of the transcript or changes the common
|
|
rendered prefix.
|
|
- Local asset tests protect only nonredundant package behavior and the two
|
|
purpose-specific prompt sequences.
|
|
- The test suite contains no prompt-language change detector.
|
|
- All prompt tests remain deterministic, offline, and independent of
|
|
credentials.
|
|
|
|
## Stage 4: Update Canonical Internal Documentation
|
|
|
|
Update `docs/internal/llm.md` as the generic owner of prompt-cache mechanics:
|
|
|
|
- explain that backend reuse depends on identical preceding roles, rendered
|
|
bytes, and cache metadata, not merely equivalent semantics;
|
|
- state the general sibling-prompt ordering rule: universal shared context,
|
|
request source material, then module-specific suffixes;
|
|
- explain that a cache boundary should be placed at a useful reusable prefix
|
|
and that redundant intermediate boundaries add no value;
|
|
- retain the rule that prompt-family owners may choose a different sequence
|
|
when their inputs and reuse pattern differ; and
|
|
- replace the current statement that D&D extraction puts transcripts last with
|
|
a link to the D&D-specific policy.
|
|
|
|
Update `docs/internal/dnd.md` as the concrete owner of maintained D&D prompt
|
|
composition:
|
|
|
|
- document the four-message extraction prefix and its cache boundaries;
|
|
- state that extraction-evidence policy, generated NPC registries, catalogs,
|
|
tasks, and instructions follow the transcript because they are not universal
|
|
across all extraction lanes;
|
|
- summarize the scene-chunking and NPC-normalization exceptions and their
|
|
rationale; and
|
|
- retain existing rules about shared-asset reuse, canonical reference input,
|
|
prompt fingerprints, and prompt behavior.
|
|
|
|
Do not update the README, configuration reference, architecture policy, CLI
|
|
documentation, or integration contracts. Prompt message order is an internal
|
|
runtime policy and has no user-visible configuration or wire-contract change.
|
|
|
|
### Stage 4 completion criteria
|
|
|
|
- No current-behavior document still says that D&D extraction transcripts are
|
|
final messages.
|
|
- Generic cache mechanics and D&D-specific ordering each have one canonical
|
|
owner.
|
|
- Documentation describes implemented behavior once the prompt changes land
|
|
and does not duplicate volatile manifest inventories unnecessarily.
|
|
|
|
## Stage 5: Verification
|
|
|
|
Inspect representative prepared prompts to confirm that the test inputs produce
|
|
the intended shared prefix and lane-specific suffixes. Then run:
|
|
|
|
```sh
|
|
git diff --check
|
|
go test ./internal/modules/dnd/chunk/scenes
|
|
go test ./internal/modules/dnd/extract/...
|
|
go test ./internal/modules/dnd/normalize/npcs
|
|
go test ./internal/modules/dnd/register
|
|
go test ./internal/modules/dnd/...
|
|
go test ./internal/modules/integration/...
|
|
go test ./...
|
|
go vet ./...
|
|
go build ./cmd/notarius
|
|
```
|
|
|
|
The focused commands should be used while implementing their respective
|
|
stages; the repository-wide commands are the final acceptance gate.
|
|
|
|
## Expected Compatibility Effects
|
|
|
|
- Reordered prompt content changes prompt fingerprints and may intentionally
|
|
cause one-time cold misses for affected development checkpoints.
|
|
- Prompt IDs, prompt versions, schema identities, checkpoint format, durable
|
|
artifacts, and public behavior do not change.
|
|
- Backend cache reuse remains provider-dependent, but Notarius supplies a
|
|
longer common extraction prefix that includes the transcript chunk.
|
|
|
|
## Open Questions
|
|
|
|
None. The message sequences, cache boundaries, test ownership, documentation
|
|
owners, and compatibility policy are specified above.
|