From de8ed41b34e229149b5ad69b81e487cc552e5292 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 27 Jul 2026 20:16:11 +0000 Subject: [PATCH] Document D&D prompt cache ordering --- docs/internal/dnd.md | 28 +++++++++++++++++++++++++--- docs/internal/llm.md | 27 +++++++++++++++------------ 2 files changed, 40 insertions(+), 15 deletions(-) diff --git a/docs/internal/dnd.md b/docs/internal/dnd.md index 7372eca..57be402 100644 --- a/docs/internal/dnd.md +++ b/docs/internal/dnd.md @@ -39,9 +39,31 @@ D&D extractors assemble prompts from an ordered manifest of shared and module-owned assets. Reuse the shared D&D system, evidence, identity, reference, and transcript assets instead of copying their text into individual modules. A manifest’s declared sequence, including cache-control placement, is -part of the prompt behavior, and the chunk transcript is the final message. -Preserve that order when changing an extractor or its assets so prompt-cache -behavior remains stable. +part of the prompt behavior. + +All extraction prompts share this four-message rendered prefix: the system +message without cache control, the identity message without cache control, the +campaign-reference message with ephemeral cache control, and the chunk +transcript message with ephemeral cache control. This gives equivalent +extraction requests the same reusable prefix through their source material. + +Extraction-evidence policy, generated NPC registries, spell catalogs, module +tasks, and instructions follow the transcript because they are not universal +across all extraction lanes. The final instructions message carries ephemeral +cache control; evidence, registry, catalog, and task messages do not. Preserve +this division when changing an extractor or its assets so prompt-cache behavior +remains stable. + +The other D&D LLM prompts intentionally follow different patterns. Scene +chunking has no sibling extraction lane with which to share its full transcript, +so it renders campaign references before its task and instructions, then places +the cacheable full transcript last. NPC normalization keeps its task and +cacheable instructions before the candidate collection, followed by the +cacheable transcript windows: candidates must be available before their +supporting evidence is evaluated, and those windows are not a cross-lane +prefix. Mounted assets and their declared message order determine the prompt +fingerprint, so intentional prompt edits continue to invalidate stale +checkpoints. All extractors use the shared prompt-input preparation rules. The current chunk is copied into transcript material; player, party, glossary, and compatible diff --git a/docs/internal/llm.md b/docs/internal/llm.md index 85d66b6..c62f741 100644 --- a/docs/internal/llm.md +++ b/docs/internal/llm.md @@ -76,19 +76,22 @@ contract. Durable formats and compatibility rules remain in the ## Prompt Maintenance And Backend Caching Prompt message order and shared asset bytes are runtime behavior. Backend cache -reuse depends on the same preceding messages and content, not merely equivalent -meaning. Keep reusable shared assets byte-identical and keep stable material -before the inputs that vary per request wherever a prompt’s declared sequence -supports caching. Preserve the existing manifest order and cache-control hints -when editing a prompt. +reuse depends on identical preceding roles, rendered bytes, and cache-control +metadata—not merely equivalent meaning. Keep reusable shared assets +byte-identical and preserve each prompt’s declared ordering and cache controls +when editing it. -D&D extraction manifests place the changing chunk transcript at the end of the -prompt after their reusable context. Scene chunking and NPC normalization use -their own declared message sequences because their inputs and work differ. The -family-specific asset and ordering rules belong in [D&D Module Internals](dnd.md). -Do not add tests that enforce a fixed message-prefix length; prompt-asset tests -should instead verify the meaningful asset sequence, inputs, and cache controls -of the prompt being changed. +For sibling prompts that can reuse the same source material, order universal +shared context first, request source material next, and module-specific +suffixes last. Put a cache boundary at a reusable prefix that is useful to the +backend. Redundant intermediate cache boundaries do not extend that reusable +prefix and add no value. + +Prompt-family owners may choose a different sequence when their inputs and +reuse pattern differ. The D&D family’s extraction, scene-chunking, and NPC +normalization policies are maintained in [D&D Module Internals](dnd.md#prompt-construction). +Do not add tests that enforce prompt prose; prompt tests should verify the +meaningful input placement and cache controls of the prompt being changed. ## Validation, Repair, And Retries