diff --git a/docs/roadmap/dnd-prompt-cache-ordering.md b/docs/roadmap/dnd-prompt-cache-ordering.md deleted file mode 100644 index 60196cb..0000000 --- a/docs/roadmap/dnd-prompt-cache-ordering.md +++ /dev/null @@ -1,90 +0,0 @@ -# D&D Prompt Cache Ordering - -## Purpose - -Arrange maintained D&D prompts so sibling extraction lanes share the longest -useful byte-identical prefix, including the transcript chunk, and can therefore -reuse provider-side prompt caches effectively. - -This policy concerns rendered message order and cache boundaries. It does not -change extraction semantics, artifact contracts, module inputs, durable output, -or the responsibilities of deterministic validators and normalizers. - -## Extraction Prompt Policy - -Every D&D extraction prompt shares this rendered prefix: - -1. the shared D&D system prompt; -2. the shared D&D identity prompt; -3. the shared campaign-reference prompt; and -4. the shared transcript-chunk prompt. - -The messages must render identically, including roles, content, and cache -metadata, when two lanes receive the same transcript and common references. -The campaign-reference and transcript messages establish ephemeral cache -boundaries. - -Only material used by every extraction lane belongs before the transcript. -Shared wording that applies to only a subset of lanes is placed after the -transcript rather than being added to unrelated prompts solely to lengthen the -common prefix. Module instructions, optional generated references, catalogs, -and other lane-specific context also follow the transcript. - -The lane-specific suffixes are: - -- NPCs: shared extraction-evidence policy, task, then instructions. -- Item events: shared extraction-evidence policy, task, then instructions. -- Scene descriptions: task, then instructions. -- Combat turns: shared extraction-evidence policy, NPC registry, task, then - instructions. -- NPC interactions: shared extraction-evidence policy, NPC registry, task, - then instructions. -- Spells: shared extraction-evidence policy, NPC registry, spell catalog, task, - then instructions. - -The final instructions message establishes an ephemeral cache boundary. -Intermediate cache markers on the shared identity message or lane-specific -reference messages are unnecessary. - -## Purpose-Specific Prompt Families - -The extraction prefix is a sibling-lane policy, not a universal ordering rule -for every D&D LLM call. - -Scene chunking has no sibling lane with which to share a transcript prefix. It -orders its prompt as system, common campaign references, task, instructions, -and full transcript. The campaign-reference and transcript messages establish -ephemeral cache boundaries. - -NPC normalization orders its prompt as system, task, instructions, candidate -NPCs, and transcript windows. The instructions and transcript-window messages -establish ephemeral cache boundaries. Candidate artifacts remain ahead of the -evidence windows needed to evaluate them. - -Future D&D prompt families should identify their actual reuse boundary rather -than mechanically copying either exception or the extraction sequence. - -## Compatibility And Observability - -Existing prompt IDs, prompt versions, response schemas, and module contracts -remain unchanged. Prompt-content fingerprints already make the reordered -assets part of checkpoint identity, so old development checkpoints may become -cold misses without a checkpoint-format migration. - -Prompt-order verification should exercise rendered messages with unique input -sentinels. Tests must protect roles, cache metadata, input isolation, and the -shared-prefix invariant without requiring particular prose, words, or phrases -to remain in prompt assets. - -## Desired End State - -- All six D&D extraction prompts render the same four-message prefix for - equivalent common inputs. -- The transcript chunk is included in that common prefix and is followed only - by lane-specific material. -- Cache-control hints identify useful prefix boundaries without redundant - intermediate markers. -- Scene chunking and NPC normalization retain orderings suited to their - distinct inputs and reuse opportunities. -- Internal documentation explains both the general provider-cache principle - and the concrete D&D prompt-family policy. diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index ee254c3..24a043c 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -7,6 +7,37 @@ not as committed release dates. ## Near-Term D&D Pipeline +### Combat Enemy Ledger + +- Add a D&D artifact that identifies enemies faced during combat and supports + an end-of-session encounter ledger. +- Track each enemy's observed state using a small controlled vocabulary such as + `active`, `killed`, `fled`, `captured`, or `incapacitated`, while preserving + an explicit unresolved state when the transcript does not establish an + outcome. +- Preserve the evidence for enemy participation and state changes rather than + inferring a terminal outcome from combat ending or an enemy disappearing + from the conversation. +- Define how repeated mentions, groups of unnamed enemies, summoned or allied + creatures, and the same enemy appearing in multiple combats affect identity + and ledger entries. +- Evaluate whether the ledger should be extracted directly, derived from + combat-turn artifacts, or use a sequential pipeline that consumes combat + turns and the normalized NPC registry as grounding references. + +### Location Extraction + +- Add a D&D artifact for locations visited by the party or otherwise mentioned + in the transcript. +- Distinguish observed visits from references, plans, recalled places, and + uncertain or inferred locations so a mention alone is not reported as a + visit. +- Preserve transcript evidence for each visit or mention and reconcile aliases, + nested places, and repeated appearances without collapsing distinct + locations that share a generic name. +- Define how the location artifact should ground later narrative reports and + whether future event artifacts should retain canonical location identities. + ### Evaluate Spell Extraction And Normalization - Evaluate ordinary extraction retries and the completed normalization path diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index 08cc2f3..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,257 +0,0 @@ -# D&D Prompt Cache Ordering Implementation Plan - -## Summary - -Implement the target state in -[D&D Prompt Cache Ordering](dnd-prompt-cache-ordering.md): give all six D&D -extraction prompts a byte-identical rendered prefix through the transcript -chunk, retain purpose-specific ordering for scene chunking and NPC -normalization, consolidate tests around durable prompt-order invariants, and -replace the current transcript-last documentation. - -This is a prompt-asset and documentation change. Do not change Go module APIs, -LLM request contracts, prompt IDs or versions, response schemas, artifact -schemas, reference declarations, extraction behavior, retry behavior, or -durable output. Existing prompt fingerprints must continue to provide -checkpoint invalidation; no checkpoint migration or cleanup is required. - -## Stage 1: Reorder The Six Extraction Prompts - -Update these manifests: - -- `internal/modules/dnd/extract/npcs/assets/prompts/dnd.npcs.yaml` -- `internal/modules/dnd/extract/itemevents/assets/prompts/dnd.item_events.yaml` -- `internal/modules/dnd/extract/scenedescriptions/assets/prompts/dnd.scene_descriptions.yaml` -- `internal/modules/dnd/extract/combatturns/assets/prompts/dnd.combat_turns.yaml` -- `internal/modules/dnd/extract/npcinteractions/assets/prompts/dnd.npc_interactions.yaml` -- `internal/modules/dnd/extract/spells/assets/prompts/dnd.spells.yaml` - -Give every manifest this exact leading message sequence: - -1. `common-dnd-system.md`, with role `system` and no cache control; -2. `common-dnd-identity.md`, with role `user` and no cache control; -3. `common-dnd-references.md`, with role `user` and ephemeral cache control; -4. `common-dnd-transcript.md`, with role `user` and ephemeral cache control. - -The common prefix must contain no extraction-evidence policy, NPC registry, -spell catalog, task, or module instructions. Those messages vary across lanes -and would prevent the transcript from participating in a shared cache prefix. - -Append the following lane-specific sequences after the transcript: - -| Prompt | Post-transcript sequence | -| --- | --- | -| `dnd.npcs` | `common-dnd-extraction-evidence.md`, `task.md`, `instructions.md` | -| `dnd.item_events` | `common-dnd-extraction-evidence.md`, `task.md`, `instructions.md` | -| `dnd.scene_descriptions` | `task.md`, `instructions.md` | -| `dnd.combat_turns` | `common-dnd-extraction-evidence.md`, `common-dnd-npcs.md`, `task.md`, `instructions.md` | -| `dnd.npc_interactions` | `common-dnd-extraction-evidence.md`, `common-dnd-npcs.md`, `task.md`, `instructions.md` | -| `dnd.spells` | `common-dnd-extraction-evidence.md`, `common-dnd-npcs.md`, `catalog.md`, `task.md`, `instructions.md` | - -For every extraction prompt: - -- apply ephemeral cache control to the final `instructions.md` message; -- do not apply cache control to the extraction-evidence, NPC-registry, catalog, - or task messages; -- remove the existing cache control from `common-dnd-identity.md` and - `common-dnd-npcs.md`; and -- retain the existing inputs, requiredness, media types, output contract, - profile, repair setting, prompt identity, and schema path. - -Do not add the extraction-evidence asset to the scene-description prompt merely -to make it resemble the other lanes. It is not universal shared context and -therefore belongs outside the common-prefix contract. - -### Stage 1 completion criteria - -- Equivalent transcript, player, party, and glossary inputs render the same - first four messages for all six extraction prompts, including identical - roles, content, and cache metadata. -- Each transcript input is rendered exactly once. -- Every lane-specific message follows the transcript. -- Prompt preparation and schema wiring remain successful for every lane. - -## Stage 2: Apply Purpose-Specific Ordering To Other D&D LLM Prompts - -Update -`internal/modules/dnd/chunk/scenes/assets/prompts/dnd.scenes.yaml` to use: - -1. `common-dnd-system.md`, without cache control; -2. `common-dnd-references.md`, with ephemeral cache control; -3. `task.md`, without cache control; -4. `instructions.md`, without cache control; and -5. `common-dnd-transcript.md`, with ephemeral cache control. - -The scene chunker consumes the full transcript once and has no sibling lanes -that can share a transcript prefix. Stable task instructions therefore remain -before the changing full transcript. - -Update -`internal/modules/dnd/normalize/npcs/assets/prompts/dnd.npcs.normalize.yaml` to -use: - -1. `common-dnd-system.md`, without cache control; -2. `task.md`, without cache control; -3. `instructions.md`, with ephemeral cache control; -4. `candidates.md`, without cache control; and -5. `common-dnd-transcript.md`, with ephemeral cache control. - -Remove the normalizer system message's current cache control. Preserve the -candidate-before-evidence ordering: the transcript windows support evaluation -of the candidate collection and are not a cross-lane transcript prefix. - -Do not change input material construction, transcript windowing, prompt -fragments, module implementations, or response handling in either package. - -### Stage 2 completion criteria - -- Scene chunking renders references separately before the task and the full - transcript exactly once at the end. -- NPC normalization renders candidates and transcript windows in separate - messages, in that order, with the agreed cache boundaries. -- Both prompts retain their current identities, schemas, inputs, and runtime - behavior. - -## Stage 3: Consolidate Prompt-Ordering Verification - -Read and follow `docs/policy/testing.md` before modifying tests. The meaningful -risk is loss of the shared rendered prefix or incorrect placement of -request-specific material, not edits to prompt prose. - -Add one family-level extraction invariant test in -`internal/modules/dnd/register/prompt_cache_test.go` (or an equivalently scoped -new register-package test file): - -- register the complete D&D prompt asset family in a real in-memory - `llm.AssetRegistry`; -- construct a Scriptorium engine with a local, non-networked test profile; -- prepare all six extraction prompts using their exported prompt IDs and - versions; -- supply the same unique transcript, players, party, and glossary sentinels to - every prompt, plus only the lane-specific required inputs such as the NPC - registry and spell catalog; -- locate the one rendered message containing the transcript sentinel rather - than locating it through prompt prose or a hard-coded phrase; -- assert that the transcript sentinel occurs exactly once in each prepared - prompt; -- compare the rendered messages from the start through the transcript message - across all six prompts, including role, full rendered content, and complete - cache-control metadata; -- assert that the shared prefix contains exactly the four agreed messages and - that at least one lane-specific message follows it in every extraction - prompt; and -- assert that NPC-registry and spell-catalog sentinels, where supplied, occur - only after the transcript. - -Use this family-level test as the single owner of the cross-lane prefix policy. -Do not duplicate the same prefix assertion in every extractor package. - -Revise the existing prompt-asset tests in: - -- `internal/modules/dnd/extract/npcs/scriptorium_assets_test.go` -- `internal/modules/dnd/extract/itemevents/scriptorium_assets_test.go` -- `internal/modules/dnd/extract/scenedescriptions/scriptorium_assets_test.go` -- `internal/modules/dnd/extract/combatturns/scriptorium_assets_test.go` -- `internal/modules/dnd/extract/npcinteractions/scriptorium_assets_test.go` -- `internal/modules/dnd/extract/spells/scriptorium_assets_test.go` -- `internal/modules/dnd/chunk/scenes/scriptorium_assets_test.go` -- `internal/modules/dnd/normalize/npcs/scriptorium_assets_test.go` - -Remove transcript-last expectations that contradict the new extraction policy. -Keep package-local coverage only where it protects package-owned behavior, such -as prompt/schema registration, declared shared assets, isolation and placement -of package-specific input material, cache boundaries unique to the scene -chunker or NPC normalizer, and non-disclosure properties. Consolidate or remove -positional and prose-marker checks already covered by the family-level -invariant. - -Tests may search for unique input sentinels created by the test. They must not -require the presence of specific words, phrases, sentences, or other prose in a -prompt asset. Do not introduce golden snapshots of complete prompts or a test -that treats harmless wording changes as failures. - -### Stage 3 completion criteria - -- One durable family-level test fails if any extraction lane moves - lane-specific material ahead of the transcript or changes the common - rendered prefix. -- Local asset tests protect only nonredundant package behavior and the two - purpose-specific prompt sequences. -- The test suite contains no prompt-language change detector. -- All prompt tests remain deterministic, offline, and independent of - credentials. - -## Stage 4: Update Canonical Internal Documentation - -Update `docs/internal/llm.md` as the generic owner of prompt-cache mechanics: - -- explain that backend reuse depends on identical preceding roles, rendered - bytes, and cache metadata, not merely equivalent semantics; -- state the general sibling-prompt ordering rule: universal shared context, - request source material, then module-specific suffixes; -- explain that a cache boundary should be placed at a useful reusable prefix - and that redundant intermediate boundaries add no value; -- retain the rule that prompt-family owners may choose a different sequence - when their inputs and reuse pattern differ; and -- replace the current statement that D&D extraction puts transcripts last with - a link to the D&D-specific policy. - -Update `docs/internal/dnd.md` as the concrete owner of maintained D&D prompt -composition: - -- document the four-message extraction prefix and its cache boundaries; -- state that extraction-evidence policy, generated NPC registries, catalogs, - tasks, and instructions follow the transcript because they are not universal - across all extraction lanes; -- summarize the scene-chunking and NPC-normalization exceptions and their - rationale; and -- retain existing rules about shared-asset reuse, canonical reference input, - prompt fingerprints, and prompt behavior. - -Do not update the README, configuration reference, architecture policy, CLI -documentation, or integration contracts. Prompt message order is an internal -runtime policy and has no user-visible configuration or wire-contract change. - -### Stage 4 completion criteria - -- No current-behavior document still says that D&D extraction transcripts are - final messages. -- Generic cache mechanics and D&D-specific ordering each have one canonical - owner. -- Documentation describes implemented behavior once the prompt changes land - and does not duplicate volatile manifest inventories unnecessarily. - -## Stage 5: Verification - -Inspect representative prepared prompts to confirm that the test inputs produce -the intended shared prefix and lane-specific suffixes. Then run: - -```sh -git diff --check -go test ./internal/modules/dnd/chunk/scenes -go test ./internal/modules/dnd/extract/... -go test ./internal/modules/dnd/normalize/npcs -go test ./internal/modules/dnd/register -go test ./internal/modules/dnd/... -go test ./internal/modules/integration/... -go test ./... -go vet ./... -go build ./cmd/notarius -``` - -The focused commands should be used while implementing their respective -stages; the repository-wide commands are the final acceptance gate. - -## Expected Compatibility Effects - -- Reordered prompt content changes prompt fingerprints and may intentionally - cause one-time cold misses for affected development checkpoints. -- Prompt IDs, prompt versions, schema identities, checkpoint format, durable - artifacts, and public behavior do not change. -- Backend cache reuse remains provider-dependent, but Notarius supplies a - longer common extraction prefix that includes the transcript chunk. - -## Open Questions - -None. The message sequences, cache boundaries, test ownership, documentation -owners, and compatibility policy are specified above.