Document D&D prompt conventions and verification
This commit is contained in:
@@ -43,59 +43,63 @@ the contracts above define durable data.
|
|||||||
|
|
||||||
## Prompt Construction
|
## Prompt Construction
|
||||||
|
|
||||||
D&D LLM-facing content lives beneath `assets/dnd/`. New extractor content uses
|
D&D LLM-facing content lives beneath `assets/dnd/`. Each module contributes a
|
||||||
its feature subtree; when a family has both extraction and normalization
|
local `prompt.yaml` declaration and `instructions.md`; input-specific files
|
||||||
content, keep those in its `extract` and `normalize` subtrees. The owning module
|
such as a catalog, registry, grounding projection, or candidate collection are
|
||||||
still defines the ordered manifest and registers the resulting scoped filesystem.
|
local only when that module needs them. New extractor content uses its feature
|
||||||
Shared fragments belong to the D&D shared implementation and are selected by
|
subtree, while families with both extraction and normalization content use their
|
||||||
name, never copied into individual module subtrees.
|
`extract` and `normalize` subtrees. Shared visual-provenance fragments use
|
||||||
|
the `common-dnd-` prefix.
|
||||||
|
|
||||||
D&D extractors assemble prompts from an ordered manifest of shared and
|
The owning module’s manifest is the source of truth for which local and shared
|
||||||
module-selected assets. The location extractor and occurrence extractor reuse
|
assets are selected, their mount paths, their message order, cache controls,
|
||||||
the shared D&D system, evidence, identity, reference, and transcript assets
|
and the files included in its prompt fingerprint. Shared fragments belong to
|
||||||
instead of copying their text into individual modules. A manifest’s declared
|
the D&D shared implementation and are selected by name rather than copied into
|
||||||
sequence, including cache-control placement, is part of the prompt behavior.
|
module directories. The root `assets` package is a content-only boundary; its
|
||||||
|
physical ownership and rationale are defined by
|
||||||
|
[ADR-0011](../adr/0011-centralize-llm-assets.md).
|
||||||
|
|
||||||
|
Put each rule at its narrowest owner:
|
||||||
|
|
||||||
|
- universal behavior belongs in the shared system asset;
|
||||||
|
- D&D-family behavior belongs in a selected `common-dnd-` asset;
|
||||||
|
- rules for an input projection belong with that input asset;
|
||||||
|
- lane-specific policy belongs in the module’s `instructions.md`; and
|
||||||
|
- transport-envelope shape belongs in the private response schema.
|
||||||
|
|
||||||
|
A rule is eligible for the system prompt only when every D&D LLM prompt needs
|
||||||
|
it regardless of lane, inputs, or response shape. Module instructions must not
|
||||||
|
repeat rules selected from shared assets or schemas. Reintroduce such repetition
|
||||||
|
only after observational evaluation with representative transcripts shows that
|
||||||
|
it improves results at the intended target models and cost; structural prompt
|
||||||
|
tests alone are not that evidence.
|
||||||
|
|
||||||
Every maintained D&D LLM prompt selects `dnd-extraction` as its default
|
Every maintained D&D LLM prompt selects `dnd-extraction` as its default
|
||||||
profile. The D&D registrar registers that fallback profile with the maintained
|
profile. The D&D registrar registers the fallback, while an operator can
|
||||||
OpenRouter model, timeout, and service-tier policy. An operator may provide a
|
replace it with a complete profile of the same ID from the configured PromptKit
|
||||||
complete profile with the same ID through the configured PromptKit source; that
|
source. Deployment profile selection is documented in
|
||||||
definition replaces the fallback rather than merging with it. The fallback
|
|
||||||
leaves reasoning and optional sampling controls unspecified. Deployment profile
|
|
||||||
selection and the maintained operator example are documented in
|
|
||||||
[Configuration](../config.md#promptkit-profiles).
|
[Configuration](../config.md#promptkit-profiles).
|
||||||
|
|
||||||
All extraction prompts share this four-message rendered prefix: the system
|
The transcript assets have distinct consumers. Scene chunking consumes the
|
||||||
message without cache control, the identity message without cache control, the
|
complete-session `common-dnd-transcript-full.md`; extraction prompts consume
|
||||||
campaign-reference message with ephemeral cache control, and the chunk
|
the current-chunk `common-dnd-transcript-chunk.md`; and NPC and location
|
||||||
transcript message with ephemeral cache control. This gives equivalent
|
normalization consume `common-dnd-transcript-windows.md` alongside their
|
||||||
extraction requests the same reusable prefix through their source material.
|
candidate collections. Player, party, glossary, and compatible campaign
|
||||||
|
references provide disambiguating context, not evidence. Reference material is
|
||||||
|
canonically ordered before rendering so equivalent inputs remain stable.
|
||||||
|
|
||||||
Extraction-evidence policy, generated NPC registries, spell catalogs, module
|
Extraction prompts render the common system and identity messages first, then
|
||||||
tasks, and instructions follow the transcript because they are not universal
|
cached campaign references and the cached chunk transcript. Evidence policy and
|
||||||
across all extraction lanes. The final instructions message carries ephemeral
|
any lane-specific registry, catalog, or grounding projection follow that
|
||||||
cache control; evidence, registry, catalog, and task messages do not. Preserve
|
prefix. The final module instructions message is ephemeral. This keeps the
|
||||||
this division when changing an extractor or its assets so prompt-cache behavior
|
reusable extraction prefix identical while preserving the lane-specific suffix.
|
||||||
remains stable.
|
|
||||||
|
|
||||||
The other D&D LLM prompts intentionally follow different patterns. Scene
|
Scene chunking intentionally uses a different order: system, cached campaign
|
||||||
chunking has no sibling extraction lane with which to share its full transcript,
|
references, uncached module instructions, then the final ephemeral full
|
||||||
so it renders campaign references before its task and instructions, then places
|
transcript. Entity normalization also has its own order: system, uncached
|
||||||
the cacheable full transcript last. NPC and location normalization share the
|
module instructions, ephemeral reconciliation policy, uncached candidates, and
|
||||||
entity-reconciliation response schema and safety boundary while retaining their
|
final ephemeral transcript windows. These orders and cache controls are prompt
|
||||||
own task and identity rules. NPC normalization keeps its task and
|
behavior; change them only through the owning manifest and prompt declaration.
|
||||||
cacheable instructions before the candidate collection, followed by the
|
|
||||||
cacheable transcript windows: candidates must be available before their
|
|
||||||
supporting evidence is evaluated, and those windows are not a cross-lane
|
|
||||||
prefix. Mounted assets and their declared message order determine the prompt
|
|
||||||
fingerprint, so intentional prompt edits continue to invalidate stale
|
|
||||||
checkpoints.
|
|
||||||
|
|
||||||
All extractors use the shared prompt-input preparation rules. The current chunk
|
|
||||||
is copied into transcript material; player, party, glossary, and compatible
|
|
||||||
campaign references are context for disambiguation, not source evidence.
|
|
||||||
Reference prompt material is canonically ordered before it is rendered, which
|
|
||||||
keeps equivalent inputs stable across runs.
|
|
||||||
|
|
||||||
## Evidence, Candidates, And Normalization
|
## Evidence, Candidates, And Normalization
|
||||||
|
|
||||||
|
|||||||
@@ -185,13 +185,11 @@ func TestAssetRegistryFallbackProfileDigestTracksContentWithoutLeakingIt(t *test
|
|||||||
func TestAssetRegistryCombinesNamespacedPromptSources(t *testing.T) {
|
func TestAssetRegistryCombinesNamespacedPromptSources(t *testing.T) {
|
||||||
registry := NewAssetRegistry()
|
registry := NewAssetRegistry()
|
||||||
mustRegisterPromptFS(t, registry, fstest.MapFS{
|
mustRegisterPromptFS(t, registry, fstest.MapFS{
|
||||||
"dnd.spells/dnd.spells.yaml": {Data: []byte(validPromptYAML("schema.json"))},
|
"dnd.spells/prompt.yaml": {Data: []byte(validPromptYAML("schema.json"))},
|
||||||
"dnd.spells/task.md": {Data: []byte("spell task")},
|
|
||||||
"dnd.spells/instructions.md": {Data: []byte("spell instructions")},
|
"dnd.spells/instructions.md": {Data: []byte("spell instructions")},
|
||||||
}, ".")
|
}, ".")
|
||||||
mustRegisterPromptFS(t, registry, fstest.MapFS{
|
mustRegisterPromptFS(t, registry, fstest.MapFS{
|
||||||
"dnd.scenes/dnd.scenes.yaml": {Data: []byte(validPromptYAML("schema.json"))},
|
"dnd.scenes/prompt.yaml": {Data: []byte(validPromptYAML("schema.json"))},
|
||||||
"dnd.scenes/task.md": {Data: []byte("scene task")},
|
|
||||||
"dnd.scenes/instructions.md": {Data: []byte("scene instructions")},
|
"dnd.scenes/instructions.md": {Data: []byte("scene instructions")},
|
||||||
}, ".")
|
}, ".")
|
||||||
|
|
||||||
@@ -200,11 +198,9 @@ func TestAssetRegistryCombinesNamespacedPromptSources(t *testing.T) {
|
|||||||
t.Fatalf("PromptFS() error = %v, want nil", err)
|
t.Fatalf("PromptFS() error = %v, want nil", err)
|
||||||
}
|
}
|
||||||
for _, name := range []string{
|
for _, name := range []string{
|
||||||
"dnd.spells/dnd.spells.yaml",
|
"dnd.spells/prompt.yaml",
|
||||||
"dnd.spells/task.md",
|
|
||||||
"dnd.spells/instructions.md",
|
"dnd.spells/instructions.md",
|
||||||
"dnd.scenes/dnd.scenes.yaml",
|
"dnd.scenes/prompt.yaml",
|
||||||
"dnd.scenes/task.md",
|
|
||||||
"dnd.scenes/instructions.md",
|
"dnd.scenes/instructions.md",
|
||||||
} {
|
} {
|
||||||
if _, err := fsys.Open(name); err != nil {
|
if _, err := fsys.Open(name); err != nil {
|
||||||
|
|||||||
@@ -14,11 +14,11 @@ func TestModulePromptFSCombinesModuleAndSharedPrompts(t *testing.T) {
|
|||||||
"nested/transcript.md": {Data: []byte("transcript")},
|
"nested/transcript.md": {Data: []byte("transcript")},
|
||||||
}
|
}
|
||||||
fsys, err := ModulePromptFS("module.test", fstest.MapFS{
|
fsys, err := ModulePromptFS("module.test", fstest.MapFS{
|
||||||
"assets/prompts/module.test.yaml": {Data: []byte("id: module.test")},
|
"assets/prompts/prompt.yaml": {Data: []byte("id: module.test")},
|
||||||
"assets/prompts/task.md": {Data: []byte("task")},
|
"assets/prompts/instructions.md": {Data: []byte("instructions")},
|
||||||
}, []ModulePromptFile{
|
}, []ModulePromptFile{
|
||||||
{Name: "module.test.yaml", Path: "assets/prompts/module.test.yaml"},
|
{Name: "prompt.yaml", Path: "assets/prompts/prompt.yaml"},
|
||||||
{Name: "task.md", Path: "assets/prompts/task.md"},
|
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||||
},
|
},
|
||||||
SharedPromptFile{Name: "system.md", FS: sharedFS, Path: "shared/system.md"},
|
SharedPromptFile{Name: "system.md", FS: sharedFS, Path: "shared/system.md"},
|
||||||
SharedPromptFile{Name: "reference.md", FS: sharedFS, Path: "shared/reference.md"},
|
SharedPromptFile{Name: "reference.md", FS: sharedFS, Path: "shared/reference.md"},
|
||||||
@@ -29,8 +29,8 @@ func TestModulePromptFSCombinesModuleAndSharedPrompts(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
tests := map[string]string{
|
tests := map[string]string{
|
||||||
"assets/prompts/module.test/module.test.yaml": "id: module.test",
|
"assets/prompts/module.test/prompt.yaml": "id: module.test",
|
||||||
"assets/prompts/module.test/task.md": "task",
|
"assets/prompts/module.test/instructions.md": "instructions",
|
||||||
"assets/prompts/module.test/sharedassets/system.md": "system",
|
"assets/prompts/module.test/sharedassets/system.md": "system",
|
||||||
"assets/prompts/module.test/sharedassets/reference.md": "reference",
|
"assets/prompts/module.test/sharedassets/reference.md": "reference",
|
||||||
"assets/prompts/module.test/sharedassets/transcript.md": "transcript",
|
"assets/prompts/module.test/sharedassets/transcript.md": "transcript",
|
||||||
@@ -63,18 +63,18 @@ func TestModulePromptFSCombinesModuleAndSharedPrompts(t *testing.T) {
|
|||||||
|
|
||||||
func TestModulePromptFSRejectsMissingModuleFile(t *testing.T) {
|
func TestModulePromptFSRejectsMissingModuleFile(t *testing.T) {
|
||||||
_, err := ModulePromptFS("module.test", fstest.MapFS{}, []ModulePromptFile{
|
_, err := ModulePromptFS("module.test", fstest.MapFS{}, []ModulePromptFile{
|
||||||
{Name: "task.md", Path: "assets/prompts/task.md"},
|
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||||
})
|
})
|
||||||
if err == nil || !strings.Contains(err.Error(), "read module prompt asset assets/prompts/task.md") {
|
if err == nil || !strings.Contains(err.Error(), "read module prompt asset assets/prompts/instructions.md") {
|
||||||
t.Fatalf("ModulePromptFS() error = %v, want missing module asset context", err)
|
t.Fatalf("ModulePromptFS() error = %v, want missing module asset context", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestModulePromptFSRejectsMissingSharedFile(t *testing.T) {
|
func TestModulePromptFSRejectsMissingSharedFile(t *testing.T) {
|
||||||
_, err := ModulePromptFS("module.test", fstest.MapFS{
|
_, err := ModulePromptFS("module.test", fstest.MapFS{
|
||||||
"assets/prompts/task.md": {Data: []byte("task")},
|
"assets/prompts/instructions.md": {Data: []byte("instructions")},
|
||||||
}, []ModulePromptFile{
|
}, []ModulePromptFile{
|
||||||
{Name: "task.md", Path: "assets/prompts/task.md"},
|
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||||
}, SharedPromptFile{Name: "system.md", FS: fstest.MapFS{}, Path: "shared/system.md"})
|
}, SharedPromptFile{Name: "system.md", FS: fstest.MapFS{}, Path: "shared/system.md"})
|
||||||
if err == nil || !strings.Contains(err.Error(), "read shared prompt asset shared/system.md") {
|
if err == nil || !strings.Contains(err.Error(), "read shared prompt asset shared/system.md") {
|
||||||
t.Fatalf("ModulePromptFS() error = %v, want missing shared asset context", err)
|
t.Fatalf("ModulePromptFS() error = %v, want missing shared asset context", err)
|
||||||
@@ -83,9 +83,9 @@ func TestModulePromptFSRejectsMissingSharedFile(t *testing.T) {
|
|||||||
|
|
||||||
func TestModulePromptFSRejectsNestedVirtualFileName(t *testing.T) {
|
func TestModulePromptFSRejectsNestedVirtualFileName(t *testing.T) {
|
||||||
_, err := ModulePromptFS("module.test", fstest.MapFS{
|
_, err := ModulePromptFS("module.test", fstest.MapFS{
|
||||||
"assets/prompts/task.md": {Data: []byte("task")},
|
"assets/prompts/instructions.md": {Data: []byte("instructions")},
|
||||||
}, []ModulePromptFile{
|
}, []ModulePromptFile{
|
||||||
{Name: "nested/task.md", Path: "assets/prompts/task.md"},
|
{Name: "nested/instructions.md", Path: "assets/prompts/instructions.md"},
|
||||||
})
|
})
|
||||||
if err == nil || !strings.Contains(err.Error(), "must not contain path separators") {
|
if err == nil || !strings.Contains(err.Error(), "must not contain path separators") {
|
||||||
t.Fatalf("ModulePromptFS() error = %v, want nested file name error", err)
|
t.Fatalf("ModulePromptFS() error = %v, want nested file name error", err)
|
||||||
@@ -94,9 +94,9 @@ func TestModulePromptFSRejectsNestedVirtualFileName(t *testing.T) {
|
|||||||
|
|
||||||
func TestModulePromptFSRejectsNestedSharedFileName(t *testing.T) {
|
func TestModulePromptFSRejectsNestedSharedFileName(t *testing.T) {
|
||||||
_, err := ModulePromptFS("module.test", fstest.MapFS{
|
_, err := ModulePromptFS("module.test", fstest.MapFS{
|
||||||
"assets/prompts/task.md": {Data: []byte("task")},
|
"assets/prompts/instructions.md": {Data: []byte("instructions")},
|
||||||
}, []ModulePromptFile{
|
}, []ModulePromptFile{
|
||||||
{Name: "task.md", Path: "assets/prompts/task.md"},
|
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||||
}, SharedPromptFile{Name: "nested/system.md", FS: fstest.MapFS{}, Path: "shared/system.md"})
|
}, SharedPromptFile{Name: "nested/system.md", FS: fstest.MapFS{}, Path: "shared/system.md"})
|
||||||
if err == nil || !strings.Contains(err.Error(), "shared prompt file name") || !strings.Contains(err.Error(), "must not contain path separators") {
|
if err == nil || !strings.Contains(err.Error(), "shared prompt file name") || !strings.Contains(err.Error(), "must not contain path separators") {
|
||||||
t.Fatalf("ModulePromptFS() error = %v, want nested shared file name error", err)
|
t.Fatalf("ModulePromptFS() error = %v, want nested shared file name error", err)
|
||||||
|
|||||||
@@ -25,9 +25,6 @@ func TestPromptAssetsPrepareTranscriptWithInstructionOrdering(t *testing.T) {
|
|||||||
if prepared.OutputContract.SchemaPath != "dnd_scenes_llm.v1.json" {
|
if prepared.OutputContract.SchemaPath != "dnd_scenes_llm.v1.json" {
|
||||||
t.Fatalf("scene output schema path = %q, want private LLM schema", prepared.OutputContract.SchemaPath)
|
t.Fatalf("scene output schema path = %q, want private LLM schema", prepared.OutputContract.SchemaPath)
|
||||||
}
|
}
|
||||||
if len(prepared.Messages) < 4 {
|
|
||||||
t.Fatalf("prepared messages = %#v, want system, references, instructions, and transcript", prepared.Messages)
|
|
||||||
}
|
|
||||||
if prepared.Messages[0].Role != "system" || prepared.Messages[1].Role != "user" {
|
if prepared.Messages[0].Role != "system" || prepared.Messages[1].Role != "user" {
|
||||||
t.Fatalf("initial prompt roles = %#v, want system followed by campaign references", prepared.Messages[:2])
|
t.Fatalf("initial prompt roles = %#v, want system followed by campaign references", prepared.Messages[:2])
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -62,9 +62,6 @@ func TestPromptAssetsPrepareRequiredInputs(t *testing.T) {
|
|||||||
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_combat_turns_llm.v1.json" {
|
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_combat_turns_llm.v1.json" {
|
||||||
t.Fatalf("prepared prompt = %#v, want combat prompt identity and schema", prepared)
|
t.Fatalf("prepared prompt = %#v, want combat prompt identity and schema", prepared)
|
||||||
}
|
}
|
||||||
if len(prepared.Messages) < 6 {
|
|
||||||
t.Fatalf("prepared messages = %#v, want shared context, NPC grounding, and instructions", prepared.Messages)
|
|
||||||
}
|
|
||||||
referenceIndex := -1
|
referenceIndex := -1
|
||||||
transcriptIndex := -1
|
transcriptIndex := -1
|
||||||
npcIndex := -1
|
npcIndex := -1
|
||||||
|
|||||||
@@ -41,7 +41,7 @@ func TestRegisterPromptAssetsAndPrepareEnemyEventPrompt(t *testing.T) {
|
|||||||
groundingIndices = append(groundingIndices, index)
|
groundingIndices = append(groundingIndices, index)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
instructionIndex := renderedMessageIndex(t, prepared.Messages, "Extract Dungeons & Dragons enemy events")
|
instructionIndex := len(prepared.Messages) - 1
|
||||||
for _, index := range groundingIndices {
|
for _, index := range groundingIndices {
|
||||||
if instructionIndex <= index {
|
if instructionIndex <= index {
|
||||||
t.Fatalf("instruction index = %d, want after grounding index %d", instructionIndex, index)
|
t.Fatalf("instruction index = %d, want after grounding index %d", instructionIndex, index)
|
||||||
|
|||||||
@@ -3,7 +3,6 @@ package itemevents
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"reflect"
|
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
@@ -43,9 +42,6 @@ func TestPromptAssetsPrepareItemEventPrompt(t *testing.T) {
|
|||||||
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_item_events_llm.v1.json" {
|
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_item_events_llm.v1.json" {
|
||||||
t.Fatalf("prepared prompt = %#v", prepared)
|
t.Fatalf("prepared prompt = %#v", prepared)
|
||||||
}
|
}
|
||||||
if len(prepared.Messages) < 5 {
|
|
||||||
t.Fatalf("prepared messages = %#v, want shared policy, references, transcript, and instructions", prepared.Messages)
|
|
||||||
}
|
|
||||||
referenceIndex := -1
|
referenceIndex := -1
|
||||||
transcriptIndex := -1
|
transcriptIndex := -1
|
||||||
for index, message := range prepared.Messages {
|
for index, message := range prepared.Messages {
|
||||||
@@ -91,16 +87,3 @@ func TestPromptAssetsDoNotLeakIntoMetadata(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPromptManifestReusesOnlySharedAssets(t *testing.T) {
|
|
||||||
want := []string{
|
|
||||||
"common-dnd-system.md",
|
|
||||||
"common-dnd-extraction-evidence.md",
|
|
||||||
"common-dnd-identity.md",
|
|
||||||
"common-dnd-references.md",
|
|
||||||
"common-dnd-transcript-chunk.md",
|
|
||||||
}
|
|
||||||
if !reflect.DeepEqual(promptAssetManifest.SharedFiles, want) {
|
|
||||||
t.Fatalf("shared assets = %#v, want %#v", promptAssetManifest.SharedFiles, want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ package locations
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"slices"
|
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
@@ -12,11 +11,6 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
func TestRegisterPromptAssetsPreparesLocationPrompt(t *testing.T) {
|
func TestRegisterPromptAssetsPreparesLocationPrompt(t *testing.T) {
|
||||||
for _, name := range []string{"common-dnd-system.md", "common-dnd-identity.md", "common-dnd-references.md", "common-dnd-transcript-chunk.md", "common-dnd-extraction-evidence.md"} {
|
|
||||||
if !slices.Contains(promptAssetManifest.SharedFiles, name) {
|
|
||||||
t.Fatalf("shared prompt assets = %#v, missing %q", promptAssetManifest.SharedFiles, name)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
registry := llm.NewAssetRegistry()
|
registry := llm.NewAssetRegistry()
|
||||||
if err := RegisterPromptAssets(registry); err != nil {
|
if err := RegisterPromptAssets(registry); err != nil {
|
||||||
t.Fatalf("RegisterPromptAssets() error = %v", err)
|
t.Fatalf("RegisterPromptAssets() error = %v", err)
|
||||||
@@ -39,9 +33,6 @@ func TestRegisterPromptAssetsPreparesLocationPrompt(t *testing.T) {
|
|||||||
if prepared.OutputContract.SchemaPath != "dnd_locations_llm.v1.json" {
|
if prepared.OutputContract.SchemaPath != "dnd_locations_llm.v1.json" {
|
||||||
t.Fatalf("output contract = %#v", prepared.OutputContract)
|
t.Fatalf("output contract = %#v", prepared.OutputContract)
|
||||||
}
|
}
|
||||||
if len(prepared.Messages) < 5 {
|
|
||||||
t.Fatalf("prepared messages = %#v, want shared policy, references, transcript, and instructions", prepared.Messages)
|
|
||||||
}
|
|
||||||
referenceIndex := -1
|
referenceIndex := -1
|
||||||
transcriptIndex := -1
|
transcriptIndex := -1
|
||||||
for index, message := range prepared.Messages {
|
for index, message := range prepared.Messages {
|
||||||
|
|||||||
@@ -51,9 +51,6 @@ func TestRegisterPromptAssetsAndPrepareInteractionPrompt(t *testing.T) {
|
|||||||
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_npc_interactions_llm.v1.json" {
|
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_npc_interactions_llm.v1.json" {
|
||||||
t.Fatalf("prepared prompt = %#v", prepared)
|
t.Fatalf("prepared prompt = %#v", prepared)
|
||||||
}
|
}
|
||||||
if len(prepared.Messages) < 6 {
|
|
||||||
t.Fatalf("prepared messages = %#v, want shared context, NPC grounding, and instructions", prepared.Messages)
|
|
||||||
}
|
|
||||||
referenceIndex := -1
|
referenceIndex := -1
|
||||||
transcriptIndex := -1
|
transcriptIndex := -1
|
||||||
npcIndex := -1
|
npcIndex := -1
|
||||||
|
|||||||
@@ -42,9 +42,6 @@ func TestRegisterPromptAssetsAndPrepareNPCPrompt(t *testing.T) {
|
|||||||
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_npcs_llm.v1.json" {
|
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_npcs_llm.v1.json" {
|
||||||
t.Fatalf("prepared prompt = %#v, want NPC prompt identity and wiring", prepared)
|
t.Fatalf("prepared prompt = %#v, want NPC prompt identity and wiring", prepared)
|
||||||
}
|
}
|
||||||
if len(prepared.Messages) < 5 {
|
|
||||||
t.Fatalf("prepared messages = %#v, want shared policy, references, transcript, and instructions", prepared.Messages)
|
|
||||||
}
|
|
||||||
referenceIndex := -1
|
referenceIndex := -1
|
||||||
transcriptIndex := -1
|
transcriptIndex := -1
|
||||||
for index, message := range prepared.Messages {
|
for index, message := range prepared.Messages {
|
||||||
|
|||||||
@@ -42,9 +42,6 @@ func TestRegisterPromptAssetsPreparesSceneDescriptionPrompt(t *testing.T) {
|
|||||||
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_scene_descriptions_llm.v1.json" {
|
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_scene_descriptions_llm.v1.json" {
|
||||||
t.Fatalf("prepared prompt = %#v, want scene-description prompt identity and schema wiring", prepared)
|
t.Fatalf("prepared prompt = %#v, want scene-description prompt identity and schema wiring", prepared)
|
||||||
}
|
}
|
||||||
if len(prepared.Messages) < 5 {
|
|
||||||
t.Fatalf("prepared messages = %#v, want shared policy, references, transcript, and instructions", prepared.Messages)
|
|
||||||
}
|
|
||||||
referenceIndex := -1
|
referenceIndex := -1
|
||||||
transcriptIndex := -1
|
transcriptIndex := -1
|
||||||
for index, message := range prepared.Messages {
|
for index, message := range prepared.Messages {
|
||||||
|
|||||||
@@ -21,9 +21,6 @@ func TestPromptAssetsPrepareSpellPrompt(t *testing.T) {
|
|||||||
if prepared.OutputContract.SchemaPath != "dnd_spells_llm.v1.json" {
|
if prepared.OutputContract.SchemaPath != "dnd_spells_llm.v1.json" {
|
||||||
t.Fatalf("schema path = %q, want LLM-only schema", prepared.OutputContract.SchemaPath)
|
t.Fatalf("schema path = %q, want LLM-only schema", prepared.OutputContract.SchemaPath)
|
||||||
}
|
}
|
||||||
if len(prepared.Messages) < 7 {
|
|
||||||
t.Fatalf("prepared messages = %#v, want shared context, grounded inputs, and instructions", prepared.Messages)
|
|
||||||
}
|
|
||||||
indices := map[string]int{
|
indices := map[string]int{
|
||||||
"spell-player": -1,
|
"spell-player": -1,
|
||||||
"spell-transcript": -1,
|
"spell-transcript": -1,
|
||||||
|
|||||||
@@ -12,18 +12,6 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
func TestRegisterPromptAssetsPreparesLocationNormalizationPrompt(t *testing.T) {
|
func TestRegisterPromptAssetsPreparesLocationNormalizationPrompt(t *testing.T) {
|
||||||
hasInstructions := false
|
|
||||||
for _, file := range promptAssetManifest.ModuleFiles {
|
|
||||||
if file.Name == "task.md" {
|
|
||||||
t.Fatalf("module prompt files = %#v, want no retired task asset", promptAssetManifest.ModuleFiles)
|
|
||||||
}
|
|
||||||
if file.Name == "instructions.md" {
|
|
||||||
hasInstructions = true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if !hasInstructions {
|
|
||||||
t.Fatalf("module prompt files = %#v, want instructions asset", promptAssetManifest.ModuleFiles)
|
|
||||||
}
|
|
||||||
registry := llm.NewAssetRegistry()
|
registry := llm.NewAssetRegistry()
|
||||||
if err := entityreconcile.RegisterSchemaAssets(registry); err != nil {
|
if err := entityreconcile.RegisterSchemaAssets(registry); err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
@@ -44,7 +32,7 @@ func TestRegisterPromptAssetsPreparesLocationNormalizationPrompt(t *testing.T) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
if prepared.OutputContract.SchemaPath != "dnd_entity_reconcile_llm.v1.json" || len(prepared.Messages) < 5 {
|
if prepared.OutputContract.SchemaPath != "dnd_entity_reconcile_llm.v1.json" {
|
||||||
t.Fatalf("prepared prompt = %#v", prepared)
|
t.Fatalf("prepared prompt = %#v", prepared)
|
||||||
}
|
}
|
||||||
for _, index := range []int{2, 4} {
|
for _, index := range []int{2, 4} {
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ package npcs
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"reflect"
|
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
@@ -13,21 +12,6 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
func TestRegisterPromptAssetsPreparesNormalizationPrompt(t *testing.T) {
|
func TestRegisterPromptAssetsPreparesNormalizationPrompt(t *testing.T) {
|
||||||
if want := []string{"common-dnd-system.md", "common-dnd-entity-reconciliation.md", "common-dnd-transcript-windows.md"}; !reflect.DeepEqual(promptAssetManifest.SharedFiles, want) {
|
|
||||||
t.Fatalf("shared prompt assets = %#v, want %#v", promptAssetManifest.SharedFiles, want)
|
|
||||||
}
|
|
||||||
hasInstructions := false
|
|
||||||
for _, file := range promptAssetManifest.ModuleFiles {
|
|
||||||
if file.Name == "task.md" {
|
|
||||||
t.Fatalf("module prompt files = %#v, want no retired task asset", promptAssetManifest.ModuleFiles)
|
|
||||||
}
|
|
||||||
if file.Name == "instructions.md" {
|
|
||||||
hasInstructions = true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if !hasInstructions {
|
|
||||||
t.Fatalf("module prompt files = %#v, want instructions asset", promptAssetManifest.ModuleFiles)
|
|
||||||
}
|
|
||||||
if promptHash, err := promptAssetMetadata(); err != nil || promptHash == "" {
|
if promptHash, err := promptAssetMetadata(); err != nil || promptHash == "" {
|
||||||
t.Fatalf("promptAssetMetadata() = %q, %v; want prompt fingerprint", promptHash, err)
|
t.Fatalf("promptAssetMetadata() = %q, %v; want prompt fingerprint", promptHash, err)
|
||||||
}
|
}
|
||||||
@@ -62,9 +46,6 @@ func TestRegisterPromptAssetsPreparesNormalizationPrompt(t *testing.T) {
|
|||||||
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_entity_reconcile_llm.v1.json" {
|
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_entity_reconcile_llm.v1.json" {
|
||||||
t.Fatalf("prepared prompt = %#v, want normalization prompt identity and schema", prepared)
|
t.Fatalf("prepared prompt = %#v, want normalization prompt identity and schema", prepared)
|
||||||
}
|
}
|
||||||
if len(prepared.Messages) < 5 {
|
|
||||||
t.Fatalf("prepared messages = %#v, want system, instructions, reconciliation policy, candidates, and transcript windows", prepared.Messages)
|
|
||||||
}
|
|
||||||
if prepared.Messages[0].Role != "system" {
|
if prepared.Messages[0].Role != "system" {
|
||||||
t.Fatalf("initial message role = %q, want system", prepared.Messages[0].Role)
|
t.Fatalf("initial message role = %q, want system", prepared.Messages[0].Role)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -15,8 +15,8 @@ func TestPromptAssetManifestPromptFS(t *testing.T) {
|
|||||||
manifest := PromptAssetManifest{
|
manifest := PromptAssetManifest{
|
||||||
ModuleDir: "dnd.test",
|
ModuleDir: "dnd.test",
|
||||||
ModuleFiles: []promptfs.ModulePromptFile{
|
ModuleFiles: []promptfs.ModulePromptFile{
|
||||||
{Name: "dnd.test.yaml", Path: "assets/prompts/dnd.test.yaml"},
|
{Name: "prompt.yaml", Path: "assets/prompts/prompt.yaml"},
|
||||||
{Name: "task.md", Path: "assets/prompts/task.md"},
|
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||||
},
|
},
|
||||||
SharedFiles: []string{
|
SharedFiles: []string{
|
||||||
"common-dnd-system.md",
|
"common-dnd-system.md",
|
||||||
@@ -27,16 +27,16 @@ func TestPromptAssetManifestPromptFS(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fsys, err := manifest.PromptFS(fstest.MapFS{
|
fsys, err := manifest.PromptFS(fstest.MapFS{
|
||||||
"assets/prompts/dnd.test.yaml": {Data: []byte("id: dnd.test")},
|
"assets/prompts/prompt.yaml": {Data: []byte("id: dnd.test")},
|
||||||
"assets/prompts/task.md": {Data: []byte("task")},
|
"assets/prompts/instructions.md": {Data: []byte("instructions")},
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("PromptFS() error = %v, want nil", err)
|
t.Fatalf("PromptFS() error = %v, want nil", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantModule := map[string]string{
|
wantModule := map[string]string{
|
||||||
"assets/prompts/dnd.test/dnd.test.yaml": "id: dnd.test",
|
"assets/prompts/dnd.test/prompt.yaml": "id: dnd.test",
|
||||||
"assets/prompts/dnd.test/task.md": "task",
|
"assets/prompts/dnd.test/instructions.md": "instructions",
|
||||||
}
|
}
|
||||||
for path, wantContent := range wantModule {
|
for path, wantContent := range wantModule {
|
||||||
content, err := fs.ReadFile(fsys, path)
|
content, err := fs.ReadFile(fsys, path)
|
||||||
@@ -88,11 +88,11 @@ func TestPromptAssetManifestRejectsInvalidSharedNames(t *testing.T) {
|
|||||||
_, err := (PromptAssetManifest{
|
_, err := (PromptAssetManifest{
|
||||||
ModuleDir: "dnd.test",
|
ModuleDir: "dnd.test",
|
||||||
ModuleFiles: []promptfs.ModulePromptFile{
|
ModuleFiles: []promptfs.ModulePromptFile{
|
||||||
{Name: "dnd.test.yaml", Path: "assets/prompts/dnd.test.yaml"},
|
{Name: "prompt.yaml", Path: "assets/prompts/prompt.yaml"},
|
||||||
},
|
},
|
||||||
SharedFiles: test.sharedFiles,
|
SharedFiles: test.sharedFiles,
|
||||||
}).PromptFS(fstest.MapFS{
|
}).PromptFS(fstest.MapFS{
|
||||||
"assets/prompts/dnd.test.yaml": {Data: []byte("id: dnd.test")},
|
"assets/prompts/prompt.yaml": {Data: []byte("id: dnd.test")},
|
||||||
})
|
})
|
||||||
if err == nil || !strings.Contains(err.Error(), test.wantError) {
|
if err == nil || !strings.Contains(err.Error(), test.wantError) {
|
||||||
t.Fatalf("PromptFS() error = %v, want %q", err, test.wantError)
|
t.Fatalf("PromptFS() error = %v, want %q", err, test.wantError)
|
||||||
@@ -105,10 +105,10 @@ func TestPromptAssetManifestRejectsMissingModuleFile(t *testing.T) {
|
|||||||
_, err := (PromptAssetManifest{
|
_, err := (PromptAssetManifest{
|
||||||
ModuleDir: "dnd.test",
|
ModuleDir: "dnd.test",
|
||||||
ModuleFiles: []promptfs.ModulePromptFile{
|
ModuleFiles: []promptfs.ModulePromptFile{
|
||||||
{Name: "task.md", Path: "assets/prompts/task.md"},
|
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||||
},
|
},
|
||||||
}).PromptFS(fstest.MapFS{})
|
}).PromptFS(fstest.MapFS{})
|
||||||
if err == nil || !strings.Contains(err.Error(), "read module prompt asset assets/prompts/task.md") {
|
if err == nil || !strings.Contains(err.Error(), "read module prompt asset assets/prompts/instructions.md") {
|
||||||
t.Fatalf("PromptFS() error = %v, want missing module asset context", err)
|
t.Fatalf("PromptFS() error = %v, want missing module asset context", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -129,11 +129,11 @@ func TestPromptAssetManifestRejectsMissingSharedFile(t *testing.T) {
|
|||||||
_, err := (PromptAssetManifest{
|
_, err := (PromptAssetManifest{
|
||||||
ModuleDir: "dnd.test",
|
ModuleDir: "dnd.test",
|
||||||
ModuleFiles: []promptfs.ModulePromptFile{
|
ModuleFiles: []promptfs.ModulePromptFile{
|
||||||
{Name: "dnd.test.yaml", Path: "assets/prompts/dnd.test.yaml"},
|
{Name: "prompt.yaml", Path: "assets/prompts/prompt.yaml"},
|
||||||
},
|
},
|
||||||
SharedFiles: []string{name},
|
SharedFiles: []string{name},
|
||||||
}).PromptFS(fstest.MapFS{
|
}).PromptFS(fstest.MapFS{
|
||||||
"assets/prompts/dnd.test.yaml": {Data: []byte("id: dnd.test")},
|
"assets/prompts/prompt.yaml": {Data: []byte("id: dnd.test")},
|
||||||
})
|
})
|
||||||
if err == nil || !strings.Contains(err.Error(), "read shared prompt asset "+path) {
|
if err == nil || !strings.Contains(err.Error(), "read shared prompt asset "+path) {
|
||||||
t.Fatalf("PromptFS() error = %v, want missing shared asset context", err)
|
t.Fatalf("PromptFS() error = %v, want missing shared asset context", err)
|
||||||
@@ -142,14 +142,14 @@ func TestPromptAssetManifestRejectsMissingSharedFile(t *testing.T) {
|
|||||||
|
|
||||||
func TestPromptAssetManifestHashMatchesManifestParts(t *testing.T) {
|
func TestPromptAssetManifestHashMatchesManifestParts(t *testing.T) {
|
||||||
moduleFS := fstest.MapFS{
|
moduleFS := fstest.MapFS{
|
||||||
"assets/prompts/dnd.test.yaml": {Data: []byte("id: dnd.test")},
|
"assets/prompts/prompt.yaml": {Data: []byte("id: dnd.test")},
|
||||||
"assets/prompts/task.md": {Data: []byte("task")},
|
"assets/prompts/instructions.md": {Data: []byte("instructions")},
|
||||||
}
|
}
|
||||||
manifest := PromptAssetManifest{
|
manifest := PromptAssetManifest{
|
||||||
ModuleDir: "dnd.test",
|
ModuleDir: "dnd.test",
|
||||||
ModuleFiles: []promptfs.ModulePromptFile{
|
ModuleFiles: []promptfs.ModulePromptFile{
|
||||||
{Name: "dnd.test.yaml", Path: "assets/prompts/dnd.test.yaml"},
|
{Name: "prompt.yaml", Path: "assets/prompts/prompt.yaml"},
|
||||||
{Name: "task.md", Path: "assets/prompts/task.md"},
|
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||||
},
|
},
|
||||||
SharedFiles: []string{
|
SharedFiles: []string{
|
||||||
"common-dnd-transcript-chunk.md",
|
"common-dnd-transcript-chunk.md",
|
||||||
@@ -166,8 +166,8 @@ func TestPromptAssetManifestHashMatchesManifestParts(t *testing.T) {
|
|||||||
t.Fatalf("sharedAssetFS() error = %v, want nil", err)
|
t.Fatalf("sharedAssetFS() error = %v, want nil", err)
|
||||||
}
|
}
|
||||||
want, err := llm.HashAssets([]llm.AssetHashPart{
|
want, err := llm.HashAssets([]llm.AssetHashPart{
|
||||||
{FS: moduleFS, Path: "assets/prompts/dnd.test.yaml"},
|
{FS: moduleFS, Path: "assets/prompts/prompt.yaml"},
|
||||||
{FS: moduleFS, Path: "assets/prompts/task.md"},
|
{FS: moduleFS, Path: "assets/prompts/instructions.md"},
|
||||||
{FS: sharedFS, Path: "prompts/common-dnd-transcript-chunk.md"},
|
{FS: sharedFS, Path: "prompts/common-dnd-transcript-chunk.md"},
|
||||||
{FS: sharedFS, Path: "prompts/common-dnd-system.md"},
|
{FS: sharedFS, Path: "prompts/common-dnd-system.md"},
|
||||||
})
|
})
|
||||||
@@ -178,8 +178,8 @@ func TestPromptAssetManifestHashMatchesManifestParts(t *testing.T) {
|
|||||||
t.Fatalf("Hash() = %q, want independently assembled manifest hash %q", got, want)
|
t.Fatalf("Hash() = %q, want independently assembled manifest hash %q", got, want)
|
||||||
}
|
}
|
||||||
withUnused, err := llm.HashAssets([]llm.AssetHashPart{
|
withUnused, err := llm.HashAssets([]llm.AssetHashPart{
|
||||||
{FS: moduleFS, Path: "assets/prompts/dnd.test.yaml"},
|
{FS: moduleFS, Path: "assets/prompts/prompt.yaml"},
|
||||||
{FS: moduleFS, Path: "assets/prompts/task.md"},
|
{FS: moduleFS, Path: "assets/prompts/instructions.md"},
|
||||||
{FS: sharedFS, Path: "prompts/common-dnd-transcript-chunk.md"},
|
{FS: sharedFS, Path: "prompts/common-dnd-transcript-chunk.md"},
|
||||||
{FS: sharedFS, Path: "prompts/common-dnd-system.md"},
|
{FS: sharedFS, Path: "prompts/common-dnd-system.md"},
|
||||||
{FS: sharedFS, Path: "prompts/common-dnd-npcs.md"},
|
{FS: sharedFS, Path: "prompts/common-dnd-npcs.md"},
|
||||||
|
|||||||
@@ -61,8 +61,8 @@ func TestRunnerProcessesSeriatimInputWithDNDSpellsExtractor(t *testing.T) {
|
|||||||
t.Fatalf("len(NormalizeOutputs) = %d, want 1", len(output.NormalizeOutputs))
|
t.Fatalf("len(NormalizeOutputs) = %d, want 1", len(output.NormalizeOutputs))
|
||||||
}
|
}
|
||||||
serializedOutput := output.NormalizeOutputs[0]
|
serializedOutput := output.NormalizeOutputs[0]
|
||||||
if serializedOutput.LaneID != "spells" || serializedOutput.Artifact.Schema.ID != spells.ResponseSchemaID || serializedOutput.Artifact.Schema.Version != spells.SchemaVersion {
|
if serializedOutput.LaneID != "spells" || serializedOutput.Artifact.Schema.ID != spellcodec.SchemaID || serializedOutput.Artifact.Schema.Version != spellcodec.SchemaVersion {
|
||||||
t.Fatalf("serialized output envelope = %#v, want dnd spells schema on spells lane", serializedOutput)
|
t.Fatalf("serialized output envelope = %#v, want durable spell artifact schema on spells lane", serializedOutput)
|
||||||
}
|
}
|
||||||
response := decodeRunnerSpellResponse(t, serializedOutput.Artifact.Content)
|
response := decodeRunnerSpellResponse(t, serializedOutput.Artifact.Content)
|
||||||
if len(response.SpellCasts) != 2 {
|
if len(response.SpellCasts) != 2 {
|
||||||
|
|||||||
Reference in New Issue
Block a user