diff --git a/docs/internal/llm.md b/docs/internal/llm.md index 7293168..4feccc2 100644 --- a/docs/internal/llm.md +++ b/docs/internal/llm.md @@ -107,10 +107,12 @@ The spell extractor's package-owned prompt declares a required `application/json` `spell_catalog` input and an optional `application/json` `npcs` input. The extractor generates the catalog input from its prepared effective catalog as `{"spell_names":[...]}` using sorted canonical names only. -When an NPC registry is bound, it strictly decodes and identity-validates one -durable artifact, re-encodes canonical JSON, and generates a semantic digest -over those bytes. The unbound input is exactly `{"npcs":[]}`. Input digests -cover the generated bytes; manifests record catalog identity and optional NPC +The shared D&D prompt assets include a generic NPC grounding fragment directly +after the campaign reference message. When an NPC registry is bound, the +domain registry boundary strictly decodes and identity-validates one durable +artifact, re-encodes canonical JSON, and generates a semantic digest over +those bytes. The unbound input is exactly `{"npcs":[]}`. Input digests cover +the generated bytes; manifests record catalog identity and optional NPC registry digest/count rather than names, aliases, overlay bytes, registry paths, or source metadata. diff --git a/docs/internal/modules.md b/docs/internal/modules.md index dba15f0..4caaa96 100644 --- a/docs/internal/modules.md +++ b/docs/internal/modules.md @@ -47,7 +47,8 @@ LLM-backed extensions own their prompt definitions and response schemas under package-local embedded assets. Shared filesystem composition belongs in `internal/framework/promptfs`; reusable D&D prompt fragments, reference declarations, prompt-input assembly, and source-unit helpers belong in -`internal/modules/dnd/shared`. Stage contracts expose only Notarius structured- +`internal/modules/dnd/shared`, which also owns bounded D&D diagnostics. Stage +contracts expose only Notarius structured- completion types, not Scriptorium public types. Reference material may inform a module or prompt but must not become source @@ -82,14 +83,18 @@ semantic digest; overlay content remains contextual reference material rather than source evidence. Its external JSON contract is defined in the [spell-catalog overlay contract](../integrations/dnd-spell-catalog-overlays.md). -### `internal/modules/dnd/npcs/identity` and `internal/modules/dnd/codec/npcs` +### `internal/modules/dnd/npcs/identity`, `internal/modules/dnd/npcs/registry`, and `internal/modules/dnd/codec/npcs` The NPC identity package owns Unicode comparison keys, deterministic `npc:sha256:` IDs, display normalization, and whole-registry collision issues. -The NPC codec owns the strict durable `dnd/npc-list` JSON boundary and exposes -candidate versus approved encode/decode operations. NPC source references are -durable provenance and may later be consumed by another pipeline as registry -context without being treated as evidence for that pipeline. +The registry package resolves one optional normalized artifact through the +strict codec, validates whole-registry identity, canonicalizes its JSON, and +provides immutable records, prompt input, semantic digest, count, and exact +canonical-name/alias lookup. It owns the `npcs` slot and its bounded, +content-safe preparation failures. NPC source references are durable +provenance and are not treated as evidence for a consuming pipeline. The NPC +codec owns the strict durable `dnd/npc-list` JSON boundary and exposes +candidate versus approved encode/decode operations. ## Input Adapter @@ -196,13 +201,13 @@ stages, using the codec only for checkpoint, debug, and output boundaries. Shared D&D helpers keep prompt input names and source-unit reference conversion consistent with the scene chunker. -The extractor also declares the optional `npcs` registry slot. Preparation -requires one approved `application/json` item no larger than 1 MiB, validates -identity without relating registry source references to the current transcript, -and supplies canonical JSON to a spell-owned prompt message. A bound registry -adds only `npc_registry_digest` and `npc_count` to manifest metadata and an -`npc_registry` checkpoint fingerprint. The unbound prompt input is exactly -`{"npcs":[]}` and has no registry provenance or fingerprint. +The extractor also declares the optional `npcs` registry slot and consumes the +prepared immutable registry boundary from `internal/modules/dnd/npcs/registry`. +A bound registry adds only `npc_registry_digest` and `npc_count` to manifest +metadata and an `npc_registry` checkpoint fingerprint. The unbound prompt +input is exactly `{"npcs":[]}` and has no registry provenance or fingerprint. +The shared NPC grounding fragment is placed immediately after the common +campaign reference message and is included in the spell prompt fingerprint. The durable payload and manifest metadata shapes are defined in the [D&D spell artifact contract](../integrations/dnd-spell-artifacts.md). @@ -212,7 +217,7 @@ The durable payload and manifest metadata shapes are defined in the The NPC extractor maps private model output to the canonical `dnd.NPCList`, assigns source identity and deterministic NPC IDs, and preserves source references for deterministic validation. It uses the shared campaign -references only for disambiguation and does not consume the spell-owned NPC +references only for disambiguation and does not consume the optional NPC registry slot. Its prompt and private response schema are package-owned. ### `internal/modules/dnd/normalize/npcs` diff --git a/docs/internal/overview.md b/docs/internal/overview.md index 50968be..e04c33b 100644 --- a/docs/internal/overview.md +++ b/docs/internal/overview.md @@ -89,6 +89,7 @@ Configuration. The implemented module packages are: | `internal/modules/dnd/codec/npcs` | Strictly decodes and stably encodes the durable D&D NPC-list representation. | | `internal/modules/dnd/extract/spells` | Maps private structured model output to canonical source-grounded D&D spell lists. | | `internal/modules/dnd/extract/npcs` | Maps private structured model output to canonical source-grounded D&D NPC lists. | +| `internal/modules/dnd/npcs/registry` | Resolves validated normalized NPC references into immutable grounding data and exact identity lookup. | | `internal/modules/dnd/npcs/identity` | Owns Unicode-aware NPC identity, ID derivation, and registry collision validation. | | `internal/modules/dnd/spells/catalog` | Embeds and validates the versioned D&D 5e 2014 SRD catalog, composes optional overlays, and provides immutable effective lookup. | | `internal/modules/generic/merge/appendorder` | Combines accepted extraction results in chunk order. | @@ -98,17 +99,19 @@ Configuration. The implemented module packages are: | `internal/modules/generic/output/json` | Encodes manifests, lane payloads, warnings, and rejections as logical JSON files. | `internal/modules/dnd/shared` owns reusable D&D prompt fragments, -reference declarations, prompt input assembly, and source-unit reference -helpers. Domain-neutral prompt filesystem composition lives in +reference declarations, prompt input assembly, source-unit reference helpers, +and bounded diagnostics under `internal/modules/dnd/shared/diagnostics`. +The shared NPC grounding fragment is mounted for D&D prompts and is owned by +this package. Domain-neutral prompt filesystem composition lives in `internal/framework/promptfs`. -The spell extractor owns its optional `npcs` registry boundary. Preparation -strictly decodes and identity-validates one normalized JSON artifact, emits -canonical registry JSON to the spell prompt, and records only its semantic -digest and count in prepared metadata. The raw reference remains independently -tracked by pipeline provenance. An absent registry is represented only by the -empty prompt value `{"npcs":[]}`; the shared D&D reference fragment is not -changed. +The `dnd/npcs/registry` package owns the optional `npcs` registry boundary. +Preparation strictly decodes and identity-validates one normalized JSON +artifact, emits canonical registry JSON to the spell prompt, and records only +its semantic digest and count in prepared metadata. The raw reference remains +independently tracked by pipeline provenance. An absent registry is represented +only by the empty prompt value `{"npcs":[]}`. Spell extraction consumes this +shared registry boundary without changing its public module contract. Generic validators under `internal/modules/generic/validate` provide unconditional test decisions, JSON syntax validation, and JSON Schema diff --git a/internal/modules/dnd/extract/spells/assets/prompts/dnd.spells.yaml b/internal/modules/dnd/extract/spells/assets/prompts/dnd.spells.yaml index ccc097f..9c5fc79 100644 --- a/internal/modules/dnd/extract/spells/assets/prompts/dnd.spells.yaml +++ b/internal/modules/dnd/extract/spells/assets/prompts/dnd.spells.yaml @@ -32,9 +32,9 @@ messages: cache_control: type: ephemeral - role: user - content_file: ./catalog.md + content_file: ./sharedassets/common-dnd-npcs.md - role: user - content_file: ./npc_registry.md + content_file: ./catalog.md - role: user content_file: ./task.md - role: user diff --git a/internal/modules/dnd/extract/spells/assets/prompts/npc_registry.md b/internal/modules/dnd/extract/spells/assets/prompts/npc_registry.md deleted file mode 100644 index bf074a1..0000000 --- a/internal/modules/dnd/extract/spells/assets/prompts/npc_registry.md +++ /dev/null @@ -1,11 +0,0 @@ -The optional canonical NPC registry for this extraction is provided below as -durable JSON. Use it only to prefer exact canonical NPC names and recognize -their aliases when the transcript identifies a caster. - -Registry entries are grounding material, not evidence that a spell was cast. -Do not extract a spell, caster, effect, or source reference from the registry. -NPC source references describe registry provenance and may belong to another -session; they are never spell evidence. Preserve the existing player and party -policy for identifying PCs from the transcript. - -{{ input "npcs" }} diff --git a/internal/modules/dnd/extract/spells/extractor.go b/internal/modules/dnd/extract/spells/extractor.go index 014d1fc..9f43c57 100644 --- a/internal/modules/dnd/extract/spells/extractor.go +++ b/internal/modules/dnd/extract/spells/extractor.go @@ -9,6 +9,7 @@ import ( "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd" + npcregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/registry" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared" spellcatalog "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/spells/catalog" ) @@ -17,6 +18,11 @@ const Key = "dnd/spells" const ArtifactType = "dnd.spell_cast" const SchemaVersion = "v1" +const ( + NPCRegistryReferenceSlot = npcregistry.ReferenceSlot + NPCRegistryMaxBytes = npcregistry.MaxBytes +) + var requiredCapabilities = []string{ "chunks", "source.transcript", @@ -59,7 +65,7 @@ type Extractor struct { llm contracts.StructuredLLMClient effectiveCatalog spellcatalog.EffectiveCatalog catalogPromptInput contracts.LLMInputMaterial - npcRegistry npcRegistryPromptInput + npcRegistry *npcregistry.Registry promptSHA string responseSchemaSHA string } @@ -83,7 +89,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr if err != nil { return nil, extractorErrorf("prepare spell catalog prompt input: %w", err) } - npcRegistry, err := resolveNPCRegistry(referenceSet) + npcRegistry, err := npcregistry.Resolve(referenceSet) if err != nil { return nil, extractorErrorf("prepare NPC registry prompt input: %w", err) } @@ -127,9 +133,9 @@ func (e *Extractor) ManifestMetadata() map[string]any { "response_schema_version": SchemaVersion, "response_schema_sha256": e.responseSchemaSHA, } - if e.npcRegistry.bound { - metadata["npc_registry_digest"] = e.npcRegistry.digest - metadata["npc_count"] = e.npcRegistry.count + if e.npcRegistry.Bound() { + metadata["npc_registry_digest"] = e.npcRegistry.Digest() + metadata["npc_count"] = e.npcRegistry.Count() } return metadata } @@ -143,8 +149,8 @@ func (e *Extractor) CheckpointFingerprints() []pipeline.CheckpointFingerprint { {Name: "prompt", Value: e.promptSHA}, {Name: "response_schema", Value: e.responseSchemaSHA}, } - if e.npcRegistry.bound { - fingerprints = append(fingerprints, pipeline.CheckpointFingerprint{Name: "npc_registry", Value: e.npcRegistry.digest}) + if e.npcRegistry.Bound() { + fingerprints = append(fingerprints, pipeline.CheckpointFingerprint{Name: "npc_registry", Value: e.npcRegistry.Digest()}) } return fingerprints } @@ -179,7 +185,7 @@ func (e *Extractor) Extract(ctx context.Context, req contracts.TypedExtractionRe var response extractionResponse inputs := shared.PromptInputs(sourceInput, req.References) inputs[spellcatalog.SpellCatalogReferenceSlot] = e.catalogPromptInput.Clone() - inputs[NPCRegistryReferenceSlot] = e.npcRegistry.input.Clone() + inputs[NPCRegistryReferenceSlot] = e.npcRegistry.PromptInput() if _, err := e.llm.CompleteStructured(ctx, contracts.StructuredCompletionRequest{ StageName: Key, PromptID: PromptID, diff --git a/internal/modules/dnd/extract/spells/npc_registry.go b/internal/modules/dnd/extract/spells/npc_registry.go deleted file mode 100644 index de9afb2..0000000 --- a/internal/modules/dnd/extract/spells/npc_registry.go +++ /dev/null @@ -1,94 +0,0 @@ -package spells - -import ( - "crypto/sha256" - "encoding/hex" - "fmt" - "mime" - "strings" - - "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" - npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs" - "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/diagnostics" - "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity" -) - -const ( - NPCRegistryReferenceSlot = "npcs" - NPCRegistryMaxBytes = 1048576 -) - -type npcRegistryPromptInput struct { - input contracts.LLMInputMaterial - digest string - count int - bound bool -} - -func resolveNPCRegistry(references contracts.ReferenceSet) (npcRegistryPromptInput, error) { - slot, ok := references.Slots[NPCRegistryReferenceSlot] - if !ok { - return npcRegistryPromptInput{ - input: contracts.NewLLMInputMaterial( - NPCRegistryReferenceSlot, - "application/json", - []byte(`{"npcs":[]}`), - "", - "", - ), - }, nil - } - if len(slot.Items) != 1 { - return npcRegistryPromptInput{}, fmt.Errorf("reference slot %q must contain exactly one item", NPCRegistryReferenceSlot) - } - - item := slot.Items[0] - mediaType, _, err := mime.ParseMediaType(item.MediaType) - if err != nil { - return npcRegistryPromptInput{}, fmt.Errorf("reference slot %q item media type %q is invalid: %w", NPCRegistryReferenceSlot, item.MediaType, err) - } - if !strings.EqualFold(mediaType, "application/json") { - return npcRegistryPromptInput{}, fmt.Errorf("reference slot %q item media type %q must be application/json", NPCRegistryReferenceSlot, item.MediaType) - } - if len(item.Content) > NPCRegistryMaxBytes { - return npcRegistryPromptInput{}, fmt.Errorf("reference slot %q item is %d bytes, limit %d", NPCRegistryReferenceSlot, len(item.Content), NPCRegistryMaxBytes) - } - - codec := npccodec.New() - value, err := codec.Decode(item.Content) - if err != nil { - return npcRegistryPromptInput{}, fmt.Errorf("decode NPC registry: invalid approved NPC JSON") - } - if issues := identity.ValidateList(value); len(issues) > 0 { - return npcRegistryPromptInput{}, fmt.Errorf("%s", formatNPCIdentityIssues(issues)) - } - content, err := codec.Encode(value) - if err != nil { - return npcRegistryPromptInput{}, fmt.Errorf("encode canonical NPC registry: approved NPC value could not be encoded") - } - - digest := semanticNPCRegistryDigest(content) - return npcRegistryPromptInput{ - input: contracts.NewLLMInputMaterial(NPCRegistryReferenceSlot, "application/json", content, digest, ""), - digest: digest, - count: len(value.NPCs), - bound: true, - }, nil -} - -func semanticNPCRegistryDigest(content []byte) string { - sum := sha256.Sum256(content) - return "sha256:" + hex.EncodeToString(sum[:]) -} - -func formatNPCIdentityIssues(issues []identity.Issue) string { - parts := make([]string, len(issues)) - for index, issue := range issues { - location := fmt.Sprintf("record %d", issue.RecordIndex) - if issue.AliasIndex >= 0 { - location += fmt.Sprintf(" alias %d", issue.AliasIndex) - } - parts[index] = fmt.Sprintf("%s at %s", issue.Code, location) - } - return diagnostics.Aggregate("validate NPC registry identity", parts) -} diff --git a/internal/modules/dnd/extract/spells/npc_registry_test.go b/internal/modules/dnd/extract/spells/npc_registry_test.go deleted file mode 100644 index cabaadc..0000000 --- a/internal/modules/dnd/extract/spells/npc_registry_test.go +++ /dev/null @@ -1,279 +0,0 @@ -package spells - -import ( - "bytes" - "context" - "encoding/json" - "fmt" - "strings" - "testing" - "unicode/utf8" - - "gitea.maximumdirect.net/eric/notarius/internal/core/source" - "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" - "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" - "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd" - npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs" - "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/diagnostics" - "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity" -) - -func TestResolveNPCRegistryUsesExactEmptyPromptWhenUnbound(t *testing.T) { - resolved, err := resolveNPCRegistry(contracts.ReferenceSet{}) - if err != nil { - t.Fatalf("resolveNPCRegistry() error = %v, want nil", err) - } - if resolved.bound || resolved.digest != "" || resolved.count != 0 { - t.Fatalf("resolved unbound registry = %#v, want no semantic metadata", resolved) - } - if resolved.input.Name != NPCRegistryReferenceSlot || resolved.input.MediaType != "application/json" || resolved.input.Digest != "" || resolved.input.OriginURI != "" { - t.Fatalf("unbound prompt input metadata = %#v, want name/media type only", resolved.input) - } - if got := string(resolved.input.Content); got != `{"npcs":[]}` { - t.Fatalf("unbound prompt input = %q, want exact empty registry", got) - } - if resolved.input.SizeBytes != int64(len(`{"npcs":[]}`)) { - t.Fatalf("unbound prompt input size = %d, want %d", resolved.input.SizeBytes, len(`{"npcs":[]}`)) - } - if metadata := newExtractor(t, &fakeSpellsLLMClient{}).ManifestMetadata(); metadata["npc_registry_digest"] != nil || metadata["npc_count"] != nil { - t.Fatalf("unbound extractor metadata = %#v, want no NPC registry fields", metadata) - } - fingerprints := newExtractor(t, &fakeSpellsLLMClient{}).CheckpointFingerprints() - for _, fingerprint := range fingerprints { - if fingerprint.Name == "npc_registry" { - t.Fatalf("unbound checkpoint fingerprints = %#v, want no NPC registry fingerprint", fingerprints) - } - } -} - -func TestResolveNPCRegistryCanonicalizesContentAndUsesSemanticDigest(t *testing.T) { - value := validNPCRegistryList() - canonical := encodeNPCRegistry(t, value) - raw := append([]byte(" \n"), canonical...) - raw = append(raw, []byte("\n ")...) - - resolved, err := resolveNPCRegistry(npcRegistryReference(raw, "file:///another-session/npcs.json")) - if err != nil { - t.Fatalf("resolveNPCRegistry() error = %v, want nil", err) - } - if !resolved.bound || resolved.count != len(value.NPCs) { - t.Fatalf("resolved registry = %#v, want bound registry with %d NPC", resolved, len(value.NPCs)) - } - if !bytes.Equal(resolved.input.Content, canonical) { - t.Fatalf("canonical prompt input = %s, want %s", resolved.input.Content, canonical) - } - if resolved.input.Digest != semanticNPCRegistryDigest(canonical) || resolved.digest != resolved.input.Digest { - t.Fatalf("semantic digest = %q/%q, want %q", resolved.input.Digest, resolved.digest, semanticNPCRegistryDigest(canonical)) - } - if resolved.input.OriginURI != "" { - t.Fatalf("prompt input origin = %q, want no provenance path", resolved.input.OriginURI) - } - - resolved.input.Content[0] = 'X' - again, err := resolveNPCRegistry(npcRegistryReference(raw, "file:///another-session/npcs.json")) - if err != nil { - t.Fatalf("second resolveNPCRegistry() error = %v, want nil", err) - } - if !bytes.Equal(again.input.Content, canonical) { - t.Fatalf("canonical content changed after caller mutation = %s, want %s", again.input.Content, canonical) - } -} - -func TestResolveNPCRegistryRejectsInvalidBoundaryValues(t *testing.T) { - valid := validNPCRegistryList() - second := validNPCRegistryList().NPCs[0] - second.ID = identity.DeriveID("Captain Vale") - second.Name = "Captain Vale" - second.Aliases = []string{"The Greencloak"} - valueWithAliasCollision := dnd.NPCList{NPCs: []dnd.NPC{valid.NPCs[0], second}} - invalidID := valid - invalidID.NPCs[0].ID = "not-an-npc-id" - - tests := []struct { - name string - reference contracts.ReferenceSet - wantError string - forbidden []string - }{ - {name: "zero items", reference: contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{NPCRegistryReferenceSlot: {Items: []contracts.ReferenceItem{}}}}, wantError: "exactly one"}, - {name: "multiple", reference: contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{NPCRegistryReferenceSlot: {Items: []contracts.ReferenceItem{{Content: []byte(`{"npcs":[]}`)}, {Content: []byte(`{"npcs":[]}`)}}}}}, wantError: "exactly one"}, - {name: "wrong media type", reference: npcRegistryReferenceWithMedia([]byte(`{"npcs":[]}`), "text/plain"), wantError: "must be application/json"}, - {name: "malformed JSON", reference: npcRegistryReference([]byte(`{"npcs":[],"MALFORMED_REGISTRY_SECRET":`), "file:///private.json"), wantError: "invalid approved NPC JSON", forbidden: []string{"MALFORMED_REGISTRY_SECRET"}}, - {name: "unknown field", reference: npcRegistryReference([]byte(`{"npcs":[],"UNKNOWN_FIELD_SECRET":true}`), "file:///private.json"), wantError: "invalid approved NPC JSON", forbidden: []string{"UNKNOWN_FIELD_SECRET"}}, - {name: "invalid ID", reference: npcRegistryReference(marshalNPCRegistry(t, invalidID), "file:///private.json"), wantError: "decode NPC registry"}, - {name: "alias collision", reference: npcRegistryReference(encodeNPCRegistry(t, valueWithAliasCollision), "file:///private.json"), wantError: string(identity.IssueAliasOwnershipCollision)}, - {name: "byte limit", reference: npcRegistryReference(bytes.Repeat([]byte("x"), NPCRegistryMaxBytes+1), "file:///private.json"), wantError: "limit"}, - } - for _, test := range tests { - t.Run(test.name, func(t *testing.T) { - _, err := resolveNPCRegistry(test.reference) - if err == nil || !strings.Contains(err.Error(), test.wantError) { - t.Fatalf("resolveNPCRegistry() error = %v, want %q", err, test.wantError) - } - for _, forbidden := range append(test.forbidden, "Mira Thorn", "The Greencloak", "private.json") { - if strings.Contains(err.Error(), forbidden) { - t.Fatalf("error leaked registry content or provenance %q: %v", forbidden, err) - } - } - }) - } -} - -func TestResolveNPCRegistryBoundsIdentityDiagnosticsWithoutContent(t *testing.T) { - const recordCount = 30 - value := dnd.NPCList{NPCs: make([]dnd.NPC, recordCount)} - for index := range value.NPCs { - value.NPCs[index] = dnd.NPC{ - ID: "npc:sha256:0000000000000000000000000000000000000000000000000000000000000000", - Name: fmt.Sprintf("PRIVATE NPC %d", index), - Aliases: []string{"PRIVATE SHARED ALIAS"}, - Description: "PRIVATE DESCRIPTION", - Relationships: []dnd.NPCRelationship{}, - SourceRefs: []source.SourceRef{{SourceID: "private-source", StartUnitID: 1, EndUnitID: 1}}, - } - } - issues := identity.ValidateList(value) - if len(issues) <= diagnostics.MaxIssues { - t.Fatalf("identity issues = %d, want more than display limit", len(issues)) - } - - _, err := resolveNPCRegistry(npcRegistryReference(marshalNPCRegistry(t, value), "file:///private-registry.json")) - if err == nil { - t.Fatal("resolveNPCRegistry() error = nil, want bounded identity rejection") - } - message := err.Error() - if !utf8.ValidString(message) || len([]byte(message)) > diagnostics.MaxMessageBytes { - t.Fatalf("identity error has invalid encoding or size: bytes=%d message=%q", len([]byte(message)), message) - } - wantOmitted := fmt.Sprintf("%d additional issue(s) omitted", len(issues)-diagnostics.MaxIssues) - if !strings.Contains(message, wantOmitted) { - t.Fatalf("identity error = %q, want %q", message, wantOmitted) - } - for _, forbidden := range []string{"PRIVATE NPC", "PRIVATE SHARED ALIAS", "PRIVATE DESCRIPTION", "private-source", "private-registry.json"} { - if strings.Contains(message, forbidden) { - t.Fatalf("identity error leaked %q: %s", forbidden, message) - } - } -} - -func TestNPCRegistryFingerprintIsSemanticAndDefensive(t *testing.T) { - value := validNPCRegistryList() - canonical := encodeNPCRegistry(t, value) - pretty, err := json.MarshalIndent(value, "", " ") - if err != nil { - t.Fatalf("MarshalIndent() error = %v", err) - } - first := newExtractor(t, &fakeSpellsLLMClient{}, npcRegistryReference(canonical, "file:///one.json")) - second := newExtractor(t, &fakeSpellsLLMClient{}, npcRegistryReference(pretty, "file:///two.json")) - firstFingerprints := checkpointFingerprintMap(first.CheckpointFingerprints()) - secondFingerprints := checkpointFingerprintMap(second.CheckpointFingerprints()) - if firstFingerprints["npc_registry"] == "" || firstFingerprints["npc_registry"] != secondFingerprints["npc_registry"] { - t.Fatalf("semantic NPC fingerprints = %#v and %#v, want same npc_registry value", firstFingerprints, secondFingerprints) - } - returned := first.CheckpointFingerprints() - returned[0].Name = "caller-mutated" - if first.CheckpointFingerprints()[0].Name == "caller-mutated" { - t.Fatal("CheckpointFingerprints() returned caller-mutable slice state") - } - - changed := validNPCRegistryList() - changed.NPCs[0].Description = "A different description." - changedFingerprint := checkpointFingerprintMap(newExtractor(t, &fakeSpellsLLMClient{}, npcRegistryReference(encodeNPCRegistry(t, changed), "file:///three.json")).CheckpointFingerprints()) - if changedFingerprint["npc_registry"] == firstFingerprints["npc_registry"] { - t.Fatalf("semantic NPC fingerprint did not change: %#v", changedFingerprint) - } - - metadata := first.ManifestMetadata() - encoded, err := json.Marshal(map[string]any{"metadata": metadata, "fingerprints": firstFingerprints}) - if err != nil { - t.Fatalf("marshal metadata: %v", err) - } - for _, forbidden := range []string{"Mira Thorn", "The Greencloak", "another-session", "one.json"} { - if strings.Contains(string(encoded), forbidden) { - t.Fatalf("metadata or fingerprints leaked %q: %s", forbidden, encoded) - } - } - if metadata["npc_registry_digest"] != firstFingerprints["npc_registry"] || metadata["npc_count"] != 1 { - t.Fatalf("NPC registry metadata = %#v, want digest and count only", metadata) - } - -} - -func TestExtractPassesCanonicalNPCRegistryToLLMWithoutProvenance(t *testing.T) { - client := &fakeSpellsLLMClient{response: extractionResponse{SpellCasts: []spellCastResponse{}}} - canonical := encodeNPCRegistry(t, validNPCRegistryList()) - extractor := newExtractor(t, client, npcRegistryReference(append([]byte("\n"), canonical...), "file:///npc-session.json")) - if _, err := extractor.Extract(context.Background(), extractionRequest()); err != nil { - t.Fatalf("Extract() error = %v, want nil", err) - } - input := client.requests[0].Inputs[NPCRegistryReferenceSlot] - if input.Name != NPCRegistryReferenceSlot || input.MediaType != "application/json" || input.Digest != semanticNPCRegistryDigest(canonical) || input.OriginURI != "" { - t.Fatalf("NPC prompt input metadata = %#v, want semantic metadata without provenance", input) - } - if !bytes.Equal(input.Content, canonical) { - t.Fatalf("NPC prompt input = %s, want canonical JSON %s", input.Content, canonical) - } -} - -func validNPCRegistryList() dnd.NPCList { - return dnd.NPCList{NPCs: []dnd.NPC{{ - ID: identity.DeriveID("Mira Thorn"), - Name: "Mira Thorn", - Aliases: []string{"The Greencloak"}, - Description: "A guarded ranger who watches the northern road.", - Relationships: []dnd.NPCRelationship{{ - Target: "Captain Vale", Relationship: "reports to", - }}, - SourceRefs: []source.SourceRef{{SourceID: "npc-session", StartUnitID: 41, EndUnitID: 43}}, - }}} -} - -func encodeNPCRegistry(t *testing.T, value dnd.NPCList) []byte { - t.Helper() - content, err := npccodec.New().Encode(value) - if err != nil { - t.Fatalf("encode NPC registry: %v", err) - } - return content -} - -func marshalNPCRegistry(t *testing.T, value dnd.NPCList) []byte { - t.Helper() - content, err := json.Marshal(value) - if err != nil { - t.Fatalf("marshal NPC registry: %v", err) - } - return content -} - -func npcRegistryReference(content []byte, origin string) contracts.ReferenceSet { - references := npcRegistryReferenceWithMedia(content, "application/json; charset=utf-8") - item := references.Slots[NPCRegistryReferenceSlot].Items[0] - item.Origin.URI = origin - slot := references.Slots[NPCRegistryReferenceSlot] - slot.Items[0] = item - references.Slots[NPCRegistryReferenceSlot] = slot - return references -} - -func npcRegistryReferenceWithMedia(content []byte, mediaType string) contracts.ReferenceSet { - return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{ - NPCRegistryReferenceSlot: { - Slot: contracts.ReferenceSlot{Name: NPCRegistryReferenceSlot, AcceptedMediaTypes: []string{"application/json"}, MaxBytes: NPCRegistryMaxBytes}, - Items: []contracts.ReferenceItem{{ - SlotName: NPCRegistryReferenceSlot, - MediaType: mediaType, - Content: append([]byte(nil), content...), - Origin: contracts.ReferenceOrigin{Type: "file", URI: "file:///npc-registry.json"}, - }}, - }, - }} -} - -func checkpointFingerprintMap(values []pipeline.CheckpointFingerprint) map[string]string { - result := make(map[string]string, len(values)) - for _, value := range values { - result[value.Name] = value.Value - } - return result -} diff --git a/internal/modules/dnd/extract/spells/npc_registry_wiring_test.go b/internal/modules/dnd/extract/spells/npc_registry_wiring_test.go new file mode 100644 index 0000000..e1dc967 --- /dev/null +++ b/internal/modules/dnd/extract/spells/npc_registry_wiring_test.go @@ -0,0 +1,120 @@ +package spells + +import ( + "bytes" + "context" + "encoding/json" + "strings" + "testing" + + "gitea.maximumdirect.net/eric/notarius/internal/core/source" + "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" + "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" + "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd" + npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs" + "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity" + "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/registry" +) + +func TestSpellExtractorUsesExactUnboundNPCPromptAndOmitsRegistryIdentity(t *testing.T) { + extractor := newExtractor(t, &fakeSpellsLLMClient{}) + metadata := extractor.ManifestMetadata() + if metadata["npc_registry_digest"] != nil || metadata["npc_count"] != nil { + t.Fatalf("unbound extractor metadata = %#v, want no NPC registry fields", metadata) + } + for _, fingerprint := range extractor.CheckpointFingerprints() { + if fingerprint.Name == "npc_registry" { + t.Fatalf("unbound checkpoint fingerprints = %#v, want no NPC registry fingerprint", extractor.CheckpointFingerprints()) + } + } + + client := &fakeSpellsLLMClient{response: extractionResponse{SpellCasts: []spellCastResponse{}}} + extractor = newExtractor(t, client) + if _, err := extractor.Extract(context.Background(), extractionRequest()); err != nil { + t.Fatalf("Extract() error = %v, want nil", err) + } + input := client.requests[0].Inputs[NPCRegistryReferenceSlot] + if input.Name != NPCRegistryReferenceSlot || input.MediaType != npccodec.MediaType || input.Digest != "" || input.OriginURI != "" || string(input.Content) != `{"npcs":[]}` { + t.Fatalf("NPC prompt input = %#v, want exact empty registry material", input) + } +} + +func TestSpellExtractorPreservesSemanticNPCRegistryFingerprintAndPromptWiring(t *testing.T) { + value := registryFixture() + canonical, err := npccodec.New().Encode(value) + if err != nil { + t.Fatalf("encode NPC registry: %v", err) + } + pretty, err := json.MarshalIndent(value, "", " ") + if err != nil { + t.Fatalf("MarshalIndent() error = %v", err) + } + first := newExtractor(t, &fakeSpellsLLMClient{}, spellNPCRegistryReference(canonical, "file:///one.json")) + second := newExtractor(t, &fakeSpellsLLMClient{}, spellNPCRegistryReference(pretty, "file:///two.json")) + firstFingerprints := checkpointFingerprintMap(first.CheckpointFingerprints()) + secondFingerprints := checkpointFingerprintMap(second.CheckpointFingerprints()) + if firstFingerprints["npc_registry"] == "" || firstFingerprints["npc_registry"] != secondFingerprints["npc_registry"] { + t.Fatalf("semantic NPC fingerprints = %#v and %#v, want same npc_registry value", firstFingerprints, secondFingerprints) + } + metadata := first.ManifestMetadata() + if metadata["npc_registry_digest"] != firstFingerprints["npc_registry"] || metadata["npc_count"] != 1 { + t.Fatalf("NPC registry metadata = %#v, want digest and count only", metadata) + } + + client := &fakeSpellsLLMClient{response: extractionResponse{SpellCasts: []spellCastResponse{}}} + extractor := newExtractor(t, client, spellNPCRegistryReference(append([]byte("\n"), canonical...), "file:///npc-session.json")) + if _, err := extractor.Extract(context.Background(), extractionRequest()); err != nil { + t.Fatalf("Extract() error = %v, want nil", err) + } + input := client.requests[0].Inputs[NPCRegistryReferenceSlot] + if input.Name != NPCRegistryReferenceSlot || input.MediaType != npccodec.MediaType || input.Digest != firstFingerprints["npc_registry"] || input.OriginURI != "" { + t.Fatalf("NPC prompt input metadata = %#v, want semantic metadata without provenance", input) + } + if !bytes.Equal(input.Content, canonical) { + t.Fatalf("NPC prompt input = %s, want canonical JSON %s", input.Content, canonical) + } + encoded, err := json.Marshal(map[string]any{"metadata": metadata, "fingerprints": firstFingerprints}) + if err != nil { + t.Fatalf("marshal metadata: %v", err) + } + for _, forbidden := range []string{"Mira Thorn", "The Greencloak", "one.json", "two.json"} { + if strings.Contains(string(encoded), forbidden) { + t.Fatalf("metadata or fingerprints leaked %q: %s", forbidden, encoded) + } + } +} + +func registryFixture() dnd.NPCList { + return dnd.NPCList{NPCs: []dnd.NPC{{ + ID: identity.DeriveID("Mira Thorn"), + Name: "Mira Thorn", + Aliases: []string{"The Greencloak"}, + Description: "A guarded ranger who watches the northern road.", + Relationships: []dnd.NPCRelationship{{ + Target: "Captain Vale", Relationship: "reports to", + }}, + SourceRefs: []source.SourceRef{{SourceID: "npc-session", StartUnitID: 41, EndUnitID: 43}}, + }}} +} + +func spellNPCRegistryReference(content []byte, origin string) contracts.ReferenceSet { + return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{ + registry.ReferenceSlot: { + Slot: contracts.ReferenceSlot{Name: registry.ReferenceSlot, AcceptedMediaTypes: []string{npccodec.MediaType}, MaxBytes: registry.MaxBytes}, + Items: []contracts.ReferenceItem{{ + SlotName: registry.ReferenceSlot, + MediaType: npccodec.MediaType, + Content: append([]byte(nil), content...), + Origin: contracts.ReferenceOrigin{Type: "file", URI: origin}, + }}, + }, + }} +} + +func checkpointFingerprintMap(values []pipeline.CheckpointFingerprint) map[string]string { + result := make(map[string]string, len(values)) + for _, value := range values { + result[value.Name] = value.Value + } + return result +} diff --git a/internal/modules/dnd/extract/spells/scriptorium_assets.go b/internal/modules/dnd/extract/spells/scriptorium_assets.go index 3930bd5..b12095b 100644 --- a/internal/modules/dnd/extract/spells/scriptorium_assets.go +++ b/internal/modules/dnd/extract/spells/scriptorium_assets.go @@ -15,7 +15,6 @@ func RegisterPromptAssets(registry *llm.AssetRegistry) error { promptFS, err := shared.ModulePromptFS("dnd.spells", embeddedAssets, []promptfs.ModulePromptFile{ {Name: "dnd.spells.yaml", Path: "assets/prompts/dnd.spells.yaml"}, {Name: "catalog.md", Path: "assets/prompts/catalog.md"}, - {Name: "npc_registry.md", Path: "assets/prompts/npc_registry.md"}, {Name: "task.md", Path: "assets/prompts/task.md"}, {Name: "instructions.md", Path: "assets/prompts/instructions.md"}, }) @@ -33,7 +32,6 @@ func scriptoriumPromptMetadata() (string, error) { parts := append([]llm.AssetHashPart{ {FS: embeddedAssets, Path: "assets/prompts/dnd.spells.yaml"}, {FS: embeddedAssets, Path: "assets/prompts/catalog.md"}, - {FS: embeddedAssets, Path: "assets/prompts/npc_registry.md"}, {FS: embeddedAssets, Path: "assets/prompts/task.md"}, {FS: embeddedAssets, Path: "assets/prompts/instructions.md"}, }, append(shared.CommonHashParts(), shared.ReferenceHashParts()...)...) diff --git a/internal/modules/dnd/extract/spells/scriptorium_assets_test.go b/internal/modules/dnd/extract/spells/scriptorium_assets_test.go index e48017f..3c4299d 100644 --- a/internal/modules/dnd/extract/spells/scriptorium_assets_test.go +++ b/internal/modules/dnd/extract/spells/scriptorium_assets_test.go @@ -39,11 +39,11 @@ func TestScriptoriumPromptPreparesTranscriptReferencesAndTaskMessages(t *testing if !strings.Contains(prepared.Messages[2].Content, "Shield: abjuration") { t.Fatalf("reference message missing glossary content") } - if !strings.Contains(prepared.Messages[3].Content, `{"spell_names":["Cure Wounds"]}`) { - t.Fatalf("catalog message missing canonical spell-name input: %s", prepared.Messages[3].Content) + if !strings.Contains(prepared.Messages[4].Content, `{"spell_names":["Cure Wounds"]}`) { + t.Fatalf("catalog message missing canonical spell-name input: %s", prepared.Messages[4].Content) } - if !strings.Contains(prepared.Messages[4].Content, `{"npcs":[]}`) { - t.Fatalf("NPC registry message missing empty registry input: %s", prepared.Messages[4].Content) + if !strings.Contains(prepared.Messages[3].Content, `{"npcs":[]}`) { + t.Fatalf("NPC registry message missing empty registry input: %s", prepared.Messages[3].Content) } if strings.Contains(prepared.Messages[5].Content, string(transcript)) { t.Fatalf("task message leaked transcript bytes") diff --git a/internal/modules/dnd/normalize/npcs/normalizer.go b/internal/modules/dnd/normalize/npcs/normalizer.go index 3966c26..d36f2ee 100644 --- a/internal/modules/dnd/normalize/npcs/normalizer.go +++ b/internal/modules/dnd/normalize/npcs/normalizer.go @@ -13,8 +13,8 @@ import ( "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd" - "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/diagnostics" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity" + "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics" ) const ( diff --git a/internal/modules/dnd/npcs/registry/registry.go b/internal/modules/dnd/npcs/registry/registry.go new file mode 100644 index 0000000..96ff62c --- /dev/null +++ b/internal/modules/dnd/npcs/registry/registry.go @@ -0,0 +1,204 @@ +// Package registry resolves normalized NPC artifacts into immutable grounding +// data for D&D extraction modules. +package registry + +import ( + "crypto/sha256" + "encoding/hex" + "fmt" + "mime" + "strings" + + "gitea.maximumdirect.net/eric/notarius/internal/core/source" + "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" + "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd" + npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs" + "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity" + "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics" +) + +const ( + ReferenceSlot = "npcs" + MaxBytes = 1048576 + emptyPrompt = `{"npcs":[]}` +) + +// Registry is an immutable, validated NPC registry prepared for prompt +// grounding. All accessors return defensive copies. +type Registry struct { + bound bool + list dnd.NPCList + canonical []byte + digest string + promptInput contracts.LLMInputMaterial + lookupByKey map[string]int +} + +// Resolve prepares the optional NPC registry reference. An absent slot +// produces the exact empty prompt input and no semantic registry identity. +func Resolve(references contracts.ReferenceSet) (*Registry, error) { + slot, ok := references.Slots[ReferenceSlot] + if !ok { + content := []byte(emptyPrompt) + return &Registry{ + list: dnd.NPCList{NPCs: []dnd.NPC{}}, + canonical: append([]byte(nil), content...), + promptInput: contracts.NewLLMInputMaterial(ReferenceSlot, npccodec.MediaType, content, "", ""), + lookupByKey: map[string]int{}, + }, nil + } + if len(slot.Items) != 1 { + return nil, fmt.Errorf("reference slot %q must contain exactly one item", ReferenceSlot) + } + + item := slot.Items[0] + mediaType, _, err := mime.ParseMediaType(item.MediaType) + if err != nil { + return nil, fmt.Errorf("reference slot %q item media type is invalid", ReferenceSlot) + } + if !strings.EqualFold(mediaType, npccodec.MediaType) { + return nil, fmt.Errorf("reference slot %q item media type must be %s", ReferenceSlot, npccodec.MediaType) + } + if len(item.Content) > MaxBytes { + return nil, fmt.Errorf("reference slot %q item is %d bytes, limit %d", ReferenceSlot, len(item.Content), MaxBytes) + } + + codec := npccodec.New() + value, err := codec.Decode(item.Content) + if err != nil { + return nil, fmt.Errorf("decode NPC registry: invalid approved NPC JSON") + } + if issues := identity.ValidateList(value); len(issues) > 0 { + return nil, fmt.Errorf("%s", formatIdentityIssues(issues)) + } + content, err := codec.Encode(value) + if err != nil { + return nil, fmt.Errorf("encode canonical NPC registry: approved NPC value could not be encoded") + } + + list := cloneNPCList(value) + lookupByKey := make(map[string]int, len(list.NPCs)*2) + for index, npc := range list.NPCs { + lookupByKey[identity.ComparisonKey(npc.Name)] = index + for _, alias := range npc.Aliases { + lookupByKey[identity.ComparisonKey(alias)] = index + } + } + digest := semanticDigest(content) + return &Registry{ + bound: true, + list: list, + canonical: append([]byte(nil), content...), + digest: digest, + promptInput: contracts.NewLLMInputMaterial(ReferenceSlot, npccodec.MediaType, content, digest, ""), + lookupByKey: lookupByKey, + }, nil +} + +// New is an alias for Resolve for callers constructing a prepared registry. +func New(references contracts.ReferenceSet) (*Registry, error) { return Resolve(references) } + +// Bound reports whether an NPC reference was supplied and validated. +func (r *Registry) Bound() bool { return r != nil && r.bound } + +// NPCs returns a defensive copy of the validated NPC records. +func (r *Registry) NPCs() []dnd.NPC { + if r == nil { + return nil + } + return cloneNPCs(r.list.NPCs) +} + +// List returns a defensive copy of the validated NPC list. +func (r *Registry) List() dnd.NPCList { + if r == nil { + return dnd.NPCList{} + } + return cloneNPCList(r.list) +} + +// CanonicalBytes returns a defensive copy of the canonical durable JSON. +func (r *Registry) CanonicalBytes() []byte { + if r == nil { + return nil + } + return append([]byte(nil), r.canonical...) +} + +// Digest returns the semantic SHA-256 digest of the canonical JSON, or an +// empty string when the registry is unbound. +func (r *Registry) Digest() string { + if r == nil { + return "" + } + return r.digest +} + +// Count returns the number of validated NPC records. +func (r *Registry) Count() int { + if r == nil { + return 0 + } + return len(r.list.NPCs) +} + +// PromptInput returns the canonical registry as a content-safe prompt input. +// Reference provenance is deliberately omitted. +func (r *Registry) PromptInput() contracts.LLMInputMaterial { + if r == nil { + return contracts.LLMInputMaterial{} + } + return r.promptInput.Clone() +} + +// Lookup returns the canonical NPC for an exact canonical-name or alias match +// under the NPC identity comparison policy. +func (r *Registry) Lookup(value string) (dnd.NPC, bool) { + if r == nil { + return dnd.NPC{}, false + } + index, ok := r.lookupByKey[identity.ComparisonKey(value)] + if !ok { + return dnd.NPC{}, false + } + return cloneNPC(r.list.NPCs[index]), true +} + +func semanticDigest(content []byte) string { + sum := sha256.Sum256(content) + return "sha256:" + hex.EncodeToString(sum[:]) +} + +func formatIdentityIssues(issues []identity.Issue) string { + parts := make([]string, len(issues)) + for index, issue := range issues { + location := fmt.Sprintf("record %d", issue.RecordIndex) + if issue.AliasIndex >= 0 { + location += fmt.Sprintf(" alias %d", issue.AliasIndex) + } + parts[index] = fmt.Sprintf("%s at %s", issue.Code, location) + } + return diagnostics.Aggregate("validate NPC registry identity", parts) +} + +func cloneNPCList(value dnd.NPCList) dnd.NPCList { + return dnd.NPCList{NPCs: cloneNPCs(value.NPCs)} +} + +func cloneNPCs(values []dnd.NPC) []dnd.NPC { + if values == nil { + return nil + } + cloned := make([]dnd.NPC, len(values)) + for index, value := range values { + cloned[index] = cloneNPC(value) + } + return cloned +} + +func cloneNPC(value dnd.NPC) dnd.NPC { + value.Aliases = append([]string(nil), value.Aliases...) + value.Relationships = append([]dnd.NPCRelationship(nil), value.Relationships...) + value.SourceRefs = append([]source.SourceRef(nil), value.SourceRefs...) + return value +} diff --git a/internal/modules/dnd/npcs/registry/registry_test.go b/internal/modules/dnd/npcs/registry/registry_test.go new file mode 100644 index 0000000..77226ef --- /dev/null +++ b/internal/modules/dnd/npcs/registry/registry_test.go @@ -0,0 +1,224 @@ +package registry + +import ( + "bytes" + "encoding/json" + "fmt" + "strings" + "testing" + "unicode/utf8" + + "gitea.maximumdirect.net/eric/notarius/internal/core/source" + "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" + "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd" + npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs" + "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity" + "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics" +) + +func TestResolveAbsentRegistryUsesExactEmptyPrompt(t *testing.T) { + resolved, err := Resolve(contracts.ReferenceSet{}) + if err != nil { + t.Fatalf("Resolve() error = %v, want nil", err) + } + if resolved.Bound() || resolved.Digest() != "" || resolved.Count() != 0 { + t.Fatalf("resolved unbound registry = %#v, want no semantic metadata", resolved) + } + input := resolved.PromptInput() + if input.Name != ReferenceSlot || input.MediaType != npccodec.MediaType || input.Digest != "" || input.OriginURI != "" { + t.Fatalf("unbound prompt input metadata = %#v, want name/media type only", input) + } + if got := string(input.Content); got != emptyPrompt { + t.Fatalf("unbound prompt input = %q, want exact empty registry", got) + } + if got := string(resolved.CanonicalBytes()); got != emptyPrompt { + t.Fatalf("unbound canonical bytes = %q, want exact empty registry", got) + } +} + +func TestResolveCanonicalizesAndProvidesSemanticIdentity(t *testing.T) { + value := validRegistryList() + canonical := encodeRegistry(t, value) + raw := append([]byte(" \n"), canonical...) + raw = append(raw, []byte("\n ")...) + + resolved, err := Resolve(registryReference(raw, "file:///another-session/npcs.json")) + if err != nil { + t.Fatalf("Resolve() error = %v, want nil", err) + } + if !resolved.Bound() || resolved.Count() != len(value.NPCs) { + t.Fatalf("resolved registry = %#v, want bound registry with %d NPC", resolved, len(value.NPCs)) + } + if !bytes.Equal(resolved.CanonicalBytes(), canonical) || !bytes.Equal(resolved.PromptInput().Content, canonical) { + t.Fatalf("canonical content = %s, want %s", resolved.CanonicalBytes(), canonical) + } + if resolved.PromptInput().Digest != resolved.Digest() || !strings.HasPrefix(resolved.Digest(), "sha256:") { + t.Fatalf("semantic digest = %q, want SHA-256 digest", resolved.Digest()) + } + if resolved.PromptInput().OriginURI != "" { + t.Fatalf("prompt input origin = %q, want no provenance path", resolved.PromptInput().OriginURI) + } +} + +func TestResolveRejectsInvalidBoundaryValuesWithoutContent(t *testing.T) { + valid := validRegistryList() + second := valid.NPCs[0] + second.ID = identity.DeriveID("Captain Vale") + second.Name = "Captain Vale" + second.Aliases = []string{"The Greencloak"} + valueWithAliasCollision := dnd.NPCList{NPCs: []dnd.NPC{valid.NPCs[0], second}} + invalidID := valid + invalidID.NPCs[0].ID = "not-an-npc-id" + + tests := []struct { + name string + reference contracts.ReferenceSet + wantError string + forbidden []string + }{ + {name: "zero items", reference: contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{ReferenceSlot: {Items: []contracts.ReferenceItem{}}}}, wantError: "exactly one"}, + {name: "multiple", reference: contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{ReferenceSlot: {Items: []contracts.ReferenceItem{{Content: []byte(emptyPrompt)}, {Content: []byte(emptyPrompt)}}}}}, wantError: "exactly one"}, + {name: "wrong media type", reference: registryReferenceWithMedia([]byte(emptyPrompt), "text/plain"), wantError: "must be application/json"}, + {name: "malformed JSON", reference: registryReference([]byte(`{"npcs":[],"MALFORMED_REGISTRY_SECRET":`), "file:///private.json"), wantError: "invalid approved NPC JSON", forbidden: []string{"MALFORMED_REGISTRY_SECRET"}}, + {name: "unknown field", reference: registryReference([]byte(`{"npcs":[],"UNKNOWN_FIELD_SECRET":true}`), "file:///private.json"), wantError: "invalid approved NPC JSON", forbidden: []string{"UNKNOWN_FIELD_SECRET"}}, + {name: "invalid ID", reference: registryReference(marshalRegistry(t, invalidID), "file:///private.json"), wantError: "decode NPC registry"}, + {name: "alias collision", reference: registryReference(encodeRegistry(t, valueWithAliasCollision), "file:///private.json"), wantError: string(identity.IssueAliasOwnershipCollision)}, + {name: "byte limit", reference: registryReference(bytes.Repeat([]byte("x"), MaxBytes+1), "file:///private.json"), wantError: "limit"}, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + _, err := Resolve(test.reference) + if err == nil || !strings.Contains(err.Error(), test.wantError) { + t.Fatalf("Resolve() error = %v, want %q", err, test.wantError) + } + for _, forbidden := range append(test.forbidden, "Mira Thorn", "The Greencloak", "private.json") { + if strings.Contains(err.Error(), forbidden) { + t.Fatalf("error leaked registry content or provenance %q: %v", forbidden, err) + } + } + }) + } +} + +func TestResolveBoundsIdentityDiagnosticsWithoutContent(t *testing.T) { + const recordCount = 30 + value := dnd.NPCList{NPCs: make([]dnd.NPC, recordCount)} + for index := range value.NPCs { + value.NPCs[index] = dnd.NPC{ + ID: "npc:sha256:0000000000000000000000000000000000000000000000000000000000000000", + Name: fmt.Sprintf("PRIVATE NPC %d", index), + Aliases: []string{"PRIVATE SHARED ALIAS"}, + Description: "PRIVATE DESCRIPTION", + Relationships: []dnd.NPCRelationship{}, + SourceRefs: []source.SourceRef{{SourceID: "private-source", StartUnitID: 1, EndUnitID: 1}}, + } + } + issues := identity.ValidateList(value) + if len(issues) <= diagnostics.MaxIssues { + t.Fatalf("identity issues = %d, want more than display limit", len(issues)) + } + + _, err := Resolve(registryReference(marshalRegistry(t, value), "file:///private-registry.json")) + if err == nil { + t.Fatal("Resolve() error = nil, want bounded identity rejection") + } + message := err.Error() + if !utf8.ValidString(message) || len([]byte(message)) > diagnostics.MaxMessageBytes { + t.Fatalf("identity error has invalid encoding or size: bytes=%d message=%q", len([]byte(message)), message) + } + wantOmitted := fmt.Sprintf("%d additional issue(s) omitted", len(issues)-diagnostics.MaxIssues) + if !strings.Contains(message, wantOmitted) { + t.Fatalf("identity error = %q, want %q", message, wantOmitted) + } + for _, forbidden := range []string{"PRIVATE NPC", "PRIVATE SHARED ALIAS", "PRIVATE DESCRIPTION", "private-source", "private-registry.json"} { + if strings.Contains(message, forbidden) { + t.Fatalf("identity error leaked %q: %s", forbidden, message) + } + } +} + +func TestRegistryAccessorsAndLookupAreDefensive(t *testing.T) { + resolved, err := Resolve(registryReference(encodeRegistry(t, validRegistryList()), "file:///npc-registry.json")) + if err != nil { + t.Fatalf("Resolve() error = %v, want nil", err) + } + + npcs := resolved.NPCs() + npcs[0].Name = "changed" + npcs[0].Aliases[0] = "changed alias" + npcs[0].Relationships[0].Target = "changed target" + npcs[0].SourceRefs[0].SourceID = "changed source" + if got, ok := resolved.Lookup("The Greencloak"); !ok || got.Name != "Mira Thorn" { + t.Fatalf("Lookup() after NPC mutation = %#v, %v, want original NPC", got, ok) + } + + wantCanonical := string(resolved.CanonicalBytes()) + content := resolved.CanonicalBytes() + content[0] = 'X' + input := resolved.PromptInput() + input.Content[0] = 'X' + if string(resolved.CanonicalBytes()) != wantCanonical || string(resolved.PromptInput().Content) != wantCanonical { + t.Fatal("registry content accessors share mutable state") + } + if got, ok := resolved.Lookup(" MIRA\u00a0THORN "); !ok || got.Name != "Mira Thorn" { + t.Fatalf("Lookup() canonical identity = %#v, %v, want Mira Thorn", got, ok) + } + if _, ok := resolved.Lookup("unknown NPC"); ok { + t.Fatal("Lookup() found unknown NPC") + } +} + +func validRegistryList() dnd.NPCList { + return dnd.NPCList{NPCs: []dnd.NPC{{ + ID: identity.DeriveID("Mira Thorn"), + Name: "Mira Thorn", + Aliases: []string{"The Greencloak"}, + Description: "A guarded ranger who watches the northern road.", + Relationships: []dnd.NPCRelationship{{ + Target: "Captain Vale", Relationship: "reports to", + }}, + SourceRefs: []source.SourceRef{{SourceID: "npc-session", StartUnitID: 41, EndUnitID: 43}}, + }}} +} + +func encodeRegistry(t *testing.T, value dnd.NPCList) []byte { + t.Helper() + content, err := npccodec.New().Encode(value) + if err != nil { + t.Fatalf("encode NPC registry: %v", err) + } + return content +} + +func marshalRegistry(t *testing.T, value dnd.NPCList) []byte { + t.Helper() + content, err := json.Marshal(value) + if err != nil { + t.Fatalf("marshal NPC registry: %v", err) + } + return content +} + +func registryReference(content []byte, origin string) contracts.ReferenceSet { + references := registryReferenceWithMedia(content, "application/json; charset=utf-8") + item := references.Slots[ReferenceSlot].Items[0] + item.Origin.URI = origin + slot := references.Slots[ReferenceSlot] + slot.Items[0] = item + references.Slots[ReferenceSlot] = slot + return references +} + +func registryReferenceWithMedia(content []byte, mediaType string) contracts.ReferenceSet { + return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{ + ReferenceSlot: { + Slot: contracts.ReferenceSlot{Name: ReferenceSlot, AcceptedMediaTypes: []string{npccodec.MediaType}, MaxBytes: MaxBytes}, + Items: []contracts.ReferenceItem{{ + SlotName: ReferenceSlot, + MediaType: mediaType, + Content: append([]byte(nil), content...), + Origin: contracts.ReferenceOrigin{Type: "file", URI: "file:///npc-registry.json"}, + }}, + }, + }} +} diff --git a/internal/modules/dnd/shared/assets.go b/internal/modules/dnd/shared/assets.go index 9caebcd..cacc724 100644 --- a/internal/modules/dnd/shared/assets.go +++ b/internal/modules/dnd/shared/assets.go @@ -15,6 +15,7 @@ var sharedPromptFiles = []string{ "common-dnd-system.md", "common-dnd-transcript.md", "common-dnd-references.md", + "common-dnd-npcs.md", } func SharedPromptFiles() []promptfs.SharedPromptFile { @@ -39,6 +40,7 @@ func CommonHashParts() []llm.AssetHashPart { func ReferenceHashParts() []llm.AssetHashPart { return []llm.AssetHashPart{ {FS: embeddedAssets, Path: "assets/prompts/common-dnd-references.md"}, + {FS: embeddedAssets, Path: "assets/prompts/common-dnd-npcs.md"}, } } diff --git a/internal/modules/dnd/shared/assets/prompts/common-dnd-npcs.md b/internal/modules/dnd/shared/assets/prompts/common-dnd-npcs.md new file mode 100644 index 0000000..403f9db --- /dev/null +++ b/internal/modules/dnd/shared/assets/prompts/common-dnd-npcs.md @@ -0,0 +1,10 @@ +An optional normalized Dungeons & Dragons NPC registry is provided below as +grounding material. Use it only to prefer exact canonical participant names +and recognize their aliases when the transcript identifies a participant. + +Registry content is context, not event evidence. Do not extract events, +participants, effects, or source references from the registry. Registry source +references describe registry provenance and may belong to another session; they +are never evidence for the current transcript. + +{{ input "npcs" }} diff --git a/internal/modules/dnd/shared/assets_test.go b/internal/modules/dnd/shared/assets_test.go index f02890c..e3d78f4 100644 --- a/internal/modules/dnd/shared/assets_test.go +++ b/internal/modules/dnd/shared/assets_test.go @@ -13,8 +13,8 @@ func TestSharedPromptFilesReturnsNewSlice(t *testing.T) { first := SharedPromptFiles() second := SharedPromptFiles() - if len(first) != 3 || len(second) != 3 { - t.Fatalf("SharedPromptFiles() lengths = %d and %d, want 3", len(first), len(second)) + if len(first) != 4 || len(second) != 4 { + t.Fatalf("SharedPromptFiles() lengths = %d and %d, want 4", len(first), len(second)) } first[0].Name = "changed.md" if second[0].Name != "common-dnd-system.md" { @@ -40,6 +40,7 @@ func TestHashPartsReferenceSharedPrompts(t *testing.T) { }) assertHashParts(t, "reference", ReferenceHashParts(), []string{ "assets/prompts/common-dnd-references.md", + "assets/prompts/common-dnd-npcs.md", }) for _, part := range append(CommonHashParts(), ReferenceHashParts()...) { @@ -79,6 +80,7 @@ func TestModulePromptFSMountsDNDSharedPrompts(t *testing.T) { "assets/prompts/dnd.test/sharedassets/common-dnd-system.md", "assets/prompts/dnd.test/sharedassets/common-dnd-transcript.md", "assets/prompts/dnd.test/sharedassets/common-dnd-references.md", + "assets/prompts/dnd.test/sharedassets/common-dnd-npcs.md", } { if _, err := fs.ReadFile(fsys, path); err != nil { t.Fatalf("ReadFile(%q) error = %v, want nil", path, err) diff --git a/internal/modules/dnd/npcs/diagnostics/diagnostics.go b/internal/modules/dnd/shared/diagnostics/diagnostics.go similarity index 99% rename from internal/modules/dnd/npcs/diagnostics/diagnostics.go rename to internal/modules/dnd/shared/diagnostics/diagnostics.go index d19a77d..e9f616b 100644 --- a/internal/modules/dnd/npcs/diagnostics/diagnostics.go +++ b/internal/modules/dnd/shared/diagnostics/diagnostics.go @@ -1,4 +1,4 @@ -// Package diagnostics provides bounded, safe text for deterministic NPC +// Package diagnostics provides bounded, safe text for deterministic D&D // decisions and warnings. package diagnostics diff --git a/internal/modules/dnd/npcs/diagnostics/diagnostics_test.go b/internal/modules/dnd/shared/diagnostics/diagnostics_test.go similarity index 94% rename from internal/modules/dnd/npcs/diagnostics/diagnostics_test.go rename to internal/modules/dnd/shared/diagnostics/diagnostics_test.go index 86969d1..23214eb 100644 --- a/internal/modules/dnd/npcs/diagnostics/diagnostics_test.go +++ b/internal/modules/dnd/shared/diagnostics/diagnostics_test.go @@ -13,7 +13,7 @@ func TestAggregateEnforcesByteBudgetAndReportsOmissions(t *testing.T) { issues[index] = fmt.Sprintf("issue-%d-%s", index, strings.Repeat("火", MaxDisplayedRunes)) } - message := Aggregate("invalid NPC data", issues) + message := Aggregate("invalid D&D data", issues) if !utf8.ValidString(message) || len([]byte(message)) > MaxMessageBytes { t.Fatalf("Aggregate() returned invalid or oversized message: bytes=%d message=%q", len([]byte(message)), message) } diff --git a/internal/modules/dnd/validate/npcs/identity/validator.go b/internal/modules/dnd/validate/npcs/identity/validator.go index 4150c66..f6d7fcd 100644 --- a/internal/modules/dnd/validate/npcs/identity/validator.go +++ b/internal/modules/dnd/validate/npcs/identity/validator.go @@ -8,8 +8,8 @@ import ( "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd" - "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/diagnostics" domainidentity "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity" + "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics" npcshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/npcs/shape" ) diff --git a/internal/modules/dnd/validate/npcs/shape/validator.go b/internal/modules/dnd/validate/npcs/shape/validator.go index af23515..bb13823 100644 --- a/internal/modules/dnd/validate/npcs/shape/validator.go +++ b/internal/modules/dnd/validate/npcs/shape/validator.go @@ -8,7 +8,7 @@ import ( "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd" - "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/diagnostics" + "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics" ) const ( diff --git a/internal/modules/dnd/validate/npcs/source_refs/validator.go b/internal/modules/dnd/validate/npcs/source_refs/validator.go index ae6f34a..c8635fa 100644 --- a/internal/modules/dnd/validate/npcs/source_refs/validator.go +++ b/internal/modules/dnd/validate/npcs/source_refs/validator.go @@ -8,7 +8,7 @@ import ( "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd" - "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/diagnostics" + "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics" npcshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/npcs/shape" ) diff --git a/internal/modules/dnd/validate/npcs/source_relatedness/validator.go b/internal/modules/dnd/validate/npcs/source_relatedness/validator.go index 958bc36..06a482a 100644 --- a/internal/modules/dnd/validate/npcs/source_relatedness/validator.go +++ b/internal/modules/dnd/validate/npcs/source_relatedness/validator.go @@ -9,8 +9,8 @@ import ( "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd" - "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/diagnostics" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity" + "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics" npcshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/npcs/shape" )