Compare commits

...

6 Commits

36 changed files with 2034 additions and 93 deletions

View File

@@ -296,6 +296,7 @@ production validators do not call the LLM and must not set `llm_profile`.
| `generic/valid_json` | deterministic | Rejects payloads that are not syntactically valid JSON. | | `generic/valid_json` | deterministic | Rejects payloads that are not syntactically valid JSON. |
| `generic/valid_json_schema` | deterministic | Rejects invalid JSON or JSON that does not conform to the module response schema. | | `generic/valid_json_schema` | deterministic | Rejects invalid JSON or JSON that does not conform to the module response schema. |
| `extract/dnd/spells/shape` | deterministic | Rejects malformed D&D spell-list artifacts. | | `extract/dnd/spells/shape` | deterministic | Rejects malformed D&D spell-list artifacts. |
| `extract/dnd/spells/catalog` | deterministic | Rejects spell-list artifacts containing names outside the effective SRD and overlay catalog. |
| `extract/dnd/spells/source_refs` | deterministic | Rejects missing or invalid D&D spell source references. | | `extract/dnd/spells/source_refs` | deterministic | Rejects missing or invalid D&D spell source references. |
| `extract/dnd/spells/source_relatedness` | deterministic | Emits warnings when a spell name is not found near its cited source text. | | `extract/dnd/spells/source_relatedness` | deterministic | Emits warnings when a spell name is not found near its cited source text. |
@@ -306,6 +307,7 @@ validators:
- generic/valid_json - generic/valid_json
- generic/valid_json_schema - generic/valid_json_schema
- extract/dnd/spells/shape - extract/dnd/spells/shape
- extract/dnd/spells/catalog
- extract/dnd/spells/source_refs - extract/dnd/spells/source_refs
- extract/dnd/spells/source_relatedness - extract/dnd/spells/source_relatedness
``` ```
@@ -325,16 +327,20 @@ declares optional `players`, `party`, and `glossary` references for scene
disambiguation, and accepts `roster` as a deprecated compatibility alias for disambiguation, and accepts `roster` as a deprecated compatibility alias for
`party`. `party`.
The `dnd/spells` extractor declares optional reference slots: The `dnd/spells` extractor declares optional campaign reference slots:
- `players` - `players`
- `party` - `party`
- `glossary` - `glossary`
- `roster` as a deprecated compatibility alias for `party` - `roster` as a deprecated compatibility alias for `party`
Both modules accept UTF-8 plain text, Markdown, YAML, or JSON reference files. These campaign slots accept UTF-8 plain text, Markdown, YAML, or JSON reference
The extractor uses references only as supporting disambiguation material; spell files. The extractor also declares an optional `spell_catalog` slot that accepts
casts still must be present in the source transcript. one UTF-8 `application/json` overlay bundle no larger than 1 MiB. The slot does
not allow multiple files. Its format is defined in the
[spell-catalog overlay contract](integrations/dnd-spell-catalog-overlays.md).
The extractor uses campaign references only as supporting disambiguation
material; spell casts still must be present in the source transcript.
## State Surfaces ## State Surfaces

View File

@@ -51,7 +51,10 @@ Each spell cast contains exactly these required fields:
IDs and evidence unit ranges. It must contain at least one entry. IDs and evidence unit ranges. It must contain at least one entry.
All four string fields must be non-empty. `caster` is the in-world caster, not All four string fields must be non-empty. `caster` is the in-world caster, not
the transcript speaker. Unknown fields are rejected. the transcript speaker. The `spell` value must resolve through the effective
SRD-plus-overlay catalog as either a canonical name or alias. Catalog
validation accepts aliases but does not rewrite them; unknown fields are
rejected.
## Source References ## Source References
@@ -81,10 +84,18 @@ manifest metadata:
"response_schema_id": "notarius.dnd.spells", "response_schema_id": "notarius.dnd.spells",
"response_schema_name": "notarius_dnd_spells_v1", "response_schema_name": "notarius_dnd_spells_v1",
"response_schema_version": "v1", "response_schema_version": "v1",
"response_schema_sha256": "sha256:..." "response_schema_sha256": "sha256:...",
"catalog_base_id": "dnd-5e-2014-srd-spells",
"catalog_digest": "sha256:...",
"catalog_overlay_ids": ["campaign.example"]
} }
} }
} }
``` ```
Raw prompt and schema content are not included in manifest metadata. `catalog_digest` identifies the effective semantic catalog, while
`catalog_overlay_ids` is sorted and empty for a base-only configuration. Raw
prompt, schema, catalog, alias, and local overlay-file content are not
included in manifest metadata. Overlay origin, media type, byte size, and raw
digest are recorded separately in the manifest's reference provenance; see
the [JSON output contract](json-output.md#manifestjson).

View File

@@ -0,0 +1,70 @@
# D&D Spell-Catalog Overlay Contract
This document defines the JSON format accepted by the D&D spell catalog
resolver. An overlay supplies campaign-specific spell names and aliases for
recognition. It does not supply spell rules, levels, classes, effects, or
source evidence.
The `dnd/spells` extractor accepts one optional UTF-8 `application/json` overlay
bundle through its `spell_catalog` reference slot. The framework materializes
that file relative to the configuration or command-line binding, enforces the
1 MiB slot limit, and records its origin and raw digest separately from the
effective catalog digest.
## Shape
An overlay bundle has this shape:
```json
{
"schema_version": "notarius.dnd.spell-catalog-overlay.v1",
"catalogs": [
{
"id": "campaign.example",
"ruleset": "dnd-5e-2014",
"source": {
"title": "Example campaign spells",
"version": "1",
"url": "",
"license": ""
},
"spells": [
{
"name": "Aegis of Emberfall",
"aliases": ["Emberfall Aegis"]
}
]
}
]
}
```
The top-level `schema_version` and `catalogs` fields are required. The schema
version must be exactly `notarius.dnd.spell-catalog-overlay.v1`, and at least
one catalog is required. Catalogs require a unique, non-empty, trimmed `id`,
the exact `dnd-5e-2014` `ruleset`, a `source`, and a non-empty `spells` array.
`source.title` is required and must be non-empty and trimmed. `source.version`,
`source.url`, and `source.license` are optional strings and may be empty.
Each spell requires a non-empty, trimmed `name`. `aliases` may be omitted or
may be an array of trimmed, non-empty strings; JSON `null` is not an alias
array. Overlay objects contain no other supported spell fields.
Decoding is strict: unknown fields, malformed JSON, trailing JSON values, and
non-string optional source fields are rejected.
## Composition
The resolver always starts with the embedded D&D 5e 2014 SRD catalog. Overlay
catalogs are sorted by `id` before composition, so the input order does not
affect the result. A new canonical name adds a recognition entry. A canonical
name matching an existing canonical name augments that spell and keeps the
established canonical display spelling. Repeated aliases for the same spell
are idempotent.
Canonical-name display conflicts and canonical/alias or alias/alias collisions
between different spells are errors, including collisions with the embedded
catalog. Canonical names and aliases use the catalog's case, whitespace, and
common-apostrophe normalization rules. The effective catalog returns canonical
names in sorted order and produces a semantic SHA-256 digest that is stable
under JSON formatting, object-key, catalog, spell, and alias reordering.

View File

@@ -103,6 +103,13 @@ return defensive copies, and expose a diagnostics map that omits schema bytes.
The small framework registry contains only generic test schemas; production The small framework registry contains only generic test schemas; production
schemas remain package-owned. schemas remain package-owned.
The spell extractor's package-owned prompt declares a required
`application/json` `spell_catalog` input. The extractor generates that input
from its prepared effective catalog as `{"spell_names":[...]}` using sorted
canonical names only. Its input digest covers those generated bytes; manifests
record catalog identity and digest rather than names, aliases, overlay bytes,
or source metadata.
## Debug And Redaction Boundaries ## Debug And Redaction Boundaries
The pipeline may wrap the client with a debug recorder that captures prepared The pipeline may wrap the client with a debug recorder that captures prepared

View File

@@ -19,11 +19,14 @@ validator chains and prompt asset collection.
Production input, chunk, output, and D&D spell-extract packages register strict Production input, chunk, output, and D&D spell-extract packages register strict
option decoders and run-local builders. Preparation decodes their options into option decoders and run-local builders. Preparation decodes their options into
implementation-owned values and injects dependencies. The spell extractor is implementation-owned values and injects dependencies plus the materialized
typed over the canonical D&D model. D&D validators, merge, and normalize use reference set for the selected target. Each builder receives an isolated clone
typed variants; JSON representation validators use serialized requests; and of that set; input and output builders receive no references. The spell
unconditional validators expose separate chunk and typed variants. The D&D extractor is typed over the canonical D&D model. D&D validators, merge, and
production registrar registers only the canonical typed spell implementations. normalize use typed variants; JSON representation validators use serialized
requests; and unconditional validators expose separate chunk and typed
variants. The D&D production registrar registers only the canonical typed spell
implementations.
Prepared extractors, extract validators, and codecs may be reused concurrently Prepared extractors, extract validators, and codecs may be reused concurrently
by the run-wide extract pool. Production implementations are immutable after by the run-wide extract pool. Production implementations are immutable after
@@ -67,6 +70,17 @@ license details live beside the asset in `SOURCES.md`. This domain-owned data is
separate from `internal/modules/dnd/shared`, which is reserved for reusable separate from `internal/modules/dnd/shared`, which is reserved for reusable
prompt and source-reference machinery. prompt and source-reference machinery.
`ResolveEffectiveCatalog` builds the immutable recognition view used by the
spell extractor and catalog validator. It starts with the embedded SRD catalog
and optionally applies one strict JSON overlay from the `spell_catalog` item in
a materialized reference set. Overlay catalogs are ordered by ID, may add names
and aliases, and may augment an existing canonical spell without replacing its
display name. Cross-spell lookup collisions are errors. The effective view
exposes sorted canonical names, normalized lookup, overlay identities, and a
semantic digest; overlay content remains contextual reference material rather
than source evidence. Its external JSON contract is defined in the
[spell-catalog overlay contract](../integrations/dnd-spell-catalog-overlays.md).
## Input Adapter ## Input Adapter
### `internal/modules/seriatim/input/transcript` ### `internal/modules/seriatim/input/transcript`
@@ -139,7 +153,12 @@ earliest cited unit, and returns `dnd.SpellList`.
The extractor owns its private model-response DTO, embedded prompt, LLM response The extractor owns its private model-response DTO, embedded prompt, LLM response
schema, strict option decoder, injected shared LLM client, and prompt/schema schema, strict option decoder, injected shared LLM client, and prompt/schema
manifest metadata. The separate `internal/modules/dnd/codec/spells` package manifest metadata. During preparation it resolves the optional `spell_catalog`
reference into an immutable effective catalog and adds a generated
canonical-name-only JSON input to every structured completion request. Overlay
failures therefore stop construction before source parsing or an LLM call;
campaign references remain separate disambiguation inputs and never become
source evidence. The separate `internal/modules/dnd/codec/spells` package
owns the durable schema and stable JSON representation for artifact kind owns the durable schema and stable JSON representation for artifact kind
`dnd/spell-list`. The runner keeps the result typed through validators and later `dnd/spell-list`. The runner keeps the result typed through validators and later
stages, using the codec only for checkpoint, debug, and output boundaries. stages, using the codec only for checkpoint, debug, and output boundaries.
@@ -192,8 +211,12 @@ codec bytes according to its target context. Neither validator calls the LLM.
## D&D Spell Validators ## D&D Spell Validators
All three validators receive `dnd.SpellList` directly. The shape validator All four validators receive `dnd.SpellList` directly. The shape validator
rejects missing or empty spell fields and empty reference lists. The rejects missing or empty spell fields and empty reference lists. The catalog
validator defers when shape is invalid, then checks every non-empty spell name
against the immutable effective SRD and overlay catalog. It accepts normalized
canonical names and aliases without rewriting the artifact; unknown names
reject the complete result with bounded, stable index/name diagnostics. The
source-reference validator applies generic source-reference validation to every source-reference validator applies generic source-reference validation to every
cited range. The relatedness validator warns when a case-insensitive spell name cited range. The relatedness validator warns when a case-insensitive spell name
is absent from all cited source text. is absent from all cited source text.

View File

@@ -87,7 +87,7 @@ Configuration. The implemented module packages are:
| `internal/modules/dnd` | Owns the canonical D&D spell-list and spell-cast artifact types. | | `internal/modules/dnd` | Owns the canonical D&D spell-list and spell-cast artifact types. |
| `internal/modules/dnd/codec/spells` | Strictly decodes and stably encodes the durable D&D spell-list representation. | | `internal/modules/dnd/codec/spells` | Strictly decodes and stably encodes the durable D&D spell-list representation. |
| `internal/modules/dnd/extract/spells` | Maps private structured model output to canonical source-grounded D&D spell lists. | | `internal/modules/dnd/extract/spells` | Maps private structured model output to canonical source-grounded D&D spell lists. |
| `internal/modules/dnd/spells/catalog` | Embeds and validates the versioned D&D 5e 2014 SRD spell catalog and provides immutable lookup. | | `internal/modules/dnd/spells/catalog` | Embeds and validates the versioned D&D 5e 2014 SRD catalog, composes optional overlays, and provides immutable effective lookup. |
| `internal/modules/generic/merge/appendorder` | Combines accepted extraction results in chunk order. | | `internal/modules/generic/merge/appendorder` | Combines accepted extraction results in chunk order. |
| `internal/modules/generic/normalize/noop` | Preserves accepted merged output. | | `internal/modules/generic/normalize/noop` | Preserves accepted merged output. |
| `internal/modules/generic/output/json` | Encodes manifests, lane payloads, warnings, and rejections as logical JSON files. | | `internal/modules/generic/output/json` | Encodes manifests, lane payloads, warnings, and rejections as logical JSON files. |
@@ -101,7 +101,7 @@ Generic validators under `internal/modules/generic/validate` provide
unconditional test decisions, JSON syntax validation, and JSON Schema unconditional test decisions, JSON syntax validation, and JSON Schema
validation. D&D spell validators under `internal/modules/dnd/validate/spells` validation. D&D spell validators under `internal/modules/dnd/validate/spells`
consume the canonical spell-list type directly to provide shape, consume the canonical spell-list type directly to provide shape,
source-reference, and source-relatedness decisions. effective-catalog, source-reference, and source-relatedness decisions.
Production composition is grouped behind package-family registrars, and every Production composition is grouped behind package-family registrars, and every
implemented production extension uses its domain-first tree: implemented production extension uses its domain-first tree:

View File

@@ -53,10 +53,19 @@ against its resolved target declaration, reads and validates the file, and
builds both a `contracts.ReferenceSet` and provenance-only metadata on the builds both a `contracts.ReferenceSet` and provenance-only metadata on the
corresponding `ResolvedReferenceTarget`. corresponding `ResolvedReferenceTarget`.
The runner clones the resulting set into the chunk, extract, merge, or normalize Preparation delivers the materialized set for each target through
request that owns the target. LLM-backed extensions may convert those items into `pipeline.BuildRequest`: chunkers and chunk validators receive the chunk target;
named prompt inputs. Reference content remains separate from source evidence and extractors and extract validators receive the lane extract target; mergers and
source digests. merge validators receive the lane merge target; and normalizers and normalize
validators receive the lane normalize target. Input and output builders receive
an empty set because those stages cannot declare references. Every builder gets
an isolated deep clone of its target set, so construction-time mutation cannot
change another builder, the resolved pipeline, or later runtime requests.
The runner continues to clone the resulting set into the chunk, extract, merge,
or normalize request that owns the target. LLM-backed extensions may convert
those items into named prompt inputs. Reference content remains separate from
source evidence and source digests.
Binding precedence, path resolution, accepted content, and media-type behavior Binding precedence, path resolution, accepted content, and media-type behavior
are configuration contracts; see [Configuration](../config.md#pipelines). are configuration contracts; see [Configuration](../config.md#pipelines).

View File

@@ -2,9 +2,31 @@
## Status ## Status
The immediate baseline feature is accepted and pending implementation. The accepted baseline implementation is complete. It provides effective
Feedback-aware repair and semantic LLM validation are deferred until the SRD-plus-overlay catalog composition, canonical-name prompt grounding,
baseline has been evaluated. deterministic catalog validation, the maintained `retries: 2` policy, and
provenance and checkpoint identity coverage through the assembled offline
workflow.
External quality evaluation is pending. This repository contains the
maintained example and offline fake-LLM coverage, but no approved
human-reviewed transcript corpus or authorized live-model evaluation was
available for this implementation run. The maintained example can be run
from the repository root with:
```sh
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells-production.config.yml \
--input examples/seriatim-minimal-transcript.json \
--chunk_cache bypass
```
That command requires the configured Scriptorium profile credential. When an
approved profile and reviewed corpus are available, record anonymized
aggregate false positives, false negatives, unknown-name rejections,
acceptance by attempt, and model-call cost here without committing transcripts
or raw prompts. Feedback-aware repair and semantic LLM validation remain
deferred until that baseline evaluation is available.
## Objective ## Objective

View File

@@ -9,14 +9,9 @@ not as committed release dates.
### Solidify Spell Extraction ### Solidify Spell Extraction
- Implement the immediate overlay-capable catalog-grounding and - Evaluate the implemented baseline in
deterministic-validation baseline in [D&D Spell Extraction Quality](dnd-spell-extraction.md), then reconsider its
[D&D Spell Extraction Quality](dnd-spell-extraction.md), using ordinary [deferred retry and validation work](dnd-spell-extraction.md#deferred-retry-and-validation-work).
`retries: 2` behavior as the comparison point.
- After evaluating that baseline, reconsider the roadmap's deferred structured
diagnostics, retryability classification, repair-capable extractor contract,
hybrid repair/fresh retry policy, and narrowly scoped semantic LLM
validation.
- Replace the no-op spell normalizer with a D&D-specific implementation that - Replace the no-op spell normalizer with a D&D-specific implementation that
canonicalizes recognized spell names, performs only high-confidence repairs, canonicalizes recognized spell names, performs only high-confidence repairs,
retains uncertain values for review, and emits scoped warnings. retains uncertain values for review, and emits scoped warnings.

View File

@@ -2,9 +2,10 @@
## Status ## Status
Ready for implementation. Follow the stages in order. This plan covers the The accepted baseline implementation is complete through the assembled
accepted baseline in [D&D Spell Extraction Quality](dnd-spell-extraction.md), offline workflow. External quality evaluation remains pending as recorded in
not its deferred repair-aware retry or semantic LLM-validation work. [D&D Spell Extraction Quality](dnd-spell-extraction.md). This plan covers that
baseline, not its deferred repair-aware retry or semantic LLM-validation work.
This active plan replaces the completed test-suite implementation record that This active plan replaces the completed test-suite implementation record that
previously occupied this filename. That review remains documented in previously occupied this filename. That review remains documented in

View File

@@ -0,0 +1,21 @@
{
"schema_version": "notarius.dnd.spell-catalog-overlay.v1",
"catalogs": [
{
"id": "notarius.example-campaign",
"ruleset": "dnd-5e-2014",
"source": {
"title": "Notarius example campaign spell names",
"version": "1",
"url": "",
"license": ""
},
"spells": [
{
"name": "Aegis of Emberfall",
"aliases": ["Emberfall Aegis"]
}
]
}
]
}

View File

@@ -29,3 +29,5 @@ pipelines:
extract: extract:
module: dnd/spells module: dnd/spells
retries: 2 retries: 2
references:
spell_catalog: ./dnd-spells-catalog.json

View File

@@ -10,6 +10,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts" "gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/core/config" "gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/core/debugbundle" "gitea.maximumdirect.net/eric/notarius/internal/core/debugbundle"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
) )
@@ -18,9 +19,22 @@ func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
for _, example := range maintainedExampleFiles(t) { for _, example := range maintainedExampleFiles(t) {
t.Run(example.name, func(t *testing.T) { t.Run(example.name, func(t *testing.T) {
cfg := loadMaintainedExample(t, example.path) cfg := loadMaintainedExample(t, example.path)
if _, err := cfg.Resolve(resolveInputForMaintainedExample(components, "dnd-session")); err != nil { effective, err := cfg.Resolve(resolveInputForMaintainedExample(components, "dnd-session"))
if err != nil {
t.Fatalf("resolve maintained example: %v", err) t.Fatalf("resolve maintained example: %v", err)
} }
materialized, _, err := pipeline.MaterializeReferences(effective.ResolvedPipeline, catalogFromRegistries(components.registries), pipeline.ReferenceMaterializationOptions{
ConfigPath: example.path,
WorkingDir: filepath.Dir(example.path),
})
if err != nil {
t.Fatalf("materialize maintained example references: %v", err)
}
if example.name == "production" {
if len(materialized.ArtifactLanes) != 1 || len(materialized.ArtifactLanes[0].ExtractReferences.ReferenceSet.Slots["spell_catalog"].Items) != 1 {
t.Fatalf("production spell catalog reference was not materialized: %#v", materialized.ArtifactLanes)
}
}
var stdout, stderr strings.Builder var stdout, stderr strings.Builder
code := RunWithOptions([]string{"pipelines", "list", "--config", example.path}, &stdout, &stderr, productionOptionsFromComponents(components)) code := RunWithOptions([]string{"pipelines", "list", "--config", example.path}, &stdout, &stderr, productionOptionsFromComponents(components))
if code != 0 || stdout.String() != "dnd-session\n" || stderr.Len() != 0 { if code != 0 || stdout.String() != "dnd-session\n" || stderr.Len() != 0 {
@@ -69,6 +83,13 @@ func TestMaintainedMinimalInvocationProducesJSONBundle(t *testing.T) {
if lane.ID != "spells" || lane.Extractor != "dnd/spells" || lane.Merger != "appendorder" || lane.Normalizer != "noop" { if lane.ID != "spells" || lane.Extractor != "dnd/spells" || lane.Merger != "appendorder" || lane.Normalizer != "noop" {
t.Fatalf("manifest lane = %#v, want production spells composition", lane) t.Fatalf("manifest lane = %#v, want production spells composition", lane)
} }
if len(manifest.References) != 0 {
t.Fatalf("base-only manifest references = %#v, want no overlay provenance", manifest.References)
}
extractorMetadata, ok := lane.Metadata["extractor"].(map[string]any)
if !ok || len(stringValues(extractorMetadata["catalog_overlay_ids"])) != 0 {
t.Fatalf("base-only extractor metadata = %#v, want no overlay IDs", lane.Metadata)
}
artifact := readProductionJSON[dnd.SpellList](t, filepath.Join(runRoot, entry.File)) artifact := readProductionJSON[dnd.SpellList](t, filepath.Join(runRoot, entry.File))
if len(artifact.SpellCasts) != 1 || artifact.SpellCasts[0].Spell != "Cure Wounds" || artifact.SpellCasts[0].SourceRefs[0].SourceID != "session-alpha" { if len(artifact.SpellCasts) != 1 || artifact.SpellCasts[0].Spell != "Cure Wounds" || artifact.SpellCasts[0].SourceRefs[0].SourceID != "session-alpha" {

View File

@@ -38,6 +38,7 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
assertProductionContains(t, "normalizers", registries.Normalizers.RegisteredKeys(), []string{"noop"}) assertProductionContains(t, "normalizers", registries.Normalizers.RegisteredKeys(), []string{"noop"})
assertProductionContains(t, "outputs", registries.Outputs.RegisteredKeys(), []string{"json"}) assertProductionContains(t, "outputs", registries.Outputs.RegisteredKeys(), []string{"json"})
assertProductionContains(t, "validators", registries.Validators.RegisteredKeys(), []string{ assertProductionContains(t, "validators", registries.Validators.RegisteredKeys(), []string{
"extract/dnd/spells/catalog",
"extract/dnd/spells/shape", "extract/dnd/spells/shape",
"extract/dnd/spells/source_refs", "extract/dnd/spells/source_refs",
"extract/dnd/spells/source_relatedness", "extract/dnd/spells/source_relatedness",
@@ -54,6 +55,7 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
pipeline.Binding("generic/valid_json"), pipeline.Binding("generic/valid_json"),
pipeline.Binding("generic/valid_json_schema"), pipeline.Binding("generic/valid_json_schema"),
pipeline.Binding("extract/dnd/spells/shape"), pipeline.Binding("extract/dnd/spells/shape"),
pipeline.Binding("extract/dnd/spells/catalog"),
pipeline.Binding("extract/dnd/spells/source_refs"), pipeline.Binding("extract/dnd/spells/source_refs"),
pipeline.Binding("extract/dnd/spells/source_relatedness"), pipeline.Binding("extract/dnd/spells/source_relatedness"),
} }
@@ -70,6 +72,7 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
"dnd.scenes/sharedassets/common-dnd-transcript.md", "dnd.scenes/sharedassets/common-dnd-transcript.md",
"dnd.scenes/task.md", "dnd.scenes/task.md",
"dnd.spells/dnd.spells.yaml", "dnd.spells/dnd.spells.yaml",
"dnd.spells/catalog.md",
"dnd.spells/instructions.md", "dnd.spells/instructions.md",
"dnd.spells/sharedassets/common-dnd-references.md", "dnd.spells/sharedassets/common-dnd-references.md",
"dnd.spells/sharedassets/common-dnd-system.md", "dnd.spells/sharedassets/common-dnd-system.md",
@@ -123,6 +126,29 @@ func TestProductionPromptAssetsPrepareWithoutProviderCredentials(t *testing.T) {
} }
} }
func TestProductionSpellValidatorsPrepareFromMaterializedCatalog(t *testing.T) {
components := productionTestComponents(t)
configPath := repositoryPath("examples", "dnd-spells-production.config.yml")
effective, err := loadMaintainedExample(t, configPath).Resolve(resolveInputForMaintainedExample(components, "dnd-session"))
if err != nil {
t.Fatalf("resolve production spell configuration: %v", err)
}
materialized, _, err := pipeline.MaterializeReferences(effective.ResolvedPipeline, catalogFromRegistries(components.registries), pipeline.ReferenceMaterializationOptions{
ConfigPath: configPath,
WorkingDir: filepath.Dir(configPath),
})
if err != nil {
t.Fatalf("materialize production spell references: %v", err)
}
items := materialized.ArtifactLanes[0].ExtractReferences.ReferenceSet.Slots["spell_catalog"].Items
if len(items) != 1 || items[0].MediaType != "application/json" || len(items[0].Content) == 0 {
t.Fatalf("materialized spell catalog items = %#v, want one JSON item", items)
}
if _, err := pipeline.Prepare(materialized, components.registries, pipeline.ModuleDependencies{LLM: &productionFakeLLMClient{}}); err != nil {
t.Fatalf("prepare production spell pipeline from materialized catalog: %v", err)
}
}
func TestProductionLLMClientFactoriesBuildOfflineRuntime(t *testing.T) { func TestProductionLLMClientFactoriesBuildOfflineRuntime(t *testing.T) {
components := productionTestComponents(t) components := productionTestComponents(t)
factories := []struct { factories := []struct {
@@ -431,6 +457,7 @@ func readProductionJSON[T any](t *testing.T, path string) T {
type productionFakeLLMClient struct { type productionFakeLLMClient struct {
mu sync.Mutex mu sync.Mutex
requests []contracts.StructuredCompletionRequest requests []contracts.StructuredCompletionRequest
spellResponse string
} }
func (client *productionFakeLLMClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) { func (client *productionFakeLLMClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
@@ -442,7 +469,11 @@ func (client *productionFakeLLMClient) CompleteStructured(ctx context.Context, r
case scenes.PromptID: case scenes.PromptID:
content = []byte(`{"scenes":[{"start_unit_id":1,"end_unit_id":2,"short_title":"Opening scene","primary_mode":"Narrative","main_participants":["Aria"],"summary":"The session opens.","boundary_note":"The opening covers the available transcript.","boundary_confidence":"High"}],"boundary_caveats":["The opening boundary is inferred from the short transcript."]}`) content = []byte(`{"scenes":[{"start_unit_id":1,"end_unit_id":2,"short_title":"Opening scene","primary_mode":"Narrative","main_participants":["Aria"],"summary":"The session opens.","boundary_note":"The opening covers the available transcript.","boundary_confidence":"High"}],"boundary_caveats":["The opening boundary is inferred from the short transcript."]}`)
case spells.PromptID: case spells.PromptID:
if client.spellResponse != "" {
content = []byte(client.spellResponse)
} else {
content = []byte(`{"spell_casts":[{"caster":"Aria","spell":"Cure Wounds","effect":"Heals an injured ally.","narrative_description":"Aria restores the fighter after the fight.","source_refs":[{"source_id":"session-alpha","start_unit_id":1,"end_unit_id":1}]}]}`) content = []byte(`{"spell_casts":[{"caster":"Aria","spell":"Cure Wounds","effect":"Heals an injured ally.","narrative_description":"Aria restores the fighter after the fight.","source_refs":[{"source_id":"session-alpha","start_unit_id":1,"end_unit_id":1}]}]}`)
}
default: default:
return contracts.StructuredCompletionResponse{}, fmt.Errorf("unexpected prompt %q", req.PromptID) return contracts.StructuredCompletionResponse{}, fmt.Errorf("unexpected prompt %q", req.PromptID)
} }

View File

@@ -0,0 +1,255 @@
package cli
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/checkpoint"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
spellcatalog "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/spells/catalog"
)
func TestSpellCatalogBytesAffectCheckpointIdentityButNotSemanticDigest(t *testing.T) {
components := productionTestComponents(t)
configPath := repositoryPath("examples", "dnd-spells-production.config.yml")
effective, err := loadMaintainedExample(t, configPath).Resolve(resolveInputForMaintainedExample(components, "dnd-session"))
if err != nil {
t.Fatalf("resolve production configuration: %v", err)
}
overlayPath := filepath.Join(t.TempDir(), "catalog.json")
resolved := effective.ResolvedPipeline
bindings := resolved.ArtifactLanes[0].ExtractReferences.Bindings
catalogBindingIndex := -1
for index, binding := range bindings {
if binding.SlotName == spellcatalog.SpellCatalogReferenceSlot {
catalogBindingIndex = index
break
}
}
if catalogBindingIndex < 0 {
t.Fatalf("spell catalog bindings = %#v, want catalog binding", bindings)
}
resolved.ArtifactLanes[0].ExtractReferences.Bindings[catalogBindingIndex].Source = overlayPath
if err := os.WriteFile(overlayPath, []byte(reorderedOverlayA), 0o600); err != nil {
t.Fatal(err)
}
materializedA, _, err := pipeline.MaterializeReferences(resolved, catalogFromRegistries(components.registries), pipeline.ReferenceMaterializationOptions{ConfigPath: configPath, WorkingDir: filepath.Dir(configPath)})
if err != nil {
t.Fatalf("materialize first catalog: %v", err)
}
identityA := catalogCheckpointIdentity(t, materializedA)
metadataA := catalogExtractorMetadata(t, materializedA)
referenceA := catalogReference(t, materializedA)
if err := os.WriteFile(overlayPath, []byte(reorderedOverlayB), 0o600); err != nil {
t.Fatal(err)
}
materializedB, _, err := pipeline.MaterializeReferences(resolved, catalogFromRegistries(components.registries), pipeline.ReferenceMaterializationOptions{ConfigPath: configPath, WorkingDir: filepath.Dir(configPath)})
if err != nil {
t.Fatalf("materialize reordered catalog: %v", err)
}
identityB := catalogCheckpointIdentity(t, materializedB)
metadataB := catalogExtractorMetadata(t, materializedB)
referenceB := catalogReference(t, materializedB)
if identityA.Digest == identityB.Digest {
t.Fatalf("checkpoint identity digest = %q for both raw catalog files, want invalidation", identityA.Digest)
}
if referenceA.Digest == referenceB.Digest || referenceA.OriginURI != referenceB.OriginURI {
t.Fatalf("catalog reference provenance changed from %#v to %#v, want same origin and different raw digest", referenceA, referenceB)
}
digestA, ok := metadataA["catalog_digest"].(string)
if !ok {
t.Fatalf("first extractor catalog metadata = %#v, want digest", metadataA)
}
digestB, ok := metadataB["catalog_digest"].(string)
if !ok || digestA != digestB {
t.Fatalf("extractor catalog digests = %q and %q, want same semantic digest", digestA, digestB)
}
if got, want := metadataA["catalog_overlay_ids"], []string{"campaign.a", "campaign.b"}; !reflect.DeepEqual(got, want) || !reflect.DeepEqual(metadataB["catalog_overlay_ids"], want) {
t.Fatalf("extractor overlay IDs = %#v and %#v, want %#v", got, metadataB["catalog_overlay_ids"], want)
}
}
func TestConfiguredSpellCatalogBindingChangesResolvedPipelineIdentity(t *testing.T) {
base := string(readRepositoryFile(t, "examples", "dnd-spells-production.config.yml"))
changed := strings.Replace(base, "./dnd-spells-catalog.json", "./alternate-spell-catalog.json", 1)
if changed == base {
t.Fatal("production configuration did not contain the maintained catalog binding")
}
root := t.TempDir()
firstPath := filepath.Join(root, "first.yml")
secondPath := filepath.Join(root, "second.yml")
if err := os.WriteFile(firstPath, []byte(base), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(secondPath, []byte(changed), 0o600); err != nil {
t.Fatal(err)
}
components := productionTestComponents(t)
first, err := loadMaintainedExample(t, firstPath).Resolve(resolveInputForMaintainedExample(components, "dnd-session"))
if err != nil {
t.Fatalf("resolve first configuration: %v", err)
}
second, err := loadMaintainedExample(t, secondPath).Resolve(resolveInputForMaintainedExample(components, "dnd-session"))
if err != nil {
t.Fatalf("resolve changed configuration: %v", err)
}
if first.ResolvedPipeline.Digest == second.ResolvedPipeline.Digest {
t.Fatalf("resolved pipeline digest = %q for different catalog bindings, want change", first.ResolvedPipeline.Digest)
}
}
func TestMaintainedProductionOverlayRunAlignsGroundingValidationAndProvenance(t *testing.T) {
outputRoot := filepath.Join(t.TempDir(), "output")
fake := &productionFakeLLMClient{spellResponse: productionSpellResponse("Aegis of Emberfall")}
options := productionRunOptions(t, fake)
var stdout, stderr strings.Builder
code := RunWithOptions([]string{
"run", "dnd-session",
"--config", repositoryPath("examples", "dnd-spells-production.config.yml"),
"--input", repositoryPath("examples", "seriatim-minimal-transcript.json"),
"--only", "spells", "--chunk_cache", "bypass", "--output-dir", outputRoot,
}, &stdout, &stderr, options)
if code != 0 {
t.Fatalf("code=%d stdout=%q stderr=%q", code, stdout.String(), stderr.String())
}
runRoot := filepath.Join(outputRoot, productionRunID)
manifest := readProductionJSON[artifacts.RunManifest](t, filepath.Join(runRoot, "manifest.json"))
if manifest.ValidationStatus != "approved" || len(manifest.References) == 0 || len(manifest.ArtifactLanes) != 1 {
t.Fatalf("manifest = %#v, want approved overlay run with one lane and references", manifest)
}
lane := manifest.ArtifactLanes[0]
extractorMetadata, ok := lane.Metadata["extractor"].(map[string]any)
if !ok {
t.Fatalf("lane metadata = %#v, want extractor metadata", lane.Metadata)
}
if extractorMetadata["catalog_base_id"] != spellcatalog.SRD5E2014ID || !strings.HasPrefix(stringValue(extractorMetadata["catalog_digest"]), "sha256:") {
t.Fatalf("extractor catalog metadata = %#v, want base ID and semantic digest", extractorMetadata)
}
if got := stringValues(extractorMetadata["catalog_overlay_ids"]); !reflect.DeepEqual(got, []string{"notarius.example-campaign"}) {
t.Fatalf("catalog overlay IDs = %#v, want maintained overlay", got)
}
var catalogProvenance *artifacts.ReferenceProvenance
for index := range manifest.References {
reference := &manifest.References[index]
if reference.SlotName == spellcatalog.SpellCatalogReferenceSlot {
catalogProvenance = reference
break
}
}
if catalogProvenance == nil {
t.Fatalf("manifest references = %#v, want spell catalog provenance", manifest.References)
}
overlayBytes := readRepositoryFile(t, "examples", "dnd-spells-catalog.json")
if catalogProvenance.Stage != "extract" || catalogProvenance.LaneID != "spells" || catalogProvenance.OriginType != "file" || catalogProvenance.MediaType != "application/json" || catalogProvenance.SizeBytes != int64(len(overlayBytes)) || catalogProvenance.Digest != digestBytes(overlayBytes) || !strings.Contains(catalogProvenance.OriginURI, "dnd-spells-catalog.json") {
t.Fatalf("catalog provenance = %#v, want extract origin, media, size, and raw digest", catalogProvenance)
}
manifestBytes, err := json.Marshal(manifest)
if err != nil {
t.Fatal(err)
}
for _, leaked := range []string{"Aegis of Emberfall", "Emberfall Aegis", "Notarius example campaign spell names"} {
if strings.Contains(string(manifestBytes), leaked) {
t.Fatalf("manifest leaked overlay content %q", leaked)
}
}
requests := fake.requestsFor(spells.PromptID)
if len(requests) != 1 {
t.Fatalf("spell requests = %d, want one", len(requests))
}
catalogInput, ok := requests[0].Inputs[spellcatalog.SpellCatalogReferenceSlot]
if !ok || !strings.Contains(string(catalogInput.Content), "Aegis of Emberfall") || strings.Contains(string(catalogInput.Content), "Emberfall Aegis") {
t.Fatalf("spell catalog prompt input = %#v, want canonical overlay name without alias", catalogInput)
}
artifact := readProductionJSON[dnd.SpellList](t, filepath.Join(runRoot, "lanes", "spells.json"))
if len(artifact.SpellCasts) != 1 || artifact.SpellCasts[0].Spell != "Aegis of Emberfall" {
t.Fatalf("artifact = %#v, want accepted overlay-only canonical spell", artifact)
}
rejected := readProductionJSON[struct {
Rejected []json.RawMessage `json:"rejected"`
}](t, filepath.Join(runRoot, "rejected.json"))
if len(rejected.Rejected) != 0 {
t.Fatalf("rejected = %#v, want no rejected output", rejected.Rejected)
}
}
func catalogCheckpointIdentity(t *testing.T, resolved pipeline.ResolvedPipeline) checkpoint.Identity {
t.Helper()
identity, err := checkpoint.NewIdentity(checkpoint.IdentityInput{
Pipeline: resolved,
InputKey: resolved.Input.Module,
RawInputDigest: "sha256:catalog-test-input",
References: pipeline.ReferenceProvenance(resolved),
})
if err != nil {
t.Fatalf("create checkpoint identity: %v", err)
}
return identity
}
func catalogExtractorMetadata(t *testing.T, resolved pipeline.ResolvedPipeline) map[string]any {
t.Helper()
lane := resolved.ArtifactLanes[0]
extractor, err := spells.New(&productionFakeLLMClient{}, spells.Options{}, lane.ExtractReferences.ReferenceSet)
if err != nil {
t.Fatalf("construct extractor: %v", err)
}
return extractor.ManifestMetadata()
}
func catalogReference(t *testing.T, resolved pipeline.ResolvedPipeline) artifacts.ReferenceProvenance {
t.Helper()
for _, reference := range pipeline.ReferenceProvenance(resolved) {
if reference.SlotName == spellcatalog.SpellCatalogReferenceSlot && reference.Stage == "extract" && reference.LaneID == "spells" {
return reference
}
}
t.Fatalf("resolved references = %#v, want spell catalog provenance", pipeline.ReferenceProvenance(resolved))
return artifacts.ReferenceProvenance{}
}
func stringValue(value any) string {
result, _ := value.(string)
return result
}
func stringValues(value any) []string {
raw, err := json.Marshal(value)
if err != nil {
return nil
}
var values []string
if err := json.Unmarshal(raw, &values); err != nil {
return nil
}
return values
}
func digestBytes(value []byte) string {
sum := sha256.Sum256(value)
return "sha256:" + hex.EncodeToString(sum[:])
}
const reorderedOverlayA = `{
"schema_version": "notarius.dnd.spell-catalog-overlay.v1",
"catalogs": [
{"id":"campaign.a","ruleset":"dnd-5e-2014","source":{"title":"Campaign A"},"spells":[{"name":"Aegis of Emberfall","aliases":["Emberfall Aegis"]}]},
{"id":"campaign.b","ruleset":"dnd-5e-2014","source":{"title":"Campaign B"},"spells":[{"name":"Cinder Veil","aliases":["Veil of Cinder","Cinder Shroud"]}]}
]
}`
const reorderedOverlayB = `{"catalogs":[{"spells":[{"aliases":["Cinder Shroud","Veil of Cinder"],"name":"Cinder Veil"}],"source":{"title":"Campaign B"},"ruleset":"dnd-5e-2014","id":"campaign.b"},{"spells":[{"aliases":["Emberfall Aegis"],"name":"Aegis of Emberfall"}],"source":{"title":"Campaign A"},"ruleset":"dnd-5e-2014","id":"campaign.a"}],"schema_version":"notarius.dnd.spell-catalog-overlay.v1"}`

View File

@@ -0,0 +1,161 @@
package cli
import (
"context"
"encoding/json"
"fmt"
"sync"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
)
func TestProductionSpellCatalogValidationRetries(t *testing.T) {
const retries = 2
tests := []struct {
name string
responses []string
wantCalls int
wantRejected bool
wantSpell string
wantWarningCode string
}{
{
name: "unknown spell remains rejected after exhaustion",
responses: []string{
productionSpellResponse("Unknown Spell"),
productionSpellResponse("Unknown Spell"),
productionSpellResponse("Unknown Spell"),
},
wantCalls: retries + 1,
wantRejected: true,
},
{
name: "overlay spell becomes valid on retry",
responses: []string{
productionSpellResponse("Unknown Spell"),
productionSpellResponse("Aegis of Emberfall"),
},
wantCalls: 2,
wantSpell: "Aegis of Emberfall",
wantWarningCode: "spell_not_near_source",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
components := productionTestComponents(t)
configPath := repositoryPath("examples", "dnd-spells-production.config.yml")
cfg := loadMaintainedExample(t, configPath)
effective, err := cfg.Resolve(config.ResolveInput{PipelineID: "dnd-session", Catalog: catalogFromRegistries(components.registries)})
if err != nil {
t.Fatalf("resolve production configuration: %v", err)
}
materialized, _, err := pipeline.MaterializeReferences(effective.ResolvedPipeline, catalogFromRegistries(components.registries), pipeline.ReferenceMaterializationOptions{
ConfigPath: configPath,
WorkingDir: repositoryPath("examples"),
})
if err != nil {
t.Fatalf("materialize production references: %v", err)
}
materialized.ArtifactLanes[0].Extract.Retries = retries
llmClient := &catalogRetryLLMClient{responses: tt.responses}
prepared, err := pipeline.Prepare(materialized, components.registries, pipeline.ModuleDependencies{LLM: llmClient})
if err != nil {
t.Fatalf("prepare production pipeline: %v", err)
}
output, err := pipeline.New().Run(context.Background(), pipeline.RunInput{
Prepared: prepared,
RawInput: readRepositoryFile(t, "examples", "seriatim-minimal-transcript.json"),
ChunkCacheMode: pipeline.ChunkCacheBypass,
})
if err != nil {
t.Fatalf("Run() error = %v, want nil", err)
}
if calls := llmClient.CallCount(); calls > retries+1 || calls != tt.wantCalls {
t.Fatalf("LLM calls = %d, want %d and no more than %d", calls, tt.wantCalls, retries+1)
}
if tt.wantRejected {
if len(output.Rejected) != 1 || len(output.NormalizeOutputs) != 0 {
t.Fatalf("rejected = %#v normalized = %#v, want one nonfatal rejection and no merge output", output.Rejected, output.NormalizeOutputs)
}
rejection := output.Rejected[0]
if rejection.ReasonCode != "unknown_spell" || rejection.AttemptCount != retries+1 {
t.Fatalf("rejection = %#v, want exhausted unknown-spell rejection", rejection)
}
if len(output.Warnings) != 0 {
t.Fatalf("warnings = %#v, want no warnings from rejected attempts", output.Warnings)
}
return
}
if len(output.Rejected) != 0 || len(output.NormalizeOutputs) != 1 {
t.Fatalf("rejected = %#v normalized = %#v, want only accepted output", output.Rejected, output.NormalizeOutputs)
}
var value dnd.SpellList
if err := json.Unmarshal(output.NormalizeOutputs[0].Artifact.Content, &value); err != nil {
t.Fatalf("decode normalized spell list: %v", err)
}
if len(value.SpellCasts) != 1 || value.SpellCasts[0].Spell != tt.wantSpell {
t.Fatalf("normalized spell list = %#v, want accepted overlay spell", value)
}
if len(output.Warnings) != 1 || output.Warnings[0].ReasonCode != tt.wantWarningCode {
t.Fatalf("warnings = %#v, want only accepted-attempt warning", output.Warnings)
}
})
}
}
type catalogRetryLLMClient struct {
mu sync.Mutex
responses []string
calls int
}
func (client *catalogRetryLLMClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
if err := ctx.Err(); err != nil {
return contracts.StructuredCompletionResponse{}, err
}
if req.PromptID != spells.PromptID {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("unexpected prompt %q", req.PromptID)
}
client.mu.Lock()
index := client.calls
client.calls++
client.mu.Unlock()
if index >= len(client.responses) {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("missing fake response %d", index)
}
content := []byte(client.responses[index])
if err := json.Unmarshal(content, out); err != nil {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("populate fake structured target: %w", err)
}
return contracts.StructuredCompletionResponse{Content: content, Provider: "test", Model: "deterministic", ProfileID: req.ProfileID}, nil
}
func (client *catalogRetryLLMClient) CallCount() int {
client.mu.Lock()
defer client.mu.Unlock()
return client.calls
}
func productionSpellResponse(name string) string {
content, err := json.Marshal(dnd.SpellList{SpellCasts: []dnd.SpellCast{{
Caster: "Aria",
Spell: name,
Effect: "The spell takes effect.",
NarrativeDescription: "Aria casts the spell.",
SourceRefs: []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 1}},
}}})
if err != nil {
panic(err)
}
return string(content)
}

View File

@@ -19,6 +19,7 @@ type ModuleDependencies struct {
type BuildRequest struct { type BuildRequest struct {
Dependencies ModuleDependencies Dependencies ModuleDependencies
Options map[string]any Options map[string]any
References contracts.ReferenceSet
} }
// OptionValidator validates one module binding without constructing it. // OptionValidator validates one module binding without constructing it.
@@ -39,6 +40,7 @@ func cloneBuildRequest(request BuildRequest) BuildRequest {
return BuildRequest{ return BuildRequest{
Dependencies: request.Dependencies, Dependencies: request.Dependencies,
Options: cloneOptions(request.Options), Options: cloneOptions(request.Options),
References: CloneReferenceSet(request.References),
} }
} }

View File

@@ -72,6 +72,66 @@ func TestPrepareConstructsEverythingInStableOrder(t *testing.T) {
} }
} }
func TestPrepareDeliversTargetReferencesAsIndependentBuildInputs(t *testing.T) {
var built []string
var observations []constructionBuildObservation
registries, input := constructionRegistriesWithHooks(t, &built, nil,
func(name string, request BuildRequest) {
observations = append(observations, constructionBuildObservation{Name: name, Request: request})
},
func(name string, request *BuildRequest) {
if name != "extract" {
return
}
slot := request.References.Slots["extract"]
slot.Items[0].Content = []byte("mutated by extractor builder")
request.References.Slots["extract"] = slot
},
)
resolved, err := ResolvePipeline(constructionProfile(), ResolveOptions{}, registries.catalog())
if err != nil {
t.Fatalf("ResolvePipeline() error = %v, want nil", err)
}
resolved.ChunkReferences.ReferenceSet = constructionReferenceSet("chunk", "chunk reference")
resolved.ArtifactLanes[0].ExtractReferences.ReferenceSet = constructionReferenceSet("extract", "extract reference")
resolved.ArtifactLanes[0].MergeReferences.ReferenceSet = constructionReferenceSet("merge", "merge reference")
resolved.ArtifactLanes[0].NormalizeReferences.ReferenceSet = constructionReferenceSet("normalize", "normalize reference")
prepared, err := Prepare(resolved, registries, ModuleDependencies{})
if err != nil {
t.Fatalf("Prepare() error = %v, want nil", err)
}
wantNames := []string{"input", "chunk", "validator", "extract", "validator", "merge", "validator", "normalize", "validator", "output"}
if !reflect.DeepEqual(built, wantNames) {
t.Fatalf("construction order = %#v, want %#v", built, wantNames)
}
wantContents := []string{"", "chunk reference", "chunk reference", "extract reference", "extract reference", "merge reference", "merge reference", "normalize reference", "normalize reference", ""}
if len(observations) != len(wantContents) {
t.Fatalf("observed %d build requests, want %d", len(observations), len(wantContents))
}
for i, want := range wantContents {
if got := constructionReferenceContent(observations[i].Request.References); got != want {
t.Errorf("build request %d (%s) reference content = %q, want %q", i, observations[i].Name, got, want)
}
}
if got := constructionReferenceContent(resolved.ArtifactLanes[0].ExtractReferences.ReferenceSet); got != "extract reference" {
t.Fatalf("resolved extract references = %q, want original content", got)
}
_, err = prepared.lanes[0].typed.extract(context.Background(), prepared.lanes[0].typed.extractor, contracts.TypedExtractionRequest{
References: CloneReferenceSet(resolved.ArtifactLanes[0].ExtractReferences.ReferenceSet),
})
if err != nil {
t.Fatalf("prepared extractor operation error = %v, want nil", err)
}
if len(input.extractRequests) != 1 {
t.Fatalf("runtime extraction requests = %d, want one", len(input.extractRequests))
}
if got := constructionReferenceContent(input.extractRequests[0].References); got != "extract reference" {
t.Fatalf("runtime extraction references = %q, want original content", got)
}
}
func TestPrepareFailuresOccurBeforeInputParse(t *testing.T) { func TestPrepareFailuresOccurBeforeInputParse(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
@@ -146,6 +206,15 @@ func constructionProfile() PipelineProfile {
} }
func constructionRegistries(t *testing.T, built *[]string, failure *constructionFailure) (Registries, *constructionInput) { func constructionRegistries(t *testing.T, built *[]string, failure *constructionFailure) (Registries, *constructionInput) {
return constructionRegistriesWithHooks(t, built, failure, nil, nil)
}
type constructionBuildObservation struct {
Name string
Request BuildRequest
}
func constructionRegistriesWithHooks(t *testing.T, built *[]string, failure *constructionFailure, observe func(string, BuildRequest), mutate func(string, *BuildRequest)) (Registries, *constructionInput) {
t.Helper() t.Helper()
if built == nil { if built == nil {
built = &[]string{} built = &[]string{}
@@ -153,7 +222,15 @@ func constructionRegistries(t *testing.T, built *[]string, failure *construction
if failure == nil { if failure == nil {
failure = &constructionFailure{} failure = &constructionFailure{}
} }
record := func(name string) { *built = append(*built, name) } record := func(name string, request *BuildRequest) {
*built = append(*built, name)
if observe != nil {
observe(name, cloneBuildRequest(*request))
}
if mutate != nil {
mutate(name, request)
}
}
strict := func(options map[string]any) error { return RejectUnknownOptions(options, "known") } strict := func(options map[string]any) error { return RejectUnknownOptions(options, "known") }
input := &constructionInput{key: "input"} input := &constructionInput{key: "input"}
registries := Registries{ registries := Registries{
@@ -164,14 +241,14 @@ func constructionRegistries(t *testing.T, built *[]string, failure *construction
if err := RegisterArtifactCodec(registries.ArtifactCodecs, notesCodec()); err != nil { if err := RegisterArtifactCodec(registries.ArtifactCodecs, notesCodec()); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if err := registries.Inputs.RegisterBuilderWithSpec(defaultModuleSpec("input", StageInput), strict, func(BuildRequest) (contracts.InputAdapter, error) { if err := registries.Inputs.RegisterBuilderWithSpec(defaultModuleSpec("input", StageInput), strict, func(request BuildRequest) (contracts.InputAdapter, error) {
record("input") record("input", &request)
return input, nil return input, nil
}); err != nil { }); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if err := registries.Chunkers.RegisterBuilderWithSpec(defaultModuleSpec("chunk", StageChunk), strict, func(BuildRequest) (contracts.Chunker, error) { if err := registries.Chunkers.RegisterBuilderWithSpec(defaultModuleSpec("chunk", StageChunk), strict, func(request BuildRequest) (contracts.Chunker, error) {
record("chunk") record("chunk", &request)
return &typedTestChunker{key: "chunk"}, nil return &typedTestChunker{key: "chunk"}, nil
}); err != nil { }); err != nil {
t.Fatal(err) t.Fatal(err)
@@ -179,45 +256,45 @@ func constructionRegistries(t *testing.T, built *[]string, failure *construction
extractSpec := defaultModuleSpec("extract", StageExtract) extractSpec := defaultModuleSpec("extract", StageExtract)
extractSpec.ArtifactKind = "test/notes" extractSpec.ArtifactKind = "test/notes"
if err := RegisterExtractorBuilder(registries.Extractors, extractSpec, strict, func(request BuildRequest) (contracts.Extractor[codecNotes], error) { if err := RegisterExtractorBuilder(registries.Extractors, extractSpec, strict, func(request BuildRequest) (contracts.Extractor[codecNotes], error) {
record("extract") record("extract", &request)
if failure.requireExtractorLLM && request.Dependencies.LLM == nil { if failure.requireExtractorLLM && request.Dependencies.LLM == nil {
return nil, errors.New("structured LLM client is required") return nil, errors.New("structured LLM client is required")
} }
return typedTestExtractor[codecNotes]{key: "extract"}, nil return &constructionExtractor{key: "extract", requests: &input.extractRequests}, nil
}); err != nil { }); err != nil {
t.Fatal(err) t.Fatal(err)
} }
mergeSpec := defaultModuleSpec("merge", StageMerge) mergeSpec := defaultModuleSpec("merge", StageMerge)
mergeSpec.ArtifactKind = "test/notes" mergeSpec.ArtifactKind = "test/notes"
if err := RegisterMergerBuilder(registries.Mergers, mergeSpec, strict, func(BuildRequest) (contracts.Merger[codecNotes], error) { if err := RegisterMergerBuilder(registries.Mergers, mergeSpec, strict, func(request BuildRequest) (contracts.Merger[codecNotes], error) {
record("merge") record("merge", &request)
return typedTestMerger[codecNotes]{key: "merge"}, nil return typedTestMerger[codecNotes]{key: "merge"}, nil
}); err != nil { }); err != nil {
t.Fatal(err) t.Fatal(err)
} }
normalizeSpec := defaultModuleSpec("normalize", StageNormalize) normalizeSpec := defaultModuleSpec("normalize", StageNormalize)
normalizeSpec.ArtifactKind = "test/notes" normalizeSpec.ArtifactKind = "test/notes"
if err := RegisterNormalizerBuilder(registries.Normalizers, normalizeSpec, strict, func(BuildRequest) (contracts.Normalizer[codecNotes], error) { if err := RegisterNormalizerBuilder(registries.Normalizers, normalizeSpec, strict, func(request BuildRequest) (contracts.Normalizer[codecNotes], error) {
record("normalize") record("normalize", &request)
return typedTestNormalizer[codecNotes]{key: "normalize"}, nil return typedTestNormalizer[codecNotes]{key: "normalize"}, nil
}); err != nil { }); err != nil {
t.Fatal(err) t.Fatal(err)
} }
validatorSpec := ValidatorSpec{Key: "configured", ExecutionClass: contracts.ExecutionClassDeterministic} validatorSpec := ValidatorSpec{Key: "configured", ExecutionClass: contracts.ExecutionClassDeterministic}
if err := RegisterChunkValidatorBuilder(registries.Validators, validatorSpec, strict, func(BuildRequest) (contracts.ChunkValidator, error) { if err := RegisterChunkValidatorBuilder(registries.Validators, validatorSpec, strict, func(request BuildRequest) (contracts.ChunkValidator, error) {
record("validator") record("validator", &request)
return typedTestChunkValidator{key: "configured"}, nil return typedTestChunkValidator{key: "configured"}, nil
}); err != nil { }); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if err := RegisterTypedValidatorBuilder(registries.Validators, "test/notes", validatorSpec, strict, func(BuildRequest) (contracts.TypedValidator[codecNotes], error) { if err := RegisterTypedValidatorBuilder(registries.Validators, "test/notes", validatorSpec, strict, func(request BuildRequest) (contracts.TypedValidator[codecNotes], error) {
record("validator") record("validator", &request)
return typedTestValidator[codecNotes]{key: "configured"}, nil return typedTestValidator[codecNotes]{key: "configured"}, nil
}); err != nil { }); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if err := registries.Outputs.RegisterBuilderWithSpec(defaultModuleSpec("output", StageOutput), strict, func(BuildRequest) (contracts.OutputEncoder, error) { if err := registries.Outputs.RegisterBuilderWithSpec(defaultModuleSpec("output", StageOutput), strict, func(request BuildRequest) (contracts.OutputEncoder, error) {
record("output") record("output", &request)
if failure.output != nil { if failure.output != nil {
return nil, failure.output return nil, failure.output
} }
@@ -231,6 +308,39 @@ func constructionRegistries(t *testing.T, built *[]string, failure *construction
type constructionInput struct { type constructionInput struct {
key string key string
requests []contracts.ParseRequest requests []contracts.ParseRequest
extractRequests []contracts.TypedExtractionRequest
}
type constructionExtractor struct {
key string
requests *[]contracts.TypedExtractionRequest
}
func (extractor *constructionExtractor) Key() string { return extractor.key }
func (*constructionExtractor) ReferenceSlots() []contracts.ReferenceSlot { return nil }
func (extractor *constructionExtractor) Extract(_ context.Context, request contracts.TypedExtractionRequest) (contracts.TypedExtractionResult[codecNotes], error) {
if extractor.requests != nil {
*extractor.requests = append(*extractor.requests, request)
}
return contracts.TypedExtractionResult[codecNotes]{}, nil
}
func constructionReferenceSet(slotName, content string) contracts.ReferenceSet {
return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
slotName: {
Slot: contracts.ReferenceSlot{Name: slotName},
Items: []contracts.ReferenceItem{{SlotName: slotName, Content: []byte(content)}},
},
}}
}
func constructionReferenceContent(references contracts.ReferenceSet) string {
for _, slot := range references.Slots {
if len(slot.Items) > 0 {
return string(slot.Items[0].Content)
}
}
return ""
} }
func (input *constructionInput) Key() string { return input.key } func (input *constructionInput) Key() string { return input.key }

View File

@@ -78,22 +78,22 @@ func Prepare(resolved ResolvedPipeline, registries Registries, deps ModuleDepend
resolved: stable, resolved: stable,
dependencies: deps, dependencies: deps,
} }
request := func(binding ModuleBinding) BuildRequest { request := func(binding ModuleBinding, references contracts.ReferenceSet) BuildRequest {
return BuildRequest{Dependencies: deps, Options: cloneOptions(binding.Options)} return BuildRequest{Dependencies: deps, Options: cloneOptions(binding.Options), References: references}
} }
input, err := registries.Inputs.BuildWithRequest(stable.Input.Module, request(stable.Input)) input, err := registries.Inputs.BuildWithRequest(stable.Input.Module, request(stable.Input, contracts.ReferenceSet{}))
if err != nil { if err != nil {
return nil, constructionError(stable.ID, "", StageInput, stable.Input.Module, "", err) return nil, constructionError(stable.ID, "", StageInput, stable.Input.Module, "", err)
} }
prepared.input = input prepared.input = input
chunker, err := registries.Chunkers.BuildWithRequest(stable.Chunk.Module, request(stable.Chunk)) chunker, err := registries.Chunkers.BuildWithRequest(stable.Chunk.Module, request(stable.Chunk, stable.ChunkReferences.ReferenceSet))
if err != nil { if err != nil {
return nil, constructionError(stable.ID, "", StageChunk, stable.Chunk.Module, "", err) return nil, constructionError(stable.ID, "", StageChunk, stable.Chunk.Module, "", err)
} }
prepared.chunker = chunker prepared.chunker = chunker
prepared.chunkValidators, err = prepareValidatorChain(stable, registries, deps, StageChunk, "", stable.Chunk.Module) prepared.chunkValidators, err = prepareValidatorChain(stable, registries, deps, StageChunk, "", stable.Chunk.Module, stable.ChunkReferences.ReferenceSet)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -109,7 +109,7 @@ func Prepare(resolved ResolvedPipeline, registries Registries, deps ModuleDepend
prepared.lanes = append(prepared.lanes, executor) prepared.lanes = append(prepared.lanes, executor)
} }
output, err := registries.Outputs.BuildWithRequest(stable.Output.Module, request(stable.Output)) output, err := registries.Outputs.BuildWithRequest(stable.Output.Module, request(stable.Output, contracts.ReferenceSet{}))
if err != nil { if err != nil {
return nil, constructionError(stable.ID, "", StageOutput, stable.Output.Module, "", err) return nil, constructionError(stable.ID, "", StageOutput, stable.Output.Module, "", err)
} }
@@ -119,14 +119,14 @@ func Prepare(resolved ResolvedPipeline, registries Registries, deps ModuleDepend
func prepareLane(pipeline ResolvedPipeline, lane ResolvedArtifactLane, registries Registries, deps ModuleDependencies) (preparedLaneExecutor, error) { func prepareLane(pipeline ResolvedPipeline, lane ResolvedArtifactLane, registries Registries, deps ModuleDependencies) (preparedLaneExecutor, error) {
executor := preparedLaneExecutor{resolved: cloneResolvedArtifactLane(lane)} executor := preparedLaneExecutor{resolved: cloneResolvedArtifactLane(lane)}
request := func(binding ModuleBinding) BuildRequest { request := func(binding ModuleBinding, references contracts.ReferenceSet) BuildRequest {
return BuildRequest{Dependencies: deps, Options: cloneOptions(binding.Options)} return BuildRequest{Dependencies: deps, Options: cloneOptions(binding.Options), References: references}
} }
extractEntry, ok := registries.Extractors.typedEntry(lane.Extract.Module) extractEntry, ok := registries.Extractors.typedEntry(lane.Extract.Module)
if !ok { if !ok {
return preparedLaneExecutor{}, constructionError(pipeline.ID, lane.ID, StageExtract, lane.Extract.Module, "", fmt.Errorf("typed construction entry is not registered")) return preparedLaneExecutor{}, constructionError(pipeline.ID, lane.ID, StageExtract, lane.Extract.Module, "", fmt.Errorf("typed construction entry is not registered"))
} }
module, err := buildErasedModule(extractEntry.builder, request(lane.Extract), lane.Extract.Module, "extractor") module, err := buildErasedModule(extractEntry.builder, request(lane.Extract, lane.ExtractReferences.ReferenceSet), lane.Extract.Module, "extractor")
if err != nil { if err != nil {
return preparedLaneExecutor{}, constructionError(pipeline.ID, lane.ID, StageExtract, lane.Extract.Module, "", err) return preparedLaneExecutor{}, constructionError(pipeline.ID, lane.ID, StageExtract, lane.Extract.Module, "", err)
} }
@@ -136,7 +136,7 @@ func prepareLane(pipeline ResolvedPipeline, lane ResolvedArtifactLane, registrie
} }
executor.typed = &preparedTypedLane{extractor: module, extract: extractEntry.extract, codec: codec} executor.typed = &preparedTypedLane{extractor: module, extract: extractEntry.extract, codec: codec}
executor.extractValidators, err = prepareValidatorChain(pipeline, registries, deps, StageExtract, lane.ID, lane.Extract.Module) executor.extractValidators, err = prepareValidatorChain(pipeline, registries, deps, StageExtract, lane.ID, lane.Extract.Module, lane.ExtractReferences.ReferenceSet)
if err != nil { if err != nil {
return preparedLaneExecutor{}, err return preparedLaneExecutor{}, err
} }
@@ -145,13 +145,13 @@ func prepareLane(pipeline ResolvedPipeline, lane ResolvedArtifactLane, registrie
if !ok { if !ok {
return preparedLaneExecutor{}, constructionError(pipeline.ID, lane.ID, StageMerge, lane.Merge.Module, "", fmt.Errorf("typed construction entry is not registered")) return preparedLaneExecutor{}, constructionError(pipeline.ID, lane.ID, StageMerge, lane.Merge.Module, "", fmt.Errorf("typed construction entry is not registered"))
} }
module, err = buildErasedModule(mergeEntry.builder, request(lane.Merge), lane.Merge.Module, "merger") module, err = buildErasedModule(mergeEntry.builder, request(lane.Merge, lane.MergeReferences.ReferenceSet), lane.Merge.Module, "merger")
if err != nil { if err != nil {
return preparedLaneExecutor{}, constructionError(pipeline.ID, lane.ID, StageMerge, lane.Merge.Module, "", err) return preparedLaneExecutor{}, constructionError(pipeline.ID, lane.ID, StageMerge, lane.Merge.Module, "", err)
} }
executor.typed.merger = module executor.typed.merger = module
executor.typed.merge = mergeEntry.merge executor.typed.merge = mergeEntry.merge
executor.mergeValidators, err = prepareValidatorChain(pipeline, registries, deps, StageMerge, lane.ID, lane.Merge.Module) executor.mergeValidators, err = prepareValidatorChain(pipeline, registries, deps, StageMerge, lane.ID, lane.Merge.Module, lane.MergeReferences.ReferenceSet)
if err != nil { if err != nil {
return preparedLaneExecutor{}, err return preparedLaneExecutor{}, err
} }
@@ -160,24 +160,24 @@ func prepareLane(pipeline ResolvedPipeline, lane ResolvedArtifactLane, registrie
if !ok { if !ok {
return preparedLaneExecutor{}, constructionError(pipeline.ID, lane.ID, StageNormalize, lane.Normalize.Module, "", fmt.Errorf("typed construction entry is not registered")) return preparedLaneExecutor{}, constructionError(pipeline.ID, lane.ID, StageNormalize, lane.Normalize.Module, "", fmt.Errorf("typed construction entry is not registered"))
} }
module, err = buildErasedModule(normalizeEntry.builder, request(lane.Normalize), lane.Normalize.Module, "normalizer") module, err = buildErasedModule(normalizeEntry.builder, request(lane.Normalize, lane.NormalizeReferences.ReferenceSet), lane.Normalize.Module, "normalizer")
if err != nil { if err != nil {
return preparedLaneExecutor{}, constructionError(pipeline.ID, lane.ID, StageNormalize, lane.Normalize.Module, "", err) return preparedLaneExecutor{}, constructionError(pipeline.ID, lane.ID, StageNormalize, lane.Normalize.Module, "", err)
} }
executor.typed.normalizer = module executor.typed.normalizer = module
executor.typed.normalize = normalizeEntry.normalize executor.typed.normalize = normalizeEntry.normalize
executor.normalizeValidators, err = prepareValidatorChain(pipeline, registries, deps, StageNormalize, lane.ID, lane.Normalize.Module) executor.normalizeValidators, err = prepareValidatorChain(pipeline, registries, deps, StageNormalize, lane.ID, lane.Normalize.Module, lane.NormalizeReferences.ReferenceSet)
if err != nil { if err != nil {
return preparedLaneExecutor{}, err return preparedLaneExecutor{}, err
} }
return executor, nil return executor, nil
} }
func prepareValidatorChain(pipeline ResolvedPipeline, registries Registries, deps ModuleDependencies, stage ModuleStage, laneID, moduleKey string) (preparedValidatorChain, error) { func prepareValidatorChain(pipeline ResolvedPipeline, registries Registries, deps ModuleDependencies, stage ModuleStage, laneID, moduleKey string, references contracts.ReferenceSet) (preparedValidatorChain, error) {
resolved := resolvedValidatorChain(stage, laneID, moduleKey, pipeline.ValidatorChains) resolved := resolvedValidatorChain(stage, laneID, moduleKey, pipeline.ValidatorChains)
prepared := preparedValidatorChain{resolved: resolved} prepared := preparedValidatorChain{resolved: resolved}
for _, validator := range resolved.Validators { for _, validator := range resolved.Validators {
request := BuildRequest{Dependencies: deps, Options: cloneOptions(validator.Binding.Options)} request := BuildRequest{Dependencies: deps, Options: cloneOptions(validator.Binding.Options), References: references}
built, err := buildPreparedValidator(registries.Validators, validator, request) built, err := buildPreparedValidator(registries.Validators, validator, request)
if err != nil { if err != nil {
return preparedValidatorChain{}, constructionError(pipeline.ID, laneID, stage, moduleKey, validator.Binding.Module, err) return preparedValidatorChain{}, constructionError(pipeline.ID, laneID, stage, moduleKey, validator.Binding.Module, err)

View File

@@ -0,0 +1,6 @@
The canonical spell-name catalog for this extraction is provided below as JSON.
Return spell names using the catalog's canonical spelling exactly. Aliases and
other campaign reference material are not part of this catalog input and must
not be copied into the output as spell names.
{{ input "spell_catalog" }}

View File

@@ -5,6 +5,9 @@ inputs:
- name: transcript - name: transcript
required: true required: true
content_type: application/json content_type: application/json
- name: spell_catalog
required: true
content_type: application/json
- name: players - name: players
required: false required: false
content_type: text/plain content_type: text/plain
@@ -25,6 +28,8 @@ messages:
content_file: ./sharedassets/common-dnd-references.md content_file: ./sharedassets/common-dnd-references.md
cache_control: cache_control:
type: ephemeral type: ephemeral
- role: user
content_file: ./catalog.md
- role: user - role: user
content_file: ./task.md content_file: ./task.md
- role: user - role: user

View File

@@ -5,8 +5,12 @@ source_id automatically.
Return only D&D spell-cast artifacts. For each spell cast, identify the in-world Return only D&D spell-cast artifacts. For each spell cast, identify the in-world
caster, spell name, effect, narrative description, and source references. caster, spell name, effect, narrative description, and source references.
Use player, party, and glossary reference material only to clarify source text. Use the canonical spell-name catalog to select spell names. Do not return a
Do not return spells, casters, or effects that are mentioned only in reference spell name absent from that catalog, even when it is suggested by general D&D
material. knowledge or reference material.
Use player, party, and glossary reference material only to clarify source text;
references are not source evidence for a spell cast. Do not return spells,
casters, or effects that are mentioned only in reference material.
Return exactly one JSON object and no explanatory text. Return exactly one JSON object and no explanatory text.

View File

@@ -3,3 +3,7 @@ Extract Dungeons & Dragons spell-cast artifacts from the provided transcript.
Extract only spell casts that are supported by the transcript. Do not infer Extract only spell casts that are supported by the transcript. Do not infer
spells from general D&D knowledge or from table chatter that does not identify a spells from general D&D knowledge or from table chatter that does not identify a
spell being cast. spell being cast.
Use the provided canonical spell-name catalog when naming each extracted spell.
Return the canonical catalog spelling exactly. The catalog is a recognition
aid; it does not establish that a spell was cast.

View File

@@ -0,0 +1,29 @@
package spells
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
spellcatalog "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/spells/catalog"
)
func newCatalogPromptInput(effective spellcatalog.EffectiveCatalog) (contracts.LLMInputMaterial, error) {
content, err := json.Marshal(struct {
SpellNames []string `json:"spell_names"`
}{SpellNames: effective.CanonicalNames()})
if err != nil {
return contracts.LLMInputMaterial{}, fmt.Errorf("encode canonical spell names: %w", err)
}
sum := sha256.Sum256(content)
digest := "sha256:" + hex.EncodeToString(sum[:])
return contracts.NewLLMInputMaterial(
spellcatalog.SpellCatalogReferenceSlot,
"application/json",
content,
digest,
"",
), nil
}

View File

@@ -9,6 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
spellcatalog "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/spells/catalog"
) )
const Key = "dnd/spells" const Key = "dnd/spells"
@@ -31,19 +32,50 @@ var referenceSlotDescriptions = shared.ReferenceSlotDescriptions{
Roster: "Deprecated alias for party roster reference material used only for disambiguation.", Roster: "Deprecated alias for party roster reference material used only for disambiguation.",
} }
func referenceSlots() []contracts.ReferenceSlot {
slots := shared.ReferenceSlots(referenceSlotDescriptions)
return append(slots, contracts.ReferenceSlot{
Name: spellcatalog.SpellCatalogReferenceSlot,
Description: "Optional canonical spell-name catalog used for extraction grounding.",
AcceptedMediaTypes: []string{"application/json"},
MaxBytes: 1048576,
})
}
var _ contracts.Extractor[dnd.SpellList] = (*Extractor)(nil) var _ contracts.Extractor[dnd.SpellList] = (*Extractor)(nil)
type Options struct{} type Options struct{}
type Extractor struct { type Extractor struct {
llm contracts.StructuredLLMClient llm contracts.StructuredLLMClient
effectiveCatalog spellcatalog.EffectiveCatalog
catalogPromptInput contracts.LLMInputMaterial
} }
func New(llmClient contracts.StructuredLLMClient, _ Options) (*Extractor, error) { func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contracts.ReferenceSet) (*Extractor, error) {
if llmClient == nil { if llmClient == nil {
return nil, extractorErrorf("LLM client must not be nil") return nil, extractorErrorf("LLM client must not be nil")
} }
return &Extractor{llm: llmClient}, nil if len(references) > 1 {
return nil, extractorErrorf("at most one reference set may be supplied")
}
var referenceSet contracts.ReferenceSet
if len(references) == 1 {
referenceSet = references[0]
}
effectiveCatalog, err := spellcatalog.ResolveEffectiveCatalog(referenceSet)
if err != nil {
return nil, extractorErrorf("resolve effective spell catalog: %w", err)
}
catalogPromptInput, err := newCatalogPromptInput(effectiveCatalog)
if err != nil {
return nil, extractorErrorf("prepare spell catalog prompt input: %w", err)
}
return &Extractor{
llm: llmClient,
effectiveCatalog: effectiveCatalog,
catalogPromptInput: catalogPromptInput,
}, nil
} }
func (e *Extractor) Key() string { func (e *Extractor) Key() string {
@@ -51,7 +83,7 @@ func (e *Extractor) Key() string {
} }
func (e *Extractor) ReferenceSlots() []contracts.ReferenceSlot { func (e *Extractor) ReferenceSlots() []contracts.ReferenceSlot {
return shared.ReferenceSlots(referenceSlotDescriptions) return referenceSlots()
} }
func (e *Extractor) ManifestMetadata() map[string]any { func (e *Extractor) ManifestMetadata() map[string]any {
@@ -63,6 +95,9 @@ func (e *Extractor) ManifestMetadata() map[string]any {
"prompt_id": PromptID, "prompt_id": PromptID,
"prompt_version": SchemaVersion, "prompt_version": SchemaVersion,
"prompt_sha256": promptSHA, "prompt_sha256": promptSHA,
"catalog_base_id": e.effectiveCatalog.BaseID(),
"catalog_digest": e.effectiveCatalog.Digest(),
"catalog_overlay_ids": e.effectiveCatalog.OverlayIDs(),
"response_schema_key": string(ResponseSchemaKey), "response_schema_key": string(ResponseSchemaKey),
"response_schema_id": ResponseSchemaID, "response_schema_id": ResponseSchemaID,
"response_schema_name": ResponseSchemaName, "response_schema_name": ResponseSchemaName,
@@ -102,13 +137,15 @@ func (e *Extractor) Extract(ctx context.Context, req contracts.TypedExtractionRe
} }
var response extractionResponse var response extractionResponse
inputs := shared.PromptInputs(sourceInput, req.References)
inputs[spellcatalog.SpellCatalogReferenceSlot] = e.catalogPromptInput.Clone()
if _, err := e.llm.CompleteStructured(ctx, contracts.StructuredCompletionRequest{ if _, err := e.llm.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
StageName: Key, StageName: Key,
PromptID: PromptID, PromptID: PromptID,
PromptVersion: SchemaVersion, PromptVersion: SchemaVersion,
ProfileID: req.LLMProfile, ProfileID: req.LLMProfile,
SessionID: req.SessionID, SessionID: req.SessionID,
Inputs: shared.PromptInputs(sourceInput, req.References), Inputs: inputs,
}, &response); err != nil { }, &response); err != nil {
return contracts.TypedExtractionResult[dnd.SpellList]{}, extractorErrorf("complete structured output: %w", err) return contracts.TypedExtractionResult[dnd.SpellList]{}, extractorErrorf("complete structured output: %w", err)
} }
@@ -143,7 +180,7 @@ func ModuleSpec() pipeline.ModuleSpec {
Requires: append([]string(nil), requiredCapabilities...), Requires: append([]string(nil), requiredCapabilities...),
Provides: append([]string(nil), providedCapabilities...), Provides: append([]string(nil), providedCapabilities...),
ArtifactKind: dnd.SpellListKind, ArtifactKind: dnd.SpellListKind,
ReferenceSlots: shared.ReferenceSlots(referenceSlotDescriptions), ReferenceSlots: referenceSlots(),
} }
} }
@@ -153,7 +190,7 @@ func Register(registry *pipeline.ExtractorRegistry) error {
if err != nil { if err != nil {
return nil, err return nil, err
} }
return New(request.Dependencies.LLM, options) return New(request.Dependencies.LLM, options, request.References)
}) })
} }

View File

@@ -2,8 +2,10 @@ package spells
import ( import (
"context" "context"
"encoding/json"
"errors" "errors"
"reflect" "reflect"
"sort"
"strings" "strings"
"testing" "testing"
@@ -11,6 +13,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
spellcatalog "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/spells/catalog"
) )
func TestExtractReturnsCanonicalSpellListFromPrivateResponse(t *testing.T) { func TestExtractReturnsCanonicalSpellListFromPrivateResponse(t *testing.T) {
@@ -62,6 +65,78 @@ func TestExtractReturnsCanonicalSpellListFromPrivateResponse(t *testing.T) {
if got := string(transcript.Content); got != string(req.Chunk.Content) { if got := string(transcript.Content); got != string(req.Chunk.Content) {
t.Fatalf("transcript content = %q, want chunk content %q", got, req.Chunk.Content) t.Fatalf("transcript content = %q, want chunk content %q", got, req.Chunk.Content)
} }
catalogInput := llmReq.Inputs[spellcatalog.SpellCatalogReferenceSlot]
if catalogInput.Name != spellcatalog.SpellCatalogReferenceSlot || catalogInput.MediaType != "application/json" || catalogInput.OriginURI != "" || !strings.HasPrefix(catalogInput.Digest, "sha256:") {
t.Fatalf("catalog prompt input metadata = %#v", catalogInput)
}
var catalogPayload struct {
SpellNames []string `json:"spell_names"`
}
if err := json.Unmarshal(catalogInput.Content, &catalogPayload); err != nil {
t.Fatalf("decode catalog prompt input: %v", err)
}
base, err := spellcatalog.LoadSRD5E2014()
if err != nil {
t.Fatal(err)
}
wantNames := make([]string, 0, len(base.Spells()))
for _, spell := range base.Spells() {
wantNames = append(wantNames, spell.Name)
}
sort.Strings(wantNames)
if !reflect.DeepEqual(catalogPayload.SpellNames, wantNames) || !sort.StringsAreSorted(catalogPayload.SpellNames) {
t.Fatalf("catalog prompt names = %d entries, want sorted base catalog", len(catalogPayload.SpellNames))
}
}
func TestExtractPromptUsesCanonicalOverlayNamesWithoutAliasesOrMetadata(t *testing.T) {
client := &fakeSpellsLLMClient{response: extractionResponse{SpellCasts: []spellCastResponse{}}}
if _, err := newExtractor(t, client, overlaySpellCatalogReference()).Extract(context.Background(), extractionRequest()); err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
input := client.requests[0].Inputs[spellcatalog.SpellCatalogReferenceSlot]
content := string(input.Content)
for _, expected := range []string{"Aegis of Emberfall", `"spell_names"`} {
if !strings.Contains(content, expected) {
t.Fatalf("catalog prompt input = %q, want %q", content, expected)
}
}
for _, forbidden := range []string{"Emberfall Aegis", "Private campaign source", "file:///private-source.json", "private"} {
if strings.Contains(content, forbidden) {
t.Fatalf("catalog prompt input leaked %q: %s", forbidden, content)
}
}
metadata := newExtractor(t, &fakeSpellsLLMClient{}, overlaySpellCatalogReference()).ManifestMetadata()
if metadata["catalog_base_id"] != spellcatalog.SRD5E2014ID {
t.Fatalf("catalog base metadata = %#v", metadata["catalog_base_id"])
}
if digest, ok := metadata["catalog_digest"].(string); !ok || !strings.HasPrefix(digest, "sha256:") {
t.Fatalf("catalog digest metadata = %#v", metadata["catalog_digest"])
}
if got, ok := metadata["catalog_overlay_ids"].([]string); !ok || !reflect.DeepEqual(got, []string{"campaign.example"}) {
t.Fatalf("catalog overlay metadata = %#v", metadata["catalog_overlay_ids"])
}
encoded, err := json.Marshal(metadata)
if err != nil {
t.Fatal(err)
}
for _, forbidden := range []string{"Aegis of Emberfall", "Emberfall Aegis", "Private campaign source", "file:///private-source.json"} {
if strings.Contains(string(encoded), forbidden) {
t.Fatalf("manifest metadata leaked %q: %s", forbidden, encoded)
}
}
}
func TestNewRejectsMalformedCatalogBeforeLLMCall(t *testing.T) {
client := &fakeSpellsLLMClient{}
_, err := New(client, Options{}, spellCatalogReference(`{"schema_version":"notarius.dnd.spell-catalog-overlay.v2","catalogs":[]}`))
if err == nil || !strings.Contains(err.Error(), "resolve effective spell catalog") {
t.Fatalf("New() error = %v, want effective catalog error", err)
}
if len(client.requests) != 0 {
t.Fatalf("LLM calls = %d, want none during failed construction", len(client.requests))
}
} }
func TestExtractorManifestMetadataIncludesLLMSchemaProvenance(t *testing.T) { func TestExtractorManifestMetadataIncludesLLMSchemaProvenance(t *testing.T) {

View File

@@ -54,6 +54,12 @@ func TestModuleSpec(t *testing.T) {
Description: "Deprecated alias for party roster reference material used only for disambiguation.", Description: "Deprecated alias for party roster reference material used only for disambiguation.",
AcceptedMediaTypes: []string{"application/json", "application/x-yaml", "application/yaml", "text/markdown", "text/plain"}, AcceptedMediaTypes: []string{"application/json", "application/x-yaml", "application/yaml", "text/markdown", "text/plain"},
}, },
{
Name: "spell_catalog",
Description: "Optional canonical spell-name catalog used for extraction grounding.",
AcceptedMediaTypes: []string{"application/json"},
MaxBytes: 1048576,
},
}, },
} }
if !reflect.DeepEqual(got, want) { if !reflect.DeepEqual(got, want) {

View File

@@ -14,6 +14,7 @@ const scriptoriumPromptRoot = "assets/prompts"
func RegisterPromptAssets(registry *llm.AssetRegistry) error { func RegisterPromptAssets(registry *llm.AssetRegistry) error {
promptFS, err := shared.ModulePromptFS("dnd.spells", embeddedAssets, []promptfs.ModulePromptFile{ promptFS, err := shared.ModulePromptFS("dnd.spells", embeddedAssets, []promptfs.ModulePromptFile{
{Name: "dnd.spells.yaml", Path: "assets/prompts/dnd.spells.yaml"}, {Name: "dnd.spells.yaml", Path: "assets/prompts/dnd.spells.yaml"},
{Name: "catalog.md", Path: "assets/prompts/catalog.md"},
{Name: "task.md", Path: "assets/prompts/task.md"}, {Name: "task.md", Path: "assets/prompts/task.md"},
{Name: "instructions.md", Path: "assets/prompts/instructions.md"}, {Name: "instructions.md", Path: "assets/prompts/instructions.md"},
}) })
@@ -30,6 +31,7 @@ func scriptoriumPromptMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() { scriptoriumPromptHashOnce.Do(func() {
parts := append([]llm.AssetHashPart{ parts := append([]llm.AssetHashPart{
{FS: embeddedAssets, Path: "assets/prompts/dnd.spells.yaml"}, {FS: embeddedAssets, Path: "assets/prompts/dnd.spells.yaml"},
{FS: embeddedAssets, Path: "assets/prompts/catalog.md"},
{FS: embeddedAssets, Path: "assets/prompts/task.md"}, {FS: embeddedAssets, Path: "assets/prompts/task.md"},
{FS: embeddedAssets, Path: "assets/prompts/instructions.md"}, {FS: embeddedAssets, Path: "assets/prompts/instructions.md"},
}, append(shared.CommonHashParts(), shared.ReferenceHashParts()...)...) }, append(shared.CommonHashParts(), shared.ReferenceHashParts()...)...)

View File

@@ -21,8 +21,8 @@ func TestScriptoriumPromptPreparesTranscriptReferencesAndTaskMessages(t *testing
if prepared.OutputContract.SchemaPath != "dnd_spells_llm.v1.json" { if prepared.OutputContract.SchemaPath != "dnd_spells_llm.v1.json" {
t.Fatalf("schema path = %q, want LLM-only schema", prepared.OutputContract.SchemaPath) t.Fatalf("schema path = %q, want LLM-only schema", prepared.OutputContract.SchemaPath)
} }
if got := len(prepared.Messages); got != 5 { if got := len(prepared.Messages); got != 6 {
t.Fatalf("message count = %d, want 5", got) t.Fatalf("message count = %d, want 6", got)
} }
if !strings.Contains(prepared.Messages[1].Content, string(transcript)) { if !strings.Contains(prepared.Messages[1].Content, string(transcript)) {
t.Fatalf("transcript message did not include source input") t.Fatalf("transcript message did not include source input")
@@ -39,7 +39,10 @@ func TestScriptoriumPromptPreparesTranscriptReferencesAndTaskMessages(t *testing
if !strings.Contains(prepared.Messages[2].Content, "Shield: abjuration") { if !strings.Contains(prepared.Messages[2].Content, "Shield: abjuration") {
t.Fatalf("reference message missing glossary content") t.Fatalf("reference message missing glossary content")
} }
if strings.Contains(prepared.Messages[3].Content, string(transcript)) { if !strings.Contains(prepared.Messages[3].Content, `{"spell_names":["Cure Wounds"]}`) {
t.Fatalf("catalog message missing canonical spell-name input: %s", prepared.Messages[3].Content)
}
if strings.Contains(prepared.Messages[4].Content, string(transcript)) {
t.Fatalf("task message leaked transcript bytes") t.Fatalf("task message leaked transcript bytes")
} }
} }
@@ -57,7 +60,7 @@ func TestScriptoriumPromptDiagnosticsOmitRawMaterials(t *testing.T) {
transcript := []byte(`{"secret":"source text"}`) transcript := []byte(`{"secret":"source text"}`)
reference := "private party note" reference := "private party note"
prepared := prepareSpellsPrompt(t, transcript, "private player note", reference, " ") prepared := prepareSpellsPrompt(t, transcript, "private player note", reference, " ")
metadata := newExtractor(t, &fakeSpellsLLMClient{}).ManifestMetadata() metadata := newExtractor(t, &fakeSpellsLLMClient{}, overlaySpellCatalogReference()).ManifestMetadata()
payload, err := json.Marshal(map[string]any{ payload, err := json.Marshal(map[string]any{
"prepared": map[string]any{ "prepared": map[string]any{
@@ -80,6 +83,11 @@ func TestScriptoriumPromptDiagnosticsOmitRawMaterials(t *testing.T) {
"source text", "source text",
"private player note", "private player note",
reference, reference,
"Cure Wounds",
"Aegis of Emberfall",
"Emberfall Aegis",
"Private campaign source",
"file:///private-source.json",
`"properties"`, `"properties"`,
"spell_casts", "spell_casts",
} { } {
@@ -120,6 +128,7 @@ func prepareSpellsPrompt(t *testing.T, transcript []byte, players string, party
ProfileID: "spell-test-profile", ProfileID: "spell-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{ Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", string(transcript)), "transcript": scriptorium.InlineWithURI("file:///session.json", string(transcript)),
"spell_catalog": scriptorium.Inline(`{"spell_names":["Cure Wounds"]}`),
"players": scriptorium.Inline(players), "players": scriptorium.Inline(players),
"party": scriptorium.Inline(party), "party": scriptorium.Inline(party),
"glossary": scriptorium.Inline(glossary), "glossary": scriptorium.Inline(glossary),

View File

@@ -9,6 +9,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/core/source" "gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
spellcatalog "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/spells/catalog"
) )
func promptExtractionRequest() contracts.TypedExtractionRequest { func promptExtractionRequest() contracts.TypedExtractionRequest {
@@ -109,15 +110,31 @@ func mismatchedSourceInputRequest(req contracts.TypedExtractionRequest) contract
return req return req
} }
func newExtractor(t *testing.T, client contracts.StructuredLLMClient) *Extractor { func newExtractor(t *testing.T, client contracts.StructuredLLMClient, references ...contracts.ReferenceSet) *Extractor {
t.Helper() t.Helper()
extractor, err := New(client, Options{}) extractor, err := New(client, Options{}, references...)
if err != nil { if err != nil {
t.Fatalf("New() error = %v, want nil", err) t.Fatalf("New() error = %v, want nil", err)
} }
return extractor return extractor
} }
func spellCatalogReference(content string) contracts.ReferenceSet {
return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
spellcatalog.SpellCatalogReferenceSlot: {
Items: []contracts.ReferenceItem{{
SlotName: spellcatalog.SpellCatalogReferenceSlot,
MediaType: "application/json",
Content: []byte(content),
}},
},
}}
}
func overlaySpellCatalogReference() contracts.ReferenceSet {
return spellCatalogReference(`{"schema_version":"notarius.dnd.spell-catalog-overlay.v1","catalogs":[{"id":"campaign.example","ruleset":"dnd-5e-2014","source":{"title":"Private campaign source","version":"1","url":"file:///private-source.json","license":"private"},"spells":[{"name":"Aegis of Emberfall","aliases":["Emberfall Aegis"]}]}]}`)
}
type fakeSpellsLLMClient struct { type fakeSpellsLLMClient struct {
response extractionResponse response extractionResponse
content []byte content []byte

View File

@@ -10,6 +10,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/chunk/scenes" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/chunk/scenes"
spellcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/spells" spellcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/spells"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells" "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
spellcatalog "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/catalog"
spellshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/shape" spellshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/shape"
spellsourcerefs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/source_refs" spellsourcerefs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/source_refs"
spellrelatedness "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/source_relatedness" spellrelatedness "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/source_relatedness"
@@ -39,6 +40,7 @@ func Register(registries pipeline.Registries, assets *llm.AssetRegistry) error {
}}, }},
{name: "spell-list noop normalizer", register: func() error { return noop.RegisterTyped[dnd.SpellList](registries.Normalizers, dnd.SpellListKind) }}, {name: "spell-list noop normalizer", register: func() error { return noop.RegisterTyped[dnd.SpellList](registries.Normalizers, dnd.SpellListKind) }},
{name: "spell shape validator", register: func() error { return spellshape.Register(registries.Validators) }}, {name: "spell shape validator", register: func() error { return spellshape.Register(registries.Validators) }},
{name: "spell catalog validator", register: func() error { return spellcatalog.Register(registries.Validators) }},
{name: "spell source references validator", register: func() error { return spellsourcerefs.Register(registries.Validators) }}, {name: "spell source references validator", register: func() error { return spellsourcerefs.Register(registries.Validators) }},
{name: "spell source relatedness validator", register: func() error { return spellrelatedness.Register(registries.Validators) }}, {name: "spell source relatedness validator", register: func() error { return spellrelatedness.Register(registries.Validators) }},
{name: "spell-list always accept validator", register: func() error { {name: "spell-list always accept validator", register: func() error {
@@ -62,6 +64,7 @@ func Register(registries pipeline.Registries, assets *llm.AssetRegistry) error {
pipeline.Binding(validjson.Key), pipeline.Binding(validjson.Key),
pipeline.Binding(validjsonschema.Key), pipeline.Binding(validjsonschema.Key),
pipeline.Binding(spellshape.Key), pipeline.Binding(spellshape.Key),
pipeline.Binding(spellcatalog.Key),
pipeline.Binding(spellsourcerefs.Key), pipeline.Binding(spellsourcerefs.Key),
pipeline.Binding(spellrelatedness.Key), pipeline.Binding(spellrelatedness.Key),
}, },

View File

@@ -24,6 +24,7 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
assertContainsKeys(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells"}) assertContainsKeys(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells"})
assertContainsArtifactKinds(t, registries.ArtifactCodecs.RegisteredKinds(), []contracts.ArtifactKind{dnd.SpellListKind}) assertContainsArtifactKinds(t, registries.ArtifactCodecs.RegisteredKinds(), []contracts.ArtifactKind{dnd.SpellListKind})
assertContainsKeys(t, "validators", registries.Validators.RegisteredKeys(), []string{ assertContainsKeys(t, "validators", registries.Validators.RegisteredKeys(), []string{
"extract/dnd/spells/catalog",
"extract/dnd/spells/shape", "extract/dnd/spells/shape",
"extract/dnd/spells/source_refs", "extract/dnd/spells/source_refs",
"extract/dnd/spells/source_relatedness", "extract/dnd/spells/source_relatedness",
@@ -34,6 +35,7 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
pipeline.Binding("generic/valid_json"), pipeline.Binding("generic/valid_json"),
pipeline.Binding("generic/valid_json_schema"), pipeline.Binding("generic/valid_json_schema"),
pipeline.Binding("extract/dnd/spells/shape"), pipeline.Binding("extract/dnd/spells/shape"),
pipeline.Binding("extract/dnd/spells/catalog"),
pipeline.Binding("extract/dnd/spells/source_refs"), pipeline.Binding("extract/dnd/spells/source_refs"),
pipeline.Binding("extract/dnd/spells/source_relatedness"), pipeline.Binding("extract/dnd/spells/source_relatedness"),
} }

View File

@@ -0,0 +1,427 @@
package catalog
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"mime"
"sort"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
)
const (
SpellCatalogReferenceSlot = "spell_catalog"
overlaySchemaVersion = "notarius.dnd.spell-catalog-overlay.v1"
)
// EffectiveCatalog is the immutable spell-name recognition catalog assembled
// from the embedded SRD catalog and an optional set of campaign overlays.
type EffectiveCatalog struct {
baseID string
ruleset string
overlayIDs []string
canonicalNames []string
lookup map[string]string
digest string
}
func (c EffectiveCatalog) BaseID() string { return c.baseID }
func (c EffectiveCatalog) Ruleset() string { return c.ruleset }
func (c EffectiveCatalog) Digest() string { return c.digest }
func (c EffectiveCatalog) OverlayIDs() []string { return append([]string(nil), c.overlayIDs...) }
// CanonicalNames returns the globally sorted canonical spell names without
// exposing the catalog's internal storage.
func (c EffectiveCatalog) CanonicalNames() []string {
return append([]string(nil), c.canonicalNames...)
}
// Lookup matches canonical names and aliases after applying the same
// normalization used by the embedded catalog. The returned string is the
// established canonical display name.
func (c EffectiveCatalog) Lookup(name string) (string, bool) {
canonical, ok := c.lookup[lookupKey(name)]
return canonical, ok
}
// ResolveEffectiveCatalog loads the embedded SRD catalog and applies the
// optional spell-catalog overlay found in the cloned reference set. It does
// not resolve paths or perform filesystem access.
func ResolveEffectiveCatalog(references contracts.ReferenceSet) (EffectiveCatalog, error) {
base, err := LoadSRD5E2014()
if err != nil {
return EffectiveCatalog{}, err
}
cloned := cloneReferenceSet(references)
slot, ok := cloned.Slots[SpellCatalogReferenceSlot]
if !ok || len(slot.Items) == 0 {
return composeEffectiveCatalog(base, nil)
}
if len(slot.Items) != 1 {
return EffectiveCatalog{}, fmt.Errorf("reference slot %q must contain zero or one item", SpellCatalogReferenceSlot)
}
item := slot.Items[0]
mediaType, _, err := mime.ParseMediaType(item.MediaType)
if err != nil {
return EffectiveCatalog{}, fmt.Errorf("reference slot %q item media type %q is invalid: %w", SpellCatalogReferenceSlot, item.MediaType, err)
}
if !strings.EqualFold(mediaType, "application/json") {
return EffectiveCatalog{}, fmt.Errorf("reference slot %q item media type %q must be application/json", SpellCatalogReferenceSlot, item.MediaType)
}
overlays, err := decodeOverlayBundle(item.Content)
if err != nil {
return EffectiveCatalog{}, err
}
return composeEffectiveCatalog(base, overlays)
}
type overlayBundle struct {
SchemaVersion string `json:"schema_version"`
Catalogs []overlayCatalog `json:"catalogs"`
}
type overlayCatalog struct {
ID string `json:"id"`
Ruleset string `json:"ruleset"`
Source overlaySource `json:"source"`
Spells []overlaySpell `json:"spells"`
}
type overlaySource struct {
Title string `json:"title"`
Version json.RawMessage `json:"version"`
URL json.RawMessage `json:"url"`
License json.RawMessage `json:"license"`
}
type overlaySpell struct {
Name string `json:"name"`
Aliases json.RawMessage `json:"aliases,omitempty"`
}
func decodeOverlayBundle(content []byte) ([]overlayCatalog, error) {
decoder := json.NewDecoder(bytes.NewReader(content))
decoder.DisallowUnknownFields()
var bundle overlayBundle
if err := decoder.Decode(&bundle); err != nil {
return nil, fmt.Errorf("decode spell catalog overlay: %w", err)
}
var trailing any
if err := decoder.Decode(&trailing); err != io.EOF {
return nil, fmt.Errorf("decode spell catalog overlay: multiple JSON values")
}
if bundle.SchemaVersion != overlaySchemaVersion {
return nil, fmt.Errorf("spell catalog overlay schema_version %q does not match %q", bundle.SchemaVersion, overlaySchemaVersion)
}
if len(bundle.Catalogs) == 0 {
return nil, fmt.Errorf("spell catalog overlay catalogs must not be empty")
}
seenIDs := make(map[string]struct{}, len(bundle.Catalogs))
for catalogIndex := range bundle.Catalogs {
catalog := &bundle.Catalogs[catalogIndex]
if catalog.ID != strings.TrimSpace(catalog.ID) || catalog.ID == "" {
return nil, fmt.Errorf("catalog[%d] id must be non-empty and trimmed", catalogIndex)
}
if _, exists := seenIDs[catalog.ID]; exists {
return nil, fmt.Errorf("catalog id %q is duplicated", catalog.ID)
}
seenIDs[catalog.ID] = struct{}{}
if catalog.Ruleset != SRD5E2014Ruleset {
return nil, fmt.Errorf("catalog %q ruleset %q does not match %q", catalog.ID, catalog.Ruleset, SRD5E2014Ruleset)
}
if catalog.Source.Title != strings.TrimSpace(catalog.Source.Title) || catalog.Source.Title == "" {
return nil, fmt.Errorf("catalog %q source title must be non-empty and trimmed", catalog.ID)
}
for field, raw := range map[string]json.RawMessage{
"version": catalog.Source.Version,
"url": catalog.Source.URL,
"license": catalog.Source.License,
} {
if _, err := decodeOptionalString(raw); err != nil {
return nil, fmt.Errorf("catalog %q source %s: %w", catalog.ID, field, err)
}
}
if len(catalog.Spells) == 0 {
return nil, fmt.Errorf("catalog %q spells must not be empty", catalog.ID)
}
for spellIndex := range catalog.Spells {
spell := &catalog.Spells[spellIndex]
if spell.Name != strings.TrimSpace(spell.Name) || spell.Name == "" {
return nil, fmt.Errorf("catalog %q spell[%d] name must be non-empty and trimmed", catalog.ID, spellIndex)
}
_, err := decodeAliases(spell.Aliases)
if err != nil {
return nil, fmt.Errorf("catalog %q spell %q aliases: %w", catalog.ID, spell.Name, err)
}
}
}
return bundle.Catalogs, nil
}
func decodeOptionalString(raw json.RawMessage) (string, error) {
if len(raw) == 0 {
return "", nil
}
if bytes.Equal(bytes.TrimSpace(raw), []byte("null")) {
return "", fmt.Errorf("must be a string when present")
}
var value string
if err := json.Unmarshal(raw, &value); err != nil {
return "", fmt.Errorf("must be a string when present: %w", err)
}
return value, nil
}
func decodeAliases(raw json.RawMessage) ([]string, error) {
if len(raw) == 0 {
return nil, nil
}
if bytes.Equal(bytes.TrimSpace(raw), []byte("null")) {
return nil, fmt.Errorf("must be an array when present")
}
var aliases []string
if err := json.Unmarshal(raw, &aliases); err != nil {
return nil, fmt.Errorf("must be an array of strings: %w", err)
}
for index, alias := range aliases {
if alias != strings.TrimSpace(alias) || alias == "" {
return nil, fmt.Errorf("value at index %d must be non-empty and trimmed", index)
}
}
return aliases, nil
}
type effectiveSpell struct {
name string
canonicalKey string
aliases map[string]string
}
type effectiveBuilder struct {
spells map[string]*effectiveSpell
canonicalByKey map[string]string
lookup map[string]string
}
func composeEffectiveCatalog(base Catalog, overlays []overlayCatalog) (EffectiveCatalog, error) {
builder := effectiveBuilder{
spells: make(map[string]*effectiveSpell, len(base.spells)),
canonicalByKey: make(map[string]string, len(base.lookup)),
lookup: make(map[string]string, len(base.lookup)),
}
for _, spell := range base.Spells() {
if err := builder.addCanonical(spell.Name); err != nil {
return EffectiveCatalog{}, fmt.Errorf("base catalog: %w", err)
}
for _, alias := range spell.Aliases {
if err := builder.addAlias(spell.Name, alias); err != nil {
return EffectiveCatalog{}, fmt.Errorf("base catalog: %w", err)
}
}
}
sort.Slice(overlays, func(i, j int) bool { return overlays[i].ID < overlays[j].ID })
overlayIDs := make([]string, len(overlays))
for i, overlay := range overlays {
overlayIDs[i] = overlay.ID
for _, spell := range overlay.Spells {
canonical, err := builder.ensureCanonical(spell.Name)
if err != nil {
return EffectiveCatalog{}, fmt.Errorf("catalog %q: %w", overlay.ID, err)
}
aliases, err := decodeAliases(spell.Aliases)
if err != nil {
return EffectiveCatalog{}, fmt.Errorf("catalog %q spell %q aliases: %w", overlay.ID, spell.Name, err)
}
for _, alias := range aliases {
if err := builder.addAlias(canonical, alias); err != nil {
return EffectiveCatalog{}, fmt.Errorf("catalog %q spell %q: %w", overlay.ID, spell.Name, err)
}
}
}
}
canonicalNames := make([]string, 0, len(builder.spells))
for _, spell := range builder.spells {
canonicalNames = append(canonicalNames, spell.name)
}
sort.Strings(canonicalNames)
digest, err := effectiveDigest(base, overlays, builder, canonicalNames)
if err != nil {
return EffectiveCatalog{}, err
}
return EffectiveCatalog{
baseID: base.ID(),
ruleset: base.Ruleset(),
overlayIDs: overlayIDs,
canonicalNames: canonicalNames,
lookup: cloneStringMap(builder.lookup),
digest: digest,
}, nil
}
func (b *effectiveBuilder) ensureCanonical(name string) (string, error) {
key := lookupKey(name)
if key == "" {
return "", fmt.Errorf("spell %q has an empty lookup key", name)
}
if canonical, exists := b.canonicalByKey[key]; exists {
if canonical != name {
return "", fmt.Errorf("canonical lookup key %q has conflicting display names %q and %q", key, canonical, name)
}
return canonical, nil
}
if canonical, exists := b.lookup[key]; exists {
return "", fmt.Errorf("canonical lookup key %q collides with alias of spell %q", key, canonical)
}
spell := &effectiveSpell{name: name, canonicalKey: key, aliases: make(map[string]string)}
b.spells[name] = spell
b.canonicalByKey[key] = name
b.lookup[key] = name
return name, nil
}
func (b *effectiveBuilder) addCanonical(name string) error {
_, err := b.ensureCanonical(name)
return err
}
func (b *effectiveBuilder) addAlias(canonical string, alias string) error {
key := lookupKey(alias)
if key == "" {
return fmt.Errorf("spell %q has an empty alias lookup key", canonical)
}
if existing, exists := b.lookup[key]; exists && existing != canonical {
return fmt.Errorf("lookup key %q maps to spells %q and %q", key, existing, canonical)
}
spell, exists := b.spells[canonical]
if !exists {
return fmt.Errorf("spell %q is not present", canonical)
}
if key == spell.canonicalKey {
return nil
}
b.lookup[key] = canonical
if previous, exists := spell.aliases[key]; !exists || alias < previous {
spell.aliases[key] = alias
}
return nil
}
type effectiveDigestDocument struct {
BaseID string `json:"base_id"`
BaseRuleset string `json:"base_ruleset"`
BaseSource Source `json:"base_source"`
Overlays []effectiveDigestOverlay `json:"overlays"`
Spells []effectiveDigestSpell `json:"spells"`
}
type effectiveDigestOverlay struct {
ID string `json:"id"`
Ruleset string `json:"ruleset"`
Source struct {
Title string `json:"title"`
Version string `json:"version"`
URL string `json:"url"`
License string `json:"license"`
} `json:"source"`
}
type effectiveDigestSpell struct {
Key string `json:"key"`
Name string `json:"name"`
Aliases []effectiveDigestAlias `json:"aliases,omitempty"`
}
type effectiveDigestAlias struct {
Key string `json:"key"`
Display string `json:"display"`
}
func effectiveDigest(base Catalog, overlays []overlayCatalog, builder effectiveBuilder, canonicalNames []string) (string, error) {
document := effectiveDigestDocument{
BaseID: base.ID(),
BaseRuleset: base.Ruleset(),
BaseSource: base.Source(),
Overlays: make([]effectiveDigestOverlay, len(overlays)),
Spells: make([]effectiveDigestSpell, 0, len(canonicalNames)),
}
for index, overlay := range overlays {
document.Overlays[index].ID = overlay.ID
document.Overlays[index].Ruleset = overlay.Ruleset
version, err := decodeOptionalString(overlay.Source.Version)
if err != nil {
return "", fmt.Errorf("catalog %q source version: %w", overlay.ID, err)
}
url, err := decodeOptionalString(overlay.Source.URL)
if err != nil {
return "", fmt.Errorf("catalog %q source url: %w", overlay.ID, err)
}
license, err := decodeOptionalString(overlay.Source.License)
if err != nil {
return "", fmt.Errorf("catalog %q source license: %w", overlay.ID, err)
}
document.Overlays[index].Source.Title = overlay.Source.Title
document.Overlays[index].Source.Version = version
document.Overlays[index].Source.URL = url
document.Overlays[index].Source.License = license
}
for _, name := range canonicalNames {
spell := builder.spells[name]
digestSpell := effectiveDigestSpell{Key: spell.canonicalKey, Name: spell.name}
keys := make([]string, 0, len(spell.aliases))
for key := range spell.aliases {
keys = append(keys, key)
}
sort.Strings(keys)
for _, key := range keys {
digestSpell.Aliases = append(digestSpell.Aliases, effectiveDigestAlias{Key: key, Display: spell.aliases[key]})
}
document.Spells = append(document.Spells, digestSpell)
}
raw, err := json.Marshal(document)
if err != nil {
return "", fmt.Errorf("encode effective spell catalog digest: %w", err)
}
sum := sha256.Sum256(raw)
return "sha256:" + hex.EncodeToString(sum[:]), nil
}
func cloneStringMap(values map[string]string) map[string]string {
if len(values) == 0 {
return nil
}
out := make(map[string]string, len(values))
for key, value := range values {
out[key] = value
}
return out
}
func cloneReferenceSet(in contracts.ReferenceSet) contracts.ReferenceSet {
if len(in.Slots) == 0 {
return contracts.ReferenceSet{}
}
out := contracts.ReferenceSet{Slots: make(map[string]contracts.ResolvedReferenceSlot, len(in.Slots))}
for name, slot := range in.Slots {
slot.Slot.AcceptedMediaTypes = append([]string(nil), slot.Slot.AcceptedMediaTypes...)
items := make([]contracts.ReferenceItem, len(slot.Items))
for index, item := range slot.Items {
item.Content = append([]byte(nil), item.Content...)
items[index] = item
}
slot.Items = items
out.Slots[name] = slot
}
return out
}

View File

@@ -0,0 +1,279 @@
package catalog
import (
"encoding/json"
"reflect"
"sort"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
)
func TestResolveEffectiveCatalogBaseAndImmutability(t *testing.T) {
effective, err := ResolveEffectiveCatalog(contracts.ReferenceSet{})
if err != nil {
t.Fatal(err)
}
if effective.BaseID() != SRD5E2014ID || effective.Ruleset() != SRD5E2014Ruleset {
t.Fatalf("identity = %q/%q", effective.BaseID(), effective.Ruleset())
}
if got := effective.OverlayIDs(); len(got) != 0 {
t.Fatalf("overlay IDs = %#v", got)
}
names := effective.CanonicalNames()
if len(names) != 319 || !sort.StringsAreSorted(names) {
t.Fatalf("canonical names count/order = %d/%t", len(names), sort.StringsAreSorted(names))
}
if got, ok := effective.Lookup(" HUNTER'S MARK "); !ok || got != "Hunters Mark" {
t.Fatalf("Hunter's Mark lookup = %q, present=%t", got, ok)
}
if got, ok := effective.Lookup("Definitely Not A Spell"); ok || got != "" {
t.Fatalf("unknown lookup = %q, present=%t", got, ok)
}
if !strings.HasPrefix(effective.Digest(), "sha256:") {
t.Fatalf("digest = %q", effective.Digest())
}
names[0] = "changed"
ids := effective.OverlayIDs()
ids = append(ids, "changed")
again, err := ResolveEffectiveCatalog(contracts.ReferenceSet{})
if err != nil {
t.Fatal(err)
}
if reflect.DeepEqual(names, again.CanonicalNames()) || len(again.OverlayIDs()) != 0 {
t.Fatal("effective catalog exposed mutable result storage")
}
}
func TestResolveEffectiveCatalogAddsAndAugmentsSpells(t *testing.T) {
overlay := testOverlayJSON(t, testOverlayCatalog(
"campaign.example",
testOverlaySpell("Aegis of Emberfall", "Emberfall Aegis"),
testOverlaySpell("Cure Wounds", "Healing Touch"),
))
effective, err := ResolveEffectiveCatalog(overlayReference([]byte(overlay), "application/json"))
if err != nil {
t.Fatal(err)
}
if got := effective.OverlayIDs(); !reflect.DeepEqual(got, []string{"campaign.example"}) {
t.Fatalf("overlay IDs = %#v", got)
}
if len(effective.CanonicalNames()) != 320 {
t.Fatalf("canonical name count = %d, want 320", len(effective.CanonicalNames()))
}
if got, ok := effective.Lookup("aegis of emberfall"); !ok || got != "Aegis of Emberfall" {
t.Fatalf("new spell lookup = %q, present=%t", got, ok)
}
if got, ok := effective.Lookup("Emberfall Aegis"); !ok || got != "Aegis of Emberfall" {
t.Fatalf("new alias lookup = %q, present=%t", got, ok)
}
if got, ok := effective.Lookup("Healing Touch"); !ok || got != "Cure Wounds" {
t.Fatalf("augmentation alias lookup = %q, present=%t", got, ok)
}
for _, name := range effective.CanonicalNames() {
if name == "Emberfall Aegis" || name == "Healing Touch" {
t.Fatalf("alias %q was exposed as a canonical name", name)
}
}
}
func TestResolveEffectiveCatalogTreatsRepeatedAliasesAsIdempotent(t *testing.T) {
single := testOverlayJSON(t, testOverlayCatalog(
"campaign.example",
testOverlaySpell("Aegis of Emberfall", "Emberfall Aegis"),
))
repeated := testOverlayJSON(t, testOverlayCatalog(
"campaign.example",
testOverlaySpell("Aegis of Emberfall", "Emberfall Aegis"),
testOverlaySpell("Aegis of Emberfall", "emberfall aegis"),
))
first, err := ResolveEffectiveCatalog(overlayReference([]byte(single), "application/json"))
if err != nil {
t.Fatal(err)
}
second, err := ResolveEffectiveCatalog(overlayReference([]byte(repeated), "application/json"))
if err != nil {
t.Fatal(err)
}
if first.Digest() != second.Digest() || !reflect.DeepEqual(first.CanonicalNames(), second.CanonicalNames()) {
t.Fatalf("repeated alias changed effective result: digest=%q/%q", first.Digest(), second.Digest())
}
}
func TestResolveEffectiveCatalogDigestIgnoresOrderingAndFormatting(t *testing.T) {
first := `{"schema_version":"notarius.dnd.spell-catalog-overlay.v1","catalogs":[{"id":"campaign.z","ruleset":"dnd-5e-2014","source":{"title":"Z spells","version":"1","url":"","license":""},"spells":[{"name":"Aegis of Emberfall","aliases":["Z Alias","Another Alias"]}]},{"id":"campaign.a","ruleset":"dnd-5e-2014","source":{"title":"A spells","version":"1","url":"","license":""},"spells":[{"name":"Cure Wounds","aliases":["Healing Touch","Cure Mend"]}]}]}`
second := `{
"catalogs": [
{"source":{"license":"","url":"","version":"1","title":"A spells"},"spells":[{"aliases":["Cure Mend","Healing Touch"],"name":"Cure Wounds"}],"ruleset":"dnd-5e-2014","id":"campaign.a"},
{"spells":[{"aliases":["Another Alias","Z Alias"],"name":"Aegis of Emberfall"}],"source":{"title":"Z spells","version":"1","url":"","license":""},"ruleset":"dnd-5e-2014","id":"campaign.z"}
],
"schema_version":"notarius.dnd.spell-catalog-overlay.v1"
}`
firstCatalog, err := ResolveEffectiveCatalog(overlayReference([]byte(first), "application/json"))
if err != nil {
t.Fatal(err)
}
secondCatalog, err := ResolveEffectiveCatalog(overlayReference([]byte(second), "application/json"))
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(firstCatalog.CanonicalNames(), secondCatalog.CanonicalNames()) || firstCatalog.Digest() != secondCatalog.Digest() {
t.Fatalf("reordered overlays changed effective result: names=%t digest=%q/%q", reflect.DeepEqual(firstCatalog.CanonicalNames(), secondCatalog.CanonicalNames()), firstCatalog.Digest(), secondCatalog.Digest())
}
}
func TestResolveEffectiveCatalogRejectsInvalidOverlays(t *testing.T) {
valid := validOverlayJSON()
tests := []struct {
name string
content string
}{
{name: "wrong schema version", content: strings.Replace(valid, "notarius.dnd.spell-catalog-overlay.v1", "notarius.dnd.spell-catalog-overlay.v2", 1)},
{name: "empty catalogs", content: `{"schema_version":"notarius.dnd.spell-catalog-overlay.v1","catalogs":[]}`},
{name: "duplicate IDs", content: testOverlayJSON(t, testOverlayCatalog("campaign.example", testOverlaySpell("Aegis of Emberfall")), testOverlayCatalog("campaign.example", testOverlaySpell("Another Spell")))},
{name: "wrong ruleset", content: strings.Replace(valid, `"ruleset":"dnd-5e-2014"`, `"ruleset":"dnd-5e-other"`, 1)},
{name: "empty ID", content: strings.Replace(valid, `"id":"campaign.example"`, `"id":""`, 1)},
{name: "empty source title", content: strings.Replace(valid, `"title":"Example campaign spells"`, `"title":""`, 1)},
{name: "empty spells", content: testOverlayJSON(t, testOverlayCatalog("campaign.example"))},
{name: "empty spell name", content: strings.Replace(valid, `"name":"Aegis of Emberfall"`, `"name":""`, 1)},
{name: "null aliases", content: strings.Replace(valid, `"aliases":["Emberfall Aegis"]`, `"aliases":null`, 1)},
{name: "unknown bundle field", content: strings.Replace(valid, `"schema_version":"notarius.dnd.spell-catalog-overlay.v1","catalogs"`, `"schema_version":"notarius.dnd.spell-catalog-overlay.v1","unexpected":true,"catalogs"`, 1)},
{name: "unknown catalog field", content: strings.Replace(valid, `"id":"campaign.example","ruleset"`, `"id":"campaign.example","unexpected":true,"ruleset"`, 1)},
{name: "unknown source field", content: strings.Replace(valid, `"title":"Example campaign spells","version"`, `"title":"Example campaign spells","unexpected":true,"version"`, 1)},
{name: "unknown spell field", content: strings.Replace(valid, `"name":"Aegis of Emberfall","aliases"`, `"name":"Aegis of Emberfall","unexpected":true,"aliases"`, 1)},
{name: "trailing JSON", content: valid + `{}`},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
if _, err := ResolveEffectiveCatalog(overlayReference([]byte(test.content), "application/json")); err == nil {
t.Fatal("invalid overlay was accepted")
}
})
}
}
func TestResolveEffectiveCatalogRejectsReferenceMultiplicityAndMediaType(t *testing.T) {
valid := []byte(validOverlayJSON())
tests := []struct {
name string
refs contracts.ReferenceSet
}{
{
name: "multiple items",
refs: contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
SpellCatalogReferenceSlot: {Items: []contracts.ReferenceItem{
{MediaType: "application/json", Content: valid},
{MediaType: "application/json", Content: valid},
}},
}},
},
{
name: "non JSON item",
refs: overlayReference(valid, "text/plain"),
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
if _, err := ResolveEffectiveCatalog(test.refs); err == nil {
t.Fatal("invalid reference set was accepted")
}
})
}
for _, refs := range []contracts.ReferenceSet{
{}, {Slots: map[string]contracts.ResolvedReferenceSlot{SpellCatalogReferenceSlot: {}}},
} {
if effective, err := ResolveEffectiveCatalog(refs); err != nil || len(effective.CanonicalNames()) != 319 {
t.Fatalf("missing overlay should use base catalog: effective=%#v err=%v", effective, err)
}
}
}
func TestResolveEffectiveCatalogRejectsCrossSpellCollisions(t *testing.T) {
tests := []struct {
name string
spells []overlaySpell
content string
}{
{
name: "same key display conflict with embedded canonical",
spells: []overlaySpell{testOverlaySpell("cure wounds")},
},
{
name: "same key display conflict between canonical names",
spells: []overlaySpell{testOverlaySpell("Moon Beam"), testOverlaySpell("moon beam")},
},
{
name: "canonical versus alias",
spells: []overlaySpell{testOverlaySpell("Alpha Spell", "Beta Spell"), testOverlaySpell("Beta Spell")},
},
{
name: "alias versus canonical",
spells: []overlaySpell{testOverlaySpell("Alpha Spell"), testOverlaySpell("Beta Spell", "Alpha Spell")},
},
{
name: "alias versus alias",
spells: []overlaySpell{testOverlaySpell("Alpha Spell", "Shared Name"), testOverlaySpell("Beta Spell", "Shared Name")},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
content := test.content
if content == "" {
content = testOverlayJSON(t, testOverlayCatalog("campaign.example", test.spells...))
}
if _, err := ResolveEffectiveCatalog(overlayReference([]byte(content), "application/json")); err == nil {
t.Fatal("cross-spell collision was accepted")
}
})
}
}
func overlayReference(content []byte, mediaType string) contracts.ReferenceSet {
return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
SpellCatalogReferenceSlot: {
Items: []contracts.ReferenceItem{{
SlotName: SpellCatalogReferenceSlot,
MediaType: mediaType,
Content: append([]byte(nil), content...),
}},
},
}}
}
func testOverlaySpell(name string, aliases ...string) overlaySpell {
content, err := json.Marshal(aliases)
if err != nil {
panic(err)
}
return overlaySpell{Name: name, Aliases: content}
}
func testOverlayCatalog(id string, spells ...overlaySpell) overlayCatalog {
return overlayCatalog{
ID: id,
Ruleset: SRD5E2014Ruleset,
Source: overlaySource{
Title: id + " spells",
Version: json.RawMessage(`"1"`),
URL: json.RawMessage(`""`),
License: json.RawMessage(`""`),
},
Spells: spells,
}
}
func testOverlayJSON(t *testing.T, catalogs ...overlayCatalog) string {
t.Helper()
content, err := json.Marshal(overlayBundle{SchemaVersion: overlaySchemaVersion, Catalogs: catalogs})
if err != nil {
t.Fatal(err)
}
return string(content)
}
func validOverlayJSON() string {
return `{"schema_version":"notarius.dnd.spell-catalog-overlay.v1","catalogs":[{"id":"campaign.example","ruleset":"dnd-5e-2014","source":{"title":"Example campaign spells","version":"1","url":"","license":""},"spells":[{"name":"Aegis of Emberfall","aliases":["Emberfall Aegis"]}]}]}`
}

View File

@@ -0,0 +1,113 @@
package catalog
import (
"context"
"fmt"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
spellcatalog "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/spells/catalog"
spellshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/shape"
)
const (
Key = "extract/dnd/spells/catalog"
ReasonCode = "unknown_spell"
maxIssues = 20
)
type Options struct{}
type Validator struct {
catalog spellcatalog.EffectiveCatalog
}
var _ contracts.TypedValidator[dnd.SpellList] = (*Validator)(nil)
func New(_ Options, references ...contracts.ReferenceSet) (*Validator, error) {
if len(references) > 1 {
return nil, fmt.Errorf("spell catalog validator accepts at most one reference set")
}
var referenceSet contracts.ReferenceSet
if len(references) == 1 {
referenceSet = references[0]
}
effective, err := spellcatalog.ResolveEffectiveCatalog(referenceSet)
if err != nil {
return nil, err
}
return &Validator{catalog: effective}, nil
}
func (v *Validator) Name() string { return Key }
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *Validator) Validate(_ context.Context, req contracts.TypedValidationRequest[dnd.SpellList]) (contracts.ValidationResult, error) {
if err := spellshape.Validate(req.Value); err != nil {
return contracts.ValidationResult{Approved: true}, nil
}
unknown := make([]unknownSpell, 0)
for index, spell := range req.Value.SpellCasts {
name := strings.TrimSpace(spell.Spell)
if name == "" {
continue
}
if _, ok := v.catalog.Lookup(name); !ok {
unknown = append(unknown, unknownSpell{index: index, name: name})
}
}
if len(unknown) == 0 {
return contracts.ValidationResult{Approved: true}, nil
}
return rejection(unknown), nil
}
type unknownSpell struct {
index int
name string
}
func rejection(unknown []unknownSpell) contracts.ValidationResult {
displayed := unknown
if len(displayed) > maxIssues {
displayed = displayed[:maxIssues]
}
issues := make([]string, len(displayed))
for index, item := range displayed {
issues[index] = fmt.Sprintf("spell_casts[%d].spell %q", item.index, item.name)
}
message := fmt.Sprintf("unknown spell names: %s", strings.Join(issues, ", "))
if omitted := len(unknown) - len(displayed); omitted > 0 {
message += fmt.Sprintf("; %d additional issue(s) omitted", omitted)
}
return contracts.ValidationResult{Approved: false, ReasonCode: ReasonCode, Message: message}
}
func Spec() pipeline.ValidatorSpec {
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
}
func Register(registry *pipeline.ValidatorRegistry) error {
return pipeline.RegisterTypedValidatorBuilder(registry, dnd.SpellListKind, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.TypedValidator[dnd.SpellList], error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return New(options, request.References)
})
}
func DecodeOptions(options map[string]any) (Options, error) {
if err := pipeline.RejectUnknownOptions(options); err != nil {
return Options{}, err
}
return Options{}, nil
}
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }

View File

@@ -0,0 +1,176 @@
package catalog
import (
"context"
"fmt"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
spellreference "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/spells/catalog"
)
func TestValidatorApprovesCanonicalNormalizedAndOverlayAliasNames(t *testing.T) {
validator, err := New(Options{}, overlayReferences())
if err != nil {
t.Fatalf("New() error = %v, want nil", err)
}
value := spellList(" cure wounds ", "Emberfall Aegis")
result, err := validator.Validate(context.Background(), validationRequest(value))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
if !result.Approved {
t.Fatalf("Validate() = %#v, want approved", result)
}
}
func TestValidatorRejectsMultipleUnknownCastsInStableOrder(t *testing.T) {
value := spellList("Unknown First", "Cure Wounds", "Unknown Second")
validator, err := New(Options{})
if err != nil {
t.Fatalf("New() error = %v, want nil", err)
}
result, err := validator.Validate(context.Background(), validationRequest(value))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
if result.Approved || result.ReasonCode != ReasonCode {
t.Fatalf("Validate() = %#v, want unknown-spell rejection", result)
}
if want := `spell_casts[0].spell "Unknown First", spell_casts[2].spell "Unknown Second"`; !strings.Contains(result.Message, want) {
t.Fatalf("message = %q, want %q", result.Message, want)
}
}
func TestValidatorBoundsUnknownCastMessage(t *testing.T) {
value := dnd.SpellList{SpellCasts: make([]dnd.SpellCast, 22)}
for index := range value.SpellCasts {
value.SpellCasts[index] = validCast(fmt.Sprintf("Unknown Spell %02d", index))
}
validator, err := New(Options{})
if err != nil {
t.Fatalf("New() error = %v, want nil", err)
}
result, err := validator.Validate(context.Background(), validationRequest(value))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
if result.Approved || result.ReasonCode != ReasonCode {
t.Fatalf("Validate() = %#v, want unknown-spell rejection", result)
}
if got := strings.Count(result.Message, "spell_casts["); got != maxIssues {
t.Fatalf("message includes %d issues, want %d: %q", got, maxIssues, result.Message)
}
if !strings.Contains(result.Message, "2 additional issue(s) omitted") || strings.Contains(result.Message, "Unknown Spell 21") {
t.Fatalf("message = %q, want bounded diagnostics", result.Message)
}
}
func TestValidatorDefersInvalidShape(t *testing.T) {
value := dnd.SpellList{SpellCasts: []dnd.SpellCast{{Spell: "Unknown Spell"}}}
validator, err := New(Options{})
if err != nil {
t.Fatalf("New() error = %v, want nil", err)
}
result, err := validator.Validate(context.Background(), validationRequest(value))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
if !result.Approved || result.ReasonCode != "" || result.Message != "" {
t.Fatalf("Validate() = %#v, want approval without catalog diagnostics", result)
}
}
func TestValidatorDoesNotMutateArtifactOrCatalog(t *testing.T) {
value := spellList("Cure Wounds")
before := value
validator, err := New(Options{}, overlayReferences())
if err != nil {
t.Fatalf("New() error = %v, want nil", err)
}
names := validator.catalog.CanonicalNames()
names[0] = "mutated"
result, err := validator.Validate(context.Background(), validationRequest(value))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
if !result.Approved || !reflect.DeepEqual(value, before) {
t.Fatalf("Validate() = %#v and value = %#v, want approval without mutation", result, value)
}
if _, ok := validator.catalog.Lookup("Aegis of Emberfall"); !ok {
t.Fatal("catalog lost overlay canonical name after mutating returned names")
}
}
func TestValidatorUsesStrictEmptyOptions(t *testing.T) {
if _, err := DecodeOptions(nil); err != nil {
t.Fatalf("DecodeOptions(nil) error = %v, want nil", err)
}
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
t.Fatal("DecodeOptions() error = nil, want unknown option error")
}
}
func TestValidatorSpecAndRegister(t *testing.T) {
spec := Spec()
if spec.Key != Key || spec.ExecutionClass != contracts.ExecutionClassDeterministic {
t.Fatalf("Spec() = %#v, want deterministic catalog validator", spec)
}
registry := pipeline.NewValidatorRegistry()
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
registered, ok := registry.Spec(Key)
if !ok || registered != spec {
t.Fatalf("registry.Spec(%q) = %#v, present = %t; want %#v", Key, registered, ok, spec)
}
}
func validationRequest(value dnd.SpellList) contracts.TypedValidationRequest[dnd.SpellList] {
return contracts.TypedValidationRequest[dnd.SpellList]{Value: value}
}
func spellList(names ...string) dnd.SpellList {
value := dnd.SpellList{SpellCasts: make([]dnd.SpellCast, len(names))}
for index, name := range names {
value.SpellCasts[index] = validCast(name)
}
return value
}
func validCast(name string) dnd.SpellCast {
return dnd.SpellCast{
Caster: "Aria",
Spell: name,
Effect: "heals an ally",
NarrativeDescription: "Aria restores Borin.",
SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 1, EndUnitID: 1}},
}
}
func overlayReferences() contracts.ReferenceSet {
content := []byte(`{
"schema_version": "notarius.dnd.spell-catalog-overlay.v1",
"catalogs": [{
"id": "campaign-spells",
"ruleset": "dnd-5e-2014",
"source": {"title": "Campaign spell names"},
"spells": [{"name": "Aegis of Emberfall", "aliases": ["Emberfall Aegis"]}]
}]
}`)
return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
spellreference.SpellCatalogReferenceSlot: {
Slot: contracts.ReferenceSlot{Name: spellreference.SpellCatalogReferenceSlot},
Items: []contracts.ReferenceItem{{
SlotName: spellreference.SpellCatalogReferenceSlot,
MediaType: "application/json",
Content: content,
}},
},
}}
}