From 3eb68baca639555c295b9272c435af8914728eab Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 20 Jul 2026 21:25:02 +0000 Subject: [PATCH] Document D&D spell normalization behavior --- docs/config.md | 11 ++- docs/integrations/dnd-spell-artifacts.md | 45 ++++++++- docs/internal/modules.md | 26 ++++- docs/internal/overview.md | 5 +- docs/roadmap/dnd-spell-normalization.md | 80 ++++----------- docs/roadmap/future.md | 5 +- examples/dnd-spells-production.config.yml | 2 + examples/dnd-spells.config.yml | 1 + .../dnd/normalize/spells/fixture_test.go | 56 +++++++++++ .../testdata/normalizer_accepted_cases.json | 99 +++++++++++++++++++ 10 files changed, 257 insertions(+), 73 deletions(-) create mode 100644 internal/modules/dnd/normalize/spells/fixture_test.go create mode 100644 internal/modules/dnd/normalize/spells/testdata/normalizer_accepted_cases.json diff --git a/docs/config.md b/docs/config.md index 9a0c0f7..7093bd6 100644 --- a/docs/config.md +++ b/docs/config.md @@ -285,6 +285,7 @@ production validators do not call the LLM and must not set `llm_profile`. | extract | `dnd/spells` | Extracts typed D&D spell-list artifacts. | | merge | `appendorder` | Combines typed artifacts in chunk order. | | normalize | `noop` | Passes merged typed artifacts through unchanged. | +| normalize | `dnd/spells` | Deterministically canonicalizes and de-duplicates typed D&D spell-list artifacts. | | output | `json` | Produces JSON output files for normalized `application/json` lanes. | ## Implemented Production Validators @@ -300,7 +301,8 @@ production validators do not call the LLM and must not set `llm_profile`. | `extract/dnd/spells/source_refs` | deterministic | Rejects missing or invalid D&D spell source references. | | `extract/dnd/spells/source_relatedness` | deterministic | Emits warnings when a spell name is not found near its cited source text. | -The production default chain for the `dnd/spells` extractor is: +The production default chain for `dnd/spells` is used for both its extract and +normalize stages: ```yaml validators: @@ -342,6 +344,13 @@ not allow multiple files. Its format is defined in the The extractor uses campaign references only as supporting disambiguation material; spell casts still must be present in the source transcript. +The `dnd/spells` normalizer declares the same optional `spell_catalog` slot. +When an overlay is used, bind it independently under +`artifacts..normalize.references.spell_catalog`; normalize-stage +references are local to that stage and are not inherited from extraction. The +normalizer uses the embedded SRD catalog when no normalize-stage overlay is +bound. + ## State Surfaces The `output`, `cache`, and `debug` top-level fields select independent physical diff --git a/docs/integrations/dnd-spell-artifacts.md b/docs/integrations/dnd-spell-artifacts.md index adece4b..d1f0588 100644 --- a/docs/integrations/dnd-spell-artifacts.md +++ b/docs/integrations/dnd-spell-artifacts.md @@ -68,6 +68,39 @@ Reference slot keys and accepted file types are defined in supporting disambiguation material, not source evidence, and are not addressable through `source_refs`. +## Normalization Behavior + +When the `dnd/spells` normalizer is selected, each recognized spell name is +rewritten to the effective catalog's canonical display name. Lookup uses the +catalog's case-insensitive, whitespace-normalizing, apostrophe-normalizing, and +alias rules. Unknown names are preserved exactly for the normalize validators; +the normalizer does not guess or apply fuzzy matching. + +Each cast's `source_refs` is copied, sorted by exact `source_id`, +`start_unit_id`, and `end_unit_id`, and stripped of exact structural +duplicates. Adjacent or overlapping ranges are not merged, and the normalizer +does not synthesize references or change their boundaries. + +After those per-cast changes, duplicate identity requires the same canonical +spell name, the same caster after case folding and whitespace normalization, +and the same complete, non-empty set of source references valid for the source +document. Only the first occurrence is retained, in stable order. Its caster, +effect, narrative description, and canonical references are preserved without +prose merging or source union. Unknown names, empty or invalid evidence, and +casts with different evidence remain separate. + +Mutation and duplicate decisions are returned through the normal warnings +surface. Warning scopes use the merged input index, such as `spell_casts[0]`, +so they remain meaningful even when a later duplicate is removed. The +normalizer uses these reason codes: + +| Reason code | Meaning | +| --- | --- | +| `spell_name_canonicalized` | A catalog lookup replaced an input name with its canonical display name. | +| `spell_name_unresolved` | A name was not found in the effective catalog and was retained unchanged. | +| `source_references_normalized` | Reference order changed or exact duplicate references were removed. | +| `duplicate_spell_cast_collapsed` | A later cast matched the retained cast's complete duplicate identity. | + ## Manifest Metadata The extractor adds prompt and response-schema provenance under the artifact lane @@ -88,6 +121,11 @@ manifest metadata: "catalog_base_id": "dnd-5e-2014-srd-spells", "catalog_digest": "sha256:...", "catalog_overlay_ids": ["campaign.example"] + }, + "normalizer": { + "catalog_base_id": "dnd-5e-2014-srd-spells", + "catalog_digest": "sha256:...", + "catalog_overlay_ids": ["campaign.example"] } } } @@ -96,6 +134,7 @@ manifest metadata: `catalog_digest` identifies the effective semantic catalog, while `catalog_overlay_ids` is sorted and empty for a base-only configuration. Raw prompt, schema, catalog, alias, and local overlay-file content are not -included in manifest metadata. Overlay origin, media type, byte size, and raw -digest are recorded separately in the manifest's reference provenance; see -the [JSON output contract](json-output.md#manifestjson). +included in manifest metadata. The `normalizer` metadata uses the same catalog +identity fields when that module is selected. Overlay origin, media type, byte +size, and raw digest are recorded separately in the manifest's reference +provenance; see the [JSON output contract](json-output.md#manifestjson). diff --git a/docs/internal/modules.md b/docs/internal/modules.md index 429f9d3..fe87e7d 100644 --- a/docs/internal/modules.md +++ b/docs/internal/modules.md @@ -186,6 +186,30 @@ function. The normalizer returns the merged domain value unchanged and is reusable for any registered artifact type. +### `internal/modules/dnd/normalize/spells` + +The typed spell normalizer resolves the optional `spell_catalog` reference into +the same immutable SRD-plus-overlay effective catalog used by spell extraction +and catalog validation. It performs no LLM calls. For each spell cast it +canonicalizes recognized names using the catalog's case, whitespace, +apostrophe, and alias rules; sorts source references by source identity and +unit boundaries; removes only exact reference duplicates; and emits bounded, +scoped warnings for each mutation or unresolved name. + +After those per-cast changes, it collapses only casts with the same canonical +spell, case-folded and whitespace-normalized caster, and complete non-empty +valid source-reference set. It retains the first occurrence and its caster, +effect, narrative description, and stable order. Unknown names, empty or +invalid evidence, and adjacent or overlapping but different ranges remain +unchanged for validation. + +The normalizer exposes the effective catalog digest as its independently scoped +`effective_catalog` checkpoint fingerprint and reports catalog base ID, digest, +and overlay IDs as manifest metadata. Catalog contents, reference paths, and +raw overlay bytes are not included in either surface. The normalize-stage +reference is stage-local, so an overlay-capable pipeline binds the catalog +independently for extraction and normalization. + ## Output Encoder ### `internal/modules/generic/output/json` @@ -240,7 +264,7 @@ complete framework registry set and one LLM asset registry. It invokes `internal/modules/seriatim/register`, and `internal/modules/dnd/register` in that order, then exposes the matching catalog for resolution. The generic and Seriatim registrars own their production leaf registrations. The D&D registrar -owns D&D leaf registrations, the spell default-validator chain, and D&D +owns D&D leaf registrations, the typed spell default-validator chains, and D&D prompt/schema asset collection. Concrete implementation packages do not import generic implementation diff --git a/docs/internal/overview.md b/docs/internal/overview.md index c22b38e..46d1f21 100644 --- a/docs/internal/overview.md +++ b/docs/internal/overview.md @@ -65,8 +65,8 @@ run-local construction closures. Preparation injects shared dependencies and constructs input, chunk, validators, ordered lanes, and output before source parsing. Production modules use strict construction-time option decoding, and LLM-backed modules retain the injected shared client. The D&D family registers -the canonical `dnd/spell-list` codec, typed spell extractor and validators, and -kind-specific generic merge and normalize strategies; generic JSON validators +the canonical `dnd/spell-list` codec, typed spell extractor, normalizer, and +validators, plus kind-specific generic merge strategies; generic JSON validators use the serialized-validation contract. The runner executes lanes through private exact-type-checked closures, coordinates extract results independently of completion timing, and serializes artifacts only through their codec at @@ -90,6 +90,7 @@ Configuration. The implemented module packages are: | `internal/modules/dnd/spells/catalog` | Embeds and validates the versioned D&D 5e 2014 SRD catalog, composes optional overlays, and provides immutable effective lookup. | | `internal/modules/generic/merge/appendorder` | Combines accepted extraction results in chunk order. | | `internal/modules/generic/normalize/noop` | Preserves accepted merged output. | +| `internal/modules/dnd/normalize/spells` | Canonicalizes catalog-backed spell names and exact source references, conservatively collapses duplicate casts, and reports deterministic warnings and independently scoped catalog checkpoint identity. | | `internal/modules/generic/output/json` | Encodes manifests, lane payloads, warnings, and rejections as logical JSON files. | `internal/modules/dnd/shared` owns reusable D&D prompt fragments, diff --git a/docs/roadmap/dnd-spell-normalization.md b/docs/roadmap/dnd-spell-normalization.md index b23c24c..37eb859 100644 --- a/docs/roadmap/dnd-spell-normalization.md +++ b/docs/roadmap/dnd-spell-normalization.md @@ -2,14 +2,8 @@ ## Status -<<<<<<< HEAD -Proposed as the next D&D pipeline milestone. This feature completes the first -domain-specific normalize stage for the spell pipeline before work begins on -NPC and combat-turn artifacts. -======= -Accepted target state. Implementation details are maintained separately in -`docs/roadmap/implementation.md`. ->>>>>>> 6fbdf6b (Add feature roadmap and implementation plan for D&D spell normalization module) +Implemented deterministic feature. Implementation details are maintained +separately in `docs/roadmap/implementation.md`. ## Objective @@ -18,13 +12,8 @@ that emits canonical catalog names, removes only clearly identical duplicate casts, preserves source provenance, and makes every mutation visible through scoped warnings. -<<<<<<< HEAD -The milestone should improve the consistency of durable spell output without -adding another LLM call or introducing fuzzy repair policy. -======= The normalizer improves the consistency of durable spell output without adding another LLM call or introducing fuzzy repair policy. ->>>>>>> 6fbdf6b (Add feature roadmap and implementation plan for D&D spell normalization module) ## Target Behavior @@ -43,24 +32,16 @@ depend on extractor-private state. The normalizer contributes its effective catalog digest through `pipeline.CheckpointFingerprintProvider`. Changing the embedded catalog, composition policy, or normalize-stage overlay therefore invalidates reusable -<<<<<<< HEAD -normalize checkpoints. -======= normalize checkpoints. It also records catalog base ID, digest, and overlay IDs as manifest metadata. Neither identity surface includes catalog contents or reference paths. ->>>>>>> 6fbdf6b (Add feature roadmap and implementation plan for D&D spell normalization module) ### Canonical spell names For every spell cast, look up the extracted name using the effective catalog's existing case, whitespace, apostrophe, and alias rules. Replace a recognized value with its canonical display name. This is the only spell-name repair in -<<<<<<< HEAD -the initial feature. -======= this feature. ->>>>>>> 6fbdf6b (Add feature roadmap and implementation plan for D&D spell normalization module) Do not use edit distance, phonetic matching, model judgment, or another fuzzy heuristic. If a value does not resolve, retain it unchanged and emit a scoped @@ -69,12 +50,8 @@ acceptance or rejection. Emit a warning for each changed spell name. Diagnostics should identify the artifact index and the original and canonical values without modifying other -<<<<<<< HEAD -fields. -======= fields. Diagnostics are deterministic and bounded: user-controlled spell names are truncated to 128 Unicode code points before display. ->>>>>>> 6fbdf6b (Add feature roadmap and implementation plan for D&D spell normalization module) ### Source-reference normalization @@ -84,12 +61,8 @@ overlapping ranges, because doing so could broaden the evidence attributed to an event. The normalizer must not synthesize source references, alter source-unit -<<<<<<< HEAD -boundaries, or use auxiliary references as evidence. -======= boundaries, or use auxiliary references as evidence. Reordering references or removing exact duplicates emits a scoped warning for the affected input cast. ->>>>>>> 6fbdf6b (Add feature roadmap and implementation plan for D&D spell normalization module) ### Conservative duplicate collapse @@ -98,12 +71,8 @@ event only when all of the following match: - canonical spell name; - caster after case folding and whitespace normalization; and -<<<<<<< HEAD -- the complete canonical source-reference set. -======= - the complete canonical source-reference set, which must be non-empty and valid against the source document. ->>>>>>> 6fbdf6b (Add feature roadmap and implementation plan for D&D spell normalization module) Collapse each such group into its first occurrence, preserving stable pipeline order. Retain the first cast's caster, effect, and narrative description. Do @@ -116,14 +85,11 @@ the removed indices. Casts with different evidence remain distinct even when their spell and caster match. In particular, adjacency at a chunk or scene boundary is not sufficient evidence of duplication. -<<<<<<< HEAD -======= Unknown spell names, empty evidence sets, and invalid source references cannot establish duplicate identity. They remain in the artifact for the configured normalize validators to accept or reject. A duplicate warning displays at most 20 removed input indices and reports the exact omitted count. ->>>>>>> 6fbdf6b (Add feature roadmap and implementation plan for D&D spell normalization module) ### Production composition and validation Register the typed spell normalizer in the D&D family and make it the default @@ -141,40 +107,34 @@ without a catalog reference. ## Acceptance Criteria -- Base and overlay aliases are written using canonical catalog display names. -- Unknown names are never guessed, deleted, or silently changed. -- Source references are deterministically sorted and exactly deduplicated. -- Only casts with matching canonical spell, normalized caster, and identical -<<<<<<< HEAD - evidence sets collapse; near or adjacent events remain separate. -- Normalization preserves the first cast's non-name fields and stable order. -- Every name change and duplicate collapse emits a scoped warning. -======= +- [x] Base and overlay aliases are written using canonical catalog display names. +- [x] Unknown names are never guessed, deleted, or silently changed. +- [x] Source references are deterministically sorted and exactly deduplicated. +- [x] Only casts with matching canonical spell, normalized caster, and identical non-empty valid evidence sets collapse; near, invalid, unknown, or adjacent events remain separate. -- Normalization preserves the first cast's non-name fields and stable order. -- Every name change, unresolved name, source-reference mutation, and duplicate +- [x] Normalization preserves the first cast's non-name fields and stable order. +- [x] Every name change, unresolved name, source-reference mutation, and duplicate collapse emits a scoped warning. ->>>>>>> 6fbdf6b (Add feature roadmap and implementation plan for D&D spell normalization module) -- Extract, normalize, and catalog-validation catalog identities agree for the +- [x] Extract, normalize, and catalog-validation catalog identities agree for the same bound references, and catalog changes invalidate normalize checkpoints. -- The maintained production pipeline and current-behavior documentation use +- [x] The maintained production pipeline and current-behavior documentation use the D&D normalizer without changing the durable artifact schema. ## Evaluation -Maintain a small human-reviewed set of representative spell outputs covering -canonical names, aliases, repeated casts, duplicate model output, and scene -boundaries. Use it to review normalization behavior and warnings, not as a -claim that LLM extraction is deterministically correct. +Status: pending qualitative review. No approved human-reviewed transcript +corpus is available locally under repository policy, so no sensitive transcript +content is committed and no claims about real transcript quality are made. + +A compact deterministic fixture set covers canonical names, exact source +reference normalization, repeated casts, duplicate output, and distinct +evidence. It exercises the normalizer contract without requiring paid or +network LLM calls. Record cases where likely duplicates remain because their evidence differs. Those examples should inform later LLM-assisted deduplication work rather than -<<<<<<< HEAD -causing this deterministic milestone to adopt broader heuristics. -======= causing this deterministic feature to adopt broader heuristics. ->>>>>>> 6fbdf6b (Add feature roadmap and implementation plan for D&D spell normalization module) ## Deferred Work @@ -186,10 +146,6 @@ causing this deterministic feature to adopt broader heuristics. - Repair-aware extraction retries or LLM-backed semantic validation. - Changes to the spell artifact schema, including stable cast IDs. -<<<<<<< HEAD -An LLM-backed normalizer is explicitly outside this milestone. If later -======= An LLM-backed normalizer is explicitly outside this feature. If later ->>>>>>> 6fbdf6b (Add feature roadmap and implementation plan for D&D spell normalization module) evaluation justifies one, shared spell-catalog prompt material should be designed at that time around the actual normalization or repair request. diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index 7b523f5..f405791 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -7,11 +7,8 @@ not as committed release dates. ## Near-Term D&D Pipeline -### Solidify Spell Extraction +### Evaluate Spell Extraction And Normalization -- Implement the deterministic catalog-aware normalizer defined in - [D&D Spell Normalization](dnd-spell-normalization.md), including conservative - exact-evidence duplicate collapse. - Evaluate ordinary extraction retries and the completed normalization path against a human-reviewed transcript set before adding repair-aware retries or an LLM-backed semantic validator. diff --git a/examples/dnd-spells-production.config.yml b/examples/dnd-spells-production.config.yml index ca92caf..a870015 100644 --- a/examples/dnd-spells-production.config.yml +++ b/examples/dnd-spells-production.config.yml @@ -29,9 +29,11 @@ pipelines: extract: module: dnd/spells retries: 2 + # Overlay behavior binds the same catalog independently at each stage. references: spell_catalog: ./dnd-spells-catalog.json normalize: module: dnd/spells + # Normalize-stage references are local and must be bound explicitly. references: spell_catalog: ./dnd-spells-catalog.json diff --git a/examples/dnd-spells.config.yml b/examples/dnd-spells.config.yml index be0bca8..5038cb6 100644 --- a/examples/dnd-spells.config.yml +++ b/examples/dnd-spells.config.yml @@ -15,4 +15,5 @@ pipelines: artifacts: spells: extract: dnd/spells + # Base-only behavior: normalization uses the embedded SRD catalog. normalize: dnd/spells diff --git a/internal/modules/dnd/normalize/spells/fixture_test.go b/internal/modules/dnd/normalize/spells/fixture_test.go new file mode 100644 index 0000000..6162add --- /dev/null +++ b/internal/modules/dnd/normalize/spells/fixture_test.go @@ -0,0 +1,56 @@ +package spells + +import ( + "context" + "encoding/json" + "os" + "reflect" + "testing" + + "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd" +) + +type normalizerFixtureSet struct { + Cases []normalizerFixtureCase `json:"cases"` +} + +type normalizerFixtureCase struct { + Name string `json:"name"` + Input dnd.SpellList `json:"input"` + Output dnd.SpellList `json:"output"` + WarningReasonCodes []string `json:"warning_reason_codes"` +} + +func TestNormalizeAcceptedFixtures(t *testing.T) { + data, err := os.ReadFile("testdata/normalizer_accepted_cases.json") + if err != nil { + t.Fatalf("read fixture: %v", err) + } + var fixtures normalizerFixtureSet + if err := json.Unmarshal(data, &fixtures); err != nil { + t.Fatalf("decode fixture: %v", err) + } + if len(fixtures.Cases) == 0 { + t.Fatal("fixture set is empty") + } + + for _, fixture := range fixtures.Cases { + t.Run(fixture.Name, func(t *testing.T) { + result, err := newNormalizer(t).Normalize(context.Background(), normalizeRequestWithSource(fixture.Input, sourceDocument(2))) + if err != nil { + t.Fatalf("Normalize() error = %v, want nil", err) + } + if !reflect.DeepEqual(result.Value, fixture.Output) { + t.Fatalf("normalized value = %#v, want %#v", result.Value, fixture.Output) + } + + gotReasonCodes := make([]string, 0, len(result.Warnings)) + for _, warning := range result.Warnings { + gotReasonCodes = append(gotReasonCodes, warning.ReasonCode) + } + if !reflect.DeepEqual(gotReasonCodes, fixture.WarningReasonCodes) { + t.Fatalf("warning reason codes = %#v, want %#v", gotReasonCodes, fixture.WarningReasonCodes) + } + }) + } +} diff --git a/internal/modules/dnd/normalize/spells/testdata/normalizer_accepted_cases.json b/internal/modules/dnd/normalize/spells/testdata/normalizer_accepted_cases.json new file mode 100644 index 0000000..ebe69b5 --- /dev/null +++ b/internal/modules/dnd/normalize/spells/testdata/normalizer_accepted_cases.json @@ -0,0 +1,99 @@ +{ + "cases": [ + { + "name": "canonicalizes references and collapses exact duplicate", + "input": { + "spell_casts": [ + { + "caster": " Aria ", + "spell": " cure wounds ", + "effect": "first effect", + "narrative_description": "first narrative", + "source_refs": [ + {"source_id": "source", "start_unit_id": 2, "end_unit_id": 2}, + {"source_id": "source", "start_unit_id": 1, "end_unit_id": 1}, + {"source_id": "source", "start_unit_id": 1, "end_unit_id": 1} + ] + }, + { + "caster": "aria", + "spell": "Cure Wounds", + "effect": "duplicate effect", + "narrative_description": "duplicate narrative", + "source_refs": [ + {"source_id": "source", "start_unit_id": 1, "end_unit_id": 1}, + {"source_id": "source", "start_unit_id": 2, "end_unit_id": 2} + ] + }, + { + "caster": "aria", + "spell": "Cure Wounds", + "effect": "distinct effect", + "narrative_description": "distinct narrative", + "source_refs": [ + {"source_id": "source", "start_unit_id": 2, "end_unit_id": 2} + ] + } + ] + }, + "output": { + "spell_casts": [ + { + "caster": " Aria ", + "spell": "Cure Wounds", + "effect": "first effect", + "narrative_description": "first narrative", + "source_refs": [ + {"source_id": "source", "start_unit_id": 1, "end_unit_id": 1}, + {"source_id": "source", "start_unit_id": 2, "end_unit_id": 2} + ] + }, + { + "caster": "aria", + "spell": "Cure Wounds", + "effect": "distinct effect", + "narrative_description": "distinct narrative", + "source_refs": [ + {"source_id": "source", "start_unit_id": 2, "end_unit_id": 2} + ] + } + ] + }, + "warning_reason_codes": [ + "spell_name_canonicalized", + "source_references_normalized", + "duplicate_spell_cast_collapsed" + ] + }, + { + "name": "leaves already normalized output unchanged", + "input": { + "spell_casts": [ + { + "caster": "Aria", + "spell": "Cure Wounds", + "effect": "heals an ally", + "narrative_description": "Aria restores an ally's wounds.", + "source_refs": [ + {"source_id": "source", "start_unit_id": 1, "end_unit_id": 1} + ] + } + ] + }, + "output": { + "spell_casts": [ + { + "caster": "Aria", + "spell": "Cure Wounds", + "effect": "heals an ally", + "narrative_description": "Aria restores an ally's wounds.", + "source_refs": [ + {"source_id": "source", "start_unit_id": 1, "end_unit_id": 1} + ] + } + ] + }, + "warning_reason_codes": [] + } + ] +}