Add feature roadmap and implementation plan for D&D spell normalization module

docs/roadmap/dnd-spell-normalization.md

@@ -0,0 +1,136 @@
+# D&D Spell Normalization
+
+## Status
+
+Proposed as the next D&D pipeline milestone. This feature completes the first
+domain-specific normalize stage for the spell pipeline before work begins on
+NPC and combat-turn artifacts.
+
+## Objective
+
+Replace the spell lane's no-op normalizer with a deterministic D&D normalizer
+that emits canonical catalog names, removes only clearly identical duplicate
+casts, preserves source provenance, and makes every mutation visible through
+scoped warnings.
+
+The milestone should improve the consistency of durable spell output without
+adding another LLM call or introducing fuzzy repair policy.
+
+## Target Behavior
+
+### Effective catalog construction
+
+The normalizer constructs the same immutable SRD-plus-overlay effective
+catalog used by spell extraction and catalog validation. It declares the
+optional `spell_catalog` normalize-stage reference slot and resolves it during
+pipeline preparation, before source parsing or module execution.
+
+An overlay-capable pipeline binds the same catalog file independently at the
+extract and normalize stages. This repetition is intentional: references are
+stage-local, and the normalizer must not reach into a constructed extractor or
+depend on extractor-private state.
+
+The normalizer contributes its effective catalog digest through
+`pipeline.CheckpointFingerprintProvider`. Changing the embedded catalog,
+composition policy, or normalize-stage overlay therefore invalidates reusable
+normalize checkpoints.
+
+### Canonical spell names
+
+For every spell cast, look up the extracted name using the effective catalog's
+existing case, whitespace, apostrophe, and alias rules. Replace a recognized
+value with its canonical display name. This is the only spell-name repair in
+the initial feature.
+
+Do not use edit distance, phonetic matching, model judgment, or another fuzzy
+heuristic. If a value does not resolve, retain it unchanged and emit a scoped
+warning; the configured normalize validator chain remains responsible for
+acceptance or rejection.
+
+Emit a warning for each changed spell name. Diagnostics should identify the
+artifact index and the original and canonical values without modifying other
+fields.
+
+### Source-reference normalization
+
+Sort each cast's source references by source identity, start unit, and end
+unit, then remove exact duplicate references. Do not merge adjacent or merely
+overlapping ranges, because doing so could broaden the evidence attributed to
+an event.
+
+The normalizer must not synthesize source references, alter source-unit
+boundaries, or use auxiliary references as evidence.
+
+### Conservative duplicate collapse
+
+After name and source-reference canonicalization, treat two casts as the same
+event only when all of the following match:
+
+- canonical spell name;
+- caster after case folding and whitespace normalization; and
+- the complete canonical source-reference set.
+
+Collapse each such group into its first occurrence, preserving stable pipeline
+order. Retain the first cast's caster, effect, and narrative description. Do
+not combine prose fields or select a winner based on length, confidence, or
+model-like semantic judgment. The retained cast receives the group's already
+canonical source-reference set.
+
+Emit one scoped warning per collapsed group, including the retained index and
+the removed indices. Casts with different evidence remain distinct even when
+their spell and caster match. In particular, adjacency at a chunk or scene
+boundary is not sufficient evidence of duplication.
+
+### Production composition and validation
+
+Register the typed spell normalizer in the D&D family and make it the default
+normalizer for the maintained production spell pipeline. Keep the artifact
+kind and durable spell-list schema unchanged.
+
+Add a normalize-stage production validator chain using the existing generic
+JSON and JSON Schema validators followed by the existing spell shape, catalog,
+source-reference, and source-relatedness validators in their current order.
+Explicit validator overrides remain authoritative.
+
+The maintained overlay-capable example should bind `spell_catalog` at both the
+extract and normalize stages. The base-only example should continue to work
+without a catalog reference.
+
+## Acceptance Criteria
+
+- Base and overlay aliases are written using canonical catalog display names.
+- Unknown names are never guessed, deleted, or silently changed.
+- Source references are deterministically sorted and exactly deduplicated.
+- Only casts with matching canonical spell, normalized caster, and identical
+  evidence sets collapse; near or adjacent events remain separate.
+- Normalization preserves the first cast's non-name fields and stable order.
+- Every name change and duplicate collapse emits a scoped warning.
+- Extract, normalize, and catalog-validation catalog identities agree for the
+  same bound references, and catalog changes invalidate normalize checkpoints.
+- The maintained production pipeline and current-behavior documentation use
+  the D&D normalizer without changing the durable artifact schema.
+
+## Evaluation
+
+Maintain a small human-reviewed set of representative spell outputs covering
+canonical names, aliases, repeated casts, duplicate model output, and scene
+boundaries. Use it to review normalization behavior and warnings, not as a
+claim that LLM extraction is deterministically correct.
+
+Record cases where likely duplicates remain because their evidence differs.
+Those examples should inform later LLM-assisted deduplication work rather than
+causing this deterministic milestone to adopt broader heuristics.
+
+## Deferred Work
+
+- Fuzzy correction of unknown spell names.
+- Collapsing casts based only on overlapping, adjacent, or semantically similar
+  evidence.
+- LLM-assisted duplicate proposals or prose reconciliation.
+- A generic deduplication contract based on stable artifact-element IDs.
+- Repair-aware extraction retries or LLM-backed semantic validation.
+- Changes to the spell artifact schema, including stable cast IDs.
+
+An LLM-backed normalizer is explicitly outside this milestone. If later
+evaluation justifies one, shared spell-catalog prompt material should be
+designed at that time around the actual normalization or repair request.