diff --git a/docs/roadmap/dnd-npc-extraction.md b/docs/roadmap/dnd-npc-extraction.md new file mode 100644 index 0000000..4837e54 --- /dev/null +++ b/docs/roadmap/dnd-npc-extraction.md @@ -0,0 +1,209 @@ +# D&D NPC Extraction And Registry + +Status: Accepted. + +## Purpose + +Add a production D&D NPC pipeline that turns a session transcript into a +canonical, evidence-backed NPC registry. The registry should be useful on its +own and should serve as reference material for later sequential spell and +combat-turn pipelines. + +This is the next recommended increment because participant identity is already +a demonstrated source of extraction error, while combat-turn extraction will +depend on consistent identities across many more events. Establishing the NPC +artifact first gives later pipelines a durable vocabulary without introducing +a DAG or cross-lane orchestration. + +## Target Outcome + +An operator can: + +1. run an NPC pipeline against a D&D transcript; +2. receive a validated, normalized JSON NPC registry; +3. provide that registry as an explicit reference to a later spell pipeline; + and +4. receive spell artifacts that prefer the registry's canonical NPC identities + when resolving casters, targets, and other named participants. + +The pipelines remain independent CLI invocations over the same input. Notarius +does not automatically schedule them, discover prior output, or reconcile +their results concurrently. + +## NPC Artifact Contract + +The new typed artifact kind represents a list of NPC records. Each record has: + +- a stable, deterministic ID derived by Notarius from the normalized NPC + identity rather than authored freely by the model; +- one canonical in-world name or, for an unnamed but individually + distinguishable NPC, one stable descriptive label; +- zero or more observed aliases; +- a concise transcript-grounded description; +- zero or more explicitly supported relationships to PCs, NPCs, groups, or + locations; and +- one or more source references that collectively support the record's + identity and every reported description, alias, and relationship. + +The external contract defines the exact JSON shape, ID syntax, ordering, +normalization, and compatibility policy. IDs are stable for equivalent +normalized identities under the same policy; they do not promise permanence +when human review or new evidence changes the canonical identity. + +Descriptions report only facts established by the session transcript. General +D&D knowledge, campaign reference material, and model inference do not supply +biography, statistics, alignment, motivations, relationships, or outcomes. +Reference material may disambiguate an identity but is not source evidence. + +## Inclusion Policy + +Include an in-world non-PC participant when the transcript establishes that it +appears, acts, speaks, or is materially discussed and either: + +- gives it a proper name; +- gives it a stable alias or title; or +- distinguishes it individually in a way useful to later extraction, such as + a titled attendant or a distinct item-bearing guard. + +Exclude: + +- human players, transcript speakers, and the GM as out-of-world people; +- player characters identified by the party or player references; +- incidental name drops, hypothetical examples, corrected transcription + mistakes, and characters mentioned only by reference material; +- interchangeable unnamed crowds or groups whose members cannot be + distinguished; and +- temporary summoned creatures or spell effects unless the transcript gives + one an individual, persistent identity relevant beyond the summoning event. + +An NPC may be friendly, hostile, neutral, allied with the party, controlled by +a player temporarily, or absent from the current combat. Classification rests +on whether the entity is an in-world non-PC participant, not on disposition or +who rolls its dice. + +## Extraction Semantics + +The extractor uses the existing shared D&D transcript, player, party, and +glossary prompt inputs. It returns canonical in-world identities rather than +speaker names and cites only transcript units. + +Each record is intentionally concise. It captures identity and stable facts +useful for recognizing the NPC elsewhere; it does not summarize every action, +follow the NPC through the entire session, or duplicate combat-turn and +narrative artifacts. Relationships must be stated or directly demonstrated by +the cited transcript rather than inferred from D&D lore. + +The model may propose canonical names, aliases, descriptions, relationships, +and evidence. Deterministic code owns artifact IDs, structural integrity, +ordering, safe consolidation, and validation of all source ranges. + +## Validation And Normalization + +The production default validator chain remains deterministic. It must reject: + +- malformed or empty required fields; +- missing, invalid, or foreign source references; +- duplicate or colliding IDs; +- an alias that resolves to more than one retained NPC; +- unsupported structural combinations defined by the artifact contract. + +Relatedness checks may emit warnings when a proposed canonical name or alias is +absent from its cited source text. A name legitimately disambiguated through +opaque player, party, or glossary reference material may still produce that +warning; the warning does not turn the reference into event evidence. Exclusion +of PCs is prompt policy and a human-evaluation dimension until a future +structured roster contract makes deterministic membership checks reliable. + +Normalization is deterministic and conservative. It: + +- normalizes whitespace, case-insensitive comparison keys, apostrophe variants, + and exact source-reference order; +- consolidates records whose normalized canonical names match or whose + canonical-name and alias sets establish one unambiguous identity; +- unions unique aliases, relationships, and source references in stable order; +- retains one supported description without synthesizing new prose; and +- emits scoped warnings for every canonicalization or collapsed record. + +Ambiguous identities remain separate or cause a bounded validation rejection; +the normalizer does not guess. Broader semantic reconciliation belongs to the +future generic LLM-assisted deduplication work after real NPC output shows that +deterministic identity overlap is insufficient. + +## Sequential Reference Use + +The normalized NPC registry is accepted as an optional structured JSON +reference by the D&D spell extractor. The spell prompt uses it to select exact +canonical NPC names and aliases, just as party and player references ground PC +identities. The registry does not establish that an NPC cast a spell and never +becomes spell source evidence. + +Reference binding stays explicit in configuration or through the existing CLI +reference override. The maintained examples and operator documentation should +demonstrate separate NPC and spell invocations over the same transcript, with +the first output bound to the second invocation. + +The reference contract should be designed for reuse by the future combat-turn +extractor without adding combat-specific fields to the NPC artifact. + +## Provenance, Checkpoints, And Diagnostics + +The NPC extractor and normalizer report their prompt, private schema, and +normalization-policy identities through the existing manifest and prepared +checkpoint-fingerprint contracts as applicable. Changes to any semantic input +that can change NPC identity or consolidation intentionally produce cold +checkpoint misses. + +Raw transcript, reference, prompt, schema, and NPC content remain outside +manifest metadata and checkpoint fingerprints. Existing debug and rejected- +attempt behavior applies without a domain-specific filesystem surface. + +## Evaluation + +Evaluation is human-reviewed and model-aware rather than a deterministic golden +output gate. The initial corpus should include the existing D&D sessions that +exposed player-versus-character attribution and repeated NPC appearances. + +Review should score separately: + +- NPC detection precision and recall; +- exclusion of PCs and out-of-world people; +- canonical identity and alias quality; +- duplicate consolidation; +- description and relationship fidelity; +- source-reference completeness; and +- usefulness as grounding for a subsequent spell extraction run. + +Small-model development runs may expose more semantic errors than production +frontier models. Deterministic contracts and validators should protect +structure and provenance without attempting to turn subjective extraction +quality into brittle fixture equality. + +## Out Of Scope + +- Combat turns, initiative, actions, damage, conditions, or encounter state. +- General narrative or scene summaries. +- PC extraction or replacement of the existing party and player references. +- Automatic multi-pipeline scheduling, prior-output discovery, or DAG + execution. +- An LLM-backed NPC validator or generic semantic deduplication normalizer. +- Campaign-wide identity persistence, a database, cross-session entity merges, + or manual identity-editing UI. +- Retrieval, embeddings, reference summarization, or token-budget management. + +## Acceptance Criteria + +- A configured D&D NPC lane produces a typed, durable JSON registry with the + identity, evidence, and inclusion semantics above. +- Production shape, source-reference, identity-collision, and relatedness + validators are registered with a deterministic default chain. +- The NPC normalizer assigns stable IDs and safely consolidates only + unambiguous identities while preserving provenance and warnings. +- The registry can be bound directly to the spell extractor as an optional NPC + reference, and spell extraction prefers its canonical NPC names without + treating it as source evidence. +- Prompt, schema, reference, and normalization-policy changes participate in + checkpoint identity where they can affect output. +- External, configuration, internal, operational, and maintained-example + documentation is updated in its canonical location when the feature lands. +- Human review on representative transcripts demonstrates useful NPC grounding + for a subsequent spell run without requiring exact model-output fixtures. diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index f405791..bf9ce9c 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -18,8 +18,10 @@ not as committed release dates. ### Add Sequential D&D Artifacts -- Add NPC extraction, including identity, aliases, descriptions, relationships, - and source evidence suitable for use as later reference material. +- Add the proposed [D&D NPC extraction and registry](dnd-npc-extraction.md) as + the next sequential artifact. It defines canonical identity, aliases, + descriptions, relationships, evidence, deterministic consolidation, and + direct reuse as spell-pipeline reference material. - Add combat-turn extraction with explicit event and source-reference semantics. Use earlier NPC output as a reference to improve participant identity and consistency. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..da9a79a --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,528 @@ +# D&D NPC Extraction And Registry Implementation Plan + +Status: Ready for implementation. + +Implement this plan in order. The feature policy and target state are defined +in [D&D NPC Extraction And Registry](dnd-npc-extraction.md); this document owns +implementation sequencing and concrete technical decisions. + +Do not introduce a DAG, implicit prior-run discovery, an LLM-backed validator, +or generic semantic deduplication. Preserve the fixed +`input -> chunk -> extract -> merge -> normalize -> output` architecture and +the existing framework retry, warning, rejection, checkpoint, debug, and +output contracts. + +## Cross-Stage Decisions + +### Artifact And Module Identities + +Use these exact production identities: + +- artifact kind: `dnd/npc-list`; +- extractor key: `dnd/npcs`; +- normalizer key: `dnd/npcs`; +- extractor capability: `dnd.npcs`; +- prompt ID: `dnd.npcs`, version `v1`; +- private LLM response-schema key: `dnd_npcs_llm`; +- private schema ID: `notarius.dnd.npcs.llm`; +- private schema name: `notarius_dnd_npcs_llm_v1`; +- durable codec schema ID: `notarius.dnd.npcs`; +- durable codec schema name: `notarius_dnd_npcs_v1`; +- durable schema version: `v1`; and +- durable media type: `application/json`. + +The private LLM schema omits framework-assigned `id` and `source_id` fields. +The durable codec schema includes both. + +### Durable Go And JSON Shape + +Add the following canonical D&D types in `internal/modules/dnd/types.go`: + +```go +type NPCList struct { + NPCs []NPC `json:"npcs"` +} + +type NPC struct { + ID string `json:"id"` + Name string `json:"name"` + Aliases []string `json:"aliases"` + Description string `json:"description"` + Relationships []NPCRelationship `json:"relationships"` + SourceRefs []source.SourceRef `json:"source_refs"` +} + +type NPCRelationship struct { + Target string `json:"target"` + Relationship string `json:"relationship"` +} +``` + +All six NPC fields and both relationship fields are required in durable JSON. +`npcs`, `aliases`, and `relationships` must be present arrays; the first may be +empty and the latter two may be empty for an individual record. `source_refs` +must contain at least one item. Required strings must be non-empty after +trimming. Unknown JSON fields are rejected. + +One NPC-level `source_refs` collection collectively supports the canonical +name, aliases, description, and relationships. Do not add per-alias or +per-relationship evidence in this version. + +### Canonical Identity And IDs + +Create `internal/modules/dnd/npcs/identity` as the sole owner of NPC identity +comparison and ID derivation. Define comparison keys by applying, in order: + +1. Unicode NFKC normalization; +2. replacement of U+2018, U+2019, and U+02BC with ASCII apostrophe; +3. Unicode whitespace collapsing with `strings.Fields` and one ASCII space; +4. Unicode case folding with `golang.org/x/text/cases.Fold`. + +Do not alter display punctuation merely to match a comparison key. Display +normalization trims and collapses whitespace but otherwise retains the first +observed spelling. + +For a non-empty comparison key, derive the ID as: + +```text +npc:sha256:<64 lowercase hexadecimal SHA-256 characters> +``` + +The hash input is the UTF-8 comparison key with no prefix, suffix, separator, +or salt. An empty identity produces an empty ID and is rejected by shape +validation. Export a stable identity-policy value +`dnd.npcs.identity.v1`; components that depend on these rules use it as their +`identity_policy` checkpoint fingerprint. A future semantic identity change +must change this policy value. + +### Deterministic Consolidation + +Normalizer identity components are built in merged input order. Union two +records only when: + +- their canonical-name comparison keys match; or +- either record's canonical-name key appears in the other record's alias keys. + +Do not union records merely because their alias sets intersect. This prevents a +shared title from silently merging otherwise distinct NPCs. After +consolidation, an alias owned by more than one retained component, or an alias +matching another retained canonical name, is an identity collision and causes +normalize-stage validation rejection. + +The first record in a component supplies the retained canonical display name, +description, and output position. Add every distinct later canonical display +name to its aliases, followed by later aliases in encounter order. Union +relationships by the pair of target and relationship comparison keys, and +union exact source references. Recompute the retained ID from the retained +canonical name. Never ask an LLM to choose merged prose. + +After components are known, rewrite a relationship target to a retained NPC's +canonical display name when its comparison key matches exactly one retained +canonical name or alias. Preserve targets that have no NPC match; they may name +a PC, group, place, or other non-registry entity. An ambiguous target remains +unchanged and is covered by identity-collision validation when the ambiguity is +an alias collision. + +### Validation And Diagnostics + +All new production validators are deterministic. Use these exact keys and +reason codes: + +| Validator key | Rejection/warning reason | +| --- | --- | +| `extract/dnd/npcs/shape` | `invalid_npc_shape` | +| `extract/dnd/npcs/source_refs` | `invalid_npc_source_refs` | +| `extract/dnd/npcs/source_relatedness` | warning `npc_not_near_source` | +| `normalize/dnd/npcs/identity` | `invalid_npc_identity` | + +Shape validation owns required fields and non-empty values. Source-reference +validation owns document identity, unit existence, and range ordering through +`source.ValidateRef`. Identity validation owns ID syntax and recomputation, +duplicate canonical identities, duplicate IDs, duplicate aliases, aliases +equal to their own canonical name, cross-record alias/canonical collisions, and +cross-record alias ownership. + +Relatedness approves the artifact and emits at most one warning per NPC when +neither its canonical name nor any alias appears in the combined cited source +text after the same Unicode comparison normalization. Opaque campaign +references may legitimately explain such a warning; do not treat them as +source evidence or attempt to parse them deterministically. + +Bound diagnostics using the established spell-validator policy: display at +most 20 issues, truncate displayed identity values to 128 Unicode code points +with an ellipsis, keep encoded rejection messages at or below 4,096 bytes, and +report the total omitted issue count. Preserve valid UTF-8 and use Go quoting +for control characters. Warnings must likewise use bounded displayed values. + +Each new validator implements `CheckpointFingerprintProvider` with one local +`policy` fingerprint. Use these exact values: + +- shape: `dnd.npcs.validator.shape.v1`; +- source references: `dnd.npcs.validator.source_refs.v1`; +- source relatedness: `dnd.npcs.validator.source_relatedness.v1`; and +- identity: `dnd.npcs.identity.v1`. + +Bump only the affected value when validator semantics change. + +### Testing Rules + +Follow `docs/policy/testing.md`. Tests must be offline and deterministic. Use a +fake structured LLM only at the external completion boundary. Protect durable +artifact contracts, identity and normalization invariants, validation outcomes, +registration, checkpoint identity, and representative assembled workflows. + +Do not add tests that require specific words or phrases to remain in prompt +prose. Prompt-asset tests may verify that assets register, required inputs are +wired, the selected prompt and schema identities are correct, and raw material +does not leak into diagnostics. Human review, not exact model-output fixtures, +owns semantic extraction quality. + +## Stage 1: NPC Domain Contract, Identity Policy, And Codec + +### Goal + +Establish the typed artifact, deterministic identity primitives, and durable +serialization contract without registering a selectable production module. + +### Changes + +- Add `NPCListKind`, `NPCList`, `NPC`, and `NPCRelationship` to the canonical + D&D model using the exact shape above. +- Add `internal/modules/dnd/npcs/identity` with defensive, concurrency-safe pure + functions for display normalization, comparison keys, ID derivation, ID + syntax checks, and whole-registry identity validation. +- Identity validation must return structured or otherwise inspectable issues so + the normalize validator can build bounded aggregate diagnostics. Do not make + it depend on pipeline validator types. +- Add `internal/modules/dnd/codec/npcs`, following the spell codec's candidate + versus approved encode/decode boundary: + - strict single-value JSON decoding with unknown-field rejection; + - `EncodeCandidate` and `DecodeCandidate` preserve validator-visible invalid + values; + - approved `Encode` and `Decode` enforce structural validity; + - schema and metadata are defensive copies; and + - metadata reports `npc_count` only. +- Add the durable JSON Schema at + `internal/modules/dnd/codec/npcs/assets/schemas/dnd_npcs.v1.json` with the + exact required fields, ID pattern, array rules, source-reference shape, and + `additionalProperties: false` at every object level. +- Create `docs/integrations/dnd-npc-artifacts.md` as the canonical durable + contract. At this stage describe the implemented artifact and codec only; do + not yet claim that production pipelines can select it. + +### Tests + +- Test identity equivalence across case, Unicode compatibility forms, + whitespace, and supported apostrophes; different names must remain distinct. +- Test the exact ID format, deterministic repeatability, empty-name behavior, + and registry identity issues for every collision category. +- Test codec candidate preservation, approved round trips, schema identity and + defensive copies, nil-versus-empty arrays, malformed/trailing/unknown JSON, + required strings, relationship shape, ID pattern, and source-reference + structure. +- Keep one small intentional durable JSON fixture for round-trip compatibility; + do not snapshot incidental error text. + +### Completion Check + +Run `go test ./internal/modules/dnd/npcs/... ./internal/modules/dnd/codec/npcs` and +`git diff --check`. + +## Stage 2: NPC Extractor And Extraction Validators + +### Goal + +Add a package-complete LLM-backed NPC extractor and deterministic extraction +validation, still without composing it into the production D&D registrar. + +### Changes + +- Add `internal/modules/dnd/extract/npcs`, mirroring the established spell + extractor boundaries without sharing spell-specific code. +- The module spec requires `chunks` and `source.transcript`, provides + `dnd.npcs`, uses artifact kind `dnd/npc-list`, accepts no options, and declares + the existing optional `players`, `party`, `glossary`, and deprecated `roster` + campaign slots with the existing shared media types. +- Embed and register package-owned prompt assets and a private LLM JSON Schema. + Reuse the shared D&D system, transcript, and campaign-reference prompt + fragments through `shared.ModulePromptFS`. +- The prompt implements the feature-roadmap inclusion and exclusion policy, + requires canonical in-world names rather than speakers, keeps descriptions + concise, requires every claim to be supported by transcript citations, and + treats auxiliary references only as disambiguation. +- The private response has top-level `npcs`; each item contains `name`, + `aliases`, `description`, `relationships`, and source ranges without + `source_id`. Require `aliases` and `relationships` arrays even when empty. +- Preserve malformed model values for validators. Canonicalize and de-duplicate + exact source ranges, order response records by earliest valid cited unit, map + every range to the current source ID, and assign IDs with the identity + package. Do not merge NPC records in the extractor. +- Validate constructor dependencies and embedded prompt/schema metadata during + preparation. Expose manifest-safe prompt and private-schema identities plus + `identity_policy`; expose `prompt`, `response_schema`, and `identity_policy` + checkpoint fingerprints. +- Add typed validators under `internal/modules/dnd/validate/npcs/shape`, + `source_refs`, and `source_relatedness` using the contracts above. Validators + defer appropriately after shape failure rather than emitting misleading + secondary diagnostics. +- Give each validator a strict empty-options decoder, immutable spec, correct + execution class, typed builder, policy fingerprint, and nil/dependency error + behavior consistent with existing production validators. + +### Tests + +- Through a fake LLM, test request identity, profile/session propagation, + chunk-scoped transcript bytes, campaign reference inputs, response mapping, + deterministic IDs, source-ID assignment, evidence ordering, and preservation + of invalid candidates for retries. +- Test cancellation and contextual provider errors without asserting entire + messages. +- Test prompt and schema asset registration, input wiring, metadata redaction, + and checkpoint fingerprints without asserting prompt prose. +- Test each validator's approval, rejection, warning, malformed-shape deferral, + strict options, typed registration, diagnostic limits, Unicode truncation, + and immutability. +- Test source relatedness using canonical names, aliases, case/whitespace/ + apostrophe variants, invalid ranges, and names resolved only through opaque + references. + +### Completion Check + +Run `go test ./internal/modules/dnd/extract/npcs ./internal/modules/dnd/validate/npcs/...` +and `git diff --check`. + +## Stage 3: Deterministic NPC Normalization + +### Goal + +Implement conservative cross-chunk identity consolidation and final identity +validation without changing framework normalization contracts. + +### Changes + +- Add `internal/modules/dnd/normalize/npcs` with key `dnd/npcs`, artifact kind + `dnd/npc-list`, no options, no references, requirements `merged`, and + capability `normalized`. +- Deep-clone all nested slices before mutation. A result must never alias the + merged input or another retained record. +- Per record, normalize display whitespace, trim description and relationship + strings, de-duplicate aliases and relationships by comparison keys, remove + aliases equal to the canonical name, sort and exactly de-duplicate source + references, and recompute the ID. +- Build and merge identity components with the exact cross-stage algorithm. + Preserve component and member encounter order. Add removed canonical names as + aliases, union provenance, retain only the first description, and + canonicalize unambiguous relationship targets after all components exist. +- Use these stable warning reason codes: + - `npc_fields_normalized`; + - `npc_id_recomputed`; + - `source_references_normalized`; + - `duplicate_npc_collapsed`; and + - `relationship_target_canonicalized`. +- Warning scopes use merged input indexes such as `npcs[3]`. A collapsed-group + warning is scoped to the retained input index and reports bounded removed + indexes. Accepted-only warning promotion remains framework policy. +- Expose manifest metadata `identity_policy` and + `normalization_policy: dnd.npcs.normalize.v1`. Provide those same two local + checkpoint fingerprints. Bump the normalization value whenever consolidation + or retained-field semantics change. +- Add `internal/modules/dnd/validate/npcs/identity`. It delegates identity issue + detection to the domain identity package, builds bounded diagnostics, and is + registered only in the normalize default chain during Stage 4. + +### Tests + +- Use table-driven domain cases for per-record normalization and alias, + relationship, source-reference, and ID behavior. +- Use a compact accepted-output fixture for representative multi-record + consolidation: exact canonical match, canonical-to-alias match, transitive + unambiguous match, shared-alias non-merge, first-description retention, + relationship target rewriting, and stable ordering. +- Prove ambiguous shared aliases remain separate for identity-validator + rejection and that invalid evidence is not made valid by normalization. +- Prove input immutability and independent nested output storage. +- Test warning scopes/reason codes, normalization and identity fingerprints, + strict options, registration, cancellation, nil input, and bounded identity + rejection diagnostics. +- Do not separately test private union-find or graph helpers when the public + normalizer cases already protect the behavior. + +### Completion Check + +Run `go test ./internal/modules/dnd/normalize/npcs ./internal/modules/dnd/validate/npcs/identity` +and `git diff --check`. + +## Stage 4: Production Composition And NPC Pipeline + +### Goal + +Make the complete NPC lane selectable and verify the assembled framework path +before adding it as a reference consumer elsewhere. + +### Changes + +- Extend `internal/modules/dnd/register` to register: + - the NPC codec; + - NPC extractor and prompt assets; + - typed `appendorder` merger specialization for `dnd.NPCList`; + - NPC normalizer and typed `noop` specialization; + - all four NPC validators; + - typed always-accept and always-reject validator variants; and + - default validator-chain mappings. +- Add a small `appendNPCLists` function that preserves chunk and record order + and distinguishes nil from present-empty output consistently with the spell + merger. +- Register the extract default chain in this order: + `generic/valid_json`, `generic/valid_json_schema`, NPC shape, NPC source refs, + NPC source relatedness. +- Register the normalize default chain in this order: + `generic/valid_json`, `generic/valid_json_schema`, NPC shape, NPC identity, + NPC source refs, NPC source relatedness. +- Do not register an NPC merge default chain and do not change global framework + defaults. +- Add a synthetic, non-sensitive Seriatim integration fixture with a PC, a + repeated named NPC, an alias, an unnamed but distinguishable NPC, and an + interchangeable group that the fake response omits. +- Add `examples/dnd-npcs.config.yml` as a complete version-3 configuration. Use + a generic chunker, an `npcs` artifact lane, `dnd/npcs` extraction with + `retries: 2`, `dnd/npcs` normalization, explicit checkpoint `enabled: false`, + and safe relative output/debug paths. + +### Tests + +- Extend family registration tests for keys, artifact variants, default chain + order, prompt/schema assets, nil registry failures, and duplicate + registration errors. Test the public catalog outcome, not private registrar + call order. +- Add config-resolution coverage for the assembled NPC lane, capabilities, + typed codec/merger/normalizer compatibility, strict options, references, and + invalid validator placement. +- Add one runner integration test using a fake LLM that exercises + Seriatim -> chunks -> NPC extract -> validators -> append merge -> NPC + normalize -> validators -> JSON output. Assert normalized identities, + provenance, warnings, manifest metadata, and durable lane schema. +- Add one representative retry/rejection case at the assembled boundary only + if existing generic runner tests do not already protect the same mechanism; + do not duplicate the framework retry matrix. +- Verify the maintained example through the existing config/example contract + tests. + +### Completion Check + +Run `go test ./internal/modules/dnd/register ./internal/modules/integration ./internal/core/config` +and `git diff --check`. + +## Stage 5: NPC Registry Reference For Spell Extraction + +### Goal + +Complete the operator-driven sequential workflow by making normalized NPC JSON +an explicit, validated spell-extractor reference. + +### Changes + +- Add an optional `npcs` reference slot to the spell extractor only. Accept + exactly one `application/json` item with a maximum size of 1,048,576 bytes. + Do not add it to the shared scene or NPC-extractor campaign slots. +- During spell-extractor preparation, when `npcs` is bound: + - require exactly one item; + - decode it with the approved NPC codec; + - validate its IDs and alias ownership through the NPC identity package; + - re-encode canonical durable JSON for the prompt; and + - compute a semantic SHA-256 digest over those canonical bytes. +- Permit source references inside the registry to identify another session; + they are registry provenance, not spell evidence. Do not validate them + against the current transcript. +- When the slot is unbound, supply the prompt with exactly `{"npcs":[]}` as the + empty structured value. Do not create reference manifest provenance or an + `npc_registry` fingerprint for the absent slot. +- Declare an optional `application/json` `npcs` prompt input and render it in a + spell-owned prompt message. Do not modify the shared D&D reference fragment, + because the scene and NPC prompts do not consume this registry. +- Instruct the spell model to prefer exact canonical NPC names, recognize + aliases, and never treat registry content as proof that a spell was cast or + as source evidence. Preserve the existing party/player policy for PCs. +- When bound, add manifest metadata `npc_registry_digest` and `npc_count`, and + add local checkpoint fingerprint `npc_registry` with the semantic digest. + Never expose names, aliases, reference content, paths, or raw bytes in + metadata or fingerprints. Existing raw reference provenance continues to + participate independently in checkpoint identity. +- Add `examples/dnd-npc-spell-sequential.config.yml` with separate `dnd-npcs` + and `dnd-spells` pipeline definitions over the same Seriatim input shape. The + NPC output remains intentionally unbound in the static spell definition + because its run directory is dynamic; demonstrate the explicit runtime + binding in documentation. +- Update current-behavior documentation in canonical locations: + - `docs/config.md`: module and validator catalogs, NPC reference slot, limits, + default chains, and maintained examples; + - `docs/cli.md`: one explicit selector example using + `spells.extract.npcs=/lanes/npcs.json`; + - `docs/operations.md`: the two-invocation sequential workflow and state + sensitivity; + - `docs/integrations/dnd-npc-artifacts.md`: production identity, final + normalization, warnings, manifest metadata, and reference-consumer + semantics; + - `docs/integrations/dnd-spell-artifacts.md`: optional NPC grounding and its + non-evidence rule; + - `docs/integrations/json-output.md`: link the NPC lane payload contract; + - `docs/internal/overview.md`, `docs/internal/modules.md`, and + `docs/internal/llm.md`: implemented package inventory and internal prompt, + preparation, fingerprint, and registration behavior. +- At feature completion, mark the feature roadmap and this implementation plan + complete. Remove the now-implemented NPC item from `future.md`; keep combat + turns as the next future sequential artifact. Do not leave future behavior in + current-behavior documentation. + +### Tests + +- Test spell preparation with an absent registry, a valid normalized registry, + malformed JSON, unknown fields, invalid IDs, alias collisions, multiple + items, wrong media type, and the configured byte limit. Materialization-limit + failures must occur before pipeline execution and checkpoint construction. +- Through the fake spell client, assert canonical NPC JSON input, input + metadata, manifest-safe count/digest, and absence of content leakage. Do not + assert prompt wording. +- Test that identical semantic registries produce the same component + fingerprint, semantic changes change it, and returned fingerprints cannot be + mutated through caller-owned slices. Do not repeat generic checkpoint ordering + tests already owned by the framework. +- Add one sequential integration case: run the assembled NPC pipeline with a + fake NPC response, serialize its normalized lane payload, materialize that + payload as the spell extractor's `npcs` reference, and run the spell pipeline + with a fake spell response. Verify reference provenance, prompt input, typed + output, and that no NPC source reference becomes spell evidence. +- Validate all new documentation links and maintained example configurations. + +### Completion Check + +Run the final verification suite below. + +## Final Verification + +Run: + +```sh +git diff --check +go test ./... +go vet ./... +go build ./cmd/notarius +go test -race ./internal/modules/dnd/... ./internal/framework/pipeline \ + ./internal/cli ./internal/modules/integration +``` + +Review the final diff for: + +- accidental framework or provider-specific D&D behavior; +- mutation of caller-owned artifacts, schemas, references, metadata, or + fingerprints; +- prompt/schema/content leakage into manifests, errors, checkpoint identity, + or redacted summaries; +- current documentation claiming behavior before its implementing stage exists; +- redundant tests or prompt-prose change detectors; and +- unrelated changes in the worktree. + +## Open Questions + +None. The artifact shape, identity algorithm, merge policy, validator placement, +reference parsing, checkpoint semantics, and documentation ownership are fixed +by this plan.