notarius/docs/roadmap/implementation.md

D&D NPC Interactions Implementation Plan

Status: Ready for implementation

Objective

Implement the accepted D&D NPC Interactions roadmap as a production D&D artifact lane. The finished lane must extract an ordered list of minimal, evidence-grounded NPC interaction occurrences from each accepted transcript chunk, ground every NPC against a required accepted NPC registry, and preserve the existing fixed pipeline and generated-reference architecture.

This plan is the implementation authority for sequencing and file-level work. The feature roadmap remains authoritative for product intent, category semantics, occurrence boundaries, evidence policy, and non-goals. Follow ADR-0009 throughout: do not add descriptive, analytical, relationship, or state fields.

Fixed Decisions

The implementation must use these identities:

Concern	Identity
Extractor key	dnd/npc-interactions
Normalizer key	dnd/npc-interactions
Artifact kind	dnd/npc-interaction-list
Durable schema ID	notarius.dnd.npc_interactions
Durable schema name	notarius_dnd_npc_interactions_v1
Durable schema version	v1
Durable media type	application/json
Private prompt ID	dnd.npc_interactions
Private response schema ID	notarius.dnd.npc_interactions.llm
Reference slot	npcs

Use a top-level interactions array. Each durable record has exactly name, kind, and source_refs. Do not add an interaction ID. The closed kind vocabulary is mentioned, noncombat_presence, dialogue, combat_ally, combat_opponent, and other.

The npcs slot is required for both extraction and normalization. It accepts only the existing dnd/npc-list JSON artifact within the existing NPC registry size limit. A bound registry with an empty npcs array is valid and requires an empty interaction artifact. An absent binding, malformed registry, failed generated handoff, or unknown interaction name is not permission to perform ungrounded extraction.

Use the existing internal/modules/dnd/npcs/registry.Resolver for construction- time external references and operation-time generated references. The extractor presents only Registry.PromptInput() to the model. Name lookup uses the registry's established comparison-key policy before normalization; the normalizer replaces a recognized variant with the exact canonical registry display name. Registry source references and campaign references never become interaction evidence.

An occurrence is one NPC, one kind, and one coherent passage within one accepted chunk. The model must split a category or combat-alignment transition. No occurrence spans chunks. Merge preserves chunk order. Normalization:

1. replaces recognized names with canonical registry display names;
2. sorts and exact-deduplicates each record's source references;
3. orders records by earliest valid source position, then canonical name, interaction kind, and the canonical source-reference sequence;
4. removes records only when canonical name, kind, and the complete canonical source-reference sequence are all identical; and
5. never semantically combines nearby, overlapping, or cross-chunk records.

The private model response remains structural and contains name, kind, and integer start/end unit candidates. Mapping attaches the current source ID. Invalid strings, kinds, and ranges must survive mapping far enough for the deterministic validator that owns the rule to reject them.

For the final record comparator, use source-document unit position for the earliest valid reference. Compare canonical names first by the existing NPC identity comparison key and then by exact display string; compare kinds by their string values; and compare canonical reference sequences lexicographically by source_id, start-unit document position, and end-unit document position. Records with valid evidence precede records without it; normalized output validation rejects the latter before durable encoding.

No framework change, new identity subsystem, DAG behavior, LLM-assisted deduplication, or scene dependency is part of this plan.

Testing And Documentation Rules

Apply Testing Policy at every stage:

• Protect artifact round trips, required fields, enum values, source provenance, registry grounding, deterministic ordering, exact deduplication, reference immutability, and generated handoff behavior.
• Use the real codec, registry, resolver, normalizer, and pipeline machinery when they are fast and deterministic. Stub only the structured LLM boundary.
• Keep provider calls offline and deterministic.
• Test each policy at its narrowest stable owner. Do not repeat every codec, validator, and normalizer case in an end-to-end test.
• Do not add a prompt-cache “change detector” based on exact message count, shared-prefix length, token count, or prompt hash. Reuse the shared prompt assets, test rendered behavior and manifest composition, and document the ordering policy instead.
• Do not use broad golden snapshots for prompts or diagnostics. A maintained durable JSON fixture is appropriate because the complete serialized artifact is an intentional integration contract.

Until the final stage, keep unimplemented behavior only in roadmap documentation. Update current-behavior documentation and examples in the same stage that makes the production lane selectable.

Stage 1 — Add The Typed Artifact And Durable Codec

Goal

Establish the domain type and strict serialized boundary without changing the production module catalog.

Changes

1. Extend internal/modules/dnd/types.go with:
   • NPCInteractionListKind;
   • a string-backed NPCInteractionKind;
   • exported constants for all six accepted values;
   • NPCInteractionList with Interactions []NPCInteraction; and
   • NPCInteraction with Name string, Kind NPCInteractionKind, and SourceRefs []source.SourceRef.
2. Add internal/modules/dnd/codec/npcinteractions following the existing NPC and combat-turn codec boundary:
   • expose the fixed schema constants and MediaType;
   • implement the typed artifact codec for dnd.NPCInteractionList;
   • keep EncodeCandidate and DecodeCandidate strict about one JSON value and unknown fields while preserving typed semantic candidates;
   • make approved Encode and Decode enforce non-empty names, the closed kind enum, at least one structurally valid source reference, and the durable schema shape; and
   • report only interaction_count in codec metadata.
3. Add internal/modules/dnd/codec/npcinteractions/assets/schemas/dnd_npc_interactions.v1.json. It must require the top-level array and all three record fields, reject unknown fields at every object level, encode the six-value enum, require at least one source reference, and use the established durable source-reference shape.
4. Add a small maintained fixture under internal/modules/dnd/codec/npcinteractions/testdata/ containing at least two ordered records with distinct kinds and source ranges.

Tests

At the codec package boundary, cover:

• fixture decode/encode round trip and canonical compact JSON;
• exact kind, schema identity, version, media type, and Go type registration;
• empty-list support and nil-versus-present-empty behavior where the existing codecs distinguish it;
• strict rejection of malformed JSON, trailing values, and unknown fields;
• approved-boundary rejection of each meaningful required-field, enum, and source-reference violation;
• candidate-boundary preservation of semantic values for later validators; and
• defensive copies for schema bytes and metadata.

Do not duplicate source-document existence checks in the codec; those belong to the source-reference validator.

Completion Criteria

• The new types and codec compile and pass focused tests.
• The codec can be registered into an isolated artifact codec registry with exact type dnd.NPCInteractionList.
• No production registrar or configuration catalog exposes the new kind yet.

Stage 2 — Implement The Chunk-Scoped Extractor And Prompt

Goal

Add an independently testable LLM-backed extractor that requires NPC grounding, maps only minimal private output, and follows the established D&D prompt-cache layout.

Changes

1. Add internal/modules/dnd/extract/npcinteractions using the current D&D extractor organization:
   • assets.go;
   • canonicalize.go;
   • extractor.go;
   • model.go;
   • schema.go;
   • scriptorium_assets.go;
   • corresponding focused tests; and
   • package-local embedded prompt and schema assets.
2. Give the extractor strict empty options, required capabilities chunks and source.transcript, provided capability dnd.npc_interactions, and artifact kind dnd.NPCInteractionListKind.
3. Build its reference slots from shared.ReferenceSlots(...), then append the npcs slot with:
   • Required: true;
   • accepted media type application/json;
   • accepted artifact kind dnd.NPCListKind;
   • the existing registry maximum byte count; and
   • a description stating that the accepted registry is required identity grounding, not evidence. Return defensive, consistently sorted slot declarations from both ModuleSpec() and the constructed extractor.
4. Construct an npcregistry.Resolver from the preparation-time reference set. Resolve operation references inside Extract, require Registry.Bound(), and fail before the LLM call if the required registry is absent or invalid. Preserve the resolver's content-safe error behavior.
5. Use shared.ChunkPromptMaterial and shared.PromptInputs. Replace the ordinary npcs input with the resolved names-only Registry.PromptInput(). Do not place full registry bytes, NPC IDs, registry evidence, or provenance in prompt inputs.
6. Add these assets:
   • assets/prompts/dnd.npc_interactions.yaml;
   • assets/prompts/task.md;
   • assets/prompts/instructions.md; and
   • assets/schemas/dnd_npc_interactions_llm.v1.json.
7. Use the existing shared prompt assets in this exact semantic order: common system; common extraction evidence; common identity; common campaign references; common NPC grounding; lane task; lane instructions; variable transcript. Preserve the established cache-control placement used by the spell and combat-turn manifests.
8. Revise the shared common-dnd-npcs.md wording once so it truthfully applies to all three consumers: a normalized registry is always presented to the prompt and may be empty. Keep its identity, context-only rule, and prohibition on treating registry provenance as event evidence. Do not fork a nearly identical interaction-specific grounding asset.
9. The lane prompt must state the category definitions, precedence, occurrence splitting rules, registry-only name restriction, empty-output behavior, and exclusions from the feature roadmap. Keep the response schema structural: it requires the envelope and fields but leaves enum membership, non-empty values, and source semantics to deterministic validators.
10. Map private records to dnd.NPCInteractionList, attach the current source ID, sort and exact-deduplicate ranges within each candidate, and stable-sort candidates with valid evidence by earliest source-document position. Preserve invalid candidate fields rather than repairing or dropping them.
11. Expose prompt, response-schema, and mapping-policy identities through manifest metadata and checkpoint fingerprints, following the current D&D extractors. Include the NPC names-only projection digest in local checkpoint identity. For a preparation-time external registry, expose only bounded digest/count metadata; do not place names or source content in metadata. Generated-reference provenance remains framework-owned.

Tests

At stable package boundaries, cover:

• nil/cancelled/invalid extraction requests and provider failures;
• required-slot declaration and rejection of an unbound or malformed registry before any LLM call;
• operation-time generated registry resolution by an extractor prepared without generated bytes;
• exact names-only registry prompt input and separation from transcript evidence;
• empty registry plus empty response;
• mapping of every category, current-source attachment, evidence canonicalization, and source-position ordering;
• preservation of invalid names, kinds, and ranges for validators;
• strict private JSON shape and unknown-field rejection;
• prompt rendering with required inputs, category policy, shared assets, and transcript-last ordering;
• prompt/schema registration and content-safe metadata/fingerprints; and
• defensive module specifications and strict rejection of unknown options.

Run the existing spell and combat-turn prompt/asset tests after changing the shared NPC prompt fragment. Do not assert an exact shared-prefix length.

Completion Criteria

• The extractor is constructible and testable through the typed extractor contract.
• A valid request makes one scheduled structured-completion call and returns a typed candidate with current-source provenance.
• Missing required NPC grounding cannot reach the model.
• The extractor remains unregistered in the production D&D family until later stages provide the rest of the lane.

Stage 3 — Add Deterministic Artifact Validators

Goal

Give each semantic invariant one clear validation owner and make invalid model output a rejection rather than a normalization repair or framework error.

Changes

Add these typed validator packages, each with strict empty options, bounded diagnostics, typed registration, and the fixed artifact kind:

1. internal/modules/dnd/validate/npcinteractions/shape (extract/dnd/npc-interactions/shape):
   • require a non-empty trimmed name without mutating it;
   • require one of the six kinds; and
   • require at least one source-reference candidate with structurally required values.
2. internal/modules/dnd/validate/npcinteractions/registry (extract/dnd/npc-interactions/registry):
   • use npcregistry.Resolver at construction and operation time;
   • require a bound registry;
   • approve a name when Registry.Lookup recognizes it under the established comparison policy;
   • reject unknown names in stable record order; and
   • never use registry evidence as transcript evidence.
3. internal/modules/dnd/validate/npcinteractions/source_refs (extract/dnd/npc-interactions/source_refs):
   • require the current source identity;
   • require referenced units to exist; and
   • require valid inclusive start/end order through the current document. Follow the established D&D citation helpers and rejection aggregation limits rather than introducing a generic framework dependency on the interaction type.
4. internal/modules/dnd/validate/npcinteractions/source_relatedness (extract/dnd/npc-interactions/source_relatedness):
   • inspect only current transcript text covered by the cited ranges;
   • emit at most one bounded warning per record when the NPC name cannot be related to that evidence;
   • do not claim to validate category agreement; and
   • ignore campaign and registry content for relatedness.
5. internal/modules/dnd/validate/npcinteractions/invariants (normalize/dnd/npc-interactions/invariants):
   • require the exact canonical display name returned by registry lookup;
   • require canonical source-reference order with no duplicate ranges;
   • require the complete list order defined in Fixed Decisions; and
   • reject exact duplicate normalized records.

The registry and invariants validators must support operation-time generated references. Construction-time metadata and fingerprints follow the resolver pattern used by registry-aware modules and must not include NPC names or content.

Tests

Give each policy one primary test owner:

• shape tests own empty names, enum membership, and missing evidence;
• registry tests own required binding, comparison-key recognition, unknown names, operation-time generated overrides, empty registries, and content-safe failures;
• source-reference tests own wrong source IDs, missing units, reversed ranges, and valid multi-range records;
• relatedness tests own current-transcript-only warnings and bounded diagnostics; and
• invariant tests own exact canonical names, final ordering, canonical evidence sequences, and duplicate rejection.

Also verify that validators do not mutate artifacts, source documents, or registries and that their specs register only for dnd.NPCInteractionListKind.

Completion Criteria

• Valid typed candidates are approved and malformed or ungrounded candidates are rejected at the intended boundary.
• No validator silently canonicalizes, drops, or merges records.
• References can explain identity but cannot satisfy source evidence checks.

Stage 4 — Add Deterministic Merge And Normalization

Goal

Preserve chunk-scoped occurrences through merge, then produce the canonical ordered artifact without semantic inference.

Changes

1. Add internal/modules/dnd/normalize/npcinteractions with strict empty options and the fixed module key and artifact kind.
2. Declare the same required npcs reference slot as the extractor. Construct and resolve npcregistry.Resolver at the same preparation/operation boundaries, and require a bound registry.
3. Implement the normalization algorithm in Fixed Decisions:
   • clone all nested values;
   • normalize display whitespace only as needed for lookup;
   • replace every recognized name with the registry's exact canonical display name;
   • canonicalize each reference list without merging overlapping ranges;
   • compute order from source-document positions rather than assuming numeric unit IDs are contiguous;
   • apply every specified tie-breaker; and
   • collapse only exact records after canonicalization.
4. Preserve nil versus present-empty list behavior consistently with the other D&D normalizers.
5. Emit bounded warnings for visible canonical-name changes, reference canonicalization, record reordering, and exact duplicate removal. Use stable reason codes and the shared D&D diagnostics helpers. Never include reference content in a warning.
6. Add an append-only merger function for dnd.NPCInteractionList beside the other D&D typed append functions in internal/modules/dnd/register/merge.go. It must preserve accepted chunk/lane order, preserve present-empty semantics, and deep-clone source reference slices so outputs do not alias inputs. Do not register it until Stage 5.
7. Add normalization-policy and NPC projection identities to checkpoint fingerprints. Manifest metadata may include the policy identity and bounded external-registry digest/count, but not names or source content.

Tests

Cover:

• canonical registry name replacement, including Unicode/case/spacing lookup;
• required and operation-time generated registry behavior;
• source-reference ordering and exact range deduplication;
• ordering by real document position with all deterministic tie-breakers;
• separation of category transitions, alignment transitions, nearby records, overlapping-but-nonidentical evidence, and nonidentical records from different chunks;
• collapse of exact duplicates only;
• stable, bounded warnings;
• nil/present-empty behavior;
• input ownership and nested-slice cloning; and
• append-order merger behavior independently of normalization.

Use table-driven pure normalization cases for the dense ordering and duplicate rules, and package-level normalizer tests for resolver and warning behavior.

Completion Criteria

• Normalization is deterministic and idempotent.
• Running normalization twice does not change the value or emit new transformation warnings on the second pass.
• Semantically distinct occurrences remain distinct.
• The module and merger compile but are not yet selectable through production composition.

Stage 5 — Compose The Complete Production Lane

Goal

Register the complete typed lane atomically so configuration cannot select a partial implementation.

Changes

Update the D&D registrar:

1. In internal/modules/dnd/register/modules.go, register:
   • the NPC-interaction codec;
   • extractor and prompt assets;
   • the append-order merger variant for dnd.NPCInteractionListKind;
   • the dnd/npc-interactions normalizer; and
   • the generic no-op normalizer variant for the new exact Go type.
2. In internal/modules/dnd/register/validators.go, register all five domain validators plus generic always-accept and always-reject typed variants for the new artifact type.
3. In internal/modules/dnd/register/chains.go, register:
   • extraction chain: generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness;
   • normalization chain: generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, normalize/dnd/npc-interactions/invariants, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness.
4. Do not add a default merge validator chain. The append merger performs no semantic mutation, and extraction plus normalized boundaries own the consequential policies.
5. Extend registrar contract tests to verify exact typed coverage, keys, artifact kinds, prompt/schema assets, reference-slot agreement, and default validator order.

Tests

• Extend the existing D&D family registration test rather than creating parallel catalog snapshots.
• Verify that the extractor and normalizer both expose required compatible npcs slots and that generated dnd/npc-list bindings resolve only from an earlier step.
• Verify production resolution rejects a missing required binding, a later or same-step producer, a wrong artifact kind, and incompatible media/schema metadata through existing resolver behavior.
• Avoid retesting the framework's general ordered-step failure matrix; add only interaction-specific composition cases not already covered generically.

Completion Criteria

• Production registration exposes one complete type-consistent lane.
• A valid two-step NPC-to-interactions pipeline resolves and prepares.
• Invalid or missing NPC dependencies fail before source parsing or LLM work whenever statically discoverable.

Stage 6 — Prove The Workflow And Publish Current Contracts

Goal

Exercise the assembled generated-reference workflow and move all implemented behavior into its canonical current documentation.

Changes

1. Add one representative integration test under internal/modules/integration with focused testdata:
   • step 1 extracts and normalizes NPCs;
   • step 2 consumes the generated NPC artifact in an interaction lane;
   • the fake structured LLM returns multiple interaction kinds and at least one name variant;
   • the assertion proves the names-only handoff, current-transcript evidence, canonical name, stable chronology, and final durable artifact.
2. Include one failure assertion showing that a rejected or absent normalized NPC producer prevents interaction extraction. Reuse the existing runner dependency behavior; do not duplicate every framework failure case.
3. Add examples/dnd-npc-interactions.config.yml as a complete copyable two-step profile. The first step produces npcs; the second step binds that exact step/lane artifact to its npcs reference and selects dnd/npc-interactions for extract and normalize.
4. Add the example to the existing CLI production configuration contract test so it is parsed and resolved offline.
5. Add the durable external contract at docs/integrations/dnd-npc-interaction-artifacts.md. It owns schema identity, JSON shape, categories, evidence semantics, ordering, normalization, validator chains, generated-reference behavior, and manifest metadata.
6. Update current canonical documentation:
   • docs/config.md for module/validator catalogs, default chains, required reference slot, and the maintained example;
   • docs/internal/modules.md for implementation ownership, prompt inputs, registry resolution, validation, merge, and normalization;
   • docs/internal/overview.md for the implemented component inventory;
   • docs/internal/llm.md only as needed to include the new lane in the existing D&D prompt-order/cache policy without duplicating the manifest;
   • docs/integrations/dnd-npc-artifacts.md to identify NPC interactions as a consumer while retaining the rule that registry evidence is not event evidence; and
   • any directly affected configuration or integration links.
7. Remove the NPC-interactions entry from docs/roadmap/future.md once the production behavior and current documentation are complete. Change docs/roadmap/dnd-npc-interactions.md to Status: Implemented pending its later retirement; do not leave it as the canonical current contract.
8. Create a small human-review worksheet or fixture set only if the repository already has an appropriate non-test evaluation home. Otherwise record the manual evaluation results in the implementation handoff rather than inventing a new framework. Exercise all six categories, transitions, repeated occurrences, mentions followed by presence, combat alignment changes, cross-chunk repetition, and empty output on at least one intended smaller model. Do not make model agreement a deterministic CI gate.

Validation

Run focused tests while implementing, then run:

go test ./...
go vet ./...
go build ./cmd/notarius

Also run git diff --check, inspect the complete documentation diff for current-versus-future claims, and verify every new relative documentation link.

Completion Criteria

• All repository checks pass offline.
• The maintained example resolves with an explicit earlier NPC producer.
• The integration test proves the accepted generated artifact, not registry source references, grounds the later model request.
• Durable output contains only name, kind, and current-transcript source_refs per occurrence.
• Current documentation owns implemented contracts, while the feature roadmap is marked implemented and future.md no longer advertises the work as pending.

Open Questions

None. The feature roadmap and fixed decisions above define the product and architectural choices required for implementation.