notarius/docs/roadmap/implementation.md

# D&D NPC Interactions Implementation Plan

Status: Ready for implementation

## Objective

Implement the accepted [D&D NPC Interactions](dnd-npc-interactions.md)
roadmap as a production D&D artifact lane. The finished lane must extract an
ordered list of minimal, evidence-grounded NPC interaction occurrences from
each accepted transcript chunk, ground every NPC against a required accepted
NPC registry, and preserve the existing fixed pipeline and generated-reference
architecture.

This plan is the implementation authority for sequencing and file-level work.
The feature roadmap remains authoritative for product intent, category
semantics, occurrence boundaries, evidence policy, and non-goals. Follow
[ADR-0009](../adr/0009-minimal-evidence-grounded-extraction-artifacts.md)
throughout: do not add descriptive, analytical, relationship, or state fields.

## Fixed Decisions

The implementation must use these identities:

| Concern | Identity |
| --- | --- |
| Extractor key | `dnd/npc-interactions` |
| Normalizer key | `dnd/npc-interactions` |
| Artifact kind | `dnd/npc-interaction-list` |
| Durable schema ID | `notarius.dnd.npc_interactions` |
| Durable schema name | `notarius_dnd_npc_interactions_v1` |
| Durable schema version | `v1` |
| Durable media type | `application/json` |
| Private prompt ID | `dnd.npc_interactions` |
| Private response schema ID | `notarius.dnd.npc_interactions.llm` |
| Reference slot | `npcs` |

Use a top-level `interactions` array. Each durable record has exactly `name`,
`kind`, and `source_refs`. Do not add an interaction ID. The closed kind
vocabulary is `mentioned`, `noncombat_presence`, `dialogue`, `combat_ally`,
`combat_opponent`, and `other`.

The `npcs` slot is required for both extraction and normalization. It accepts
only the existing `dnd/npc-list` JSON artifact within the existing NPC registry
size limit. A bound registry with an empty `npcs` array is valid and requires an
empty interaction artifact. An absent binding, malformed registry, failed
generated handoff, or unknown interaction name is not permission to perform
ungrounded extraction.

Use the existing `internal/modules/dnd/npcs/registry.Resolver` for construction-
time external references and operation-time generated references. The
extractor presents only `Registry.PromptInput()` to the model. Name lookup uses
the registry's established comparison-key policy before normalization; the
normalizer replaces a recognized variant with the exact canonical registry
display name. Registry source references and campaign references never become
interaction evidence.

An occurrence is one NPC, one kind, and one coherent passage within one
accepted chunk. The model must split a category or combat-alignment transition.
No occurrence spans chunks. Merge preserves chunk order. Normalization:

1. replaces recognized names with canonical registry display names;
2. sorts and exact-deduplicates each record's source references;
3. orders records by earliest valid source position, then canonical name,
   interaction kind, and the canonical source-reference sequence;
4. removes records only when canonical name, kind, and the complete canonical
   source-reference sequence are all identical; and
5. never semantically combines nearby, overlapping, or cross-chunk records.

The private model response remains structural and contains name, kind, and
integer start/end unit candidates. Mapping attaches the current source ID.
Invalid strings, kinds, and ranges must survive mapping far enough for the
deterministic validator that owns the rule to reject them.

For the final record comparator, use source-document unit position for the
earliest valid reference. Compare canonical names first by the existing NPC
identity comparison key and then by exact display string; compare kinds by
their string values; and compare canonical reference sequences
lexicographically by `source_id`, start-unit document position, and end-unit
document position. Records with valid evidence precede records without it;
normalized output validation rejects the latter before durable encoding.

No framework change, new identity subsystem, DAG behavior, LLM-assisted
deduplication, or scene dependency is part of this plan.

## Testing And Documentation Rules

Apply [Testing Policy](../policy/testing.md) at every stage:

- Protect artifact round trips, required fields, enum values, source
  provenance, registry grounding, deterministic ordering, exact
  deduplication, reference immutability, and generated handoff behavior.
- Use the real codec, registry, resolver, normalizer, and pipeline machinery
  when they are fast and deterministic. Stub only the structured LLM boundary.
- Keep provider calls offline and deterministic.
- Test each policy at its narrowest stable owner. Do not repeat every codec,
  validator, and normalizer case in an end-to-end test.
- Do not add a prompt-cache “change detector” based on exact message count,
  shared-prefix length, token count, or prompt hash. Reuse the shared prompt
  assets, test rendered behavior and manifest composition, and document the
  ordering policy instead.
- Do not use broad golden snapshots for prompts or diagnostics. A maintained
  durable JSON fixture is appropriate because the complete serialized artifact
  is an intentional integration contract.

Until the final stage, keep unimplemented behavior only in roadmap
documentation. Update current-behavior documentation and examples in the same
stage that makes the production lane selectable.

## Stage 1 — Add The Typed Artifact And Durable Codec

### Goal

Establish the domain type and strict serialized boundary without changing the
production module catalog.

### Changes

1. Extend `internal/modules/dnd/types.go` with:
   - `NPCInteractionListKind`;
   - a string-backed `NPCInteractionKind`;
   - exported constants for all six accepted values;
   - `NPCInteractionList` with `Interactions []NPCInteraction`; and
   - `NPCInteraction` with `Name string`, `Kind NPCInteractionKind`, and
     `SourceRefs []source.SourceRef`.
2. Add `internal/modules/dnd/codec/npcinteractions` following the existing NPC
   and combat-turn codec boundary:
   - expose the fixed schema constants and `MediaType`;
   - implement the typed artifact codec for `dnd.NPCInteractionList`;
   - keep `EncodeCandidate` and `DecodeCandidate` strict about one JSON value
     and unknown fields while preserving typed semantic candidates;
   - make approved `Encode` and `Decode` enforce non-empty names, the closed
     kind enum, at least one structurally valid source reference, and the
     durable schema shape; and
   - report only `interaction_count` in codec metadata.
3. Add
   `internal/modules/dnd/codec/npcinteractions/assets/schemas/dnd_npc_interactions.v1.json`.
   It must require the top-level array and all three record fields, reject
   unknown fields at every object level, encode the six-value enum, require at
   least one source reference, and use the established durable source-reference
   shape.
4. Add a small maintained fixture under
   `internal/modules/dnd/codec/npcinteractions/testdata/` containing at least
   two ordered records with distinct kinds and source ranges.

### Tests

At the codec package boundary, cover:

- fixture decode/encode round trip and canonical compact JSON;
- exact kind, schema identity, version, media type, and Go type registration;
- empty-list support and nil-versus-present-empty behavior where the existing
  codecs distinguish it;
- strict rejection of malformed JSON, trailing values, and unknown fields;
- approved-boundary rejection of each meaningful required-field, enum, and
  source-reference violation;
- candidate-boundary preservation of semantic values for later validators; and
- defensive copies for schema bytes and metadata.

Do not duplicate source-document existence checks in the codec; those belong to
the source-reference validator.

### Completion Criteria

- The new types and codec compile and pass focused tests.
- The codec can be registered into an isolated artifact codec registry with
  exact type `dnd.NPCInteractionList`.
- No production registrar or configuration catalog exposes the new kind yet.

## Stage 2 — Implement The Chunk-Scoped Extractor And Prompt

### Goal

Add an independently testable LLM-backed extractor that requires NPC grounding,
maps only minimal private output, and follows the established D&D prompt-cache
layout.

### Changes

1. Add `internal/modules/dnd/extract/npcinteractions` using the current D&D
   extractor organization:
   - `assets.go`;
   - `canonicalize.go`;
   - `extractor.go`;
   - `model.go`;
   - `schema.go`;
   - `scriptorium_assets.go`;
   - corresponding focused tests; and
   - package-local embedded prompt and schema assets.
2. Give the extractor strict empty options, required capabilities `chunks` and
   `source.transcript`, provided capability `dnd.npc_interactions`, and artifact
   kind `dnd.NPCInteractionListKind`.
3. Build its reference slots from `shared.ReferenceSlots(...)`, then append the
   `npcs` slot with:
   - `Required: true`;
   - accepted media type `application/json`;
   - accepted artifact kind `dnd.NPCListKind`;
   - the existing registry maximum byte count; and
   - a description stating that the accepted registry is required identity
     grounding, not evidence.
   Return defensive, consistently sorted slot declarations from both
   `ModuleSpec()` and the constructed extractor.
4. Construct an `npcregistry.Resolver` from the preparation-time reference set.
   Resolve operation references inside `Extract`, require `Registry.Bound()`,
   and fail before the LLM call if the required registry is absent or invalid.
   Preserve the resolver's content-safe error behavior.
5. Use `shared.ChunkPromptMaterial` and `shared.PromptInputs`. Replace the
   ordinary `npcs` input with the resolved names-only `Registry.PromptInput()`.
   Do not place full registry bytes, NPC IDs, registry evidence, or provenance
   in prompt inputs.
6. Add these assets:
   - `assets/prompts/dnd.npc_interactions.yaml`;
   - `assets/prompts/task.md`;
   - `assets/prompts/instructions.md`; and
   - `assets/schemas/dnd_npc_interactions_llm.v1.json`.
7. Use the existing shared prompt assets in this exact semantic order:
   common system; common extraction evidence; common identity; common campaign
   references; common NPC grounding; lane task; lane instructions; variable
   transcript. Preserve the established cache-control placement used by the
   spell and combat-turn manifests.
8. Revise the shared `common-dnd-npcs.md` wording once so it truthfully applies
   to all three consumers: a normalized registry is always presented to the
   prompt and may be empty. Keep its identity, context-only rule, and
   prohibition on treating registry provenance as event evidence. Do not fork a
   nearly identical interaction-specific grounding asset.
9. The lane prompt must state the category definitions, precedence, occurrence
   splitting rules, registry-only name restriction, empty-output behavior, and
   exclusions from the feature roadmap. Keep the response schema structural:
   it requires the envelope and fields but leaves enum membership, non-empty
   values, and source semantics to deterministic validators.
10. Map private records to `dnd.NPCInteractionList`, attach the current source
    ID, sort and exact-deduplicate ranges within each candidate, and stable-sort
    candidates with valid evidence by earliest source-document position.
    Preserve invalid candidate fields rather than repairing or dropping them.
11. Expose prompt, response-schema, and mapping-policy identities through
    manifest metadata and checkpoint fingerprints, following the current D&D
    extractors. Include the NPC names-only projection digest in local checkpoint
    identity. For a preparation-time external registry, expose only bounded
    digest/count metadata; do not place names or source content in metadata.
    Generated-reference provenance remains framework-owned.

### Tests

At stable package boundaries, cover:

- nil/cancelled/invalid extraction requests and provider failures;
- required-slot declaration and rejection of an unbound or malformed registry
  before any LLM call;
- operation-time generated registry resolution by an extractor prepared
  without generated bytes;
- exact names-only registry prompt input and separation from transcript
  evidence;
- empty registry plus empty response;
- mapping of every category, current-source attachment, evidence
  canonicalization, and source-position ordering;
- preservation of invalid names, kinds, and ranges for validators;
- strict private JSON shape and unknown-field rejection;
- prompt rendering with required inputs, category policy, shared assets, and
  transcript-last ordering;
- prompt/schema registration and content-safe metadata/fingerprints; and
- defensive module specifications and strict rejection of unknown options.

Run the existing spell and combat-turn prompt/asset tests after changing the
shared NPC prompt fragment. Do not assert an exact shared-prefix length.

### Completion Criteria

- The extractor is constructible and testable through the typed extractor
  contract.
- A valid request makes one scheduled structured-completion call and returns a
  typed candidate with current-source provenance.
- Missing required NPC grounding cannot reach the model.
- The extractor remains unregistered in the production D&D family until later
  stages provide the rest of the lane.

## Stage 3 — Add Deterministic Artifact Validators

### Goal

Give each semantic invariant one clear validation owner and make invalid model
output a rejection rather than a normalization repair or framework error.

### Changes

Add these typed validator packages, each with strict empty options, bounded
diagnostics, typed registration, and the fixed artifact kind:

1. `internal/modules/dnd/validate/npcinteractions/shape`
   (`extract/dnd/npc-interactions/shape`):
   - require a non-empty trimmed name without mutating it;
   - require one of the six kinds; and
   - require at least one source-reference candidate with structurally required
     values.
2. `internal/modules/dnd/validate/npcinteractions/registry`
   (`extract/dnd/npc-interactions/registry`):
   - use `npcregistry.Resolver` at construction and operation time;
   - require a bound registry;
   - approve a name when `Registry.Lookup` recognizes it under the established
     comparison policy;
   - reject unknown names in stable record order; and
   - never use registry evidence as transcript evidence.
3. `internal/modules/dnd/validate/npcinteractions/source_refs`
   (`extract/dnd/npc-interactions/source_refs`):
   - require the current source identity;
   - require referenced units to exist; and
   - require valid inclusive start/end order through the current document.
   Follow the established D&D citation helpers and rejection aggregation
   limits rather than introducing a generic framework dependency on the
   interaction type.
4. `internal/modules/dnd/validate/npcinteractions/source_relatedness`
   (`extract/dnd/npc-interactions/source_relatedness`):
   - inspect only current transcript text covered by the cited ranges;
   - emit at most one bounded warning per record when the NPC name cannot be
     related to that evidence;
   - do not claim to validate category agreement; and
   - ignore campaign and registry content for relatedness.
5. `internal/modules/dnd/validate/npcinteractions/invariants`
   (`normalize/dnd/npc-interactions/invariants`):
   - require the exact canonical display name returned by registry lookup;
   - require canonical source-reference order with no duplicate ranges;
   - require the complete list order defined in Fixed Decisions; and
   - reject exact duplicate normalized records.

The registry and invariants validators must support operation-time generated
references. Construction-time metadata and fingerprints follow the resolver
pattern used by registry-aware modules and must not include NPC names or
content.

### Tests

Give each policy one primary test owner:

- shape tests own empty names, enum membership, and missing evidence;
- registry tests own required binding, comparison-key recognition, unknown
  names, operation-time generated overrides, empty registries, and content-safe
  failures;
- source-reference tests own wrong source IDs, missing units, reversed ranges,
  and valid multi-range records;
- relatedness tests own current-transcript-only warnings and bounded
  diagnostics; and
- invariant tests own exact canonical names, final ordering, canonical evidence
  sequences, and duplicate rejection.

Also verify that validators do not mutate artifacts, source documents, or
registries and that their specs register only for
`dnd.NPCInteractionListKind`.

### Completion Criteria

- Valid typed candidates are approved and malformed or ungrounded candidates
  are rejected at the intended boundary.
- No validator silently canonicalizes, drops, or merges records.
- References can explain identity but cannot satisfy source evidence checks.

## Stage 4 — Add Deterministic Merge And Normalization

### Goal

Preserve chunk-scoped occurrences through merge, then produce the canonical
ordered artifact without semantic inference.

### Changes

1. Add `internal/modules/dnd/normalize/npcinteractions` with strict empty
   options and the fixed module key and artifact kind.
2. Declare the same required `npcs` reference slot as the extractor. Construct
   and resolve `npcregistry.Resolver` at the same preparation/operation
   boundaries, and require a bound registry.
3. Implement the normalization algorithm in Fixed Decisions:
   - clone all nested values;
   - normalize display whitespace only as needed for lookup;
   - replace every recognized name with the registry's exact canonical display
     name;
   - canonicalize each reference list without merging overlapping ranges;
   - compute order from source-document positions rather than assuming numeric
     unit IDs are contiguous;
   - apply every specified tie-breaker; and
   - collapse only exact records after canonicalization.
4. Preserve nil versus present-empty list behavior consistently with the other
   D&D normalizers.
5. Emit bounded warnings for visible canonical-name changes, reference
   canonicalization, record reordering, and exact duplicate removal. Use stable
   reason codes and the shared D&D diagnostics helpers. Never include reference
   content in a warning.
6. Add an append-only merger function for `dnd.NPCInteractionList` beside the
   other D&D typed append functions in
   `internal/modules/dnd/register/merge.go`. It must preserve accepted
   chunk/lane order, preserve present-empty semantics, and deep-clone source
   reference slices so outputs do not alias inputs. Do not register it until
   Stage 5.
7. Add normalization-policy and NPC projection identities to checkpoint
   fingerprints. Manifest metadata may include the policy identity and bounded
   external-registry digest/count, but not names or source content.

### Tests

Cover:

- canonical registry name replacement, including Unicode/case/spacing lookup;
- required and operation-time generated registry behavior;
- source-reference ordering and exact range deduplication;
- ordering by real document position with all deterministic tie-breakers;
- separation of category transitions, alignment transitions, nearby records,
  overlapping-but-nonidentical evidence, and nonidentical records from
  different chunks;
- collapse of exact duplicates only;
- stable, bounded warnings;
- nil/present-empty behavior;
- input ownership and nested-slice cloning; and
- append-order merger behavior independently of normalization.

Use table-driven pure normalization cases for the dense ordering and duplicate
rules, and package-level normalizer tests for resolver and warning behavior.

### Completion Criteria

- Normalization is deterministic and idempotent.
- Running normalization twice does not change the value or emit new
  transformation warnings on the second pass.
- Semantically distinct occurrences remain distinct.
- The module and merger compile but are not yet selectable through production
  composition.

## Stage 5 — Compose The Complete Production Lane

### Goal

Register the complete typed lane atomically so configuration cannot select a
partial implementation.

### Changes

Update the D&D registrar:

1. In `internal/modules/dnd/register/modules.go`, register:
   - the NPC-interaction codec;
   - extractor and prompt assets;
   - the append-order merger variant for
     `dnd.NPCInteractionListKind`;
   - the `dnd/npc-interactions` normalizer; and
   - the generic no-op normalizer variant for the new exact Go type.
2. In `internal/modules/dnd/register/validators.go`, register all five domain
   validators plus generic always-accept and always-reject typed variants for
   the new artifact type.
3. In `internal/modules/dnd/register/chains.go`, register:
   - extraction chain:
     `generic/valid_json`,
     `extract/dnd/npc-interactions/shape`,
     `extract/dnd/npc-interactions/registry`,
     `extract/dnd/npc-interactions/source_refs`,
     `generic/valid_json_schema`,
     `extract/dnd/npc-interactions/source_relatedness`;
   - normalization chain:
     `generic/valid_json`,
     `extract/dnd/npc-interactions/shape`,
     `extract/dnd/npc-interactions/registry`,
     `normalize/dnd/npc-interactions/invariants`,
     `extract/dnd/npc-interactions/source_refs`,
     `generic/valid_json_schema`,
     `extract/dnd/npc-interactions/source_relatedness`.
4. Do not add a default merge validator chain. The append merger performs no
   semantic mutation, and extraction plus normalized boundaries own the
   consequential policies.
5. Extend registrar contract tests to verify exact typed coverage, keys,
   artifact kinds, prompt/schema assets, reference-slot agreement, and default
   validator order.

### Tests

- Extend the existing D&D family registration test rather than creating
  parallel catalog snapshots.
- Verify that the extractor and normalizer both expose required compatible
  `npcs` slots and that generated `dnd/npc-list` bindings resolve only from an
  earlier step.
- Verify production resolution rejects a missing required binding, a later or
  same-step producer, a wrong artifact kind, and incompatible media/schema
  metadata through existing resolver behavior.
- Avoid retesting the framework's general ordered-step failure matrix; add only
  interaction-specific composition cases not already covered generically.

### Completion Criteria

- Production registration exposes one complete type-consistent lane.
- A valid two-step NPC-to-interactions pipeline resolves and prepares.
- Invalid or missing NPC dependencies fail before source parsing or LLM work
  whenever statically discoverable.

## Stage 6 — Prove The Workflow And Publish Current Contracts

### Goal

Exercise the assembled generated-reference workflow and move all implemented
behavior into its canonical current documentation.

### Changes

1. Add one representative integration test under
   `internal/modules/integration` with focused testdata:
   - step 1 extracts and normalizes NPCs;
   - step 2 consumes the generated NPC artifact in an interaction lane;
   - the fake structured LLM returns multiple interaction kinds and at least
     one name variant;
   - the assertion proves the names-only handoff, current-transcript evidence,
     canonical name, stable chronology, and final durable artifact.
2. Include one failure assertion showing that a rejected or absent normalized
   NPC producer prevents interaction extraction. Reuse the existing runner
   dependency behavior; do not duplicate every framework failure case.
3. Add `examples/dnd-npc-interactions.config.yml` as a complete copyable
   two-step profile. The first step produces `npcs`; the second step binds that
   exact step/lane artifact to its `npcs` reference and selects
   `dnd/npc-interactions` for extract and normalize.
4. Add the example to the existing CLI production configuration contract test
   so it is parsed and resolved offline.
5. Add the durable external contract at
   `docs/integrations/dnd-npc-interaction-artifacts.md`. It owns schema
   identity, JSON shape, categories, evidence semantics, ordering,
   normalization, validator chains, generated-reference behavior, and manifest
   metadata.
6. Update current canonical documentation:
   - `docs/config.md` for module/validator catalogs, default chains, required
     reference slot, and the maintained example;
   - `docs/internal/modules.md` for implementation ownership, prompt inputs,
     registry resolution, validation, merge, and normalization;
   - `docs/internal/overview.md` for the implemented component inventory;
   - `docs/internal/llm.md` only as needed to include the new lane in the
     existing D&D prompt-order/cache policy without duplicating the manifest;
   - `docs/integrations/dnd-npc-artifacts.md` to identify NPC interactions as a
     consumer while retaining the rule that registry evidence is not event
     evidence; and
   - any directly affected configuration or integration links.
7. Remove the NPC-interactions entry from `docs/roadmap/future.md` once the
   production behavior and current documentation are complete. Change
   `docs/roadmap/dnd-npc-interactions.md` to `Status: Implemented` pending its
   later retirement; do not leave it as the canonical current contract.
8. Create a small human-review worksheet or fixture set only if the repository
   already has an appropriate non-test evaluation home. Otherwise record the
   manual evaluation results in the implementation handoff rather than
   inventing a new framework. Exercise all six categories, transitions,
   repeated occurrences, mentions followed by presence, combat alignment
   changes, cross-chunk repetition, and empty output on at least one intended
   smaller model. Do not make model agreement a deterministic CI gate.

### Validation

Run focused tests while implementing, then run:

```sh
go test ./...
go vet ./...
go build ./cmd/notarius
```

Also run `git diff --check`, inspect the complete documentation diff for
current-versus-future claims, and verify every new relative documentation link.

### Completion Criteria

- All repository checks pass offline.
- The maintained example resolves with an explicit earlier NPC producer.
- The integration test proves the accepted generated artifact, not registry
  source references, grounds the later model request.
- Durable output contains only `name`, `kind`, and current-transcript
  `source_refs` per occurrence.
- Current documentation owns implemented contracts, while the feature roadmap
  is marked implemented and `future.md` no longer advertises the work as
  pending.

## Open Questions

None. The feature roadmap and fixed decisions above define the product and
architectural choices required for implementation.