notarius/docs/roadmap/entity-registries.md

Canonical D&D Entity Registries And Occurrences

Purpose

This roadmap defines the target state for D&D entity extraction in Notarius. NPCs, locations, and items should follow one legible architecture: a registry lane first identifies canonical entities, and a later occurrence lane consumes that registry to extract independently evidenced facts about those entities.

The work is a clean pre-release contract redesign. It should remove the current mixture of bare plural, interaction, occurrence, and event terminology; add the missing item registry; tighten entity eligibility; and make generated-reference relationships visible in module, artifact, schema, prompt, validation, example, and documentation names.

This roadmap owns the desired behavior and policy. The implementation plan owns sequencing and task breakdowns.

Background

Notarius already implements most of the required platform mechanics:

• ordered pipeline steps create a barrier between registry production and occurrence extraction;
• generated artifact references are validated for topology, media type, artifact kind, schema, and codec compatibility before execution;
• NPC and location registries have deterministic identities and bounded LLM-assisted reconciliation;
• NPC interactions and location occurrences consume normalized registry artifacts as source-free grounding; and
• validators keep registry provenance separate from occurrence evidence.

The remaining D&D domain contracts are inconsistent:

• dnd/npcs produces a registry, but its name does not communicate that role;
• dnd/npc-interactions is an occurrence lane under a narrower name;
• dnd/locations produces a registry while dnd/location-occurrences already uses the desired downstream terminology;
• dnd/item-events extracts occurrences directly and has no canonical item registry; and
• current location eligibility permits generic labels that are not suitable canonical nouns for reuse by downstream consumers.

ADR-0009 already establishes that changing encounter-level facts belong in occurrence artifacts rather than registry attributes. ADR-0008 already establishes explicit ordered artifact handoffs. This feature applies those decisions consistently; it does not change the platform architecture.

Goals

The completed feature should:

• make registry producers and occurrence consumers immediately recognizable by their names;
• provide registry and occurrence lanes for NPCs, locations, and items;
• require every entity occurrence to resolve to one canonical registry ID and display name;
• tighten registry eligibility so entries are stable nouns or designations that a downstream consumer can reuse;
• preserve minimal, evidence-grounded durable artifacts;
• keep registry provenance distinct from occurrence evidence;
• use ordered generated references rather than inferred dependencies;
• preserve domain-specific occurrence categories and identity rules where the entity types genuinely differ;
• use a clean pre-release migration with no aliases or compatibility shims; and
• leave the codebase with three concrete registry implementations from which a later generic deduplication design can be evaluated.

Non-Goals

This work does not introduce:

• a general workflow DAG or implicit dependency inference;
• a generic framework-level LLM deduplication module;
• campaign-wide or cross-run registries;
• an inventory, currency balance, possession ledger, or current-state model;
• item-instance tracking for interchangeable objects;
• narrative descriptions, biographies, relationships, ownership summaries, or other enrichment on registry entries;
• occurrence-derived attributes copied back onto registry entries;
• NPC IDs on spell, combat-turn, or enemy-event artifacts merely because those lanes consume NPC grounding;
• migration tooling for old checkpoints or durable artifacts; or
• compatibility aliases for retired module keys, artifact kinds, prompt IDs, reference slots, schemas, or validator keys.

Canonical Naming

Modules

Registry extractors and their matching normalizers use dnd/<singular-entity>-registry. Occurrence extractors and their matching normalizers use dnd/<singular-entity>-occurrences.

Role	Current module key	Target module key
NPC registry	dnd/npcs	dnd/npc-registry
NPC occurrences	dnd/npc-interactions	dnd/npc-occurrences
Location registry	dnd/locations	dnd/location-registry
Location occurrences	dnd/location-occurrences	dnd/location-occurrences
Item registry	none	dnd/item-registry
Item occurrences	dnd/item-events	dnd/item-occurrences

The same key is used for a lane's extract and normalize modules, as it is for other current D&D lanes. Stage identity continues to distinguish those module registrations.

Durable artifacts

Registry artifact kinds name the registry directly. Occurrence artifact kinds retain the -list suffix because their durable value is an ordered collection of independently evidenced observations.

Role	Target artifact kind	Target schema ID	Root collection
NPC registry	dnd/npc-registry	notarius.dnd.npc_registry	npcs
NPC occurrences	dnd/npc-occurrence-list	notarius.dnd.npc_occurrences	occurrences
Location registry	dnd/location-registry	notarius.dnd.location_registry	locations
Location occurrences	dnd/location-occurrence-list	notarius.dnd.location_occurrences	occurrences
Item registry	dnd/item-registry	notarius.dnd.item_registry	items
Item occurrences	dnd/item-occurrence-list	notarius.dnd.item_occurrences	occurrences

All redesigned durable schemas remain version v1. Notarius is pre-release, and these names replace rather than version or alias the retired contracts. Schema names should follow the corresponding underscore form, such as notarius_dnd_item_registry_v1.

Canonical Go types, codec packages, validator packages, prompt IDs, private response-schema identities, capability names, and asset subtrees should use the same registry/occurrence vocabulary. Prompt IDs should be dnd.<entity>_registry, dnd.<entity>_registry.normalize, and dnd.<entity>_occurrences as applicable.

The corresponding durable integration contracts should be:

• docs/integrations/dnd-npc-registry-artifacts.md;
• docs/integrations/dnd-npc-occurrence-artifacts.md;
• docs/integrations/dnd-location-registry-artifacts.md;
• docs/integrations/dnd-location-occurrence-artifacts.md;
• docs/integrations/dnd-item-registry-artifacts.md; and
• docs/integrations/dnd-item-occurrence-artifacts.md.

Those documents will become canonical only when the contracts are implemented; until then, this roadmap is the sole owner of the proposed names and behavior.

References

Registry-consuming modules expose explicit reference slots named:

• npc_registry;
• location_registry; and
• item_registry.

Generated-reference aliases in maintained examples should use those same names. The reference slot, not a matching lane label, owns dependency resolution. Each slot accepts only the corresponding registry artifact kind and media type and retains an explicit size bound.

Existing NPC-grounded consumers such as spells, combat turns, enemy events, and NPC occurrences should use npc_registry. Location occurrences should use location_registry, and item occurrences should require item_registry at both extraction and normalization.

Registry Contract

Every registry answers one narrow question: which reusable canonical entities of this type are established by the current transcript?

Each registry record contains only:

• a deterministic application-owned id;
• a canonical display name; and
• one or more transcript source_refs establishing the entity identity.

Registries do not contain encounter roles, occurrence categories, descriptions, summaries, aliases, relationships, inventories, holder state, or inferred attributes. Their source references establish registry provenance only.

Registry extraction remains chunk-scoped. Merge and normalization combine candidate records across chunks, canonicalize evidence in source-document order, apply deterministic identity rules, and may use a bounded structured-LLM proposal to reconcile semantic duplicates. The model proposes duplicate groups; deterministic code validates keys, membership, overlap, canonical selection, and application. Invalid or uncertain proposals preserve the safe deterministic result and produce bounded diagnostics.

The existing shared D&D entity-reconciliation substrate should support all three registries. Entity-specific candidate construction, identity judgment, canonical-name preference, and consolidation remain in the owning D&D module. This feature must not move D&D policy into generic framework packages.

Entity Eligibility And Identity

Eligibility is based on whether the transcript establishes a stable reusable referent, not on capitalization alone. Transcript capitalization is unreliable, and a capitalized generic phrase is not automatically a canonical entity.

NPC registry

Include an NPC when the transcript establishes an individually identifiable non-player character through a proper name or a stable unique designation. A title or descriptive designation qualifies only when the transcript uses it as a persistent identity, such as the Masked Envoy; a transient generic role such as a guard, the bartender, or one of the bandits does not qualify.

Exclude player characters, speakers known only by player identity, anonymous groups, creature categories, generic roles, speculative identities, and labels invented merely to make an occurrence referable.

NPC IDs use the npc:sha256: prefix and identity policy dnd.npc_registry.identity.v1. The digest is the lowercase SHA-256 of compact JSON for [policy, comparison_name], where the comparison name uses the existing Unicode, apostrophe, whitespace, and case normalization. Semantic reconciliation may collapse supported aliases but may not merge distinct people merely because their names or roles are similar.

Location registry

Include a location only when the transcript establishes a stable proper name or unique in-world designation, such as Waterdeep, The Yawning Portal, or a uniquely named chamber. Exclude generic, temporary, relative, or purely descriptive references such as the room, the bar, the hallway, outside, or upstairs.

A later occurrence may resolve a generic phrase such as the bar to an existing named registry location when the current transcript context supports that coreference. The generic phrase must not create a new registry entry.

Location IDs use the location:sha256: prefix and identity policy dnd.location_registry.identity.v1. The digest is the lowercase SHA-256 of compact JSON for [policy, comparison_name, source_id, start_unit_id, end_unit_id], using the earliest canonical evidence anchor. Source anchoring keeps distinct places with the same display name distinguishable. Nested places and same-name places remain distinct unless a validated semantic proposal establishes that candidates identify the same physical place.

Item registry

Items do not need proper names. Include a reusable, transcript-established item name at the narrowest useful type or unique-designation level, including:

• named unique objects;
• concrete item types such as Healing Potion when the transcript establishes that type;
• stable unique designations used by the session; and
• individual currency denominations such as Silver Pieces.

Exclude vague or non-reusable descriptions such as loot, treasure, some gear, a weapon, or something valuable unless the transcript supplies a more stable name or designation. Do not infer mechanical subtype, magic properties, denomination, quantity, or uniqueness.

An item-registry record identifies a canonical item kind or designation, not a physical instance. Interchangeable objects with the same canonical name share one registry identity and are distinguished in occurrences by evidence, quantity, and holder transitions. A genuinely named unique object naturally has its own canonical identity.

Item IDs use the item:sha256: prefix and identity policy dnd.item_registry.identity.v1. The digest is the lowercase SHA-256 of compact JSON for [policy, comparison_name], using the same text normalization family as other entity registries. Item normalization may reconcile transcript- supported aliases and abbreviation variants but may not collapse distinct denominations or materially different item types.

Occurrence Contract

Every occurrence answers a separate question: what source-grounded event or relationship to a known registry entity occurred in this transcript passage?

Each occurrence record contains:

• the exact entity ID from its required registry;
• the exact canonical display name associated with that ID;
• one bounded occurrence kind; and
• one or more source references from the current transcript.

Entity-specific fields are allowed only when required by that occurrence contract. Registry IDs and names provide grounding; registry source references never become occurrence evidence. Unknown IDs and mismatched ID/name pairs are rejected rather than guessed, reassigned, or repaired from display-name similarity.

Occurrence extractors may resolve an alias, pronoun, generic reference, or abbreviation to an existing registry entry only when the current chunk supports that coreference. They may not add entities to the registry. Omit an occurrence when entity resolution or the occurrence itself is not adequately supported.

Occurrence normalization is deterministic. It validates registry membership, canonicalizes the display name from the registry, orders and deduplicates source references, preserves distinct observations, orders records by source-document chronology and stable domain tie-breakers, and collapses only exact duplicates.

NPC occurrences

The durable record contains required npc_id, name, kind, and source_refs. It retains the current interaction categories:

• mentioned;
• noncombat_presence;
• dialogue;
• combat_ally;
• combat_opponent; and
• other.

The precedence and splitting rules of the current NPC-interaction contract remain applicable. Renaming the artifact does not turn it into a biography, relationship model, sentiment analysis, or persistent NPC state.

Enemy-event grounding consumes combat_opponent NPC occurrences under the new contract. Other NPC-grounded lanes continue to consume the registry directly.

Location occurrences

The durable record continues to contain required location_id, name, kind, and source_refs. It retains the current categories and precedence:

• visited;
• planned;
• recalled; and
• mentioned.

Generic current-chunk language may resolve to a named registry location, but the occurrence must cite the current transcript passage that supports both the resolution and the occurrence category.

Item occurrences

The durable record contains required item_id, name, kind, and source_refs, plus the current semantically conditional quantity, from, and to fields. It retains the current categories:

• discovered;
• acquired;
• lost;
• consumed; and
• transferred.

Holder and quantity rules remain unchanged: discovery has no holder; acquisition requires a gaining holder; loss and consumption require a losing holder; and transfer requires distinct losing and gaining holders. Currency remains an ordinary item occurrence with an explicit denomination and a quantity only when the transcript establishes one.

Item occurrences do not compute inventory, merge successive state changes, convert denominations, infer unspoken holders, or distinguish physical instances that the transcript does not identify.

Private LLM response schemas should remain strict and compatible with providers that require every declared property to be required. Nullable or explicit absence representations may be used privately for semantically conditional item fields; deterministic mapping owns omission from the durable artifact when the public contract calls for an absent field.

Pipeline End State

A complete D&D pipeline produces all three registries in an earlier step and binds them explicitly into later consumers. Conceptually:

registry step
  npc-registry
  location-registry
  item-registry
  scene-descriptions

occurrence and action step
  npc-occurrences      <- npc-registry
  location-occurrences <- location-registry
  item-occurrences     <- item-registry
  spells               <- optional npc-registry
  combat-turns         <- optional npc-registry + required scene descriptions

derived enemy step
  enemy-events <- npc-registry + npc-occurrences + combat turns + scene descriptions

Steps remain barriers with the existing whole-run failure behavior. A required registry producer that fails, rejects, or does not publish an accepted normalized artifact prevents its dependent later step and fails the run under the existing ordered-pipeline contract. Configuration with a same-step or forward generated reference remains invalid.

The maintained complete D&D example should use generated references for these same-run handoffs. Registry slots continue to accept compatible external path references under the existing configuration contract; this feature does not remove the ability to supply a previously produced or independently maintained registry artifact.

The shared scene plan remains pipeline-wide. This feature changes eligible lanes and reference bindings, not chunking or scheduling semantics.

Validation And Normalization

Each registry receives production defaults for:

• JSON and durable-schema validity;
• registry shape and required evidence;
• deterministic identity and uniqueness;
• source-reference validity;
• source relatedness; and
• safe bounded reconciliation during normalization.

Each occurrence family receives production defaults for:

• JSON and durable-schema validity;
• occurrence shape and kind-specific invariants;
• exact registry ID/name membership;
• source-reference validity;
• source relatedness; and
• deterministic ordering and exact-duplicate removal.

Validator and capability keys should adopt the new module vocabulary. Registry membership validators consume the same immutable registry projection used by the extractor and normalizer. Validators do not use registry provenance as a substitute for occurrence evidence and do not mutate supplied artifacts or references.

Warnings remain bounded, content-safe, and deterministic. Prompt, schema, registry, and identity fingerprints remain represented by hashes or stable identities rather than raw content in manifests and diagnostics.

Package And Asset Organization

Implementation packages should make the two roles visible, using Go-compatible names such as npcregistry, npcoccurrences, locationregistry, and itemoccurrences beneath the existing extract, normalize, codec, and validate families. Entity-specific identity and immutable registry-resolution helpers remain inside the D&D domain.

LLM assets should use corresponding subtrees beneath assets/dnd/, for example assets/dnd/npc-registry/{extract,normalize} and assets/dnd/item-occurrences. The root assets package remains a single-file, content-only filesystem boundary with no business logic, registration, or PromptKit dependency.

All extraction prompts retain the shared D&D prompt prefix and chunk scope. Registry prompts own only entity eligibility and canonical-name policy; occurrence prompts own only occurrence semantics and entity-specific fields. Registry input fragments own the meaning of their source-free projections.

Migration Policy

The redesign is intentionally breaking and immediate:

• remove retired module and validator keys;
• remove retired artifact kinds, schema IDs, schema names, Go types, prompt IDs, private schema identities, reference slots, capability names, asset paths, and generated-reference aliases;
• do not register aliases or decode old shapes;
• do not retain duplicate integration documents for old contracts; and
• accept invalidation of old checkpoints, prompt fingerprints, provider caches, example output, and other reconstructible pre-release state.

New contracts use version v1; v2 does not communicate useful compatibility information while there is no supported release to preserve. Current-behavior documentation, maintained examples, module catalogs, and integration contracts must change atomically with implementation.

Evaluation Expectations

Implementation should be assessed against representative transcript cases that cover:

• named NPCs, stable NPC designations, transient roles, aliases, and anonymous groups;
• named locations, same-name locations, nested locations, generic spatial phrases, and supported generic coreferences to named places;
• named items, ordinary item types, ambiguous descriptions, aliases, quantities, transfers, consumption, and each currency denomination;
• registry candidates mentioned across multiple chunks;
• unsupported occurrence links and mismatched registry ID/name pairs; and
• output from cost-effective smaller models as well as the default production profile.

Deterministic tests should protect schemas, identity rules, registry resolution, reference compatibility, ordering, validation, normalization, and assembled multi-step configuration. Live model quality remains an opt-in human evaluation activity rather than a default automated gate.

Target End State

The feature is complete when:

• the six canonical entity modules are the only selectable NPC, location, and item registry/occurrence modules;
• all six durable artifacts use the target kinds, schema identities, root collections, minimal record shapes, and v1 contracts defined here;
• every occurrence contains and validates an exact registry ID/name pair;
• NPC, location, and item registry eligibility follows the policies above;
• the maintained complete pipeline uses explicit generated registry references across ordered steps, while compatible external registry references remain supported;
• item occurrences no longer run independently of an accepted item registry;
• spell, combat, enemy, and occurrence consumers use the renamed registry slots without treating registry provenance as evidence;
• current module catalogs, default validator chains, prompts, assets, examples, integration contracts, internal documentation, and tests use only the new vocabulary; and
• repository searches find no production compatibility aliases or stale current- behavior claims for dnd/npcs, dnd/npc-interactions, dnd/locations, or dnd/item-events.