Finish D&D entity contract cleanup
This commit is contained in:
@@ -1,503 +0,0 @@
|
||||
# Canonical D&D Entity Registries And Occurrences
|
||||
|
||||
## Purpose
|
||||
|
||||
This roadmap defines the target state for D&D entity extraction in Notarius.
|
||||
NPCs, locations, and items should follow one legible architecture: a registry
|
||||
lane first identifies canonical entities, and a later occurrence lane consumes
|
||||
that registry to extract independently evidenced facts about those entities.
|
||||
|
||||
The work is a clean pre-release contract redesign. It should remove the current
|
||||
mixture of bare plural, interaction, occurrence, and event terminology; add the
|
||||
missing item registry; tighten entity eligibility; and make generated-reference
|
||||
relationships visible in module, artifact, schema, prompt, validation, example,
|
||||
and documentation names.
|
||||
|
||||
This roadmap owns the desired behavior and policy. The
|
||||
[implementation plan](implementation.md) owns sequencing and task breakdowns.
|
||||
|
||||
## Background
|
||||
|
||||
Notarius already implements most of the required platform mechanics:
|
||||
|
||||
- ordered pipeline steps create a barrier between registry production and
|
||||
occurrence extraction;
|
||||
- generated artifact references are validated for topology, media type,
|
||||
artifact kind, schema, and codec compatibility before execution;
|
||||
- NPC and location registries have deterministic identities and bounded
|
||||
LLM-assisted reconciliation;
|
||||
- NPC interactions and location occurrences consume normalized registry
|
||||
artifacts as source-free grounding; and
|
||||
- validators keep registry provenance separate from occurrence evidence.
|
||||
|
||||
The remaining D&D domain contracts are inconsistent:
|
||||
|
||||
- `dnd/npcs` produces a registry, but its name does not communicate that role;
|
||||
- `dnd/npc-interactions` is an occurrence lane under a narrower name;
|
||||
- `dnd/locations` produces a registry while `dnd/location-occurrences` already
|
||||
uses the desired downstream terminology;
|
||||
- `dnd/item-events` extracts occurrences directly and has no canonical item
|
||||
registry; and
|
||||
- current location eligibility permits generic labels that are not suitable
|
||||
canonical nouns for reuse by downstream consumers.
|
||||
|
||||
[ADR-0009](../adr/0009-minimal-evidence-grounded-extraction-artifacts.md)
|
||||
already establishes that changing encounter-level facts belong in occurrence
|
||||
artifacts rather than registry attributes. [ADR-0008](../adr/0008-ordered-pipeline-steps.md)
|
||||
already establishes explicit ordered artifact handoffs. This feature applies
|
||||
those decisions consistently; it does not change the platform architecture.
|
||||
|
||||
## Goals
|
||||
|
||||
The completed feature should:
|
||||
|
||||
- make registry producers and occurrence consumers immediately recognizable by
|
||||
their names;
|
||||
- provide registry and occurrence lanes for NPCs, locations, and items;
|
||||
- require every entity occurrence to resolve to one canonical registry ID and
|
||||
display name;
|
||||
- tighten registry eligibility so entries are stable nouns or designations
|
||||
that a downstream consumer can reuse;
|
||||
- preserve minimal, evidence-grounded durable artifacts;
|
||||
- keep registry provenance distinct from occurrence evidence;
|
||||
- use ordered generated references rather than inferred dependencies;
|
||||
- preserve domain-specific occurrence categories and identity rules where the
|
||||
entity types genuinely differ;
|
||||
- use a clean pre-release migration with no aliases or compatibility shims; and
|
||||
- leave the codebase with three concrete registry implementations from which a
|
||||
later generic deduplication design can be evaluated.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This work does not introduce:
|
||||
|
||||
- a general workflow DAG or implicit dependency inference;
|
||||
- a generic framework-level LLM deduplication module;
|
||||
- campaign-wide or cross-run registries;
|
||||
- an inventory, currency balance, possession ledger, or current-state model;
|
||||
- item-instance tracking for interchangeable objects;
|
||||
- narrative descriptions, biographies, relationships, ownership summaries, or
|
||||
other enrichment on registry entries;
|
||||
- occurrence-derived attributes copied back onto registry entries;
|
||||
- NPC IDs on spell, combat-turn, or enemy-event artifacts merely because those
|
||||
lanes consume NPC grounding;
|
||||
- migration tooling for old checkpoints or durable artifacts; or
|
||||
- compatibility aliases for retired module keys, artifact kinds, prompt IDs,
|
||||
reference slots, schemas, or validator keys.
|
||||
|
||||
## Canonical Naming
|
||||
|
||||
### Modules
|
||||
|
||||
Registry extractors and their matching normalizers use
|
||||
`dnd/<singular-entity>-registry`. Occurrence extractors and their matching
|
||||
normalizers use `dnd/<singular-entity>-occurrences`.
|
||||
|
||||
| Role | Current module key | Target module key |
|
||||
| --- | --- | --- |
|
||||
| NPC registry | `dnd/npcs` | `dnd/npc-registry` |
|
||||
| NPC occurrences | `dnd/npc-interactions` | `dnd/npc-occurrences` |
|
||||
| Location registry | `dnd/locations` | `dnd/location-registry` |
|
||||
| Location occurrences | `dnd/location-occurrences` | `dnd/location-occurrences` |
|
||||
| Item registry | none | `dnd/item-registry` |
|
||||
| Item occurrences | `dnd/item-events` | `dnd/item-occurrences` |
|
||||
|
||||
The same key is used for a lane's extract and normalize modules, as it is for
|
||||
other current D&D lanes. Stage identity continues to distinguish those module
|
||||
registrations.
|
||||
|
||||
### Durable artifacts
|
||||
|
||||
Registry artifact kinds name the registry directly. Occurrence artifact kinds
|
||||
retain the `-list` suffix because their durable value is an ordered collection
|
||||
of independently evidenced observations.
|
||||
|
||||
| Role | Target artifact kind | Target schema ID | Root collection |
|
||||
| --- | --- | --- | --- |
|
||||
| NPC registry | `dnd/npc-registry` | `notarius.dnd.npc_registry` | `npcs` |
|
||||
| NPC occurrences | `dnd/npc-occurrence-list` | `notarius.dnd.npc_occurrences` | `occurrences` |
|
||||
| Location registry | `dnd/location-registry` | `notarius.dnd.location_registry` | `locations` |
|
||||
| Location occurrences | `dnd/location-occurrence-list` | `notarius.dnd.location_occurrences` | `occurrences` |
|
||||
| Item registry | `dnd/item-registry` | `notarius.dnd.item_registry` | `items` |
|
||||
| Item occurrences | `dnd/item-occurrence-list` | `notarius.dnd.item_occurrences` | `occurrences` |
|
||||
|
||||
All redesigned durable schemas remain version `v1`. Notarius is pre-release,
|
||||
and these names replace rather than version or alias the retired contracts.
|
||||
Schema names should follow the corresponding underscore form, such as
|
||||
`notarius_dnd_item_registry_v1`.
|
||||
|
||||
Canonical Go types, codec packages, validator packages, prompt IDs, private
|
||||
response-schema identities, capability names, and asset subtrees should use the
|
||||
same registry/occurrence vocabulary. Prompt IDs should be
|
||||
`dnd.<entity>_registry`, `dnd.<entity>_registry.normalize`, and
|
||||
`dnd.<entity>_occurrences` as applicable.
|
||||
|
||||
The corresponding durable integration contracts should be:
|
||||
|
||||
- `docs/integrations/dnd-npc-registry-artifacts.md`;
|
||||
- `docs/integrations/dnd-npc-occurrence-artifacts.md`;
|
||||
- `docs/integrations/dnd-location-registry-artifacts.md`;
|
||||
- `docs/integrations/dnd-location-occurrence-artifacts.md`;
|
||||
- `docs/integrations/dnd-item-registry-artifacts.md`; and
|
||||
- `docs/integrations/dnd-item-occurrence-artifacts.md`.
|
||||
|
||||
Those documents will become canonical only when the contracts are implemented;
|
||||
until then, this roadmap is the sole owner of the proposed names and behavior.
|
||||
|
||||
### References
|
||||
|
||||
Registry-consuming modules expose explicit reference slots named:
|
||||
|
||||
- `npc_registry`;
|
||||
- `location_registry`; and
|
||||
- `item_registry`.
|
||||
|
||||
Generated-reference aliases in maintained examples should use those same names.
|
||||
The reference slot, not a matching lane label, owns dependency resolution. Each
|
||||
slot accepts only the corresponding registry artifact kind and media type and
|
||||
retains an explicit size bound.
|
||||
|
||||
Existing NPC-grounded consumers such as spells, combat turns, enemy events, and
|
||||
NPC occurrences should use `npc_registry`. Location occurrences should use
|
||||
`location_registry`, and item occurrences should require `item_registry` at
|
||||
both extraction and normalization.
|
||||
|
||||
## Registry Contract
|
||||
|
||||
Every registry answers one narrow question: which reusable canonical entities
|
||||
of this type are established by the current transcript?
|
||||
|
||||
Each registry record contains only:
|
||||
|
||||
- a deterministic application-owned `id`;
|
||||
- a canonical display `name`; and
|
||||
- one or more transcript `source_refs` establishing the entity identity.
|
||||
|
||||
Registries do not contain encounter roles, occurrence categories, descriptions,
|
||||
summaries, aliases, relationships, inventories, holder state, or inferred
|
||||
attributes. Their source references establish registry provenance only.
|
||||
|
||||
Registry extraction remains chunk-scoped. Merge and normalization combine
|
||||
candidate records across chunks, canonicalize evidence in source-document
|
||||
order, apply deterministic identity rules, and may use a bounded structured-LLM
|
||||
proposal to reconcile semantic duplicates. The model proposes duplicate groups;
|
||||
deterministic code validates keys, membership, overlap, canonical selection,
|
||||
and application. Invalid or uncertain proposals preserve the safe deterministic
|
||||
result and produce bounded diagnostics.
|
||||
|
||||
The existing shared D&D entity-reconciliation substrate should support all
|
||||
three registries. Entity-specific candidate construction, identity judgment,
|
||||
canonical-name preference, and consolidation remain in the owning D&D module.
|
||||
This feature must not move D&D policy into generic framework packages.
|
||||
|
||||
## Entity Eligibility And Identity
|
||||
|
||||
Eligibility is based on whether the transcript establishes a stable reusable
|
||||
referent, not on capitalization alone. Transcript capitalization is unreliable,
|
||||
and a capitalized generic phrase is not automatically a canonical entity.
|
||||
|
||||
### NPC registry
|
||||
|
||||
Include an NPC when the transcript establishes an individually identifiable
|
||||
non-player character through a proper name or a stable unique designation. A
|
||||
title or descriptive designation qualifies only when the transcript uses it as
|
||||
a persistent identity, such as `the Masked Envoy`; a transient generic role
|
||||
such as `a guard`, `the bartender`, or `one of the bandits` does not qualify.
|
||||
|
||||
Exclude player characters, speakers known only by player identity, anonymous
|
||||
groups, creature categories, generic roles, speculative identities, and labels
|
||||
invented merely to make an occurrence referable.
|
||||
|
||||
NPC IDs use the `npc:sha256:` prefix and identity policy
|
||||
`dnd.npc_registry.identity.v1`. The digest is the lowercase SHA-256 of compact
|
||||
JSON for `[policy, comparison_name]`, where the comparison name uses the
|
||||
existing Unicode, apostrophe, whitespace, and case normalization. Semantic
|
||||
reconciliation may collapse supported aliases but may not merge distinct people
|
||||
merely because their names or roles are similar.
|
||||
|
||||
### Location registry
|
||||
|
||||
Include a location only when the transcript establishes a stable proper name or
|
||||
unique in-world designation, such as `Waterdeep`, `The Yawning Portal`, or a
|
||||
uniquely named chamber. Exclude generic, temporary, relative, or purely
|
||||
descriptive references such as `the room`, `the bar`, `the hallway`, `outside`,
|
||||
or `upstairs`.
|
||||
|
||||
A later occurrence may resolve a generic phrase such as `the bar` to an existing
|
||||
named registry location when the current transcript context supports that
|
||||
coreference. The generic phrase must not create a new registry entry.
|
||||
|
||||
Location IDs use the `location:sha256:` prefix and identity policy
|
||||
`dnd.location_registry.identity.v1`. The digest is the lowercase SHA-256 of
|
||||
compact JSON for `[policy, comparison_name, source_id, start_unit_id,
|
||||
end_unit_id]`, using the earliest canonical evidence anchor. Source anchoring
|
||||
keeps distinct places with the same display name distinguishable. Nested places
|
||||
and same-name places remain distinct unless a validated semantic proposal
|
||||
establishes that candidates identify the same physical place.
|
||||
|
||||
### Item registry
|
||||
|
||||
Items do not need proper names. Include a reusable, transcript-established item
|
||||
name at the narrowest useful type or unique-designation level, including:
|
||||
|
||||
- named unique objects;
|
||||
- concrete item types such as `Healing Potion` when the transcript establishes
|
||||
that type;
|
||||
- stable unique designations used by the session; and
|
||||
- individual currency denominations such as `Silver Pieces`.
|
||||
|
||||
Exclude vague or non-reusable descriptions such as `loot`, `treasure`, `some
|
||||
gear`, `a weapon`, or `something valuable` unless the transcript supplies a
|
||||
more stable name or designation. Do not infer mechanical subtype, magic
|
||||
properties, denomination, quantity, or uniqueness.
|
||||
|
||||
An item-registry record identifies a canonical item kind or designation, not a
|
||||
physical instance. Interchangeable objects with the same canonical name share
|
||||
one registry identity and are distinguished in occurrences by evidence,
|
||||
quantity, and holder transitions. A genuinely named unique object naturally
|
||||
has its own canonical identity.
|
||||
|
||||
Item IDs use the `item:sha256:` prefix and identity policy
|
||||
`dnd.item_registry.identity.v1`. The digest is the lowercase SHA-256 of compact
|
||||
JSON for `[policy, comparison_name]`, using the same text normalization family
|
||||
as other entity registries. Item normalization may reconcile transcript-
|
||||
supported aliases and abbreviation variants but may not collapse distinct
|
||||
denominations or materially different item types.
|
||||
|
||||
## Occurrence Contract
|
||||
|
||||
Every occurrence answers a separate question: what source-grounded event or
|
||||
relationship to a known registry entity occurred in this transcript passage?
|
||||
|
||||
Each occurrence record contains:
|
||||
|
||||
- the exact entity ID from its required registry;
|
||||
- the exact canonical display name associated with that ID;
|
||||
- one bounded occurrence kind; and
|
||||
- one or more source references from the current transcript.
|
||||
|
||||
Entity-specific fields are allowed only when required by that occurrence
|
||||
contract. Registry IDs and names provide grounding; registry source references
|
||||
never become occurrence evidence. Unknown IDs and mismatched ID/name pairs are
|
||||
rejected rather than guessed, reassigned, or repaired from display-name
|
||||
similarity.
|
||||
|
||||
Occurrence extractors may resolve an alias, pronoun, generic reference, or
|
||||
abbreviation to an existing registry entry only when the current chunk supports
|
||||
that coreference. They may not add entities to the registry. Omit an occurrence
|
||||
when entity resolution or the occurrence itself is not adequately supported.
|
||||
|
||||
Occurrence normalization is deterministic. It validates registry membership,
|
||||
canonicalizes the display name from the registry, orders and deduplicates source
|
||||
references, preserves distinct observations, orders records by source-document
|
||||
chronology and stable domain tie-breakers, and collapses only exact duplicates.
|
||||
|
||||
### NPC occurrences
|
||||
|
||||
The durable record contains required `npc_id`, `name`, `kind`, and
|
||||
`source_refs`. It retains the current interaction categories:
|
||||
|
||||
- `mentioned`;
|
||||
- `noncombat_presence`;
|
||||
- `dialogue`;
|
||||
- `combat_ally`;
|
||||
- `combat_opponent`; and
|
||||
- `other`.
|
||||
|
||||
The precedence and splitting rules of the current NPC-interaction contract
|
||||
remain applicable. Renaming the artifact does not turn it into a biography,
|
||||
relationship model, sentiment analysis, or persistent NPC state.
|
||||
|
||||
Enemy-event grounding consumes `combat_opponent` NPC occurrences under the new
|
||||
contract. Other NPC-grounded lanes continue to consume the registry directly.
|
||||
|
||||
### Location occurrences
|
||||
|
||||
The durable record continues to contain required `location_id`, `name`, `kind`,
|
||||
and `source_refs`. It retains the current categories and precedence:
|
||||
|
||||
- `visited`;
|
||||
- `planned`;
|
||||
- `recalled`; and
|
||||
- `mentioned`.
|
||||
|
||||
Generic current-chunk language may resolve to a named registry location, but
|
||||
the occurrence must cite the current transcript passage that supports both the
|
||||
resolution and the occurrence category.
|
||||
|
||||
### Item occurrences
|
||||
|
||||
The durable record contains required `item_id`, `name`, `kind`, and
|
||||
`source_refs`, plus the current semantically conditional `quantity`, `from`,
|
||||
and `to` fields. It retains the current categories:
|
||||
|
||||
- `discovered`;
|
||||
- `acquired`;
|
||||
- `lost`;
|
||||
- `consumed`; and
|
||||
- `transferred`.
|
||||
|
||||
Holder and quantity rules remain unchanged: discovery has no holder; acquisition
|
||||
requires a gaining holder; loss and consumption require a losing holder; and
|
||||
transfer requires distinct losing and gaining holders. Currency remains an
|
||||
ordinary item occurrence with an explicit denomination and a quantity only
|
||||
when the transcript establishes one.
|
||||
|
||||
Item occurrences do not compute inventory, merge successive state changes,
|
||||
convert denominations, infer unspoken holders, or distinguish physical
|
||||
instances that the transcript does not identify.
|
||||
|
||||
Private LLM response schemas should remain strict and compatible with providers
|
||||
that require every declared property to be required. Nullable or explicit
|
||||
absence representations may be used privately for semantically conditional
|
||||
item fields; deterministic mapping owns omission from the durable artifact when
|
||||
the public contract calls for an absent field.
|
||||
|
||||
## Pipeline End State
|
||||
|
||||
A complete D&D pipeline produces all three registries in an earlier step and
|
||||
binds them explicitly into later consumers. Conceptually:
|
||||
|
||||
```text
|
||||
registry step
|
||||
npc-registry
|
||||
location-registry
|
||||
item-registry
|
||||
scene-descriptions
|
||||
|
||||
occurrence and action step
|
||||
npc-occurrences <- npc-registry
|
||||
location-occurrences <- location-registry
|
||||
item-occurrences <- item-registry
|
||||
spells <- optional npc-registry
|
||||
combat-turns <- optional npc-registry + required scene descriptions
|
||||
|
||||
derived enemy step
|
||||
enemy-events <- npc-registry + npc-occurrences + combat turns + scene descriptions
|
||||
```
|
||||
|
||||
Steps remain barriers with the existing whole-run failure behavior. A required
|
||||
registry producer that fails, rejects, or does not publish an accepted
|
||||
normalized artifact prevents its dependent later step and fails the run under
|
||||
the existing ordered-pipeline contract. Configuration with a same-step or
|
||||
forward generated reference remains invalid.
|
||||
|
||||
The maintained complete D&D example should use generated references for these
|
||||
same-run handoffs. Registry slots continue to accept compatible external path
|
||||
references under the existing configuration contract; this feature does not
|
||||
remove the ability to supply a previously produced or independently maintained
|
||||
registry artifact.
|
||||
|
||||
The shared scene plan remains pipeline-wide. This feature changes eligible
|
||||
lanes and reference bindings, not chunking or scheduling semantics.
|
||||
|
||||
## Validation And Normalization
|
||||
|
||||
Each registry receives production defaults for:
|
||||
|
||||
- JSON and durable-schema validity;
|
||||
- registry shape and required evidence;
|
||||
- deterministic identity and uniqueness;
|
||||
- source-reference validity;
|
||||
- source relatedness; and
|
||||
- safe bounded reconciliation during normalization.
|
||||
|
||||
Each occurrence family receives production defaults for:
|
||||
|
||||
- JSON and durable-schema validity;
|
||||
- occurrence shape and kind-specific invariants;
|
||||
- exact registry ID/name membership;
|
||||
- source-reference validity;
|
||||
- source relatedness; and
|
||||
- deterministic ordering and exact-duplicate removal.
|
||||
|
||||
Validator and capability keys should adopt the new module vocabulary. Registry
|
||||
membership validators consume the same immutable registry projection used by
|
||||
the extractor and normalizer. Validators do not use registry provenance as a
|
||||
substitute for occurrence evidence and do not mutate supplied artifacts or
|
||||
references.
|
||||
|
||||
Warnings remain bounded, content-safe, and deterministic. Prompt, schema,
|
||||
registry, and identity fingerprints remain represented by hashes or stable
|
||||
identities rather than raw content in manifests and diagnostics.
|
||||
|
||||
## Package And Asset Organization
|
||||
|
||||
Implementation packages should make the two roles visible, using Go-compatible
|
||||
names such as `npcregistry`, `npcoccurrences`, `locationregistry`, and
|
||||
`itemoccurrences` beneath the existing extract, normalize, codec, and validate
|
||||
families. Entity-specific identity and immutable registry-resolution helpers
|
||||
remain inside the D&D domain.
|
||||
|
||||
LLM assets should use corresponding subtrees beneath `assets/dnd/`, for example
|
||||
`assets/dnd/npc-registry/{extract,normalize}` and
|
||||
`assets/dnd/item-occurrences`. The root `assets` package remains a single-file,
|
||||
content-only filesystem boundary with no business logic, registration, or
|
||||
PromptKit dependency.
|
||||
|
||||
All extraction prompts retain the shared D&D prompt prefix and chunk scope.
|
||||
Registry prompts own only entity eligibility and canonical-name policy;
|
||||
occurrence prompts own only occurrence semantics and entity-specific fields.
|
||||
Registry input fragments own the meaning of their source-free projections.
|
||||
|
||||
## Migration Policy
|
||||
|
||||
The redesign is intentionally breaking and immediate:
|
||||
|
||||
- remove retired module and validator keys;
|
||||
- remove retired artifact kinds, schema IDs, schema names, Go types, prompt IDs,
|
||||
private schema identities, reference slots, capability names, asset paths,
|
||||
and generated-reference aliases;
|
||||
- do not register aliases or decode old shapes;
|
||||
- do not retain duplicate integration documents for old contracts; and
|
||||
- accept invalidation of old checkpoints, prompt fingerprints, provider caches,
|
||||
example output, and other reconstructible pre-release state.
|
||||
|
||||
New contracts use version `v1`; `v2` does not communicate useful compatibility
|
||||
information while there is no supported release to preserve. Current-behavior
|
||||
documentation, maintained examples, module catalogs, and integration contracts
|
||||
must change atomically with implementation.
|
||||
|
||||
## Evaluation Expectations
|
||||
|
||||
Implementation should be assessed against representative transcript cases that
|
||||
cover:
|
||||
|
||||
- named NPCs, stable NPC designations, transient roles, aliases, and anonymous
|
||||
groups;
|
||||
- named locations, same-name locations, nested locations, generic spatial
|
||||
phrases, and supported generic coreferences to named places;
|
||||
- named items, ordinary item types, ambiguous descriptions, aliases, quantities,
|
||||
transfers, consumption, and each currency denomination;
|
||||
- registry candidates mentioned across multiple chunks;
|
||||
- unsupported occurrence links and mismatched registry ID/name pairs; and
|
||||
- output from cost-effective smaller models as well as the default production
|
||||
profile.
|
||||
|
||||
Deterministic tests should protect schemas, identity rules, registry resolution,
|
||||
reference compatibility, ordering, validation, normalization, and assembled
|
||||
multi-step configuration. Live model quality remains an opt-in human evaluation
|
||||
activity rather than a default automated gate.
|
||||
|
||||
## Target End State
|
||||
|
||||
The feature is complete when:
|
||||
|
||||
- the six canonical entity modules are the only selectable NPC, location, and
|
||||
item registry/occurrence modules;
|
||||
- all six durable artifacts use the target kinds, schema identities, root
|
||||
collections, minimal record shapes, and `v1` contracts defined here;
|
||||
- every occurrence contains and validates an exact registry ID/name pair;
|
||||
- NPC, location, and item registry eligibility follows the policies above;
|
||||
- the maintained complete pipeline uses explicit generated registry references
|
||||
across ordered steps, while compatible external registry references remain
|
||||
supported;
|
||||
- item occurrences no longer run independently of an accepted item registry;
|
||||
- spell, combat, enemy, and occurrence consumers use the renamed registry slots
|
||||
without treating registry provenance as evidence;
|
||||
- current module catalogs, default validator chains, prompts, assets, examples,
|
||||
integration contracts, internal documentation, and tests use only the new
|
||||
vocabulary; and
|
||||
- repository searches find no production compatibility aliases or stale current-
|
||||
behavior claims for `dnd/npcs`, `dnd/npc-interactions`, `dnd/locations`, or
|
||||
`dnd/item-events`.
|
||||
@@ -22,14 +22,6 @@ not as committed release dates.
|
||||
spell, combat, interaction, and scene-description lanes after real-world use.
|
||||
Add more complex chunking only in response to demonstrated failures.
|
||||
|
||||
### Canonical Entity Registries And Occurrences
|
||||
|
||||
This work has been promoted to the dedicated
|
||||
[Canonical D&D Entity Registries And Occurrences](entity-registries.md)
|
||||
roadmap. That document owns the proposed module and artifact vocabulary,
|
||||
registry eligibility, identity policies, occurrence contracts, ordered
|
||||
handoffs, migration policy, and target end state.
|
||||
|
||||
## Shared Normalization And Quality Work
|
||||
|
||||
### Generic LLM-Assisted Deduplication
|
||||
|
||||
@@ -1,384 +0,0 @@
|
||||
# Canonical D&D Entity Contracts Remediation Plan
|
||||
|
||||
## Objective
|
||||
|
||||
Address the remaining correctness and maintainability findings discovered after
|
||||
implementing [Canonical D&D Entity Registries And Occurrences](entity-registries.md).
|
||||
The work must preserve the completed six-lane registry/occurrence architecture
|
||||
while fixing deterministic item identity consolidation, hardening currency
|
||||
reconciliation, making item-occurrence grounding fail closed, removing an
|
||||
unnecessary NPC-identity dependency from item behavior, and aligning NPC
|
||||
registry eligibility with the downstream `mentioned` occurrence contract.
|
||||
|
||||
Complete Stages 15 through 18 in numerical order. Stages 1 through 14 are
|
||||
already complete and are summarized below rather than retained as historical
|
||||
implementation instructions. Each remaining stage is intentionally bounded for
|
||||
one gpt-5.6-terra implementation prompt. Do not combine stages unless a later
|
||||
review explicitly revises this plan.
|
||||
|
||||
## Completed Work Summary: Stages 1 Through 14
|
||||
|
||||
The completed stages established the target state in the feature roadmap:
|
||||
|
||||
- NPCs, locations, and items now each have a canonical registry lane and a
|
||||
registry-backed occurrence lane under consistent `dnd/<entity>-registry` and
|
||||
`dnd/<entity>-occurrences` module keys.
|
||||
- Durable artifact types, kinds, `v1` schemas, codecs, evidence projections,
|
||||
mergers, validators, prompts, assets, capabilities, and package namespaces
|
||||
use registry/occurrence vocabulary without compatibility aliases.
|
||||
- Registry records contain deterministic IDs, canonical names, and transcript
|
||||
evidence. Occurrence records carry exact registry ID/name pairs and their own
|
||||
current-transcript evidence.
|
||||
- The complete D&D example produces all three registries in an earlier step and
|
||||
supplies them through explicit generated references to later occurrence and
|
||||
action lanes. Enemy events consume the NPC registry and combat-opponent NPC
|
||||
occurrences in a later derived step.
|
||||
- Production registration, default validator chains, integration contracts,
|
||||
current-behavior documentation, maintained examples, and representative
|
||||
ordered-pipeline tests were migrated to the new contracts.
|
||||
- A final repository audit confirmed that the retired pre-release module,
|
||||
artifact, schema, prompt, reference-slot, and package names no longer remain
|
||||
as production compatibility paths.
|
||||
|
||||
These completed contracts are the implementation baseline. The remaining
|
||||
stages are corrective work, not a second namespace or schema migration.
|
||||
|
||||
## Baseline Contract Decisions
|
||||
|
||||
The implementation agent must preserve these decisions throughout the four
|
||||
remaining stages:
|
||||
|
||||
- Item identity is name-based: `item:sha256:` plus the lowercase SHA-256 of
|
||||
compact JSON `["dnd.item_registry.identity.v1", comparison_name]`.
|
||||
Consequently, two item-registry records with the same comparison name cannot
|
||||
remain separate merely because their evidence differs.
|
||||
- NPC identity is likewise name-based; location identity additionally includes
|
||||
its earliest evidence anchor. Do not change any durable ID algorithm in this
|
||||
work set.
|
||||
- Exact comparison-name duplicates are resolved deterministically before any
|
||||
semantic LLM reconciliation. The model is used only for supported alias or
|
||||
semantic-equivalence proposals that deterministic identity cannot decide.
|
||||
- Currency denominations are separate item identities. Singular, plural, and
|
||||
standard abbreviations for the same denomination may be reconciled, but a
|
||||
currency name may never be consolidated with another denomination or with a
|
||||
non-currency item.
|
||||
- Every occurrence must contain an exact registry ID/name pair. An extractor
|
||||
response with an unknown ID or mismatched name is invalid output and must
|
||||
cause the extraction attempt to fail; silently dropping a record is not an
|
||||
allowed repair.
|
||||
- A named or stably designated NPC may enter the NPC registry when established
|
||||
only through a supported third-party mention. Hypothetical, speculative,
|
||||
generic, or unestablished identities remain ineligible.
|
||||
- Registry references provide source-free identity grounding and never become
|
||||
occurrence evidence. No stage may weaken this boundary.
|
||||
|
||||
## Rules For Every Remaining Stage
|
||||
|
||||
- Read and follow `docs/policy/architecture.md`,
|
||||
`docs/policy/documentation.md`, and `docs/policy/testing.md` before editing.
|
||||
- Preserve the fixed pipeline, typed artifact, generated-reference, scheduler,
|
||||
validation, checkpoint, output, and whole-run failure boundaries.
|
||||
- Keep D&D policy in `internal/modules/dnd` and `assets/dnd`. Shared text policy
|
||||
belongs in a D&D shared helper, not a generic framework package.
|
||||
- Keep `assets/assets.go` as the root assets package's only Go file and keep the
|
||||
root package free of business logic and PromptKit dependencies.
|
||||
- Preserve strict private structured-response schemas: reject unknown fields
|
||||
and list every declared property in `required`.
|
||||
- Keep prompt-cache ordering and the byte-identical shared D&D prefix intact.
|
||||
- Update the canonical current-behavior integration or internal documentation
|
||||
in the same stage as any implemented contract or failure-semantics change.
|
||||
Link to canonical owners instead of duplicating volatile facts.
|
||||
- Keep tests deterministic, offline, and credential-free. Add regression tests
|
||||
at the narrowest stable behavior boundary. Do not add live-LLM tests, exact
|
||||
prompt snapshots, prompt-prefix length detectors, private asset inventories,
|
||||
or tests that merely freeze helper placement.
|
||||
- Preserve unrelated user changes. Run `gofmt` on changed Go files and
|
||||
`git diff --check` in every stage. Do not retire this plan or the feature
|
||||
roadmap during implementation.
|
||||
|
||||
## Stage 15: Consolidate Deterministically Identical Item Registry Records
|
||||
|
||||
### Goal
|
||||
|
||||
Make item-registry normalization produce exactly one record for every item
|
||||
comparison name, regardless of whether duplicate candidates cite identical or
|
||||
different transcript ranges.
|
||||
|
||||
### Required changes
|
||||
|
||||
1. Revise deterministic preprocessing in
|
||||
`internal/modules/dnd/normalize/itemregistry` so grouping is keyed by the
|
||||
item identity comparison name alone. Remove source-reference equality as a
|
||||
condition for grouping records whose deterministic IDs are necessarily
|
||||
identical.
|
||||
2. For each comparison-name group:
|
||||
- retain the earliest input record as the deterministic display-name and
|
||||
ordering owner;
|
||||
- union source references from every member;
|
||||
- canonicalize and deduplicate the union with the existing
|
||||
`shared.SourceRefOrder` behavior;
|
||||
- retain the sorted unique set of original input indexes for diagnostics;
|
||||
and
|
||||
- derive the final ID from the retained canonical display name through the
|
||||
existing item identity package.
|
||||
3. Keep this consolidation entirely deterministic. Do not send equal-key
|
||||
candidates to the LLM reconciliation prompt, and do not change semantic
|
||||
alias reconciliation for records with different comparison names.
|
||||
4. Preserve stable output order, warning bounds, content-safe diagnostics,
|
||||
nil/empty collection conventions, and input non-mutation. Remove imports or
|
||||
helpers made obsolete by dropping source-reference equality.
|
||||
5. Replace the existing test expectation that equal-name items with different
|
||||
evidence survive as two records. Add focused behavior coverage proving that
|
||||
such records become one valid item with unioned canonical evidence, one
|
||||
deterministic ID, earliest-display-name retention, stable placement, and no
|
||||
input mutation.
|
||||
6. Include a normalizer/default-chain regression case showing that normalized
|
||||
output with cross-chunk equal-name candidates passes item-registry identity
|
||||
validation rather than producing duplicate canonical-name or duplicate-ID
|
||||
rejection.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- Item normalization cannot emit two records with the same item comparison
|
||||
name or deterministic ID solely because their evidence differs.
|
||||
- The retained record contains the canonical union of every duplicate member's
|
||||
evidence and remains stable across repeated normalization.
|
||||
- Different comparison names remain eligible for the existing bounded semantic
|
||||
reconciliation path.
|
||||
- No durable schema, item ID algorithm, module key, or artifact kind changes.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/modules/dnd/normalize/itemregistry ./internal/modules/dnd/items/identity ./internal/modules/dnd/validate/itemregistry/identity ./internal/modules/dnd/register
|
||||
go test ./internal/modules/dnd/...
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 16: Make Currency Reconciliation Fail Closed
|
||||
|
||||
### Goal
|
||||
|
||||
Prevent an LLM reconciliation proposal from combining currency with a
|
||||
non-currency item or combining distinct currency denominations.
|
||||
|
||||
### Required changes
|
||||
|
||||
1. Refactor the item-registry semantic-consolidation guard in
|
||||
`internal/modules/dnd/normalize/itemregistry/reconciliation.go` to classify
|
||||
every proposed member before applying a group. Preserve the existing five
|
||||
supported D&D denominations and their singular, plural, and standard
|
||||
abbreviation aliases.
|
||||
2. Apply this exact decision table:
|
||||
|
||||
| Proposed group contents | Result |
|
||||
| --- | --- |
|
||||
| no recognized currency members | allow the existing item reconciliation policy to decide/apply the already validated proposal |
|
||||
| only recognized currency members of one denomination | allow consolidation |
|
||||
| recognized currency members of different denominations | reject the proposal group |
|
||||
| one or more recognized currency members plus any non-currency member | reject the proposal group |
|
||||
|
||||
An unrecognized name must not be treated as the same denomination merely
|
||||
because another member is recognized currency.
|
||||
3. On rejection, preserve every deterministic input record exactly as the safe
|
||||
fallback, keep its canonicalized evidence, count the proposal as rejected
|
||||
for the existing retry/fallback behavior, and emit one bounded content-safe
|
||||
warning for the group. Do not partially consolidate a rejected group.
|
||||
4. Generalize the current warning text so it accurately covers both mixed
|
||||
currency/non-currency groups and conflicting denominations without exposing
|
||||
transcript content unnecessarily.
|
||||
5. Add table-driven regression coverage for same-denomination aliases,
|
||||
different denominations, currency plus an ordinary item, multiple ordinary
|
||||
items, and a mixed group whose canonical member is the non-currency item.
|
||||
Test observable normalized results and warnings rather than the private
|
||||
helper's implementation.
|
||||
6. Preserve proposal validation, retry counts, safe fallback, warning limits,
|
||||
deterministic ordering, and non-mutation outside this additional safety
|
||||
rule.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- `Gold Pieces` cannot be consolidated with `Longsword`, regardless of which
|
||||
member the model selects as canonical.
|
||||
- Currency aliases such as `GP`, `Gold Piece`, and `Gold Pieces` may still
|
||||
resolve to one gold-denomination item when proposed safely.
|
||||
- Different denominations always remain distinct.
|
||||
- Invalid groups preserve the deterministic candidates and follow the current
|
||||
retry/fallback contract without a partial merge.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/modules/dnd/normalize/itemregistry
|
||||
go test ./internal/modules/dnd/validate/itemregistry/... ./internal/modules/dnd/register
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 17: Enforce Item Occurrence Grounding And Own Shared Text Comparison
|
||||
|
||||
### Goal
|
||||
|
||||
Make item-occurrence extraction reject invalid registry pairs instead of
|
||||
silently losing events, and remove its accidental semantic dependency on the
|
||||
NPC identity package while preserving comparison behavior exactly.
|
||||
|
||||
### Required changes
|
||||
|
||||
1. Change response mapping in
|
||||
`internal/modules/dnd/extract/itemoccurrences/canonicalize.go` so an unknown
|
||||
`item_id` or a `name` that does not exactly match that ID's canonical
|
||||
registry name returns a bounded, content-safe extraction error identifying
|
||||
the occurrence index and violated field. Do not `continue`, omit the bad
|
||||
record, repair it by name, or return a partial artifact.
|
||||
2. Ensure the mapping error propagates through the existing extractor boundary
|
||||
as a failed extraction attempt so the configured pipeline retry policy may
|
||||
obtain a corrected structured response. Do not add a module-local retry
|
||||
loop or convert this failure into a validator rejection after accepting
|
||||
partial output.
|
||||
3. Add extraction regression cases for an unknown ID, a mismatched name, and a
|
||||
response containing a valid occurrence followed by an invalid one. All must
|
||||
return an error and no accepted partial result. Mirror the established NPC
|
||||
occurrence extractor behavior where practical without introducing a generic
|
||||
framework abstraction.
|
||||
4. Introduce one D&D-owned shared text comparison primitive under
|
||||
`internal/modules/dnd/shared` implementing the current byte-for-byte
|
||||
semantics: Unicode NFKC normalization, curly/modifier apostrophe mapping,
|
||||
whitespace collapse, and Unicode case folding. Give the semantic policy a
|
||||
stable explicit identity such as `dnd.text_comparison.v1`.
|
||||
5. Remove the `shared` package's existing import of `npcs/identity` by making
|
||||
shared token matching use the new primitive. Make NPC, location, and item
|
||||
identity `ComparisonKey` functions delegate to the shared primitive while
|
||||
retaining their public package functions and all existing entity-specific
|
||||
ID policy constants and digest algorithms.
|
||||
6. Change `internal/modules/dnd/itemoccurrences` to use the shared D&D
|
||||
comparison primitive rather than `npcs/identity`. Do not change display
|
||||
trimming, party-holder rules, exact-duplicate identity, or ordering.
|
||||
7. Add the shared comparison policy identity to item-occurrence extractor and
|
||||
normalizer manifest/checkpoint semantics wherever comparison behavior can
|
||||
affect mapped or normalized output. Keep values content-safe. Document in
|
||||
the nearest D&D internal owner that a semantic change to shared comparison
|
||||
requires an explicit policy-version review for every affected identity,
|
||||
mapping, normalization, and validator policy; do not calculate a brittle
|
||||
fingerprint from private helper source.
|
||||
8. Consolidate duplicated comparison test cases into focused shared-helper
|
||||
behavior coverage, while retaining entity identity tests that protect each
|
||||
durable ID contract. Prove existing representative NPC, location, item, and
|
||||
item-occurrence outputs are unchanged by the ownership refactor.
|
||||
9. Update `docs/integrations/dnd-item-occurrence-artifacts.md` if it currently
|
||||
says extraction omits unknown or mismatched pairs. The canonical contract
|
||||
must say the response is rejected as invalid model output and may be retried;
|
||||
normalization and validation remain defense-in-depth for artifacts entering
|
||||
through other boundaries.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- Item-occurrence extraction never succeeds with a silently shortened artifact
|
||||
after receiving an invalid registry pair.
|
||||
- The framework's existing retry behavior receives the mapping error without a
|
||||
new hidden retry mechanism.
|
||||
- No item-occurrence or shared D&D package imports NPC identity solely for text
|
||||
comparison.
|
||||
- All existing comparison keys and durable entity IDs remain unchanged for the
|
||||
same inputs.
|
||||
- Item-occurrence checkpoint reuse is invalidated when its declared comparison
|
||||
policy identity changes.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/modules/dnd/shared ./internal/modules/dnd/npcs/identity ./internal/modules/dnd/locations/identity ./internal/modules/dnd/items/identity
|
||||
go test ./internal/modules/dnd/itemoccurrences ./internal/modules/dnd/extract/itemoccurrences ./internal/modules/dnd/normalize/itemoccurrences ./internal/modules/dnd/validate/itemoccurrences/...
|
||||
go test ./internal/modules/dnd/...
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 18: Align NPC Mention Eligibility And Finish Vocabulary Cleanup
|
||||
|
||||
### Goal
|
||||
|
||||
Allow the NPC registry to establish named NPCs that appear only in supported
|
||||
third-party mentions, and remove the remaining occurrence/interactions naming
|
||||
residue without broadening the registry to speculative or generic identities.
|
||||
|
||||
### Required changes
|
||||
|
||||
1. Revise `assets/dnd/npc-registry/extract/prompts/instructions.md` so a proper
|
||||
name or stable unique NPC designation is eligible when the transcript
|
||||
establishes it through a factual third-party mention, even if the NPC is not
|
||||
physically present and does not speak or enter combat in that chunk.
|
||||
2. Keep the exclusion boundary explicit: omit hypothetical or speculative
|
||||
people, names used only in an imagined example, generic roles, anonymous
|
||||
groups, player characters, and labels invented by the model. Do not require
|
||||
dialogue, physical presence, or direct action as evidence of identity.
|
||||
3. Preserve separation of concerns. The registry prompt must extract only NPC
|
||||
identity and identity evidence; it must not classify an occurrence as
|
||||
`mentioned` or copy occurrence semantics into the registry artifact. The
|
||||
later NPC occurrence lane remains responsible for emitting the `mentioned`
|
||||
fact with independently cited current-transcript evidence.
|
||||
4. Review the shared D&D identity/evidence instructions and remove or revise
|
||||
any contradictory wording using the canonical asset owner. Do not duplicate
|
||||
the new rule across module-specific and shared assets merely for emphasis,
|
||||
and do not disturb shared prompt ordering or cacheable prefix bytes unless a
|
||||
genuinely shared contradiction must be corrected.
|
||||
5. Update the canonical NPC registry and NPC occurrence integration documents,
|
||||
and `docs/internal/dnd.md` only where needed, so they consistently distinguish
|
||||
a transcript-established mentioned-only identity from an occurrence. Link
|
||||
between canonical owners instead of copying category lists or schemas.
|
||||
6. Correct the duplicated phrase `NPC occurrence occurrences` in the NPC
|
||||
occurrence prompt. Rename stale local aliases such as `interactionmodel` in
|
||||
NPC occurrence code and stale `interactionContent` test variables to
|
||||
occurrence vocabulary. Correct similarly scoped registration labels that
|
||||
inaccurately say `locations` when they describe another entity. Do not make
|
||||
unrelated wording changes outside the entity modules.
|
||||
7. Run existing prompt preparation and module behavior tests. Add a focused
|
||||
deterministic regression test only where there is a stable behavioral seam;
|
||||
do not snapshot the prose, assert an exact prompt fragment, or add a prompt
|
||||
change detector. Use optional human prompt evaluation to assess model
|
||||
interpretation, not as an automated acceptance gate.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- The NPC registry prompt permits a factual named third-party mention to
|
||||
establish an NPC while continuing to reject hypothetical and generic names.
|
||||
- The registry output remains the minimal `{id, name, source_refs}` contract;
|
||||
mention classification remains in NPC occurrences.
|
||||
- Current integration and internal documentation describe the same boundary
|
||||
without duplicating volatile contract detail.
|
||||
- NPC occurrence production code, prompt prose, and touched tests contain no
|
||||
stale interaction-era local vocabulary.
|
||||
- Prompt manifests still prepare successfully with the established shared
|
||||
prefix and final module-instruction ordering.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/modules/dnd/extract/npcregistry ./internal/modules/dnd/extract/npcoccurrences ./internal/modules/dnd/normalize/npcoccurrences ./internal/modules/dnd/validate/npcoccurrences/... ./internal/modules/dnd/register
|
||||
go test ./internal/modules/dnd/... ./internal/modules/integration/...
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Final Verification
|
||||
|
||||
After Stage 18, run the repository-wide checks and validate both maintained D&D
|
||||
configurations:
|
||||
|
||||
```sh
|
||||
go test -count=1 ./...
|
||||
go vet ./...
|
||||
go build ./cmd/notarius
|
||||
go run ./cmd/notarius config validate --config examples/dnd-minimal.config.yml --pipeline dnd-session
|
||||
OPENROUTER_API_KEY=validation-placeholder go run ./cmd/notarius config validate --config examples/dnd-complete.config.yml --pipeline dnd-session
|
||||
git diff --check
|
||||
```
|
||||
|
||||
The implementation is complete only when all commands pass and a final review
|
||||
confirms that the feature roadmap's durable schemas, module names, evidence
|
||||
boundaries, and ordered registry handoffs remain unchanged except for the
|
||||
explicitly corrected failure and eligibility behavior above.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None. The stages above adopt the long-term-maintainable choices from the audit:
|
||||
deterministic identity consolidation, fail-closed semantic safeguards, strict
|
||||
extractor grounding, a D&D-owned shared comparison policy, and support for
|
||||
factual mentioned-only NPC identities.
|
||||
@@ -285,7 +285,7 @@ func registerDefaultChains(registry *pipeline.ValidatorChainRegistry) error {
|
||||
},
|
||||
})
|
||||
}},
|
||||
{name: "locations validator chain", register: func() error {
|
||||
{name: "location registry validator chain", register: func() error {
|
||||
return registry.Register(pipeline.ValidatorChainMapping{
|
||||
Stage: pipeline.StageExtract, Module: locationextract.Key,
|
||||
Validators: []pipeline.ModuleBinding{
|
||||
@@ -294,7 +294,7 @@ func registerDefaultChains(registry *pipeline.ValidatorChainRegistry) error {
|
||||
},
|
||||
})
|
||||
}},
|
||||
{name: "locations normalize validator chain", register: func() error {
|
||||
{name: "location registry normalize validator chain", register: func() error {
|
||||
return registry.Register(pipeline.ValidatorChainMapping{
|
||||
Stage: pipeline.StageNormalize, Module: locationnormalize.Key,
|
||||
Validators: []pipeline.ModuleBinding{
|
||||
|
||||
@@ -159,7 +159,7 @@ func registerPromptAssets(assets *llm.AssetRegistry) error {
|
||||
{name: "item registry normalization prompt assets", register: func() error { return itemregistrynormalize.RegisterPromptAssets(assets) }},
|
||||
{name: "npc occurrences prompt assets", register: func() error { return occurrenceextract.RegisterPromptAssets(assets) }},
|
||||
{name: "scene descriptions prompt assets", register: func() error { return scenedescriptionextract.RegisterPromptAssets(assets) }},
|
||||
{name: "locations prompt assets", register: func() error { return locationextract.RegisterPromptAssets(assets) }},
|
||||
{name: "location registry prompt assets", register: func() error { return locationextract.RegisterPromptAssets(assets) }},
|
||||
{name: "location normalization prompt assets", register: func() error { return locationnormalize.RegisterPromptAssets(assets) }},
|
||||
{name: "location occurrences prompt assets", register: func() error { return locationoccurrenceextract.RegisterPromptAssets(assets) }},
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user