notarius/docs/roadmap/implementation.md

Canonical D&D Entity Contracts Remediation Plan

Objective

Address the remaining correctness and maintainability findings discovered after implementing Canonical D&D Entity Registries And Occurrences. The work must preserve the completed six-lane registry/occurrence architecture while fixing deterministic item identity consolidation, hardening currency reconciliation, making item-occurrence grounding fail closed, removing an unnecessary NPC-identity dependency from item behavior, and aligning NPC registry eligibility with the downstream mentioned occurrence contract.

Complete Stages 15 through 18 in numerical order. Stages 1 through 14 are already complete and are summarized below rather than retained as historical implementation instructions. Each remaining stage is intentionally bounded for one gpt-5.6-terra implementation prompt. Do not combine stages unless a later review explicitly revises this plan.

Completed Work Summary: Stages 1 Through 14

The completed stages established the target state in the feature roadmap:

• NPCs, locations, and items now each have a canonical registry lane and a registry-backed occurrence lane under consistent dnd/<entity>-registry and dnd/<entity>-occurrences module keys.
• Durable artifact types, kinds, v1 schemas, codecs, evidence projections, mergers, validators, prompts, assets, capabilities, and package namespaces use registry/occurrence vocabulary without compatibility aliases.
• Registry records contain deterministic IDs, canonical names, and transcript evidence. Occurrence records carry exact registry ID/name pairs and their own current-transcript evidence.
• The complete D&D example produces all three registries in an earlier step and supplies them through explicit generated references to later occurrence and action lanes. Enemy events consume the NPC registry and combat-opponent NPC occurrences in a later derived step.
• Production registration, default validator chains, integration contracts, current-behavior documentation, maintained examples, and representative ordered-pipeline tests were migrated to the new contracts.
• A final repository audit confirmed that the retired pre-release module, artifact, schema, prompt, reference-slot, and package names no longer remain as production compatibility paths.

These completed contracts are the implementation baseline. The remaining stages are corrective work, not a second namespace or schema migration.

Baseline Contract Decisions

The implementation agent must preserve these decisions throughout the four remaining stages:

• Item identity is name-based: item:sha256: plus the lowercase SHA-256 of compact JSON ["dnd.item_registry.identity.v1", comparison_name]. Consequently, two item-registry records with the same comparison name cannot remain separate merely because their evidence differs.
• NPC identity is likewise name-based; location identity additionally includes its earliest evidence anchor. Do not change any durable ID algorithm in this work set.
• Exact comparison-name duplicates are resolved deterministically before any semantic LLM reconciliation. The model is used only for supported alias or semantic-equivalence proposals that deterministic identity cannot decide.
• Currency denominations are separate item identities. Singular, plural, and standard abbreviations for the same denomination may be reconciled, but a currency name may never be consolidated with another denomination or with a non-currency item.
• Every occurrence must contain an exact registry ID/name pair. An extractor response with an unknown ID or mismatched name is invalid output and must cause the extraction attempt to fail; silently dropping a record is not an allowed repair.
• A named or stably designated NPC may enter the NPC registry when established only through a supported third-party mention. Hypothetical, speculative, generic, or unestablished identities remain ineligible.
• Registry references provide source-free identity grounding and never become occurrence evidence. No stage may weaken this boundary.

Rules For Every Remaining Stage

• Read and follow docs/policy/architecture.md, docs/policy/documentation.md, and docs/policy/testing.md before editing.
• Preserve the fixed pipeline, typed artifact, generated-reference, scheduler, validation, checkpoint, output, and whole-run failure boundaries.
• Keep D&D policy in internal/modules/dnd and assets/dnd. Shared text policy belongs in a D&D shared helper, not a generic framework package.
• Keep assets/assets.go as the root assets package's only Go file and keep the root package free of business logic and PromptKit dependencies.
• Preserve strict private structured-response schemas: reject unknown fields and list every declared property in required.
• Keep prompt-cache ordering and the byte-identical shared D&D prefix intact.
• Update the canonical current-behavior integration or internal documentation in the same stage as any implemented contract or failure-semantics change. Link to canonical owners instead of duplicating volatile facts.
• Keep tests deterministic, offline, and credential-free. Add regression tests at the narrowest stable behavior boundary. Do not add live-LLM tests, exact prompt snapshots, prompt-prefix length detectors, private asset inventories, or tests that merely freeze helper placement.
• Preserve unrelated user changes. Run gofmt on changed Go files and git diff --check in every stage. Do not retire this plan or the feature roadmap during implementation.

Stage 15: Consolidate Deterministically Identical Item Registry Records

Goal

Make item-registry normalization produce exactly one record for every item comparison name, regardless of whether duplicate candidates cite identical or different transcript ranges.

Required changes

1. Revise deterministic preprocessing in internal/modules/dnd/normalize/itemregistry so grouping is keyed by the item identity comparison name alone. Remove source-reference equality as a condition for grouping records whose deterministic IDs are necessarily identical.
2. For each comparison-name group:
   • retain the earliest input record as the deterministic display-name and ordering owner;
   • union source references from every member;
   • canonicalize and deduplicate the union with the existing shared.SourceRefOrder behavior;
   • retain the sorted unique set of original input indexes for diagnostics; and
   • derive the final ID from the retained canonical display name through the existing item identity package.
3. Keep this consolidation entirely deterministic. Do not send equal-key candidates to the LLM reconciliation prompt, and do not change semantic alias reconciliation for records with different comparison names.
4. Preserve stable output order, warning bounds, content-safe diagnostics, nil/empty collection conventions, and input non-mutation. Remove imports or helpers made obsolete by dropping source-reference equality.
5. Replace the existing test expectation that equal-name items with different evidence survive as two records. Add focused behavior coverage proving that such records become one valid item with unioned canonical evidence, one deterministic ID, earliest-display-name retention, stable placement, and no input mutation.
6. Include a normalizer/default-chain regression case showing that normalized output with cross-chunk equal-name candidates passes item-registry identity validation rather than producing duplicate canonical-name or duplicate-ID rejection.

Acceptance criteria

• Item normalization cannot emit two records with the same item comparison name or deterministic ID solely because their evidence differs.
• The retained record contains the canonical union of every duplicate member's evidence and remains stable across repeated normalization.
• Different comparison names remain eligible for the existing bounded semantic reconciliation path.
• No durable schema, item ID algorithm, module key, or artifact kind changes.

Validation

go test ./internal/modules/dnd/normalize/itemregistry ./internal/modules/dnd/items/identity ./internal/modules/dnd/validate/itemregistry/identity ./internal/modules/dnd/register
go test ./internal/modules/dnd/...
git diff --check

Stage 16: Make Currency Reconciliation Fail Closed

Goal

Prevent an LLM reconciliation proposal from combining currency with a non-currency item or combining distinct currency denominations.

Required changes

1. Refactor the item-registry semantic-consolidation guard in internal/modules/dnd/normalize/itemregistry/reconciliation.go to classify every proposed member before applying a group. Preserve the existing five supported D&D denominations and their singular, plural, and standard abbreviation aliases.

2. Apply this exact decision table:

   Proposed group contents	Result
   no recognized currency members	allow the existing item reconciliation policy to decide/apply the already validated proposal
   only recognized currency members of one denomination	allow consolidation
   recognized currency members of different denominations	reject the proposal group
   one or more recognized currency members plus any non-currency member	reject the proposal group

   An unrecognized name must not be treated as the same denomination merely because another member is recognized currency.

3. On rejection, preserve every deterministic input record exactly as the safe fallback, keep its canonicalized evidence, count the proposal as rejected for the existing retry/fallback behavior, and emit one bounded content-safe warning for the group. Do not partially consolidate a rejected group.

4. Generalize the current warning text so it accurately covers both mixed currency/non-currency groups and conflicting denominations without exposing transcript content unnecessarily.

5. Add table-driven regression coverage for same-denomination aliases, different denominations, currency plus an ordinary item, multiple ordinary items, and a mixed group whose canonical member is the non-currency item. Test observable normalized results and warnings rather than the private helper's implementation.

6. Preserve proposal validation, retry counts, safe fallback, warning limits, deterministic ordering, and non-mutation outside this additional safety rule.

Acceptance criteria

• Gold Pieces cannot be consolidated with Longsword, regardless of which member the model selects as canonical.
• Currency aliases such as GP, Gold Piece, and Gold Pieces may still resolve to one gold-denomination item when proposed safely.
• Different denominations always remain distinct.
• Invalid groups preserve the deterministic candidates and follow the current retry/fallback contract without a partial merge.

Validation

go test ./internal/modules/dnd/normalize/itemregistry
go test ./internal/modules/dnd/validate/itemregistry/... ./internal/modules/dnd/register
git diff --check

Stage 17: Enforce Item Occurrence Grounding And Own Shared Text Comparison

Goal

Make item-occurrence extraction reject invalid registry pairs instead of silently losing events, and remove its accidental semantic dependency on the NPC identity package while preserving comparison behavior exactly.

Required changes

1. Change response mapping in internal/modules/dnd/extract/itemoccurrences/canonicalize.go so an unknown item_id or a name that does not exactly match that ID's canonical registry name returns a bounded, content-safe extraction error identifying the occurrence index and violated field. Do not continue, omit the bad record, repair it by name, or return a partial artifact.
2. Ensure the mapping error propagates through the existing extractor boundary as a failed extraction attempt so the configured pipeline retry policy may obtain a corrected structured response. Do not add a module-local retry loop or convert this failure into a validator rejection after accepting partial output.
3. Add extraction regression cases for an unknown ID, a mismatched name, and a response containing a valid occurrence followed by an invalid one. All must return an error and no accepted partial result. Mirror the established NPC occurrence extractor behavior where practical without introducing a generic framework abstraction.
4. Introduce one D&D-owned shared text comparison primitive under internal/modules/dnd/shared implementing the current byte-for-byte semantics: Unicode NFKC normalization, curly/modifier apostrophe mapping, whitespace collapse, and Unicode case folding. Give the semantic policy a stable explicit identity such as dnd.text_comparison.v1.
5. Remove the shared package's existing import of npcs/identity by making shared token matching use the new primitive. Make NPC, location, and item identity ComparisonKey functions delegate to the shared primitive while retaining their public package functions and all existing entity-specific ID policy constants and digest algorithms.
6. Change internal/modules/dnd/itemoccurrences to use the shared D&D comparison primitive rather than npcs/identity. Do not change display trimming, party-holder rules, exact-duplicate identity, or ordering.
7. Add the shared comparison policy identity to item-occurrence extractor and normalizer manifest/checkpoint semantics wherever comparison behavior can affect mapped or normalized output. Keep values content-safe. Document in the nearest D&D internal owner that a semantic change to shared comparison requires an explicit policy-version review for every affected identity, mapping, normalization, and validator policy; do not calculate a brittle fingerprint from private helper source.
8. Consolidate duplicated comparison test cases into focused shared-helper behavior coverage, while retaining entity identity tests that protect each durable ID contract. Prove existing representative NPC, location, item, and item-occurrence outputs are unchanged by the ownership refactor.
9. Update docs/integrations/dnd-item-occurrence-artifacts.md if it currently says extraction omits unknown or mismatched pairs. The canonical contract must say the response is rejected as invalid model output and may be retried; normalization and validation remain defense-in-depth for artifacts entering through other boundaries.

Acceptance criteria

• Item-occurrence extraction never succeeds with a silently shortened artifact after receiving an invalid registry pair.
• The framework's existing retry behavior receives the mapping error without a new hidden retry mechanism.
• No item-occurrence or shared D&D package imports NPC identity solely for text comparison.
• All existing comparison keys and durable entity IDs remain unchanged for the same inputs.
• Item-occurrence checkpoint reuse is invalidated when its declared comparison policy identity changes.

Validation

go test ./internal/modules/dnd/shared ./internal/modules/dnd/npcs/identity ./internal/modules/dnd/locations/identity ./internal/modules/dnd/items/identity
go test ./internal/modules/dnd/itemoccurrences ./internal/modules/dnd/extract/itemoccurrences ./internal/modules/dnd/normalize/itemoccurrences ./internal/modules/dnd/validate/itemoccurrences/...
go test ./internal/modules/dnd/...
git diff --check

Stage 18: Align NPC Mention Eligibility And Finish Vocabulary Cleanup

Goal

Allow the NPC registry to establish named NPCs that appear only in supported third-party mentions, and remove the remaining occurrence/interactions naming residue without broadening the registry to speculative or generic identities.

Required changes

1. Revise assets/dnd/npc-registry/extract/prompts/instructions.md so a proper name or stable unique NPC designation is eligible when the transcript establishes it through a factual third-party mention, even if the NPC is not physically present and does not speak or enter combat in that chunk.
2. Keep the exclusion boundary explicit: omit hypothetical or speculative people, names used only in an imagined example, generic roles, anonymous groups, player characters, and labels invented by the model. Do not require dialogue, physical presence, or direct action as evidence of identity.
3. Preserve separation of concerns. The registry prompt must extract only NPC identity and identity evidence; it must not classify an occurrence as mentioned or copy occurrence semantics into the registry artifact. The later NPC occurrence lane remains responsible for emitting the mentioned fact with independently cited current-transcript evidence.
4. Review the shared D&D identity/evidence instructions and remove or revise any contradictory wording using the canonical asset owner. Do not duplicate the new rule across module-specific and shared assets merely for emphasis, and do not disturb shared prompt ordering or cacheable prefix bytes unless a genuinely shared contradiction must be corrected.
5. Update the canonical NPC registry and NPC occurrence integration documents, and docs/internal/dnd.md only where needed, so they consistently distinguish a transcript-established mentioned-only identity from an occurrence. Link between canonical owners instead of copying category lists or schemas.
6. Correct the duplicated phrase NPC occurrence occurrences in the NPC occurrence prompt. Rename stale local aliases such as interactionmodel in NPC occurrence code and stale interactionContent test variables to occurrence vocabulary. Correct similarly scoped registration labels that inaccurately say locations when they describe another entity. Do not make unrelated wording changes outside the entity modules.
7. Run existing prompt preparation and module behavior tests. Add a focused deterministic regression test only where there is a stable behavioral seam; do not snapshot the prose, assert an exact prompt fragment, or add a prompt change detector. Use optional human prompt evaluation to assess model interpretation, not as an automated acceptance gate.

Acceptance criteria

• The NPC registry prompt permits a factual named third-party mention to establish an NPC while continuing to reject hypothetical and generic names.
• The registry output remains the minimal {id, name, source_refs} contract; mention classification remains in NPC occurrences.
• Current integration and internal documentation describe the same boundary without duplicating volatile contract detail.
• NPC occurrence production code, prompt prose, and touched tests contain no stale interaction-era local vocabulary.
• Prompt manifests still prepare successfully with the established shared prefix and final module-instruction ordering.

Validation

go test ./internal/modules/dnd/extract/npcregistry ./internal/modules/dnd/extract/npcoccurrences ./internal/modules/dnd/normalize/npcoccurrences ./internal/modules/dnd/validate/npcoccurrences/... ./internal/modules/dnd/register
go test ./internal/modules/dnd/... ./internal/modules/integration/...
git diff --check

Final Verification

After Stage 18, run the repository-wide checks and validate both maintained D&D configurations:

go test -count=1 ./...
go vet ./...
go build ./cmd/notarius
go run ./cmd/notarius config validate --config examples/dnd-minimal.config.yml --pipeline dnd-session
OPENROUTER_API_KEY=validation-placeholder go run ./cmd/notarius config validate --config examples/dnd-complete.config.yml --pipeline dnd-session
git diff --check

The implementation is complete only when all commands pass and a final review confirms that the feature roadmap's durable schemas, module names, evidence boundaries, and ordered registry handoffs remain unchanged except for the explicitly corrected failure and eligibility behavior above.

Open Questions

None. The stages above adopt the long-term-maintainable choices from the audit: deterministic identity consolidation, fail-closed semantic safeguards, strict extractor grounding, a D&D-owned shared comparison policy, and support for factual mentioned-only NPC identities.