notarius/docs/roadmap/implementation.md

D&D Item Events Implementation Plan

Purpose

This document is the decision-complete implementation plan for D&D Item Events. Implement the stages below in order, with one stage per coding-agent prompt. The feature roadmap is authoritative for product intent, artifact semantics, and non-goals; this document defines how to reach that target state in the current architecture.

The work adds one LLM-backed dnd/item-events extractor. It records a source-grounded, ordered history of item and currency events without attempting to maintain inventory state. A deterministic normalizer prepares the extractor output for downstream use.

Cross-Stage Rules

Apply these rules in every stage:

• Follow Architecture Policy, Documentation Policy, and Testing Policy.
• Preserve the durable boundary: private provider response types belong to the extractor package; public item-event artifacts belong to internal/modules/dnd; codecs own strict wire validation.
• Keep LLM extraction and deterministic processing separate. The extractor may map provider fields and attach the current source ID, but it must not repair, infer, merge, or semantically reinterpret events.
• Reuse the existing shared D&D prompt assets and prompt assembly helpers. Do not copy their text into item-specific assets. Stable shared references must precede item-specific instructions and the changing transcript message must remain last.
• Use strict option decoding and reject unknown module options.
• Use policy identities with a v1 suffix because this is the first item-event contract. Do not introduce a v2 schema or backward-compatibility layer; the application is pre-release.
• Keep validation diagnostics bounded and content-safe, and keep capability and identity metadata free of transcript, prompt, reference, and credential content.
• Keep all tests offline and deterministic. Do not add live-provider tests, wall-clock-dependent assertions, exact prompt-prefix-length tests, or tests coupled only to private helper structure.
• Do not make unrelated cleanup changes. Preserve user changes already present in the worktree.
• At the end of every stage, run the narrowest relevant package tests and go test ./.... Run the repository's documented formatting and static-analysis commands before completing the final stage.

Fixed Contract Decisions

The implementation must use the following names and shapes consistently.

Module and artifact identities

• Extractor key: dnd/item-events
• Extractor capability: dnd.item_events
• Durable artifact kind: dnd/item-event-list
• Durable schema ID: notarius.dnd.item_events
• Durable schema name: notarius_dnd_item_events_v1
• Durable schema version: v1
• Private LLM schema ID: notarius.dnd.item_events.llm
• Private LLM schema name: notarius_dnd_item_events_llm_v1
• Private response schema key: dnd_item_events_llm
• Prompt ID: dnd.item_events
• Extraction mapping policy: dnd.item_events.extract_mapping.v1
• Normalization policy: dnd.item_events.normalize.v1

Use the same identity formats, MIME type, metadata keys, and fingerprint construction conventions as the other current D&D extractors.

Durable Go shape

Add public D&D domain types equivalent to:

type ItemEventKind string

const (
    ItemEventKindDiscovered  ItemEventKind = "discovered"
    ItemEventKindAcquired    ItemEventKind = "acquired"
    ItemEventKindLost        ItemEventKind = "lost"
    ItemEventKindConsumed    ItemEventKind = "consumed"
    ItemEventKindTransferred ItemEventKind = "transferred"
)

type ItemEventList struct {
    Events []ItemEvent `json:"events"`
}

type ItemEvent struct {
    Name       string             `json:"name"`
    Kind       ItemEventKind      `json:"kind"`
    Quantity   *int               `json:"quantity,omitempty"`
    From       string             `json:"from,omitempty"`
    To         string             `json:"to,omitempty"`
    SourceRefs []source.SourceRef `json:"source_refs"`
}

The exact source import follows the existing D&D types file. Quantity is a pointer so omission is distinguishable from an explicitly invalid zero. Empty From and To values mean the optional property is absent.

Event semantics

The allowed combinations are:

Kind	from	to	Meaning
discovered	absent	absent	The party learned that the item exists or encountered it without establishing possession.
acquired	absent	required	A party member or the collective party gained possession.
lost	required	absent	A party member or the collective party ceased to possess the item for a reason other than intended consumption.
consumed	required	absent	Intended use depleted the item or currency.
transferred	required	required	Possession moved between two distinct named party members.

The reserved holder value party represents collective or unresolved party possession. It is allowed for acquired, lost, and consumed, but not for either side of transferred. A transfer must name two distinct party members. Compare holders case-insensitively and Unicode-normalized when determining whether they are distinct, but preserve their display spelling.

Quantity, when present, must be a positive integer grounded explicitly in the source. Currency is represented as an item whose name preserves the stated denomination; do not convert denominations or calculate balances.

Every event must contain at least one source reference. Ordinary non-consuming use is not an event. Gifts, sales, or payments to someone outside the party are lost; destruction, abandonment, and theft are also lost. Emit both discovered and acquired only when the source independently establishes both events.

Deterministic order and equality

Create one canonical item-event helper package and make both normalization and invariant validation use it. The canonical order is:

1. earliest valid source position, using the pipeline source-document order;
2. normalized comparison form of name, then exact trimmed name;
3. kind;
4. from presence, normalized comparison form, then exact trimmed value;
5. to presence, normalized comparison form, then exact trimmed value;
6. quantity presence, then numeric value;
7. the complete canonical source-reference sequence.

Absent optional fields sort before present fields. Valid source positions sort before malformed positions; comparison must remain total and panic-free for invalid candidates so validators can report them.

An exact duplicate has the same trimmed name, kind, quantity presence and value, trimmed holders, and complete canonical source-reference sequence. Do not merge records that differ in any of those fields. Use collision-safe equality or key construction rather than delimiter concatenation.

Stage 1: Domain Contract, Canonical Rules, and Durable Codec

Goal

Establish the public artifact contract and strict durable boundary without registering an incomplete production module.

Changes

1. Extend internal/modules/dnd/types.go with:
   • ItemEventListKind;
   • ItemEventKind and the five constants above;
   • ItemEventList and ItemEvent using the fixed durable shape.
2. Add internal/modules/dnd/itemevents as the canonical domain helper package. It must own:
   • supported-kind checks;
   • validation of the kind-specific holder combination;
   • holder comparison sufficient to reject party transfers and self-transfers;
   • canonical ordering;
   • exact-duplicate comparison;
   • source-reference equality and validity helpers needed by normalization and invariant validation.
3. Reuse the existing normalized comparison-key implementation used by D&D NPC identity code for Unicode- and case-insensitive comparison. Use strings.TrimSpace for item and holder display normalization. Do not broaden this stage into a generic identity-package refactor.
4. Add internal/modules/dnd/codec/itemevents following the current D&D typed codec pattern:
   • embed a strict JSON Schema;
   • distinguish candidate decoding from approved encode/decode validation;
   • reject unknown fields;
   • expose the fixed artifact and schema identities;
   • report only bounded structural metadata such as event_count;
   • deep-clone source-reference slices and quantity pointers where ownership crosses a boundary.
5. Define the durable JSON Schema with:
   • a required top-level events array, which may be empty;
   • required event properties name, kind, and non-empty source_refs;
   • the five-value kind enum;
   • non-empty strings for name and for holders when present;
   • an integer quantity with minimum 1;
   • strict item, source-reference, and top-level objects;
   • conditional requirements/prohibitions matching the event-semantics table.
6. Enforce transfer holders being distinct and not equal to party in approved codec validation, because those comparisons are not cleanly expressed by the JSON Schema. Candidate decoding must still preserve semantically invalid values for the validation/retry pipeline.

Tests

Add package tests covering:

• all valid event kinds and holder combinations;
• collective-party acquisition, loss, and consumption;
• rejection of party transfers and case/Unicode-equivalent self-transfers;
• omitted versus present quantity and rejection of zero or negative quantity;
• strict unknown-field rejection;
• empty event lists;
• missing or empty source references;
• candidate decoding preserving values that approved decoding rejects;
• canonical order, including all tie-breakers and malformed references;
• exact equality distinguishing optional-field presence/value and complete evidence;
• deep-copy behavior for source references and quantity pointers;
• stable schema identity and metadata without payload leakage.

Completion Criteria

• Public item-event values round-trip through the approved codec only when they satisfy the durable contract.
• Candidate decoding can carry invalid semantic values to validators.
• Canonical comparison and equality have a single tested implementation.
• No item-event module, validator chain, or prompt is globally registered yet.

Stage 2: Extractor, Private Response Schema, and Prompt Assets

Goal

Implement source-grounded item-event extraction while preserving the boundary between provider output and the durable artifact.

Changes

1. Add internal/modules/dnd/extract/itemevents following the common D&D extractor structure:
   • key dnd/item-events;
   • requires chunks and source.transcript;
   • provides dnd.item_events;
   • strict empty options;
   • nil-client guard;
   • shared chunk preflight and source-reference ordering;
   • the current D&D default model profile and repair-attempt convention.
2. Define private provider response structs inside the extractor package. The response contains a required events array whose entries have:
   • required name, kind, and source_refs;
   • optional pointer quantity;
   • optional from and to;
   • private source ranges containing only start_segment and end_segment.
3. Add the embedded private structured-output schema using the fixed private identities. Keep it structurally strict, but do not encode the durable semantic enum, positive-quantity, holder-combination, or transcript-range rules there. Those rules belong to deterministic application validators so invalid model output participates in the normal retry/diagnostic flow.
4. Map the private response to dnd.ItemEventList by:
   • copying all provider fields without semantic rewriting;
   • attaching the current source ID to every private source range;
   • canonicalizing and deduplicating source references using the shared source order;
   • deterministically ordering candidates by earliest evidence as the existing D&D extractors do;
   • preserving quantity presence with a fresh pointer.
5. Add item-specific prompt assets for the task and instructions only. The instructions must state all event semantics from the feature roadmap, including currency, holder rules, non-consuming use, and the distinction between discovery and acquisition.
6. Assemble the prompt from existing shared assets in this order:
   • shared D&D system message;
   • shared extraction-evidence user message;
   • shared identity/reference context messages;
   • item-event task message;
   • rendered item-event instructions;
   • shared transcript message last.
7. Accept the same prompt inputs as the NPC extractor:
   • required transcript;
   • optional players, party, and glossary references. Do not accept or inject NPC registries, scene descriptions, item registries, or output from another generated lane.
8. Use the shared prompt manifest and caching metadata conventions. Reuse shared asset descriptors rather than creating item-specific copies. Publish the prompt, response-schema, and extraction-mapping policy identities and their fingerprints in module metadata.

Tests

Add extractor and prompt tests covering:

• key, capabilities, options, nil-client behavior, and preflight errors;
• private-schema identity and strict structural decoding;
• all five kinds, currency quantity, collective party, and omitted optional fields mapping into durable candidates;
• source-ID attachment, source-reference canonicalization, deterministic ordering, and quantity-pointer independence;
• preservation of invalid semantic candidates for validators;
• empty provider results producing a valid empty candidate list;
• exact prompt asset sequence and roles, with transcript last;
• optional reference rendering and absence behavior;
• reuse of shared asset descriptors and no module-specific duplicate of shared prompt text;
• stable fingerprint metadata that changes for item-specific prompt/schema/ mapping changes without exposing rendered content.

Do not add a test that asserts the numeric length of a shared prompt prefix.

Completion Criteria

• A fake structured-completion client can drive the extractor from a transcript chunk to a typed candidate artifact.
• Prompt ordering supports backend prompt caching and reuses shared assets exactly.
• The extractor is package-complete but is not yet added to production registration.

Stage 3: Shape and Source-Reference Validators

Goal

Add deterministic blocking validation for event semantics and evidence.

Changes

1. Add internal/modules/dnd/validate/itemevents/shape:
   • key extract/dnd/item-events/shape;
   • policy identity dnd.item_events.shape.v1;
   • blocking reason code invalid_item_event_shape.
2. The shape validator must reject:
   • absent or incorrectly typed item-event artifacts;
   • blank item names after trimming;
   • unsupported kinds;
   • invalid kind-specific from/to combinations;
   • party on either side of a transfer;
   • transfers whose holders have the same normalized comparison key;
   • present quantities less than one;
   • missing source-reference lists.
3. Permit leading/trailing whitespace in nonblank display fields at the extraction boundary so the normalizer can perform deterministic trimming. Treat an empty optional holder string as absence.
4. Add internal/modules/dnd/validate/itemevents/source_refs:
   • key extract/dnd/item-events/source_refs;
   • policy identity dnd.item_events.source_refs.v1;
   • blocking reason code invalid_item_event_source_references.
5. Mirror the current D&D source-reference validator behavior:
   • validate source IDs and inclusive segment ranges against the available source document;
   • when validating extraction for a specific chunk, require each range to be wholly contained in that chunk;
   • when validating a merged or normalized artifact without a chunk, validate against the source document without imposing a single-chunk condition;
   • emit bounded, indexed diagnostics.
6. Share the canonical domain helpers from Stage 1 rather than independently encoding holder semantics or source-reference validity.

Tests

Cover every accepted and rejected row of the event-semantics table, plus:

• whitespace that is normalizable versus whitespace-only required fields;
• quantities nil, 1, 0, and negative;
• transfer holder comparison across case and Unicode normalization;
• party in each applicable and inapplicable position;
• empty lists and multiple independently diagnosed invalid records;
• unknown source IDs, reversed ranges, out-of-bounds ranges, and extraction ranges crossing the current chunk;
• valid multi-range and multi-chunk post-merge evidence;
• diagnostic caps and stable policy metadata.

Completion Criteria

• Invalid event semantics and invalid evidence fail with distinct owning validators and reason codes.
• Valid candidates, including empty lists and currency events, pass.
• The validators remain package-complete but are not yet in production chains.

Stage 4: Deterministic Normalizer and Normalized Invariants

Goal

Normalize presentation and ordering without inventing inventory semantics, then verify the normalized contract independently.

Changes

1. Add internal/modules/dnd/normalize/itemevents:
   • key dnd/item-events;
   • requires merged;
   • provides normalized;
   • strict empty options;
   • policy identity dnd.item_events.normalize.v1;
   • no generated-reference dependency.
2. For each event, the normalizer must:
   • trim leading/trailing whitespace from name, from, and to;
   • preserve case and internal spelling;
   • clone the optional quantity;
   • canonicalize and deduplicate source references using the source-document order;
   • avoid changing kind, quantity, holders, or event meaning.
3. Sort the list with the Stage 1 canonical comparator.
4. Collapse exact duplicates only after display trimming and source-reference canonicalization. Preserve invalid records when safe to do so; do not allow malformed evidence to panic or cause unrelated records to disappear.
5. Emit bounded warning diagnostics, following existing D&D normalizer conventions, for:
   • normalized display fields;
   • canonicalized source references;
   • reordered events;
   • collapsed exact duplicates;
   • omitted warnings after the cap.
6. Publish normalization policy metadata and fingerprint using the common normalizer conventions.
7. Add internal/modules/dnd/validate/itemevents/invariants:
   • key normalize/dnd/item-events/invariants;
   • policy identity dnd.item_events.normalize_invariants.v1;
   • blocking reason code invalid_normalized_item_event_invariants.
8. The invariant validator must require:
   • trimmed display fields;
   • canonical source-reference sequences;
   • canonical list order;
   • no exact duplicates. It must use the same Stage 1 helpers as the normalizer. It should defer malformed shape and range reporting to their owning validators instead of emitting competing diagnoses.

Tests

Cover:

• each individual trim and source-reference normalization;
• canonical ordering through every tie-breaker;
• exact-duplicate collapse and preservation of near-duplicates;
• quantity-pointer cloning;
• empty and singleton lists;
• deterministic behavior under repeated normalization;
• malformed candidates remaining diagnosable without panics;
• every invariant failure independently;
• agreement between normalizer output and invariant validation;
• warning caps and non-sensitive metadata/fingerprints.

Include at least one test demonstrating that no aliasing, singularization, denomination conversion, discovery inference, acquisition inference, transfer inference, or ledger calculation occurs.

Completion Criteria

• Normalizing the same merged artifact repeatedly is idempotent.
• Every valid normalizer output passes the invariant validator.
• Semantically distinct events remain distinct.
• The normalizer and invariant validator are not yet globally registered.

Stage 5: Relatedness Validator and Production Registration

Goal

Add advisory grounding checks, then register the complete module stack and its default validation chains atomically.

Changes

1. Add internal/modules/dnd/validate/itemevents/source_relatedness:

   • key extract/dnd/item-events/source_relatedness;
   • policy identity dnd.item_events.source_relatedness.v1;
   • warning reason code item_event_source_unrelated.

2. For every event with otherwise readable evidence, aggregate the cited transcript text and look for a normalized token sequence derived from the item name, using the existing shared D&D token helpers. Do not require holder names or quantities to appear in every cited range. Relatedness is advisory only, so aliases, currency abbreviations, and transcript variation may warn but must not block.

3. Bound warnings consistently with the other D&D relatedness validators and avoid duplicating range-validity diagnostics owned by the source-reference validator.

4. Update D&D production registration to add:

   • the typed item-event codec;
   • the standard always-accept and always-reject typed validators;
   • an append-order merger that deep-clones item events, quantity pointers, and source references while preserving nil-versus-empty list behavior;
   • the item-event extractor and its prompt/schema assets;
   • the custom item-event normalizer and standard no-op normalizer;
   • shape, source-reference, source-relatedness, and invariant validators.

5. Add default chains with these exact members and ordering:

   Extraction:

   1. generic valid JSON;
   2. item-event shape;
   3. item-event source references;
   4. generic valid JSON Schema;
   5. item-event source relatedness.

   Normalization:

   1. generic valid JSON;
   2. item-event shape;
   3. normalized item-event invariants;
   4. item-event source references;
   5. generic valid JSON Schema;
   6. item-event source relatedness.

6. Do not add a registry validator or a generated-reference capability. Item events intentionally operate independently of NPC, scene, and other extraction outputs.

7. Extend registration and chain contract tests so omissions, duplicate registration, incorrect order, policy-identity collisions, and capability drift fail clearly.

Tests

Add tests for:

• related and unrelated item names, punctuation/case variation, multiple source ranges, invalid ranges, and capped warnings;
• registration of every item-event component and asset;
• exact default-chain order and blocking/advisory behavior;
• append-order merge semantics, deep cloning, and nil/empty preservation;
• duplicate-key and identity uniqueness across the full production registry;
• module metadata and capabilities containing identities only, not rendered content.

Completion Criteria

• The fully assembled registry exposes the item-event extractor, codec, merger, normalizers, validators, prompt assets, and both default chains.
• The default extraction and normalization chains accept valid item events, block invalid shape/evidence, and only warn on relatedness.
• Existing registered modules and chains remain unchanged.

Stage 6: End-to-End Integration and Maintained Configuration

Goal

Prove the production path and make the complete supported configuration exercise the new independent lane.

Changes

1. Add a focused production-registry integration test that runs an item-events lane from a transcript chunk through:
   • extraction;
   • candidate validation;
   • merge;
   • deterministic normalization;
   • normalized validation;
   • durable encode/decode.
2. Use a deterministic fake structured-completion client. Exercise at least:
   • one currency event with explicit quantity;
   • one event using collective party;
   • one transfer between named party members;
   • multiple events returned out of source order;
   • an exact duplicate that normalization removes;
   • a relatedness warning that does not fail the run.
3. Add focused negative integration coverage proving a blocking shape or source-reference failure follows the configured retry/failure path rather than bypassing validation. Do not use a live LLM.
4. Add dnd/item-events to the complete D&D configuration example. Place it in the first pipeline step alongside other independent extraction lanes; do not give it a dependency on generated NPC or scene artifacts. Leave the minimal example unchanged.
5. Update maintained example contract tests and deterministic CLI/provider test doubles so they recognize dnd.item_events, return a structurally valid private response, and account for the additional output artifact. Avoid brittle whole-file snapshots where focused semantic assertions suffice.
6. Verify the example still demonstrates the architecture already selected for other lanes: independent work in the first step and only genuinely dependent lanes in later steps.

Tests and Verification

Run:

• the focused new integration tests;
• example/configuration contract tests;
• CLI tests that execute or inspect the complete example;
• go test ./....

Also perform a local configuration validation or dry run using the repository's documented offline-safe mechanism. Do not require credentials or a provider network call.

Completion Criteria

• A production-registry pipeline produces a durable, normalized dnd/item-event-list.
• The complete example configures the lane in the correct independent step and remains accepted by configuration validation.
• The minimal example and all existing module behavior remain unchanged.

Stage 7: Canonical Documentation and Final Verification

Goal

Document the implemented contract in its owning locations, retire the future work entry, and perform repository-wide verification.

Changes

1. Add a canonical item-event artifact guide under docs/integrations/, following the current D&D artifact documentation pattern. Document:
   • module, capability, artifact, and schema identities;
   • every field and event-kind rule;
   • the reserved party holder;
   • currency representation;
   • source-reference requirements;
   • deterministic order and exact-duplicate behavior;
   • representative JSON for every kind;
   • the distinction between event extraction and inventory/ledger state;
   • the absence of dependencies on generated NPC, scene, or item registries.
2. Update the canonical configuration/module documentation to list:
   • the extractor;
   • supported prompt inputs;
   • the merger and normalizers;
   • validator keys and default chains;
   • the artifact kind and codec.
3. Update internal architecture/module documentation and operational workflow documentation wherever their exhaustive lists or complete-example walkthrough would otherwise become inaccurate.
4. Update docs/roadmap/future.md to remove the implemented item-tracking entry rather than leaving duplicate current and future documentation. Preserve unrelated future ideas.
5. Mark docs/roadmap/dnd-item-events.md as implemented. Keep it as the feature intent record for now; roadmap retirement is a separate cleanup decision.
6. Check links, terminology, and identity strings across code, schemas, examples, and docs. There must be no accidental use of items, item-tracking, inventory-events, or a v2 identity where the fixed names require item-events and v1.

Final Verification

Run the repository-documented:

• formatter;
• full test suite;
• static analysis;
• documentation/link checks, if available;
• configuration/example validation.

Inspect git diff --check and the final diff. Confirm that:

• no live-provider or credential-bearing test was introduced;
• no shared prompt content was copied into item-specific assets;
• no exact shared-prefix-length change-detector test was added;
• no raw transcript, prompt, reference, or credential data appears in metadata, diagnostics, or fingerprints;
• no item ledger, balance, item identity registry, alias resolution, ordinary-use tracking, or other roadmap non-goal slipped into scope;
• unrelated roadmap and user changes are preserved.

Completion Criteria

• Code, examples, and canonical documentation describe the same item-event contract and identities.
• The future roadmap no longer presents this implemented feature as future work.
• All required repository checks pass.

Open Questions

None. The feature roadmap and the fixed decisions above define the artifact, event semantics, pipeline placement, validation ownership, normalization rules, and documentation boundary needed to implement each stage without further product decisions.