notarius/docs/roadmap/implementation.md

# D&D Item Events Implementation Plan

## Purpose

This document is the decision-complete implementation plan for
[D&D Item Events](dnd-item-events.md). Implement the stages below in order, with
one stage per coding-agent prompt. The feature roadmap is authoritative for
product intent, artifact semantics, and non-goals; this document defines how to
reach that target state in the current architecture.

The work adds one LLM-backed `dnd/item-events` extractor. It records a
source-grounded, ordered history of item and currency events without attempting
to maintain inventory state. A deterministic normalizer prepares the extractor
output for downstream use.

## Cross-Stage Rules

Apply these rules in every stage:

- Follow [Architecture Policy](../policy/architecture.md),
  [Documentation Policy](../policy/documentation.md), and
  [Testing Policy](../policy/testing.md).
- Preserve the durable boundary: private provider response types belong to the
  extractor package; public item-event artifacts belong to `internal/modules/dnd`;
  codecs own strict wire validation.
- Keep LLM extraction and deterministic processing separate. The extractor may
  map provider fields and attach the current source ID, but it must not repair,
  infer, merge, or semantically reinterpret events.
- Reuse the existing shared D&D prompt assets and prompt assembly helpers. Do not
  copy their text into item-specific assets. Stable shared references must
  precede item-specific instructions and the changing transcript message must
  remain last.
- Use strict option decoding and reject unknown module options.
- Use policy identities with a `v1` suffix because this is the first item-event
  contract. Do not introduce a `v2` schema or backward-compatibility layer; the
  application is pre-release.
- Keep validation diagnostics bounded and content-safe, and keep capability and
  identity metadata free of transcript, prompt, reference, and credential
  content.
- Keep all tests offline and deterministic. Do not add live-provider tests,
  wall-clock-dependent assertions, exact prompt-prefix-length tests, or tests
  coupled only to private helper structure.
- Do not make unrelated cleanup changes. Preserve user changes already present
  in the worktree.
- At the end of every stage, run the narrowest relevant package tests and
  `go test ./...`. Run the repository's documented formatting and static-analysis
  commands before completing the final stage.

## Fixed Contract Decisions

The implementation must use the following names and shapes consistently.

### Module and artifact identities

- Extractor key: `dnd/item-events`
- Extractor capability: `dnd.item_events`
- Durable artifact kind: `dnd/item-event-list`
- Durable schema ID: `notarius.dnd.item_events`
- Durable schema name: `notarius_dnd_item_events_v1`
- Durable schema version: `v1`
- Private LLM schema ID: `notarius.dnd.item_events.llm`
- Private LLM schema name: `notarius_dnd_item_events_llm_v1`
- Private response schema key: `dnd_item_events_llm`
- Prompt ID: `dnd.item_events`
- Extraction mapping policy: `dnd.item_events.extract_mapping.v1`
- Normalization policy: `dnd.item_events.normalize.v1`

Use the same identity formats, MIME type, metadata keys, and fingerprint
construction conventions as the other current D&D extractors.

### Durable Go shape

Add public D&D domain types equivalent to:

```go
type ItemEventKind string

const (
    ItemEventKindDiscovered  ItemEventKind = "discovered"
    ItemEventKindAcquired    ItemEventKind = "acquired"
    ItemEventKindLost        ItemEventKind = "lost"
    ItemEventKindConsumed    ItemEventKind = "consumed"
    ItemEventKindTransferred ItemEventKind = "transferred"
)

type ItemEventList struct {
    Events []ItemEvent `json:"events"`
}

type ItemEvent struct {
    Name       string             `json:"name"`
    Kind       ItemEventKind      `json:"kind"`
    Quantity   *int               `json:"quantity,omitempty"`
    From       string             `json:"from,omitempty"`
    To         string             `json:"to,omitempty"`
    SourceRefs []source.SourceRef `json:"source_refs"`
}
```

The exact source import follows the existing D&D types file. `Quantity` is a
pointer so omission is distinguishable from an explicitly invalid zero. Empty
`From` and `To` values mean the optional property is absent.

### Event semantics

The allowed combinations are:

| Kind | `from` | `to` | Meaning |
| --- | --- | --- | --- |
| `discovered` | absent | absent | The party learned that the item exists or encountered it without establishing possession. |
| `acquired` | absent | required | A party member or the collective party gained possession. |
| `lost` | required | absent | A party member or the collective party ceased to possess the item for a reason other than intended consumption. |
| `consumed` | required | absent | Intended use depleted the item or currency. |
| `transferred` | required | required | Possession moved between two distinct named party members. |

The reserved holder value `party` represents collective or unresolved party
possession. It is allowed for `acquired`, `lost`, and `consumed`, but not for
either side of `transferred`. A transfer must name two distinct party members.
Compare holders case-insensitively and Unicode-normalized when determining
whether they are distinct, but preserve their display spelling.

`Quantity`, when present, must be a positive integer grounded explicitly in the
source. Currency is represented as an item whose `name` preserves the stated
denomination; do not convert denominations or calculate balances.

Every event must contain at least one source reference. Ordinary non-consuming
use is not an event. Gifts, sales, or payments to someone outside the party are
`lost`; destruction, abandonment, and theft are also `lost`. Emit both
`discovered` and `acquired` only when the source independently establishes both
events.

### Deterministic order and equality

Create one canonical item-event helper package and make both normalization and
invariant validation use it. The canonical order is:

1. earliest valid source position, using the pipeline source-document order;
2. normalized comparison form of `name`, then exact trimmed `name`;
3. `kind`;
4. `from` presence, normalized comparison form, then exact trimmed value;
5. `to` presence, normalized comparison form, then exact trimmed value;
6. `quantity` presence, then numeric value;
7. the complete canonical source-reference sequence.

Absent optional fields sort before present fields. Valid source positions sort
before malformed positions; comparison must remain total and panic-free for
invalid candidates so validators can report them.

An exact duplicate has the same trimmed name, kind, quantity presence and value,
trimmed holders, and complete canonical source-reference sequence. Do not merge
records that differ in any of those fields. Use collision-safe equality or key
construction rather than delimiter concatenation.

## Stage 1: Domain Contract, Canonical Rules, and Durable Codec

### Goal

Establish the public artifact contract and strict durable boundary without
registering an incomplete production module.

### Changes

1. Extend `internal/modules/dnd/types.go` with:
   - `ItemEventListKind`;
   - `ItemEventKind` and the five constants above;
   - `ItemEventList` and `ItemEvent` using the fixed durable shape.
2. Add `internal/modules/dnd/itemevents` as the canonical domain helper package.
   It must own:
   - supported-kind checks;
   - validation of the kind-specific holder combination;
   - holder comparison sufficient to reject `party` transfers and
     self-transfers;
   - canonical ordering;
   - exact-duplicate comparison;
   - source-reference equality and validity helpers needed by normalization and
     invariant validation.
3. Reuse the existing normalized comparison-key implementation used by D&D NPC
   identity code for Unicode- and case-insensitive comparison. Use
   `strings.TrimSpace` for item and holder display normalization. Do not broaden
   this stage into a generic identity-package refactor.
4. Add `internal/modules/dnd/codec/itemevents` following the current D&D typed
   codec pattern:
   - embed a strict JSON Schema;
   - distinguish candidate decoding from approved encode/decode validation;
   - reject unknown fields;
   - expose the fixed artifact and schema identities;
   - report only bounded structural metadata such as `event_count`;
   - deep-clone source-reference slices and quantity pointers where ownership
     crosses a boundary.
5. Define the durable JSON Schema with:
   - a required top-level `events` array, which may be empty;
   - required event properties `name`, `kind`, and non-empty `source_refs`;
   - the five-value `kind` enum;
   - non-empty strings for `name` and for holders when present;
   - an integer `quantity` with minimum `1`;
   - strict item, source-reference, and top-level objects;
   - conditional requirements/prohibitions matching the event-semantics table.
6. Enforce transfer holders being distinct and not equal to `party` in approved
   codec validation, because those comparisons are not cleanly expressed by the
   JSON Schema. Candidate decoding must still preserve semantically invalid
   values for the validation/retry pipeline.

### Tests

Add package tests covering:

- all valid event kinds and holder combinations;
- collective-party acquisition, loss, and consumption;
- rejection of `party` transfers and case/Unicode-equivalent self-transfers;
- omitted versus present quantity and rejection of zero or negative quantity;
- strict unknown-field rejection;
- empty event lists;
- missing or empty source references;
- candidate decoding preserving values that approved decoding rejects;
- canonical order, including all tie-breakers and malformed references;
- exact equality distinguishing optional-field presence/value and complete
  evidence;
- deep-copy behavior for source references and quantity pointers;
- stable schema identity and metadata without payload leakage.

### Completion Criteria

- Public item-event values round-trip through the approved codec only when they
  satisfy the durable contract.
- Candidate decoding can carry invalid semantic values to validators.
- Canonical comparison and equality have a single tested implementation.
- No item-event module, validator chain, or prompt is globally registered yet.

## Stage 2: Extractor, Private Response Schema, and Prompt Assets

### Goal

Implement source-grounded item-event extraction while preserving the boundary
between provider output and the durable artifact.

### Changes

1. Add `internal/modules/dnd/extract/itemevents` following the common D&D
   extractor structure:
   - key `dnd/item-events`;
   - requires `chunks` and `source.transcript`;
   - provides `dnd.item_events`;
   - strict empty options;
   - nil-client guard;
   - shared chunk preflight and source-reference ordering;
   - the current D&D default model profile and repair-attempt convention.
2. Define private provider response structs inside the extractor package. The
   response contains a required `events` array whose entries have:
   - required `name`, `kind`, and `source_refs`;
   - optional pointer `quantity`;
   - optional `from` and `to`;
   - private source ranges containing only `start_segment` and `end_segment`.
3. Add the embedded private structured-output schema using the fixed private
   identities. Keep it structurally strict, but do not encode the durable
   semantic enum, positive-quantity, holder-combination, or transcript-range
   rules there. Those rules belong to deterministic application validators so
   invalid model output participates in the normal retry/diagnostic flow.
4. Map the private response to `dnd.ItemEventList` by:
   - copying all provider fields without semantic rewriting;
   - attaching the current source ID to every private source range;
   - canonicalizing and deduplicating source references using the shared source
     order;
   - deterministically ordering candidates by earliest evidence as the existing
     D&D extractors do;
   - preserving quantity presence with a fresh pointer.
5. Add item-specific prompt assets for the task and instructions only. The
   instructions must state all event semantics from the feature roadmap,
   including currency, holder rules, non-consuming use, and the distinction
   between discovery and acquisition.
6. Assemble the prompt from existing shared assets in this order:
   - shared D&D system message;
   - shared extraction-evidence user message;
   - shared identity/reference context messages;
   - item-event task message;
   - rendered item-event instructions;
   - shared transcript message last.
7. Accept the same prompt inputs as the NPC extractor:
   - required transcript;
   - optional players, party, and glossary references.
   Do not accept or inject NPC registries, scene descriptions, item registries,
   or output from another generated lane.
8. Use the shared prompt manifest and caching metadata conventions. Reuse shared
   asset descriptors rather than creating item-specific copies. Publish the
   prompt, response-schema, and extraction-mapping policy identities and their
   fingerprints in module metadata.

### Tests

Add extractor and prompt tests covering:

- key, capabilities, options, nil-client behavior, and preflight errors;
- private-schema identity and strict structural decoding;
- all five kinds, currency quantity, collective `party`, and omitted optional
  fields mapping into durable candidates;
- source-ID attachment, source-reference canonicalization, deterministic
  ordering, and quantity-pointer independence;
- preservation of invalid semantic candidates for validators;
- empty provider results producing a valid empty candidate list;
- exact prompt asset sequence and roles, with transcript last;
- optional reference rendering and absence behavior;
- reuse of shared asset descriptors and no module-specific duplicate of shared
  prompt text;
- stable fingerprint metadata that changes for item-specific prompt/schema/
  mapping changes without exposing rendered content.

Do not add a test that asserts the numeric length of a shared prompt prefix.

### Completion Criteria

- A fake structured-completion client can drive the extractor from a transcript
  chunk to a typed candidate artifact.
- Prompt ordering supports backend prompt caching and reuses shared assets
  exactly.
- The extractor is package-complete but is not yet added to production
  registration.

## Stage 3: Shape and Source-Reference Validators

### Goal

Add deterministic blocking validation for event semantics and evidence.

### Changes

1. Add `internal/modules/dnd/validate/itemevents/shape`:
   - key `extract/dnd/item-events/shape`;
   - policy identity `dnd.item_events.shape.v1`;
   - blocking reason code `invalid_item_event_shape`.
2. The shape validator must reject:
   - absent or incorrectly typed item-event artifacts;
   - blank item names after trimming;
   - unsupported kinds;
   - invalid kind-specific `from`/`to` combinations;
   - `party` on either side of a transfer;
   - transfers whose holders have the same normalized comparison key;
   - present quantities less than one;
   - missing source-reference lists.
3. Permit leading/trailing whitespace in nonblank display fields at the
   extraction boundary so the normalizer can perform deterministic trimming.
   Treat an empty optional holder string as absence.
4. Add `internal/modules/dnd/validate/itemevents/source_refs`:
   - key `extract/dnd/item-events/source_refs`;
   - policy identity `dnd.item_events.source_refs.v1`;
   - blocking reason code `invalid_item_event_source_references`.
5. Mirror the current D&D source-reference validator behavior:
   - validate source IDs and inclusive segment ranges against the available
     source document;
   - when validating extraction for a specific chunk, require each range to be
     wholly contained in that chunk;
   - when validating a merged or normalized artifact without a chunk, validate
     against the source document without imposing a single-chunk condition;
   - emit bounded, indexed diagnostics.
6. Share the canonical domain helpers from Stage 1 rather than independently
   encoding holder semantics or source-reference validity.

### Tests

Cover every accepted and rejected row of the event-semantics table, plus:

- whitespace that is normalizable versus whitespace-only required fields;
- quantities `nil`, `1`, `0`, and negative;
- transfer holder comparison across case and Unicode normalization;
- `party` in each applicable and inapplicable position;
- empty lists and multiple independently diagnosed invalid records;
- unknown source IDs, reversed ranges, out-of-bounds ranges, and extraction
  ranges crossing the current chunk;
- valid multi-range and multi-chunk post-merge evidence;
- diagnostic caps and stable policy metadata.

### Completion Criteria

- Invalid event semantics and invalid evidence fail with distinct owning
  validators and reason codes.
- Valid candidates, including empty lists and currency events, pass.
- The validators remain package-complete but are not yet in production chains.

## Stage 4: Deterministic Normalizer and Normalized Invariants

### Goal

Normalize presentation and ordering without inventing inventory semantics, then
verify the normalized contract independently.

### Changes

1. Add `internal/modules/dnd/normalize/itemevents`:
   - key `dnd/item-events`;
   - requires `merged`;
   - provides `normalized`;
   - strict empty options;
   - policy identity `dnd.item_events.normalize.v1`;
   - no generated-reference dependency.
2. For each event, the normalizer must:
   - trim leading/trailing whitespace from `name`, `from`, and `to`;
   - preserve case and internal spelling;
   - clone the optional quantity;
   - canonicalize and deduplicate source references using the source-document
     order;
   - avoid changing kind, quantity, holders, or event meaning.
3. Sort the list with the Stage 1 canonical comparator.
4. Collapse exact duplicates only after display trimming and source-reference
   canonicalization. Preserve invalid records when safe to do so; do not allow
   malformed evidence to panic or cause unrelated records to disappear.
5. Emit bounded warning diagnostics, following existing D&D normalizer
   conventions, for:
   - normalized display fields;
   - canonicalized source references;
   - reordered events;
   - collapsed exact duplicates;
   - omitted warnings after the cap.
6. Publish normalization policy metadata and fingerprint using the common
   normalizer conventions.
7. Add `internal/modules/dnd/validate/itemevents/invariants`:
   - key `normalize/dnd/item-events/invariants`;
   - policy identity `dnd.item_events.normalize_invariants.v1`;
   - blocking reason code `invalid_normalized_item_event_invariants`.
8. The invariant validator must require:
   - trimmed display fields;
   - canonical source-reference sequences;
   - canonical list order;
   - no exact duplicates.
   It must use the same Stage 1 helpers as the normalizer. It should defer
   malformed shape and range reporting to their owning validators instead of
   emitting competing diagnoses.

### Tests

Cover:

- each individual trim and source-reference normalization;
- canonical ordering through every tie-breaker;
- exact-duplicate collapse and preservation of near-duplicates;
- quantity-pointer cloning;
- empty and singleton lists;
- deterministic behavior under repeated normalization;
- malformed candidates remaining diagnosable without panics;
- every invariant failure independently;
- agreement between normalizer output and invariant validation;
- warning caps and non-sensitive metadata/fingerprints.

Include at least one test demonstrating that no aliasing, singularization,
denomination conversion, discovery inference, acquisition inference, transfer
inference, or ledger calculation occurs.

### Completion Criteria

- Normalizing the same merged artifact repeatedly is idempotent.
- Every valid normalizer output passes the invariant validator.
- Semantically distinct events remain distinct.
- The normalizer and invariant validator are not yet globally registered.

## Stage 5: Relatedness Validator and Production Registration

### Goal

Add advisory grounding checks, then register the complete module stack and its
default validation chains atomically.

### Changes

1. Add `internal/modules/dnd/validate/itemevents/source_relatedness`:
   - key `extract/dnd/item-events/source_relatedness`;
   - policy identity `dnd.item_events.source_relatedness.v1`;
   - warning reason code `item_event_source_unrelated`.
2. For every event with otherwise readable evidence, aggregate the cited
   transcript text and look for a normalized token sequence derived from the
   item `name`, using the existing shared D&D token helpers. Do not require
   holder names or quantities to appear in every cited range. Relatedness is
   advisory only, so aliases, currency abbreviations, and transcript variation
   may warn but must not block.
3. Bound warnings consistently with the other D&D relatedness validators and
   avoid duplicating range-validity diagnostics owned by the source-reference
   validator.
4. Update D&D production registration to add:
   - the typed item-event codec;
   - the standard always-accept and always-reject typed validators;
   - an append-order merger that deep-clones item events, quantity pointers, and
     source references while preserving nil-versus-empty list behavior;
   - the item-event extractor and its prompt/schema assets;
   - the custom item-event normalizer and standard no-op normalizer;
   - shape, source-reference, source-relatedness, and invariant validators.
5. Add default chains with these exact members and ordering:

   Extraction:

   1. generic valid JSON;
   2. item-event shape;
   3. item-event source references;
   4. generic valid JSON Schema;
   5. item-event source relatedness.

   Normalization:

   1. generic valid JSON;
   2. item-event shape;
   3. normalized item-event invariants;
   4. item-event source references;
   5. generic valid JSON Schema;
   6. item-event source relatedness.

6. Do not add a registry validator or a generated-reference capability. Item
   events intentionally operate independently of NPC, scene, and other
   extraction outputs.
7. Extend registration and chain contract tests so omissions, duplicate
   registration, incorrect order, policy-identity collisions, and capability
   drift fail clearly.

### Tests

Add tests for:

- related and unrelated item names, punctuation/case variation, multiple source
  ranges, invalid ranges, and capped warnings;
- registration of every item-event component and asset;
- exact default-chain order and blocking/advisory behavior;
- append-order merge semantics, deep cloning, and nil/empty preservation;
- duplicate-key and identity uniqueness across the full production registry;
- module metadata and capabilities containing identities only, not rendered
  content.

### Completion Criteria

- The fully assembled registry exposes the item-event extractor, codec, merger,
  normalizers, validators, prompt assets, and both default chains.
- The default extraction and normalization chains accept valid item events,
  block invalid shape/evidence, and only warn on relatedness.
- Existing registered modules and chains remain unchanged.

## Stage 6: End-to-End Integration and Maintained Configuration

### Goal

Prove the production path and make the complete supported configuration exercise
the new independent lane.

### Changes

1. Add a focused production-registry integration test that runs an item-events
   lane from a transcript chunk through:
   - extraction;
   - candidate validation;
   - merge;
   - deterministic normalization;
   - normalized validation;
   - durable encode/decode.
2. Use a deterministic fake structured-completion client. Exercise at least:
   - one currency event with explicit quantity;
   - one event using collective `party`;
   - one transfer between named party members;
   - multiple events returned out of source order;
   - an exact duplicate that normalization removes;
   - a relatedness warning that does not fail the run.
3. Add focused negative integration coverage proving a blocking shape or
   source-reference failure follows the configured retry/failure path rather
   than bypassing validation. Do not use a live LLM.
4. Add `dnd/item-events` to the complete D&D configuration example. Place it in
   the first pipeline step alongside other independent extraction lanes; do not
   give it a dependency on generated NPC or scene artifacts. Leave the minimal
   example unchanged.
5. Update maintained example contract tests and deterministic CLI/provider test
   doubles so they recognize `dnd.item_events`, return a structurally valid
   private response, and account for the additional output artifact. Avoid
   brittle whole-file snapshots where focused semantic assertions suffice.
6. Verify the example still demonstrates the architecture already selected for
   other lanes: independent work in the first step and only genuinely dependent
   lanes in later steps.

### Tests and Verification

Run:

- the focused new integration tests;
- example/configuration contract tests;
- CLI tests that execute or inspect the complete example;
- `go test ./...`.

Also perform a local configuration validation or dry run using the repository's
documented offline-safe mechanism. Do not require credentials or a provider
network call.

### Completion Criteria

- A production-registry pipeline produces a durable, normalized
  `dnd/item-event-list`.
- The complete example configures the lane in the correct independent step and
  remains accepted by configuration validation.
- The minimal example and all existing module behavior remain unchanged.

## Stage 7: Canonical Documentation and Final Verification

### Goal

Document the implemented contract in its owning locations, retire the future
work entry, and perform repository-wide verification.

### Changes

1. Add a canonical item-event artifact guide under `docs/integrations/`,
   following the current D&D artifact documentation pattern. Document:
   - module, capability, artifact, and schema identities;
   - every field and event-kind rule;
   - the reserved `party` holder;
   - currency representation;
   - source-reference requirements;
   - deterministic order and exact-duplicate behavior;
   - representative JSON for every kind;
   - the distinction between event extraction and inventory/ledger state;
   - the absence of dependencies on generated NPC, scene, or item registries.
2. Update the canonical configuration/module documentation to list:
   - the extractor;
   - supported prompt inputs;
   - the merger and normalizers;
   - validator keys and default chains;
   - the artifact kind and codec.
3. Update internal architecture/module documentation and operational workflow
   documentation wherever their exhaustive lists or complete-example walkthrough
   would otherwise become inaccurate.
4. Update `docs/roadmap/future.md` to remove the implemented item-tracking entry
   rather than leaving duplicate current and future documentation. Preserve
   unrelated future ideas.
5. Mark `docs/roadmap/dnd-item-events.md` as implemented. Keep it as the feature
   intent record for now; roadmap retirement is a separate cleanup decision.
6. Check links, terminology, and identity strings across code, schemas, examples,
   and docs. There must be no accidental use of `items`, `item-tracking`,
   `inventory-events`, or a `v2` identity where the fixed names require
   `item-events` and `v1`.

### Final Verification

Run the repository-documented:

- formatter;
- full test suite;
- static analysis;
- documentation/link checks, if available;
- configuration/example validation.

Inspect `git diff --check` and the final diff. Confirm that:

- no live-provider or credential-bearing test was introduced;
- no shared prompt content was copied into item-specific assets;
- no exact shared-prefix-length change-detector test was added;
- no raw transcript, prompt, reference, or credential data appears in metadata,
  diagnostics, or fingerprints;
- no item ledger, balance, item identity registry, alias resolution, ordinary-use
  tracking, or other roadmap non-goal slipped into scope;
- unrelated roadmap and user changes are preserved.

### Completion Criteria

- Code, examples, and canonical documentation describe the same item-event
  contract and identities.
- The future roadmap no longer presents this implemented feature as future work.
- All required repository checks pass.

## Open Questions

None. The feature roadmap and the fixed decisions above define the artifact,
event semantics, pipeline placement, validation ownership, normalization rules,
and documentation boundary needed to implement each stage without further
product decisions.