From b6aad4fa98ab97ff4b66ba89f308a8b124f5684d Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 3 Aug 2026 23:29:32 +0000 Subject: [PATCH] Plan D&D location tracking --- docs/roadmap/implementation.md | 649 +++++++++++++++++++++++++++++++++ docs/roadmap/location.md | 337 +++++++++++++++++ 2 files changed, 986 insertions(+) create mode 100644 docs/roadmap/implementation.md create mode 100644 docs/roadmap/location.md diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..1b384fe --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,649 @@ +# D&D Location Tracking Implementation Plan + +## Objective + +Implement the target state in [D&D Location Tracking](location.md): an +evidence-grounded `dnd/locations` registry lane and a dependent +`dnd/location-occurrences` lane, including conservative location identity, +shared D&D entity-reconciliation infrastructure, production validation, +generated-reference wiring, maintained examples, and current documentation. + +This is an ordered implementation plan for a `gpt-5.6-terra` coding agent. +Implement one stage per prompt, in order. Finish each stage's tests and leave +the repository coherent before proceeding. Do not implement later-stage +production registrations early merely to make an incomplete feature selectable. + +All stages must follow: + +- [Architecture Policy](../policy/architecture.md) +- [Testing Policy](../policy/testing.md) +- [Documentation Policy](../policy/documentation.md) +- the D&D conventions in [D&D Module Internals](../internal/dnd.md) +- the durable policy decisions in [the feature roadmap](location.md) + +Use behavior-level tests. Do not add tests that merely freeze source layout, +exact prompt wording, message counts, shared-prefix length, or other incidental +implementation details. Keep tests deterministic, offline, and owned by the +component whose behavior they exercise. + +## Stage 1: Define Location Domain Types And Identity + +### Goal + +Establish the in-process contracts and deterministic identity policy on which +both lanes depend. + +### Work + +- Extend `internal/modules/dnd/types.go` with: + - `LocationListKind` = `dnd/location-list`; + - `LocationOccurrenceListKind` = `dnd/location-occurrence-list`; + - `LocationList`, `Location`, `LocationOccurrenceList`, + `LocationOccurrence`, and `LocationOccurrenceKind`; + - exact JSON members and the four occurrence constants specified in + `location.md`. +- Add `internal/modules/dnd/locations/identity`. +- Implement display normalization and comparison normalization consistently + with the existing NPC identity policy. Share a lower-level comparison helper + only if doing so preserves NPC behavior exactly; otherwise keep the small + policy-specific function explicit. +- Implement the versioned compact-JSON-array ID derivation contract from + `location.md`, including ID syntax checks and immutable list validation. +- Identity validation must require correctly derived, unique IDs while allowing + two records to have the same comparison name when their evidence anchors + differ. +- Add focused tests for Unicode normalization, whitespace, apostrophes, + deterministic encoding, evidence ordering, same-name/different-anchor IDs, + malformed IDs, missing evidence, and non-mutation. + +### Acceptance Criteria + +- The types compile without production registration. +- ID derivation exactly follows the documented five-element compact JSON input. +- Same normalized name plus different earliest evidence yields different IDs. +- Validation does not reject same-name records solely because their names + match, and it reports duplicate or mismatched IDs deterministically. +- `go test ./internal/modules/dnd/locations/... ./internal/modules/dnd/...` passes + for the packages available at this stage. + +### Prompt Size + +Small enough for one implementation prompt. + +## Stage 2: Add Durable Codecs And Schemas + +### Goal + +Create strict durable JSON ownership for both artifact kinds without exposing +either lane as a selectable pipeline yet. + +### Work + +- Add `internal/modules/dnd/codec/locations` and + `internal/modules/dnd/codec/locationoccurrences`, following the existing D&D + candidate/approved codec pattern. +- Add embedded Draft 2020-12 schemas with the IDs, names, root members, required + fields, enums, source-reference shape, and `additionalProperties: false` + contracts in `location.md`. +- Keep both schemas at `v1`. +- Support strict candidate decoding before semantic approval and strict durable + encoding/decoding after approval. +- Add representative valid fixtures and tests for schema metadata, defensive + schema bytes, empty arrays, unknown fields, missing fields, invalid types, + invalid enum values, malformed source references, invalid ID syntax, and + round trips. + +### Acceptance Criteria + +- Each codec advertises the correct artifact kind and metadata count. +- Candidate decoding preserves semantic mistakes for validators while + rejecting structurally invalid JSON. +- Approved encoding and decoding enforce the durable shape. +- `go test ./internal/modules/dnd/codec/locations/... ./internal/modules/dnd/codec/locationoccurrences/...` + passes offline. + +### Prompt Size + +Small enough for one implementation prompt. + +## Stage 3: Extract Shared Entity-Reconciliation Infrastructure + +### Goal + +Create the D&D-shared, domain-safe proposal machinery needed by both NPC and +location normalization, without changing NPC production behavior yet. + +### Work + +- Add `internal/modules/dnd/shared/entityreconcile`. +- Move or generalize the reusable behavior currently owned by + `internal/modules/dnd/normalize/npcs/context_material.go` and `proposal.go`: + - assign deterministic opaque candidate keys such as `candidate-000001` in + input order; + - clone candidate names and source references; + - build bounded transcript windows in source-document order; + - omit candidates whose references cannot safely produce context; + - coalesce overlapping or adjacent windows without mutating the source; + - define the private `duplicate_groups` proposal with `members` and + `canonical` candidate keys; + - reject blank, unknown, repeated, ineligible, overlapping, too-small, or + canonical-not-a-member groups; and + - return defensive, immutable assessment data identifying only safe groups. +- Keep LLM calls, retry decisions, artifact mutation, canonical-name policy, + durable ID derivation, and warning wording out of this package. +- Add a shared prompt instruction asset that states the key-copying and + proposal-safety contract without NPC- or location-specific identity rules. +- Add a shared private structured-response schema and loader/registration + support with a stable `v1` key, ID, name, and fingerprint. Registering the + schema more than once must not be required. +- Add table-driven tests for context bounds, ordering, invalid references, + coalescing, every unsafe proposal category, non-overlapping safe groups, + deterministic keys, defensive copies, and non-mutation. + +### Acceptance Criteria + +- The package has no dependency on `dnd.NPC`, `dnd.Location`, either + normalizer, or a concrete LLM client. +- Proposal values can identify duplicate candidates even when display names + are equal. +- The shared response contract cannot directly supply replacement records or + evidence. +- Existing NPC packages still compile before their migration. +- `go test ./internal/modules/dnd/shared/...` passes offline. + +### Prompt Size + +Medium, but coherent and suitable for one implementation prompt. Do not combine +it with the NPC migration. + +## Stage 4: Migrate NPC Normalization To The Shared Helper + +### Goal + +Make the existing NPC normalizer the first production consumer of the shared +entity-reconciliation contract while retaining its durable behavior. + +### Work + +- Refactor `internal/modules/dnd/normalize/npcs` to use opaque candidate keys, + shared context construction, shared proposal assessment, the shared response + schema, and the shared generic reconciliation instruction asset. +- Retain NPC-owned responsibilities: + - comparison-name preparation and deterministic duplicate handling; + - the NPC-specific task and canonical display-name rules; + - LLM invocation, bounded retry, fallback, warnings, and diagnostics; + - application of safe groups, evidence union, NPC ID derivation, and output + ordering. +- Remove superseded NPC-private context/proposal code and private schema assets + once no longer referenced. +- Update NPC prompt metadata and checkpoint fingerprints for the intentional + prompt/private-schema contract change. +- Preserve public module keys, durable NPC schema, identity policy, validator + chains, warning bounds, and fallback semantics. +- Test equal display names as distinct keyed candidates, alias consolidation, + rejected unsafe groups, retry exhaustion, private input ownership, redacted + errors, deterministic fallback, and non-mutation. + +### Acceptance Criteria + +- No durable NPC artifact field or module key changes. +- NPC normalization cannot confuse two candidates merely because their display + names match. +- Unsafe proposals leave a valid deterministic result and follow existing + retry/fallback policy. +- Obsolete NPC-only reconciliation helpers and schema are removed. +- `go test ./internal/modules/dnd/normalize/npcs/... ./internal/modules/dnd/shared/...` + passes offline. + +### Prompt Size + +Medium-to-large but bounded to one existing module. Suitable for one +implementation prompt; do not add location normalization in this stage. + +## Stage 5: Implement Location Extraction + +### Goal + +Add the LLM-backed extractor that produces evidence-grounded location +candidates. + +### Work + +- Add `internal/modules/dnd/extract/locations` following current D&D extractor + conventions: strict empty options, typed builder and registration function, + `llm_backed` execution metadata, immutable inputs, redacted errors, prompt + and response-schema fingerprints, and bounded diagnostics. +- Add a private response schema containing only `name` and source ranges; the + model must not produce durable IDs or prose. +- Compose the prompt from existing shared D&D system, identity, campaign + reference, transcript, and evidence assets plus module-owned task and + instructions. Preserve the documented extraction-message ordering and cache + controls. +- Define physical-place inclusion and conservative omission exactly as in + `location.md`, including generic labels, aliases, and nested places. +- Map source ranges to the current source ID, canonicalize exact duplicate + ranges, derive candidate location IDs in code, preserve semantically invalid + candidates for validators where safe, and return deterministic ordering. +- Add prompt-asset tests that verify shared asset reuse and rendered inputs by + behavior, without asserting exact shared-prefix length or prompt wording. +- Add extractor tests for empty output, mapping, evidence ownership, generic + same-name locations with different anchors, invalid candidate preservation, + client failures, registration metadata, fingerprints, and non-mutation. + +### Acceptance Criteria + +- The extractor cannot manufacture source identities or accept campaign + references as evidence. +- The private model response does not contain a durable ID. +- Same-name candidates with different evidence survive extraction as distinct + candidates. +- The package is testable through its local registration but is not yet added + to the production D&D registrar. +- `go test ./internal/modules/dnd/extract/locations/...` passes offline. + +### Prompt Size + +Medium and suitable for one implementation prompt. + +## Stage 6: Implement Semantic Location Normalization + +### Goal + +Add conservative alias and repeated-place reconciliation without collapsing +same-named or nested locations by default. + +### Work + +- Add `internal/modules/dnd/normalize/locations` as an `llm_backed` normalizer + using the shared entity-reconciliation package and private response schema. +- Deterministically clone and prepare the merged candidates first: + - normalize display whitespace; + - canonicalize and deduplicate source references; + - remove only exact duplicates with the same comparison name and exact + canonical evidence; + - assign opaque reconciliation keys; and + - retain same-name records with different evidence. +- Use bounded transcript windows and a module-owned location task that permits + grouping only when evidence clearly identifies one physical place. Explicitly + prohibit grouping solely by equal names, proximity, nesting, or generic + labels. +- Validate proposals through the shared package. Apply only safe groups in + deterministic code, choose the canonical name from the selected existing + candidate, union evidence, and derive the final evidence-anchored ID. +- Retain the deterministic candidate set on unusable proposals and follow the + existing NPC retry/fallback and bounded-warning conventions. +- Publish prompt, response-schema, identity-policy, normalization-policy, and + semantic-context fingerprints. +- Test aliases, repeated appearances, same-name distinct places, parent/child + locations, invalid and overlapping proposals, proposal retries, fallback, + ordering, ID recomputation, warning bounds, idempotent deterministic + application, and non-mutation. + +### Acceptance Criteria + +- The model proposes groups but cannot directly replace durable locations. +- A failed or ambiguous proposal cannot lose a valid candidate. +- Same-name locations remain distinct unless an approved evidence-backed group + joins them. +- Final IDs are derived only after group evidence is unioned. +- `go test ./internal/modules/dnd/normalize/locations/...` passes offline. + +### Prompt Size + +Medium-to-large but scoped to one normalizer and suitable for one implementation +prompt. + +## Stage 7: Add The Immutable Location Registry + +### Goal + +Provide safe generated-reference resolution and an unambiguous prompt +projection for downstream occurrence extraction. + +### Work + +- Add `internal/modules/dnd/locations/registry`, modeled on the immutable NPC + registry and its operation-time resolver. +- Define `ReferenceSlot = "locations"`, a 1,048,576-byte limit, and exactly one + accepted `application/json` location-list item when bound. +- Validate durable decoding and location identity before constructing a + registry. +- Store canonical durable bytes and semantic digests without retaining mutable + caller-owned content. Return defensive copies from all accessors. +- Produce a compact, source-free prompt projection containing ordered + `{id, name}` pairs. Do not include source references or generated-reference + provenance. +- Support exact lookup by ID and verify the matching canonical name; do not + provide an ambiguous name-only lookup as the occurrence linkage mechanism. +- Preserve the established seeded/operation resolver behavior and concurrency- + safe semantic caching. +- Test absent, empty, malformed, oversized, wrong-media-type, invalid-identity, + and valid registries; projections; ID lookup; defensive copies; raw and + semantic cache reuse; and concurrent resolution. + +### Acceptance Criteria + +- Distinct same-name records are both representable and addressable by ID. +- Registry evidence cannot appear in the prompt projection. +- Malformed static references fail during construction and malformed generated + references fail at operation resolution through existing boundaries. +- `go test ./internal/modules/dnd/locations/registry/...` passes offline. + +### Prompt Size + +Medium and suitable for one implementation prompt. + +## Stage 8: Implement Location-Occurrence Extraction + +### Goal + +Add the dependent LLM-backed lane that classifies source-grounded location +occurrences. + +### Work + +- Add `internal/modules/dnd/extract/locationoccurrences` with module key + `dnd/location-occurrences`, `llm_backed` execution metadata, strict empty + options, typed construction, and a required `locations` reference slot. +- Resolve the immutable registry at construction and for each operation using + the established generated-reference pattern. +- Add a private response schema requiring `location_id`, `name`, `kind`, and + source ranges. Restrict kinds to `visited`, `planned`, `recalled`, and + `mentioned`. +- Reuse the shared D&D extraction prompt assets and ordering. Place the compact + location registry after the shared transcript/evidence material and before + module task/instructions, consistent with current generated grounding. +- Encode the exact classification rules, precedence, multi-fact behavior, and + conservative omission policy from `location.md`. +- Map evidence only to the current source. Copy candidate IDs and names without + silently repairing unknown or mismatched values so deterministic validators + retain ownership of those diagnostics. +- Canonically order output and exact duplicates without dropping distinct + kinds or independent evidence. +- Test every kind, precedence, multiple supported facts, no-location and no- + occurrence outputs, required registry failures, same-name ID selection, + source-free prompt projection, current-transcript evidence, prompt/profile + metadata, client failures, non-mutation, and local registration. + +### Acceptance Criteria + +- Construction and operation specs declare `locations` as required and accept + only `dnd/location-list` JSON. +- The model sees IDs and names but no registry evidence. +- Registry context never becomes occurrence evidence. +- The extractor remains locally testable but is not production-selectable yet. +- `go test ./internal/modules/dnd/extract/locationoccurrences/...` passes + offline. + +### Prompt Size + +Medium and suitable for one implementation prompt. + +## Stage 9: Implement Deterministic Occurrence Normalization + +### Goal + +Canonicalize occurrence records against the exact location registry without a +second LLM call. + +### Work + +- Add `internal/modules/dnd/normalize/locationoccurrences` as a deterministic + normalizer with the same required `locations` slot. +- Clone all inputs. Normalize source ranges, names, ordering, and exact + duplicates. +- For a known `location_id`, replace display-name variation with the registry's + exact canonical name. Do not perform a name-only guess. +- Preserve an unknown ID or otherwise invalid record for validator diagnostics + and emit bounded warnings where current D&D normalizer conventions require + them. +- Sort using the complete order defined in `location.md`. +- Publish normalization and registry-projection fingerprints consistent with + the other registry-backed normalizers. +- Test all kind values, canonical name replacement, same-name distinct IDs, + exact-duplicate removal, distinct evidence retention, stable ordering, + unknown IDs, malformed registry resolution, warnings, idempotence, and + non-mutation. + +### Acceptance Criteria + +- No LLM client or prompt assets are required. +- Canonicalization is exclusively ID-based. +- Invalid records are not silently redirected to a different location. +- `go test ./internal/modules/dnd/normalize/locationoccurrences/...` passes + offline. + +### Prompt Size + +Small enough for one implementation prompt. + +## Stage 10: Add Location Registry Validators + +### Goal + +Give `dnd/location-list` the complete validator ownership expected of a +production D&D artifact. + +### Work + +- Add location validator packages under + `internal/modules/dnd/validate/locations` for: + - extraction shape; + - source-reference bounds/current-chunk ownership; + - advisory source relatedness; and + - normalized identity derivation and ID uniqueness. +- Use shared D&D citation, unit-reference, diagnostic, and matching helpers + where their contracts apply. +- The identity validator must allow repeated comparison names and validate the + evidence-anchored derivation policy instead of importing NPC uniqueness + assumptions. +- Keep diagnostics indexed, aggregated, bounded, stable, and free of raw prompt + or reference content. +- Add tests for accepted values, every owned failure, same-name distinct + locations, malformed/unreadable citations, advisory relatedness, bounds, + registration metadata, fingerprints, nil safety where applicable, and + non-mutation. + +### Acceptance Criteria + +- Validator responsibilities do not overlap merely to increase test coverage. +- Relatedness remains advisory and uses only cited current-transcript text. +- Validators do not repair or mutate artifacts. +- `go test ./internal/modules/dnd/validate/locations/...` passes offline. + +### Prompt Size + +Medium and suitable for one implementation prompt. + +## Stage 11: Add Location-Occurrence Validators + +### Goal + +Give `dnd/location-occurrence-list` complete structural, registry, ordering, +evidence, and advisory validation. + +### Work + +- Add validator packages under + `internal/modules/dnd/validate/locationoccurrences` for: + - extraction shape and supported kinds; + - required registry membership and exact `location_id`/`name` pairing; + - normalized ordering and exact-duplicate invariants; + - source-reference bounds/current-chunk ownership; and + - advisory source relatedness. +- Reuse the immutable location resolver rather than decoding caller-owned + references independently in each validator. +- Ensure same-name registry records remain distinguishable by ID. +- Keep registry context out of evidence checks. +- Add focused tests for each kind, unknown IDs, mismatched names, same-name + locations, ordering, duplicates, malformed required references, invalid + evidence, advisory diagnostics, registration metadata, fingerprints, + diagnostic bounds, and non-mutation. + +### Acceptance Criteria + +- An ID/name mismatch is rejected even when another registry record has the + supplied name. +- Missing or malformed required registry references fail at the established + boundary. +- Validators remain deterministic and do not alter occurrence records or the + registry. +- `go test ./internal/modules/dnd/validate/locationoccurrences/...` passes + offline. + +### Prompt Size + +Medium and suitable for one implementation prompt. + +## Stage 12: Compose The Production D&D Family + +### Goal + +Make both lanes selectable as one coherent production addition after all +component contracts are present. + +### Work + +- Extend `internal/modules/dnd/register` to register, in dependency-safe order: + - both codecs; + - both extractors; + - typed append-order mergers; + - the LLM-backed location normalizer; + - the deterministic occurrence normalizer; + - all validators; + - shared reconciliation schema assets and both new prompt manifests; + - evidence projectors; and + - extract and normalize default validator chains. +- Ensure both LLM-backed modules select the maintained `dnd-extraction` + fallback profile and inherit the existing profile policy. +- Define chain order consistently with existing D&D artifacts: generic JSON, + shape, registry/identity or normalized invariants at the appropriate stage, + source references, durable JSON Schema, then advisory relatedness. +- Update registrar tests for artifact kinds, keys, execution classes, reference + slot requirements, builder construction, assets, profile use, evidence + projection, chain contents/order, duplicate registration, and failure + propagation. +- Update any integration-level artifact-kind allowlists or typed registries + required by the framework; do not add module-specific orchestration logic. + +### Acceptance Criteria + +- One D&D registration call exposes both complete lanes and no partial + registration succeeds silently. +- Catalog inspection reports correct artifact kinds, stages, execution classes, + reference slots, profiles, and fingerprints. +- Both artifact kinds support evidence projection without registry evidence + leakage. +- `go test ./internal/modules/dnd/register/... ./internal/modules/dnd/...` passes + offline. + +### Prompt Size + +Medium-to-large but limited to composition and suitable for one implementation +prompt. + +## Stage 13: Add Maintained Pipeline And Handoff Coverage + +### Goal + +Exercise the feature through real configuration, ordered generated references, +chunk operations, acceptance gates, and durable output. + +### Work + +- Update `examples/dnd-complete.config.yml`: + - add `locations` to the first descriptive step with extract, append-order + merge, and LLM-backed normalize bindings; + - add a generated `locations` reference to the next step; + - add `location-occurrences` to that step with extract, append-order merge, + and deterministic normalize bindings; and + - add both lanes to evidence-context output where appropriate. +- Keep the minimal example minimal unless its stated purpose requires a + location lane; do not turn it into a second complete example. +- Extend maintained example-loading/config-validation tests. +- Add integration tests that prove: + - the normalized accepted registry is handed off in memory; + - the occurrence lane cannot run before its producer; + - missing, cyclic, wrong-kind, wrong-media-type, rejected, or unaccepted + producers are rejected at the existing boundaries; + - same-name locations remain distinguishable by ID through the handoff; + - registry evidence never becomes occurrence evidence; + - retries and checkpoints honor prompt, schema, identity, and generated- + reference fingerprints; and + - output contains both durable artifact envelopes and evidence context. +- Use recording/fake structured clients only; no network-dependent tests. + +### Acceptance Criteria + +- The complete example loads through the real config path and exercises all + registered D&D lanes. +- Ordered handoff failure semantics match the framework's existing fail-whole- + run policy. +- Integration tests cover behavior rather than duplicating package internals. +- `go test ./internal/modules/integration/... ./internal/config/...` and any + example-specific test targets pass offline. + +### Prompt Size + +Medium-to-large but coherent as one end-to-end integration prompt. + +## Stage 14: Publish Current Documentation And Perform Final Verification + +### Goal + +Make the implemented feature discoverable and retire fulfilled future-work +language without leaving development-history documentation behind. + +### Work + +- Create canonical integration contracts: + - `docs/integrations/dnd-location-artifacts.md`; + - `docs/integrations/dnd-location-occurrence-artifacts.md`. +- Update `docs/config.md` with both selectable keys, the `locations` reference + slot and compatibility, execution classes, validators, default chains, and + the complete-example link. +- Update `docs/internal/dnd.md` with nine-lane composition, shared entity + reconciliation, evidence-anchored identity, occurrence grounding, prompt + asset reuse, and intentional lane differences. Link to integration contracts + instead of duplicating their JSON shapes. +- Update `docs/integrations/json-output.md`, `README.md`, and other current + canonical inventories only where repository inspection shows that the new + artifact kinds or maintained example must be listed. +- Remove the fulfilled `Location Extraction` section from + `docs/roadmap/future.md`. Keep the generic LLM-assisted deduplication item and + clarify only if needed that the new D&D helper does not fulfill that broader + feature. +- Verify all relative links and search current documentation for stale + seven-lane counts, missing keys, obsolete location-planning claims, and + accidental claims that references are evidence. +- Run formatting, focused tests, the full Go test suite, static analysis, and + repository-provided config/example checks. Inspect `git diff --check` and + confirm no unrelated files changed. +- After implementation and verification are complete, leave `location.md` and + this plan in place for the user's separate roadmap-retirement step; do not + delete them unless explicitly asked. + +### Acceptance Criteria + +- Current documentation describes the implemented contracts and only + implemented behavior outside `docs/roadmap/`. +- The roadmap no longer presents completed location tracking as future work. +- Links, examples, module inventories, and default-chain tables agree with + production registration. +- `go test ./internal/modules/dnd/...` passes. +- `go test ./internal/modules/integration/...` passes. +- `go test ./...` passes. +- `go vet ./...` passes. +- Repository-provided configuration/example validation passes. +- `git diff --check` reports no errors. + +### Prompt Size + +Medium and suitable for one implementation prompt. + +## Open Questions + +None. The feature roadmap fixes the artifact shapes, identity scope and +derivation, occurrence categories, classification precedence, reference +dependency, reconciliation safety boundary, pipeline placement, and non-goals +needed to implement every stage without an additional product decision. diff --git a/docs/roadmap/location.md b/docs/roadmap/location.md new file mode 100644 index 0000000..4e3d34d --- /dev/null +++ b/docs/roadmap/location.md @@ -0,0 +1,337 @@ +# D&D Location Tracking + +## Purpose + +Add evidence-grounded D&D location tracking without turning a single extractor +into both an entity registry and an event classifier. The target design follows +the established NPC pattern: one lane identifies canonical location records and +a later lane records how the party related to those locations in the transcript. + +This roadmap defines the desired end state and policy choices. The ordered work +needed to reach that state is in [the implementation plan](implementation.md). + +## User Intent + +- Record locations the party visits or that the session otherwise discusses. +- Distinguish current physical presence from plans, recollections, and ordinary + mentions. +- Preserve transcript evidence for every durable record. +- Reconcile aliases and repeated appearances conservatively. +- Keep distinct places separate when they happen to share a generic name. +- Keep the schemas minimal. Location description, hierarchy, participants, and + narrative analysis belong in other artifacts or deterministic joins. + +## Target Capability + +The D&D module family will have two new lanes: + +1. `dnd/locations` produces a session-scoped registry of physical places. +2. `dnd/location-occurrences` consumes the normalized location registry and + produces an ordered list of source-grounded relationships between the party + and those places. + +The normalized location artifact is handed to the occurrence lane through a +required generated reference named `locations`. The occurrence lane must use +that registry for identity grounding, but the current transcript remains its +only evidence source. + +## Durable Artifact Contracts + +Both contracts remain at `v1`; Notarius is pre-release and does not need a +compatibility layer for these new artifacts. + +### Location registry + +The location lane uses: + +- artifact kind: `dnd/location-list` +- module key: `dnd/locations` +- schema ID: `notarius.dnd.locations` +- schema name: `notarius_dnd_locations_v1` +- media type: `application/json` +- root member: `locations` + +Each location contains exactly: + +| Field | Type | Meaning | +| --- | --- | --- | +| `id` | string | Deterministic, session-scoped canonical location identity. | +| `name` | string | Evidence-grounded display name or transcript-established label. | +| `source_refs` | non-empty source-reference array | Current-transcript evidence that identifies the place. | + +Locations are physical or spatial places: planes, regions, settlements, +districts, buildings, rooms, landmarks, routes, and geographic features. A +generic label such as `the tavern` is permitted only when the transcript uses +it for a specific place. The extractor must not invent a qualifier merely to +distinguish that place from another place with the same label. + +The registry does not contain type, parent, description, summary, coordinates, +participants, visit status, or occurrence data. Parent and child places are +separate identities when the transcript identifies both; nesting alone is not +a reason to merge them. + +### Location occurrences + +The occurrence lane uses: + +- artifact kind: `dnd/location-occurrence-list` +- module key: `dnd/location-occurrences` +- schema ID: `notarius.dnd.location_occurrences` +- schema name: `notarius_dnd_location_occurrences_v1` +- media type: `application/json` +- root member: `occurrences` + +Each occurrence contains exactly: + +| Field | Type | Meaning | +| --- | --- | --- | +| `location_id` | string | An exact ID from the consumed normalized location registry. | +| `name` | string | The canonical display name associated with `location_id`. | +| `kind` | enum | `visited`, `planned`, `recalled`, or `mentioned`. | +| `source_refs` | non-empty source-reference array | Current-transcript evidence for both the place and the classified occurrence. | + +`location_id` is required even though existing NPC interactions currently use +name-only grounding. Locations can legitimately share the same display name, +so a name alone cannot provide an unambiguous cross-artifact link. The name is +retained for readable standalone output and must exactly match the registry +record selected by the ID after normalization. + +## Identity Policy + +Location identity is conservative and scoped to one source document. It is not +a campaign-wide or cross-session world identity. + +Display normalization trims surrounding whitespace and collapses internal +Unicode whitespace. Comparison normalization uses the existing D&D entity +rules: Unicode NFKC normalization, normalized apostrophes, collapsed +whitespace, and Unicode case folding. + +The canonical ID is: + +~~~text +location:sha256: +~~~ + +The digest input is the UTF-8 encoding of compact JSON for this five-element +array: + +~~~text +["dnd.locations.identity.v1", comparison_name, source_id, start_unit_id, end_unit_id] +~~~ + +The source values come from the earliest reference after canonical reference +sorting and exact deduplication. Compact JSON array encoding is part of the +identity contract: it avoids delimiter ambiguity and must not be replaced +without changing the policy version. A blank comparison name or missing valid +source reference produces no manufactured ID and remains a validation error. + +Including the evidence anchor prevents two unrelated places called `the +tavern` from receiving the same ID. When semantic normalization safely groups +aliases or repeated appearances, it first chooses an existing canonical display +name and unions the evidence; it then derives the final ID from that name and +the earliest unioned reference. + +The normalizer may merge records only when transcript evidence clearly shows +that they denote the same physical place. It must not merge records solely +because: + +- their comparison names are equal; +- they are near one another in the transcript; +- one is spatially nested inside the other; or +- their labels are both generic. + +Distinct normalized records may therefore have the same comparison name, but +their IDs must be unique and correctly derived. Exact duplicates with the same +comparison name and canonical evidence may be collapsed deterministically. + +## Occurrence Semantics + +Each occurrence has one kind: + +- `visited`: current-session gameplay establishes that one or more party + members are physically present at the location, including an arrival, + continuing presence, or departure. +- `planned`: the party explicitly proposes, intends, or agrees to future travel + to the location. Mere hypotheticals or speculation are not plans. +- `recalled`: the transcript explicitly recounts or recaps the party being at + the location before the current session's live events. +- `mentioned`: the location is explicitly referenced but the occurrence does + not meet a stronger definition. This includes lore, directions, third-party + activity, non-actionable speculation, and out-of-character discussion. + +An inferred but unstated place produces no location or occurrence. Uncertainty +is handled by conservative omission rather than an `uncertain` enum value. + +For one occurrence supported by overlapping evidence, classification precedence +is `visited`, then `planned`, then `recalled`, then `mentioned`; `mentioned` is +the fallback. A passage may produce multiple records when it independently +supports separate facts, such as recalling an earlier visit while planning a +return. Exact duplicates with the same ID, kind, and canonical evidence are +collapsed. Different kinds or independently supported evidence remain. + +Output is ordered by earliest evidence in source-document order, then by +`location_id`, `name`, kind order (`visited`, `planned`, `recalled`, +`mentioned`), and the remaining canonical reference sequence. + +## Extraction, Normalization, And Evidence + +### Location registry lane + +The extractor is LLM-backed and follows the shared D&D extraction prompt and +input conventions. It emits names and source ranges through a private response +schema; deterministic mapping supplies the current source ID and derives +candidate IDs. Campaign references may disambiguate terminology but never +become durable evidence. + +The merger uses the typed append-order convention. The normalizer is LLM-backed: +it deterministically prepares names and evidence, then asks the model only for +duplicate groups. The model may identify groups and choose a canonical member, +but it may not create, delete, rewrite, or directly replace durable records. +Code validates the proposal, applies non-overlapping safe groups, unions +evidence, derives final IDs, orders output, and emits bounded warnings. + +Malformed, unknown, overlapping, or ambiguous proposal groups are rejected. +The normalizer uses the existing bounded retry behavior and falls back to the +safe deterministic candidate set if no usable proposal is obtained. + +### Location occurrence lane + +The extractor is LLM-backed and requires exactly one validated `locations` +reference. The prompt projection contains only ordered `{id, name}` pairs; it +omits registry evidence and reference provenance. The model must copy both +values from one projected record and cite current-transcript source ranges for +the occurrence. + +The occurrence normalizer is deterministic. It canonicalizes names by exact +registry ID, normalizes evidence and ordering, and removes exact duplicates. +Unknown IDs and mismatched ID/name pairs remain inspectable validation failures +rather than being guessed or silently reassigned. + +The occurrence lane cannot add a missing location to the registry. If the +location extractor omitted a place, the correct behavior is to omit its +occurrence and improve the upstream extraction later. + +## Shared Entity Reconciliation + +Adding a second LLM-assisted entity registry demonstrates a concrete shared +need in the D&D domain. The existing NPC normalization context-window and +proposal-safety logic will move to +`internal/modules/dnd/shared/entityreconcile` and serve both NPC and location +normalizers. + +The shared package owns: + +- deterministic opaque candidate keys; +- source-window construction and canonical prompt materials; +- a common private duplicate-group response contract; +- validation of unknown, repeated, overlapping, malformed, or ineligible + candidate keys; and +- immutable assessment results identifying safe groups. + +It does not call the LLM, choose domain-specific canonical names, derive +durable IDs, mutate domain artifacts, or format domain warnings. Those +responsibilities remain in each normalizer. + +NPC normalization will migrate to the shared key-based proposal contract +without changing its durable NPC behavior. Its prompt and private response +schema fingerprints are expected to change, so stale NPC normalization +checkpoints will invalidate normally. + +The two normalizers will reuse an exactly identical shared reconciliation +instruction asset and private response schema. Module-owned task text will +continue to define the different NPC and location identity rules. This keeps +shared prompt content identical without pretending the two domains have the +same semantic merge policy. + +This helper is intentionally D&D-specific. It does not implement the broader +domain-neutral replacement-element normalizer still described in +[future work](future.md). + +## Reference Contract And Pipeline Placement + +The generated reference slot is named `locations` and accepts exactly one JSON +artifact of kind `dnd/location-list`, with the established 1 MiB limit. It is +required by both extraction and normalization for +`dnd/location-occurrences`. Static file bindings remain valid where the +framework permits them, but the maintained complete example uses a generated +same-run artifact. + +The complete D&D pipeline places `locations` in the first descriptive step +alongside the independent NPC, item-event, and scene-description lanes. It +places `location-occurrences` in the next step and binds the accepted normalized +location artifact from the first step. The occurrence lane has no mandatory +NPC or scene-description dependency. + +No current downstream lane is changed to consume location artifacts. Future +narrative reports or joins may use the canonical IDs after defining their own +contracts. + +## Validation And Production Defaults + +The location registry receives production validators for: + +- required shape and supported ID syntax; +- current-document and current-chunk source ranges; +- normalized identity derivation and ID uniqueness; and +- advisory source relatedness. + +The location occurrence artifact receives production validators for: + +- required shape and the four supported kinds; +- registry membership and exact ID/name correspondence; +- normalized ordering and exact-duplicate invariants; +- current-document and current-chunk source ranges; and +- advisory source relatedness. + +Validators remain immutable and diagnostic. Durable JSON Schema validation +stays in the production chains after semantic shape and source-reference +checks, consistent with the existing D&D lanes. + +## Documentation End State + +Implementation will add canonical integration documents for both durable +artifacts and update current-state documentation to cover: + +- both module and artifact keys; +- the `locations` generated-reference slot; +- production validators and default chains; +- D&D family composition, reconciliation, identity, and grounding behavior; +- the complete maintained pipeline example; and +- JSON output and evidence-context support. + +After the feature is implemented, the fulfilled Location Extraction section is +removed from `future.md`. Historical implementation narration remains in +version control rather than current documentation. + +## Out Of Scope + +- Campaign-wide or cross-session canonical location IDs. +- A location ontology, hierarchy, map, coordinates, or containment graph. +- Location descriptions, summaries, participants, ownership, or encounter + analysis. +- Inferring a location that the transcript does not identify. +- Automatically creating registry records from occurrence output. +- Changing NPC-interaction artifacts to use NPC IDs. +- Making other lanes consume location references. +- A generic domain-neutral LLM deduplication framework. +- Long-term artifact-version migration machinery. + +## Acceptance Criteria + +- Both durable contracts are minimal, strict, versioned, and registered. +- Location IDs are deterministic under the documented policy and do not force + same-named places to collapse. +- Alias and repeat reconciliation is proposal-only, conservatively validated, + and safe on retry exhaustion. +- Occurrences use one of the four defined kinds and carry an unambiguous + registry ID/name pair plus current-transcript evidence. +- Missing, malformed, oversized, or incompatible `locations` references fail + through the established configuration or operation boundaries. +- Both lanes have typed mergers, normalizers, evidence projectors, validators, + default chains, prompt/profile metadata, and registration coverage consistent + with the D&D family. +- NPC normalization retains its durable behavior after adopting the shared + reconciliation helper. +- The maintained complete example loads and exercises the generated handoff. +- Focused D&D and integration tests pass offline, and current documentation + describes only implemented behavior once the work is complete.