diff --git a/docs/integrations/dnd-npc-artifacts.md b/docs/integrations/dnd-npc-artifacts.md index 221ce2b..87f5d14 100644 --- a/docs/integrations/dnd-npc-artifacts.md +++ b/docs/integrations/dnd-npc-artifacts.md @@ -55,6 +55,14 @@ with the same canonical identity, retains their earliest position, and merges their canonicalized evidence; it does not add aliases, roles, descriptions, or relationship fields. +When evidence supports a semantically duplicate group, the canonical display +name is one of that group's supplied candidates. A complete, stable proper name +is preferred over an abbreviation. An unadorned proper name is preferred over +the same name plus a contextual class, role, title, or relationship descriptor +unless the transcript establishes that descriptor as part of the person's +name. A longer candidate is not preferred solely because it includes such a +descriptor. + ## Scope and consumers Only individually identifiable NPC names with transcript evidence belong in diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 1b384fe..25a2e87 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -2,648 +2,431 @@ ## Objective -Implement the target state in [D&D Location Tracking](location.md): an -evidence-grounded `dnd/locations` registry lane and a dependent -`dnd/location-occurrences` lane, including conservative location identity, -shared D&D entity-reconciliation infrastructure, production validation, -generated-reference wiring, maintained examples, and current documentation. +Complete the target state in [D&D Location Tracking](location.md) by correcting +the behavioral and maintainability issues found in the post-implementation +audit. The location registry and occurrence lanes are already implemented and +production-selectable; the remaining work restores one NPC normalization +contract, aligns location-occurrence classification with the roadmap, narrows a +module's declared references, and factors demonstrated registry-resolution +mechanics into a D&D-shared helper. This is an ordered implementation plan for a `gpt-5.6-terra` coding agent. -Implement one stage per prompt, in order. Finish each stage's tests and leave -the repository coherent before proceeding. Do not implement later-stage -production registrations early merely to make an incomplete feature selectable. +Stages 1 through 14 are complete and must not be reimplemented. Implement one +new stage per prompt, beginning with Stage 15, and complete each stage's focused +validation before proceeding. -All stages must follow: +All remaining stages must follow: -- [Architecture Policy](../policy/architecture.md) -- [Testing Policy](../policy/testing.md) -- [Documentation Policy](../policy/documentation.md) -- the D&D conventions in [D&D Module Internals](../internal/dnd.md) -- the durable policy decisions in [the feature roadmap](location.md) +- [Architecture Policy](../policy/architecture.md); +- [Testing Policy](../policy/testing.md); +- [Documentation Policy](../policy/documentation.md); +- [D&D Module Internals](../internal/dnd.md); and +- the durable policy decisions in [the feature roadmap](location.md). -Use behavior-level tests. Do not add tests that merely freeze source layout, -exact prompt wording, message counts, shared-prefix length, or other incidental -implementation details. Keep tests deterministic, offline, and owned by the -component whose behavior they exercise. +Use behavior-level tests. Do not add tests that freeze exact prompt wording, +message counts, source layout, shared-prefix length, or other incidental +implementation details. Prompts and canonical documentation must agree, but +that agreement is a review responsibility rather than a reason to add a +change-detector test. -## Stage 1: Define Location Domain Types And Identity +## Completed Work Summary + +Stages 1 through 14 delivered the following production state: + +| Completed stage | Result | +| --- | --- | +| Stage 1 | Added location and location-occurrence domain types plus evidence-anchored location identity. | +| Stage 2 | Added strict `v1` durable codecs and JSON Schemas for both artifact kinds. | +| Stage 3 | Added shared, opaque-key D&D entity-reconciliation context and proposal safety infrastructure. | +| Stage 4 | Migrated NPC normalization to the shared reconciliation machinery. | +| Stage 5 | Added evidence-grounded `dnd/locations` extraction. | +| Stage 6 | Added conservative LLM-assisted location normalization. | +| Stage 7 | Added immutable location-registry resolution and source-free `{id, name}` projection. | +| Stage 8 | Added registry-grounded `dnd/location-occurrences` extraction. | +| Stage 9 | Added deterministic location-occurrence normalization. | +| Stage 10 | Added location shape, identity, evidence, and advisory validators. | +| Stage 11 | Added location-occurrence shape, registry, invariant, evidence, and advisory validators. | +| Stage 12 | Registered both lanes, their assets, evidence projectors, default profiles, and validator chains. | +| Stage 13 | Added the ordered generated-reference handoff to the complete D&D example and integration coverage. | +| Stage 14 | Published the current integration, configuration, internal, output, and top-level documentation and ran final verification. | + +The post-implementation audit confirmed that the feature's durable types, +identity derivation, codecs, registration, validators, generated handoff, +evidence publication, maintained example, and integration coverage are in +place. At the audit baseline, `go test ./...`, focused race tests, `go vet +./...`, `go build ./cmd/notarius`, and `git diff --check` all passed. + +## Stage 15: Restore NPC Canonical-Name Selection Semantics ### Goal -Establish the in-process contracts and deterministic identity policy on which -both lanes depend. +Restore the NPC-specific canonical display-name policy that was weakened during +the shared reconciliation migration, without changing the shared proposal +contract or any durable artifact shape. ### Work -- Extend `internal/modules/dnd/types.go` with: - - `LocationListKind` = `dnd/location-list`; - - `LocationOccurrenceListKind` = `dnd/location-occurrence-list`; - - `LocationList`, `Location`, `LocationOccurrenceList`, - `LocationOccurrence`, and `LocationOccurrenceKind`; - - exact JSON members and the four occurrence constants specified in - `location.md`. -- Add `internal/modules/dnd/locations/identity`. -- Implement display normalization and comparison normalization consistently - with the existing NPC identity policy. Share a lower-level comparison helper - only if doing so preserves NPC behavior exactly; otherwise keep the small - policy-specific function explicit. -- Implement the versioned compact-JSON-array ID derivation contract from - `location.md`, including ID syntax checks and immutable list validation. -- Identity validation must require correctly derived, unique IDs while allowing - two records to have the same comparison name when their evidence anchors - differ. -- Add focused tests for Unicode normalization, whitespace, apostrophes, - deterministic encoding, evidence ordering, same-name/different-anchor IDs, - malformed IDs, missing evidence, and non-mutation. +- Update the NPC normalization task asset under + `internal/modules/dnd/normalize/npcs/assets/prompts/` so canonical selection + again applies all of these rules: + - choose only a supplied candidate from an evidence-supported duplicate + group; + - prefer a complete, stable proper name over an abbreviation; and + - prefer an unadorned proper name over that name plus a contextual class, + role, title, or relationship descriptor unless the transcript establishes + the descriptor as part of the person's name. +- Preserve the opaque-key shared reconciliation instructions, prompt-message + ordering, response schema, retry/fallback behavior, and deterministic group + application. Do not move NPC-specific naming policy into + `shared/entityreconcile`. +- Update the canonical NPC artifact contract in + `docs/integrations/dnd-npc-artifacts.md` with the observable canonical-name + selection rule. Keep `docs/internal/dnd.md` focused on normalization + mechanics and link to the integration contract rather than maintaining a + competing definition; add or adjust a link only if needed for clarity. +- Allow the prompt fingerprint and any derived manifest/checkpoint metadata to + change through the existing asset-fingerprint mechanism. Do not manually + pin a digest. +- Retain existing behavior tests for opaque candidate selection, evidence + union, ID derivation, retry/fallback, and non-mutation. Add a test only if it + exercises deterministic code behavior; do not assert prompt prose. ### Acceptance Criteria -- The types compile without production registration. -- ID derivation exactly follows the documented five-element compact JSON input. -- Same normalized name plus different earliest evidence yields different IDs. -- Validation does not reject same-name records solely because their names - match, and it reports duplicate or mismatched IDs deterministically. -- `go test ./internal/modules/dnd/locations/... ./internal/modules/dnd/...` passes - for the packages available at this stage. - -### Prompt Size - -Small enough for one implementation prompt. - -## Stage 2: Add Durable Codecs And Schemas - -### Goal - -Create strict durable JSON ownership for both artifact kinds without exposing -either lane as a selectable pipeline yet. - -### Work - -- Add `internal/modules/dnd/codec/locations` and - `internal/modules/dnd/codec/locationoccurrences`, following the existing D&D - candidate/approved codec pattern. -- Add embedded Draft 2020-12 schemas with the IDs, names, root members, required - fields, enums, source-reference shape, and `additionalProperties: false` - contracts in `location.md`. -- Keep both schemas at `v1`. -- Support strict candidate decoding before semantic approval and strict durable - encoding/decoding after approval. -- Add representative valid fixtures and tests for schema metadata, defensive - schema bytes, empty arrays, unknown fields, missing fields, invalid types, - invalid enum values, malformed source references, invalid ID syntax, and - round trips. - -### Acceptance Criteria - -- Each codec advertises the correct artifact kind and metadata count. -- Candidate decoding preserves semantic mistakes for validators while - rejecting structurally invalid JSON. -- Approved encoding and decoding enforce the durable shape. -- `go test ./internal/modules/dnd/codec/locations/... ./internal/modules/dnd/codec/locationoccurrences/...` +- The rendered NPC normalization prompt contains one unambiguous NPC-owned + policy that distinguishes proper names from contextual descriptors. +- `Captain Aria` is not preferred over `Aria` solely because it is longer when + `Captain` is only a contextual title, while an established full name may + still outrank an abbreviation. +- No module key, schema, identity policy, response schema, validator chain, or + shared reconciliation asset changes. +- Canonical current documentation, rather than this implementation plan, owns + the restored externally observable rule. +- `go test ./internal/modules/dnd/normalize/npcs/... ./internal/modules/dnd/shared/entityreconcile/... ./internal/modules/dnd/register/...` passes offline. +- `git diff --check` reports no errors. ### Prompt Size -Small enough for one implementation prompt. +Small and suitable for one implementation prompt. -## Stage 3: Extract Shared Entity-Reconciliation Infrastructure +## Stage 16: Correct Hypothetical Location-Occurrence Classification ### Goal -Create the D&D-shared, domain-safe proposal machinery needed by both NPC and -location normalization, without changing NPC production behavior yet. +Make the location-occurrence prompt and durable documentation implement the +roadmap's distinction between an explicit speculative mention and an inferred +or unsupported occurrence. ### Work -- Add `internal/modules/dnd/shared/entityreconcile`. -- Move or generalize the reusable behavior currently owned by - `internal/modules/dnd/normalize/npcs/context_material.go` and `proposal.go`: - - assign deterministic opaque candidate keys such as `candidate-000001` in - input order; - - clone candidate names and source references; - - build bounded transcript windows in source-document order; - - omit candidates whose references cannot safely produce context; - - coalesce overlapping or adjacent windows without mutating the source; - - define the private `duplicate_groups` proposal with `members` and - `canonical` candidate keys; - - reject blank, unknown, repeated, ineligible, overlapping, too-small, or - canonical-not-a-member groups; and - - return defensive, immutable assessment data identifying only safe groups. -- Keep LLM calls, retry decisions, artifact mutation, canonical-name policy, - durable ID derivation, and warning wording out of this package. -- Add a shared prompt instruction asset that states the key-copying and - proposal-safety contract without NPC- or location-specific identity rules. -- Add a shared private structured-response schema and loader/registration - support with a stable `v1` key, ID, name, and fingerprint. Registering the - schema more than once must not be required. -- Add table-driven tests for context bounds, ordering, invalid references, - coalescing, every unsafe proposal category, non-overlapping safe groups, - deterministic keys, defensive copies, and non-mutation. +- Update the location-occurrence task asset under + `internal/modules/dnd/extract/locationoccurrences/assets/prompts/` to encode + these exact semantic outcomes: + - explicit non-actionable speculation or a mere hypothetical reference to a + registry location is `mentioned`; + - an explicit proposal, intention, or agreement to travel is `planned`; + - a mere hypothetical or speculation is not `planned`; and + - an inferred, unstated, uncertain, or unsupported place or occurrence is + omitted. +- Preserve the documented precedence `visited`, `planned`, `recalled`, then + `mentioned`, including the ability to return multiple independently + evidenced facts from one passage. +- Update the `mentioned` definition in + `docs/integrations/dnd-location-occurrence-artifacts.md` to explicitly include + non-actionable speculation. Keep durable category semantics in that + integration contract and link from internal documentation rather than + duplicating the taxonomy elsewhere. +- Do not change the occurrence enum, durable schema, normalizer, validators, or + generated-reference contract. +- Allow existing prompt fingerprinting to record the asset change. Do not add + an exact-text or exact-digest test. ### Acceptance Criteria -- The package has no dependency on `dnd.NPC`, `dnd.Location`, either - normalizer, or a concrete LLM client. -- Proposal values can identify duplicate candidates even when display names - are equal. -- The shared response contract cannot directly supply replacement records or - evidence. -- Existing NPC packages still compile before their migration. -- `go test ./internal/modules/dnd/shared/...` passes offline. - -### Prompt Size - -Medium, but coherent and suitable for one implementation prompt. Do not combine -it with the NPC migration. - -## Stage 4: Migrate NPC Normalization To The Shared Helper - -### Goal - -Make the existing NPC normalizer the first production consumer of the shared -entity-reconciliation contract while retaining its durable behavior. - -### Work - -- Refactor `internal/modules/dnd/normalize/npcs` to use opaque candidate keys, - shared context construction, shared proposal assessment, the shared response - schema, and the shared generic reconciliation instruction asset. -- Retain NPC-owned responsibilities: - - comparison-name preparation and deterministic duplicate handling; - - the NPC-specific task and canonical display-name rules; - - LLM invocation, bounded retry, fallback, warnings, and diagnostics; - - application of safe groups, evidence union, NPC ID derivation, and output - ordering. -- Remove superseded NPC-private context/proposal code and private schema assets - once no longer referenced. -- Update NPC prompt metadata and checkpoint fingerprints for the intentional - prompt/private-schema contract change. -- Preserve public module keys, durable NPC schema, identity policy, validator - chains, warning bounds, and fallback semantics. -- Test equal display names as distinct keyed candidates, alias consolidation, - rejected unsafe groups, retry exhaustion, private input ownership, redacted - errors, deterministic fallback, and non-mutation. - -### Acceptance Criteria - -- No durable NPC artifact field or module key changes. -- NPC normalization cannot confuse two candidates merely because their display - names match. -- Unsafe proposals leave a valid deterministic result and follow existing - retry/fallback policy. -- Obsolete NPC-only reconciliation helpers and schema are removed. -- `go test ./internal/modules/dnd/normalize/npcs/... ./internal/modules/dnd/shared/...` +- The prompt no longer instructs the model to omit every hypothetical + reference. +- An explicit statement such as “What if we went to Moon Gate?” is eligible as + `mentioned` when it is supported by a narrow transcript range, but is not a + `planned` occurrence without an actual proposal or intention. +- An unstated location inferred only from surrounding events remains omitted. +- The feature roadmap, prompt, and canonical integration contract agree on all + four categories and their precedence. +- `go test ./internal/modules/dnd/extract/locationoccurrences/... ./internal/modules/dnd/register/...` passes offline. +- `git diff --check` reports no errors. ### Prompt Size -Medium-to-large but bounded to one existing module. Suitable for one -implementation prompt; do not add location normalization in this stage. +Small and suitable for one implementation prompt. -## Stage 5: Implement Location Extraction +## Stage 17: Narrow Location-Occurrence Normalizer References ### Goal -Add the LLM-backed extractor that produces evidence-grounded location -candidates. +Make the deterministic normalizer advertise exactly the reference material it +uses. ### Work -- Add `internal/modules/dnd/extract/locations` following current D&D extractor - conventions: strict empty options, typed builder and registration function, - `llm_backed` execution metadata, immutable inputs, redacted errors, prompt - and response-schema fingerprints, and bounded diagnostics. -- Add a private response schema containing only `name` and source ranges; the - model must not produce durable IDs or prose. -- Compose the prompt from existing shared D&D system, identity, campaign - reference, transcript, and evidence assets plus module-owned task and - instructions. Preserve the documented extraction-message ordering and cache - controls. -- Define physical-place inclusion and conservative omission exactly as in - `location.md`, including generic labels, aliases, and nested places. -- Map source ranges to the current source ID, canonicalize exact duplicate - ranges, derive candidate location IDs in code, preserve semantically invalid - candidates for validators where safe, and return deterministic ordering. -- Add prompt-asset tests that verify shared asset reuse and rendered inputs by - behavior, without asserting exact shared-prefix length or prompt wording. -- Add extractor tests for empty output, mapping, evidence ownership, generic - same-name locations with different anchors, invalid candidate preservation, - client failures, registration metadata, fingerprints, and non-mutation. +- Change `internal/modules/dnd/normalize/locationoccurrences` so its + `ReferenceSlots` and `ModuleSpec` expose only the required `locations` slot. +- Keep that slot's existing contract unchanged: exactly one + `dnd/location-list` artifact, `application/json`, at most 1,048,576 bytes, + required at extraction and normalization. +- Remove the normalizer's unused `party`, `roster`, `players`, and `glossary` + declarations and any imports or description tables made dead by that change. + Do not remove those optional slots from the LLM-backed occurrence extractor, + which actually presents campaign material to the model. +- Add or revise focused module-contract tests to compare the normalizer's + complete declared slot set, including requiredness, media type, artifact kind, + and size. This is a stable module contract test, not a source-layout test. +- Verify that pipeline-level campaign reference defaults continue to bind only + to eligible targets and that the complete example still resolves. Current + configuration documentation already states that `locations` is required by + both occurrence stages; update it only if repository inspection finds a claim + that optional campaign references are consumed by the normalizer. +- Do not change normalization output, registry fingerprints, warnings, or + checkpoint handling for the required generated registry. ### Acceptance Criteria -- The extractor cannot manufacture source identities or accept campaign - references as evidence. -- The private model response does not contain a durable ID. -- Same-name candidates with different evidence survive extraction as distinct - candidates. -- The package is testable through its local registration but is not yet added - to the production D&D registrar. -- `go test ./internal/modules/dnd/extract/locations/...` passes offline. +- Catalog and `ModuleSpec` inspection report exactly one normalizer reference + slot: required `locations`. +- The extractor continues to accept its existing optional campaign context and + required location registry. +- Binding optional campaign references no longer implies that deterministic + occurrence normalization consumes them. +- The complete configuration and generated location handoff still resolve and + run through existing tests. +- `go test ./internal/modules/dnd/normalize/locationoccurrences/... ./internal/modules/dnd/register/... ./internal/modules/integration/... ./internal/cli/...` + passes offline. +- `git diff --check` reports no errors. ### Prompt Size -Medium and suitable for one implementation prompt. +Small and suitable for one implementation prompt. -## Stage 6: Implement Semantic Location Normalization +## Stage 18: Add Shared Registry-Resolution Infrastructure ### Goal -Add conservative alias and repeated-place reconciliation without collapsing -same-named or nested locations by default. +Introduce a focused D&D-shared primitive for the demonstrated immutable +registry-resolution mechanics, without changing any production consumer in the +same prompt. ### Work -- Add `internal/modules/dnd/normalize/locations` as an `llm_backed` normalizer - using the shared entity-reconciliation package and private response schema. -- Deterministically clone and prepare the merged candidates first: - - normalize display whitespace; - - canonicalize and deduplicate source references; - - remove only exact duplicates with the same comparison name and exact - canonical evidence; - - assign opaque reconciliation keys; and - - retain same-name records with different evidence. -- Use bounded transcript windows and a module-owned location task that permits - grouping only when evidence clearly identifies one physical place. Explicitly - prohibit grouping solely by equal names, proximity, nesting, or generic - labels. -- Validate proposals through the shared package. Apply only safe groups in - deterministic code, choose the canonical name from the selected existing - candidate, union evidence, and derive the final evidence-anchored ID. -- Retain the deterministic candidate set on unusable proposals and follow the - existing NPC retry/fallback and bounded-warning conventions. -- Publish prompt, response-schema, identity-policy, normalization-policy, and - semantic-context fingerprints. -- Test aliases, repeated appearances, same-name distinct places, parent/child - locations, invalid and overlapping proposals, proposal retries, fallback, - ordering, ID recomputation, warning bounds, idempotent deterministic - application, and non-mutation. +- Add `internal/modules/dnd/shared/registryresolver` as a domain-shared package; + do not put this helper in the generic framework. +- Provide an idiomatic generic resolver driven by callbacks or a small adapter + so it can operate on immutable domain registry views without importing NPC, + location, scene-description, or codec packages. +- The resolver core must own the behavior currently duplicated by the NPC and + location resolvers: + - validate and retain one construction-time seeded view; + - treat a declared generated-reference placeholder with zero construction- + time items as absent only during construction; + - use the seeded view when an operation does not supply the target slot; + - resolve an operation-time override when the target slot is present; + - memoize valid immutable views by a raw key and by semantic identity; + - reuse the seeded view when bound state and semantic identity match; + - synchronize concurrent resolution; and + - never retain caller-owned reference maps, item content, or mutable values. +- Raw cache identity must normalize the media-type spelling and hash the actual + item content. Do not trust a caller-supplied digest as proof that different + bytes are equal. +- Provide a narrow helper for resolving an optional single reference item that + validates the configured slot name, exact item count when bound, parsed media + type, and maximum size. Return owned bytes or require immediate decode so the + helper cannot retain caller-owned content. +- Keep artifact decoding, semantic identity validation, canonical encoding, + prompt projection, lookup indexes, clone policy, and domain-specific error + context in the owning registry packages. +- Keep errors bounded and content-free. Do not include raw reference bytes, + origin paths, or campaign data in diagnostics. +- Add focused tests for absent construction state, zero-item generated + placeholders, seeded fallback, operation overrides, malformed cardinality, + media types, size limits, raw reuse, semantic reuse, caller-supplied digest + mismatch, concurrent access, loader failure, and non-retention/non-mutation. ### Acceptance Criteria -- The model proposes groups but cannot directly replace durable locations. -- A failed or ambiguous proposal cannot lose a valid candidate. -- Same-name locations remain distinct unless an approved evidence-backed group - joins them. -- Final IDs are derived only after group evidence is unioned. -- `go test ./internal/modules/dnd/normalize/locations/...` passes offline. - -### Prompt Size - -Medium-to-large but scoped to one normalizer and suitable for one implementation -prompt. - -## Stage 7: Add The Immutable Location Registry - -### Goal - -Provide safe generated-reference resolution and an unambiguous prompt -projection for downstream occurrence extraction. - -### Work - -- Add `internal/modules/dnd/locations/registry`, modeled on the immutable NPC - registry and its operation-time resolver. -- Define `ReferenceSlot = "locations"`, a 1,048,576-byte limit, and exactly one - accepted `application/json` location-list item when bound. -- Validate durable decoding and location identity before constructing a - registry. -- Store canonical durable bytes and semantic digests without retaining mutable - caller-owned content. Return defensive copies from all accessors. -- Produce a compact, source-free prompt projection containing ordered - `{id, name}` pairs. Do not include source references or generated-reference - provenance. -- Support exact lookup by ID and verify the matching canonical name; do not - provide an ambiguous name-only lookup as the occurrence linkage mechanism. -- Preserve the established seeded/operation resolver behavior and concurrency- - safe semantic caching. -- Test absent, empty, malformed, oversized, wrong-media-type, invalid-identity, - and valid registries; projections; ID lookup; defensive copies; raw and - semantic cache reuse; and concurrent resolution. - -### Acceptance Criteria - -- Distinct same-name records are both representable and addressable by ID. -- Registry evidence cannot appear in the prompt projection. -- Malformed static references fail during construction and malformed generated - references fail at operation resolution through existing boundaries. -- `go test ./internal/modules/dnd/locations/registry/...` passes offline. - -### Prompt Size - -Medium and suitable for one implementation prompt. - -## Stage 8: Implement Location-Occurrence Extraction - -### Goal - -Add the dependent LLM-backed lane that classifies source-grounded location -occurrences. - -### Work - -- Add `internal/modules/dnd/extract/locationoccurrences` with module key - `dnd/location-occurrences`, `llm_backed` execution metadata, strict empty - options, typed construction, and a required `locations` reference slot. -- Resolve the immutable registry at construction and for each operation using - the established generated-reference pattern. -- Add a private response schema requiring `location_id`, `name`, `kind`, and - source ranges. Restrict kinds to `visited`, `planned`, `recalled`, and - `mentioned`. -- Reuse the shared D&D extraction prompt assets and ordering. Place the compact - location registry after the shared transcript/evidence material and before - module task/instructions, consistent with current generated grounding. -- Encode the exact classification rules, precedence, multi-fact behavior, and - conservative omission policy from `location.md`. -- Map evidence only to the current source. Copy candidate IDs and names without - silently repairing unknown or mismatched values so deterministic validators - retain ownership of those diagnostics. -- Canonically order output and exact duplicates without dropping distinct - kinds or independent evidence. -- Test every kind, precedence, multiple supported facts, no-location and no- - occurrence outputs, required registry failures, same-name ID selection, - source-free prompt projection, current-transcript evidence, prompt/profile - metadata, client failures, non-mutation, and local registration. - -### Acceptance Criteria - -- Construction and operation specs declare `locations` as required and accept - only `dnd/location-list` JSON. -- The model sees IDs and names but no registry evidence. -- Registry context never becomes occurrence evidence. -- The extractor remains locally testable but is not production-selectable yet. -- `go test ./internal/modules/dnd/extract/locationoccurrences/...` passes +- The shared package depends only on generic source/framework contracts and Go + standard-library packages, not concrete D&D artifact or codec packages. +- The helper owns only demonstrated resolution, validation, caching, and + ownership mechanics; it does not become a universal artifact registry. +- Invalid values and loader errors are never cached. +- Concurrent callers resolving identical bytes receive the same prepared + immutable view without races. +- Existing production registry packages remain unchanged in this stage. +- `go test -race ./internal/modules/dnd/shared/registryresolver/...` passes offline. +- `git diff --check` reports no errors. ### Prompt Size -Medium and suitable for one implementation prompt. +Medium and suitable for one implementation prompt. Keep production migrations +out of this stage so the generic ownership and concurrency contract can be +reviewed independently. -## Stage 9: Implement Deterministic Occurrence Normalization +## Stage 19: Migrate The NPC Registry To The Shared Resolver ### Goal -Canonicalize occurrence records against the exact location registry without a -second LLM call. +Make the established NPC registry the first production consumer of the shared +resolver while preserving all public and operational behavior. ### Work -- Add `internal/modules/dnd/normalize/locationoccurrences` as a deterministic - normalizer with the same required `locations` slot. -- Clone all inputs. Normalize source ranges, names, ordering, and exact - duplicates. -- For a known `location_id`, replace display-name variation with the registry's - exact canonical name. Do not perform a name-only guess. -- Preserve an unknown ID or otherwise invalid record for validator diagnostics - and emit bounded warnings where current D&D normalizer conventions require - them. -- Sort using the complete order defined in `location.md`. -- Publish normalization and registry-projection fingerprints consistent with - the other registry-backed normalizers. -- Test all kind values, canonical name replacement, same-name distinct IDs, - exact-duplicate removal, distinct evidence retention, stable ordering, - unknown IDs, malformed registry resolution, warnings, idempotence, and - non-mutation. +- Refactor `internal/modules/dnd/npcs/registry` to compose the Stage 18 helper + for construction/operation selection, single-item validation, raw caching, + semantic caching, and synchronization. +- Preserve the existing package API used by extractors, normalizers, + validators, manifests, and tests, including `NewResolver`, `Resolve`, + `Seeded`, `Bound`, `NPCs`, `List`, `CanonicalBytes`, `Digest`, + `ProjectionDigest`, `Count`, `PromptInput`, `CanonicalName`, and `Matches`. +- Keep NPC-owned behavior in the NPC package: + - strict approved NPC codec use; + - NPC identity validation; + - canonical durable JSON and names-only prompt projection; + - comparison-key lookup; + - defensive NPC/source-reference cloning; and + - NPC-specific redacted error context. +- Preserve the exact empty projection, semantic and projection digest behavior, + seeded/generated resolution semantics, manifest metadata, checkpoint + fingerprints, and source-free prompt input. +- Remove superseded NPC-private resolver/cache/media/digest code only after the + shared helper supplies the same behavior. Do not migrate locations or scene + descriptions in this stage. +- Update focused tests as needed to verify behavior through the NPC package, + including malformed static and operation-time references, raw/semantic cache + reuse, concurrent resolution, defensive copies, prompt projection, and + downstream grounding. ### Acceptance Criteria -- No LLM client or prompt assets are required. -- Canonicalization is exclusively ID-based. -- Invalid records are not silently redirected to a different location. -- `go test ./internal/modules/dnd/normalize/locationoccurrences/...` passes - offline. - -### Prompt Size - -Small enough for one implementation prompt. - -## Stage 10: Add Location Registry Validators - -### Goal - -Give `dnd/location-list` the complete validator ownership expected of a -production D&D artifact. - -### Work - -- Add location validator packages under - `internal/modules/dnd/validate/locations` for: - - extraction shape; - - source-reference bounds/current-chunk ownership; - - advisory source relatedness; and - - normalized identity derivation and ID uniqueness. -- Use shared D&D citation, unit-reference, diagnostic, and matching helpers - where their contracts apply. -- The identity validator must allow repeated comparison names and validate the - evidence-anchored derivation policy instead of importing NPC uniqueness - assumptions. -- Keep diagnostics indexed, aggregated, bounded, stable, and free of raw prompt - or reference content. -- Add tests for accepted values, every owned failure, same-name distinct - locations, malformed/unreadable citations, advisory relatedness, bounds, - registration metadata, fingerprints, nil safety where applicable, and - non-mutation. - -### Acceptance Criteria - -- Validator responsibilities do not overlap merely to increase test coverage. -- Relatedness remains advisory and uses only cited current-transcript text. -- Validators do not repair or mutate artifacts. -- `go test ./internal/modules/dnd/validate/locations/...` passes offline. - -### Prompt Size - -Medium and suitable for one implementation prompt. - -## Stage 11: Add Location-Occurrence Validators - -### Goal - -Give `dnd/location-occurrence-list` complete structural, registry, ordering, -evidence, and advisory validation. - -### Work - -- Add validator packages under - `internal/modules/dnd/validate/locationoccurrences` for: - - extraction shape and supported kinds; - - required registry membership and exact `location_id`/`name` pairing; - - normalized ordering and exact-duplicate invariants; - - source-reference bounds/current-chunk ownership; and - - advisory source relatedness. -- Reuse the immutable location resolver rather than decoding caller-owned - references independently in each validator. -- Ensure same-name registry records remain distinguishable by ID. -- Keep registry context out of evidence checks. -- Add focused tests for each kind, unknown IDs, mismatched names, same-name - locations, ordering, duplicates, malformed required references, invalid - evidence, advisory diagnostics, registration metadata, fingerprints, - diagnostic bounds, and non-mutation. - -### Acceptance Criteria - -- An ID/name mismatch is rejected even when another registry record has the - supplied name. -- Missing or malformed required registry references fail at the established - boundary. -- Validators remain deterministic and do not alter occurrence records or the - registry. -- `go test ./internal/modules/dnd/validate/locationoccurrences/...` passes - offline. - -### Prompt Size - -Medium and suitable for one implementation prompt. - -## Stage 12: Compose The Production D&D Family - -### Goal - -Make both lanes selectable as one coherent production addition after all -component contracts are present. - -### Work - -- Extend `internal/modules/dnd/register` to register, in dependency-safe order: - - both codecs; - - both extractors; - - typed append-order mergers; - - the LLM-backed location normalizer; - - the deterministic occurrence normalizer; - - all validators; - - shared reconciliation schema assets and both new prompt manifests; - - evidence projectors; and - - extract and normalize default validator chains. -- Ensure both LLM-backed modules select the maintained `dnd-extraction` - fallback profile and inherit the existing profile policy. -- Define chain order consistently with existing D&D artifacts: generic JSON, - shape, registry/identity or normalized invariants at the appropriate stage, - source references, durable JSON Schema, then advisory relatedness. -- Update registrar tests for artifact kinds, keys, execution classes, reference - slot requirements, builder construction, assets, profile use, evidence - projection, chain contents/order, duplicate registration, and failure - propagation. -- Update any integration-level artifact-kind allowlists or typed registries - required by the framework; do not add module-specific orchestration logic. - -### Acceptance Criteria - -- One D&D registration call exposes both complete lanes and no partial - registration succeeds silently. -- Catalog inspection reports correct artifact kinds, stages, execution classes, - reference slots, profiles, and fingerprints. -- Both artifact kinds support evidence projection without registry evidence - leakage. -- `go test ./internal/modules/dnd/register/... ./internal/modules/dnd/...` passes - offline. - -### Prompt Size - -Medium-to-large but limited to composition and suitable for one implementation -prompt. - -## Stage 13: Add Maintained Pipeline And Handoff Coverage - -### Goal - -Exercise the feature through real configuration, ordered generated references, -chunk operations, acceptance gates, and durable output. - -### Work - -- Update `examples/dnd-complete.config.yml`: - - add `locations` to the first descriptive step with extract, append-order - merge, and LLM-backed normalize bindings; - - add a generated `locations` reference to the next step; - - add `location-occurrences` to that step with extract, append-order merge, - and deterministic normalize bindings; and - - add both lanes to evidence-context output where appropriate. -- Keep the minimal example minimal unless its stated purpose requires a - location lane; do not turn it into a second complete example. -- Extend maintained example-loading/config-validation tests. -- Add integration tests that prove: - - the normalized accepted registry is handed off in memory; - - the occurrence lane cannot run before its producer; - - missing, cyclic, wrong-kind, wrong-media-type, rejected, or unaccepted - producers are rejected at the existing boundaries; - - same-name locations remain distinguishable by ID through the handoff; - - registry evidence never becomes occurrence evidence; - - retries and checkpoints honor prompt, schema, identity, and generated- - reference fingerprints; and - - output contains both durable artifact envelopes and evidence context. -- Use recording/fake structured clients only; no network-dependent tests. - -### Acceptance Criteria - -- The complete example loads through the real config path and exercises all - registered D&D lanes. -- Ordered handoff failure semantics match the framework's existing fail-whole- - run policy. -- Integration tests cover behavior rather than duplicating package internals. -- `go test ./internal/modules/integration/... ./internal/config/...` and any - example-specific test targets pass offline. - -### Prompt Size - -Medium-to-large but coherent as one end-to-end integration prompt. - -## Stage 14: Publish Current Documentation And Perform Final Verification - -### Goal - -Make the implemented feature discoverable and retire fulfilled future-work -language without leaving development-history documentation behind. - -### Work - -- Create canonical integration contracts: - - `docs/integrations/dnd-location-artifacts.md`; - - `docs/integrations/dnd-location-occurrence-artifacts.md`. -- Update `docs/config.md` with both selectable keys, the `locations` reference - slot and compatibility, execution classes, validators, default chains, and - the complete-example link. -- Update `docs/internal/dnd.md` with nine-lane composition, shared entity - reconciliation, evidence-anchored identity, occurrence grounding, prompt - asset reuse, and intentional lane differences. Link to integration contracts - instead of duplicating their JSON shapes. -- Update `docs/integrations/json-output.md`, `README.md`, and other current - canonical inventories only where repository inspection shows that the new - artifact kinds or maintained example must be listed. -- Remove the fulfilled `Location Extraction` section from - `docs/roadmap/future.md`. Keep the generic LLM-assisted deduplication item and - clarify only if needed that the new D&D helper does not fulfill that broader - feature. -- Verify all relative links and search current documentation for stale - seven-lane counts, missing keys, obsolete location-planning claims, and - accidental claims that references are evidence. -- Run formatting, focused tests, the full Go test suite, static analysis, and - repository-provided config/example checks. Inspect `git diff --check` and - confirm no unrelated files changed. -- After implementation and verification are complete, leave `location.md` and - this plan in place for the user's separate roadmap-retirement step; do not - delete them unless explicitly asked. - -### Acceptance Criteria - -- Current documentation describes the implemented contracts and only - implemented behavior outside `docs/roadmap/`. -- The roadmap no longer presents completed location tracking as future work. -- Links, examples, module inventories, and default-chain tables agree with - production registration. -- `go test ./internal/modules/dnd/...` passes. -- `go test ./internal/modules/integration/...` passes. -- `go test ./...` passes. -- `go vet ./...` passes. -- Repository-provided configuration/example validation passes. +- Existing NPC consumers compile without API changes. +- NPC registry output bytes, prompt projection, lookup behavior, digests, + reference limits, errors, manifests, and checkpoints retain their established + semantics. +- The NPC package no longer contains a second implementation of resolver + selection, caching, or synchronization. +- `go test -race ./internal/modules/dnd/npcs/registry/... ./internal/modules/dnd/extract/npcinteractions/... ./internal/modules/dnd/normalize/npcinteractions/... ./internal/modules/dnd/validate/npcinteractions/...` + passes offline. +- `go test ./internal/modules/dnd/register/...` passes offline. - `git diff --check` reports no errors. ### Prompt Size Medium and suitable for one implementation prompt. +## Stage 20: Migrate The Location Registry To The Shared Resolver + +### Goal + +Remove the newly introduced NPC/location resolver duplication while preserving +the location identity and generated-handoff contracts. + +### Work + +- Refactor `internal/modules/dnd/locations/registry` to compose the Stage 18 + helper for the same resolver, reference-item validation, cache, and + synchronization responsibilities migrated in Stage 19. +- Preserve the existing package API, including `NewResolver`, `Resolve`, + `Seeded`, `Bound`, `Locations`, `List`, `CanonicalBytes`, `Digest`, + `ProjectionDigest`, `Count`, `PromptInput`, `Lookup`, and `Matches`. +- Keep location-owned behavior in the location package: + - strict approved location codec use; + - evidence-anchored identity validation; + - canonical durable JSON and ordered source-free `{id, name}` projection; + - exact ID lookup and exact ID/name matching; + - defensive location/source-reference cloning; and + - location-specific redacted error context. +- Preserve absent/empty/bound distinctions, same-name locations with distinct + IDs, seeded/generated resolution, semantic and projection digests, manifest + metadata, checkpoint fingerprints, and the 1 MiB limit. +- Remove superseded location-private resolver/cache/media/digest code only + after parity is established. +- Do not migrate the scene-description eligibility registry in this stage. It + has a different projection and matching contract; including it is not needed + to eliminate the exact NPC/location duplication identified by the audit. +- Update focused and integration tests as needed for static and generated + references, malformed inputs, raw/semantic reuse, concurrency, defensive + copies, source-free projections, same-name ID lookup, normalizer grounding, + validator grounding, and handoff checkpoint invalidation. + +### Acceptance Criteria + +- Existing location consumers compile without API changes. +- Location registry output bytes, projection, identity validation, lookup, + digests, reference errors, manifests, and checkpoints retain their + established semantics. +- NPC and location registries share one implementation of construction/ + operation resolution, raw and semantic caching, and synchronization. +- No registry evidence or provenance appears in the occurrence prompt + projection. +- `go test -race ./internal/modules/dnd/locations/registry/... ./internal/modules/dnd/extract/locationoccurrences/... ./internal/modules/dnd/normalize/locationoccurrences/... ./internal/modules/dnd/validate/locationoccurrences/... ./internal/modules/integration/...` + passes offline. +- `git diff --check` reports no errors. + +### Prompt Size + +Medium and suitable for one implementation prompt. + +## Stage 21: Final Regression And Documentation Verification + +### Goal + +Verify the remediations as one coherent production state without expanding the +feature or prematurely retiring its roadmap documents. + +### Work + +- Run `gofmt` on all changed Go files and inspect the final diff for unrelated + changes, stale private resolver implementations, unused imports, and + accidental artifact-contract changes. +- Re-read the affected prompt assets and canonical integration documents to + verify that: + - NPC canonical-name selection retains the proper-name/title distinction; + - explicit non-actionable speculation maps to `mentioned`; + - inferred, uncertain, and unsupported occurrences remain omitted; and + - prompt changes did not reorder or duplicate shared cacheable prompt assets. +- Inspect module catalog output or registration tests to confirm the + location-occurrence normalizer declares only `locations`, while its extractor + retains the intended optional campaign slots. +- Confirm the shared resolver has exactly the NPC and location consumers + planned here and that domain-specific codecs, identity, projections, lookups, + and error context remain in their owning packages. +- Run the maintained complete example/configuration loading path and all + repository-wide validation. +- Leave `docs/roadmap/location.md` and this implementation plan in place for the + user's separate roadmap-retirement step. Do not delete them in this stage. + +### Acceptance Criteria + +- `go test ./internal/modules/dnd/...` passes. +- `go test ./internal/modules/integration/... ./internal/cli/...` passes. +- `go test -race ./internal/modules/dnd/shared/registryresolver/... ./internal/modules/dnd/npcs/registry/... ./internal/modules/dnd/locations/registry/...` passes. +- `go test ./...` passes. +- `go vet ./...` passes. +- `go build ./cmd/notarius` passes. +- Maintained configuration/example validation passes through its repository + test target. +- `git diff --check` reports no errors. +- Current documentation contains no stale contradiction about hypothetical + location mentions or NPC canonical-name selection. +- The worktree contains only the intended remediation changes and the roadmap + documents remain available for later retirement. + +### Prompt Size + +Small-to-medium and suitable for one implementation prompt. + ## Open Questions -None. The feature roadmap fixes the artifact shapes, identity scope and -derivation, occurrence categories, classification precedence, reference -dependency, reconciliation safety boundary, pipeline placement, and non-goals -needed to implement every stage without an additional product decision. +None. The feature roadmap and audit findings determine the required behavioral +semantics, module reference contract, abstraction boundary, migration order, +and validation expectations. The scene-description eligibility registry is +explicitly outside this remediation because its projection and matching +semantics differ from the duplicated NPC/location grounding registries. diff --git a/internal/modules/dnd/normalize/npcs/assets/prompts/task.md b/internal/modules/dnd/normalize/npcs/assets/prompts/task.md index 6063b29..a7482f4 100644 --- a/internal/modules/dnd/normalize/npcs/assets/prompts/task.md +++ b/internal/modules/dnd/normalize/npcs/assets/prompts/task.md @@ -3,6 +3,11 @@ that refer to the same individual. Propose only groups supported by the transcript, and preserve distinct individuals even when their names are similar. -For every accepted group, choose as canonical the supplied candidate whose -display name is the most complete and clear NPC name. Do not invent, edit, or -combine display names. +For every accepted group, choose as canonical only a supplied candidate from +that evidence-supported duplicate group. Prefer a complete, stable proper name +over an abbreviation. Prefer an unadorned proper name over that name plus a +contextual class, role, title, or relationship descriptor unless the transcript +establishes the descriptor as part of the person's name. A longer display name +is not inherently more canonical; for example, do not prefer `Captain Aria` +over `Aria` solely because it includes the contextual title `Captain`. Do not +invent, edit, or combine display names.