notarius/docs/roadmap/dnd-npc-semantic-normalization.md

D&D NPC Semantic Normalization

Status

Accepted scope; not implemented.

Purpose

Reconcile NPC records that extraction produced under different display names when the complete transcript establishes that they represent the same individual. This work addresses identities split across scene or chunk boundaries while preserving the existing rule that extraction records narrow source evidence and normalization owns document-wide reconciliation.

The model should make only the semantic identity judgment. Deterministic code must continue to own identity derivation, proposal validation, artifact mutation, evidence preservation, ordering, diagnostics, and final validation.

Target Behavior

The target dnd/npcs normalization contract combines:

• the current deterministic display-name normalization, stable-ID derivation, source-reference canonicalization, and equal-comparison-key consolidation;
• one LLM determination of whether remaining, distinctly named NPC records represent the same individual and which existing display name is canonical; and
• deterministic proposal validation and application followed by the configured normalize validator chain.

Each configured normalization attempt should make at most one semantic pass over the merged document-level NPC list, not one pass per extraction chunk or candidate pair. The pass should be skipped when fewer than two distinct candidate identities remain after deterministic preprocessing.

False consolidation is more damaging than a missed consolidation. Prompt policy should therefore require affirmative contextual evidence that names identify the same individual and should prefer no group when identity remains ambiguous.

Canonical selection should favor the most complete stable proper name supported by the transcript. A complete proper name is preferable to an abbreviation, while an unadorned proper name is preferable to the same name plus a contextual class, role, title, or relationship descriptor unless the transcript establishes that descriptor as part of the character's name. The model must still select one supplied display name rather than synthesize a better one.

Model Proposal Contract

The private structured response should contain only proposed duplicate groups:

{
  "duplicate_groups": [
    {
      "members": [
        "Billy",
        "Billy the druid"
      ],
      "canonical_name": "Billy"
    }
  ]
}

members identifies candidates by their supplied display names. canonical_name selects one existing member as the retained display name. An empty duplicate_groups array is a valid determination that no sufficiently supported duplicates exist.

The model must not receive or return deterministic npc:sha256: IDs. Those IDs are long, non-semantic implementation identities and remain exclusively owned by deterministic code. The model must not invent a replacement name, rewrite an NPC record, propose new source references, or return a complete replacement artifact.

Before the call, deterministic preprocessing ensures that each supplied display name has a unique NPC identity comparison key. Returned names may resolve using the existing comparison-key equivalences for case, whitespace, Unicode compatibility, and supported apostrophes. Resolution must not use fuzzy, substring, edit-distance, embedding, or other approximate matching.

An applicable group must:

• contain at least two distinct, known members;
• resolve every member uniquely against the supplied candidate set;
• select a known canonical_name that belongs to the group;
• contain no repeated member; and
• share no resolved member with any other proposed group.

Unknown or ambiguous names, singleton groups, invalid canonical selections, and other unsafe groups must not mutate the artifact. Groups are otherwise independent: a locally valid group may be applied when none of its resolved members appears in any other proposed group.

Overlap is a proposal conflict even when one participating group is already locally invalid. Deterministic code must discard every group in the connected conflict set rather than selecting a winner by response order. Locally valid, non-conflicting groups remain safe to apply. This permits useful partial reconciliation without allowing an unsafe group to influence an NPC identity that another group would mutate.

Retry And Safe Fallback

An invalid or partially unsafe private proposal is a retryable normalization attempt. This includes:

• structurally invalid model output classified by the LLM boundary as an invalid structured completion; and
• a structurally valid response containing any locally invalid or conflicting group.

The framework owns retry counting and attempt diagnostics. The normalizer returns a safe candidate together with a bounded, content-safe retry diagnostic. For structurally invalid output, that candidate is the deterministic pre-LLM result. For a decoded proposal with unsafe groups, it also includes every independently valid, non-conflicting group from that attempt. When normalize retries remain, the framework invokes the normalizer again from the same merged input; safe groups are not accumulated across attempts. A later completely safe proposal is applied normally.

When the configured attempt budget is exhausted, the framework validates and accepts the final attempt's safe fallback instead of failing or rejecting the NPC lane, provided that fallback passes the configured normalize validators. It promotes one durable warning stating that one or more proposed groups were omitted. The fallback may therefore contain a safe partial reconciliation, or only the deterministic pre-LLM result when no group could be applied. Earlier retry diagnostics, candidate values, and warnings remain attempt-local in debug artifacts.

The normal module-binding default remains retries: 0, meaning one total normalization attempt and immediate fallback after its invalid proposal. Retries occur only when the user configures a positive normalize retry count. For example, retries: 2 permits the initial proposal plus two additional attempts before fallback.

Transport, authentication, prompt-preparation, cancellation, input-encoding, and other operational errors are not safe-proposal failures. They retain the existing module/framework error behavior rather than being converted into an accepted fallback.

Transcript Context Policy

The model should receive every candidate's display name and source references, together with transcript windows derived from those references. It should not receive the complete transcript by default.

Each window must contain:

• the complete inclusive source range cited by the NPC record;
• up to two source units immediately before the cited range; and
• up to two source units immediately after the cited range.

The surrounding-unit count is defined once as a named module policy constant with value 2; it is not a user-accessible configuration field. Window construction consumes that value through one clear boundary so a later configuration option can replace the fixed value without changing prompt or reconciliation contracts. The context-radius policy participates in module metadata and checkpoint identity so changing it invalidates incompatible normalize checkpoints.

The value was selected by reviewing the July 19 evaluation transcript. Its NPC citations are generally self-contained; where additional context is useful, two surrounding units capture the relevant question-and-answer exchange, speaker transition, or short anaphora chain. A third unit frequently begins a separate joke or table exchange and adds distraction without improving the identity evidence.

Window expansion must use source-document positions rather than arithmetic on unit IDs. It must clamp at document boundaries, preserve document order, and coalesce overlapping or adjacent expanded windows without duplicating units. The prepared model input must distinguish originally cited units from surrounding context.

Only records with a non-empty NPC identity comparison key and a non-empty source-reference collection that is wholly valid against the current source document are eligible for semantic reconciliation. Ineligible records remain unchanged so the configured deterministic validators retain ownership of their rejection. They are not supplied to the model and cannot participate in a proposed group.

Surrounding units inform the semantic decision but do not automatically become durable NPC evidence. An applied group unions only the source references already present on its member records. The LLM cannot add references from the context window.

Full-transcript mode and a configurable context radius may be evaluated later. They are not part of this scope. Provider prompt-cache behavior should be measured rather than assumed before expanding context solely for cache economics.

Deterministic Application

For each approved group, deterministic code should:

• retain the model-selected existing display name;
• union source references from every group member;
• canonicalize and exact-deduplicate those references in source-document order;
• derive the resulting stable NPC ID from the retained display name under the existing NPC identity policy; and
• emit one bounded duplicate-collapse warning describing the applied group.

The consolidated record should occupy the earliest original member position so model response ordering cannot reorder the artifact. Unrelated records must remain present and retain their relative order. Every input record must be represented by exactly one output record, either unchanged or through one approved consolidation.

The existing NPC shape, identity, source-reference, schema, and relatedness validators remain the final artifact boundary. No LLM-backed validator is needed: model judgment occurs in the normalizer, while deterministic validators continue to enforce the durable artifact contract.

Prompt, Provenance, And Diagnostics

NPC semantic-normalization instructions and the private response schema should be module-owned prompt assets. Stable D&D-wide identity or transcript guidance may be reused through the existing shared prompt-asset mechanism where its meaning is genuinely common.

The normalizer should use the configured normalize-stage LLM profile and the application-wide scheduled LLM client. Invalid structured output and unsafe semantic proposals use the framework-owned retryable-fallback contract; operational failures retain existing error semantics.

Manifest metadata and component checkpoint fingerprints should identify the semantic-normalization policy, prompt identity, private response-schema identity, deterministic NPC identity policy, and evidence-context policy. They must not contain transcript text, NPC names, source paths, raw model responses, or other source content.

Warnings and preparation or runtime errors must follow the established bounded diagnostic and content-safety policies. Debug artifacts may retain the normal attempt-local model request, response, proposal decisions, and warnings under the existing debug sensitivity contract.

Configuration And Documentation

The maintained complete D&D configuration should demonstrate a normalize-stage LLM profile and an explicit positive retry count for the NPC lane. The application-wide default remains zero additional retries. No new configuration field is introduced for the context radius in this scope.

Canonical documentation ownership is:

• Configuration owns the LLM-backed dnd/npcs selection and its use of the existing normalize binding's profile and retry fields.
• Operations owns the document-level semantic reconciliation call and its checkpoint behavior.
• Pipeline internals own the provider-neutral invalid-structured-output classification and normalize retryable-fallback contract.
• The NPC integration contract owns externally observable consolidation, evidence, ordering, and warning behavior.
• Internal LLM and module documentation own the proposal boundary, context-window construction, deterministic application, metadata, and fingerprints.
• The broader generic LLM-assisted deduplication item in future.md remains future work until another artifact demonstrates that extracting a shared generic facility is worthwhile.

Quality Expectations

Tests should protect the semantic and safety boundaries through deterministic LLM fakes rather than live-provider calls. Coverage should demonstrate:

• distinct display variants can be consolidated when the model proposes a valid group;
• model-facing requests and responses use display names rather than stable hash IDs;
• the selected existing canonical name controls ID derivation while all member evidence is preserved;
• window construction uses source-document position, handles document edges, coalesces overlap, and distinguishes cited evidence from context;
• no surrounding context is promoted into durable source references;
• empty proposals and fewer-than-two-candidate inputs preserve deterministic normalization behavior;
• unknown, ambiguous, repeated, overlapping, and otherwise malformed semantic groups cannot corrupt or reorder the artifact;
• locally valid groups are applied independently, while every group that participates in a resolved-member conflict is discarded;
• an invalid proposal consumes only configured retry budget, a later valid proposal can succeed, and exhaustion accepts the final attempt's safe fallback with one durable warning;
• the default zero-retry binding makes exactly one proposal attempt before fallback;
• warnings and failures remain bounded and do not expose transcript content;
• prompt, schema, policy, or context-policy changes invalidate relevant checkpoint reuse; and
• an assembled ordered D&D pipeline supplies the reconciled NPC registry to downstream consumers.

Prompt tests should assert prepared message structure, supplied materials, and cache-boundary behavior. They must not act as change detectors for particular words or phrases in natural-language prompt text.

Human review of representative transcripts should compare missed and false consolidations, latency, input-token cost, and provider cache use. Probabilistic model quality is an evaluation activity, not a deterministic CI assertion.

Non-Goals

This scope does not:

• change the durable NPC artifact schema;
• add a durable alias collection or identity history;
• expose the context radius or full-transcript selection as configuration;
• ask an LLM to read, reproduce, or derive stable NPC IDs;
• permit model-authored source references or arbitrary replacement records;
• add fuzzy deterministic name matching;
• add an LLM-backed normalize validator;
• change the global default retry count;
• reconcile NPCs concurrently across lanes or runs;
• add canonical NPC IDs to downstream artifact schemas;
• implement a general DAG or implicit cross-lane dependency; or
• implement the generic cross-artifact deduplication normalizer described in future.md.

Completion Criteria

The scope is complete when dnd/npcs can use one document-level LLM proposal to reconcile differently named records conservatively, deterministic code validates and independently applies only safe non-conflicting name-based groups, the resulting NPC preserves all member evidence under a newly derived stable ID, the two-unit context policy is centralized and fingerprinted, invalid or partially unsafe proposals use configured framework retries and then an accepted safe fallback, downstream ordered steps receive the reconciled registry, and the canonical current-behavior documentation reflects the implemented contract.