notarius/docs/roadmap/dnd-npc-semantic-normalization.md

# D&D NPC Semantic Normalization

## Status

Accepted scope; not implemented.

## Purpose

Reconcile NPC records that extraction produced under different display names
when the complete transcript establishes that they represent the same
individual. This work addresses identities split across scene or chunk
boundaries while preserving the existing rule that extraction records narrow
source evidence and normalization owns document-wide reconciliation.

The model should make only the semantic identity judgment. Deterministic code
must continue to own identity derivation, proposal validation, artifact
mutation, evidence preservation, ordering, diagnostics, and final validation.

## Target Behavior

The target `dnd/npcs` normalization contract combines:

- the current deterministic display-name normalization, stable-ID derivation,
  source-reference canonicalization, and equal-comparison-key consolidation;
- one LLM determination of whether remaining, distinctly named NPC records
  represent the same individual and which existing display name is canonical;
  and
- deterministic proposal validation and application followed by the configured
  normalize validator chain.

Each configured normalization attempt should make at most one semantic pass
over the merged document-level NPC list, not one pass per extraction chunk or
candidate pair. The pass should be skipped when fewer than two distinct
candidate identities remain after deterministic preprocessing.

False consolidation is more damaging than a missed consolidation. Prompt
policy should therefore require affirmative contextual evidence that names
identify the same individual and should prefer no group when identity remains
ambiguous.

Canonical selection should favor the most complete stable proper name supported
by the transcript. A complete proper name is preferable to an abbreviation,
while an unadorned proper name is preferable to the same name plus a contextual
class, role, title, or relationship descriptor unless the transcript establishes
that descriptor as part of the character's name. The model must still select
one supplied display name rather than synthesize a better one.

## Model Proposal Contract

The private structured response should contain only proposed duplicate groups:

```json
{
  "duplicate_groups": [
    {
      "members": [
        "Billy",
        "Billy the druid"
      ],
      "canonical_name": "Billy"
    }
  ]
}
```

`members` identifies candidates by their supplied display names.
`canonical_name` selects one existing member as the retained display name. An
empty `duplicate_groups` array is a valid determination that no sufficiently
supported duplicates exist.

The model must not receive or return deterministic `npc:sha256:` IDs. Those IDs
are long, non-semantic implementation identities and remain exclusively owned
by deterministic code. The model must not invent a replacement name, rewrite
an NPC record, propose new source references, or return a complete replacement
artifact.

Before the call, deterministic preprocessing ensures that each supplied display
name has a unique NPC identity comparison key. Returned names may resolve using
the existing comparison-key equivalences for case, whitespace, Unicode
compatibility, and supported apostrophes. Resolution must not use fuzzy,
substring, edit-distance, embedding, or other approximate matching.

An applicable group must:

- contain at least two distinct, known members;
- resolve every member uniquely against the supplied candidate set;
- select a known `canonical_name` that belongs to the group;
- contain no repeated member; and
- share no resolved member with any other proposed group.

Unknown or ambiguous names, singleton groups, invalid canonical selections, and
other unsafe groups must not mutate the artifact. Groups are otherwise
independent: a locally valid group may be applied when none of its resolved
members appears in any other proposed group.

Overlap is a proposal conflict even when one participating group is already
locally invalid. Deterministic code must discard every group in the connected
conflict set rather than selecting a winner by response order. Locally valid,
non-conflicting groups remain safe to apply. This permits useful partial
reconciliation without allowing an unsafe group to influence an NPC identity
that another group would mutate.

## Retry And Safe Fallback

An invalid or partially unsafe private proposal is a retryable normalization
attempt. This includes:

- structurally invalid model output classified by the LLM boundary as an
  invalid structured completion; and
- a structurally valid response containing any locally invalid or conflicting
  group.

The framework owns retry counting and attempt diagnostics. The normalizer
returns a safe candidate together with a bounded, content-safe retry
diagnostic. For structurally invalid output, that candidate is the
deterministic pre-LLM result. For a decoded proposal with unsafe groups, it
also includes every independently valid, non-conflicting group from that
attempt. When normalize retries remain, the framework invokes the normalizer
again from the same merged input; safe groups are not accumulated across
attempts. A later completely safe proposal is applied normally.

When the configured attempt budget is exhausted, the framework validates and
accepts the final attempt's safe fallback instead of failing or rejecting the
NPC lane, provided that fallback passes the configured normalize validators.
It promotes one durable warning stating that one or more proposed groups were
omitted. The fallback may therefore contain a safe partial reconciliation, or
only the deterministic pre-LLM result when no group could be applied. Earlier
retry diagnostics, candidate values, and warnings remain attempt-local in
debug artifacts.

The normal module-binding default remains `retries: 0`, meaning one total
normalization attempt and immediate fallback after its invalid proposal.
Retries occur only when the user configures a positive normalize retry count.
For example, `retries: 2` permits the initial proposal plus two additional
attempts before fallback.

Transport, authentication, prompt-preparation, cancellation, input-encoding,
and other operational errors are not safe-proposal failures. They retain the
existing module/framework error behavior rather than being converted into an
accepted fallback.

## Transcript Context Policy

The model should receive every candidate's display name and source references,
together with transcript windows derived from those references. It should not
receive the complete transcript by default.

Each window must contain:

- the complete inclusive source range cited by the NPC record;
- up to two source units immediately before the cited range; and
- up to two source units immediately after the cited range.

The surrounding-unit count is defined once as a named module policy constant
with value `2`; it is not a user-accessible configuration field. Window
construction consumes that value through one clear boundary so a later
configuration option can replace the fixed value without changing prompt or
reconciliation contracts. The context-radius policy participates in module
metadata and checkpoint identity so changing it invalidates incompatible
normalize checkpoints.

The value was selected by reviewing the July 19 evaluation transcript. Its NPC
citations are generally self-contained; where additional context is useful,
two surrounding units capture the relevant question-and-answer exchange,
speaker transition, or short anaphora chain. A third unit frequently begins a
separate joke or table exchange and adds distraction without improving the
identity evidence.

Window expansion must use source-document positions rather than arithmetic on
unit IDs. It must clamp at document boundaries, preserve document order, and
coalesce overlapping or adjacent expanded windows without duplicating units.
The prepared model input must distinguish originally cited units from
surrounding context.

Only records with a non-empty NPC identity comparison key and a non-empty
source-reference collection that is wholly valid against the current source
document are eligible for semantic reconciliation. Ineligible records remain
unchanged so the configured deterministic validators retain ownership of their
rejection. They are not supplied to the model and cannot participate in a
proposed group.

Surrounding units inform the semantic decision but do not automatically become
durable NPC evidence. An applied group unions only the source references
already present on its member records. The LLM cannot add references from the
context window.

Full-transcript mode and a configurable context radius may be evaluated later.
They are not part of this scope. Provider prompt-cache behavior should be
measured rather than assumed before expanding context solely for cache
economics.

## Deterministic Application

For each approved group, deterministic code should:

- retain the model-selected existing display name;
- union source references from every group member;
- canonicalize and exact-deduplicate those references in source-document order;
- derive the resulting stable NPC ID from the retained display name under the
  existing NPC identity policy; and
- emit one bounded duplicate-collapse warning describing the applied group.

The consolidated record should occupy the earliest original member position so
model response ordering cannot reorder the artifact. Unrelated records must
remain present and retain their relative order. Every input record must be
represented by exactly one output record, either unchanged or through one
approved consolidation.

The existing NPC shape, identity, source-reference, schema, and relatedness
validators remain the final artifact boundary. No LLM-backed validator is
needed: model judgment occurs in the normalizer, while deterministic validators
continue to enforce the durable artifact contract.

## Prompt, Provenance, And Diagnostics

NPC semantic-normalization instructions and the private response schema should
be module-owned prompt assets. Stable D&D-wide identity or transcript guidance
may be reused through the existing shared prompt-asset mechanism where its
meaning is genuinely common.

The normalizer should use the configured normalize-stage LLM profile and the
application-wide scheduled LLM client. Invalid structured output and unsafe
semantic proposals use the framework-owned retryable-fallback contract;
operational failures retain existing error semantics.

Manifest metadata and component checkpoint fingerprints should identify the
semantic-normalization policy, prompt identity, private response-schema
identity, deterministic NPC identity policy, and evidence-context policy.
They must not contain transcript text, NPC names, source paths, raw model
responses, or other source content.

Warnings and preparation or runtime errors must follow the established bounded
diagnostic and content-safety policies. Debug artifacts may retain the normal
attempt-local model request, response, proposal decisions, and warnings under
the existing debug sensitivity contract.

## Configuration And Documentation

The maintained complete D&D configuration should demonstrate a normalize-stage
LLM profile and an explicit positive retry count for the NPC lane. The
application-wide default remains zero additional retries. No new configuration
field is introduced for the context radius in this scope.

Canonical documentation ownership is:

- Configuration owns the LLM-backed `dnd/npcs` selection and its use of the
  existing normalize binding's profile and retry fields.
- Operations owns the document-level semantic reconciliation call and its
  checkpoint behavior.
- Pipeline internals own the provider-neutral invalid-structured-output
  classification and normalize retryable-fallback contract.
- The NPC integration contract owns externally observable consolidation,
  evidence, ordering, and warning behavior.
- Internal LLM and module documentation own the proposal boundary,
  context-window construction, deterministic application, metadata, and
  fingerprints.
- The broader generic LLM-assisted deduplication item in
  [future.md](future.md) remains future work until another artifact demonstrates
  that extracting a shared generic facility is worthwhile.

## Quality Expectations

Tests should protect the semantic and safety boundaries through deterministic
LLM fakes rather than live-provider calls. Coverage should demonstrate:

- distinct display variants can be consolidated when the model proposes a
  valid group;
- model-facing requests and responses use display names rather than stable
  hash IDs;
- the selected existing canonical name controls ID derivation while all member
  evidence is preserved;
- window construction uses source-document position, handles document edges,
  coalesces overlap, and distinguishes cited evidence from context;
- no surrounding context is promoted into durable source references;
- empty proposals and fewer-than-two-candidate inputs preserve deterministic
  normalization behavior;
- unknown, ambiguous, repeated, overlapping, and otherwise malformed semantic
  groups cannot corrupt or reorder the artifact;
- locally valid groups are applied independently, while every group that
  participates in a resolved-member conflict is discarded;
- an invalid proposal consumes only configured retry budget, a later valid
  proposal can succeed, and exhaustion accepts the final attempt's safe
  fallback with one durable warning;
- the default zero-retry binding makes exactly one proposal attempt before
  fallback;
- warnings and failures remain bounded and do not expose transcript content;
- prompt, schema, policy, or context-policy changes invalidate relevant
  checkpoint reuse; and
- an assembled ordered D&D pipeline supplies the reconciled NPC registry to
  downstream consumers.

Prompt tests should assert prepared message structure, supplied materials, and
cache-boundary behavior. They must not act as change detectors for particular
words or phrases in natural-language prompt text.

Human review of representative transcripts should compare missed and false
consolidations, latency, input-token cost, and provider cache use. Probabilistic
model quality is an evaluation activity, not a deterministic CI assertion.

## Non-Goals

This scope does not:

- change the durable NPC artifact schema;
- add a durable alias collection or identity history;
- expose the context radius or full-transcript selection as configuration;
- ask an LLM to read, reproduce, or derive stable NPC IDs;
- permit model-authored source references or arbitrary replacement records;
- add fuzzy deterministic name matching;
- add an LLM-backed normalize validator;
- change the global default retry count;
- reconcile NPCs concurrently across lanes or runs;
- add canonical NPC IDs to downstream artifact schemas;
- implement a general DAG or implicit cross-lane dependency; or
- implement the generic cross-artifact deduplication normalizer described in
  `future.md`.

## Completion Criteria

The scope is complete when `dnd/npcs` can use one document-level LLM proposal
to reconcile differently named records conservatively, deterministic code
validates and independently applies only safe non-conflicting name-based
groups, the resulting NPC preserves all member evidence under a newly derived
stable ID, the two-unit context policy is centralized and fingerprinted,
invalid or partially unsafe proposals use configured framework retries and
then an accepted safe fallback, downstream ordered steps receive the
reconciled registry, and the canonical current-behavior documentation reflects
the implemented contract.