notarius/docs/roadmap/implementation.md

D&D NPC Semantic Normalization Implementation Plan

Status

Ready for implementation.

Objective

Implement the accepted D&D NPC Semantic Normalization contract. Enhance the existing dnd/npcs normalizer with one document-level LLM identity proposal per configured normalization attempt, but keep name resolution, proposal safety, evidence handling, ordering, stable-ID derivation, mutation, diagnostics, and final artifact validation deterministic.

Complete the stages below in order. Each stage should leave its affected packages passing before the next stage begins. Do not introduce a generic deduplication framework during this work.

Decisions Applying To Every Stage

• Retain the production module key dnd/npcs, artifact kind dnd/npc-list, durable schema notarius.dnd.npcs v1, codec, default normalize validator chain, and empty reference-slot contract.
• Give normalization its own private prompt identity. Use:
  • prompt ID and prompt filesystem directory: dnd.npcs.normalize;
  • prompt version: v1;
  • response-schema key: dnd_npcs_normalize_llm;
  • response-schema ID: notarius.dnd.npcs.normalize.llm;
  • response-schema name: notarius_dnd_npcs_normalize_llm_v1; and
  • response-schema version: v1.
• Use gemini-2-flash as the normalize prompt's default profile, matching the current NPC extraction prompt. An explicit normalize binding llm_profile continues to override that prompt default through the existing framework contract.
• Retain empty, strict Options. This scope adds no user-configurable context mode, radius, batching, or prompt option.
• Define one package-owned context-radius constant with value 2. Pass that constant into a context builder that accepts a radius argument, so future configuration can replace the caller-supplied value without redesigning the builder.
• Use two model inputs:
  • candidates, application/json, containing display names and integer start/end source ranges; and
  • transcript, application/json, containing the coalesced context windows. Compute a sha256: digest over each exact encoded input, leave its origin URI empty, and do not pass the original full-source SourceInput to the prompt.
• Never place npc:sha256: IDs in either model input, the private response schema, or natural-language prompt material. Stable IDs remain internal to deterministic application and durable artifacts.
• Keep the private response schema structural: require the top-level duplicate_groups array and each group's members array and canonical_name string; reject unknown fields and incompatible JSON types. Do not encode semantic constraints such as non-empty strings, minimum or unique array sizes, known names, canonical membership, or non-overlap in JSON Schema.
• Resolve model-returned names only through identity.ComparisonKey. Do not add fuzzy, substring, edit-distance, embedding, or heuristic name resolution.
• Classify structurally invalid model output separately from operational LLM failures at the provider-neutral structured-completion boundary. Invalid output and partially unsafe semantic proposals request a framework-owned normalize retry with a safe candidate fallback.
• Apply locally valid proposal groups independently only when none of their resolved members appears in another group. Discard all groups participating in a resolved-member conflict, including locally valid groups that conflict with an invalid group. Do not invent repairs, choose a conflict winner by response order, produce a durable rejection, or add a normalizer-local retry loop.
• Preserve accepted-only warning promotion. If retry budget remains, all fallback warnings remain attempt-local. On the last configured attempt, the runner validates and accepts the supplied safe fallback with one durable exhaustion warning.
• Preserve the binding default of retries: 0; this feature does not change global configuration defaults. Zero retries means one proposal call followed immediately by fallback when that proposal is invalid. A configured value of 2 permits three total proposal attempts.
• Do not add an LLM-backed validator. The existing deterministic validators remain the artifact-acceptance boundary.
• Do not add a new stage, reference type, durable alias field, schema migration, or compatibility adapter. The only framework contract expansion is the provider-neutral invalid-output classification and normalize retryable fallback described below.

Stage 1: Framework Retryable Normalize Fallback

Goal

Add the smallest domain-neutral result contract needed for a normalizer to ask the runner for another attempt while supplying a safe candidate to accept if the configured retry budget is exhausted.

Invalid Structured-Output Classification

Add a provider-neutral ErrInvalidStructuredOutput sentinel in internal/framework/contracts. It identifies failures in which a provider returned no usable value for the caller's declared structured-output contract.

Update internal/framework/llm/scriptorium_client.go to wrap that sentinel for:

• Scriptorium structured-output validation failure;
• an empty structured completion; and
• failure to decode returned structured content into the caller's output target.

Preserve the current contextual and redacted error text. Do not apply the sentinel to nil clients or output targets, prompt preparation, authentication, transport/provider execution, context cancellation, or other operational failures. Existing modules that do not inspect the sentinel retain their current behavior.

Normalize Result Contract

Add a normalize-only, domain-neutral retry directive to contracts.TypedNormalizeResult[T]. Use a named structure rather than Boolean fields:

type NormalizeRetry struct {
    ReasonCode       string
    Message          string
    FallbackWarnings []Warning
}

type TypedNormalizeResult[T any] struct {
    Value    T
    Warnings []Warning
    Retry    *NormalizeRetry
}

Value is always the safe fallback candidate when Retry is non-nil. Warnings are ordinary attempt warnings. FallbackWarnings are promoted only if the runner exhausts retry budget and accepts that fallback. ReasonCode and Message are attempt-local retry diagnostics for debug artifacts and must be non-empty, bounded, and content-safe when supplied by a module.

Update normalizer type erasure and cloning so the directive and both warning collections cross the registry boundary without mutable shared backing storage. Normalizers returning Retry: nil remain behaviorally unchanged.

Runner Semantics

In the normalize attempt path, serialize the returned candidate before acting on a retry directive so debug can inspect the safe fallback through the existing candidate envelope.

When Retry is non-nil:

• Reject a blank reason code or message as a framework/module contract error.
• Add a structured retry object to the normalize attempt debug payload with its reason code, message, whether another attempt remains, and whether the fallback is being accepted. Do not encode this as a contracts.RejectedOutput.
• If attempt <= lane.Normalize.Retries, record the terminal attempt envelope and return an unaccepted result with no rejection or error to runWithRetry. The loop then performs the next configured attempt.
• On the final attempt, append defensively copied FallbackWarnings to the attempt warnings and continue through the ordinary typed and serialized normalize validator chain using Value.

If final fallback validation approves, store and publish it exactly like any other accepted normalized result, including its accepted warnings and checkpoint. If validation rejects it, preserve the ordinary terminal normalize rejection with the correct attempt count. Do not record intermediate fallbacks as checkpoints or durable warnings.

A later attempt with Retry: nil follows the existing candidate validation and acceptance path; no earlier retry diagnostic or fallback warning becomes durable. Debug persistence failure, cancellation, and ordinary module or validator errors retain their current non-fallback behavior.

This contract is normalize-specific because a safe reconciled fallback is the demonstrated requirement. Do not add speculative retry fields to extraction, merge, chunking, or validation results.

Stage 1 Tests

Add generic framework tests at stable contracts:

• errors.Is recognizes invalid structured output for validation, empty-body, and decode failures, but not operational or cancellation failures.
• Normalizer registry erasure preserves a retry directive and returns defensive warning copies.
• With retries: 0, one retryable result validates and accepts its fallback, promotes only fallback and final-attempt warnings, and records one LLM/module attempt.
• With positive retries, an invalid result followed by a regular valid result accepts the later value and promotes no earlier fallback warnings.
• Exhausting positive retries validates and accepts only the last safe fallback and promotes its fallback warning.
• A final fallback rejected by normalize validators remains a normal rejection with the total attempt count.
• Intermediate fallback attempts produce debug retry diagnostics but no checkpoint or durable warning.
• Blank retry reason fields, debug persistence failures, errors, and cancellation retain the correct failure semantics.

Keep these tests domain-neutral and use test-controlled retry values. Do not assert the global default retry count here; configuration contract tests already own that default.

Run:

go test ./internal/framework/contracts
go test ./internal/framework/llm
go test ./internal/framework/pipeline

Stage 1 Completion Gate

Any typed normalizer can request a framework-managed retry with a safe fallback; zero and positive retry budgets behave deterministically; invalid structured output is distinguishable from operational failure; and existing normalizers remain unchanged.

Stage 2: Private Prompt Contract And Context Material

Goal

Add the package-owned prompt, schema, input models, and deterministic context-window builder without changing normalizer runtime behavior yet.

Prompt And Schema Assets

Under internal/modules/dnd/normalize/npcs, add the same package-owned asset structure used by LLM-backed D&D extractors:

• assets.go embedding only this package's prompt YAML, Markdown, and private response schema;
• schema.go owning the identities listed above and loading assets/schemas/dnd_npcs_normalize_llm.v1.json;
• scriptorium_assets.go owning the ordered prompt manifest, registration, and prompt-content digest;
• assets/prompts/dnd.npcs.normalize.yaml;
• package-local task.md, instructions.md, and a small candidate-input rendering template; and
• assets/schemas/dnd_npcs_normalize_llm.v1.json.

The prompt manifest should reuse only semantically applicable shared assets:

1. common-dnd-system.md;
2. common-dnd-identity.md;
3. package-owned task and normalization instructions;
4. the package-owned candidates input message; and
5. common-dnd-transcript.md rendering the windowed transcript input.

Place cache boundaries after the stable shared identity tier and after the stable package instructions. Candidate data and transcript windows are document-specific and should follow the final stable boundary. Do not reuse the extraction-evidence or campaign-reference prompt fragments: normalization does not extract new events, return citations, or consume campaign references. Set Scriptorium repair_attempts to 0, consistent with the current module-owned structured-completion boundary.

Prompt policy must tell the model to:

• return only duplicate groups that the supplied transcript context clearly establishes as one individual;
• prefer no group when identity is ambiguous;
• copy supplied display names into members;
• select canonical_name from the same group's supplied members;
• prefer a complete stable proper name over an abbreviation, but prefer an unadorned proper name over that name plus a contextual class, role, title, or relationship descriptor unless the descriptor is established as part of the name; and
• avoid inventing names, source references, replacement records, or explanatory output.

Model Input Types

Add private DTOs rather than exposing durable NPC types directly:

• Candidate input:
  • a required npcs array in deterministic current-record order;
  • each item contains only name and source_refs; and
  • each source range contains only start_unit_id and end_unit_id.
• Transcript input:
  • a required windows array in source-document order;
  • each window contains its ordered units; and
  • each unit contains generic source-unit id, kind, text, a defensively cloned metadata value when present, and a Boolean cited marker.
• Proposal response:
  • duplicate_groups;
  • each group has members []string and canonical_name string.

The candidate DTO deliberately omits stable NPC IDs and source-document IDs. The transcript DTO deliberately preserves generic source-unit metadata because the Seriatim adapter records speaker and timing context there; it must not import or interpret Seriatim-specific types or metadata keys.

Encode candidates and windows with encoding/json into independently owned bytes. Return fixed contextual errors without including transcript text, NPC names, encoded payloads, source paths, or model response content.

Context Builder

Implement context selection inside internal/modules/dnd/normalize/npcs; do not add it to generic framework or shared D&D packages for this first concrete use.

Use source.NewDocumentIndex and source-document slice positions. For each preprocessed record:

1. Require a non-empty identity.ComparisonKey for its display name.
2. Require a non-empty source-reference list.
3. Require every reference to validate against the current document. A record with any invalid reference is ineligible for the semantic call, remains unchanged, and is left to the configured normalize validators.
4. Convert each valid inclusive range to source slice positions.
5. Expand its start and end positions by the supplied radius, clamping to the document bounds.
6. Sort intervals by document position and coalesce intervals that overlap or are directly adjacent.
7. Emit each source unit at most once and preserve document order.
8. Mark a unit cited: true only when its position belongs to at least one original unexpanded reference; surrounding units remain false.

Only eligible records appear in the model's candidate list. Skip semantic normalization when fewer than two eligible, comparison-distinct candidates remain. Do not emit a new warning merely because a record is ineligible; shape and source-reference validators already own that diagnostic.

No artificial candidate-count, byte, token, or window-count limit is added in this scope. If the selected material exceeds a provider's context limit, the normal structured-completion failure remains a normalizer error. Batching and token budgeting stay deferred pending observed need.

Stage 2 Tests

Use package-level behavioral tests and the real prompt asset registry with offline Scriptorium preparation:

• The private schema accepts structurally valid empty and non-empty group arrays, including semantically invalid values reserved for deterministic checking, while rejecting missing fields, unknown fields, and wrong JSON types.
• Prepared prompt messages contain exactly the two declared dynamic inputs in the intended order and at the intended cache boundaries.
• Candidate and transcript input JSON contain display names, ranges, selected source units, and citation markers but no stable NPC hash IDs or full-source origin path.
• Context construction is based on source slice order for non-monotonic unit IDs, clamps document edges, expands complete multi-unit citations, coalesces overlap and adjacency, and does not duplicate units.
• Records with empty, missing, foreign-source, reversed, or otherwise invalid references are excluded from semantic candidates without being mutated.
• Returned materials and copied metadata do not alias caller-owned values.

Test the window mechanism with a caller-supplied test radius rather than duplicating the production constant throughout the suite. Prompt tests must assert message structure, inputs, schema identity, and cache behavior; they must not require particular words or phrases in natural-language prompt files.

Run:

go test ./internal/modules/dnd/normalize/npcs
go test ./internal/modules/dnd/register

Stage 2 Completion Gate

The new prompt and schema can be registered and prepared offline; context material is deterministic, content-safe, document-position-aware, and contains no stable NPC IDs; existing normalization behavior remains unchanged.

Stage 3: Deterministic Proposal Validation And LLM-Assisted Normalization

Goal

Wire the private structured completion into dnd/npcs and apply only safe name-based groups while preserving all existing deterministic behavior.

Normalizer Construction And Identity

Change the normalizer to retain:

• the injected contracts.StructuredLLMClient;
• the prompt asset digest; and
• the private response-schema digest.

Change New to accept the LLM client and return (*Normalizer, error). Reject a nil client and fail construction if prompt or schema metadata cannot load. Update the normalizer registry builder to pass request.Dependencies.LLM. Keep Options strict and empty and ReferenceSlots() empty.

Bump the complete normalization policy from dnd.npcs.normalize.v2 to dnd.npcs.normalize.v3. Add a stable semantic-context policy identity such as dnd.npcs.semantic_context.v1 and keep the radius as a separately inspectable metadata value.

Extend manifest metadata with:

• prompt_id, prompt_version, and prompt_sha256;
• response_schema_key, response_schema_id, response_schema_name, response_schema_version, and response_schema_sha256;
• identity_policy;
• normalization_policy;
• semantic_context_policy; and
• semantic_context_radius.

Extend component fingerprints with local names:

• prompt;
• response_schema;
• identity_policy;
• normalization_policy; and
• semantic_context_policy.

The semantic-context fingerprint value must cover both its policy identity and the radius value. Metadata and fingerprints must not contain names, source text, model output, paths, timestamps, or invocation-specific data. Existing pipeline scoping will prefix these local names and incorporate them into checkpoint identity. The new fingerprints intentionally make prior NPC normalize checkpoints cold misses.

Deterministic Preprocessing

Refactor the current normalizeList implementation into a form that preserves all existing behavior before the semantic call:

• display whitespace normalization;
• stable-ID recomputation;
• source-reference canonicalization and exact deduplication;
• equal-identity.ComparisonKey consolidation;
• first-occurrence output anchoring;
• source-reference union; and
• existing field, ID, evidence, and duplicate warnings.

Each preprocessed record must retain the sorted original input indexes it represents and its earliest input index. This provenance is internal only and supports deterministic application, ordering, scopes, and warnings.

Return the deterministic result immediately, without an LLM call, when fewer than two semantically eligible records remain. Preserve nil versus empty list behavior and do not mutate or alias the merge input.

Structured Completion

For two or more eligible records on each framework attempt:

1. Build the candidate and transcript-window inputs from Stage 2 using the production radius constant.
2. Call the injected scheduled client exactly once with:
   • StageName: Key;
   • the normalization prompt ID and v1 version;
   • ProfileID: req.LLMProfile;
   • SessionID: req.SessionID; and
   • only the candidates and transcript input materials.
3. Decode into the private proposal DTO.
4. If completion returns contracts.ErrInvalidStructuredOutput, return the deterministic pre-LLM artifact as Value with a normalize retry directive.
5. Wrap every other completion failure with normalizer and completion context, relying on the existing Scriptorium redaction boundary and never appending raw response content.

Do not perform an LLM call per candidate pair or group. Do not make a second module-local repair call. A framework retry invokes the normalizer again and therefore makes one new proposal call with the same deterministic inputs.

Proposal Validation

Validate the complete decoded proposal and isolate independently safe groups before applying them:

1. Build the unique map from identity.ComparisonKey(displayName) to eligible preprocessed record position.
2. For each response group, resolve every member and canonical_name.
3. Mark a group locally invalid when it has:
   • fewer than two distinct members;
   • a blank, unknown, or ambiguous member;
   • a repeated resolved member;
   • a blank, unknown, or ambiguous canonical name; or
   • a canonical record that is not a group member.
4. For every group, including locally invalid groups, retain each member that resolves uniquely and count ownership of those resolved record positions across the complete proposal.
5. Mark every group containing a record position owned by more than one proposal group as conflicting. Discard every participant rather than choosing a winner. Applying this rule to every group also discards all groups in a chained conflict.
6. Classify a group as safe only when it is locally valid and non-conflicting. Apply every safe group independently; discard every other group.
7. If any group is discarded, return the artifact containing the safe groups as a retryable fallback. If none is discarded, return the applied artifact through the ordinary successful result path. Response order may order attempt diagnostics but must never choose artifact order or conflict winners.

Name resolution may accept only the equivalences already implemented by identity.ComparisonKey. The validator must not infer which supplied name the model intended.

Add:

• npc_semantic_proposal_invalid as the attempt-local retry reason for structurally or semantically invalid proposals;
• npc_semantic_reconciliation_exhausted as the durable fallback-warning reason; and
• npc_normalization_warnings_omitted for total accepted-warning truncation.

For a partially unsafe semantic proposal, build one bounded retry message with shared/diagnostics.Aggregate. It may identify proposal-group indexes and stable issue categories, including every group participating in an overlap, but must not echo model-returned names, transcript content, source paths, or raw completion text.

Apply the safe groups to the deterministic pre-LLM artifact and return that partial result as the retry directive's safe Value. If structured output could not be decoded or no group is safe, Value remains the deterministic pre-LLM artifact. Ordinary deterministic preprocessing warnings and warnings for safely applied groups remain in Warnings.

Put one bounded warning with reason npc_semantic_reconciliation_exhausted in FallbackWarnings; it should state the exact number of groups omitted from the final proposal after the configured attempt budget. For structurally invalid output, where no decoded group count exists, state that the semantic proposal could not be applied. That warning becomes durable only when the runner accepts final fallback.

Each framework attempt starts from the same merge output and performs deterministic preprocessing again. Do not retain or accumulate safe groups from earlier attempts. If retry budget is exhausted, the final attempt's safe candidate is authoritative.

Deterministic Group Application

Apply approved groups against the preprocessed record positions:

• Iterate records in existing order.
• Emit one consolidated record at the earliest member position and suppress later members.
• Take the display name from the model-selected existing canonical member, regardless of which member supplies the output anchor.
• Union every member's existing source references and canonicalize them with the current source-document order.
• Derive a new stable ID from the selected display name.
• Combine the represented original input-index sets without loss.
• Leave unrelated records present and in their prior relative order.

Use the existing duplicate_npc_collapsed reason code for both deterministic equal-name and approved semantic consolidation. The semantic warning scope is the earliest represented input index. Its bounded message should identify input indexes and, when different, the input index that supplied the canonical display name; it should not include raw names or transcript text.

Build duplicate details with shared/diagnostics.Aggregate, and pass the complete NPC normalization warning list through diagnostics.LimitWarnings(..., "npcs", ReasonCodeNPCNormalizationWarningsOmitted) before returning. This total cap applies to old and new normalizer warnings and must preserve deterministic warning order. When returning a retry directive, reserve one position within diagnostics.MaxWarnings for the exhaustion fallback warning. If deterministic or safely applied-group warnings require truncation, include their omission-summary warning before the reserved exhaustion warning so final accepted fallback still contains at most MaxWarnings warnings and reports the exact omitted-warning count.

For a proposal whose groups are all safe, return the applied artifact with Retry: nil. The configured normalize validator chain receives it through the ordinary framework path. For a final fallback, Stage 1 sends the safe candidate through that same validator chain. No validator, codec, artifact schema, or warning JSON shape changes are needed.

Stage 3 Tests

Use a small recording LLM fake at the normalizer boundary and real deterministic collaborators:

• A nil client fails construction before execution, while normal construction exposes valid prompt and schema metadata.
• Nil receiver, nil context, canceled context, operational completion failure, and JSON input-encoding failure follow content-safe error paths.
• Invalid structured output returns a retryable deterministic fallback rather than an ordinary error.
• Zero or one eligible candidate performs no LLM call and retains existing deterministic normalization.
• A valid proposal consolidates differently named records, selects an existing canonical display name, recomputes its ID, unions and orders all member evidence, occupies the earliest member position, preserves unrelated order, and does not alias input.
• The completion request carries the normalize profile and session ID and contains names but no stable hash IDs.
• Empty proposals leave comparison-distinct records unchanged.
• Case, whitespace, Unicode compatibility, and supported apostrophe variants resolve through the existing comparison policy.
• Unknown, blank, repeated, singleton, non-member-canonical, and overlapping groups are discarded and return a retryable safe fallback.
• A valid independent group is applied even when a disjoint group is invalid.
• A locally valid group sharing a resolved member with an invalid group is discarded, and chained overlaps discard every group in the connected conflict set without response-order dependence.
• A partially safe final attempt preserves its applied groups in the accepted fallback and reports the exact omitted-group count; a later retry starts from the original merge output rather than accumulating earlier groups.
• Invalid or ineligible source-reference records cannot participate in semantic groups and remain available for final validator rejection.
• Surrounding context never becomes output evidence.
• Retry diagnostics and fallback warnings remain valid UTF-8 and individually bounded. Accepted deterministic, applied-group, and exhaustion warnings are subject to the total warning cap with an accurate omission summary.
• Metadata and fingerprints contain all stable policy and asset identities but no names, source text, raw model responses, or paths.

Do not add tests that assert exact natural-language prompt wording. Do not repeat generic checkpoint-identity tests: package tests should prove that the normalizer contributes the required stable fingerprints, while existing framework tests continue to own the rule that fingerprint changes alter identity.

Run:

go test ./internal/modules/dnd/normalize/npcs
go test ./internal/modules/dnd/validate/npcs/...
go test ./internal/framework/pipeline

Stage 3 Completion Gate

Each normalizer invocation performs at most one scheduled LLM call over eligible document-level candidates. It independently applies safe, non-conflicting name-based groups, requests a framework retry when any group is discarded, supplies the resulting safe candidate as fallback, preserves artifact integrity and evidence, emits bounded diagnostics, and contributes complete checkpoint identity.

Stage 4: Production Registration, Pipeline Integration, And Checkpoint Coverage

Goal

Make the new normalize prompt available in production composition and prove that reconciled NPCs cross an ordered step barrier correctly.

Registration

Update internal/modules/dnd/register/modules.go to register NPC normalize prompt and schema assets in addition to the existing NPC extraction assets. Use a distinct registration label and call the normalize package's RegisterPromptAssets.

Update registrar tests to cover the new prompt filesystem and schema registration under dnd.npcs.normalize. Preserve existing module key, artifact-kind registration, normalizer spec, reference slots, options, and default normalize validator chain.

Update every production test constructor or direct normalizer call for the new LLM-injected constructor. Preparation tests should continue using the existing production fake client; no real provider is permitted.

Existing Fake And Fixture Adaptation

Several integration fakes currently assume that every dnd/npcs pipeline LLM request is extraction. Change them to dispatch by PromptID:

• extraction prompt requests continue returning the existing private NPC extraction response;
• normalization prompt requests return a private duplicate_groups response; and
• unrelated prompt IDs retain their existing behavior.

For tests unrelated to semantic reconciliation, return {"duplicate_groups":[]} and update only call-count assertions whose observable contract now includes the one document-level normalize call. Do not duplicate proposal edge cases already owned by normalize-package tests.

Assembled Behavior

Add one focused assembled D&D workflow case with two extraction chunks that produce a short proper name and a role-qualified variant for the same NPC. Have the normalize fake propose one group using those display names. Prove:

• extraction still produces and merge still preserves both candidates;
• normalization emits one canonical NPC at the earliest member position;
• its evidence is the canonical union of both chunks' source references;
• its ID is derived from the selected canonical display name;
• the accepted normalized artifact crosses the existing ordered step barrier;
• a downstream NPC consumer receives a names-only registry projection containing the canonical name once; and
• no stable NPC hash ID or transcript content leaks into manifest component metadata or reference provenance.

Keep this integration test focused on collaboration among extraction, merge, normalization, validation, and generated handoff. Proposal-malformation, context-window edge cases, and schema parsing remain at their narrower owners.

Checkpoint And Retry Coverage

Update the existing NPC preparation and grounded-pipeline assertions to include the scoped normalize fingerprints for prompt, response schema, identity policy, normalization policy, and semantic-context policy. Confirm the normalize prompt/schema metadata appears under the normalizer manifest component.

Rely on existing generic checkpoint tests for ordering and identity sensitivity; do not add another generic fingerprint mutation test. The assembled NPC test should demonstrate that the new fingerprints are present before execution. Existing checkpoint construction will then cause a one-time cold miss for old NPC normalize checkpoints.

Stage 1 framework tests own the generic retryable-fallback mechanism. Add only the narrow NPC integration coverage needed to prove feature wiring:

• with normalize retries omitted, one partially unsafe NPC proposal makes one normalize completion call, accepts its safe partial fallback, and promotes the exhaustion warning; and
• with retries: 1, a partially unsafe NPC proposal followed by a completely safe proposal makes two normalize completion calls, applies the later proposal, and does not promote the earlier fallback warning.

Do not repeat every proposal-invalidity category or generic final-validator case at integration scope. Existing NPC fakes should return a fresh normalization response for each actual attempt.

Stage 4 Tests

Run:

go test ./internal/modules/dnd/register
go test ./internal/modules/integration
go test ./internal/cli
go test ./internal/framework/pipeline

Stage 4 Completion Gate

Production preparation registers and constructs the LLM-assisted normalizer, representative pipelines route both NPC prompt identities correctly, prepared identity contains all new fingerprints, and downstream generated references observe the reconciled registry.

Stage 5: Maintained Configuration And Canonical Documentation

Goal

Update each canonical owner to describe the implemented behavior without duplicating volatile contracts.

Configuration And Example

Update examples/dnd-complete.config.yml so the NPC normalize binding uses object form and demonstrates the normalize-stage profile and retry knobs:

normalize:
  module: dnd/npcs
  llm_profile: gemini-2-flash
  retries: 2

Do not add a radius or context-mode option. Keep the minimal example unchanged unless its current NPC composition requires object form for correctness. Retain the maintained-example resolution tests as the owner of example validity.

Update docs/config.md to:

• identify dnd/npcs normalization as LLM-assisted;
• explain that llm_profile and retries use the ordinary normalize binding contract;
• retain and clearly state the global retries: 0 default, including that it allows one initial normalization attempt and no additional attempts;
• retain the absence of normalizer references and options;
• state that context selection is currently module policy rather than configuration; and
• accurately list the affected prompt default without duplicating the complete example.

Operations, Integration, And Internals

Update:

• docs/operations.md with the document-level NPC semantic-normalization call, its place before the ordered generated-reference barrier, configured retry behavior, safe partial fallback after invalid-proposal exhaustion, and checkpoint reuse;
• docs/internal/pipeline.md with the provider-neutral invalid structured output classification, normalize retry directive, attempt accounting, accepted-only warning promotion, fallback validation, debug behavior, and checkpoint rules;
• docs/integrations/dnd-npc-artifacts.md with externally observable name-based consolidation, canonical selection, evidence union, ordering, warnings, and unchanged durable v1 artifact shape;
• docs/internal/modules.md with deterministic preprocessing, candidate eligibility, prompt inputs, private response contract, proposal validation, application, diagnostics, metadata, and fingerprints;
• docs/internal/llm.md with the normalization prompt's stable and variable message tiers, cache boundaries, windowed transcript input, and distinct prompt/schema identity; and
• docs/internal/overview.md with a concise current package responsibility.

Do not put implementation details in configuration or operations documentation. Do not describe the private proposal schema as part of the durable NPC integration schema. Keep the generic LLM-assisted deduplication item in future.md; this feature is intentionally the first D&D-specific use and does not complete that generic work.

After all implementation and documentation checks pass:

• change the feature roadmap status to Implemented and preserve it as the feature contract and rationale; and
• change this implementation plan's status to Completed.

Documentation And Test Policy

Update existing behavioral and contract tests where their owned behavior changes. Do not add prompt-language change detectors, broad golden snapshots, live-provider tests, redundant schema cases at higher layers, or assertions against private helper shape.

Run:

go test ./internal/cli
go test ./internal/modules/integration

Stage 5 Completion Gate

The maintained complete example resolves, canonical current-behavior documentation describes the shipped feature in its proper owners, future work still distinguishes generic deduplication, and both roadmap statuses agree with implementation state.

Final Verification

From the repository root, run:

git diff --check
go test ./...
go vet ./...
go build ./cmd/notarius
go test -race ./internal/modules/dnd/normalize/npcs ./internal/modules/dnd/register ./internal/framework/pipeline ./internal/cli ./internal/modules/integration

Review the final diff for:

• no durable artifact or public configuration schema change;
• no model-facing stable NPC IDs;
• no full-transcript normalize input;
• no transcript, name, path, or raw response content in metadata, fingerprints, provenance, or content-safe errors;
• no fuzzy matching or response-order-dependent application;
• no retryable-result expansion beyond the demonstrated normalize fallback or domain behavior in the framework contract;
• no real-provider or prompt-wording change-detector tests; and
• no documentation describing future behavior as already implemented before the final implementation stage is complete.

Assumptions

• The current source document supplied by the runner remains the authoritative source-unit order; unit IDs are identifiers and may be non-monotonic.
• Default production extraction validators normally provide valid NPC references before merge. The normalizer still excludes invalid records from semantic reconciliation so custom validator chains cannot make them unsafe model operands.
• A deterministically validated partial reconciliation after retry exhaustion is preferable to failing the complete NPC lane or discarding independent safe groups. Operators can inspect its durable bounded warning and attempt-local debug diagnostics.
• Invalid structured output is distinguishable from operational LLM failure. The former uses retryable fallback; the latter retains existing error behavior.
• The configuration-wide retry default remains zero. Any future decision to change it to one or two is separate work with application-wide cost, latency, and failure-semantics consequences.
• One document-level request is sufficient for the current workload. Batching, token budgeting, retrieval, full-transcript mode, and context-radius configuration require separate evidence and scope.
• Checkpoint compatibility is subordinate to semantic correctness. No migration or cleanup is required for normalize checkpoints made cold by the new component fingerprints.

Open Questions

None. The feature roadmap and the decisions above are sufficient to implement the work without additional product or architecture choices.