notarius/docs/roadmap/implementation.md

# Minimal D&D Extraction Contracts Implementation Plan

**Status:** Ready for implementation

## Objective

Implement the durable contract redesign defined by
[Minimal D&D Extraction Contracts](minimal-dnd-extraction-contracts.md) and
[ADR-0009](../adr/0009-minimal-evidence-grounded-extraction-artifacts.md).
The result is a coordinated in-place redesign of NPC, spell-cast, and combat-turn
extraction. Each lane must emit only its narrow, evidence-grounded facts; no
removed rich-schema field may survive as an optional field, placeholder, compatibility
shim, or deterministic synthesis.

This plan deliberately starts with NPCs because the NPC registry is a generated
reference consumed by spell and combat extraction. Spell and combat then change
independently, followed by one repository-wide contract and documentation
pass.

## Fixed Decisions And Guardrails

The implementing agent must treat these as decisions, not open design choices:

- Keep the artifact kinds `dnd/npc-list`, `dnd/spell-list`, and
  `dnd/combat-turn-list`, their module keys and capabilities, their media types,
  and their prompt IDs stable.
- Retain exactly the existing v1 schema keys, IDs, names, versions, filenames,
  prompt IDs, and prompt versions listed in the feature roadmap. Change their
  unpublished shapes and content in place; do not harmonize the private spell
  identity as part of this work.
- Do not add v2 assets, runtime version negotiation, migration code, dual-write
  behavior, a second module registration, or compatibility fixtures for the
  superseded pre-release shapes. Existing fixtures should be rewritten or
  deleted according to whether they still protect current behavior.
- Require the top-level list field and every record field. A list may be empty,
  but every returned record must have at least one source reference. Use strict
  JSON objects with unknown fields rejected and no nullable or optional legacy
  fields.
- Keep private source references limited to `start_unit_id` and `end_unit_id`.
  Mapping assigns the current source document ID; campaign references and an
  NPC registry never become event evidence.
- Keep the shared prompt ordering policy: stable shared instructions first,
  stable campaign and generated references next, module task material after
  those references, and the chunk-variable transcript last.
- Preserve the existing production validator-chain order and the rule that a
  configured chain is authoritative. Remove or simplify validators; do not
  silently reorder chains.
- Preserve the existing NPC ID derivation algorithm and comparison/display
  normalization for canonical names. Retain existing semantic-policy
  identifiers as well as schema and prompt versions; pre-redesign development
  state is disposable. Unchanged canonical names must not receive new IDs.
- Use deterministic code only for display normalization, known catalog or NPC
  canonicalization, application-owned IDs, evidence canonicalization, ordering,
  and exact duplicate collapse. Do not introduce fuzzy matching or inferred
  enrichment.
- Follow the testing policy: protect schemas and behavior at their owning
  boundaries, delete obsolete tests, and avoid tests that snapshot prompt prose
  or detect exact shared-prefix length. No live or paid model call is a stage
  completion requirement.
- Each stage must leave the repository compiling and `go test ./...` passing.
  Update all affected fixtures and callers within the stage that changes a
  public Go type; do not leave an intentionally broken intermediate commit.

## Stage 1: Cut Over The NPC Contract And Name Projection

This stage establishes the redesigned generated reference on which later stages
depend.

### 1.1 Replace the durable and private data shapes

- Reduce `dnd.NPC` in `internal/modules/dnd/types.go` to exactly `ID`, `Name`,
  and `SourceRefs`. Delete `NPCRelationship` and every alias, description, and
  relationship field or helper that becomes unused.
- Rewrite the existing strict durable schema `dnd_npcs.v1.json` with required
  top-level `npcs`, and records containing only required `id`, `name`, and
  `source_refs`. Keep the NPC codec at schema version `v1`, name
  `notarius_dnd_npcs_v1`, and its existing schema ID and media type.
- Rewrite the strict private schema `dnd_npcs_llm.v1.json`. Its records contain
  only required `name` and model-facing source ranges. Keep the extractor schema
  identity and prompt ID/version unchanged.
- Reduce the private response DTO and mapping accordingly. Preserve candidate
  data at the mapping boundary so deterministic validators, rather than mapping
  defaults, reject blank names or invalid ranges. Assign the current source ID
  and derive the NPC ID in application code.
- Rewrite the NPC task and instruction assets to ask only for individually
  identifiable NPC names and supporting transcript ranges. Explicitly exclude
  anonymous groups, generic roles, invented labels, descriptions, aliases, and
  relationships. Keep transcript material last in the manifest.

### 1.2 Simplify NPC identity, normalization, and validation

- Keep the identity policy at `dnd.npcs.identity.v1`. Retain display
  normalization, comparison keys, ID syntax, and deterministic ID derivation.
  Validate nonblank canonical names, exact ID/name agreement, duplicate
  canonical names, and duplicate IDs. Delete alias validation and alias-specific
  issue locations/codes.
- Keep NPC normalization at `dnd.npcs.normalize.v1`. Normalize the retained
  display name, derive its ID, canonicalize and deduplicate source references,
  consolidate records only by the canonical-name comparison key, preserve the
  first stable record, and union exact evidence. Remove alias promotion,
  relationship merge/rewrites, and their warning codes.
- Keep the existing NPC shape and source-relatedness policy identifiers. The
  shape validator checks only list presence, nonblank `id` and `name`, and
  nonempty source references; source-relatedness grounds a record only through
  its retained name. The existing source-ref validator remains the owner of
  range validity.
- Keep the default validation order domain-first, then JSON Schema, then the
  advisory relatedness validator. Simplify only the validator implementations
  and selections whose owned behavior changed.

### 1.3 Separate durable registry provenance from model input

- Continue to canonicalize and retain the complete redesigned NPC artifact for
  registry validation, cache identity, manifests, and provenance.
  `Registry.Digest()` remains the digest of that complete canonical artifact
  when bound and remains empty when no registry was supplied.
- Build a second structural JSON projection for model grounding with the exact
  shape `{"npcs":[{"name":"Mira Thorn"}]}` in normalized registry order. It
  contains names only: no IDs, source references, origin URI, aliases, or other
  provenance. Generate it with typed values and `json.Marshal`, not string
  concatenation. The empty projection is exactly `{"npcs":[]}`.
- Add an explicit projection digest accessor. The digest is SHA-256 over the
  exact projected bytes, including for an absent or empty registry, and
  `PromptInput().Digest` must equal it. Keep full-artifact and projection digests
  distinct even when their content happens to coincide.
- Index `Registry.Lookup` by canonical name only. Retain immutable return values
  and defensive copies. Remove alias indexing and alias-aware comments.
- In spell extraction, combat extraction, and combat normalization, use the NPC
  name-projection digest for the component-local checkpoint fingerprint because
  it exactly describes the names that affect those operations. Keep the full
  registry digest and count in manifest metadata for provenance. Framework-owned
  generated-reference fingerprints may still invalidate a run when any upstream
  artifact byte changes; do not broaden this stage into a framework cache
  redesign.

### 1.4 Update owners, consumers, and tests

- Update NPC codec, schema, extractor, identity, validator, normalizer, registry,
  registration, pipeline-integration, and CLI fixtures to the new shape. Adapt
  spell and combat tests that construct `dnd.NPC` values so the repository
  remains buildable, but do not change their own artifact contracts yet.
- Add or rewrite focused tests for strict schema acceptance/rejection,
  codec round trips, unchanged ID derivation for known names, name-only
  consolidation, evidence union, registry immutability, and canonical-name-only
  lookup. Rewrite the existing fixture for the minimal current contract; do not
  retain the superseded rich fixture solely to test backwards incompatibility.
- Test the projection as a data contract: it contains only ordered names,
  equivalent normalized registries produce identical bytes and digest,
  evidence/ID-only changes do not change the projection digest, and name/order
  changes do. This is not authorization to snapshot assembled prompt prose or
  prefix lengths.
- Update `docs/integrations/dnd-npc-artifacts.md` to own the redesigned v1
  durable schema. Update `docs/internal/modules.md` and
  `docs/internal/llm.md` only for current NPC behavior that lands in this stage.

### Stage 1 completion criteria

- NPC durable and private schemas expose no removed enrichment fields.
- The generated NPC prompt input contains names only while manifests retain
  full registry provenance.
- Spell and combat consumers accept generated or external registries in the
  redesigned v1 shape; rich pre-redesign registry JSON fails strict decoding.
- Focused NPC, spell-wiring, combat-wiring, integration, and CLI tests pass, and
  `go test ./...` passes.

## Stage 2: Cut Over Spell-Cast Extraction

### 2.1 Replace the spell contract and prompt

- Reduce `dnd.SpellCast` to exactly `Caster`, `Spell`, and `SourceRefs`.
- Rewrite `dnd_spells.v1.json` in place. Keep the durable codec at version v1,
  name `notarius_dnd_spells_v1`, and its existing schema ID. Its strict record
  contains only required `caster`, `spell`, and `source_refs`.
- Rewrite `dnd_spells_llm.v1.json` with the same logical fields and model-facing
  ranges. Keep the existing private key, ID, name, prompt version, and schema
  path unchanged.
- Reduce the private DTO, canonicalization, and mapping to the retained fields.
  Continue assigning current source IDs, stable-ordering candidates by evidence,
  and preserving semantically invalid candidates for deterministic validation.
- Rewrite spell task and instruction assets around the narrow casting-event
  boundary. Retain the spell catalog and name-only NPC projection as
  disambiguation inputs, never evidence. Remove effect, outcome, and narrative
  duties and remove the `common-dnd-immediate-resolution.md` message from the
  spell manifest and spell asset registration. Do not delete the shared file in
  this stage because combat still uses it.

### 2.2 Simplify spell policies

- Keep the spell shape policy identifier unchanged and validate only list
  presence, nonblank caster/spell, and nonempty source references.
- Keep catalog validation, source-reference validation, and source-relatedness
  behavior and policy identities unchanged where their actual semantics are
  already limited to spell name, caster, and evidence.
- Simplify normalization and duplicate comparison to caster, catalog-canonical
  spell name, and the complete valid evidence set. Remove all prose selection,
  copying, fixture fields, and assertions. Preserve catalog fingerprints and
  metadata.
- Keep the exact inclusion rule from the feature roadmap: an actual casting or
  unambiguous declared attempt is included; mentions, plans, rules discussion,
  and catalog matches are not.

### 2.3 Update owners and tests

- Update spell codec, private schema, extractor, validators, normalizer, merge
  and registration tests, pipeline integration, CLI output fixtures, and any
  maintained examples to the minimal shape. Rewrite or delete rich-schema
  fixtures rather than retaining them as compatibility cases.
- Replace tests of effects and narrative descriptions with focused tests of the
  retained contract: strict unknown-field rejection, codec round trips, current
  source-ID assignment, catalog canonicalization, evidence ordering, duplicate
  collapse, and NPC name-projection wiring.
- Update `docs/integrations/dnd-spell-artifacts.md` as the canonical redesigned
  v1 contract and update current internal module/LLM documentation for the
  smaller prompt and response. Do not duplicate the spell catalog contract
  owned by its existing integration document.

### Stage 2 completion criteria

- No production spell type, schema, prompt, validator, normalizer, fixture, or
  documentation contract refers to effect or narrative description.
- Spell prompt/schema identities remain exactly their existing v1 values.
- Focused spell and pipeline tests pass, and `go test ./...` passes.

## Stage 3: Cut Over Combat-Turn Extraction

### 3.1 Replace the combat contract and prompt

- Reduce `dnd.CombatTurn` to exactly `Actor`, `TurnKind`, and `SourceRefs`.
  Delete `CombatAction`, `CombatActionCategory`, their constants, and all
  now-unused helpers. Retain the existing five `CombatTurnKind` values.
- Rewrite `dnd_combat_turns.v1.json` in place. Keep the durable codec at version
  v1, name `notarius_dnd_combat_turns_v1`, and its existing schema ID. Use a
  strict required record with `actor`, `turn_kind`, and `source_refs` only.
- Simplify the durable codec to direct strict encoding/decoding if its custom
  wire representation exists only to distinguish removed nullable fields.
  Preserve presence semantics for the top-level list and strict unknown-field
  rejection.
- Rewrite `dnd_combat_turns_llm.v1.json` while retaining its private schema and
  prompt identities, and reduce the DTO/mapping to actor, turn kind, and
  model-facing ranges. Keep the semantic mapping policy identifier unchanged.
- Rewrite combat task/instruction assets around detecting ordered turns and
  discrete interrupting events. Remove round, action, target, declaration,
  resolution, outcome, and summary duties. Remove the shared immediate-
  resolution message from the combat manifest.
- Once both spell and combat manifests no longer use it, delete
  `common-dnd-immediate-resolution.md` and its shared and module asset
  registrations. Retain all other shared evidence, identity, reference, NPC,
  and transcript assets in their cache-friendly order.

### 3.2 Simplify combat normalization and validation

- Keep combat normalization at `dnd.combat_turns.normalize.v1`. Continue to
  display-normalize and registry-canonicalize actors, canonicalize evidence,
  order records chronologically, and collapse exact duplicates by actor, turn
  kind, and complete valid evidence. Delete action/target/prose normalization
  and warning codes.
- Keep the invariant policy identifier unchanged. It checks canonical actor
  display, canonical and chronological evidence, stable event ordering, and
  absence of duplicate event identities; it performs no nested-action checks.
- Keep the shape policy identifier unchanged and validate only list presence,
  nonblank actor, allowed turn kind, and nonempty source references.
- Keep the source-relatedness policy identifier unchanged and compare only the
  actor against cited transcript material. Remove declaration-token heuristics
  and their now-unused helpers. Keep source-reference validation unchanged if
  its semantics did not change.
- Simplify merge/clone behavior to copy only retained values and source refs.
  Preserve stable ordering and defensive ownership.

### 3.3 Update owners and tests

- Update codec, private schema, extractor, normalizer, validators, merge,
  registration, pipeline integration, CLI fixtures, and examples to the minimal
  shape. Rewrite or delete rich-schema fixtures rather than retaining them as
  compatibility cases.
- Delete tests whose sole policy was round/action/summary handling. Add or
  rewrite focused tests for strict schemas, enum validation, mapping and
  source-ID assignment, actor canonicalization through the redesigned NPC
  registry, chronology, exact duplicate collapse, and invariant validation.
- Update `docs/integrations/dnd-combat-turn-artifacts.md` as the canonical
  redesigned v1 contract and update current internal module/LLM documentation
  for the implemented behavior.

### Stage 3 completion criteria

- No production combat type, prompt, schema, policy, normalizer, fixture, or
  current documentation refers to rounds, actions, summaries, declarations,
  targets, resolutions, or action categories.
- The immediate-resolution shared asset has no remaining registration or file.
- Focused combat and pipeline tests pass, and `go test ./...` passes.

## Stage 4: Complete The Repository-Wide Cutover

### 4.1 Audit contract identity and stale surface area

- Search code, embedded assets, tests, examples, and current documentation for
  all removed field names and unintended v2 schema/prompt names. Removed fields
  may remain only in historical ADR or roadmap context; active schema, prompt,
  fixture, and current-behavior surfaces must describe the minimal v1 contract.
- Verify the three artifact registrations still bind their original kinds and
  exact redesigned Go types through extract, merge, normalize, codec, and
  validators.
- Verify all prompt manifests still point at their v1 private schemas, stable
  reference material precedes module-variable material, and transcript content
  remains last. Do not add a change-detector test for prompt message count,
  prose, shared-prefix content, or prefix length.
- Verify pre-redesign development checkpoints naturally miss through changed
  prompt/schema content and generated-dependency fingerprints. Do not add
  migration or compatibility handling for disposable pre-release state.

### 4.2 Exercise representative assembled behavior

- Keep unit case matrices at their owning schema, validator, normalizer, and
  registry boundaries. At the assembled-pipeline boundary, retain only
  representative tests proving each redesigned lane is registered, strict
  JSON/schema failures are attributed to the correct validator, semantic
  failures are attributed to the correct domain validator, and generated NPC
  output is accepted by later spell/combat stages.
- Cover an ordered multi-step run in which NPC extraction produces the minimal
  artifact and spell/combat consume its name projection. Assert that downstream
  event source references point only to the current transcript and not to NPC
  registry evidence.
- Verify CLI logical output and manifests retain the v1 schema/prompt identities,
  full NPC registry provenance, and the relevant component fingerprints without
  exposing prompt bodies or reference payloads.

### 4.3 Finish documentation and lifecycle state

- Reconcile the three integration documents, `docs/internal/modules.md`, and
  `docs/internal/llm.md` with the final code. Correct any stale validator-order
  descriptions while doing so; current docs must describe the registered order,
  not preserve an older generic ordering example.
- Mark ADR-0009 `Accepted` and the feature roadmap `Implemented` when the code,
  tests, and current-behavior documentation all land. Remove the completed item
  from `docs/roadmap/future.md`; retain the feature roadmap and ADR as design
  rationale unless the repository's normal roadmap-retirement practice calls
  for moving the completed roadmap later.
- Record any human-reviewed rich/minimal model evaluation separately from CI
  results. Evaluation may motivate later prompt tuning but does not reopen the
  approved durable minimal field set within this implementation.

### 4.4 Final verification

Run, in order:

```sh
gofmt -w <changed Go files>
go test ./...
go vet ./...
go build ./cmd/notarius
go test -race ./internal/modules/dnd/...
git diff --check
```

If the repository-wide race command exposes an unrelated, pre-existing failure,
document it with the narrower affected package result; do not weaken or skip
ordinary tests for the changed D&D packages.

### Stage 4 completion criteria

- All three lanes use only their minimal v1 contracts from model response
  through durable output.
- No compatibility shim, removed-field policy, unintended v2 asset, or stale
  current-behavior documentation remains.
- Generated NPC references are name-only for LLM input and remain full-fidelity
  for durable provenance.
- Repository tests, vet, build, race checks for the changed domain, and diff
  hygiene checks pass.

## Open Questions

None. The feature roadmap, ADR, and fixed decisions above define the cutover,
pre-release schema, evidence, projection, validation, testing, and documentation
policies needed to implement each stage without further product decisions.