From b2c076946b85a184000b1de15b0a0e308300c519 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Thu, 23 Jul 2026 04:33:18 +0000 Subject: [PATCH] Plan chunk map exports and scene descriptions --- docs/roadmap/future.md | 96 ++++- docs/roadmap/implementation.md | 373 ------------------ .../minimal-dnd-extraction-contracts.md | 220 ----------- 3 files changed, 78 insertions(+), 611 deletions(-) delete mode 100644 docs/roadmap/implementation.md delete mode 100644 docs/roadmap/minimal-dnd-extraction-contracts.md diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index 411af44..a2bbe46 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -16,28 +16,88 @@ not as committed release dates. validator, and normalizer development. Treat model-quality review as an iterative human evaluation aid, not a deterministic correctness gate. -### Expand Sequential D&D Artifacts +### Extract NPC Interactions -- Add narrative extraction for scene summaries, party actions, and NPCs - encountered when that output proves useful beyond the dedicated NPC artifact. -- Use ordered pipeline steps when a later artifact needs an accepted earlier - artifact as context. Keep independent lanes in the same step and do not - introduce a general DAG or concurrent cross-lane reconciliation model. +- Add an ordered NPC-interaction artifact that records how an identifiable NPC + participates in the session without adding occurrence-level state to the + normalized NPC registry. Use `npc-interactions` as the working lane and + product name; the exact artifact kind may be finalized with its contract. +- Keep each record minimal: canonical NPC `name`, one bounded interaction + `kind`, and transcript `source_refs` supporting both the identity and + classification. +- Start with the mutually exclusive vocabulary `mentioned`, + `noncombat_presence`, `dialogue`, `combat_ally`, `combat_opponent`, and + `other`. Define narrow inclusion rules and category precedence before + implementation so overlapping activity does not produce arbitrary labels. +- Model interactions as ordered occurrences rather than one scalar NPC + category. The same NPC may therefore have separate records when the + transcript establishes distinct interactions, such as dialogue followed by + hostile combat. +- Run NPC identity extraction first and provide its accepted names-only + projection to the interaction extractor for grounding. Registry names may + disambiguate identity but never establish that an interaction occurred, and + registry source references must not be copied into interaction evidence. +- Evaluate category agreement, evidence sufficiency, duplicate behavior, and + smaller-model reliability on human-reviewed transcripts before expanding the + enum or adding additional fields. -### Improve D&D Scene Classification +### Export Accepted Chunk Maps -- Extend scene annotations with classifications that downstream extractors can - use, including reliable combat and narrative indicators. -- Strengthen the scene prompt so every scene containing combat turns is marked - as combat, and add validation capable of detecting missing or inconsistent - combat classifications. -- Allow the combat extractor to no-op for chunks that are not classified as - combat, avoiding unnecessary model calls where practical. -- Allow a narrative extractor to select the corresponding scene classification - rather than processing every chunk indiscriminately. +- Add an option to emit the accepted materialized chunk map as a proper, + framework-owned artifact with a documented schema identity, version, media + type, canonical encoding, and source/chunker provenance. +- Export the exact ordered chunks used for lane execution, including stable + chunk IDs and current-source ranges. Do not expose a model's raw boundary + proposal or require a second model call to reconstruct information already + owned by the framework. +- Treat chunk-map export as an output concern rather than an ordinary + extraction lane. Chunking is pipeline-wide and precedes lane extraction; a + pseudo-extractor would duplicate work and obscure that ownership boundary. +- Keep the generic contract independent of D&D interpretation. Namespaced + annotations may be preserved when they are part of the accepted chunk plan, + but downstream applications should not need domain-specific annotations to + understand chunk identity, order, or source coverage. + +### Extract D&D Scene Descriptions + +- Add a `dnd/scene-descriptions` extractor that runs once for each accepted + scene chunk and explicitly owns the small amount of scene synthesis useful to + downstream applications. +- Keep the private model response to exactly `kind`, `title`, and `summary`. + Use the enum `combat`, `narrative`, `recap`, and `meta`: `narrative` means + current-session in-world gameplay that is not combat, recap, or sustained + out-of-character discussion. +- Treat brief table talk or rules clarification as incidental to the enclosing + gameplay scene. A sustained transition between kinds should normally create + a chunk boundary; define a primary-kind rule for residual mixed chunks before + implementation. +- Deterministically attach the accepted chunk ID and its exact source range + while mapping the private response into the durable artifact. Do not ask the + model to reproduce IDs or segment boundaries, and do not defer required + identity or evidence until normalization. +- Keep normalization limited to stable ordering, exact deduplication, and + canonical invariant enforcement. Titles and summaries are explicit, + source-bounded synthesis owned by this artifact rather than by the chunker. +- Do not add `participants`. Derive participant-oriented views by joining scene + ranges with NPC evidence or, preferably, NPC-interaction occurrences. An NPC + registry reference proves identity, not exhaustive presence in every scene. + +### Minimize And Use D&D Scene Chunking + +- Reduce the D&D scene chunker toward its narrow responsibility: identifying + coherent scene boundaries. Retain boundary confidence or caveats only when a + demonstrated validator or operator workflow consumes them; move title, + summary, scene kind, and participant duties to dedicated artifacts. +- Allow the combat extractor to no-op for chunks classified as non-combat only + after the scene-description artifact can be supplied through an explicit + ordered dependency. Do not make generic chunk materialization depend on a D&D + classification. +- Use ordered pipeline steps whenever a later artifact needs an accepted + earlier artifact as context. Keep independent lanes in the same step and do + not introduce a general DAG or concurrent cross-lane reconciliation model. - Reassess whether one shared scene plan provides enough context for NPC, - spell, combat, and narrative pipelines after these extractors have real-world - usage. Add more complex chunking only in response to demonstrated failures. + spell, combat, interaction, and scene-description lanes after real-world use. + Add more complex chunking only in response to demonstrated failures. ## Shared Normalization And Quality Work diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index 63d459a..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,373 +0,0 @@ -# Minimal D&D Extraction Contracts Implementation Plan - -**Status:** Implemented - -## Objective - -Implement the durable contract redesign defined by -[Minimal D&D Extraction Contracts](minimal-dnd-extraction-contracts.md) and -[ADR-0009](../adr/0009-minimal-evidence-grounded-extraction-artifacts.md). -The result is a coordinated in-place redesign of NPC, spell-cast, and combat-turn -extraction. Each lane must emit only its narrow, evidence-grounded facts; no -removed rich-schema field may survive as an optional field, placeholder, compatibility -shim, or deterministic synthesis. - -This plan deliberately starts with NPCs because the NPC registry is a generated -reference consumed by spell and combat extraction. Spell and combat then change -independently, followed by one repository-wide contract and documentation -pass. - -## Fixed Decisions And Guardrails - -The implementing agent must treat these as decisions, not open design choices: - -- Keep the artifact kinds `dnd/npc-list`, `dnd/spell-list`, and - `dnd/combat-turn-list`, their module keys and capabilities, their media types, - and their prompt IDs stable. -- Retain exactly the existing v1 schema keys, IDs, names, versions, filenames, - prompt IDs, and prompt versions listed in the feature roadmap. Change their - unpublished shapes and content in place; do not harmonize the private spell - identity as part of this work. -- Do not add v2 assets, runtime version negotiation, migration code, dual-write - behavior, a second module registration, or compatibility fixtures for the - superseded pre-release shapes. Existing fixtures should be rewritten or - deleted according to whether they still protect current behavior. -- Require the top-level list field and every record field. A list may be empty, - but every returned record must have at least one source reference. Use strict - JSON objects with unknown fields rejected and no nullable or optional legacy - fields. -- Keep private source references limited to `start_unit_id` and `end_unit_id`. - Mapping assigns the current source document ID; campaign references and an - NPC registry never become event evidence. -- Keep the shared prompt ordering policy: stable shared instructions first, - stable campaign and generated references next, module task material after - those references, and the chunk-variable transcript last. -- Preserve the existing production validator-chain order and the rule that a - configured chain is authoritative. Remove or simplify validators; do not - silently reorder chains. -- Preserve the existing NPC ID derivation algorithm and comparison/display - normalization for canonical names. Retain existing semantic-policy - identifiers as well as schema and prompt versions; pre-redesign development - state is disposable. Unchanged canonical names must not receive new IDs. -- Use deterministic code only for display normalization, known catalog or NPC - canonicalization, application-owned IDs, evidence canonicalization, ordering, - and exact duplicate collapse. Do not introduce fuzzy matching or inferred - enrichment. -- Follow the testing policy: protect schemas and behavior at their owning - boundaries, delete obsolete tests, and avoid tests that snapshot prompt prose - or detect exact shared-prefix length. No live or paid model call is a stage - completion requirement. -- Each stage must leave the repository compiling and `go test ./...` passing. - Update all affected fixtures and callers within the stage that changes a - public Go type; do not leave an intentionally broken intermediate commit. - -## Stage 1: Cut Over The NPC Contract And Name Projection - -This stage establishes the redesigned generated reference on which later stages -depend. - -### 1.1 Replace the durable and private data shapes - -- Reduce `dnd.NPC` in `internal/modules/dnd/types.go` to exactly `ID`, `Name`, - and `SourceRefs`. Delete `NPCRelationship` and every alias, description, and - relationship field or helper that becomes unused. -- Rewrite the existing strict durable schema `dnd_npcs.v1.json` with required - top-level `npcs`, and records containing only required `id`, `name`, and - `source_refs`. Keep the NPC codec at schema version `v1`, name - `notarius_dnd_npcs_v1`, and its existing schema ID and media type. -- Rewrite the strict private schema `dnd_npcs_llm.v1.json`. Its records contain - only required `name` and model-facing source ranges. Keep the extractor schema - identity and prompt ID/version unchanged. -- Reduce the private response DTO and mapping accordingly. Preserve candidate - data at the mapping boundary so deterministic validators, rather than mapping - defaults, reject blank names or invalid ranges. Assign the current source ID - and derive the NPC ID in application code. -- Rewrite the NPC task and instruction assets to ask only for individually - identifiable NPC names and supporting transcript ranges. Explicitly exclude - anonymous groups, generic roles, invented labels, descriptions, aliases, and - relationships. Keep transcript material last in the manifest. - -### 1.2 Simplify NPC identity, normalization, and validation - -- Keep the identity policy at `dnd.npcs.identity.v1`. Retain display - normalization, comparison keys, ID syntax, and deterministic ID derivation. - Validate nonblank canonical names, exact ID/name agreement, duplicate - canonical names, and duplicate IDs. Delete alias validation and alias-specific - issue locations/codes. -- Keep NPC normalization at `dnd.npcs.normalize.v1`. Normalize the retained - display name, derive its ID, canonicalize and deduplicate source references, - consolidate records only by the canonical-name comparison key, preserve the - first stable record, and union exact evidence. Remove alias promotion, - relationship merge/rewrites, and their warning codes. -- Keep the existing NPC shape and source-relatedness policy identifiers. The - shape validator checks only list presence, nonblank `id` and `name`, and - nonempty source references; source-relatedness grounds a record only through - its retained name. The existing source-ref validator remains the owner of - range validity. -- Keep the default validation order domain-first, then JSON Schema, then the - advisory relatedness validator. Simplify only the validator implementations - and selections whose owned behavior changed. - -### 1.3 Separate durable registry provenance from model input - -- Continue to canonicalize and retain the complete redesigned NPC artifact for - registry validation, cache identity, manifests, and provenance. - `Registry.Digest()` remains the digest of that complete canonical artifact - when bound and remains empty when no registry was supplied. -- Build a second structural JSON projection for model grounding with the exact - shape `{"npcs":[{"name":"Mira Thorn"}]}` in normalized registry order. It - contains names only: no IDs, source references, origin URI, aliases, or other - provenance. Generate it with typed values and `json.Marshal`, not string - concatenation. The empty projection is exactly `{"npcs":[]}`. -- Add an explicit projection digest accessor. The digest is SHA-256 over the - exact projected bytes, including for an absent or empty registry, and - `PromptInput().Digest` must equal it. Keep full-artifact and projection digests - distinct even when their content happens to coincide. -- Index `Registry.Lookup` by canonical name only. Retain immutable return values - and defensive copies. Remove alias indexing and alias-aware comments. -- In spell extraction, combat extraction, and combat normalization, use the NPC - name-projection digest for the component-local checkpoint fingerprint because - it exactly describes the names that affect those operations. Keep the full - registry digest and count in manifest metadata for provenance. Framework-owned - generated-reference fingerprints may still invalidate a run when any upstream - artifact byte changes; do not broaden this stage into a framework cache - redesign. - -### 1.4 Update owners, consumers, and tests - -- Update NPC codec, schema, extractor, identity, validator, normalizer, registry, - registration, pipeline-integration, and CLI fixtures to the new shape. Adapt - spell and combat tests that construct `dnd.NPC` values so the repository - remains buildable, but do not change their own artifact contracts yet. -- Add or rewrite focused tests for strict schema acceptance/rejection, - codec round trips, unchanged ID derivation for known names, name-only - consolidation, evidence union, registry immutability, and canonical-name-only - lookup. Rewrite the existing fixture for the minimal current contract; do not - retain the superseded rich fixture solely to test backwards incompatibility. -- Test the projection as a data contract: it contains only ordered names, - equivalent normalized registries produce identical bytes and digest, - evidence/ID-only changes do not change the projection digest, and name/order - changes do. This is not authorization to snapshot assembled prompt prose or - prefix lengths. -- Update `docs/integrations/dnd-npc-artifacts.md` to own the redesigned v1 - durable schema. Update `docs/internal/modules.md` and - `docs/internal/llm.md` only for current NPC behavior that lands in this stage. - -### Stage 1 completion criteria - -- NPC durable and private schemas expose no removed enrichment fields. -- The generated NPC prompt input contains names only while manifests retain - full registry provenance. -- Spell and combat consumers accept generated or external registries in the - redesigned v1 shape; rich pre-redesign registry JSON fails strict decoding. -- Focused NPC, spell-wiring, combat-wiring, integration, and CLI tests pass, and - `go test ./...` passes. - -## Stage 2: Cut Over Spell-Cast Extraction - -### 2.1 Replace the spell contract and prompt - -- Reduce `dnd.SpellCast` to exactly `Caster`, `Spell`, and `SourceRefs`. -- Rewrite `dnd_spells.v1.json` in place. Keep the durable codec at version v1, - name `notarius_dnd_spells_v1`, and its existing schema ID. Its strict record - contains only required `caster`, `spell`, and `source_refs`. -- Rewrite `dnd_spells_llm.v1.json` with the same logical fields and model-facing - ranges. Keep the existing private key, ID, name, prompt version, and schema - path unchanged. -- Reduce the private DTO, canonicalization, and mapping to the retained fields. - Continue assigning current source IDs, stable-ordering candidates by evidence, - and preserving semantically invalid candidates for deterministic validation. -- Rewrite spell task and instruction assets around the narrow casting-event - boundary. Retain the spell catalog and name-only NPC projection as - disambiguation inputs, never evidence. Remove effect, outcome, and narrative - duties and remove the `common-dnd-immediate-resolution.md` message from the - spell manifest and spell asset registration. Do not delete the shared file in - this stage because combat still uses it. - -### 2.2 Simplify spell policies - -- Keep the spell shape policy identifier unchanged and validate only list - presence, nonblank caster/spell, and nonempty source references. -- Keep catalog validation, source-reference validation, and source-relatedness - behavior and policy identities unchanged where their actual semantics are - already limited to spell name, caster, and evidence. -- Simplify normalization and duplicate comparison to caster, catalog-canonical - spell name, and the complete valid evidence set. Remove all prose selection, - copying, fixture fields, and assertions. Preserve catalog fingerprints and - metadata. -- Keep the exact inclusion rule from the feature roadmap: an actual casting or - unambiguous declared attempt is included; mentions, plans, rules discussion, - and catalog matches are not. - -### 2.3 Update owners and tests - -- Update spell codec, private schema, extractor, validators, normalizer, merge - and registration tests, pipeline integration, CLI output fixtures, and any - maintained examples to the minimal shape. Rewrite or delete rich-schema - fixtures rather than retaining them as compatibility cases. -- Replace tests of effects and narrative descriptions with focused tests of the - retained contract: strict unknown-field rejection, codec round trips, current - source-ID assignment, catalog canonicalization, evidence ordering, duplicate - collapse, and NPC name-projection wiring. -- Update `docs/integrations/dnd-spell-artifacts.md` as the canonical redesigned - v1 contract and update current internal module/LLM documentation for the - smaller prompt and response. Do not duplicate the spell catalog contract - owned by its existing integration document. - -### Stage 2 completion criteria - -- No production spell type, schema, prompt, validator, normalizer, fixture, or - documentation contract refers to effect or narrative description. -- Spell prompt/schema identities remain exactly their existing v1 values. -- Focused spell and pipeline tests pass, and `go test ./...` passes. - -## Stage 3: Cut Over Combat-Turn Extraction - -### 3.1 Replace the combat contract and prompt - -- Reduce `dnd.CombatTurn` to exactly `Actor`, `TurnKind`, and `SourceRefs`. - Delete `CombatAction`, `CombatActionCategory`, their constants, and all - now-unused helpers. Retain the existing five `CombatTurnKind` values. -- Rewrite `dnd_combat_turns.v1.json` in place. Keep the durable codec at version - v1, name `notarius_dnd_combat_turns_v1`, and its existing schema ID. Use a - strict required record with `actor`, `turn_kind`, and `source_refs` only. -- Simplify the durable codec to direct strict encoding/decoding if its custom - wire representation exists only to distinguish removed nullable fields. - Preserve presence semantics for the top-level list and strict unknown-field - rejection. -- Rewrite `dnd_combat_turns_llm.v1.json` while retaining its private schema and - prompt identities, and reduce the DTO/mapping to actor, turn kind, and - model-facing ranges. Keep the semantic mapping policy identifier unchanged. -- Rewrite combat task/instruction assets around detecting ordered turns and - discrete interrupting events. Remove round, action, target, declaration, - resolution, outcome, and summary duties. Remove the shared immediate- - resolution message from the combat manifest. -- Once both spell and combat manifests no longer use it, delete - `common-dnd-immediate-resolution.md` and its shared and module asset - registrations. Retain all other shared evidence, identity, reference, NPC, - and transcript assets in their cache-friendly order. - -### 3.2 Simplify combat normalization and validation - -- Keep combat normalization at `dnd.combat_turns.normalize.v1`. Continue to - display-normalize and registry-canonicalize actors, canonicalize evidence, - order records chronologically, and collapse exact duplicates by actor, turn - kind, and complete valid evidence. Delete action/target/prose normalization - and warning codes. -- Keep the invariant policy identifier unchanged. It checks canonical actor - display, canonical and chronological evidence, stable event ordering, and - absence of duplicate event identities; it performs no nested-action checks. -- Keep the shape policy identifier unchanged and validate only list presence, - nonblank actor, allowed turn kind, and nonempty source references. -- Keep the source-relatedness policy identifier unchanged and compare only the - actor against cited transcript material. Remove declaration-token heuristics - and their now-unused helpers. Keep source-reference validation unchanged if - its semantics did not change. -- Simplify merge/clone behavior to copy only retained values and source refs. - Preserve stable ordering and defensive ownership. - -### 3.3 Update owners and tests - -- Update codec, private schema, extractor, normalizer, validators, merge, - registration, pipeline integration, CLI fixtures, and examples to the minimal - shape. Rewrite or delete rich-schema fixtures rather than retaining them as - compatibility cases. -- Delete tests whose sole policy was round/action/summary handling. Add or - rewrite focused tests for strict schemas, enum validation, mapping and - source-ID assignment, actor canonicalization through the redesigned NPC - registry, chronology, exact duplicate collapse, and invariant validation. -- Update `docs/integrations/dnd-combat-turn-artifacts.md` as the canonical - redesigned v1 contract and update current internal module/LLM documentation - for the implemented behavior. - -### Stage 3 completion criteria - -- No production combat type, prompt, schema, policy, normalizer, fixture, or - current documentation refers to rounds, actions, summaries, declarations, - targets, resolutions, or action categories. -- The immediate-resolution shared asset has no remaining registration or file. -- Focused combat and pipeline tests pass, and `go test ./...` passes. - -## Stage 4: Complete The Repository-Wide Cutover - -### 4.1 Audit contract identity and stale surface area - -- Search code, embedded assets, tests, examples, and current documentation for - all removed field names and unintended v2 schema/prompt names. Removed fields - may remain only in historical ADR or roadmap context; active schema, prompt, - fixture, and current-behavior surfaces must describe the minimal v1 contract. -- Verify the three artifact registrations still bind their original kinds and - exact redesigned Go types through extract, merge, normalize, codec, and - validators. -- Verify all prompt manifests still point at their v1 private schemas, stable - reference material precedes module-variable material, and transcript content - remains last. Do not add a change-detector test for prompt message count, - prose, shared-prefix content, or prefix length. -- Verify pre-redesign development checkpoints naturally miss through changed - prompt/schema content and generated-dependency fingerprints. Do not add - migration or compatibility handling for disposable pre-release state. - -### 4.2 Exercise representative assembled behavior - -- Keep unit case matrices at their owning schema, validator, normalizer, and - registry boundaries. At the assembled-pipeline boundary, retain only - representative tests proving each redesigned lane is registered, strict - JSON/schema failures are attributed to the correct validator, semantic - failures are attributed to the correct domain validator, and generated NPC - output is accepted by later spell/combat stages. -- Cover an ordered multi-step run in which NPC extraction produces the minimal - artifact and spell/combat consume its name projection. Assert that downstream - event source references point only to the current transcript and not to NPC - registry evidence. -- Verify CLI logical output and manifests retain the v1 schema/prompt identities, - full NPC registry provenance, and the relevant component fingerprints without - exposing prompt bodies or reference payloads. - -### 4.3 Finish documentation and lifecycle state - -- Reconcile the three integration documents, `docs/internal/modules.md`, and - `docs/internal/llm.md` with the final code. Correct any stale validator-order - descriptions while doing so; current docs must describe the registered order, - not preserve an older generic ordering example. -- Mark ADR-0009 `Accepted` and the feature roadmap `Implemented` when the code, - tests, and current-behavior documentation all land. Remove the completed item - from `docs/roadmap/future.md`; retain the feature roadmap and ADR as design - rationale unless the repository's normal roadmap-retirement practice calls - for moving the completed roadmap later. -- Record any human-reviewed rich/minimal model evaluation separately from CI - results. Evaluation may motivate later prompt tuning but does not reopen the - approved durable minimal field set within this implementation. - -### 4.4 Final verification - -Run, in order: - -```sh -gofmt -w -go test ./... -go vet ./... -go build ./cmd/notarius -go test -race ./internal/modules/dnd/... -git diff --check -``` - -If the repository-wide race command exposes an unrelated, pre-existing failure, -document it with the narrower affected package result; do not weaken or skip -ordinary tests for the changed D&D packages. - -### Stage 4 completion criteria - -- All three lanes use only their minimal v1 contracts from model response - through durable output. -- No compatibility shim, removed-field policy, unintended v2 asset, or stale - current-behavior documentation remains. -- Generated NPC references are name-only for LLM input and remain full-fidelity - for durable provenance. -- Repository tests, vet, build, race checks for the changed domain, and diff - hygiene checks pass. - -## Open Questions - -None. The feature roadmap, ADR, and fixed decisions above define the cutover, -pre-release schema, evidence, projection, validation, testing, and documentation -policies needed to implement each stage without further product decisions. diff --git a/docs/roadmap/minimal-dnd-extraction-contracts.md b/docs/roadmap/minimal-dnd-extraction-contracts.md deleted file mode 100644 index 19e3606..0000000 --- a/docs/roadmap/minimal-dnd-extraction-contracts.md +++ /dev/null @@ -1,220 +0,0 @@ -# Minimal D&D Extraction Contracts - -**Status:** Implemented - -## Intent - -Redesign the D&D spell, NPC, and combat-turn artifacts around the principle in -[ADR-0009](../adr/0009-minimal-evidence-grounded-extraction-artifacts.md): -each extractor should answer one narrow question with the smallest useful set -of source-grounded fields. - -The redesign favors extraction precision, evidence quality, valid-output rate, -smaller-model reliability, and lower prompt and response cost over descriptive -richness. It removes synthesis responsibilities rather than preserving obsolete -fields as optional, nullable, empty, or application-generated placeholders. - -## Goals - -- Make every model-produced field necessary to the artifact's core question. -- Require direct transcript evidence for every extracted record. -- Remove overlapping prose, inferred enrichment, and nested structures without - a demonstrated consumer. -- Keep catalog and identity references as disambiguation aids rather than - evidence. -- Preserve deterministic canonicalization, evidence ordering, exact - deduplication, identity assignment, and bounded domain diagnostics where - those responsibilities still apply. -- Reduce downstream prompt material to the fields a consumer actually needs. -- Keep the unpublished v1 identities while replacing their pre-release shapes - in place. - -## Non-Goals - -- Generating session narrative, rules analysis, biographies, relationship - graphs, encounter summaries, or prose descriptions. -- Preserving removed fields for source compatibility through empty strings, - nullable values, or synthetic defaults. -- Adding fuzzy entity resolution, LLM-assisted enrichment, or a general schema - migration framework. -- Treating campaign references, catalogs, or earlier artifacts as evidence that - an event occurred in the current transcript. -- Combining the three D&D artifact families into one model call. - -## Shared Contract Policy - -All three artifacts remain ordered lists. Each record contains at least one -source reference, and the complete reference collection supports every -model-produced field in that record. Source IDs continue to be assigned by the -application from the current input; the model returns only source-unit ranges. - -Private LLM schemas remain strict about their transport envelope: required -fields, JSON types, object and array shape, nullability where applicable, and -unknown-field rejection. Deterministic validators continue to own semantic -rules such as nonblank identities, catalog membership, enum membership, -positive and resolvable source units, and canonical normalized invariants. - -Prompts retain the shared D&D evidence, identity, reference, NPC-grounding, and -transcript assets that remain relevant. Module-specific task and instruction -assets must delete duties associated with removed fields. The existing -cache-friendly ordering keeps stable shared and reference material before the -chunk-variable transcript. - -## Spell Cast - -The spell extractor answers: - -> Which spell was cast, by which in-world caster, and where is that event -> established in the source? - -The artifact kind remains `dnd/spell-list`. Its durable schema remains v1. -Each spell-cast record contains exactly: - -- `caster`: required nonblank in-world display identity; -- `spell`: required nonblank canonical or catalog-resolvable spell name; and -- `source_refs`: one or more current-source evidence ranges. - -The model-facing response contains the same fields except for application-owned -`source_id` values within references. The current `effect` and -`narrative_description` fields are removed from the private response, public Go -type, durable schema, codec, validators, normalizer, fixtures, and integration -contract. - -The inclusion boundary remains an actual casting event or an unambiguously -declared casting attempt, not a spell mention, hypothetical plan, rules -discussion, or catalog match. The spell catalog helps recognize and canonicalize -the name but never establishes that a cast occurred. - -Normalization continues to canonicalize spell names, canonicalize evidence, -and collapse exact duplicate events using caster, canonical spell name, and -complete valid evidence. It performs no prose selection or merging. - -## NPC Registry - -The NPC extractor answers: - -> Which individually identifiable non-player characters are established in the -> source, and where is each identity established? - -The artifact kind remains `dnd/npc-list`. Its durable schema remains v1. -Each durable NPC record contains exactly: - -- `id`: deterministic application-assigned identity derived under the NPC - identity policy; -- `name`: required nonblank source-supported display identity; and -- `source_refs`: one or more evidence ranges supporting that identity. - -The private model response omits `id` and reference `source_id` values. A -`name` may be a proper name or a stable, individually distinguishing title or -alias supported by the transcript. The extractor does not invent descriptive -labels for anonymous creatures, crowds, or generic roles. - -The current `aliases`, `description`, and `relationships` fields are removed -from the private response, public Go type, durable schema, codec, validators, -normalizer, registry, fixtures, and integration contract. Normalization -consolidates only identities supported by the retained name policy and unions -exact evidence; it does not infer alias equivalence or relationships. - -Spell and combat consumers receive a prompt projection containing only the -canonical NPC names needed for identity grounding. Application-owned NPC IDs -remain available to deterministic registry and normalization code but are not -sent to a model that cannot return or otherwise consume them. NPC source -references remain provenance in the durable registry and are not included as -current-transcript evidence or copied into downstream event artifacts. - -Encounter context is deliberately not a scalar NPC registry field. Dialogue, -combat alignment, presence, or third-party mention can vary across occurrences. -If a demonstrated consumer needs that information, add a separate ordered -NPC-occurrence artifact whose records contain `name`, a small mutually -exclusive context enum, and `source_refs`. A candidate starting vocabulary is -`dialogue`, `combat_ally`, `combat_opponent`, `noncombat_presence`, `mentioned`, -and `other`; its exact semantics require a separate feature decision. - -## Combat Event - -The combat extractor answers: - -> Which in-world participant took a turn or discrete interrupting combat -> event, what kind of event was it, and where is it established in the source? - -The existing `dnd/combat-turn-list` artifact kind and v1 durable-schema identity -remain. Each record contains exactly: - -- `actor`: required nonblank in-world display identity; -- `turn_kind`: one of `turn`, `reaction`, `legendary_action`, `lair_action`, or - `other`; and -- `source_refs`: one or more current-source evidence ranges. - -The current `round`, `actions`, and `summary` fields, including nested action -categories, declarations, targets, and resolutions, are removed from the -private response, public Go types, durable schema, codec, validators, -normalizer, fixtures, and integration contract. - -Normalization continues to display-normalize and registry-canonicalize actors, -canonicalize evidence, order events by source position, and collapse exact -duplicates using actor, turn kind, and complete valid evidence. It no longer -normalizes targets, declarations, summaries, or resolutions. - -If action-level facts later have a demonstrated consumer, they belong in a -separate combat-action artifact rather than restoring a nested synthesis -contract to combat-turn detection. Spell casts and future item events remain -owned by their dedicated artifact lanes. - -## Pre-Release Schema Policy - -Notarius and these contracts are pre-release. The existing v1 artifacts and -private model-response schemas have not been published as compatibility -contracts, so their shapes change in place. The implementation does not retain -the rich pre-redesign shape, add v2 assets, migrate old output, support multiple -versions, or preserve old fixtures solely for compatibility testing. - -Artifact kinds, schema keys and IDs, schema names and versions, prompt IDs and -versions, module keys, capabilities, and media types all remain unchanged. -Changing prompt and schema content invalidates the relevant content-addressed -development state; any remaining pre-redesign local output or cache is -disposable and may be regenerated. - -| Lane | Durable v1 schema | Private model-response v1 schema | Prompt ID | -| --- | --- | --- | --- | -| Spell cast | ID `notarius.dnd.spells`, name `notarius_dnd_spells_v1` | key `dnd_spells`, ID `notarius.dnd.spells`, name `notarius_dnd_spells_v1` | `dnd.spells` | -| NPC registry | ID `notarius.dnd.npcs`, name `notarius_dnd_npcs_v1` | key `dnd_npcs_llm`, ID `notarius.dnd.npcs.llm`, name `notarius_dnd_npcs_llm_v1` | `dnd.npcs` | -| Combat event | ID `notarius.dnd.combat_turns`, name `notarius_dnd_combat_turns_v1` | key `dnd_combat_turns_llm`, ID `notarius.dnd.combat_turns.llm`, name `notarius_dnd_combat_turns_llm_v1` | `dnd.combat_turns` | - -## Quality And Evaluation - -The implemented contract should protect the remaining meaningful risks: - -- private schemas accept only the new structural envelopes; -- durable codecs strictly round-trip the redesigned contracts; -- domain validators own blank identities, enum and catalog membership, and - invalid evidence; -- normalizers preserve source-grounded values while applying only their stated - deterministic transformations; -- generated NPC references expose the minimal identity projection and never - become event evidence; and -- representative assembled pipelines attribute retries and rejections to the - owning domain boundary. - -Post-cutover model evaluation should compare the rich-schema baseline and the -minimal-schema result on a small human-reviewed transcript set using: - -- event/entity precision and recall; -- caster, NPC, and combat-actor attribution accuracy; -- source-range validity and evidence sufficiency; -- catalog and enum accuracy; -- structurally valid completion rate and exhausted-retry rate; -- unsupported-claim rate; and -- input/output tokens, latency, and model cost. - -The evaluation exists to identify prompt or model-quality follow-up work, not -to gate the approved minimal contract on live-provider behavior. Human review -and live model calls are evaluation aids rather than deterministic CI gates. - -## Documentation Ownership - -The spell, NPC, and combat integration documents are the canonical owners of -their redesigned durable schemas. Internal LLM and module documents own the -corresponding current prompt, DTO, validator, normalizer, and NPC -prompt-projection behavior. This roadmap records the implemented design and -policy; [the implementation plan](implementation.md) records the completed -sequencing and completion criteria.