eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Close out the D&D combat turn extraction roadmap

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-21 07:49:57 -05:00
parent a1f5dce405
commit 110593ece1
15 changed files with 318 additions and 804 deletions
Download Patch File Download Diff File Expand all files Collapse all files

515
docs/roadmap/implementation.md
View File

@@ -1,515 +0,0 @@
# D&D Combat-Turn Extraction Implementation Plan
Status: Complete.
Implement this plan in order. The feature policy and target state are defined
in [D&D Combat-Turn Extraction](dnd-combat-turn-extraction.md); this document
owns implementation sequencing and concrete technical decisions.
Do not introduce a DAG, prior-run discovery, encounter state engine, scene-based
provider-call suppression, LLM-backed validator, or semantic reconciliation.
Preserve the fixed `input -> chunk -> extract -> merge -> normalize -> output`
architecture and the existing framework retry, warning, rejection, checkpoint,
debug, and output contracts.
## Cross-Stage Decisions
### Production Identities
Use these exact identities:
- artifact kind: `dnd/combat-turn-list`;
- extractor key and normalizer key: `dnd/combat-turns`;
- extractor capability: `dnd.combat_turns`;
- prompt ID: `dnd.combat_turns`, version `v1`;
- private response-schema key: `dnd_combat_turns_llm`;
- private schema ID: `notarius.dnd.combat_turns.llm`;
- private schema name: `notarius_dnd_combat_turns_llm_v1`;
- durable schema ID: `notarius.dnd.combat_turns`;
- durable schema name: `notarius_dnd_combat_turns_v1`;
- durable schema version: `v1`; and
- durable media type: `application/json`.
The private response schema omits `source_id`; mapping code assigns the current
source identity. No combat-turn ID is added in v1.
### Canonical Go And JSON Shape
Add these types to the canonical D&D model:
```go
type CombatTurnList struct {
CombatTurns []CombatTurn `json:"combat_turns"`
}
type CombatTurn struct {
Actor string `json:"actor"`
TurnKind CombatTurnKind `json:"turn_kind"`
Round *int `json:"round"`
Actions []CombatAction `json:"actions"`
Summary string `json:"summary"`
SourceRefs []source.SourceRef `json:"source_refs"`
}
type CombatAction struct {
Category CombatActionCategory `json:"category"`
Declaration string `json:"declaration"`
Targets []string `json:"targets"`
Resolution *string `json:"resolution"`
}
```
Define string types and only these lowercase durable values:
- `CombatTurnKind`: `turn`, `reaction`, `legendary_action`, `lair_action`,
`other`;
- `CombatActionCategory`: `attack`, `spell`, `movement`, `item`,
`ability_check`, `saving_throw`, `condition`, `other`.
All object fields and the top-level `combat_turns` array are required.
`combat_turns` may be empty. Every turn requires a non-empty actor, supported
turn kind, at least one action, non-empty summary, and at least one source
reference. `round` is either `null` or a positive integer. Every action requires
a supported category, non-empty declaration, and present `targets` array;
targets may be empty but may not contain empty strings. `resolution` is either
`null` or non-empty after trimming. Unknown fields are rejected at every object
level.
Strict raw decoding must distinguish a missing required `round` or `resolution`
key from an explicitly present JSON `null`, even though the accepted Go value
represents `null` with a nil pointer. Use presence-aware wire decoding or raw
key validation at codec and private-response boundaries; do not silently turn a
missing nullable field into an accepted null.
One turn-level source-reference collection collectively supports the actor,
kind, round, actions, summary, targets, and resolutions. Do not add per-action
references in v1.
### Ordering And Duplicate Identity
Source-document slice position is chronology. Numeric unit IDs are identifiers
only. An item's earliest evidence position is the minimum valid start-unit
index across all its ranges. Stable ordering puts records with valid evidence
in ascending earliest-position order, preserves encounter order for ties, and
puts records without valid evidence after valid records without reordering
them.
The normalizer may collapse two records only when all of these match:
- actor under `npcs/identity.ComparisonKey`;
- exact `turn_kind`;
- both round values, including `nil` versus a value; and
- the complete, non-empty, canonically ordered set of exact source references.
Every reference in the duplicate key must pass `source.ValidateRef` against the
current document. Invalid or empty evidence never participates in duplicate
collapse. Actions and prose do not participate in identity because two model
representations of the same cited turn should collapse; retain the complete
first record without merging prose or action lists.
### NPC Registry Contract And Prompt Reuse
Promote NPC registry preparation from the spell extractor into
`internal/modules/dnd/npcs/registry`. It owns:
- slot name `npcs` and maximum size 1,048,576 bytes;
- exact-one-item and `application/json` validation when bound;
- approved NPC codec decoding, whole-registry identity validation, canonical
re-encoding, semantic SHA-256 digest, and content-safe bounded failures;
- the exact unbound prompt value `{"npcs":[]}`;
- immutable accessors for bound state, a defensive NPC list, canonical bytes,
digest, count, and prompt input; and
- exact canonical-name/alias lookup using the NPC identity comparison policy.
Registry source references may identify another session and are not checked
against the current source. The resolver never returns content, names, aliases,
paths, or raw decoder errors in failures. Identity failures use issue codes and
record/alias indexes under the established 20-issue, 128-rune, and 4,096-byte
limits.
Move the generic bounded diagnostic helper from `dnd/npcs/diagnostics` to
`dnd/shared/diagnostics`, updating existing NPC consumers. This is an internal
package move with no diagnostic-policy change.
Add `common-dnd-npcs.md` to the shared D&D prompt assets. Its generic policy
allows exact canonical participant names and alias recognition while stating
that registry content is context, not event evidence. Both spell and combat
prompts use this exact fragment immediately after the existing shared campaign
reference message. Remove the spell-owned NPC fragment and include the shared
fragment in both prompt hashes. This intentionally changes the spell prompt
fingerprint once but must not change its request inputs, metadata shape,
registry semantics, or output contract.
Both combat extraction and normalization declare the same optional registry
slot. A bound registry contributes manifest metadata `npc_registry_digest` and
`npc_count` and a local checkpoint fingerprint `npc_registry`. An absent slot
contributes neither metadata fields nor that fingerprint.
### Validation And Diagnostics
Add deterministic validators with these exact identities:
| Validator key | Result reason | Policy fingerprint |
| --- | --- | --- |
| `extract/dnd/combat-turns/shape` | rejection `invalid_combat_turn_shape` | `dnd.combat_turns.validator.shape.v1` |
| `extract/dnd/combat-turns/source_refs` | rejection `invalid_combat_turn_source_refs` | `dnd.combat_turns.validator.source_refs.v1` |
| `extract/dnd/combat-turns/source_relatedness` | warning `combat_turn_not_near_source` | `dnd.combat_turns.validator.source_relatedness.v1` |
| `normalize/dnd/combat-turns/invariants` | rejection `invalid_combat_turn_normalization` | `dnd.combat_turns.validator.normalized.v1` |
Shape owns required arrays, enum membership, nullable fields, positive rounds,
and non-empty strings. Source-reference validation owns source identity, unit
existence, and range order through `source.ValidateRef`. Later validators defer
when shape is invalid; relatedness also defers when any source range is invalid.
Relatedness combines the turn's cited units once in document order, removing
overlap. The actor is related when its NPC comparison key is a substring of the
comparison-normalized cited text. For each action declaration, split its
comparison-normalized form on runes that are not Unicode letters or digits,
retain tokens of at least four Unicode code points, and require at least one
retained token to occur as a complete cited-text token. No retained token means
the declaration is unrelated. Emit at most one approved warning per turn,
listing whether the actor and which action indexes were not related. Do not
check targets deterministically.
The normalize invariant validator requires display-normalized strings,
comparison-unique targets within each action, canonical exact source-reference
order without duplicates, chronological record order for valid evidence, and
absence of the duplicate identity defined above. It does not require every
actor or target to exist in the optional NPC registry.
All rejection and warning messages use the shared diagnostic helper: at most 20
displayed issues, at most 128 Unicode code points per displayed value, valid
UTF-8, at most 4,096 bytes, Go quoting for control characters, and the exact
total omitted count. Validators use strict empty-options decoders and provide
one local `policy` checkpoint fingerprint with the value in the table.
### Normalization Policy
Use normalization policy `dnd.combat_turns.normalize.v1` and these warning
reason codes:
- `combat_turn_fields_normalized`;
- `combat_actor_canonicalized`;
- `combat_target_canonicalized`;
- `source_references_normalized`;
- `combat_turns_reordered`; and
- `duplicate_combat_turn_collapsed`.
Deep-clone all nested slices and pointers. Normalize actor, summary,
declarations, targets, and non-null resolutions by collapsing Unicode
whitespace through `npcs/identity.NormalizeDisplay`. Preserve `nil` resolution.
Remove comparison-duplicate targets while retaining the first display value and
target order. Do not deduplicate or reorder actions.
When a registry is bound, rewrite an actor or target only if its comparison key
matches exactly one validated canonical name or alias. Preserve unmatched
values. Registry validation makes ambiguous lookup impossible at preparation;
do not guess or perform fuzzy matching.
Sort every source-reference list by exact `source_id`, `start_unit_id`, and
`end_unit_id`, then remove exact duplicates. Stable-sort records by the
chronology rule before duplicate detection. Collapse duplicates in that order
and retain the first record unchanged after its per-record normalization.
Warning scopes use merged input indexes such as `combat_turns[3]`, even after
sorting or collapse. Emit one bounded warning for each affected record or
collapsed group; group warnings identify the retained and removed input indexes.
Only warnings from a validator-approved attempt become durable, under existing
framework policy.
The normalizer exposes manifest metadata `normalization_policy`,
`identity_policy`, and optional registry digest/count. Its checkpoint
fingerprints are `normalization_policy`, `identity_policy`, and optional
`npc_registry`; bump the normalization policy when any transformation,
ordering, or duplicate rule changes.
### Testing Rules
Follow `docs/policy/testing.md`. Tests are offline and deterministic. Use a fake
structured LLM only at the completion boundary. Protect durable shapes, domain
invariants, preparation failures, reference sensitivity, registration, and one
representative assembled workflow.
Do not add prompt-prose change detectors or tests requiring particular words or
phrases. Prompt tests may verify registration, message/input wiring, prompt and
schema identities, stable shared-fragment use, and absence of raw content from
metadata or diagnostics. Human evaluation owns semantic output quality.
## Stage 1: Shared NPC Registry And Prompt Grounding
### Goal
Create one reusable, content-safe NPC registry boundary before adding a second
consumer, while preserving spell behavior.
### Changes
- Add the domain registry resolver and immutable lookup described above; migrate
spell extraction from its private resolver without changing the spell
extractor's public module contract.
- Relocate bounded D&D diagnostics to the shared domain package and update all
NPC imports and tests.
- Add the shared NPC prompt fragment, move the spell prompt to it, place it
immediately after shared campaign references, and remove the spell-owned
fragment.
- Keep raw reference provenance in framework identity independently from the
semantic registry fingerprint. Preserve the exact empty prompt input and all
existing content-safe error behavior.
- Update the internal overview, module, and LLM documentation in this stage so
the implemented registry owner and shared prompt-fragment ownership remain
accurate while the later combat work remains independently scoped.
### Tests
- Move resolver contract tests to the domain package: absent, valid, formatted-
equivalent, malformed, unknown-field, invalid-ID, collision, media type,
item count, byte limit, defensive copies, lookup, digest, and bounded
content-free diagnostics.
- Retain spell-level tests only for spell request wiring, metadata/fingerprint
behavior, and no-regression output; remove duplicated resolver cases.
- Verify the shared prompt asset registers for spells and hashes the shared
fragment without asserting its prose.
### Completion Check
Run `go test ./internal/modules/dnd/npcs/... ./internal/modules/dnd/extract/spells ./internal/modules/dnd/validate/npcs/...`
and `git diff --check`.
## Stage 2: Combat Domain Contract And Codec
### Goal
Establish the typed artifact and durable serialization boundary without making
the combat module selectable.
### Changes
- Add the canonical combat types, enums, constants, and artifact kind using the
exact shape and values above.
- Add `internal/modules/dnd/codec/combatturns`, following the existing candidate
versus approved codec boundary:
- strict single-value JSON with unknown-field rejection;
- candidate encode/decode preserving validator-visible invalid values;
- approved encode/decode enforcing structural validity only;
- defensive schema and metadata values; and
- metadata containing `combat_turn_count` only.
- Add the durable v1 JSON Schema with required nullable fields, enum values,
source-reference shape, array rules, and `additionalProperties: false`.
- Add `docs/integrations/dnd-combat-turn-artifacts.md`, describing only the
implemented artifact and codec at this stage.
### Tests
- Test approved round trips and candidate preservation for every nullable,
enum, required-array, required-string, target, and source-reference boundary.
- Test malformed, trailing, unknown-field, and invalid structural input without
snapshotting complete errors.
- Test defensive schema/metadata copies, nil versus present-empty arrays, and
exact codec identity.
- Keep one compact durable v1 fixture for intentional compatibility coverage.
### Completion Check
Run `go test ./internal/modules/dnd/codec/combatturns` and `git diff --check`.
## Stage 3: Combat Extractor And Extraction Validators
### Goal
Implement package-complete LLM extraction and deterministic candidate
validation without production composition.
### Changes
- Add `internal/modules/dnd/extract/combatturns` with the exact module, prompt,
schema, capability, artifact, and reference identities above. It accepts no
options and requires `chunks` and `source.transcript`. The prompt manifest is
`dnd.combat_turns.yaml`, uses default profile `gemini-2-flash`, JSON Schema
validation, and zero provider-side repair attempts so pipeline retries remain
the only extraction retry policy.
- Declare existing optional `players`, `party`, `glossary`, and deprecated
`roster` slots plus the optional structured NPC registry slot. Reuse shared
D&D prompt inputs and the shared NPC prompt fragment.
- Embed a private response schema matching the durable shape except that source
ranges omit `source_id`. Require present arrays and nullable round/resolution
values exactly as in the durable contract.
- Prompt for the inclusion, exclusion, chronology, identity, round restraint,
and immediate-resolution policy in the feature roadmap. Instruct the model
that campaign and NPC references disambiguate identities but are never combat
evidence.
- Preserve malformed response values for validators. Canonicalize and exactly
deduplicate source ranges, assign the current source ID, and stable-sort
records by earliest valid source-document position. Do not merge records,
canonicalize NPC names deterministically, or skip any chunk in the extractor.
- Expose prompt and private-schema manifest metadata plus optional registry
digest/count. Fingerprint local names `prompt`, `response_schema`,
`mapping_policy` with value `dnd.combat_turns.extract_mapping.v1`, and optional
`npc_registry`.
- Implement shape, source-reference, and source-relatedness validators with the
exact contracts and policies above.
### Tests
- Through a fake LLM, test request identity, profile/session propagation,
chunk-scoped source input, campaign and NPC inputs, response mapping, source
assignment, non-monotonic-unit chronology, invalid-candidate preservation,
cancellation, and contextual provider failures.
- Test prompt/schema registration, required input wiring, metadata and
fingerprint sensitivity, defensive copies, and content redaction without
asserting prompt prose.
- Test each validator's meaningful approval, rejection, deferral, and warning
categories, including Unicode comparison, overlapping evidence, declaration
tokenization, bounded diagnostics, strict options, and typed registration.
- Verify that a bound NPC registry affects grounding metadata and checkpoint
identity but never supplies combat source references.
### Completion Check
Run `go test ./internal/modules/dnd/extract/combatturns ./internal/modules/dnd/validate/combatturns/...`
and `git diff --check`.
## Stage 4: Combat Normalization And Final Invariants
### Goal
Implement conservative identity canonicalization, chronological ordering, and
exact-evidence duplicate collapse.
### Changes
- Add `internal/modules/dnd/normalize/combatturns` with key
`dnd/combat-turns`, requirements `merged`, capability `normalized`, no
options, and the optional NPC registry slot.
- Resolve and retain the immutable registry during preparation. Runtime uses
the prepared view; it does not parse references repeatedly.
- Apply the exact per-record, registry, source-reference, chronology, duplicate,
warning, metadata, and fingerprint policies above. Never mutate or retain
caller-owned slices, pointers, registry data, or source data.
- Add the normalized-invariants validator. It remains deterministic, accepts no
references of its own, and defers malformed shape or invalid source evidence
to their owning validators.
### Tests
- Use table-driven cases for whitespace, nullable resolution, target
deduplication, actor/target registry matches, unmatched names, reference
normalization, non-monotonic source IDs, stable ties, and warning scopes.
- Cover duplicate collapse and non-collapse for every identity dimension,
especially invalid evidence, distinct ranges, different turn kinds, and
`nil` versus numbered rounds.
- Prove actions/prose come only from the first retained record, merged input is
immutable, and all nested output storage is independent.
- Test absent and bound registries, preparation failures, metadata,
fingerprints, policy sensitivity, cancellation, strict options, and module
registration.
- Test normalized-invariant rejection for each owned invariant without
duplicating shape and source-validator case matrices.
### Completion Check
Run `go test ./internal/modules/dnd/normalize/combatturns ./internal/modules/dnd/validate/combatturns/...`
and `git diff --check`.
## Stage 5: Production Composition, Sequential Workflow, And Documentation
### Goal
Make the complete combat lane selectable, verify the assembled workflow, and
document current behavior in canonical locations.
### Changes
- Extend the D&D registrar with the combat codec, extractor and prompt assets,
typed append-order merger, combat normalizer, typed no-op normalizer, all four
validators, and typed always-accept/always-reject variants.
- Add append behavior that preserves present-empty versus nil output and chunk
record order before normalization.
- Register the extract default chain in this order:
`generic/valid_json`, `generic/valid_json_schema`, combat shape, combat source
references, combat source relatedness.
- Register the normalize default chain in this order:
`generic/valid_json`, `generic/valid_json_schema`, combat shape, normalized
invariants, combat source references, combat source relatedness.
- Do not add a merge validator chain or change framework defaults.
- Add `examples/dnd-combat-turns.config.yml` and
`examples/dnd-npc-combat-sequential.config.yml`. Use version 3, safe relative
state paths, explicit checkpoint `enabled: false`, a generic chunker, combat
extraction with `retries: 2`, and combat normalization. Keep the dynamic NPC
output unbound in the static sequential example.
- Demonstrate runtime binding to both stage-local slots with selectors
`combat.extract.npcs=<npc-run>/lanes/npcs.json` and
`combat.normalize.npcs=<npc-run>/lanes/npcs.json`.
- Update canonical current-behavior documentation:
- Configuration owns module/validator catalogs, reference slots, limits,
default chains, and maintained examples;
- CLI owns the explicit two-selector invocation syntax;
- Operations owns the independent NPC-then-combat workflow and state
sensitivity;
- the combat integration contract owns durable fields, enum values,
evidence, normalization, warnings, and manifest metadata;
- the NPC integration contract notes combat as a consumer without redefining
combat fields;
- JSON output links the new lane payload; and
- internal overview, module, and LLM docs describe concrete packages,
preparation, prompt reuse, and fingerprints.
- At completion, mark this plan and the feature roadmap complete. Remove the
implemented combat proposal from `future.md` while retaining scene-routing,
narrative, generic deduplication, and other unimplemented work.
### Tests
- Extend registrar tests for production keys, typed variants, prompt assets,
default chain order, nil dependencies, and duplicate registration behavior.
- Add config/example resolution coverage for stage-local NPC bindings,
capabilities, codec compatibility, strict options, and invalid validator
placement.
- Add one assembled pipeline integration covering extract, retry-capable
validation, append merge, registry-backed normalization, final validation,
JSON output, warnings, manifest metadata, and checkpoint fingerprints using a
fake LLM.
- Add one sequential integration that materializes normalized NPC output as
both combat references and verifies canonical actor/target output, reference
provenance, and that NPC source ranges never become combat evidence.
- Add preparation-boundary coverage showing malformed or oversized NPC input
fails before checkpoint construction or pipeline execution. Do not duplicate
the shared resolver's complete malformed-input matrix.
- Validate maintained examples and documentation links through existing test
mechanisms.
### Completion Check
Run the final verification suite.
## Final Verification
Run:
```sh
git diff --check
go test ./...
go vet ./...
go build ./cmd/notarius
go test -race ./internal/modules/dnd/... ./internal/framework/pipeline \
./internal/cli ./internal/modules/integration
```
Review the final diff for:
- D&D or provider behavior leaking into generic framework packages;
- accidental scene-based call suppression or automatic pipeline composition;
- mutation or aliasing of typed artifacts, schemas, references, metadata, or
fingerprints;
- transcript, registry, prompt, schema, path, or decoder content leaking into
errors, manifests, fingerprints, or redacted summaries;
- prompt-prose change-detector tests, redundant cross-layer cases, or exact LLM
output goldens;
- current-behavior documentation claiming features before the implementing
stage lands; and
- unrelated worktree changes.
## Open Questions
None. Artifact fields and enums, nullable behavior, chronology, duplicate
identity, reference reuse, prompt composition, validator placement,
normalization, checkpoint semantics, documentation ownership, and stage
boundaries are fixed by this plan.