Compare commits

...

15 Commits

62 changed files with 5322 additions and 154 deletions

View File

@@ -103,10 +103,11 @@ go run ./cmd/notarius run dnd-session \
--reference spells.extract.glossary=./campaign-glossary.txt
```
For the maintained NPC-grounded workflow, use the explicit ordered pipeline.
The first step produces the normalized NPC artifact; the second step receives
it in memory and fans it out to spell extraction, combat extraction, and combat
normalization:
For the maintained ordered D&D workflow, use the explicit pipeline. The first
step produces normalized NPC and scene-description artifacts; the second step
receives the NPC artifact in memory for spell extraction, combat extraction, and
combat normalization, and receives the required scene-description artifact for
combat eligibility:
```sh
go run ./cmd/notarius run dnd-session \
@@ -118,6 +119,10 @@ go run ./cmd/notarius run dnd-session \
The generated NPC content remains contextual grounding, not spell or combat
evidence. It is represented in manifests and debug summaries by bounded
identity and producer provenance, not by payload content or a filesystem path.
The scene-description artifact is control context: combat extraction calls its
LLM only for an exact `combat` scene match. See the
[D&D combat-turn artifact contract](integrations/dnd-combat-turn-artifacts.md)
for the resulting empty-output and warning behavior.
The same grammar can target chunk, merge, and normalize slots when the configured
modules declare them:

View File

@@ -22,9 +22,9 @@ The explicit-path option is defined in the [CLI reference](cli.md).
- [Minimal D&D configuration](../examples/dnd-minimal.config.yml) shows the
shortest useful single-step, single-lane pipeline.
- [Complete D&D configuration](../examples/dnd-complete.config.yml) shows scene
chunking, ordered steps, all five D&D artifact lanes, a generated NPC
handoff, campaign and spell-catalog references, checkpoint storage, and
chunk-map output.
chunking, ordered steps, all six D&D artifact lanes, generated NPC and
scene-description handoffs, campaign and spell-catalog references, checkpoint
storage, and chunk-map output.
Both are complete version 3 files. The fragments below illustrate individual
fields and are not alternate complete configurations.
@@ -168,12 +168,19 @@ pipelines:
npcs:
extract: dnd/npcs
normalize: dnd/npcs
scene-descriptions:
extract: dnd/scene-descriptions
normalize: dnd/scene-descriptions
- id: grounded-events
references:
npcs:
artifact:
step: identify-npcs
lane: npcs
scene_descriptions:
artifact:
step: identify-npcs
lane: scene-descriptions
artifacts:
spells:
extract: dnd/spells
@@ -374,7 +381,8 @@ production validators do not call the LLM and must not set `llm_profile`.
| chunk | `dnd/scenes` | Uses an LLM to split transcript source units into D&D scenes. |
| extract | `dnd/spells` | Extracts typed D&D spell-list artifacts. |
| extract | `dnd/npcs` | Extracts typed D&D NPC-list artifacts. |
| extract | `dnd/combat-turns` | Extracts typed D&D combat-turn-list artifacts. |
| extract | `dnd/combat-turns` | Extracts typed D&D combat-turn-list artifacts only for exact combat scene matches. |
| extract | `dnd/item-events` | Extracts typed, source-grounded D&D item-event-list artifacts. |
| extract | `dnd/npc-interactions` | Extracts typed D&D NPC-interaction-list artifacts from a supplied NPC registry. |
| extract | `dnd/scene-descriptions` | Extracts one typed D&D scene description for each accepted chunk. |
| merge | `appendorder` | Combines typed artifacts in chunk order. |
@@ -382,6 +390,7 @@ production validators do not call the LLM and must not set `llm_profile`.
| normalize | `dnd/spells` | Deterministically canonicalizes and de-duplicates typed D&D spell-list artifacts. |
| normalize | `dnd/npcs` | Deterministically consolidates typed D&D NPC-list artifacts by canonical name and unions exact evidence. |
| normalize | `dnd/combat-turns` | Deterministically canonicalizes, orders, and de-duplicates typed D&D combat-turn artifacts. |
| normalize | `dnd/item-events` | Deterministically trims, source-orders, and removes only exact D&D item-event duplicates. |
| normalize | `dnd/npc-interactions` | Canonicalizes registry NPC names, orders interaction occurrences, and removes only exact duplicates. |
| normalize | `dnd/scene-descriptions` | Validates, orders, trims, and de-duplicates typed D&D scene-description artifacts. |
| output | `json` | Produces JSON output files for normalized `application/json` lanes and can opt in to an accepted chunk map. |
@@ -406,6 +415,10 @@ production validators do not call the LLM and must not set `llm_profile`.
| `extract/dnd/combat-turns/source_refs` | deterministic | Rejects missing or invalid D&D combat-turn source references. |
| `extract/dnd/combat-turns/source_relatedness` | deterministic | Emits warnings when an actor is not found near cited source text. |
| `normalize/dnd/combat-turns/invariants` | deterministic | Rejects normalized combat-turn identity, evidence-order, and chronology violations. |
| `extract/dnd/item-events/shape` | deterministic | Rejects malformed D&D item-event-list artifacts and invalid category, holder, or quantity combinations. |
| `extract/dnd/item-events/source_refs` | deterministic | Rejects missing, invalid, or extract-chunk-external D&D item-event source references. |
| `extract/dnd/item-events/source_relatedness` | deterministic | Emits bounded warnings when an item-event name is not found in cited source text. |
| `normalize/dnd/item-events/invariants` | deterministic | Rejects non-normalized D&D item-event order, source references, or exact duplicates. |
| `extract/dnd/npc-interactions/shape` | deterministic | Rejects malformed D&D NPC-interaction-list artifacts. |
| `extract/dnd/npc-interactions/registry` | deterministic | Rejects interaction names absent from the supplied NPC registry. |
| `extract/dnd/npc-interactions/source_refs` | deterministic | Rejects missing, invalid, or extract-chunk-external D&D interaction source references. |
@@ -469,6 +482,26 @@ normalize:
- extract/dnd/combat-turns/source_relatedness
```
The production default chains for `dnd/item-events` are:
```yaml
extract:
validators:
- generic/valid_json
- extract/dnd/item-events/shape
- extract/dnd/item-events/source_refs
- generic/valid_json_schema
- extract/dnd/item-events/source_relatedness
normalize:
validators:
- generic/valid_json
- extract/dnd/item-events/shape
- normalize/dnd/item-events/invariants
- extract/dnd/item-events/source_refs
- generic/valid_json_schema
- extract/dnd/item-events/source_relatedness
```
The production default chains for `dnd/npc-interactions` are:
```yaml
@@ -567,6 +600,13 @@ accepts no references. The
normalized output to later spell, combat-turn, and NPC-interaction targets
through an explicit ordered step.
The `dnd/item-events` extractor accepts optional `glossary`, `party`, `players`,
and deprecated `roster` campaign slots for disambiguation only. It requires no
generated artifact and its normalizer accepts no references, so the complete
example keeps this independent lane in the first step. Current transcript
references remain the only item-event evidence. Its complete durable behavior
is defined in the [D&D item-event artifact contract](integrations/dnd-item-event-artifacts.md).
The `dnd/scene-descriptions` extractor accepts only the optional `players`,
`party`, and `glossary` campaign slots. They can disambiguate terms in the
model-facing title and summary but are never source evidence. Its normalizer
@@ -574,14 +614,23 @@ accepts no references. No NPC registry or generated artifact is required. See
the [D&D scene-description artifact contract](integrations/dnd-scene-description-artifacts.md)
and the copyable [complete D&D configuration](../examples/dnd-complete.config.yml).
The `dnd/combat-turns` extractor declares the optional campaign slots and the
structured `npcs` slot. Campaign references guide only the LLM extraction
stage. The deterministic normalizer declares only `npcs`, whose operation-time
registry supports the same actor canonicalization. Each `npcs` slot
accepts exactly one UTF-8 `application/json` artifact no larger than 1 MiB. The
registry's source ranges remain provenance for the reference and never become
combat evidence. An ordered step binding fans the same generated NPC artifact
out to extraction and normalization:
The `dnd/combat-turns` extractor declares the optional campaign slots, the
optional structured `npcs` slot, and the required `scene_descriptions` slot.
Campaign references guide only the LLM extraction stage. The deterministic
normalizer declares only `npcs`, whose operation-time registry supports the
same actor canonicalization. Each `npcs` slot accepts exactly one UTF-8
`application/json` artifact no larger than 1 MiB. The registry's source ranges
remain provenance for the reference and never become combat evidence.
`scene_descriptions` accepts exactly one approved
`dnd/scene-description-list` artifact with media type `application/json`, no
larger than 1 MiB (1048576 bytes). Pipeline resolution rejects a combat
extractor whose required slot is unbound. The scene artifact is eligibility
control context only; it is neither prompt input nor combat evidence, and the
combat normalizer does not accept it.
An ordered step binds the generated NPC artifact to extraction and normalization
and the generated scene-description artifact to combat extraction:
```yaml
references:
@@ -589,14 +638,22 @@ references:
artifact:
step: identify-npcs
lane: npcs
scene_descriptions:
artifact:
step: identify-npcs
lane: scene-descriptions
```
When bound, the combat extractor and normalizer receive the generated registry
at operation time. Framework provenance and checkpoint dependencies contain its
kind, schema identity, media type, canonical digest, size, and bounded producer
identity; names, content, and paths are not recorded there. When
absent, the combat prompt receives the exact empty registry value
`{"npcs":[]}` with its projection digest and no registry provenance.
When `npcs` is bound, the combat extractor and normalizer receive the generated
NPC registry at operation time. Framework provenance and checkpoint dependencies
contain its kind, schema identity, media type, canonical digest, size, and
bounded producer identity; names, content, and paths are not recorded there.
When absent, the combat prompt receives the exact empty NPC registry value
`{"npcs":[]}` with its projection digest and no registry provenance. The
generated `scene_descriptions` artifact is supplied only to combat extraction.
The complete example uses the same explicit scene binding; an external approved
scene-description artifact may be used instead when the workflow crosses a
process or session boundary.
Both `dnd/npc-interactions` stages require the structured `npcs` slot. Bind one
accepted normalized NPC artifact from an earlier ordered step; the interaction

View File

@@ -73,15 +73,29 @@ The standalone extractor uses these identities:
| Private response-schema ID | `notarius.dnd.combat_turns.llm` |
| Default profile | `gemini-2-flash` |
It requires `chunks` and `source.transcript`, accepts no options, and makes one
structured completion for each supplied chunk. The prompt receives the
chunk-scoped transcript plus the existing `players`, `party`, and `glossary`
inputs, and optionally the deprecated `roster` reference through the shared
party mapping. The optional `npcs` reference is an approved normalized NPC
artifact used only for identity grounding; it never supplies combat evidence.
An external file is validated during preparation. In an ordered pipeline, the
same slot may receive the producer's canonical generated artifact at the step
handoff.
It requires `chunks` and `source.transcript`, accepts no options, and requires
one `scene_descriptions` reference. That reference must be exactly one approved
`dnd/scene-description-list` artifact with media type `application/json` and a
maximum size of 1 MiB (1048576 bytes). It may be an external file validated
during preparation or a canonical generated artifact supplied at an ordered
step handoff. An unbound slot is a configuration error.
The scene artifact controls eligibility, not evidence. The extractor calls the
LLM only when exactly one record has the current chunk's ID, source ID, start
unit ID, and end unit ID, and that record has `kind: combat`. An exact
`narrative`, `recap`, or `meta` record returns an accepted empty
`combat_turns` array without an LLM call, warning, or retry attempt. A missing
or mismatched exact record returns the same accepted empty result without an
LLM call or retry and emits one content-safe
`scene_classification_unavailable` warning. The scene artifact, its title and
summary, and its source references are never copied into combat turns.
For an eligible combat chunk, the prompt receives the chunk-scoped transcript
plus the existing `players`, `party`, and `glossary` inputs, and optionally the
deprecated `roster` reference through the shared party mapping. The optional
`npcs` reference is an approved normalized NPC artifact used only for identity
grounding; it never supplies combat evidence. `scene_descriptions` is never a
combat prompt input.
The private response envelope has the same turn fields and JSON types as the
durable shape except that source references contain only `start_unit_id`
@@ -151,13 +165,16 @@ merge validator chain.
## Production manifest and references
The selectable lane uses extractor and normalizer key `dnd/combat-turns`,
`appendorder` for the typed merger, and the durable codec above. A bound `npcs`
reference contributes raw-file provenance to the run manifest. A generated
binding contributes artifact kind, schema identity, media type, canonical
digest, size, and bounded producer provenance. Consumer metadata and checkpoint
fingerprints contain no registry names, content, paths, or NPC source ranges.
The component-local registry fingerprint covers only the names projected to the
consumer, while manifest provenance retains the full artifact digest. The
normalized lane is emitted as `lanes/<lane-id>.json` by the JSON
output module, and warnings and rejection summaries remain in their shared
companion files.
`appendorder` for the typed merger, and the durable codec above. Bound external
references contribute raw-file provenance to the run manifest. Generated
bindings contribute artifact kind, schema identity, media type, canonical
digest, size, and bounded producer provenance. This includes the generated
scene-description artifact for the combat extractor. Consumer metadata and
checkpoint fingerprints contain no reference names, content, paths, source
ranges, scene titles, or scene summaries. For an external scene artifact,
component metadata records the gate policy plus a semantic eligibility digest
and record count; generated scene identity remains in framework handoff
provenance and dependencies. The eligibility digest changes with scene ID,
exact source range, or kind, but not with title or summary. The normalized lane
is emitted as `lanes/<lane-id>.json` by the JSON output module, and warnings
and rejection summaries remain in their shared companion files.

View File

@@ -0,0 +1,133 @@
# D&D Item-Event Artifact Contract
This document defines the durable D&D item-event artifact and its production
boundaries. It records source-grounded discoveries and possession changes; it
does not maintain an inventory or ledger.
## Artifact Identity
| Property | Value |
| --- | --- |
| Extractor key | `dnd/item-events` |
| Extract capability | `dnd.item_events` |
| Artifact kind | `dnd/item-event-list` |
| Schema ID | `notarius.dnd.item_events` |
| Schema name | `notarius_dnd_item_events_v1` |
| Schema version | `v1` |
| Media type | `application/json` |
| Normalizer key | `dnd/item-events` |
The payload is one strict JSON object containing a required `events` array,
which may be empty. Objects reject unknown fields.
## Event Shape And Categories
Every event has `name`, `kind`, and a non-empty `source_refs` array. `quantity`
is optional and, when present, is a positive integer. `from` and `to` are
optional display values whose presence depends on `kind`.
| Field | Rule |
| --- | --- |
| `name` | Non-empty, transcript-supported display value. |
| `kind` | One of the categories below. |
| `quantity` | Optional positive integer; omitted when the source does not establish a count. |
| `from` | Holder that loses possession when the category permits it. |
| `to` | Holder that gains possession when the category permits it. |
| `source_refs` | One or more current-source references. |
| Kind | Meaning and holder rule |
| --- | --- |
| `discovered` | The party learns of or encounters an item without established possession; neither holder is present. |
| `acquired` | A party member or `party` gains possession; `to` is required and `from` is absent. |
| `lost` | A party member or `party` ceases to possess an item without consuming it; `from` is required and `to` is absent. |
| `consumed` | Use depletes, expends, or destroys an item; `from` is required and `to` is absent. |
| `transferred` | Possession moves between distinct party members; both holders are required and neither may be `party`. |
`party` is the reserved display holder for collective party possession when an
individual holder is not established. Its comparison is case- and
Unicode-insensitive. Transfers require distinct normalized holder values.
Giving an item to an NPC, spending currency, selling an item, or another move
outside party possession is `lost`, not `transferred` or `consumed`. Monetary
spending, purchases, and payments are always `lost`. Currency is `consumed` only
when the source explicitly describes its physical destruction or expenditure as
a non-payment component. Ordinary non-depleting use is not an event.
Currency is represented as an ordinary event name plus an explicit quantity
when the transcript supplies one. Each denomination remains separate. The
artifact never converts denominations, infers a missing count, calculates a
balance, or sums nearby events.
## Source Evidence And Normalization
Each source reference contains `source_id`, `start_unit_id`, and `end_unit_id`.
It must identify an ordered range in the current source document. During
extraction, every cited range must also be wholly contained in the current
accepted chunk. Campaign references may disambiguate names, but never provide
event evidence.
The deterministic normalizer trims only display-edge whitespace in names and
holders, canonicalizes source-reference order and exact duplicate references,
then orders events by earliest valid source position and stable tie-breakers
over name, kind, holders, quantity, and complete evidence. It removes only
events with the same normalized fields and complete valid evidence sequence.
Invalid evidence is never collapsed. It does not rename, singularize, resolve
aliases, infer holders or quantities, or merge nearby events.
The default extraction validation chain is JSON syntax, item-event shape,
source references, durable JSON Schema, then advisory source relatedness. The
normalization chain adds normalized invariants after shape validation and before
source references and schema validation. Relatedness warnings are advisory so
contextual names and currency notation do not reject otherwise valid evidence.
## Representative JSON
```json
{
"events": [
{
"name": "Hidden Cache",
"kind": "discovered",
"source_refs": [{"source_id": "session-7", "start_unit_id": 1, "end_unit_id": 1}]
},
{
"name": "Silver Pieces",
"kind": "acquired",
"quantity": 20,
"to": "party",
"source_refs": [{"source_id": "session-7", "start_unit_id": 2, "end_unit_id": 2}]
},
{
"name": "Torch",
"kind": "lost",
"from": "party",
"source_refs": [{"source_id": "session-7", "start_unit_id": 3, "end_unit_id": 3}]
},
{
"name": "Healing Potion",
"kind": "consumed",
"from": "Aria",
"source_refs": [{"source_id": "session-7", "start_unit_id": 4, "end_unit_id": 4}]
},
{
"name": "Moonblade",
"kind": "transferred",
"from": "Aria",
"to": "Borin",
"source_refs": [{"source_id": "session-7", "start_unit_id": 5, "end_unit_id": 5}]
}
]
}
```
## Production Boundary
The extractor requires `chunks` and `source.transcript`, and accepts optional
`glossary`, `party`, `players`, and deprecated `roster` campaign references for
disambiguation only. It has no generated NPC, scene-description, or item-registry
dependency. The append-order merger preserves chunk order, and the normalizer
has no reference slots.
The normalized lane is emitted as `lanes/<lane-id>.json` by the JSON output
module. See [Configuration](../config.md#implemented-production-modules) for
the selectable keys and default chains, and the
[JSON output contract](json-output.md) for bundle paths.

View File

@@ -109,12 +109,19 @@ steps:
npcs:
extract: dnd/npcs
normalize: dnd/npcs
scene-descriptions:
extract: dnd/scene-descriptions
normalize: dnd/scene-descriptions
- id: grounded-events
references:
npcs:
artifact:
step: identify-npcs
lane: npcs
scene_descriptions:
artifact:
step: identify-npcs
lane: scene-descriptions
artifacts:
spells:
extract: dnd/spells
@@ -124,13 +131,16 @@ steps:
normalize: dnd/combat-turns
```
The framework hands only an accepted normalized artifact across the barrier. It
validates the canonical bytes against each consumer slot and clones the
operation-time reference for the spell and combat consumers. Generated
The framework hands only accepted normalized artifacts across the barrier. It
validates the canonical bytes against each consumer slot and clones the NPC
operation-time reference for spell and combat consumers. The accompanying
scene-description reference is required by the combat extractor for eligibility
only; its consumer contract is defined in the
[D&D combat-turn artifact contract](dnd-combat-turn-artifacts.md). Generated
provenance records the artifact kind, schema identity, media type, canonical
digest, size, and producer step/lane/module, but not names, source
ranges, or payload bytes. External normalized files remain supported as
explicit references and retain their file provenance.
digest, size, and producer step/lane/module, but not names, source ranges, or
payload bytes. External normalized files remain supported as explicit references
and retain their file provenance.
NPC source references are registry provenance and are never accepted as spell
or combat evidence. Current transcript units remain the only event evidence.

View File

@@ -118,3 +118,13 @@ Use `dnd/scenes` to form accepted scene chunks, then select
[complete D&D configuration](../../examples/dnd-complete.config.yml) includes
this lane alongside the other D&D artifacts. Selectable keys and default
validator chains are defined in [Configuration](../config.md).
## Downstream combat use
The combat-turn extractor consumes this approved artifact as required
eligibility control context through an explicit ordered reference binding. It
uses only an exact chunk ID and source-range match with `kind: combat` to permit
combat extraction; titles, summaries, and scene references do not become combat
prompt material or evidence. The complete downstream behavior, including empty
results and warnings for unavailable coverage, is defined in the
[D&D combat-turn artifact contract](dnd-combat-turn-artifacts.md).

View File

@@ -39,7 +39,7 @@ without exposing Scriptorium types through stage contracts.
7. injecting that one shared client into complete pipeline preparation before
the source file is read or the runner is invoked.
The D&D scene chunker and spell, NPC, combat-turn, NPC-interaction, and
The D&D scene chunker and spell, NPC, combat-turn, item-event, NPC-interaction, and
scene-description extractors retain this
injected client and use it for every structured completion. Operation requests
do not carry an LLM client.
@@ -99,7 +99,7 @@ D&D prompt content. `internal/framework/promptfs` provides the domain-neutral
filesystem composition helper used to combine module-owned files with shared
domain prompt fragments.
The D&D scene chunker and spell, NPC, combat-turn, NPC-interaction, and
The D&D scene chunker and spell, NPC, combat-turn, item-event, NPC-interaction, and
scene-description extractors each declare an
ordered prompt asset manifest. The manifest lists the package-owned YAML and
Markdown files, then the exact shared fragments rendered by that prompt; the
@@ -134,12 +134,13 @@ prompt places its final boundary on its lane-specific instructions, immediately
before the transcript. The transcript does not carry cache control because no
reusable content follows it.
Accordingly, the common prefix of the spell, NPC, combat, and interaction
Accordingly, the common prefix of the spell, NPC, combat, item-event, and interaction
extraction prompts is system,
extraction evidence, identity, and campaign references. The NPC prompt then
renders task, instructions, and transcript. Spell renders the NPC registry,
catalog, task, instructions, and transcript. Combat renders the NPC registry,
task, instructions, and transcript. NPC interaction renders the names-only NPC
task, instructions, and transcript. Item-event renders task, instructions, and
transcript without a generated-artifact input. NPC interaction renders the names-only NPC
registry, task, instructions, and transcript. The
scene chunker is not an extraction lane: it retains its separate system,
transcript, campaign-reference, task, and instruction order and marks its
@@ -166,7 +167,7 @@ return defensive copies, and expose a diagnostics map that omits schema bytes.
The small framework registry contains only generic test schemas; production
schemas remain package-owned.
The spell, NPC, combat, NPC-interaction, and scene-description extractors'
The spell, NPC, combat, item-event, NPC-interaction, and scene-description extractors'
package-owned prompts declare their
structured JSON inputs and private response schemas. Each private response
schema remains separate from its durable artifact codec schema; this work does

View File

@@ -17,16 +17,16 @@ validator registry. Package-family registrars compose those leaf registrations
into the production catalog and own family-level policy such as default
validator chains and prompt asset collection.
Production input, chunk, output, and D&D spell-, NPC-, combat-, interaction-, and scene-description-extract packages
Production input, chunk, output, and D&D spell-, NPC-, combat-, item-event-, interaction-, and scene-description-extract packages
register strict option decoders and run-local builders. Preparation decodes their options into
implementation-owned values and injects dependencies plus the materialized
reference set for the selected target. Each builder receives an isolated clone
of that set; input and output builders receive no references. The spell, NPC,
combat, interaction, and scene-description extractors are typed over the canonical D&D model. D&D validators, merge,
combat, item-event, interaction, and scene-description extractors are typed over the canonical D&D model. D&D validators, merge,
and normalize use typed variants; JSON representation validators use serialized
requests; and unconditional validators expose separate chunk and typed
variants. The D&D production registrar registers the canonical typed spell,
NPC, combat, interaction, and scene-description implementations, including their kind-specific merge and
NPC, combat, item-event, interaction, and scene-description implementations, including their kind-specific merge and
normalize behavior.
For D&D artifact defaults, generic JSON syntax validation runs first. Rejecting
@@ -56,7 +56,7 @@ declarations, prompt-input assembly, and source-unit/citation helpers belong in
`internal/modules/dnd/shared`, which owns operation-scoped indexed
source-reference validation, citation traversal, ordering and canonicalization,
plus bounded D&D diagnostics. The
D&D scene chunker and spell, NPC, combat-turn, NPC-interaction, and scene-description extractors use ordered
D&D scene chunker and spell, NPC, combat-turn, item-event, NPC-interaction, and scene-description extractors use ordered
package-local prompt manifests for both rendering and prompt fingerprinting, so
only the shared fragments each prompt actually renders participate in either
operation. Extraction prompts place stable shared and lane-specific context
@@ -67,7 +67,7 @@ contracts expose only Notarius structured-completion types, not Scriptorium
public types.
The shared `PrepareChunkExtraction` helper owns common extraction preflight and
transcript material preparation for the spell, NPC, combat-turn,
transcript material preparation for the spell, NPC, combat-turn, item-event,
NPC-interaction, and scene-description extractors. It validates common request
state, clones supplied source metadata, falls back to the materialized chunk
when content is absent, checks that content remains chunk-identical, and fills
@@ -123,11 +123,38 @@ shared `internal/modules/dnd/codec/candidatejson` package supplies strict typed
candidate JSON mechanics; each artifact codec retains its own durable schema
and approved-value policy.
### `internal/modules/dnd/scenedescriptions/registry`
The scene-description registry owns the required `scene_descriptions` control
reference used by combat extraction. It decodes exactly one approved scene-list
artifact through the scene-description codec and retains only scene ID, exact
source reference, and kind. Titles, summaries, original bytes, paths, and
prompt material do not cross this domain boundary.
An external reference is validated during preparation; an unbound seed is
permitted only while a configured generated reference awaits the ordered
handoff. At operation time, a generated artifact overrides the seed and is
resolved into an immutable view safe for concurrent extract jobs. Matching is
strictly exact by chunk ID, source ID, start unit ID, and end unit ID, producing
an exact, missing, or mismatched result. Only an exact result exposes kind.
The registry's semantic eligibility digest is derived from a sorted projection
of ID, exact range, and kind. It ignores titles, summaries, and input order;
the unbound view has a stable empty projection digest. Combat extractor
metadata and checkpoint identity use this semantic boundary for external
references, while generated artifact identity and dependencies remain owned by
the framework handoff.
The `internal/modules/dnd/codec/combatturns` package owns the durable
`dnd/combat-turn-list` schema and candidate versus approved JSON boundary. It
is registered by the production D&D family registrar for the selectable combat
lane.
The `internal/modules/dnd/codec/itemevents` package owns the durable
`dnd/item-event-list` schema and candidate versus approved JSON boundary. It is
registered by the production D&D family registrar. Its external contract is
defined in the [D&D item-event artifact contract](../integrations/dnd-item-event-artifacts.md).
The `internal/modules/dnd/codec/npcinteractions` package owns the durable
`dnd/npc-interaction-list` schema and candidate versus approved JSON boundary.
It is registered by the production D&D family registrar for the selectable
@@ -293,18 +320,31 @@ checks. The durable contract is defined in the
### `internal/modules/dnd/extract/combatturns`
The combat extractor prepares one structured request per supplied chunk using
the shared extraction-evidence, identity, campaign-reference, NPC-grounding,
and transcript prompt inputs. It
maps the private response to `dnd.CombatTurnList`, assigns the current source
identity, removes exact duplicate source ranges, and orders turns by valid
source-document position while preserving malformed candidate fields for
deterministic validators. Its package-owned private response schema enforces
only the structural JSON envelope; semantic artifact constraints remain with
the validator chain. Its prepared metadata and checkpoint fingerprints contain
only prompt/schema/mapping identities plus an optional NPC registry digest.
The prompt follows the shared D&D extraction ordering and cache policy
documented in
The combat extractor requires the `scene_descriptions` reference and resolves
it through the immutable scene-description registry before it resolves NPC
grounding or constructs prompt inputs. It calls the LLM only for an exact
current-chunk match whose kind is `combat`. Exact `narrative`, `recap`, and
`meta` matches return an accepted empty `dnd.CombatTurnList`; missing or
mismatched coverage returns the same result with one bounded unavailable-
classification warning. These deterministic results do not consume retry
attempts. Scene descriptions are control context only and are not passed to the
combat prompt or copied into combat evidence.
For eligible chunks, the extractor prepares one structured request using the
shared extraction-evidence, identity, campaign-reference, NPC-grounding, and
transcript prompt inputs. It maps the private response to
`dnd.CombatTurnList`, assigns the current source identity, removes exact
duplicate source ranges, and orders turns by valid source-document position
while preserving malformed candidate fields for deterministic validators. Its
package-owned private response schema enforces only the structural JSON
envelope; semantic artifact constraints remain with the validator chain.
Prepared metadata and checkpoint fingerprints include prompt, response-schema,
mapping, and scene-gate identities. An external scene reference additionally
reports its semantic eligibility digest and count; generated identity remains
framework handoff provenance and dependency state. Neither surface retains
scene prose or payload bytes. The prompt follows the shared D&D extraction
ordering and cache policy documented in
[LLM Runtime](llm.md#dd-extraction-prompt-ordering-and-cache-boundaries). The
package exposes typed registration and is included in the production D&D
registrar with the default combat extraction chain.
@@ -313,6 +353,23 @@ The combat normalizer accepts only the optional structured NPC registry.
Campaign references remain extractor-only LLM context and are not materialized
for deterministic normalization.
### `internal/modules/dnd/extract/itemevents`
The item-event extractor prepares one structured request from the accepted
chunk and optional campaign references, then maps private records to
`dnd.ItemEventList` with the current source identity. It declares only optional
`glossary`, `party`, `players`, and deprecated `roster` reference slots; these
can disambiguate names but never supply evidence. It has no NPC,
scene-description, or item-registry dependency.
The private response schema owns structural transport validation. The extractor
preserves candidate category, holder, quantity, and source-range values for the
deterministic validators, removes exact duplicate ranges, and source-orders
events. The source-reference validator requires citations to fit the current
accepted chunk. Prompt, response-schema, and mapping identities participate in
checkpoint identity. The durable schema is owned separately by
`internal/modules/dnd/codec/itemevents`.
### `internal/modules/dnd/extract/npcinteractions`
The NPC interaction extractor requires the structured `npcs` registry slot. It
@@ -349,7 +406,7 @@ normalization warnings.
### `internal/modules/generic/merge/appendorder`
The merger passes typed values to an injected combine function in framework
source-chunk order. The D&D registrar specializes it for all five artifact
source-chunk order. The D&D registrar specializes it for all six artifact
lists; each append merger preserves collection presence and order while giving
the result independently owned nested source-reference slices.
@@ -397,6 +454,16 @@ digest/count metadata; generated identity is retained in framework provenance
and dependency fingerprints. The normalizer is included in the production D&D
registrar with the default combat normalization chain.
### `internal/modules/dnd/normalize/itemevents`
The item-event normalizer accepts no options or references and makes no LLM
calls. It trims display-edge whitespace in names and holders, canonicalizes
source references, source-orders events, and collapses only exact duplicates
with complete valid evidence. It does not create a ledger, calculate balances,
resolve aliases, infer quantities or holders, or reconcile nearby events. Its
policy fingerprint and bounded warnings identify deterministic normalization;
the matching invariant validator checks the resulting order and duplicate rule.
### `internal/modules/dnd/normalize/npcinteractions`
The interaction normalizer requires the same immutable NPC registry. It
@@ -506,6 +573,21 @@ expose local policy fingerprints. In the registered defaults, JSON syntax runs
first; combat shape, normalized invariants when applicable, and source-reference
validation precede JSON Schema validation; warning-only relatedness runs last.
## D&D Item-Event Validators
Item-event shape validation owns the required list, non-empty name, supported
category, category-and-holder combination, positive optional quantity, and
non-empty source-reference collection. Source-reference validation defers
malformed shapes, checks current-source identity and ordered ranges, and during
extraction requires every citation to fit the accepted chunk. Relatedness is
advisory and warning-only: it checks the event name against cited transcript
text while deferring malformed candidates and invalid ranges to their blocking
owners. The normalized-invariants validator owns display normalization,
canonical source-reference order, chronology, and exact duplicate identity.
All four validators are deterministic and expose policy fingerprints. The
registered chains run syntax and blocking checks before durable JSON Schema;
relatedness remains last.
## D&D NPC Interaction Validators
Interaction shape validation owns the required list, registry name, supported
@@ -538,7 +620,7 @@ complete framework registry set and one LLM asset registry. It invokes
`internal/modules/seriatim/register`, and `internal/modules/dnd/register` in
that order, then exposes the matching catalog for resolution. The generic and
Seriatim registrars own their production leaf registrations. The D&D registrar
owns D&D leaf registrations, typed spell, NPC, combat, interaction, and scene-description default-validator
owns D&D leaf registrations, typed spell, NPC, combat, item-event, interaction, and scene-description default-validator
chains, typed append-order specializations, and D&D prompt/schema asset
collection. Its registration helpers group module, validator, prompt-asset, and
chain composition while retaining artifact-specific merge and clone behavior in

View File

@@ -69,9 +69,9 @@ constructs input, chunk, validators, ordered lanes, and output before source
parsing. Production modules use strict construction-time option decoding, and
LLM-backed modules retain the injected shared client. The D&D family registers
the canonical `dnd/spell-list`, `dnd/npc-list`, `dnd/combat-turn-list`,
`dnd/npc-interaction-list`, and `dnd/scene-description-list` codecs, typed
spell, NPC, combat, interaction, and scene-description extractors and
normalizers, validators,
`dnd/item-event-list`, `dnd/npc-interaction-list`, and
`dnd/scene-description-list` codecs, typed spell, NPC, combat, item-event,
interaction, and scene-description extractors and normalizers, validators,
plus kind-specific generic merge strategies; generic JSON validators use the
serialized-validation contract. The runner executes lanes through
private exact-type-checked closures, coordinates extract results independently
@@ -90,25 +90,30 @@ Configuration. The implemented module packages are:
| `internal/modules/seriatim/input/transcript` | Parses the supported Seriatim transcript format into the generic source model. |
| `internal/modules/generic/chunk/units` | Splits ordered source units by unit count and overlap. |
| `internal/modules/dnd/chunk/scenes` | Produces contiguous D&D scene chunks from structured model output. |
| `internal/modules/dnd` | Owns the canonical D&D spell-list, spell-cast, NPC-list, NPC, combat-turn-list, combat-turn, NPC-interaction-list, and scene-description-list artifact types. |
| `internal/modules/dnd` | Owns the canonical D&D spell-list, spell-cast, NPC-list, NPC, combat-turn-list, combat-turn, item-event-list, item-event, NPC-interaction-list, and scene-description-list artifact types. |
| `internal/modules/dnd/codec/spells` | Strictly decodes and stably encodes the durable D&D spell-list representation. |
| `internal/modules/dnd/codec/npcs` | Strictly decodes and stably encodes the durable D&D NPC-list representation. |
| `internal/modules/dnd/codec/combatturns` | Strictly decodes and stably encodes the durable D&D combat-turn-list representation. |
| `internal/modules/dnd/codec/itemevents` | Strictly decodes and stably encodes the durable D&D item-event-list representation. |
| `internal/modules/dnd/codec/npcinteractions` | Strictly decodes and stably encodes the durable D&D NPC-interaction-list representation. |
| `internal/modules/dnd/codec/scenedescriptions` | Strictly decodes and stably encodes the durable D&D scene-description-list representation. |
| `internal/modules/dnd/extract/spells` | Maps private structured model output to canonical source-grounded D&D spell lists. |
| `internal/modules/dnd/extract/npcs` | Maps private structured model output to canonical source-grounded D&D NPC lists. |
| `internal/modules/dnd/extract/combatturns` | Maps private structured model output to source-grounded D&D combat-turn candidates and preserves chronology and invalid candidate values for validators. |
| `internal/modules/dnd/extract/combatturns` | Uses exact scene eligibility to select combat chunks, then maps private structured model output to source-grounded D&D combat-turn candidates. |
| `internal/modules/dnd/extract/itemevents` | Maps private structured model output to source-grounded D&D item-event candidates. |
| `internal/modules/dnd/extract/npcinteractions` | Maps private structured model output to current-source NPC interaction candidates grounded by a required registry. |
| `internal/modules/dnd/extract/scenedescriptions` | Maps one private scene description to the current accepted chunk's ID and exact range. |
| `internal/modules/dnd/npcinteractions` | Owns interaction occurrence ordering, valid-evidence checks, and exact interaction identity shared by normalization and invariant validation. |
| `internal/modules/dnd/normalize/combatturns` | Canonicalizes and orders merged combat turns, applies exact NPC identity matches, and collapses only exact valid-evidence duplicates. |
| `internal/modules/dnd/normalize/itemevents` | Trims, source-orders, and removes only exact valid-evidence item-event duplicates. |
| `internal/modules/dnd/normalize/npcinteractions` | Canonicalizes required-registry names, orders interaction occurrences, and collapses only exact valid-evidence duplicates. |
| `internal/modules/dnd/normalize/scenedescriptions` | Trims, source-orders, and removes only exactly identical scene descriptions while rejecting ID and range conflicts. |
| `internal/modules/dnd/validate/combatturns` | Provides deterministic shape, source-reference, source-relatedness, and normalized-invariant validation for the production combat chains. |
| `internal/modules/dnd/validate/itemevents` | Provides deterministic shape, source-reference, source-relatedness, and normalized-invariant validation for item-event chains. |
| `internal/modules/dnd/validate/npcinteractions` | Provides deterministic shape, registry, source-reference, source-relatedness, and normalized-invariant validation for interaction chains. |
| `internal/modules/dnd/validate/scenedescriptions` | Provides deterministic shape, exact extraction attachment, source-relatedness, and normalized-invariant validation for scene-description chains. |
| `internal/modules/dnd/npcs/registry` | Resolves validated normalized NPC references into immutable grounding data and exact identity lookup. |
| `internal/modules/dnd/scenedescriptions/registry` | Resolves approved scene descriptions into immutable exact-match combat eligibility data without retaining scene prose. |
| `internal/modules/dnd/npcs/identity` | Owns Unicode-aware NPC identity, ID derivation, and registry collision validation. |
| `internal/modules/dnd/spells/catalog` | Embeds and validates the versioned D&D 5e 2014 SRD catalog, composes optional overlays, and provides immutable effective lookup. |
| `internal/modules/generic/merge/appendorder` | Combines accepted extraction results in chunk order. |

View File

@@ -50,10 +50,13 @@ needed. This is data deletion, not cache cleanup.
The maintained [complete D&D configuration](../examples/dnd-complete.config.yml)
contains one pipeline with two ordered steps. The first step extracts and
normalizes NPCs and scene descriptions. Only after those lanes reach accepted
terminal results does the second step begin; its generated NPC reference is
normalizes independent item events, NPCs, and scene descriptions. Only after
the NPC and scene-description lanes reach accepted terminal results does the
second step begin; its generated NPC reference is
supplied in memory to spell, combat-turn, and NPC-interaction extraction and
the applicable normalizers.
the applicable normalizers, while its generated scene-description reference is
supplied to combat-turn extraction. The item-event lane has no generated
reference dependency and retains only current-transcript evidence.
```sh
go run ./cmd/notarius run dnd-session \
@@ -70,6 +73,14 @@ exposed through a filesystem path. The same producer artifact may fan out to
compatible consumers, while a missing or rejected producer prevents the later
step from starting.
Combat-turn extraction calls its LLM only when a scene record exactly matches
the accepted chunk and has kind `combat`. Exact `narrative`, `recap`, and
`meta` records produce an accepted empty combat-turn result without an LLM call.
Missing or mismatched scene coverage also produces an accepted empty result and
a bounded warning. Scene descriptions remain control context rather than combat
evidence or prompt material; the complete behavior is defined by the
[D&D combat-turn artifact contract](integrations/dnd-combat-turn-artifacts.md).
Standalone module configurations continue to support external NPC files when a
workflow intentionally crosses a process or session boundary. Those files are
validated against the consumer slot and must be protected as sensitive

View File

@@ -18,10 +18,6 @@ not as committed release dates.
### Use D&D Scene Chunking
- Allow the combat extractor to no-op for chunks classified as non-combat only
after the scene-description artifact can be supplied through an explicit
ordered dependency. Do not make generic chunk materialization depend on a D&D
classification.
- Use ordered pipeline steps whenever a later artifact needs an accepted
earlier artifact as context. Keep independent lanes in the same step and do
not introduce a general DAG or concurrent cross-lane reconciliation model.

View File

@@ -0,0 +1,190 @@
# Scene-Aware Combat Extraction
## Status
Implemented. This document preserves the feature contract and rationale; the
current configuration, operational, integration, and internal behavior is
maintained in their canonical documentation.
## Purpose
Use an accepted D&D scene-description artifact as the eligibility boundary for
combat-turn extraction. The combat LLM should run only for a chunk whose exact
scene record has `kind: combat`; every other chunk should be ignored by the
combat extractor. This work should connect the existing scene-description lane,
ordered generated-reference handoff, and combat-turn lane without expanding the
minimal combat-turn artifact contract or introducing D&D policy into generic
pipeline code.
## Target Workflow
The complete D&D pipeline has two ordered steps:
1. The first step extracts and normalizes NPCs and scene descriptions for the
accepted scene chunks.
2. At the step barrier, the accepted NPC and scene-description artifacts become
generated references.
3. The second step supplies the NPC artifact to its existing consumers and the
scene-description artifact to combat-turn extraction.
4. For each chunk, combat-turn extraction performs its existing LLM extraction
only for an exact `combat` classification. Every other chunk produces a
deterministic empty artifact without an LLM call.
The scene-description dependency must remain explicit in pipeline
configuration. The combat extractor must not discover another lane implicitly,
read output files directly, inspect chunker-private state, or require generic
chunk materialization to interpret D&D scene classifications.
## Scene-Description Reference Contract
The combat extractor should declare a required structured reference slot named
`scene_descriptions`. The slot accepts one approved
`dnd/scene-description-list` artifact using the existing durable scene
description schema.
The reference may be supplied as:
- a generated artifact from an earlier ordered step; or
- an external artifact through the existing reference-materialization
boundary.
External artifacts must be decoded and validated before source parsing or LLM
execution. Generated artifacts must cross the existing typed step-handoff
boundary and be validated before use. A bound artifact that is malformed,
incompatible, or internally inconsistent is an error; it must not be treated as
though the slot were unbound.
Pipeline resolution must reject combat extraction when this slot is not bound.
In an ordered same-run workflow, failure of the configured scene-description
producer to yield an accepted normalized artifact must fail the run before the
combat consumer step starts, consistent with existing required generated
reference semantics.
The prepared reference view should be immutable and safe for concurrent
extract jobs. Its metadata and checkpoint identity should be content-safe and
must not expose scene titles, summaries, paths, or source text.
## Chunk Matching And Gating Policy
A scene record authorizes combat extraction for the current chunk only when all
of the following are true:
- the scene ID exactly equals the current accepted chunk ID;
- the scene source ID exactly equals the chunk source ID;
- the scene start and end unit IDs exactly equal the chunk's inclusive source
range; and
- the approved artifact contains exactly one such record.
The normalized scene-description contract already rejects conflicting IDs and
ranges. The combat extractor must nevertheless require the exact match above at
its own decision boundary rather than relying on array position, range overlap,
title, summary, or inferred chronology.
An exact match with `kind: combat` performs the existing combat-turn LLM
extraction. An exact match with `narrative`, `recap`, or `meta` returns a typed
`dnd/combat-turn-list` containing an empty `combat_turns` array without making
an LLM call.
A valid artifact with no exact match, incomplete coverage, or a chunk identity
or range mismatch also returns the deterministic empty artifact. It must not
infer combat eligibility from an overlapping or adjacent scene. Emit a bounded,
content-safe warning for missing or mismatched coverage so operators can
distinguish an intentional non-combat classification from an unavailable exact
classification. The warning must not include scene prose or transcript text.
This policy relies on the scene contract's mixed-scene precedence: any chunk in
which combat is a substantive central activity is classified as `combat`.
Scene-aware gating must not add a second classification policy.
## Extraction, Validation, And Provenance
The deterministic empty result follows the same typed extractor and validator
boundaries as an LLM-produced empty result. It is not a rejection, does not
consume retry budget, and continues through merge and normalization normally.
The durable combat-turn schema remains unchanged.
Existing combat extraction behavior—including prompt assets, NPC grounding,
candidate mapping, validators, retries, warnings, and normalization—remains
unchanged for chunks classified as combat. The scene-description artifact is
control context only:
- it must not be added to the combat prompt;
- its title or summary must not become combat evidence;
- its source references must not be copied into combat turns; and
- it must not create, repair, or classify a combat turn.
Generated-reference provenance and dependency fingerprints should cover the
scene artifact through the existing ordered-handoff machinery. External
references should contribute their existing materialization provenance plus a
component-local semantic fingerprint sufficient to invalidate combat extract
checkpoints when a classification or chunk identity changes. Checkpoint reuse
must never preserve a skipped result after the effective scene classification
changes.
Run manifests may report bounded module metadata such as the number of approved
scene records. They must not contain scene prose or duplicate the referenced
artifact payload.
## Configuration And Documentation
The maintained complete D&D example should bind the normalized
`scene-descriptions` lane from the first step to the `scene_descriptions` slot
in the second step. The minimal example should remain unchanged.
Current-behavior documentation is maintained in these canonical locations:
- Configuration owns the new selectable reference slot and binding example.
- Operations owns the ordered scene-aware workflow.
- The combat-turn integration contract owns externally observable extraction,
empty-result, and provenance behavior.
- Internal pipeline and module documentation own preparation, handoff, matching,
and checkpoint mechanics.
- `future.md` should remove the completed scene-aware combat item.
## Quality Expectations
Tests should protect behavior and architectural boundaries rather than internal
helper shape. Coverage should demonstrate:
- exact non-combat matches produce accepted empty combat artifacts without an
LLM call;
- exact combat matches retain the existing LLM path;
- an unbound required reference is rejected during pipeline resolution;
- missing and mismatched chunk coverage produces an accepted empty combat
artifact without an LLM call and emits a bounded warning;
- malformed external and generated artifacts fail at the appropriate
preparation or handoff boundary;
- retries are neither consumed nor attempted for deterministic skips;
- scene artifact changes invalidate relevant checkpoint reuse;
- no scene prose is exposed through combat prompts, warnings, metadata, or
manifests; and
- the complete maintained configuration resolves and materializes the intended
ordered dependency.
Model-output fixtures should remain deterministic test doubles. This feature
does not require live-provider tests or assertions over probabilistic model
quality.
## Non-Goals
This scope does not:
- change the durable scene-description or combat-turn schemas;
- add scene fields to combat-turn artifacts;
- annotate generic chunks with D&D classifications;
- make the combat lane depend implicitly on the scene-description lane;
- skip spell, NPC, NPC-interaction, or scene-description extraction;
- infer combat from scene titles, summaries, overlap, or campaign references;
- introduce arbitrary DAG scheduling or concurrent cross-step execution;
- add prior-run artifact discovery or new reference syntax; or
- add an LLM-backed validator, repair pass, or generic deduplication stage.
## Completion Criteria
The scope is complete when an explicitly configured ordered D&D pipeline can
hand an approved scene-description artifact to combat extraction, exact
`combat` scene matches are the only chunks that invoke the combat LLM, every
other chunk produces a deterministic empty result, provenance and checkpoint
identity remain correct, the complete example demonstrates the workflow, and
the canonical current-behavior documentation reflects the implemented
contract.

View File

@@ -29,9 +29,15 @@ pipelines:
options:
include_chunk_map: true
steps:
# Establish session-wide reference artifacts before extracting events.
# Establish session-wide reference artifacts alongside independent item events.
- id: describe-session
artifacts:
item-events:
extract:
module: dnd/item-events
retries: 2
merge: appendorder
normalize: dnd/item-events
npcs:
extract:
module: dnd/npcs
@@ -45,13 +51,17 @@ pipelines:
merge: appendorder
normalize: dnd/scene-descriptions
- id: extract-events
# The accepted NPC artifact is supplied in memory to every compatible
# extractor and normalizer in this step.
# Accepted NPC grounding and scene-description eligibility artifacts are
# supplied in memory to their compatible consumers in this step.
references:
npcs:
artifact:
step: describe-session
lane: npcs
scene_descriptions:
artifact:
step: describe-session
lane: scene-descriptions
artifacts:
spells:
extract:

View File

@@ -48,8 +48,12 @@ func TestProductionCombatConfigurationResolvesTypedLane(t *testing.T) {
if !ok || codecSpec.Schema.ID != "notarius.dnd.combat_turns" || codecSpec.Schema.Version != "v1" {
t.Fatalf("combat codec spec = %#v, want compatible durable schema", codecSpec)
}
if !hasReferenceSlot(extractSpec.ReferenceSlots, "npcs") || !hasReferenceSlot(normalizeSpec.ReferenceSlots, "npcs") {
t.Fatalf("combat reference slots = %#v / %#v, want stage-local NPC slots", extractSpec.ReferenceSlots, normalizeSpec.ReferenceSlots)
if !hasReferenceSlot(extractSpec.ReferenceSlots, "npcs") || !hasReferenceSlot(extractSpec.ReferenceSlots, "scene_descriptions") || !hasReferenceSlot(normalizeSpec.ReferenceSlots, "npcs") {
t.Fatalf("combat reference slots = %#v / %#v, want extraction scene and NPC slots plus normalization NPC slot", extractSpec.ReferenceSlots, normalizeSpec.ReferenceSlots)
}
sceneSlot := referenceSlot(extractSpec.ReferenceSlots, "scene_descriptions")
if !sceneSlot.Required || !reflect.DeepEqual(sceneSlot.AcceptedMediaTypes, []string{"application/json"}) || !reflect.DeepEqual(sceneSlot.AcceptedArtifactKinds, []contracts.ArtifactKind{dnd.SceneDescriptionListKind}) || sceneSlot.MaxBytes != 1048576 {
t.Fatalf("scene description slot = %#v, want required approved scene artifact", sceneSlot)
}
wantExtractChain := []pipeline.ModuleBinding{
@@ -89,8 +93,19 @@ func TestProductionCombatConfigurationResolvesTypedLane(t *testing.T) {
t.Fatalf("Resolve(bound references) error = %v, want nil", err)
}
boundLane := bound.ResolvedPipeline.Steps[0].ArtifactLanes[0]
if len(boundLane.ExtractReferences.Bindings) != 1 || len(boundLane.NormalizeReferences.Bindings) != 1 || boundLane.ExtractReferences.Bindings[0].SlotName != "npcs" || boundLane.NormalizeReferences.Bindings[0].SlotName != "npcs" {
t.Fatalf("bound combat references = %#v / %#v, want one independent NPC binding per stage", boundLane.ExtractReferences, boundLane.NormalizeReferences)
if len(boundLane.ExtractReferences.Bindings) != 2 || len(boundLane.NormalizeReferences.Bindings) != 1 || !hasReferenceBinding(boundLane.ExtractReferences.Bindings, "npcs") || !hasReferenceBinding(boundLane.ExtractReferences.Bindings, "scene_descriptions") || !hasReferenceBinding(boundLane.NormalizeReferences.Bindings, "npcs") {
t.Fatalf("bound combat references = %#v / %#v, want extraction scene and NPC bindings plus normalization NPC binding", boundLane.ExtractReferences, boundLane.NormalizeReferences)
}
}
func TestProductionCombatConfigurationRequiresSceneDescriptions(t *testing.T) {
components := productionTestComponents(t)
cfg := productionCombatContractConfig()
profile := cfg.Pipelines["dnd-combat"]
profile.References = nil
cfg.Pipelines["dnd-combat"] = profile
if _, err := cfg.Resolve(config.ResolveInput{PipelineID: "dnd-combat", Catalog: catalogFromRegistries(components.registries)}); err == nil || !strings.Contains(err.Error(), "scene_descriptions") || !strings.Contains(err.Error(), "required") {
t.Fatalf("Resolve() error = %v, want required scene reference failure", err)
}
}
@@ -154,6 +169,7 @@ func productionCombatContractConfig() config.Config {
ID: "dnd-combat",
Input: pipeline.Binding("seriatim"),
Chunk: pipeline.Binding(pipeline.DefaultChunkModule),
References: map[string]pipeline.ReferenceSource{"scene_descriptions": pipeline.ExternalReference("scenes.json")},
Artifacts: map[string]pipeline.ArtifactLaneProfile{
"combat": {
Extract: pipeline.ModuleBinding{Module: combatextract.Key, Retries: 2},
@@ -172,3 +188,21 @@ func hasReferenceSlot(slots []contracts.ReferenceSlot, name string) bool {
}
return false
}
func hasReferenceBinding(bindings []pipeline.ReferenceBinding, name string) bool {
for _, binding := range bindings {
if binding.SlotName == name {
return true
}
}
return false
}
func referenceSlot(slots []contracts.ReferenceSlot, name string) contracts.ReferenceSlot {
for _, slot := range slots {
if slot.Name == name {
return slot
}
}
return contracts.ReferenceSlot{}
}

View File

@@ -34,7 +34,7 @@ func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
t.Fatalf("materialize maintained example references for %q: %v", pipelineID, err)
}
if example.name == "complete" {
if got := exampleStepLaneIDs(materialized); strings.Join(got, "|") != "describe-session:npcs,scene-descriptions|extract-events:combat-turns,npc-interactions,spells" {
if got := exampleStepLaneIDs(materialized); strings.Join(got, "|") != "describe-session:item-events,npcs,scene-descriptions|extract-events:combat-turns,npc-interactions,spells" {
t.Fatalf("complete example steps and lanes = %v, want every D&D extractor in the documented two-step composition", got)
}
spellLane := referenceContractLane(t, materialized, "spells")
@@ -42,6 +42,15 @@ func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
len(spellLane.NormalizeReferences.ReferenceSet.Slots["spell_catalog"].Items) != 1 {
t.Fatalf("complete example spell catalog reference was not materialized: %#v", spellLane)
}
itemEventLane := referenceContractLane(t, materialized, "item-events")
for _, references := range []pipeline.ResolvedReferenceTarget{itemEventLane.ExtractReferences, itemEventLane.NormalizeReferences} {
if _, found := references.ReferenceSet.Slots["npcs"]; found {
t.Fatalf("item event lane unexpectedly depends on generated NPCs: %#v", itemEventLane)
}
if _, found := references.ReferenceSet.Slots["scene_descriptions"]; found {
t.Fatalf("item event lane unexpectedly depends on generated scene descriptions: %#v", itemEventLane)
}
}
}
}
var stdout, stderr strings.Builder

View File

@@ -24,10 +24,13 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/chunk/scenes"
combatcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/combatturns"
itemeventcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/itemevents"
spellcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/spells"
combatextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
itemeventextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/itemevents"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
combatnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/combatturns"
itemeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/itemevents"
spellnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/spells"
"gitea.maximumdirect.net/eric/notarius/internal/modules/generic/normalize/noop"
)
@@ -38,9 +41,9 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
assertProductionContains(t, "inputs", registries.Inputs.RegisteredKeys(), []string{"seriatim"})
assertProductionContains(t, "chunkers", registries.Chunkers.RegisteredKeys(), []string{"dnd/scenes", "generic"})
assertProductionContains(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells", "dnd/npcs", combatextract.Key})
assertProductionContains(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells", "dnd/npcs", combatextract.Key, itemeventextract.Key})
assertProductionContains(t, "mergers", registries.Mergers.RegisteredKeys(), []string{"appendorder"})
assertProductionContains(t, "normalizers", registries.Normalizers.RegisteredKeys(), []string{"noop", spellnormalize.Key, "dnd/npcs", combatnormalize.Key})
assertProductionContains(t, "normalizers", registries.Normalizers.RegisteredKeys(), []string{"noop", spellnormalize.Key, "dnd/npcs", combatnormalize.Key, itemeventnormalize.Key})
assertProductionContains(t, "outputs", registries.Outputs.RegisteredKeys(), []string{"json"})
assertProductionContains(t, "validators", registries.Validators.RegisteredKeys(), []string{
"extract/dnd/spells/catalog",
@@ -51,16 +54,21 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
"extract/dnd/combat-turns/source_refs",
"extract/dnd/combat-turns/source_relatedness",
"normalize/dnd/combat-turns/invariants",
"extract/dnd/item-events/shape",
"extract/dnd/item-events/source_refs",
"extract/dnd/item-events/source_relatedness",
"normalize/dnd/item-events/invariants",
"generic/always_accept",
"generic/always_reject",
"generic/valid_json",
"generic/valid_json_schema",
})
assertProductionContains(t, "artifact codec kinds", registries.ArtifactCodecs.RegisteredKinds(), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind})
assertProductionContains(t, "merger variants", registries.Mergers.RegisteredArtifactKinds(pipeline.DefaultMergeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind})
assertProductionContains(t, "normalizer variants", registries.Normalizers.RegisteredArtifactKinds(pipeline.DefaultNormalizeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind})
assertProductionContains(t, "artifact codec kinds", registries.ArtifactCodecs.RegisteredKinds(), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind})
assertProductionContains(t, "merger variants", registries.Mergers.RegisteredArtifactKinds(pipeline.DefaultMergeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind})
assertProductionContains(t, "normalizer variants", registries.Normalizers.RegisteredArtifactKinds(pipeline.DefaultNormalizeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind})
assertProductionContains(t, "spell normalizer variants", registries.Normalizers.RegisteredArtifactKinds(spellnormalize.Key), []contracts.ArtifactKind{dnd.SpellListKind})
assertProductionContains(t, "combat normalizer variants", registries.Normalizers.RegisteredArtifactKinds(combatnormalize.Key), []contracts.ArtifactKind{dnd.CombatTurnListKind})
assertProductionContains(t, "item event normalizer variants", registries.Normalizers.RegisteredArtifactKinds(itemeventnormalize.Key), []contracts.ArtifactKind{dnd.ItemEventListKind})
wantChain := []pipeline.ModuleBinding{
pipeline.Binding("generic/valid_json"),
@@ -97,6 +105,27 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
if got := registries.ValidatorChains.Validators(pipeline.StageNormalize, combatnormalize.Key); !reflect.DeepEqual(got, combatNormalizeChain) {
t.Fatalf("combat normalize validator chain = %#v, want %#v", got, combatNormalizeChain)
}
itemEventExtractChain := []pipeline.ModuleBinding{
pipeline.Binding("generic/valid_json"),
pipeline.Binding("extract/dnd/item-events/shape"),
pipeline.Binding("extract/dnd/item-events/source_refs"),
pipeline.Binding("generic/valid_json_schema"),
pipeline.Binding("extract/dnd/item-events/source_relatedness"),
}
itemEventNormalizeChain := []pipeline.ModuleBinding{
pipeline.Binding("generic/valid_json"),
pipeline.Binding("extract/dnd/item-events/shape"),
pipeline.Binding("normalize/dnd/item-events/invariants"),
pipeline.Binding("extract/dnd/item-events/source_refs"),
pipeline.Binding("generic/valid_json_schema"),
pipeline.Binding("extract/dnd/item-events/source_relatedness"),
}
if got := registries.ValidatorChains.Validators(pipeline.StageExtract, itemeventextract.Key); !reflect.DeepEqual(got, itemEventExtractChain) {
t.Fatalf("item event extract validator chain = %#v, want %#v", got, itemEventExtractChain)
}
if got := registries.ValidatorChains.Validators(pipeline.StageNormalize, itemeventnormalize.Key); !reflect.DeepEqual(got, itemEventNormalizeChain) {
t.Fatalf("item event normalize validator chain = %#v, want %#v", got, itemEventNormalizeChain)
}
assetNames := productionAssetNames(t, components.assets.PromptFS)
requiredAssets := []string{
@@ -119,6 +148,14 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
"dnd.combat_turns/sharedassets/common-dnd-system.md",
"dnd.combat_turns/sharedassets/common-dnd-transcript.md",
"dnd.combat_turns/task.md",
"dnd.item_events/dnd.item_events.yaml",
"dnd.item_events/instructions.md",
"dnd.item_events/sharedassets/common-dnd-extraction-evidence.md",
"dnd.item_events/sharedassets/common-dnd-identity.md",
"dnd.item_events/sharedassets/common-dnd-references.md",
"dnd.item_events/sharedassets/common-dnd-system.md",
"dnd.item_events/sharedassets/common-dnd-transcript.md",
"dnd.item_events/task.md",
}
assertProductionContains(t, "production prompt assets", assetNames, requiredAssets)
@@ -135,6 +172,10 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
if !ok || combatCodecSpec.Kind != dnd.CombatTurnListKind || combatCodecSpec.Schema.ID != combatcodec.SchemaID {
t.Fatalf("combat codec spec = %#v, ok=%t, want typed D&D combat codec", combatCodecSpec, ok)
}
itemEventCodecSpec, ok := catalog.ArtifactCodecs.Spec(dnd.ItemEventListKind)
if !ok || itemEventCodecSpec.Kind != dnd.ItemEventListKind || itemEventCodecSpec.Schema.ID != itemeventcodec.SchemaID {
t.Fatalf("item event codec spec = %#v, ok=%t, want typed D&D item-event codec", itemEventCodecSpec, ok)
}
if got := catalog.ValidatorChains.Validators(pipeline.StageExtract, spells.Key); !reflect.DeepEqual(got, wantChain) {
t.Fatalf("catalog validator chain = %#v, want %#v", got, wantChain)
}
@@ -520,8 +561,8 @@ func TestProductionSceneRunRecordsAnnotationFreeChunkPlanAndProvenance(t *testin
if len(warnings.Warnings) != 0 {
t.Fatalf("warnings = %#v, want none", warnings.Warnings)
}
if len(fake.requestsFor(scenes.PromptID)) != 1 || len(fake.requestsFor(spells.PromptID)) != 1 {
t.Fatalf("fake prompt requests = %#v, want one scene and one spell request", fake.requestPrompts())
if len(fake.requestsFor(scenes.PromptID)) != 1 || len(fake.requestsFor(spells.PromptID)) != 1 || len(fake.requestsFor(itemeventextract.PromptID)) != 1 {
t.Fatalf("fake prompt requests = %#v, want one scene, spell, and item-event request", fake.requestPrompts())
}
}
@@ -658,6 +699,8 @@ pipelines:
artifacts:
spells:
extract: dnd/spells
item-events:
extract: dnd/item-events
`, outputRoot, filepath.Join(filepath.Dir(outputRoot), "debug"), chunkModule)
}
@@ -756,6 +799,8 @@ func (client *productionFakeLLMClient) CompleteStructured(ctx context.Context, r
} else {
content = []byte(`{"spell_casts":[{"caster":"Aria","spell":"Cure Wounds","source_refs":[{"start_unit_id":1,"end_unit_id":1}]}]}`)
}
case itemeventextract.PromptID:
content = []byte(`{"events":[{"name":"Cure Wounds","kind":"acquired","to":"party","source_refs":[{"start_segment":1,"end_segment":1}]}]}`)
default:
return contracts.StructuredCompletionResponse{}, fmt.Errorf("unexpected prompt %q", req.PromptID)
}

View File

@@ -0,0 +1,45 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.item_events",
"type": "object",
"additionalProperties": false,
"required": ["events"],
"properties": {
"events": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name", "kind", "source_refs"],
"properties": {
"name": {"type": "string", "minLength": 1},
"kind": {"type": "string", "enum": ["discovered", "acquired", "lost", "consumed", "transferred"]},
"quantity": {"type": "integer", "minimum": 1},
"from": {"type": "string", "minLength": 1},
"to": {"type": "string", "minLength": 1},
"source_refs": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"additionalProperties": false,
"required": ["source_id", "start_unit_id", "end_unit_id"],
"properties": {
"source_id": {"type": "string", "minLength": 1},
"start_unit_id": {"type": "integer", "minimum": 1},
"end_unit_id": {"type": "integer", "minimum": 1}
}
}
}
},
"allOf": [
{"if": {"properties": {"kind": {"const": "discovered"}}, "required": ["kind"]}, "then": {"not": {"anyOf": [{"required": ["from"]}, {"required": ["to"]}]}}},
{"if": {"properties": {"kind": {"const": "acquired"}}, "required": ["kind"]}, "then": {"required": ["to"], "not": {"required": ["from"]}}},
{"if": {"properties": {"kind": {"const": "lost"}}, "required": ["kind"]}, "then": {"required": ["from"], "not": {"required": ["to"]}}},
{"if": {"properties": {"kind": {"const": "consumed"}}, "required": ["kind"]}, "then": {"required": ["from"], "not": {"required": ["to"]}}},
{"if": {"properties": {"kind": {"const": "transferred"}}, "required": ["kind"]}, "then": {"required": ["from", "to"]}}
]
}
}
}
}

View File

@@ -0,0 +1,146 @@
// Package itemevents encodes durable D&D item-event artifacts.
package itemevents
import (
"embed"
"fmt"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/candidatejson"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/itemevents"
)
const (
SchemaID = "notarius.dnd.item_events"
SchemaName = "notarius_dnd_item_events_v1"
SchemaVersion = "v1"
MediaType = "application/json"
)
//go:embed assets/schemas/dnd_item_events.v1.json
var schemaAssets embed.FS
var _ contracts.ArtifactCodec[dnd.ItemEventList] = (*Codec)(nil)
type Codec struct{}
func New() *Codec { return &Codec{} }
func (c *Codec) Kind() contracts.ArtifactKind { return dnd.ItemEventListKind }
func (c *Codec) Schema() contracts.ArtifactSchema {
raw, err := schemaAssets.ReadFile("assets/schemas/dnd_item_events.v1.json")
if err != nil {
return contracts.ArtifactSchema{}
}
return contracts.ArtifactSchema{
ID: SchemaID,
Name: SchemaName,
Version: SchemaVersion,
JSONSchema: append([]byte(nil), raw...),
}
}
func (c *Codec) MediaType() string { return MediaType }
func (c *Codec) Metadata(value dnd.ItemEventList) map[string]any {
return map[string]any{"event_count": len(value.Events)}
}
func (c *Codec) Encode(value dnd.ItemEventList) ([]byte, error) {
if err := validate(value); err != nil {
return nil, fmt.Errorf("encode dnd item event list: %w", err)
}
return c.EncodeCandidate(value)
}
// EncodeCandidate provides the durable representation before semantic
// validators have approved a value.
func (c *Codec) EncodeCandidate(value dnd.ItemEventList) ([]byte, error) {
return candidatejson.EncodeCandidate("dnd item event list", cloneList(value))
}
func (c *Codec) Decode(content []byte) (dnd.ItemEventList, error) {
value, err := c.DecodeCandidate(content)
if err != nil {
return dnd.ItemEventList{}, err
}
if err := validate(value); err != nil {
return dnd.ItemEventList{}, fmt.Errorf("decode dnd item event list: %w", err)
}
return value, nil
}
// DecodeCandidate reads one strict durable JSON value before semantic
// validators have approved it.
func (c *Codec) DecodeCandidate(content []byte) (dnd.ItemEventList, error) {
value, err := candidatejson.DecodeCandidate[dnd.ItemEventList]("dnd item event list", content)
if err != nil {
return dnd.ItemEventList{}, err
}
return cloneList(value), nil
}
func validate(value dnd.ItemEventList) error {
if value.Events == nil {
return fmt.Errorf("events must be present")
}
for index, event := range value.Events {
prefix := fmt.Sprintf("events[%d]", index)
if strings.TrimSpace(event.Name) == "" {
return fmt.Errorf("%s.name must not be empty", prefix)
}
if !itemevents.SupportedKind(event.Kind) {
return fmt.Errorf("%s.kind must be supported", prefix)
}
if event.From != "" && strings.TrimSpace(event.From) == "" {
return fmt.Errorf("%s.from must not be empty when present", prefix)
}
if event.To != "" && strings.TrimSpace(event.To) == "" {
return fmt.Errorf("%s.to must not be empty when present", prefix)
}
if !itemevents.ValidHolderCombination(event.Kind, event.From, event.To) {
return fmt.Errorf("%s holders are incompatible with %q", prefix, event.Kind)
}
if event.Quantity != nil && *event.Quantity < 1 {
return fmt.Errorf("%s.quantity must be positive when present", prefix)
}
if len(event.SourceRefs) == 0 {
return fmt.Errorf("%s.source_refs must contain at least one reference", prefix)
}
for refIndex, ref := range event.SourceRefs {
refPrefix := fmt.Sprintf("%s.source_refs[%d]", prefix, refIndex)
if strings.TrimSpace(ref.SourceID) == "" {
return fmt.Errorf("%s.source_id must not be empty", refPrefix)
}
if ref.StartUnitID <= 0 {
return fmt.Errorf("%s.start_unit_id must be positive", refPrefix)
}
if ref.EndUnitID <= 0 {
return fmt.Errorf("%s.end_unit_id must be positive", refPrefix)
}
}
}
return nil
}
func cloneList(value dnd.ItemEventList) dnd.ItemEventList {
if value.Events == nil {
return dnd.ItemEventList{}
}
cloned := dnd.ItemEventList{Events: make([]dnd.ItemEvent, len(value.Events))}
for index, event := range value.Events {
cloned.Events[index] = event
if event.Quantity != nil {
quantity := *event.Quantity
cloned.Events[index].Quantity = &quantity
}
if event.SourceRefs != nil {
cloned.Events[index].SourceRefs = append([]source.SourceRef(nil), event.SourceRefs...)
}
}
return cloned
}

View File

@@ -0,0 +1,147 @@
package itemevents
import (
"bytes"
"encoding/json"
"errors"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
)
func validList() dnd.ItemEventList {
quantity := 12
return dnd.ItemEventList{Events: []dnd.ItemEvent{
{Name: "Hidden Cache", Kind: dnd.ItemEventKindDiscovered, SourceRefs: refs(1, 1)},
{Name: "Gold Pieces", Kind: dnd.ItemEventKindAcquired, Quantity: &quantity, To: "party", SourceRefs: refs(2, 2)},
{Name: "Torch", Kind: dnd.ItemEventKindLost, From: "party", SourceRefs: refs(3, 3)},
{Name: "Healing Potion", Kind: dnd.ItemEventKindConsumed, From: "party", SourceRefs: refs(4, 4)},
{Name: "Moonblade", Kind: dnd.ItemEventKindTransferred, From: "Aria", To: "Borin", SourceRefs: refs(5, 5)},
}}
}
func refs(start, end int) []source.SourceRef {
return []source.SourceRef{{SourceID: "session", StartUnitID: start, EndUnitID: end}}
}
func TestCodecRoundTripAndIdentities(t *testing.T) {
codec := New()
value := validList()
content, err := codec.Encode(value)
if err != nil {
t.Fatalf("Encode() error = %v", err)
}
decoded, err := codec.Decode(content)
if err != nil || !reflect.DeepEqual(decoded, value) {
t.Fatalf("Decode() = %#v, %v; want %#v", decoded, err, value)
}
schema := codec.Schema()
if codec.Kind() != dnd.ItemEventListKind || codec.MediaType() != MediaType || schema.ID != SchemaID || schema.Name != SchemaName || schema.Version != SchemaVersion || !json.Valid(schema.JSONSchema) {
t.Fatalf("codec identity/schema = %q/%q %#v", codec.Kind(), codec.MediaType(), schema)
}
registry := pipeline.NewArtifactCodecRegistry()
if err := pipeline.RegisterArtifactCodec(registry, codec); err != nil {
t.Fatal(err)
}
spec, ok := registry.Spec(dnd.ItemEventListKind)
if !ok || spec.SchemaDigest != contracts.DigestArtifactSchema(schema) {
t.Fatalf("registered spec = %#v, %t", spec, ok)
}
if _, err := registry.Encode(dnd.ItemEventListKind, dnd.NPCList{}); err == nil {
t.Fatal("Encode() error = nil, want exact type rejection")
} else {
var typeErr *pipeline.ArtifactCodecTypeError
if !errors.As(err, &typeErr) {
t.Fatalf("Encode() error = %T, want ArtifactCodecTypeError", err)
}
}
}
func TestCodecSupportsEmptyListAndPreservesInvalidCandidates(t *testing.T) {
codec := New()
empty := dnd.ItemEventList{Events: []dnd.ItemEvent{}}
if content, err := codec.Encode(empty); err != nil || string(content) != `{"events":[]}` {
t.Fatalf("Encode() = %s, %v", content, err)
}
zero := 0
candidate := dnd.ItemEventList{Events: []dnd.ItemEvent{{
Name: " ", Kind: dnd.ItemEventKindTransferred, Quantity: &zero, From: "party", To: "Party",
SourceRefs: []source.SourceRef{{SourceID: "", StartUnitID: 0, EndUnitID: -1}},
}}}
content, err := codec.EncodeCandidate(candidate)
if err != nil || !json.Valid(content) {
t.Fatalf("EncodeCandidate() = %s, %v", content, err)
}
decoded, err := codec.DecodeCandidate(content)
if err != nil || !reflect.DeepEqual(decoded, candidate) {
t.Fatalf("DecodeCandidate() = %#v, %v; want %#v", decoded, err, candidate)
}
if _, err := codec.Decode(content); err == nil {
t.Fatal("Decode() error = nil, want semantic candidate rejection")
}
}
func TestCodecRejectsStrictJSONAndApprovedBoundaries(t *testing.T) {
validJSON := `{"events":[{"name":"Ring","kind":"acquired","to":"party","source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]}]}`
tests := []struct {
name, raw, want string
}{
{"malformed", `{`, "decode dnd item event list"},
{"unknown top level", `{"events":[],"unexpected":true}`, "unknown field"},
{"unknown event field", strings.Replace(validJSON, `"to":"party"`, `"to":"party","unexpected":true`, 1), "unknown field"},
{"unknown reference field", strings.Replace(validJSON, `"end_unit_id":1`, `"end_unit_id":1,"unexpected":true`, 1), "unknown field"},
{"trailing", `{"events":[]} {}`, "multiple JSON values"},
{"missing list", `{}`, "events must be present"},
{"zero quantity", strings.Replace(validJSON, `"to":"party"`, `"quantity":0,"to":"party"`, 1), "quantity must be positive"},
{"negative quantity", strings.Replace(validJSON, `"to":"party"`, `"quantity":-1,"to":"party"`, 1), "quantity must be positive"},
{"party transfer", strings.Replace(validJSON, `"kind":"acquired","to":"party"`, `"kind":"transferred","from":"party","to":"Borin"`, 1), "holders are incompatible"},
{"self transfer", strings.Replace(validJSON, `"kind":"acquired","to":"party"`, `"kind":"transferred","from":"Aria","to":"aria"`, 1), "holders are incompatible"},
{"missing source refs", strings.Replace(validJSON, `,"source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]`, "", 1), "source_refs must contain"},
{"empty source refs", strings.Replace(validJSON, `[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]`, `[]`, 1), "source_refs must contain"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
if _, err := New().Decode([]byte(test.raw)); err == nil || !strings.Contains(err.Error(), test.want) {
t.Fatalf("Decode() error = %v, want %q", err, test.want)
}
})
}
}
func TestCodecDeepCopiesBoundaryValuesAndMetadata(t *testing.T) {
codec := New()
value := validList()
content, err := codec.EncodeCandidate(value)
if err != nil {
t.Fatal(err)
}
decoded, err := codec.DecodeCandidate(content)
if err != nil {
t.Fatal(err)
}
if decoded.Events[1].Quantity == value.Events[1].Quantity || &decoded.Events[1].SourceRefs[0] == &value.Events[1].SourceRefs[0] {
t.Fatal("DecodeCandidate() retained caller-owned event fields")
}
*decoded.Events[1].Quantity = 99
decoded.Events[1].SourceRefs[0].SourceID = "changed"
if *value.Events[1].Quantity != 12 || value.Events[1].SourceRefs[0].SourceID != "session" {
t.Fatal("decoded item event aliases input")
}
first := codec.Schema()
first.JSONSchema[0] = '['
if second := codec.Schema(); !json.Valid(second.JSONSchema) || second.JSONSchema[0] == '[' {
t.Fatal("Schema() returned shared bytes")
}
metadata := codec.Metadata(value)
metadata["payload"] = bytes.Repeat([]byte("x"), 10)
if next := codec.Metadata(value); len(next) != 1 || next["event_count"] != len(value.Events) {
t.Fatalf("Metadata() = %#v", next)
}
}

View File

@@ -9,17 +9,21 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
npcregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/registry"
sceneregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/scenedescriptions/registry"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
const (
Key = "dnd/combat-turns"
mappingPolicy = "dnd.combat_turns.extract_mapping.v2"
sceneGatePolicy = "dnd.combat_turns.scene_gate.v1"
)
const (
NPCRegistryReferenceSlot = npcregistry.ReferenceSlot
NPCRegistryMaxBytes = npcregistry.MaxBytes
SceneDescriptionReferenceSlot = sceneregistry.ReferenceSlot
SceneDescriptionMaxBytes = sceneregistry.MaxBytes
)
var requiredCapabilities = []string{
@@ -47,6 +51,14 @@ func referenceSlots() []contracts.ReferenceSlot {
AcceptedArtifactKinds: []contracts.ArtifactKind{dnd.NPCListKind},
MaxBytes: NPCRegistryMaxBytes,
})
slots = append(slots, contracts.ReferenceSlot{
Name: SceneDescriptionReferenceSlot,
Description: "Required scene descriptions that gate combat eligibility and are not combat evidence.",
Required: true,
AcceptedMediaTypes: []string{"application/json"},
AcceptedArtifactKinds: []contracts.ArtifactKind{dnd.SceneDescriptionListKind},
MaxBytes: SceneDescriptionMaxBytes,
})
sort.Slice(slots, func(i, j int) bool { return slots[i].Name < slots[j].Name })
return slots
}
@@ -60,6 +72,7 @@ type Options struct{}
type Extractor struct {
llm contracts.StructuredLLMClient
npcResolver *npcregistry.Resolver
sceneResolver *sceneregistry.Resolver
promptSHA string
responseSchemaSHA string
}
@@ -79,6 +92,10 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
if err != nil {
return nil, extractorErrorf("prepare NPC registry prompt input: %w", err)
}
sceneResolver, err := sceneregistry.NewResolver(referenceSet)
if err != nil {
return nil, extractorErrorf("prepare scene eligibility: %w", err)
}
promptSHA, err := scriptoriumPromptMetadata()
if err != nil {
return nil, extractorErrorf("load prompt metadata: %w", err)
@@ -90,6 +107,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
return &Extractor{
llm: llmClient,
npcResolver: npcResolver,
sceneResolver: sceneResolver,
promptSHA: promptSHA,
responseSchemaSHA: responseSchema.SHA256,
}, nil
@@ -108,6 +126,7 @@ func (e *Extractor) ManifestMetadata() map[string]any {
"prompt_version": SchemaVersion,
"prompt_sha256": e.promptSHA,
"mapping_policy": mappingPolicy,
"scene_gate_policy": sceneGatePolicy,
"response_schema_key": string(ResponseSchemaKey),
"response_schema_id": ResponseSchemaID,
"response_schema_name": ResponseSchemaName,
@@ -119,6 +138,11 @@ func (e *Extractor) ManifestMetadata() map[string]any {
metadata["npc_registry_digest"] = seeded.Digest()
metadata["npc_count"] = seeded.Count()
}
seededScenes := e.sceneResolver.Seeded()
if seededScenes.Bound() {
metadata["scene_eligibility_digest"] = seededScenes.EligibilityDigest()
metadata["scene_description_count"] = seededScenes.Count()
}
return metadata
}
@@ -130,9 +154,12 @@ func (e *Extractor) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
{Name: "prompt", Value: e.promptSHA},
{Name: "response_schema", Value: e.responseSchemaSHA},
{Name: "mapping_policy", Value: mappingPolicy},
{Name: "scene_gate_policy", Value: sceneGatePolicy},
}
seeded := e.npcResolver.Seeded()
fingerprints = append(fingerprints, pipeline.CheckpointFingerprint{Name: "npc_registry", Value: seeded.ProjectionDigest()})
seededScenes := e.sceneResolver.Seeded()
fingerprints = append(fingerprints, pipeline.CheckpointFingerprint{Name: "scene_eligibility", Value: seededScenes.EligibilityDigest()})
return fingerprints
}
@@ -147,6 +174,31 @@ func (e *Extractor) Extract(ctx context.Context, req contracts.TypedExtractionRe
if err != nil {
return contracts.TypedExtractionResult[dnd.CombatTurnList]{}, extractorErrorf("%w", err)
}
if _, ok := req.References.Slots[SceneDescriptionReferenceSlot]; !ok {
return contracts.TypedExtractionResult[dnd.CombatTurnList]{}, extractorErrorf("required scene descriptions reference is missing")
}
scenes, err := e.sceneResolver.Resolve(req.References)
if err != nil {
return contracts.TypedExtractionResult[dnd.CombatTurnList]{}, extractorErrorf("resolve scene eligibility: %w", err)
}
match := scenes.Match(req.Chunk)
switch match.State {
case sceneregistry.MatchExact:
if match.Kind != dnd.SceneKindCombat {
return contracts.TypedExtractionResult[dnd.CombatTurnList]{Value: dnd.CombatTurnList{CombatTurns: []dnd.CombatTurn{}}}, nil
}
case sceneregistry.MatchMissing, sceneregistry.MatchMismatched:
return contracts.TypedExtractionResult[dnd.CombatTurnList]{
Value: dnd.CombatTurnList{CombatTurns: []dnd.CombatTurn{}},
Warnings: []contracts.Warning{{
Scope: SceneDescriptionReferenceSlot,
ReasonCode: "scene_classification_unavailable",
Message: "No exact scene classification was available; combat extraction was skipped.",
}},
}, nil
default:
return contracts.TypedExtractionResult[dnd.CombatTurnList]{}, extractorErrorf("unsupported scene eligibility match state %q", match.State)
}
order := shared.NewSourceRefOrder(req.Source)
npcRegistry, err := e.npcResolver.Resolve(req.References)
if err != nil {

View File

@@ -13,6 +13,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs"
scenecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity"
)
@@ -34,7 +35,7 @@ func TestExtractMapsAndOrdersCombatTurnsBySourcePosition(t *testing.T) {
},
}}}
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest(t))
if err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
@@ -57,7 +58,7 @@ func TestExtractMapsAndOrdersCombatTurnsBySourcePosition(t *testing.T) {
t.Fatalf("LLM request identity = %#v", request)
}
transcript := request.Inputs["transcript"]
if transcript.Name != "transcript" || transcript.MediaType != "application/json" || transcript.Digest != "sha256:chunk" || transcript.OriginURI != "file:///session-alpha.json" || !reflect.DeepEqual(transcript.Content, extractionRequest().Chunk.Content) {
if transcript.Name != "transcript" || transcript.MediaType != "application/json" || transcript.Digest != "sha256:chunk" || transcript.OriginURI != "file:///session-alpha.json" || !reflect.DeepEqual(transcript.Content, extractionRequest(t).Chunk.Content) {
t.Fatalf("transcript input = %#v, want chunk-scoped source input", transcript)
}
}
@@ -69,7 +70,7 @@ func TestExtractPreservesInvalidCandidatesForValidators(t *testing.T) {
SourceRefs: []combatSourceRefResponse{{StartUnitID: 99, EndUnitID: 0}},
},
}}}
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest(t))
if err != nil {
t.Fatalf("Extract() error = %v, want nil for candidate values", err)
}
@@ -93,10 +94,11 @@ func TestExtractUsesDocumentOrderForReferencesAndTurns(t *testing.T) {
}},
{Actor: "Second", TurnKind: "other", SourceRefs: []combatSourceRefResponse{{StartUnitID: 30, EndUnitID: 30}}},
}}}
req := extractionRequest()
req := extractionRequest(t)
req.Source.Units = []source.SourceUnit{{ID: 30}, {ID: 10}}
req.Chunk.Units = append([]source.SourceUnit(nil), req.Source.Units...)
req.Chunk.Ref = source.SourceRef{SourceID: req.Source.ID, StartUnitID: 30, EndUnitID: 10}
req.References = sceneReferences(t, req.Chunk, dnd.SceneKindCombat)
result, err := newExtractor(t, client).Extract(context.Background(), req)
if err != nil {
@@ -127,7 +129,10 @@ func TestExtractPassesReferencesAndNPCGroundingWithoutUsingItAsEvidence(t *testi
"glossary": {Slot: contracts.ReferenceSlot{Name: "glossary"}, Items: []contracts.ReferenceItem{{SlotName: "glossary", Content: []byte("ogre: a large foe")}}},
NPCRegistryReferenceSlot: {Slot: contracts.ReferenceSlot{Name: NPCRegistryReferenceSlot}, Items: []contracts.ReferenceItem{{SlotName: NPCRegistryReferenceSlot, MediaType: "application/json", Content: npcRegistryJSON(t)}}},
}}
req := extractionRequest()
req := extractionRequest(t)
for name, slot := range req.References.Slots {
references.Slots[name] = slot
}
req.References = references
if _, err := newExtractor(t, client, references).Extract(context.Background(), req); err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
@@ -143,14 +148,20 @@ func TestExtractPassesReferencesAndNPCGroundingWithoutUsingItAsEvidence(t *testi
if strings.Contains(string(inputs["transcript"].Content), "Aria: cleric") {
t.Fatal("transcript input contains reference content")
}
if _, ok := inputs[SceneDescriptionReferenceSlot]; ok {
t.Fatalf("prompt inputs include scene descriptions: %#v", inputs)
}
metadata := newExtractor(t, &fakeCombatTurnsLLMClient{}, references).ManifestMetadata()
if metadata["npc_count"] != 1 || !strings.HasPrefix(metadata["npc_registry_digest"].(string), "sha256:") {
t.Fatalf("bound registry metadata = %#v, want digest and count", metadata)
}
if metadata["scene_description_count"] != 1 || !strings.HasPrefix(metadata["scene_eligibility_digest"].(string), "sha256:") {
t.Fatalf("bound scene metadata = %#v, want digest and count", metadata)
}
fingerprints := newExtractor(t, &fakeCombatTurnsLLMClient{}, references).CheckpointFingerprints()
if len(fingerprints) != 4 || fingerprints[3].Name != "npc_registry" {
t.Fatalf("bound fingerprints = %#v, want local identities plus NPC registry", fingerprints)
if len(fingerprints) != 6 || fingerprints[3].Name != "scene_gate_policy" || fingerprints[4].Name != "npc_registry" || fingerprints[5].Name != "scene_eligibility" {
t.Fatalf("bound fingerprints = %#v, want local identities plus scene gate and NPC registry", fingerprints)
}
encoded, err := json.Marshal(metadata)
if err != nil {
@@ -164,7 +175,7 @@ func TestExtractPassesReferencesAndNPCGroundingWithoutUsingItAsEvidence(t *testi
func TestExtractUnboundRegistryUsesExactEmptyPromptAndOmitsIdentity(t *testing.T) {
client := &fakeCombatTurnsLLMClient{response: extractionResponse{CombatTurns: []combatTurnResponse{}}}
extractor := newExtractor(t, client)
if _, err := extractor.Extract(context.Background(), extractionRequest()); err != nil {
if _, err := extractor.Extract(context.Background(), extractionRequest(t)); err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
input := client.requests[0].Inputs[NPCRegistryReferenceSlot]
@@ -175,7 +186,7 @@ func TestExtractUnboundRegistryUsesExactEmptyPromptAndOmitsIdentity(t *testing.T
if _, ok := metadata["npc_registry_digest"]; ok {
t.Fatalf("unbound metadata has registry digest: %#v", metadata)
}
if fingerprints := extractor.CheckpointFingerprints(); len(fingerprints) != 4 || fingerprints[3].Name != "npc_registry" || fingerprints[3].Value != input.Digest {
if fingerprints := extractor.CheckpointFingerprints(); len(fingerprints) != 6 || fingerprints[4].Name != "npc_registry" || fingerprints[4].Value != input.Digest || fingerprints[5].Name != "scene_eligibility" || fingerprints[5].Value == "" {
t.Fatalf("unbound fingerprints = %#v, want empty-projection identity", fingerprints)
}
}
@@ -184,13 +195,11 @@ func TestExtractorResolvesOperationNPCOverrideWithoutSingletonMetadata(t *testin
content := npcRegistryJSON(t)
client := &fakeCombatTurnsLLMClient{response: extractionResponse{CombatTurns: []combatTurnResponse{}}}
extractor := newExtractor(t, client)
request := extractionRequest()
request.References = contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
NPCRegistryReferenceSlot: {
request := extractionRequest(t)
request.References.Slots[NPCRegistryReferenceSlot] = contracts.ResolvedReferenceSlot{
Slot: contracts.ReferenceSlot{Name: NPCRegistryReferenceSlot},
Items: []contracts.ReferenceItem{{SlotName: NPCRegistryReferenceSlot, MediaType: npccodec.MediaType, Content: append([]byte(nil), content...), Origin: contracts.ReferenceOrigin{Type: "generated"}}},
},
}}
}
if _, err := extractor.Extract(context.Background(), request); err != nil {
t.Fatalf("Extract() error = %v", err)
}
@@ -198,11 +207,151 @@ func TestExtractorResolvesOperationNPCOverrideWithoutSingletonMetadata(t *testin
if input.Digest == "" || string(input.Content) != `{"npcs":[{"name":"Mira Thorn"}]}` || input.OriginURI != "" {
t.Fatalf("operation NPC input = %#v, want generated canonical grounding without provenance", input)
}
if metadata := extractor.ManifestMetadata(); metadata["npc_registry_digest"] != nil || metadata["npc_count"] != nil {
if metadata := extractor.ManifestMetadata(); metadata["npc_registry_digest"] != nil || metadata["npc_count"] != nil || metadata["scene_eligibility_digest"] != nil || metadata["scene_description_count"] != nil {
t.Fatalf("singleton metadata = %#v, want no operation-varying NPC identity", metadata)
}
}
func TestExtractAppliesSceneEligibilityBeforePromptConstruction(t *testing.T) {
for _, test := range []struct {
name string
scenes []dnd.SceneDescription
wantWarning string
wantError string
wantCalls int
wantEmptyList bool
}{
{
name: "exact combat",
scenes: []dnd.SceneDescription{sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindCombat)},
wantCalls: 1,
},
{
name: "exact narrative",
scenes: []dnd.SceneDescription{sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindNarrative)},
wantEmptyList: true,
},
{
name: "exact recap",
scenes: []dnd.SceneDescription{sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindRecap)},
wantEmptyList: true,
},
{
name: "exact meta",
scenes: []dnd.SceneDescription{sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindMeta)},
wantEmptyList: true,
},
{
name: "missing ID",
scenes: []dnd.SceneDescription{func() dnd.SceneDescription {
value := sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindCombat)
value.ID = "other-chunk"
return value
}()},
wantWarning: "scene_classification_unavailable",
wantEmptyList: true,
},
{
name: "source mismatch",
scenes: []dnd.SceneDescription{func() dnd.SceneDescription {
value := sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindCombat)
value.SourceRef.SourceID = "other-source"
return value
}()},
wantWarning: "scene_classification_unavailable",
wantEmptyList: true,
},
{
name: "start mismatch",
scenes: []dnd.SceneDescription{func() dnd.SceneDescription {
value := sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindCombat)
value.SourceRef.StartUnitID = 2
return value
}()},
wantWarning: "scene_classification_unavailable",
wantEmptyList: true,
},
{
name: "end mismatch",
scenes: []dnd.SceneDescription{func() dnd.SceneDescription {
value := sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindCombat)
value.SourceRef.EndUnitID = 10
return value
}()},
wantWarning: "scene_classification_unavailable",
wantEmptyList: true,
},
{
name: "missing required reference",
wantError: "required scene descriptions reference is missing",
},
} {
t.Run(test.name, func(t *testing.T) {
client := &fakeCombatTurnsLLMClient{response: extractionResponse{CombatTurns: []combatTurnResponse{}}}
req := extractionRequest(t)
if test.wantError != "" {
req.References = contracts.ReferenceSet{}
} else {
req.References = sceneListReferences(t, test.scenes...)
}
result, err := newExtractor(t, client).Extract(context.Background(), req)
if test.wantError != "" {
if err == nil || !strings.Contains(err.Error(), test.wantError) {
t.Fatalf("Extract() error = %v, want %q", err, test.wantError)
}
} else if err != nil {
t.Fatalf("Extract() error = %v", err)
}
if len(client.requests) != test.wantCalls {
t.Fatalf("LLM calls = %d, want %d", len(client.requests), test.wantCalls)
}
if test.wantEmptyList && result.Value.CombatTurns == nil {
t.Fatal("CombatTurns = nil, want accepted non-nil empty list")
}
if test.wantWarning != "" {
if len(result.Warnings) != 1 || result.Warnings[0].Scope != SceneDescriptionReferenceSlot || result.Warnings[0].ReasonCode != test.wantWarning {
t.Fatalf("warnings = %#v", result.Warnings)
}
} else if len(result.Warnings) != 0 {
t.Fatalf("warnings = %#v, want none", result.Warnings)
}
})
}
}
func TestSceneEligibilityMetadataAndFingerprintsTrackGatingValues(t *testing.T) {
chunk := extractionRequest(t).Chunk
base := sceneDescription(chunk, dnd.SceneKindCombat)
baseExtractor := newExtractor(t, &fakeCombatTurnsLLMClient{}, sceneListReferences(t, base))
metadata := baseExtractor.ManifestMetadata()
if metadata["scene_gate_policy"] != sceneGatePolicy || metadata["scene_description_count"] != 1 {
t.Fatalf("metadata = %#v, want scene gate policy and count", metadata)
}
baseDigest := fingerprintValue(t, baseExtractor.CheckpointFingerprints(), "scene_eligibility")
proseChanged := base
proseChanged.Title = "Different title"
proseChanged.Summary = "Different summary."
if got := fingerprintValue(t, newExtractor(t, &fakeCombatTurnsLLMClient{}, sceneListReferences(t, proseChanged)).CheckpointFingerprints(), "scene_eligibility"); got != baseDigest {
t.Fatalf("prose-only eligibility digest = %q, want %q", got, baseDigest)
}
for _, test := range []struct {
name string
change func(*dnd.SceneDescription)
}{
{"kind", func(value *dnd.SceneDescription) { value.Kind = dnd.SceneKindNarrative }},
{"range", func(value *dnd.SceneDescription) { value.SourceRef.StartUnitID = 2 }},
} {
t.Run(test.name, func(t *testing.T) {
changed := base
test.change(&changed)
if got := fingerprintValue(t, newExtractor(t, &fakeCombatTurnsLLMClient{}, sceneListReferences(t, changed)).CheckpointFingerprints(), "scene_eligibility"); got == baseDigest {
t.Fatal("eligibility digest did not change")
}
})
}
}
func TestNewRejectsMalformedNPCRegistryBeforeLLMCallWithoutContent(t *testing.T) {
client := &fakeCombatTurnsLLMClient{}
references := contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
@@ -220,8 +369,25 @@ func TestNewRejectsMalformedNPCRegistryBeforeLLMCallWithoutContent(t *testing.T)
}
}
func TestNewRejectsInvalidSceneReferenceBeforeLLMCallWithoutContent(t *testing.T) {
client := &fakeCombatTurnsLLMClient{}
references := contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
SceneDescriptionReferenceSlot: {
Slot: contracts.ReferenceSlot{Name: SceneDescriptionReferenceSlot},
Items: []contracts.ReferenceItem{{SlotName: SceneDescriptionReferenceSlot, MediaType: scenecodec.MediaType, Content: []byte(`{"scenes":[]}`)}},
},
}}
_, err := New(client, Options{}, references)
if err == nil || !strings.Contains(err.Error(), "prepare scene eligibility") || strings.Contains(err.Error(), `{"scenes":[]}`) {
t.Fatalf("New() error = %v, want bounded content-free scene failure", err)
}
if len(client.requests) != 0 {
t.Fatalf("LLM calls = %d, want none during failed construction", len(client.requests))
}
}
func TestExtractRejectsInvalidRequestsAndWrapsProviderFailures(t *testing.T) {
validReq := extractionRequest()
validReq := extractionRequest(t)
validExtractor := newExtractor(t, &fakeCombatTurnsLLMClient{response: extractionResponse{CombatTurns: []combatTurnResponse{}}})
var nilExtractor *Extractor
tests := []struct {
@@ -243,7 +409,7 @@ func TestExtractRejectsInvalidRequestsAndWrapsProviderFailures(t *testing.T) {
}
})
}
_, err := newExtractor(t, &fakeCombatTurnsLLMClient{err: errors.New("provider unavailable")}).Extract(context.Background(), extractionRequest())
_, err := newExtractor(t, &fakeCombatTurnsLLMClient{err: errors.New("provider unavailable")}).Extract(context.Background(), extractionRequest(t))
if err == nil || !strings.Contains(err.Error(), "dnd combat turns") || !strings.Contains(err.Error(), "provider unavailable") {
t.Fatalf("provider error = %v, want contextual wrapped error", err)
}
@@ -267,7 +433,7 @@ func TestExtractorManifestMetadataAndFingerprints(t *testing.T) {
t.Fatalf("metadata[%q] = %#v, want digest", key, metadata[key])
}
}
wantNames := map[string]struct{}{"prompt": {}, "response_schema": {}, "mapping_policy": {}, "npc_registry": {}}
wantNames := map[string]struct{}{"prompt": {}, "response_schema": {}, "mapping_policy": {}, "scene_gate_policy": {}, "npc_registry": {}, "scene_eligibility": {}}
for _, fingerprint := range extractor.CheckpointFingerprints() {
if _, ok := wantNames[fingerprint.Name]; !ok {
t.Fatalf("unexpected fingerprint = %#v", fingerprint)
@@ -304,7 +470,8 @@ func TestModuleSpecAndRegistration(t *testing.T) {
}
}
func extractionRequest() contracts.TypedExtractionRequest {
func extractionRequest(t *testing.T) contracts.TypedExtractionRequest {
t.Helper()
doc := combatSourceDocument()
chunk := &source.Chunk{
ID: "session-alpha:chunk:0",
@@ -315,13 +482,15 @@ func extractionRequest() contracts.TypedExtractionRequest {
MediaType: "application/json",
Units: append([]source.SourceUnit(nil), doc.Units...),
}
return contracts.TypedExtractionRequest{
request := contracts.TypedExtractionRequest{
Source: doc,
Chunk: chunk,
SourceInput: contracts.NewLLMInputMaterial("source", chunk.MediaType, chunk.Content, "sha256:chunk", "file:///session-alpha.json"),
SessionID: "session-123",
LLMProfile: "profile-combat",
}
request.References = sceneReferences(t, chunk, dnd.SceneKindCombat)
return request
}
func combatSourceDocument() *source.SourceDocument {
@@ -358,6 +527,46 @@ func npcRegistryJSON(t *testing.T) []byte {
return content
}
func sceneReferences(t *testing.T, chunk *source.Chunk, kind dnd.SceneKind) contracts.ReferenceSet {
t.Helper()
return sceneListReferences(t, sceneDescription(chunk, kind))
}
func sceneListReferences(t *testing.T, scenes ...dnd.SceneDescription) contracts.ReferenceSet {
t.Helper()
content, err := scenecodec.New().Encode(dnd.SceneDescriptionList{Scenes: scenes})
if err != nil {
t.Fatalf("encode scene descriptions: %v", err)
}
return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
SceneDescriptionReferenceSlot: {
Slot: contracts.ReferenceSlot{Name: SceneDescriptionReferenceSlot},
Items: []contracts.ReferenceItem{{SlotName: SceneDescriptionReferenceSlot, MediaType: scenecodec.MediaType, Content: content}},
},
}}
}
func sceneDescription(chunk *source.Chunk, kind dnd.SceneKind) dnd.SceneDescription {
return dnd.SceneDescription{
ID: chunk.ID,
SourceRef: chunk.Ref,
Kind: kind,
Title: "Combat eligibility",
Summary: "A valid scene description for combat extraction.",
}
}
func fingerprintValue(t *testing.T, fingerprints []pipeline.CheckpointFingerprint, name string) string {
t.Helper()
for _, fingerprint := range fingerprints {
if fingerprint.Name == name {
return fingerprint.Value
}
}
t.Fatalf("fingerprints = %#v, missing %q", fingerprints, name)
return ""
}
type fakeCombatTurnsLLMClient struct {
response extractionResponse
err error

View File

@@ -0,0 +1,6 @@
package itemevents
import "embed"
//go:embed assets/schemas/dnd_item_events_llm.v1.json assets/prompts/*.yaml assets/prompts/*.md
var embeddedAssets embed.FS

View File

@@ -0,0 +1,42 @@
id: dnd.item_events
version: "v1"
default_profile: gemini-2-flash
inputs:
- name: transcript
required: true
content_type: application/json
- name: players
required: false
content_type: text/plain
- name: party
required: false
content_type: text/plain
- name: glossary
required: false
content_type: text/plain
messages:
- role: system
content_file: ./sharedassets/common-dnd-system.md
- role: user
content_file: ./sharedassets/common-dnd-extraction-evidence.md
- role: user
content_file: ./sharedassets/common-dnd-identity.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-references.md
cache_control:
type: ephemeral
- role: user
content_file: ./task.md
- role: user
content_file: ./instructions.md
cache_control:
type: ephemeral
- role: user
content_file: ./sharedassets/common-dnd-transcript.md
output:
format: json
validation_mode: json_schema
schema_path: dnd_item_events_llm.v1.json
repair_attempts: 0

View File

@@ -0,0 +1,25 @@
Return one event only when the transcript establishes a meaningful item or
currency occurrence. Use a concise observed item name and preserve the stated
currency denomination; include quantity only when the transcript explicitly
states it.
Use `discovered` when the party learns of or encounters an item without
establishing possession. Use `acquired` when the party or a party member gains
possession. Use `lost` when party possession ends through a gift, sale, payment,
theft, abandonment, or destruction not caused by intended use. Use `consumed`
when intended use depletes an expendable item. Monetary spending, purchases, and
payments are always `lost`, not `consumed`. Classify currency as `consumed` only
when the transcript explicitly describes it being physically destroyed or
expended as a non-payment component. Use `transferred` only when possession
moves between two distinct named party members.
For `discovered`, omit both holders. For `acquired`, provide only `to`; for
`lost` and `consumed`, provide only `from`; and for `transferred`, provide both
`from` and `to`. Use `party` only for collective or unresolved party possession,
never for either side of a transfer. Do not emit a transfer for a gift, sale, or
payment outside the party.
Ordinary non-depleting use is not an event. Do not infer acquisition from a
discovery, or discovery from an acquisition: emit both only when each is
independently established. Every event needs at least one narrow transcript
range. Return no lore, inventory totals, aliases, or unstated holder details.

View File

@@ -0,0 +1,7 @@
Extract meaningful Dungeons & Dragons item and currency events established by
the transcript. Record only discoveries and changes in party possession, with
the transcript ranges that support each event.
This is an event history, not an inventory or ledger. Do not calculate balances,
resolve item identity across records, or infer ownership that the transcript
does not establish.

View File

@@ -0,0 +1,36 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.item_events.llm",
"type": "object",
"additionalProperties": false,
"required": ["events"],
"properties": {
"events": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name", "kind", "source_refs"],
"properties": {
"name": {"type": "string"},
"kind": {"type": "string"},
"quantity": {"type": "integer"},
"from": {"type": "string"},
"to": {"type": "string"},
"source_refs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["start_segment", "end_segment"],
"properties": {
"start_segment": {"type": "integer"},
"end_segment": {"type": "integer"}
}
}
}
}
}
}
}
}

View File

@@ -0,0 +1,99 @@
package itemevents
import (
"sort"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
type orderedItemEventResponse struct {
value itemEventResponse
earliest int
hasEvidence bool
}
func canonicalizeResponse(response *extractionResponse, order shared.SourceRefOrder, sourceID string) {
if response == nil {
return
}
ordered := make([]orderedItemEventResponse, len(response.Events))
for index := range response.Events {
earliest, hasEvidence := canonicalizeItemEvent(&response.Events[index], order, sourceID)
ordered[index] = orderedItemEventResponse{
value: response.Events[index],
earliest: earliest,
hasEvidence: hasEvidence,
}
}
sort.SliceStable(ordered, func(left, right int) bool {
if ordered[left].hasEvidence != ordered[right].hasEvidence {
return ordered[left].hasEvidence
}
if !ordered[left].hasEvidence {
return false
}
return ordered[left].earliest < ordered[right].earliest
})
for index := range ordered {
response.Events[index] = ordered[index].value
}
}
func canonicalizeItemEvent(event *itemEventResponse, order shared.SourceRefOrder, sourceID string) (int, bool) {
if event == nil {
return 0, false
}
refs := order.Canonicalize(itemEventSourceRefs(event.SourceRefs, sourceID))
event.SourceRefs = itemEventResponseRefs(refs)
return order.EarliestValid(refs)
}
func canonicalItemEventList(response extractionResponse, sourceID string) dnd.ItemEventList {
if response.Events == nil {
return dnd.ItemEventList{}
}
events := make([]dnd.ItemEvent, len(response.Events))
for index, event := range response.Events {
events[index] = dnd.ItemEvent{
Name: event.Name,
Kind: dnd.ItemEventKind(event.Kind),
Quantity: cloneQuantity(event.Quantity),
From: event.From,
To: event.To,
SourceRefs: itemEventSourceRefs(event.SourceRefs, sourceID),
}
}
return dnd.ItemEventList{Events: events}
}
func itemEventSourceRefs(refs []itemEventSourceRefResponse, sourceID string) []source.SourceRef {
if refs == nil {
return nil
}
values := make([]source.SourceRef, len(refs))
for index, ref := range refs {
values[index] = source.SourceRef{SourceID: sourceID, StartUnitID: ref.StartSegment, EndUnitID: ref.EndSegment}
}
return values
}
func itemEventResponseRefs(refs []source.SourceRef) []itemEventSourceRefResponse {
if refs == nil {
return nil
}
values := make([]itemEventSourceRefResponse, len(refs))
for index, ref := range refs {
values[index] = itemEventSourceRefResponse{StartSegment: ref.StartUnitID, EndSegment: ref.EndUnitID}
}
return values
}
func cloneQuantity(value *int) *int {
if value == nil {
return nil
}
quantity := *value
return &quantity
}

View File

@@ -0,0 +1,163 @@
// Package itemevents extracts source-grounded D&D item events.
package itemevents
import (
"context"
"fmt"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
const Key = "dnd/item-events"
const mappingPolicy = "dnd.item_events.extract_mapping.v1"
var requiredCapabilities = []string{
"chunks",
"source.transcript",
}
var providedCapabilities = []string{
"dnd.item_events",
}
var referenceSlotDescriptions = shared.ReferenceSlotDescriptions{
Glossary: "Optional campaign glossary reference material used only for item and holder disambiguation.",
Party: "Optional party roster reference material used only for item and holder disambiguation.",
Players: "Optional player list reference material used only for item and holder disambiguation.",
Roster: "Deprecated alias for party roster reference material used only for item and holder disambiguation.",
}
func referenceSlots() []contracts.ReferenceSlot {
return shared.ReferenceSlots(referenceSlotDescriptions)
}
var _ contracts.Extractor[dnd.ItemEventList] = (*Extractor)(nil)
var _ contracts.ManifestMetadataProvider = (*Extractor)(nil)
var _ pipeline.CheckpointFingerprintProvider = (*Extractor)(nil)
type Options struct{}
type Extractor struct {
llm contracts.StructuredLLMClient
promptSHA string
responseSchemaSHA string
}
func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contracts.ReferenceSet) (*Extractor, error) {
if llmClient == nil {
return nil, extractorErrorf("LLM client must not be nil")
}
if len(references) > 1 {
return nil, extractorErrorf("at most one reference set may be supplied")
}
promptSHA, err := scriptoriumPromptMetadata()
if err != nil {
return nil, extractorErrorf("load prompt metadata: %w", err)
}
responseSchema, err := loadResponseSchema()
if err != nil {
return nil, extractorErrorf("load response schema: %w", err)
}
return &Extractor{llm: llmClient, promptSHA: promptSHA, responseSchemaSHA: responseSchema.SHA256}, nil
}
func (e *Extractor) Key() string { return Key }
func (e *Extractor) ReferenceSlots() []contracts.ReferenceSlot { return referenceSlots() }
func (e *Extractor) ManifestMetadata() map[string]any {
if e == nil {
return nil
}
return map[string]any{
"prompt_id": PromptID,
"prompt_version": SchemaVersion,
"prompt_sha256": e.promptSHA,
"response_schema_key": string(ResponseSchemaKey),
"response_schema_id": ResponseSchemaID,
"response_schema_name": ResponseSchemaName,
"response_schema_version": SchemaVersion,
"response_schema_sha256": e.responseSchemaSHA,
"mapping_policy": mappingPolicy,
}
}
func (e *Extractor) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
if e == nil {
return nil
}
return []pipeline.CheckpointFingerprint{
{Name: "prompt", Value: e.promptSHA},
{Name: "response_schema", Value: e.responseSchemaSHA},
{Name: "mapping_policy", Value: mappingPolicy},
}
}
func (e *Extractor) Extract(ctx context.Context, req contracts.TypedExtractionRequest) (contracts.TypedExtractionResult[dnd.ItemEventList], error) {
if e == nil {
return contracts.TypedExtractionResult[dnd.ItemEventList]{}, extractorErrorf("extractor must not be nil")
}
if e.llm == nil {
return contracts.TypedExtractionResult[dnd.ItemEventList]{}, extractorErrorf("LLM client must not be nil")
}
sourceInput, err := shared.PrepareChunkExtraction(ctx, req)
if err != nil {
return contracts.TypedExtractionResult[dnd.ItemEventList]{}, extractorErrorf("%w", err)
}
order := shared.NewSourceRefOrder(req.Source)
var response extractionResponse
if _, err := e.llm.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
StageName: Key,
PromptID: PromptID,
PromptVersion: SchemaVersion,
ProfileID: req.LLMProfile,
SessionID: req.SessionID,
Inputs: shared.PromptInputs(sourceInput, req.References),
}, &response); err != nil {
return contracts.TypedExtractionResult[dnd.ItemEventList]{}, extractorErrorf("complete structured output: %w", err)
}
canonicalizeResponse(&response, order, req.Source.ID)
return contracts.TypedExtractionResult[dnd.ItemEventList]{Value: canonicalItemEventList(response, req.Source.ID)}, nil
}
func ModuleSpec() pipeline.ModuleSpec {
return pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageExtract,
Requires: append([]string(nil), requiredCapabilities...),
Provides: append([]string(nil), providedCapabilities...),
ArtifactKind: dnd.ItemEventListKind,
ReferenceSlots: referenceSlots(),
}
}
func Register(registry *pipeline.ExtractorRegistry) error {
return pipeline.RegisterExtractorBuilder(registry, ModuleSpec(), validateOptions, func(request pipeline.BuildRequest) (contracts.Extractor[dnd.ItemEventList], error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return New(request.Dependencies.LLM, options, request.References)
})
}
func validateOptions(options map[string]any) error {
_, err := DecodeOptions(options)
return err
}
func DecodeOptions(options map[string]any) (Options, error) {
if err := pipeline.RejectUnknownOptions(options); err != nil {
return Options{}, extractorErrorf("%w", err)
}
return Options{}, nil
}
func extractorErrorf(format string, args ...any) error {
return fmt.Errorf("dnd item events extractor: "+format, args...)
}

View File

@@ -0,0 +1,147 @@
package itemevents
import (
"context"
"errors"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
)
func TestExtractMapsAllEventKindsAndCanonicalizesEvidence(t *testing.T) {
quantity := 20
client := &fakeItemEventsLLMClient{response: extractionResponse{Events: []itemEventResponse{
{Name: "Torch", Kind: "lost", From: "party", SourceRefs: responseRefs(5, 5)},
{Name: "Moonblade", Kind: "transferred", From: "Aria", To: "Borin", SourceRefs: responseRefs(3, 3)},
{Name: "Hidden Cache", Kind: "discovered", SourceRefs: []itemEventSourceRefResponse{{StartSegment: 1, EndSegment: 1}, {StartSegment: 1, EndSegment: 1}}},
{Name: "Gold Pieces", Kind: "acquired", Quantity: &quantity, To: "party", SourceRefs: responseRefs(2, 2)},
{Name: "Healing Potion", Kind: "consumed", From: "party", SourceRefs: responseRefs(4, 4)},
}}}
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
if err != nil {
t.Fatalf("Extract() error = %v", err)
}
if got := []dnd.ItemEventKind{result.Value.Events[0].Kind, result.Value.Events[1].Kind, result.Value.Events[2].Kind, result.Value.Events[3].Kind, result.Value.Events[4].Kind}; !reflect.DeepEqual(got, []dnd.ItemEventKind{
dnd.ItemEventKindDiscovered, dnd.ItemEventKindAcquired, dnd.ItemEventKindTransferred, dnd.ItemEventKindConsumed, dnd.ItemEventKindLost,
}) {
t.Fatalf("event kinds = %#v, want source order", got)
}
acquired := result.Value.Events[1]
if acquired.Name != "Gold Pieces" || acquired.To != "party" || acquired.Quantity == nil || *acquired.Quantity != 20 {
t.Fatalf("acquired event = %#v", acquired)
}
if result.Value.Events[0].Quantity != nil || result.Value.Events[0].From != "" || result.Value.Events[0].To != "" {
t.Fatalf("discovered event = %#v, want omitted optional fields", result.Value.Events[0])
}
if refs := result.Value.Events[0].SourceRefs; len(refs) != 1 || refs[0] != (source.SourceRef{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 1}) {
t.Fatalf("source refs = %#v, want source ID attachment and deduplication", refs)
}
if acquired.Quantity == client.response.Events[3].Quantity {
t.Fatal("mapped quantity aliases private response")
}
if len(client.requests) != 1 {
t.Fatalf("LLM calls = %d, want 1", len(client.requests))
}
request := client.requests[0]
if request.StageName != Key || request.PromptID != PromptID || request.PromptVersion != SchemaVersion || request.ProfileID != "profile-item-events" || request.SessionID != "session-123" {
t.Fatalf("LLM request identity = %#v", request)
}
if transcript := request.Inputs["transcript"]; string(transcript.Content) != string(extractionRequest().Chunk.Content) || transcript.Name != "transcript" {
t.Fatalf("transcript input = %#v", transcript)
}
for _, name := range []string{"players", "party", "glossary"} {
if input := request.Inputs[name]; string(input.Content) != " " {
t.Fatalf("absent %s input = %#v, want retained empty prompt material", name, input)
}
}
}
func TestExtractPreservesInvalidCandidatesAndEmptyResults(t *testing.T) {
client := &fakeItemEventsLLMClient{content: []byte(`{"events":[{"name":"","kind":"transferred","quantity":0,"from":"party","to":"Party","source_refs":[{"start_segment":99,"end_segment":-1}]}]}`)}
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
if err != nil {
t.Fatalf("Extract() error = %v", err)
}
event := result.Value.Events[0]
if event.Name != "" || event.Kind != "transferred" || event.Quantity == nil || *event.Quantity != 0 || event.From != "party" || event.To != "Party" {
t.Fatalf("invalid candidate = %#v, want values preserved", event)
}
if refs := event.SourceRefs; len(refs) != 1 || refs[0] != (source.SourceRef{SourceID: "session-alpha", StartUnitID: 99, EndUnitID: -1}) {
t.Fatalf("source refs = %#v, want invalid range preserved", refs)
}
empty, err := newExtractor(t, &fakeItemEventsLLMClient{response: extractionResponse{Events: []itemEventResponse{}}}).Extract(context.Background(), extractionRequest())
if err != nil || empty.Value.Events == nil || len(empty.Value.Events) != 0 {
t.Fatalf("empty provider result = %#v, %v; want valid empty list", empty.Value, err)
}
}
func TestExtractUsesSourceDocumentOrderForCandidates(t *testing.T) {
client := &fakeItemEventsLLMClient{response: extractionResponse{Events: []itemEventResponse{
{Name: "Later", Kind: "discovered", SourceRefs: responseRefs(10, 10)},
{Name: "Earlier", Kind: "discovered", SourceRefs: responseRefs(100, 100)},
}}}
req := extractionRequest()
req.Source.Units = []source.SourceUnit{{ID: 100}, {ID: 10}}
req.Chunk.Units = append([]source.SourceUnit(nil), req.Source.Units...)
req.Chunk.Ref = source.SourceRef{SourceID: req.Source.ID, StartUnitID: 100, EndUnitID: 10}
result, err := newExtractor(t, client).Extract(context.Background(), req)
if err != nil {
t.Fatal(err)
}
if got := []string{result.Value.Events[0].Name, result.Value.Events[1].Name}; !reflect.DeepEqual(got, []string{"Earlier", "Later"}) {
t.Fatalf("event order = %#v, want source document order", got)
}
}
func TestExtractUsesOnlySupportedPromptInputs(t *testing.T) {
client := &fakeItemEventsLLMClient{response: extractionResponse{Events: []itemEventResponse{}}}
req := extractionRequest()
req.References = contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
"players": {Slot: contracts.ReferenceSlot{Name: "players"}, Items: []contracts.ReferenceItem{{SlotName: "players", Content: []byte("Dana: Aria")}}},
"party": {Slot: contracts.ReferenceSlot{Name: "party"}, Items: []contracts.ReferenceItem{{SlotName: "party", Content: []byte("Aria: ranger")}}},
"glossary": {Slot: contracts.ReferenceSlot{Name: "glossary"}, Items: []contracts.ReferenceItem{{SlotName: "glossary", Content: []byte("Moonblade: heirloom")}}},
"npcs": {Slot: contracts.ReferenceSlot{Name: "npcs"}, Items: []contracts.ReferenceItem{{SlotName: "npcs", Content: []byte("must not be used")}}},
}}
if _, err := newExtractor(t, client).Extract(context.Background(), req); err != nil {
t.Fatal(err)
}
inputs := client.requests[0].Inputs
if string(inputs["players"].Content) != "Dana: Aria" || string(inputs["party"].Content) != "Aria: ranger" || string(inputs["glossary"].Content) != "Moonblade: heirloom" {
t.Fatalf("reference inputs = %#v", inputs)
}
if _, ok := inputs["npcs"]; ok {
t.Fatalf("unexpected generated-lane input: %#v", inputs)
}
}
func TestExtractRetainsLocalErrorContext(t *testing.T) {
request := extractionRequest()
extractor := newExtractor(t, &fakeItemEventsLLMClient{})
var nilExtractor *Extractor
request.SourceInput = contracts.NewLLMInputMaterial("source", "application/json", []byte(`{"different":true}`), "sha256:other", "file:///other.json")
for _, test := range []struct {
name string
extractor *Extractor
req contracts.TypedExtractionRequest
want string
}{
{"nil extractor", nilExtractor, extractionRequest(), "extractor"},
{"nil client", &Extractor{}, extractionRequest(), "LLM client"},
{"preflight", extractor, request, "must match chunk"},
} {
t.Run(test.name, func(t *testing.T) {
if _, err := test.extractor.Extract(context.Background(), test.req); err == nil || !strings.Contains(err.Error(), "dnd item events") || !strings.Contains(err.Error(), test.want) {
t.Fatalf("Extract() error = %v", err)
}
})
}
if _, err := newExtractor(t, &fakeItemEventsLLMClient{err: errors.New("provider unavailable")}).Extract(context.Background(), extractionRequest()); err == nil || !strings.Contains(err.Error(), "provider unavailable") {
t.Fatalf("provider error = %v", err)
}
}

View File

@@ -0,0 +1,19 @@
package itemevents
type extractionResponse struct {
Events []itemEventResponse `json:"events"`
}
type itemEventResponse struct {
Name string `json:"name"`
Kind string `json:"kind"`
Quantity *int `json:"quantity,omitempty"`
From string `json:"from,omitempty"`
To string `json:"to,omitempty"`
SourceRefs []itemEventSourceRefResponse `json:"source_refs"`
}
type itemEventSourceRefResponse struct {
StartSegment int `json:"start_segment"`
EndSegment int `json:"end_segment"`
}

View File

@@ -0,0 +1,65 @@
package itemevents
import (
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
)
func TestConstructorSpecOptionsAndMetadata(t *testing.T) {
if _, err := New(nil, Options{}); err == nil || !strings.Contains(err.Error(), "LLM client") {
t.Fatalf("New(nil) error = %v", err)
}
if _, err := New(&fakeItemEventsLLMClient{}, Options{}, contracts.ReferenceSet{}, contracts.ReferenceSet{}); err == nil || !strings.Contains(err.Error(), "at most one reference set") {
t.Fatalf("New() error = %v", err)
}
want := pipeline.ModuleSpec{
Key: Key, Stage: pipeline.StageExtract, Requires: []string{"chunks", "source.transcript"}, Provides: []string{"dnd.item_events"}, ArtifactKind: dnd.ItemEventListKind,
ReferenceSlots: []contracts.ReferenceSlot{
{Name: "glossary", Description: referenceSlotDescriptions.Glossary, AcceptedMediaTypes: []string{"application/json", "application/x-yaml", "application/yaml", "text/markdown", "text/plain"}},
{Name: "party", Description: referenceSlotDescriptions.Party, AcceptedMediaTypes: []string{"application/json", "application/x-yaml", "application/yaml", "text/markdown", "text/plain"}},
{Name: "players", Description: referenceSlotDescriptions.Players, AcceptedMediaTypes: []string{"application/json", "application/x-yaml", "application/yaml", "text/markdown", "text/plain"}},
{Name: "roster", Description: referenceSlotDescriptions.Roster, AcceptedMediaTypes: []string{"application/json", "application/x-yaml", "application/yaml", "text/markdown", "text/plain"}},
},
}
if got := ModuleSpec(); !reflect.DeepEqual(got, want) {
t.Fatalf("ModuleSpec() = %#v, want %#v", got, want)
}
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil || !strings.Contains(err.Error(), "unknown option") {
t.Fatalf("DecodeOptions() error = %v", err)
}
registry := pipeline.NewExtractorRegistry()
if err := Register(registry); err != nil {
t.Fatal(err)
}
if got, ok := registry.Spec(Key); !ok || !reflect.DeepEqual(got, want) {
t.Fatalf("registry spec = %#v, %t", got, ok)
}
extractor := newExtractor(t, &fakeItemEventsLLMClient{})
metadata := extractor.ManifestMetadata()
for key, want := range map[string]string{
"prompt_id": PromptID, "prompt_version": SchemaVersion, "response_schema_key": string(ResponseSchemaKey), "response_schema_id": ResponseSchemaID,
"response_schema_name": ResponseSchemaName, "response_schema_version": SchemaVersion, "mapping_policy": mappingPolicy,
} {
if metadata[key] != want {
t.Fatalf("metadata[%q] = %#v, want %q", key, metadata[key], want)
}
}
for _, key := range []string{"prompt_sha256", "response_schema_sha256"} {
if value, ok := metadata[key].(string); !ok || !strings.HasPrefix(value, "sha256:") {
t.Fatalf("metadata[%q] = %#v", key, metadata[key])
}
}
if got := extractor.CheckpointFingerprints(); !reflect.DeepEqual(got, []pipeline.CheckpointFingerprint{
{Name: "prompt", Value: metadata["prompt_sha256"].(string)},
{Name: "response_schema", Value: metadata["response_schema_sha256"].(string)},
{Name: "mapping_policy", Value: mappingPolicy},
}) {
t.Fatalf("CheckpointFingerprints() = %#v", got)
}
}

View File

@@ -0,0 +1,21 @@
package itemevents
import "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
const (
PromptID = "dnd.item_events"
ResponseSchemaKey = llm.ResponseSchemaKey("dnd_item_events_llm")
ResponseSchemaID = "notarius.dnd.item_events.llm"
ResponseSchemaName = "notarius_dnd_item_events_llm_v1"
SchemaVersion = "v1"
)
func loadResponseSchema() (llm.ResponseSchema, error) {
return llm.LoadResponseSchema(embeddedAssets, llm.ResponseSchemaDefinition{
Key: ResponseSchemaKey,
ID: ResponseSchemaID,
Version: SchemaVersion,
Name: ResponseSchemaName,
AssetPath: "assets/schemas/dnd_item_events_llm.v1.json",
})
}

View File

@@ -0,0 +1,80 @@
package itemevents
import (
"bytes"
"encoding/json"
"strings"
"testing"
"github.com/santhosh-tekuri/jsonschema/v6"
)
func TestResponseSchemaIsStrictlyStructuralAndPrivate(t *testing.T) {
schema, err := loadResponseSchema()
if err != nil {
t.Fatal(err)
}
if schema.Key != ResponseSchemaKey || schema.ID != ResponseSchemaID || schema.Name != ResponseSchemaName || schema.Version != SchemaVersion || !strings.HasPrefix(schema.SHA256, "sha256:") || !json.Valid(schema.JSONSchema) {
t.Fatalf("schema = %#v", schema)
}
valid := map[string]any{"events": []any{map[string]any{
"name": "", "kind": "unsupported", "quantity": 0, "from": "party", "to": "Party",
"source_refs": []any{map[string]any{"start_segment": 0, "end_segment": -1}},
}}}
content, err := json.Marshal(valid)
if err != nil {
t.Fatal(err)
}
if err := validateJSONSchema(content, schema.JSONSchema); err != nil {
t.Fatalf("private schema rejected validator-owned values: %v", err)
}
for _, test := range []struct {
name string
value map[string]any
}{
{"missing events", map[string]any{}},
{"missing event name", map[string]any{"events": []any{map[string]any{"kind": "acquired", "source_refs": []any{}}}}},
{"unknown event field", map[string]any{"events": []any{map[string]any{"name": "Ring", "kind": "acquired", "source_refs": []any{}, "extra": true}}}},
{"unknown reference field", map[string]any{"events": []any{map[string]any{"name": "Ring", "kind": "acquired", "source_refs": []any{map[string]any{"start_segment": 1, "end_segment": 1, "extra": true}}}}}},
{"noninteger range", map[string]any{"events": []any{map[string]any{"name": "Ring", "kind": "acquired", "source_refs": []any{map[string]any{"start_segment": 1.5, "end_segment": 1}}}}}},
} {
t.Run(test.name, func(t *testing.T) {
content, err := json.Marshal(test.value)
if err != nil {
t.Fatal(err)
}
if err := validateJSONSchema(content, schema.JSONSchema); err == nil {
t.Fatal("private schema accepted invalid structure")
}
})
}
first := schema.JSONSchema
first[0] = '['
second, err := loadResponseSchema()
if err != nil || !json.Valid(second.JSONSchema) || bytes.Equal(first, second.JSONSchema) {
t.Fatalf("defensive schema copy = %s, %v", second.JSONSchema, err)
}
if _, ok := second.DiagnosticsMap()["json_schema"]; ok {
t.Fatalf("schema diagnostics leaked content: %#v", second.DiagnosticsMap())
}
}
func validateJSONSchema(instanceContent, schemaContent []byte) error {
instance, err := jsonschema.UnmarshalJSON(bytes.NewReader(instanceContent))
if err != nil {
return err
}
document, err := jsonschema.UnmarshalJSON(bytes.NewReader(schemaContent))
if err != nil {
return err
}
compiler := jsonschema.NewCompiler()
if err := compiler.AddResource("schema.json", document); err != nil {
return err
}
compiled, err := compiler.Compile("schema.json")
if err != nil {
return err
}
return compiled.Validate(instance)
}

View File

@@ -0,0 +1,52 @@
package itemevents
import (
"fmt"
"sync"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/notarius/internal/framework/promptfs"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
const scriptoriumPromptRoot = "assets/prompts"
var promptAssetManifest = shared.PromptAssetManifest{
ModuleDir: "dnd.item_events",
ModuleFiles: []promptfs.ModulePromptFile{
{Name: "dnd.item_events.yaml", Path: "assets/prompts/dnd.item_events.yaml"},
{Name: "task.md", Path: "assets/prompts/task.md"},
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
},
SharedFiles: []string{
"common-dnd-system.md",
"common-dnd-extraction-evidence.md",
"common-dnd-identity.md",
"common-dnd-references.md",
"common-dnd-transcript.md",
},
}
func RegisterPromptAssets(registry *llm.AssetRegistry) error {
promptFS, err := promptAssetManifest.PromptFS(embeddedAssets)
if err != nil {
return fmt.Errorf("prepare item event prompt assets: %w", err)
}
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil {
return err
}
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
}
func scriptoriumPromptMetadata() (string, error) {
scriptoriumPromptHashOnce.Do(func() {
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets)
})
return scriptoriumPromptHash, scriptoriumPromptHashErr
}
var (
scriptoriumPromptHashOnce sync.Once
scriptoriumPromptHash string
scriptoriumPromptHashErr error
)

View File

@@ -0,0 +1,114 @@
package itemevents
import (
"context"
"encoding/json"
"io/fs"
"reflect"
"strings"
"testing"
"time"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/scriptorium"
)
func TestPromptAssetsUseSharedSequenceAndTranscriptLast(t *testing.T) {
registry := llm.NewAssetRegistry()
if err := RegisterPromptAssets(registry); err != nil {
t.Fatal(err)
}
options, err := registry.ScriptoriumOptions()
if err != nil {
t.Fatal(err)
}
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
ID: "item-events-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "item-events-test-model",
})))
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...)
if err != nil {
t.Fatal(err)
}
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "item-events-test-profile",
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.InlineWithURI("file:///session.json", `{"segments":[1]}`),
"players": scriptorium.Inline("Dana: Aria"),
"party": scriptorium.Inline("Aria: ranger"),
"glossary": scriptorium.Inline("Moonblade: heirloom"),
},
})
if err != nil {
t.Fatal(err)
}
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_item_events_llm.v1.json" {
t.Fatalf("prepared prompt = %#v", prepared)
}
want := []struct {
role string
cached bool
marker string
}{
{"system", false, "Dungeons & Dragons gameplay transcripts"},
{"user", false, "Transcript units are the only evidence"},
{"user", true, "most specific supported in-world"},
{"user", true, "Dana: Aria"},
{"user", false, "item and currency events"},
{"user", true, "Ordinary non-depleting use"},
{"user", false, `{"segments":[1]}`},
}
if len(prepared.Messages) != len(want) {
t.Fatalf("prompt messages = %d, want %d", len(prepared.Messages), len(want))
}
for index, expected := range want {
message := prepared.Messages[index]
if message.Role != expected.role || !strings.Contains(message.Content, expected.marker) {
t.Fatalf("message %d = %#v", index, message)
}
if (message.CacheControl != nil) != expected.cached {
t.Fatalf("message %d cache control = %#v", index, message.CacheControl)
}
}
if strings.Contains(prepared.Messages[len(prepared.Messages)-1].Content, "Moonblade: heirloom") {
t.Fatal("transcript message contains optional reference content")
}
}
func TestPromptAssetsDoNotLeakIntoMetadata(t *testing.T) {
hash, err := scriptoriumPromptMetadata()
if err != nil || !strings.HasPrefix(hash, "sha256:") {
t.Fatalf("scriptoriumPromptMetadata() = %q, %v", hash, err)
}
metadata := newExtractor(t, &fakeItemEventsLLMClient{}).ManifestMetadata()
payload, err := json.Marshal(metadata)
if err != nil {
t.Fatal(err)
}
for _, forbidden := range []string{"meaningful Dungeons", "common-dnd-system", "start_segment", "dnd_item_events_llm.v1.json"} {
if strings.Contains(string(payload), forbidden) {
t.Fatalf("metadata leaked raw asset content %q: %s", forbidden, payload)
}
}
}
func TestPromptManifestReusesOnlySharedAssets(t *testing.T) {
want := []string{
"common-dnd-system.md",
"common-dnd-extraction-evidence.md",
"common-dnd-identity.md",
"common-dnd-references.md",
"common-dnd-transcript.md",
}
if !reflect.DeepEqual(promptAssetManifest.SharedFiles, want) {
t.Fatalf("shared assets = %#v, want %#v", promptAssetManifest.SharedFiles, want)
}
for _, path := range []string{"assets/prompts/task.md", "assets/prompts/instructions.md"} {
content, err := fs.ReadFile(embeddedAssets, path)
if err != nil {
t.Fatal(err)
}
if strings.Contains(string(content), "Transcript units are the only evidence") || strings.Contains(string(content), "Dungeons & Dragons gameplay transcripts") {
t.Fatalf("module asset %q copied shared prompt text", path)
}
}
}

View File

@@ -0,0 +1,92 @@
package itemevents
import (
"context"
"encoding/json"
"errors"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
)
func extractionRequest() contracts.TypedExtractionRequest {
doc := sourceDocument()
chunk := &source.Chunk{
ID: "session-alpha:chunk:0",
SourceID: doc.ID,
Index: 0,
Ref: source.SourceRef{SourceID: doc.ID, StartUnitID: 1, EndUnitID: 5},
Content: []byte(`{"segments":[1,2,3,4,5]}`),
MediaType: "application/json",
Units: append([]source.SourceUnit(nil), doc.Units...),
}
return contracts.TypedExtractionRequest{
Source: doc,
Chunk: chunk,
SourceInput: contracts.NewLLMInputMaterial("source", chunk.MediaType, chunk.Content, "sha256:chunk", "file:///session-alpha.json"),
SessionID: "session-123",
LLMProfile: "profile-item-events",
}
}
func sourceDocument() *source.SourceDocument {
return &source.SourceDocument{
ID: "session-alpha", Kind: "transcript", Format: "application/vnd.seriatim.minimal+json", Digest: "sha256:test",
Units: []source.SourceUnit{
{ID: 1, Kind: "transcript_segment", Text: "The party discovers a hidden cache."},
{ID: 2, Kind: "transcript_segment", Text: "They acquire 20 gold pieces."},
{ID: 3, Kind: "transcript_segment", Text: "Aria gives Borin the moonblade."},
{ID: 4, Kind: "transcript_segment", Text: "The party consumes a healing potion."},
{ID: 5, Kind: "transcript_segment", Text: "The party loses a torch."},
},
}
}
func responseRefs(start, end int) []itemEventSourceRefResponse {
return []itemEventSourceRefResponse{{StartSegment: start, EndSegment: end}}
}
func newExtractor(t *testing.T, client contracts.StructuredLLMClient, references ...contracts.ReferenceSet) *Extractor {
t.Helper()
extractor, err := New(client, Options{}, references...)
if err != nil {
t.Fatalf("New() error = %v", err)
}
return extractor
}
func cloneStructuredCompletionRequest(req contracts.StructuredCompletionRequest) contracts.StructuredCompletionRequest {
req.Inputs = req.Inputs.Clone()
return req
}
type fakeItemEventsLLMClient struct {
response extractionResponse
content []byte
err error
requests []contracts.StructuredCompletionRequest
}
func (client *fakeItemEventsLLMClient) CompleteStructured(_ context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
client.requests = append(client.requests, cloneStructuredCompletionRequest(req))
if client.err != nil {
return contracts.StructuredCompletionResponse{}, client.err
}
target, ok := out.(*extractionResponse)
if !ok {
return contracts.StructuredCompletionResponse{}, errors.New("unexpected output target")
}
content := append([]byte(nil), client.content...)
if len(content) == 0 {
var err error
content, err = json.Marshal(client.response)
if err != nil {
return contracts.StructuredCompletionResponse{}, err
}
}
if err := json.Unmarshal(content, target); err != nil {
return contracts.StructuredCompletionResponse{}, err
}
return contracts.StructuredCompletionResponse{Content: content}, nil
}

View File

@@ -0,0 +1,209 @@
// Package itemevents owns canonical ordering and durable domain rules for D&D
// item-event artifacts.
package itemevents
import (
"strconv"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
const partyHolder = "party"
// SupportedKind reports whether kind is one of the durable item-event kinds.
func SupportedKind(kind dnd.ItemEventKind) bool {
switch kind {
case dnd.ItemEventKindDiscovered,
dnd.ItemEventKindAcquired,
dnd.ItemEventKindLost,
dnd.ItemEventKindConsumed,
dnd.ItemEventKindTransferred:
return true
default:
return false
}
}
// DisplayValue returns the durable display form used for item and holder
// comparisons without changing the observed internal spelling.
func DisplayValue(value string) string { return strings.TrimSpace(value) }
// ComparisonKey returns the shared D&D Unicode- and case-insensitive key for
// a trimmed item or holder display value.
func ComparisonKey(value string) string { return identity.ComparisonKey(DisplayValue(value)) }
// HolderPresent reports whether value is a nonblank optional holder.
func HolderPresent(value string) bool { return DisplayValue(value) != "" }
// IsPartyHolder reports whether value denotes collective party possession.
func IsPartyHolder(value string) bool { return ComparisonKey(value) == partyHolder }
// ValidHolderCombination reports whether the optional holder fields satisfy
// the durable rules for kind. Blank holders are treated as absent so callers
// can preserve invalid extraction candidates for their owning validators.
func ValidHolderCombination(kind dnd.ItemEventKind, from, to string) bool {
hasFrom := HolderPresent(from)
hasTo := HolderPresent(to)
switch kind {
case dnd.ItemEventKindDiscovered:
return !hasFrom && !hasTo
case dnd.ItemEventKindAcquired:
return !hasFrom && hasTo
case dnd.ItemEventKindLost, dnd.ItemEventKindConsumed:
return hasFrom && !hasTo
case dnd.ItemEventKindTransferred:
return hasFrom && hasTo && !IsPartyHolder(from) && !IsPartyHolder(to) && ComparisonKey(from) != ComparisonKey(to)
default:
return false
}
}
// SourceRefsEqual reports whether two source-reference sequences have the
// same representation and values, including nil-versus-empty distinction.
func SourceRefsEqual(left, right []source.SourceRef) bool {
if (left == nil) != (right == nil) || len(left) != len(right) {
return false
}
for index := range left {
if left[index] != right[index] {
return false
}
}
return true
}
// ValidSourceRefs reports whether refs are non-empty and valid for index.
func ValidSourceRefs(index source.DocumentIndex, refs []source.SourceRef) bool {
if len(refs) == 0 {
return false
}
for _, ref := range refs {
if index.ValidateRef(ref) != nil {
return false
}
}
return true
}
// Less defines the canonical event order. Invalid source references remain
// comparable through SourceRefOrder's literal fallback so malformed candidates
// are still safe to sort and diagnose.
func Less(order shared.SourceRefOrder, left, right dnd.ItemEvent) bool {
leftPosition, leftHasEvidence := order.EarliestValid(left.SourceRefs)
rightPosition, rightHasEvidence := order.EarliestValid(right.SourceRefs)
if leftHasEvidence != rightHasEvidence {
return leftHasEvidence
}
if leftHasEvidence && leftPosition != rightPosition {
return leftPosition < rightPosition
}
if leftKey, rightKey := ComparisonKey(left.Name), ComparisonKey(right.Name); leftKey != rightKey {
return leftKey < rightKey
}
if leftName, rightName := DisplayValue(left.Name), DisplayValue(right.Name); leftName != rightName {
return leftName < rightName
}
if left.Kind != right.Kind {
return left.Kind < right.Kind
}
if less, decided := optionalStringLess(left.From, right.From); decided {
return less
}
if less, decided := optionalStringLess(left.To, right.To); decided {
return less
}
if less, decided := optionalQuantityLess(left.Quantity, right.Quantity); decided {
return less
}
return sourceRefsLess(order, order.Canonicalize(left.SourceRefs), order.Canonicalize(right.SourceRefs))
}
// ExactEqual reports whether events are exact duplicates after their display
// fields and evidence have been canonicalized for the supplied source order.
func ExactEqual(order shared.SourceRefOrder, left, right dnd.ItemEvent) bool {
if DisplayValue(left.Name) != DisplayValue(right.Name) || left.Kind != right.Kind ||
DisplayValue(left.From) != DisplayValue(right.From) || DisplayValue(left.To) != DisplayValue(right.To) ||
(left.Quantity == nil) != (right.Quantity == nil) {
return false
}
if left.Quantity != nil && *left.Quantity != *right.Quantity {
return false
}
return SourceRefsEqual(order.Canonicalize(left.SourceRefs), order.Canonicalize(right.SourceRefs))
}
// ExactIdentity returns a collision-safe duplicate key after display and
// evidence canonicalization. It is intended for callers that have already
// decided the event is eligible for duplicate handling.
func ExactIdentity(order shared.SourceRefOrder, event dnd.ItemEvent) string {
var key strings.Builder
writeKeyString(&key, DisplayValue(event.Name))
writeKeyString(&key, string(event.Kind))
writeKeyString(&key, DisplayValue(event.From))
writeKeyString(&key, DisplayValue(event.To))
if event.Quantity == nil {
key.WriteByte('0')
} else {
key.WriteByte('1')
writeKeyInt(&key, *event.Quantity)
}
for _, ref := range order.Canonicalize(event.SourceRefs) {
writeKeyString(&key, ref.SourceID)
writeKeyInt(&key, ref.StartUnitID)
writeKeyInt(&key, ref.EndUnitID)
}
return key.String()
}
func optionalStringLess(left, right string) (bool, bool) {
leftPresent, rightPresent := HolderPresent(left), HolderPresent(right)
if leftPresent != rightPresent {
return !leftPresent, true
}
if !leftPresent {
return false, false
}
if leftKey, rightKey := ComparisonKey(left), ComparisonKey(right); leftKey != rightKey {
return leftKey < rightKey, true
}
if leftValue, rightValue := DisplayValue(left), DisplayValue(right); leftValue != rightValue {
return leftValue < rightValue, true
}
return false, false
}
func optionalQuantityLess(left, right *int) (bool, bool) {
if (left == nil) != (right == nil) {
return left == nil, true
}
if left != nil && *left != *right {
return *left < *right, true
}
return false, false
}
func sourceRefsLess(order shared.SourceRefOrder, left, right []source.SourceRef) bool {
for index := 0; index < len(left) && index < len(right); index++ {
if left[index] == right[index] {
continue
}
return order.Less(left[index], right[index])
}
return len(left) < len(right)
}
func writeKeyString(builder *strings.Builder, value string) {
builder.WriteString(strconv.Itoa(len(value)))
builder.WriteByte(':')
builder.WriteString(value)
}
func writeKeyInt(builder *strings.Builder, value int) {
builder.WriteString(strconv.Itoa(value))
builder.WriteByte(';')
}

View File

@@ -0,0 +1,169 @@
package itemevents
import (
"sort"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
func TestValidHolderCombination(t *testing.T) {
valid := []struct {
kind dnd.ItemEventKind
from, to string
}{
{dnd.ItemEventKindDiscovered, "", ""},
{dnd.ItemEventKindAcquired, "", "party"},
{dnd.ItemEventKindLost, "party", ""},
{dnd.ItemEventKindConsumed, "party", ""},
{dnd.ItemEventKindTransferred, "Aria", "Borin"},
}
for _, test := range valid {
if !ValidHolderCombination(test.kind, test.from, test.to) {
t.Fatalf("ValidHolderCombination(%q, %q, %q) = false", test.kind, test.from, test.to)
}
}
invalid := []struct {
kind dnd.ItemEventKind
from, to string
}{
{dnd.ItemEventKindDiscovered, "Aria", ""},
{dnd.ItemEventKindAcquired, "", ""},
{dnd.ItemEventKindLost, "", ""},
{dnd.ItemEventKindConsumed, "", "Borin"},
{dnd.ItemEventKindTransferred, "party", "Borin"},
{dnd.ItemEventKindTransferred, "Aria", "Party"},
{dnd.ItemEventKindTransferred, "Aria", "aria"},
{dnd.ItemEventKindTransferred, "Åria", "Åria"},
{"unsupported", "", ""},
}
for _, test := range invalid {
if ValidHolderCombination(test.kind, test.from, test.to) {
t.Fatalf("ValidHolderCombination(%q, %q, %q) = true", test.kind, test.from, test.to)
}
}
}
func TestLessUsesEveryCanonicalTieBreaker(t *testing.T) {
order := testOrder()
ref := func(start, end int) []source.SourceRef {
return []source.SourceRef{{SourceID: "session", StartUnitID: start, EndUnitID: end}}
}
quantity := func(value int) *int { return &value }
base := dnd.ItemEvent{Name: "Amulet", Kind: dnd.ItemEventKindAcquired, To: "Borin", SourceRefs: ref(20, 20)}
tests := []struct {
name string
left, right dnd.ItemEvent
}{
{"earlier evidence", withRefs(base, ref(10, 10)), base},
{"valid evidence before malformed", base, withRefs(base, ref(999, 999))},
{"normalized name", withName(base, "Amulet"), withName(base, "Blade")},
{"exact trimmed name", withName(base, "Amulet"), withName(base, "amulet")},
{"kind", withKind(base, dnd.ItemEventKindAcquired), withKind(base, dnd.ItemEventKindLost)},
{"from presence", withFrom(base, ""), withFrom(base, "Aria")},
{"from normalized value", withFrom(base, "Aria"), withFrom(base, "Borin")},
{"from exact value", withFrom(base, "Aria"), withFrom(base, "aria")},
{"to presence", withoutTo(base), withTo(base, "Borin")},
{"to normalized value", withTo(base, "Aria"), withTo(base, "Borin")},
{"to exact value", withTo(base, "Aria"), withTo(base, "aria")},
{"quantity presence", withQuantity(base, nil), withQuantity(base, quantity(1))},
{"quantity value", withQuantity(base, quantity(1)), withQuantity(base, quantity(2))},
{"canonical source reference sequence", withRefs(base, ref(20, 20)), withRefs(base, ref(30, 30))},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
if !Less(order, test.left, test.right) || Less(order, test.right, test.left) {
t.Fatalf("Less() did not order %#v before %#v", test.left, test.right)
}
})
}
}
func TestLessAndExactEqualityCanonicalizeEvidence(t *testing.T) {
order := testOrder()
first := dnd.ItemEvent{
Name: " Silver Coin ", Kind: dnd.ItemEventKindAcquired, To: " party ",
SourceRefs: []source.SourceRef{
{SourceID: "session", StartUnitID: 30, EndUnitID: 30},
{SourceID: "session", StartUnitID: 10, EndUnitID: 10},
{SourceID: "session", StartUnitID: 10, EndUnitID: 10},
},
}
second := dnd.ItemEvent{
Name: "Silver Coin", Kind: dnd.ItemEventKindAcquired, To: "party",
SourceRefs: []source.SourceRef{
{SourceID: "session", StartUnitID: 10, EndUnitID: 10},
{SourceID: "session", StartUnitID: 30, EndUnitID: 30},
},
}
if !ExactEqual(order, first, second) {
t.Fatal("ExactEqual() = false, want canonical duplicate")
}
if Less(order, first, second) || Less(order, second, first) {
t.Fatal("Less() distinguished canonically equal records")
}
if ExactIdentity(order, first) != ExactIdentity(order, second) {
t.Fatal("ExactIdentity() differs for canonical duplicates")
}
quantity := 1
differentQuantity := second
differentQuantity.Quantity = &quantity
differentEvidence := second
differentEvidence.SourceRefs = append([]source.SourceRef(nil), second.SourceRefs...)
differentEvidence.SourceRefs[1].EndUnitID = 20
if ExactEqual(order, second, differentQuantity) || ExactEqual(order, second, differentEvidence) {
t.Fatal("ExactEqual() collapsed distinct optional field or evidence values")
}
}
func TestSourceReferenceHelpers(t *testing.T) {
refs := []source.SourceRef{{SourceID: "session", StartUnitID: 10, EndUnitID: 20}}
if !SourceRefsEqual(refs, append([]source.SourceRef(nil), refs...)) || SourceRefsEqual(nil, []source.SourceRef{}) {
t.Fatal("SourceRefsEqual() did not preserve source-reference representation")
}
doc := &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 10}, {ID: 20}, {ID: 30}}}
if !ValidSourceRefs(source.NewDocumentIndex(doc), refs) {
t.Fatal("ValidSourceRefs() = false, want valid reference")
}
if ValidSourceRefs(source.NewDocumentIndex(doc), []source.SourceRef{{SourceID: "other", StartUnitID: 10, EndUnitID: 20}}) {
t.Fatal("ValidSourceRefs() = true, want invalid source identifier rejection")
}
}
func TestLessSortsMalformedReferencesDeterministically(t *testing.T) {
order := testOrder()
events := []dnd.ItemEvent{
{Name: "A", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 999, EndUnitID: 999}}},
{Name: "A", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "other", StartUnitID: 1, EndUnitID: 1}}},
{Name: "A", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 10, EndUnitID: 10}}},
}
sort.SliceStable(events, func(left, right int) bool { return Less(order, events[left], events[right]) })
if events[0].SourceRefs[0].StartUnitID != 10 || events[1].SourceRefs[0].SourceID != "other" || events[2].SourceRefs[0].StartUnitID != 999 {
t.Fatalf("canonical sort = %#v", events)
}
}
func testOrder() shared.SourceRefOrder {
return shared.NewSourceRefOrder(&source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 10}, {ID: 20}, {ID: 30}}})
}
func withName(event dnd.ItemEvent, value string) dnd.ItemEvent { event.Name = value; return event }
func withKind(event dnd.ItemEvent, value dnd.ItemEventKind) dnd.ItemEvent {
event.Kind = value
return event
}
func withFrom(event dnd.ItemEvent, value string) dnd.ItemEvent { event.From = value; return event }
func withTo(event dnd.ItemEvent, value string) dnd.ItemEvent { event.To = value; return event }
func withoutTo(event dnd.ItemEvent) dnd.ItemEvent { event.To = ""; return event }
func withQuantity(event dnd.ItemEvent, value *int) dnd.ItemEvent {
event.Quantity = value
return event
}
func withRefs(event dnd.ItemEvent, value []source.SourceRef) dnd.ItemEvent {
event.SourceRefs = value
return event
}

View File

@@ -0,0 +1,254 @@
// Package itemevents normalizes merged D&D item-event candidates.
package itemevents
import (
"context"
"fmt"
"sort"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
itemeventmodel "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/itemevents"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics"
)
const (
Key = "dnd/item-events"
normalizationPolicy = "dnd.item_events.normalize.v1"
NormalizationPolicy = normalizationPolicy
ReasonCodeDisplayNormalized = "item_event_display_normalized"
ReasonCodeSourceRefsNormalized = "source_references_normalized"
ReasonCodeEventsReordered = "item_events_reordered"
ReasonCodeDuplicateCollapsed = "duplicate_item_event_collapsed"
ReasonCodeWarningsOmitted = "item_event_normalization_warnings_omitted"
)
var requiredCapabilities = []string{"merged"}
var providedCapabilities = []string{"normalized"}
var _ contracts.Normalizer[dnd.ItemEventList] = (*Normalizer)(nil)
var _ contracts.ManifestMetadataProvider = (*Normalizer)(nil)
var _ pipeline.CheckpointFingerprintProvider = (*Normalizer)(nil)
// Options deliberately has no fields: item-event normalization has no
// generated-reference dependency or configurable semantic behavior.
type Options struct{}
type Normalizer struct{}
func New(Options) *Normalizer { return &Normalizer{} }
func (n *Normalizer) Key() string { return Key }
func (n *Normalizer) ReferenceSlots() []contracts.ReferenceSlot { return nil }
func (n *Normalizer) ManifestMetadata() map[string]any {
if n == nil {
return nil
}
return map[string]any{"normalization_policy": normalizationPolicy}
}
func (n *Normalizer) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
if n == nil {
return nil
}
return []pipeline.CheckpointFingerprint{{Name: "normalization_policy", Value: normalizationPolicy}}
}
func (n *Normalizer) Normalize(ctx context.Context, req contracts.TypedNormalizeRequest[dnd.ItemEventList]) (contracts.TypedNormalizeResult[dnd.ItemEventList], error) {
if n == nil {
return contracts.TypedNormalizeResult[dnd.ItemEventList]{}, normalizerErrorf("normalizer must not be nil")
}
if ctx == nil {
return contracts.TypedNormalizeResult[dnd.ItemEventList]{}, normalizerErrorf("context must not be nil")
}
if err := ctx.Err(); err != nil {
return contracts.TypedNormalizeResult[dnd.ItemEventList]{}, normalizerErrorf("context error before normalize: %w", err)
}
index := source.NewDocumentIndex(req.Source)
value, warnings := normalizeList(req.MergeOutput.Value, index, shared.NewSourceRefOrderFromIndex(index))
return contracts.TypedNormalizeResult[dnd.ItemEventList]{Value: value, Warnings: warnings}, nil
}
type normalizedRecord struct {
event dnd.ItemEvent
inputIndex int
}
func normalizeList(input dnd.ItemEventList, index source.DocumentIndex, order shared.SourceRefOrder) (dnd.ItemEventList, []contracts.Warning) {
if input.Events == nil {
return dnd.ItemEventList{}, nil
}
records := make([]normalizedRecord, len(input.Events))
warnings := make([]contracts.Warning, 0)
for index, inputEvent := range input.Events {
event, changedFields, refsChanged := normalizeEvent(inputEvent, order)
records[index] = normalizedRecord{event: event, inputIndex: index}
if len(changedFields) != 0 {
warnings = append(warnings, contracts.Warning{
Scope: eventScope(index),
ReasonCode: ReasonCodeDisplayNormalized,
Message: fmt.Sprintf("input index %d: normalized display whitespace in %s", index,
diagnostics.Aggregate("fields", changedFields)),
})
}
if refsChanged {
warnings = append(warnings, contracts.Warning{
Scope: eventScope(index),
ReasonCode: ReasonCodeSourceRefsNormalized,
Message: fmt.Sprintf("input index %d: source references normalized (original count %d, final count %d)",
index, len(inputEvent.SourceRefs), len(event.SourceRefs)),
})
}
}
sort.SliceStable(records, func(left, right int) bool {
return itemeventmodel.Less(order, records[left].event, records[right].event)
})
for position, record := range records {
if position == record.inputIndex {
continue
}
warnings = append(warnings, contracts.Warning{
Scope: eventScope(record.inputIndex),
ReasonCode: ReasonCodeEventsReordered,
Message: fmt.Sprintf("input index %d moved to normalized position %d", record.inputIndex, position),
})
}
output, duplicateWarnings := collapseDuplicates(records, index, order)
warnings = append(warnings, duplicateWarnings...)
return dnd.ItemEventList{Events: output}, diagnostics.LimitWarnings(warnings, "item_events", ReasonCodeWarningsOmitted)
}
func normalizeEvent(input dnd.ItemEvent, order shared.SourceRefOrder) (dnd.ItemEvent, []string, bool) {
output := cloneEvent(input)
changedFields := make([]string, 0, 3)
for _, field := range []struct {
name string
value *string
}{
{name: "name", value: &output.Name},
{name: "from", value: &output.From},
{name: "to", value: &output.To},
} {
trimmed := itemeventmodel.DisplayValue(*field.value)
if *field.value != trimmed {
*field.value = trimmed
changedFields = append(changedFields, field.name)
}
}
output.SourceRefs = order.Canonicalize(input.SourceRefs)
return output, changedFields, !itemeventmodel.SourceRefsEqual(input.SourceRefs, output.SourceRefs)
}
func cloneEvent(input dnd.ItemEvent) dnd.ItemEvent {
output := input
if input.Quantity != nil {
quantity := *input.Quantity
output.Quantity = &quantity
}
if input.SourceRefs != nil {
output.SourceRefs = append([]source.SourceRef(nil), input.SourceRefs...)
}
return output
}
type duplicateGroup struct {
retainedIndex int
removed []int
}
func collapseDuplicates(records []normalizedRecord, index source.DocumentIndex, order shared.SourceRefOrder) ([]dnd.ItemEvent, []contracts.Warning) {
if len(records) == 0 {
return make([]dnd.ItemEvent, 0), nil
}
keep := make([]bool, len(records))
groups := make([]duplicateGroup, 0)
groupByKey := make(map[string]int)
for recordIndex, record := range records {
if !itemeventmodel.ValidSourceRefs(index, record.event.SourceRefs) {
keep[recordIndex] = true
continue
}
key := itemeventmodel.ExactIdentity(order, record.event)
groupIndex, exists := groupByKey[key]
if !exists {
groupByKey[key] = len(groups)
groups = append(groups, duplicateGroup{retainedIndex: record.inputIndex})
keep[recordIndex] = true
continue
}
groups[groupIndex].removed = append(groups[groupIndex].removed, record.inputIndex)
}
output := make([]dnd.ItemEvent, 0, len(records))
for recordIndex, record := range records {
if keep[recordIndex] {
output = append(output, cloneEvent(record.event))
}
}
warnings := make([]contracts.Warning, 0)
for _, group := range groups {
if len(group.removed) != 0 {
warnings = append(warnings, duplicateWarning(group.retainedIndex, group.removed))
}
}
return output, warnings
}
func duplicateWarning(retainedIndex int, removed []int) contracts.Warning {
issues := make([]string, len(removed))
for index, removedIndex := range removed {
issues[index] = fmt.Sprintf("removed input index %d", removedIndex)
}
return contracts.Warning{
Scope: eventScope(retainedIndex),
ReasonCode: ReasonCodeDuplicateCollapsed,
Message: diagnostics.Aggregate(
fmt.Sprintf("duplicate item event collapsed; retained input index %d", retainedIndex), issues),
}
}
func eventScope(index int) string { return fmt.Sprintf("events[%d]", index) }
func ModuleSpec() pipeline.ModuleSpec {
return pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageNormalize,
Requires: append([]string(nil), requiredCapabilities...),
Provides: append([]string(nil), providedCapabilities...),
ArtifactKind: dnd.ItemEventListKind,
}
}
func Register(registry *pipeline.NormalizerRegistry) error {
return pipeline.RegisterNormalizerBuilder(registry, ModuleSpec(), validateOptions, func(request pipeline.BuildRequest) (contracts.Normalizer[dnd.ItemEventList], error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return New(options), nil
})
}
func DecodeOptions(options map[string]any) (Options, error) {
if err := pipeline.RejectUnknownOptions(options); err != nil {
return Options{}, normalizerErrorf("%w", err)
}
return Options{}, nil
}
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }
func normalizerErrorf(format string, args ...any) error {
return fmt.Errorf("dnd item events normalizer: "+format, args...)
}

View File

@@ -0,0 +1,194 @@
package itemevents
import (
"context"
"reflect"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics"
)
func TestNormalizeTrimsCanonicalizesAndClones(t *testing.T) {
doc := &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 30}, {ID: 10}}}
quantity := 4
input := dnd.ItemEventList{Events: []dnd.ItemEvent{{
Name: " Gold Pieces ", Kind: dnd.ItemEventKindTransferred, Quantity: &quantity, From: " Borin ", To: " Aria ",
SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 10, EndUnitID: 10}, {SourceID: doc.ID, StartUnitID: 30, EndUnitID: 30}, {SourceID: doc.ID, StartUnitID: 30, EndUnitID: 30}},
}}}
result := normalize(t, doc, input)
got := result.Value.Events[0]
if got.Name != "Gold Pieces" || got.From != "Borin" || got.To != "Aria" || got.Quantity == input.Events[0].Quantity ||
!reflect.DeepEqual(got.SourceRefs, []source.SourceRef{{SourceID: doc.ID, StartUnitID: 30, EndUnitID: 30}, {SourceID: doc.ID, StartUnitID: 10, EndUnitID: 10}}) {
t.Fatalf("normalized event = %#v", got)
}
if !hasWarning(result.Warnings, ReasonCodeDisplayNormalized) || !hasWarning(result.Warnings, ReasonCodeSourceRefsNormalized) {
t.Fatalf("warnings = %#v", result.Warnings)
}
second := normalize(t, doc, result.Value)
if !reflect.DeepEqual(second.Value, result.Value) || len(second.Warnings) != 0 {
t.Fatalf("second normalization = %#v; want idempotent output without warnings", second)
}
*got.Quantity = 99
got.SourceRefs[0].StartUnitID = 999
if quantity != 4 || input.Events[0].SourceRefs[1].StartUnitID != 30 {
t.Fatalf("Normalize() aliased input: %#v", input)
}
}
func TestNormalizeOrdersAndCollapsesExactDuplicatesOnly(t *testing.T) {
doc := &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 30}, {ID: 10}, {ID: 20}}}
ref := func(unit int) source.SourceRef {
return source.SourceRef{SourceID: doc.ID, StartUnitID: unit, EndUnitID: unit}
}
quantity := 2
duplicate := dnd.ItemEvent{Name: " Potion ", Kind: dnd.ItemEventKindAcquired, Quantity: &quantity, To: " Aria ", SourceRefs: []source.SourceRef{ref(30)}}
input := dnd.ItemEventList{Events: []dnd.ItemEvent{
{Name: "Later", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(20)}},
duplicate,
{Name: "Potion", Kind: dnd.ItemEventKindAcquired, Quantity: &quantity, To: "Aria", SourceRefs: []source.SourceRef{ref(30)}},
{Name: "Potion", Kind: dnd.ItemEventKindAcquired, Quantity: &quantity, To: "Borin", SourceRefs: []source.SourceRef{ref(30)}},
{Name: "Potion", Kind: dnd.ItemEventKindAcquired, Quantity: &quantity, To: "Aria", SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 999, EndUnitID: 999}}},
}}
result := normalize(t, doc, input)
got := result.Value.Events
if len(got) != 4 || got[0].Name != "Potion" || got[0].To != "Aria" || got[1].To != "Borin" || got[2].Name != "Later" || got[3].SourceRefs[0].StartUnitID != 999 {
t.Fatalf("normalized events = %#v", got)
}
if !hasWarning(result.Warnings, ReasonCodeEventsReordered) || !hasWarning(result.Warnings, ReasonCodeDuplicateCollapsed) {
t.Fatalf("warnings = %#v", result.Warnings)
}
}
func TestNormalizeUsesEveryCanonicalTieBreaker(t *testing.T) {
doc := testDocument()
ref := func(unit int) source.SourceRef {
return source.SourceRef{SourceID: doc.ID, StartUnitID: unit, EndUnitID: unit}
}
quantity := 1
tests := []struct {
name string
earlier dnd.ItemEvent
later dnd.ItemEvent
}{
{"evidence position", dnd.ItemEvent{Name: "Z", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "A", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(20)}}},
{"normalized name", dnd.ItemEvent{Name: "A", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "B", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}},
{"exact name", dnd.ItemEvent{Name: "A", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "a", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}},
{"kind", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindAcquired, To: "Aria", SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}},
{"from presence", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, From: "Aria", SourceRefs: []source.SourceRef{ref(10)}}},
{"from comparison", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "A", To: "Z", SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "B", To: "Z", SourceRefs: []source.SourceRef{ref(10)}}},
{"exact from", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "A", To: "Z", SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "a", To: "Z", SourceRefs: []source.SourceRef{ref(10)}}},
{"to presence", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, To: "Aria", SourceRefs: []source.SourceRef{ref(10)}}},
{"to comparison", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "A", To: "A", SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "A", To: "B", SourceRefs: []source.SourceRef{ref(10)}}},
{"exact to", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "A", To: "A", SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "A", To: "a", SourceRefs: []source.SourceRef{ref(10)}}},
{"quantity", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, Quantity: &quantity, SourceRefs: []source.SourceRef{ref(10)}}},
{"reference sequence", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10), ref(20)}}},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
result := normalize(t, doc, dnd.ItemEventList{Events: []dnd.ItemEvent{test.later, test.earlier}})
if len(result.Value.Events) != 2 || !reflect.DeepEqual(result.Value.Events[0], test.earlier) || !reflect.DeepEqual(result.Value.Events[1], test.later) {
t.Fatalf("normalized events = %#v; want %#v then %#v", result.Value.Events, test.earlier, test.later)
}
})
}
}
func TestNormalizePreservesEmptyAndMalformedCandidates(t *testing.T) {
for _, input := range []dnd.ItemEventList{{}, {Events: []dnd.ItemEvent{}}} {
result := normalize(t, nil, input)
if (result.Value.Events == nil) != (input.Events == nil) {
t.Fatalf("empty representation changed: %#v", result.Value)
}
}
malformed := dnd.ItemEventList{Events: []dnd.ItemEvent{
{Name: " Coin ", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 99, EndUnitID: 99}}},
{Name: "Coin", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 99, EndUnitID: 99}}},
}}
result := normalize(t, testDocument(), malformed)
if len(result.Value.Events) != 2 || result.Value.Events[0].Name != "Coin" || result.Value.Events[1].Name != "Coin" {
t.Fatalf("malformed candidates were not safely retained: %#v", result.Value)
}
}
func TestNormalizeDoesNotInferOrConvertInventoryMeaning(t *testing.T) {
doc := testDocument()
quantity := 25
input := dnd.ItemEventList{Events: []dnd.ItemEvent{
{Name: " gold pieces ", Kind: dnd.ItemEventKindDiscovered, Quantity: &quantity, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 10, EndUnitID: 10}}},
{Name: " gp ", Kind: dnd.ItemEventKindAcquired, Quantity: &quantity, To: " party ", SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 20, EndUnitID: 20}}},
{Name: " potion of healing ", Kind: dnd.ItemEventKindTransferred, Quantity: &quantity, From: " Aria ", To: " Borin ", SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 30, EndUnitID: 30}}},
}}
result := normalize(t, doc, input)
got := make(map[string]dnd.ItemEvent, len(result.Value.Events))
for _, event := range result.Value.Events {
got[event.Name] = event
}
if event := got["gold pieces"]; event.Kind != dnd.ItemEventKindDiscovered || event.Quantity == nil || *event.Quantity != 25 {
t.Fatalf("discovery event was inferred or converted: %#v", event)
}
if event := got["gp"]; event.Kind != dnd.ItemEventKindAcquired || event.Quantity == nil || *event.Quantity != 25 || event.To != "party" {
t.Fatalf("acquisition event was inferred or denomination-converted: %#v", event)
}
if event := got["potion of healing"]; event.Kind != dnd.ItemEventKindTransferred || event.From != "Aria" || event.To != "Borin" || event.Quantity == nil || *event.Quantity != 25 {
t.Fatalf("transfer event was inferred, aliased, singularized, or ledger-adjusted: %#v", event)
}
}
func TestNormalizerContractAndBoundedWarnings(t *testing.T) {
normalizer := New(Options{})
if spec := ModuleSpec(); spec.Key != Key || spec.Stage != pipeline.StageNormalize || spec.ArtifactKind != dnd.ItemEventListKind || len(spec.ReferenceSlots) != 0 ||
!reflect.DeepEqual(spec.Requires, []string{"merged"}) || !reflect.DeepEqual(spec.Provides, []string{"normalized"}) {
t.Fatalf("ModuleSpec() = %#v", spec)
}
if metadata := normalizer.ManifestMetadata(); !reflect.DeepEqual(metadata, map[string]any{"normalization_policy": normalizationPolicy}) {
t.Fatalf("ManifestMetadata() = %#v", metadata)
}
if fingerprints := normalizer.CheckpointFingerprints(); !reflect.DeepEqual(fingerprints, []pipeline.CheckpointFingerprint{{Name: "normalization_policy", Value: normalizationPolicy}}) {
t.Fatalf("CheckpointFingerprints() = %#v", fingerprints)
}
registry := pipeline.NewNormalizerRegistry()
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v", err)
}
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
t.Fatal("DecodeOptions() accepted unknown option")
}
count := diagnostics.MaxWarnings + 5
doc := &source.SourceDocument{ID: "session", Units: make([]source.SourceUnit, count)}
input := dnd.ItemEventList{Events: make([]dnd.ItemEvent, count)}
for index := range input.Events {
doc.Units[index].ID = index + 1
input.Events[index] = dnd.ItemEvent{Name: " Coin ", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: count - index, EndUnitID: count - index}}}
}
result := normalize(t, doc, input)
if len(result.Warnings) != diagnostics.MaxWarnings || result.Warnings[len(result.Warnings)-1].ReasonCode != ReasonCodeWarningsOmitted {
t.Fatalf("warnings = %#v", result.Warnings)
}
}
func normalize(t *testing.T, doc *source.SourceDocument, input dnd.ItemEventList) contracts.TypedNormalizeResult[dnd.ItemEventList] {
t.Helper()
result, err := New(Options{}).Normalize(context.Background(), contracts.TypedNormalizeRequest[dnd.ItemEventList]{Source: doc, MergeOutput: contracts.MergeArtifact[dnd.ItemEventList]{Value: input}})
if err != nil {
t.Fatalf("Normalize() error = %v", err)
}
return result
}
func testDocument() *source.SourceDocument {
return &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 10}, {ID: 20}, {ID: 30}}}
}
func hasWarning(warnings []contracts.Warning, reason string) bool {
for _, warning := range warnings {
if warning.ReasonCode == reason {
return true
}
}
return false
}

View File

@@ -3,11 +3,13 @@ package register
import (
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
combatextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
itemeventextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/itemevents"
interactionextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcinteractions"
npcextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcs"
scenedescriptionextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/scenedescriptions"
spellextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
combatnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/combatturns"
itemeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/itemevents"
interactionnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/npcinteractions"
npcnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/npcs"
scenedescriptionnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/scenedescriptions"
@@ -16,6 +18,10 @@ import (
combatshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/combatturns/shape"
combatsourcerefs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/combatturns/source_refs"
combatrelatedness "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/combatturns/source_relatedness"
itemeventinvariants "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/invariants"
itemeventshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/shape"
itemeventrefs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/source_refs"
itemeventrelatedness "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/source_relatedness"
interactioninvariants "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/npcinteractions/invariants"
interactionregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/npcinteractions/registry"
interactionshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/npcinteractions/shape"
@@ -121,6 +127,33 @@ func registerDefaultChains(registry *pipeline.ValidatorChainRegistry) error {
},
})
}},
{name: "item events validator chain", register: func() error {
return registry.Register(pipeline.ValidatorChainMapping{
Stage: pipeline.StageExtract,
Module: itemeventextract.Key,
Validators: []pipeline.ModuleBinding{
pipeline.Binding(validjson.Key),
pipeline.Binding(itemeventshape.Key),
pipeline.Binding(itemeventrefs.Key),
pipeline.Binding(validjsonschema.Key),
pipeline.Binding(itemeventrelatedness.Key),
},
})
}},
{name: "item events normalize validator chain", register: func() error {
return registry.Register(pipeline.ValidatorChainMapping{
Stage: pipeline.StageNormalize,
Module: itemeventnormalize.Key,
Validators: []pipeline.ModuleBinding{
pipeline.Binding(validjson.Key),
pipeline.Binding(itemeventshape.Key),
pipeline.Binding(itemeventinvariants.Key),
pipeline.Binding(itemeventrefs.Key),
pipeline.Binding(validjsonschema.Key),
pipeline.Binding(itemeventrelatedness.Key),
},
})
}},
{name: "npc interactions validator chain", register: func() error {
return registry.Register(pipeline.ValidatorChainMapping{
Stage: pipeline.StageExtract,

View File

@@ -70,6 +70,27 @@ func appendCombatTurnLists(values []dnd.CombatTurnList) (dnd.CombatTurnList, err
return combined, nil
}
func appendItemEventLists(values []dnd.ItemEventList) (dnd.ItemEventList, error) {
count := 0
present := false
for _, value := range values {
if value.Events != nil {
present = true
}
count += len(value.Events)
}
if !present {
return dnd.ItemEventList{}, nil
}
combined := dnd.ItemEventList{Events: make([]dnd.ItemEvent, 0, count)}
for _, value := range values {
for _, event := range value.Events {
combined.Events = append(combined.Events, cloneItemEvent(event))
}
}
return combined, nil
}
func appendNPCInteractionLists(values []dnd.NPCInteractionList) (dnd.NPCInteractionList, error) {
count := 0
present := false
@@ -116,6 +137,16 @@ func cloneCombatTurn(value dnd.CombatTurn) dnd.CombatTurn {
return clone
}
func cloneItemEvent(value dnd.ItemEvent) dnd.ItemEvent {
clone := value
if value.Quantity != nil {
quantity := *value.Quantity
clone.Quantity = &quantity
}
clone.SourceRefs = cloneSourceRefs(value.SourceRefs)
return clone
}
func cloneSpellCast(value dnd.SpellCast) dnd.SpellCast {
clone := value
clone.SourceRefs = cloneSourceRefs(value.SourceRefs)

View File

@@ -6,16 +6,19 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/chunk/scenes"
combatcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/combatturns"
itemeventcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/itemevents"
interactioncodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcinteractions"
npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs"
scenedescriptioncodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
spellcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/spells"
combatextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
itemeventextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/itemevents"
interactionextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcinteractions"
npcextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcs"
scenedescriptionextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/scenedescriptions"
spellextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
combatnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/combatturns"
itemeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/itemevents"
interactionnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/npcinteractions"
npcnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/npcs"
scenedescriptionnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/scenedescriptions"
@@ -30,6 +33,7 @@ func registerModules(registries pipeline.Registries) error {
{name: "spells codec", register: func() error { return pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, codec) }},
{name: "npcs codec", register: func() error { return pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, npccodec.New()) }},
{name: "combat turns codec", register: func() error { return pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, combatcodec.New()) }},
{name: "item events codec", register: func() error { return pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, itemeventcodec.New()) }},
{name: "npc interactions codec", register: func() error { return pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, interactioncodec.New()) }},
{name: "scene descriptions codec", register: func() error {
return pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, scenedescriptioncodec.New())
@@ -38,6 +42,7 @@ func registerModules(registries pipeline.Registries) error {
{name: "spells extractor", register: func() error { return spellextract.Register(registries.Extractors) }},
{name: "npcs extractor", register: func() error { return npcextract.Register(registries.Extractors) }},
{name: "combat turns extractor", register: func() error { return combatextract.Register(registries.Extractors) }},
{name: "item events extractor", register: func() error { return itemeventextract.Register(registries.Extractors) }},
{name: "npc interactions extractor", register: func() error { return interactionextract.Register(registries.Extractors) }},
{name: "scene descriptions extractor", register: func() error { return scenedescriptionextract.Register(registries.Extractors) }},
{name: "spell-list appendorder merger", register: func() error {
@@ -49,6 +54,9 @@ func registerModules(registries pipeline.Registries) error {
{name: "combat-turn-list appendorder merger", register: func() error {
return appendorder.RegisterTyped(registries.Mergers, dnd.CombatTurnListKind, appendCombatTurnLists)
}},
{name: "item-event-list appendorder merger", register: func() error {
return appendorder.RegisterTyped(registries.Mergers, dnd.ItemEventListKind, appendItemEventLists)
}},
{name: "npc-interaction-list appendorder merger", register: func() error {
return appendorder.RegisterTyped(registries.Mergers, dnd.NPCInteractionListKind, appendNPCInteractionLists)
}},
@@ -58,6 +66,7 @@ func registerModules(registries pipeline.Registries) error {
{name: "spells normalizer", register: func() error { return spellnormalize.Register(registries.Normalizers) }},
{name: "npcs normalizer", register: func() error { return npcnormalize.Register(registries.Normalizers) }},
{name: "combat turns normalizer", register: func() error { return combatnormalize.Register(registries.Normalizers) }},
{name: "item events normalizer", register: func() error { return itemeventnormalize.Register(registries.Normalizers) }},
{name: "npc interactions normalizer", register: func() error { return interactionnormalize.Register(registries.Normalizers) }},
{name: "scene descriptions normalizer", register: func() error { return scenedescriptionnormalize.Register(registries.Normalizers) }},
{name: "spell-list noop normalizer", register: func() error {
@@ -69,6 +78,9 @@ func registerModules(registries pipeline.Registries) error {
{name: "combat-turn-list noop normalizer", register: func() error {
return noop.RegisterTyped[dnd.CombatTurnList](registries.Normalizers, dnd.CombatTurnListKind)
}},
{name: "item-event-list noop normalizer", register: func() error {
return noop.RegisterTyped[dnd.ItemEventList](registries.Normalizers, dnd.ItemEventListKind)
}},
{name: "npc-interaction-list noop normalizer", register: func() error {
return noop.RegisterTyped[dnd.NPCInteractionList](registries.Normalizers, dnd.NPCInteractionListKind)
}},
@@ -84,6 +96,7 @@ func registerPromptAssets(assets *llm.AssetRegistry) error {
{name: "spells prompt assets", register: func() error { return spellextract.RegisterPromptAssets(assets) }},
{name: "npcs prompt assets", register: func() error { return npcextract.RegisterPromptAssets(assets) }},
{name: "combat turns prompt assets", register: func() error { return combatextract.RegisterPromptAssets(assets) }},
{name: "item events prompt assets", register: func() error { return itemeventextract.RegisterPromptAssets(assets) }},
{name: "npc interactions prompt assets", register: func() error { return interactionextract.RegisterPromptAssets(assets) }},
{name: "scene descriptions prompt assets", register: func() error { return scenedescriptionextract.RegisterPromptAssets(assets) }},
})

View File

@@ -13,11 +13,13 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
combatextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
itemeventextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/itemevents"
interactionextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcinteractions"
npcextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcs"
scenedescriptionextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/scenedescriptions"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
combatnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/combatturns"
itemeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/itemevents"
interactionnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/npcinteractions"
npcnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/npcs"
scenedescriptionnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/scenedescriptions"
@@ -31,13 +33,14 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
t.Fatalf("Register() error = %v, want nil", err)
}
assertContainsKeys(t, "chunkers", registries.Chunkers.RegisteredKeys(), []string{"dnd/scenes"})
assertContainsKeys(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells", npcextract.Key, combatextract.Key, interactionextract.Key, scenedescriptionextract.Key})
assertContainsKeys(t, "normalizers", registries.Normalizers.RegisteredKeys(), []string{spellnormalize.Key, npcnormalize.Key, combatnormalize.Key, interactionnormalize.Key, scenedescriptionnormalize.Key, pipeline.DefaultNormalizeModule})
assertContainsArtifactKinds(t, registries.ArtifactCodecs.RegisteredKinds(), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.NPCInteractionListKind, dnd.SceneDescriptionListKind})
assertContainsArtifactKinds(t, registries.Mergers.RegisteredArtifactKinds(pipeline.DefaultMergeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.NPCInteractionListKind, dnd.SceneDescriptionListKind})
assertContainsArtifactKinds(t, registries.Normalizers.RegisteredArtifactKinds(pipeline.DefaultNormalizeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.NPCInteractionListKind, dnd.SceneDescriptionListKind})
assertContainsKeys(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells", npcextract.Key, combatextract.Key, itemeventextract.Key, interactionextract.Key, scenedescriptionextract.Key})
assertContainsKeys(t, "normalizers", registries.Normalizers.RegisteredKeys(), []string{spellnormalize.Key, npcnormalize.Key, combatnormalize.Key, itemeventnormalize.Key, interactionnormalize.Key, scenedescriptionnormalize.Key, pipeline.DefaultNormalizeModule})
assertContainsArtifactKinds(t, registries.ArtifactCodecs.RegisteredKinds(), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind, dnd.NPCInteractionListKind, dnd.SceneDescriptionListKind})
assertContainsArtifactKinds(t, registries.Mergers.RegisteredArtifactKinds(pipeline.DefaultMergeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind, dnd.NPCInteractionListKind, dnd.SceneDescriptionListKind})
assertContainsArtifactKinds(t, registries.Normalizers.RegisteredArtifactKinds(pipeline.DefaultNormalizeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind, dnd.NPCInteractionListKind, dnd.SceneDescriptionListKind})
assertContainsArtifactKinds(t, registries.Normalizers.RegisteredArtifactKinds(npcnormalize.Key), []contracts.ArtifactKind{dnd.NPCListKind})
assertContainsArtifactKinds(t, registries.Normalizers.RegisteredArtifactKinds(combatnormalize.Key), []contracts.ArtifactKind{dnd.CombatTurnListKind})
assertContainsArtifactKinds(t, registries.Normalizers.RegisteredArtifactKinds(itemeventnormalize.Key), []contracts.ArtifactKind{dnd.ItemEventListKind})
assertContainsArtifactKinds(t, registries.Normalizers.RegisteredArtifactKinds(interactionnormalize.Key), []contracts.ArtifactKind{dnd.NPCInteractionListKind})
assertContainsArtifactKinds(t, registries.Normalizers.RegisteredArtifactKinds(scenedescriptionnormalize.Key), []contracts.ArtifactKind{dnd.SceneDescriptionListKind})
assertContainsKeys(t, "validators", registries.Validators.RegisteredKeys(), []string{
@@ -53,6 +56,10 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
"extract/dnd/combat-turns/source_refs",
"extract/dnd/combat-turns/source_relatedness",
"normalize/dnd/combat-turns/invariants",
"extract/dnd/item-events/shape",
"extract/dnd/item-events/source_refs",
"extract/dnd/item-events/source_relatedness",
"normalize/dnd/item-events/invariants",
"extract/dnd/npc-interactions/shape",
"extract/dnd/npc-interactions/registry",
"extract/dnd/npc-interactions/source_refs",
@@ -121,6 +128,27 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
if got := registries.ValidatorChains.Validators(pipeline.StageNormalize, combatnormalize.Key); !reflect.DeepEqual(got, combatNormalizeChain) {
t.Fatalf("combat normalize validator chain = %#v, want %#v", got, combatNormalizeChain)
}
itemEventExtractChain := []pipeline.ModuleBinding{
pipeline.Binding("generic/valid_json"),
pipeline.Binding("extract/dnd/item-events/shape"),
pipeline.Binding("extract/dnd/item-events/source_refs"),
pipeline.Binding("generic/valid_json_schema"),
pipeline.Binding("extract/dnd/item-events/source_relatedness"),
}
itemEventNormalizeChain := []pipeline.ModuleBinding{
pipeline.Binding("generic/valid_json"),
pipeline.Binding("extract/dnd/item-events/shape"),
pipeline.Binding("normalize/dnd/item-events/invariants"),
pipeline.Binding("extract/dnd/item-events/source_refs"),
pipeline.Binding("generic/valid_json_schema"),
pipeline.Binding("extract/dnd/item-events/source_relatedness"),
}
if got := registries.ValidatorChains.Validators(pipeline.StageExtract, itemeventextract.Key); !reflect.DeepEqual(got, itemEventExtractChain) {
t.Fatalf("item event extract validator chain = %#v, want %#v", got, itemEventExtractChain)
}
if got := registries.ValidatorChains.Validators(pipeline.StageNormalize, itemeventnormalize.Key); !reflect.DeepEqual(got, itemEventNormalizeChain) {
t.Fatalf("item event normalize validator chain = %#v, want %#v", got, itemEventNormalizeChain)
}
interactionExtractChain := []pipeline.ModuleBinding{
pipeline.Binding("generic/valid_json"),
pipeline.Binding("extract/dnd/npc-interactions/shape"),
@@ -196,6 +224,14 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
"dnd.combat_turns/sharedassets/common-dnd-system.md",
"dnd.combat_turns/sharedassets/common-dnd-transcript.md",
"dnd.combat_turns/task.md",
"dnd.item_events/dnd.item_events.yaml",
"dnd.item_events/instructions.md",
"dnd.item_events/sharedassets/common-dnd-extraction-evidence.md",
"dnd.item_events/sharedassets/common-dnd-identity.md",
"dnd.item_events/sharedassets/common-dnd-references.md",
"dnd.item_events/sharedassets/common-dnd-system.md",
"dnd.item_events/sharedassets/common-dnd-transcript.md",
"dnd.item_events/task.md",
"dnd.npc_interactions/dnd.npc_interactions.yaml",
"dnd.npc_interactions/instructions.md",
"dnd.npc_interactions/sharedassets/common-dnd-npcs.md",
@@ -213,6 +249,7 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
"dnd_spells_llm.v1.json",
"dnd_npcs_llm.v1.json",
"dnd_combat_turns_llm.v1.json",
"dnd_item_events_llm.v1.json",
"dnd_npc_interactions_llm.v1.json",
"dnd_scene_descriptions_llm.v1.json",
})
@@ -237,6 +274,16 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
if spec, ok := registries.Normalizers.Spec(combatnormalize.Key); !ok || spec.ArtifactKind != dnd.CombatTurnListKind || spec.Stage != pipeline.StageNormalize {
t.Fatalf("combat normalizer spec = %#v, present = %t; want dnd combat-turn-list artifact", spec, ok)
}
itemEventExtractSpec, itemEventExtractOK := registries.Extractors.Spec(itemeventextract.Key)
itemEventNormalizeSpec, itemEventNormalizeOK := registries.Normalizers.Spec(itemeventnormalize.Key)
if !itemEventExtractOK || itemEventExtractSpec.ArtifactKind != dnd.ItemEventListKind || !itemEventNormalizeOK || itemEventNormalizeSpec.ArtifactKind != dnd.ItemEventListKind || itemEventNormalizeSpec.Stage != pipeline.StageNormalize || len(itemEventNormalizeSpec.ReferenceSlots) != 0 {
t.Fatalf("item event specs = %#v / %#v, present = %t / %t", itemEventExtractSpec, itemEventNormalizeSpec, itemEventExtractOK, itemEventNormalizeOK)
}
for _, slot := range itemEventExtractSpec.ReferenceSlots {
if slot.Required || len(slot.AcceptedArtifactKinds) != 0 {
t.Fatalf("item event extractor has a generated-reference dependency: %#v", slot)
}
}
interactionExtractSpec, extractOK := registries.Extractors.Spec(interactionextract.Key)
interactionNormalizeSpec, normalizeOK := registries.Normalizers.Spec(interactionnormalize.Key)
if !extractOK || interactionExtractSpec.ArtifactKind != dnd.NPCInteractionListKind || !normalizeOK || interactionNormalizeSpec.ArtifactKind != dnd.NPCInteractionListKind || interactionNormalizeSpec.Stage != pipeline.StageNormalize {
@@ -409,6 +456,32 @@ func TestAppendCombatTurnListsPreservesOrderPresenceAndOwnership(t *testing.T) {
}
}
func TestAppendItemEventListsPreservesOrderPresenceAndOwnership(t *testing.T) {
quantity := 3
refs := []source.SourceRef{{SourceID: "session", StartUnitID: 1, EndUnitID: 1}}
input := []dnd.ItemEventList{
{},
{Events: []dnd.ItemEvent{}},
{Events: []dnd.ItemEvent{{Name: "first", Kind: dnd.ItemEventKindDiscovered, Quantity: &quantity, SourceRefs: refs}}},
{Events: []dnd.ItemEvent{{Name: "second", Kind: dnd.ItemEventKindAcquired, To: "party", SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 2, EndUnitID: 2}}}}},
}
got, err := appendItemEventLists(input)
if err != nil || got.Events == nil || !reflect.DeepEqual([]string{got.Events[0].Name, got.Events[1].Name}, []string{"first", "second"}) {
t.Fatalf("appendItemEventLists() = %#v, error = %v", got, err)
}
*got.Events[0].Quantity = 99
got.Events[0].SourceRefs[0].StartUnitID = 999
if quantity != 3 || input[2].Events[0].SourceRefs[0].StartUnitID != 1 {
t.Fatal("merged item events share quantity or source-reference storage")
}
for _, values := range [][]dnd.ItemEventList{nil, []dnd.ItemEventList{{}, {}}} {
result, err := appendItemEventLists(values)
if err != nil || result.Events != nil {
t.Fatalf("nil-only merge = %#v, %v; want nil events", result, err)
}
}
}
func TestAppendListsPreserveNestedSourceReferencePresence(t *testing.T) {
spells, err := appendSpellLists([]dnd.SpellList{{SpellCasts: []dnd.SpellCast{{SourceRefs: []source.SourceRef{}}}}})
if err != nil || spells.SpellCasts[0].SourceRefs == nil {
@@ -429,6 +502,11 @@ func TestAppendListsPreserveNestedSourceReferencePresence(t *testing.T) {
if err != nil || interactions.Interactions[0].SourceRefs == nil {
t.Fatalf("appendNPCInteractionLists() = %#v, %v; want present-empty source refs", interactions, err)
}
events, err := appendItemEventLists([]dnd.ItemEventList{{Events: []dnd.ItemEvent{{SourceRefs: []source.SourceRef{}}}}})
if err != nil || events.Events[0].SourceRefs == nil {
t.Fatalf("appendItemEventLists() = %#v, %v; want present-empty source refs", events, err)
}
}
func TestRegisterRejectsMissingDNDDependenciesBeforeMutation(t *testing.T) {

View File

@@ -7,6 +7,10 @@ import (
combatshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/combatturns/shape"
combatsourcerefs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/combatturns/source_refs"
combatrelatedness "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/combatturns/source_relatedness"
itemeventinvariants "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/invariants"
itemeventshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/shape"
itemeventrefs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/source_refs"
itemeventrelatedness "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/source_relatedness"
interactioninvariants "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/npcinteractions/invariants"
interactionregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/npcinteractions/registry"
interactionshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/npcinteractions/shape"
@@ -42,6 +46,10 @@ func registerValidators(registries pipeline.Registries) error {
{name: "combat source references validator", register: func() error { return combatsourcerefs.Register(registries.Validators) }},
{name: "combat source relatedness validator", register: func() error { return combatrelatedness.Register(registries.Validators) }},
{name: "combat normalized invariants validator", register: func() error { return combatinvariants.Register(registries.Validators) }},
{name: "item event shape validator", register: func() error { return itemeventshape.Register(registries.Validators) }},
{name: "item event source references validator", register: func() error { return itemeventrefs.Register(registries.Validators) }},
{name: "item event source relatedness validator", register: func() error { return itemeventrelatedness.Register(registries.Validators) }},
{name: "item event normalized invariants validator", register: func() error { return itemeventinvariants.Register(registries.Validators) }},
{name: "npc interaction shape validator", register: func() error { return interactionshape.Register(registries.Validators) }},
{name: "npc interaction registry validator", register: func() error { return interactionregistry.Register(registries.Validators) }},
{name: "npc interaction source references validator", register: func() error { return interactionrefs.Register(registries.Validators) }},
@@ -69,6 +77,12 @@ func registerValidators(registries pipeline.Registries) error {
{name: "combat-turn-list always reject validator", register: func() error {
return alwaysreject.RegisterTyped[dnd.CombatTurnList](registries.Validators, dnd.CombatTurnListKind)
}},
{name: "item-event-list always accept validator", register: func() error {
return alwaysaccept.RegisterTyped[dnd.ItemEventList](registries.Validators, dnd.ItemEventListKind)
}},
{name: "item-event-list always reject validator", register: func() error {
return alwaysreject.RegisterTyped[dnd.ItemEventList](registries.Validators, dnd.ItemEventListKind)
}},
{name: "npc-interaction-list always accept validator", register: func() error {
return alwaysaccept.RegisterTyped[dnd.NPCInteractionList](registries.Validators, dnd.NPCInteractionListKind)
}},

View File

@@ -0,0 +1,291 @@
// Package registry resolves scene-description artifacts into immutable combat
// eligibility data.
package registry
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"mime"
"sort"
"strings"
"sync"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
scenecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
)
const (
ReferenceSlot = "scene_descriptions"
MaxBytes = 1048576
emptyProjection = `{"scenes":[]}`
)
// MatchState identifies how a chunk relates to the resolved scene records.
type MatchState string
const (
MatchExact MatchState = "exact"
MatchMissing MatchState = "missing"
MatchMismatched MatchState = "mismatched"
)
// ChunkMatch describes whether a scene record exactly covers a chunk. Kind is
// populated only for an exact match.
type ChunkMatch struct {
State MatchState
Kind dnd.SceneKind
}
type sceneEligibility struct {
sourceID string
startUnitID int
endUnitID int
kind dnd.SceneKind
}
// Registry is an immutable, validated eligibility view. It retains no scene
// prose, raw reference bytes, or reference provenance.
type Registry struct {
bound bool
digest string
scenesByID map[string]sceneEligibility
}
// Resolver holds an immutable construction-time view and memoizes immutable
// operation-time views by raw reference identity and eligibility digest.
type Resolver struct {
seeded *Registry
mu sync.Mutex
cache map[string]*Registry
rawCache map[string]*Registry
}
// NewResolver validates a materialized construction-time scene reference. An
// empty slot is permitted because a generated reference is supplied only at
// operation time.
func NewResolver(references contracts.ReferenceSet) (*Resolver, error) {
seeded, err := Resolve(constructionReferences(references))
if err != nil {
return nil, err
}
return &Resolver{
seeded: seeded,
cache: make(map[string]*Registry),
rawCache: make(map[string]*Registry),
}, nil
}
func constructionReferences(references contracts.ReferenceSet) contracts.ReferenceSet {
slot, ok := references.Slots[ReferenceSlot]
if !ok || len(slot.Items) > 0 {
return references
}
cloned := contracts.ReferenceSet{Slots: make(map[string]contracts.ResolvedReferenceSlot, len(references.Slots))}
for name, value := range references.Slots {
cloned.Slots[name] = value
}
delete(cloned.Slots, ReferenceSlot)
return cloned
}
// Seeded returns the construction-time eligibility view.
func (r *Resolver) Seeded() *Registry {
if r == nil {
return nil
}
return r.seeded
}
// Resolve returns the generated operation-time view when the scene slot is
// present, otherwise it returns the construction-time view.
func (r *Resolver) Resolve(references contracts.ReferenceSet) (*Registry, error) {
if r == nil {
return Resolve(references)
}
if _, ok := references.Slots[ReferenceSlot]; !ok {
return r.seeded, nil
}
slot := references.Slots[ReferenceSlot]
rawKey := ""
if len(slot.Items) == 1 {
rawKey = rawReferenceKey(slot.Items[0])
}
r.mu.Lock()
defer r.mu.Unlock()
if rawKey != "" {
if cached, ok := r.rawCache[rawKey]; ok {
return cached, nil
}
}
resolved, err := Resolve(references)
if err != nil {
return nil, err
}
if sameEligibility(r.seeded, resolved) {
if rawKey != "" {
r.rawCache[rawKey] = r.seeded
}
return r.seeded, nil
}
if cached, ok := r.cache[resolved.EligibilityDigest()]; ok {
if rawKey != "" {
r.rawCache[rawKey] = cached
}
return cached, nil
}
r.cache[resolved.EligibilityDigest()] = resolved
if rawKey != "" {
r.rawCache[rawKey] = resolved
}
return resolved, nil
}
func rawReferenceKey(item contracts.ReferenceItem) string {
digest := strings.ToLower(strings.TrimSpace(item.Digest))
if !validSHA256Digest(digest) {
digest = semanticDigest(item.Content)
}
return strings.ToLower(strings.TrimSpace(item.MediaType)) + "\x00" + digest
}
func validSHA256Digest(value string) bool {
if len(value) != len("sha256:")+sha256.Size*2 || !strings.HasPrefix(value, "sha256:") {
return false
}
decoded, err := hex.DecodeString(strings.TrimPrefix(value, "sha256:"))
return err == nil && len(decoded) == sha256.Size
}
func sameEligibility(first, second *Registry) bool {
if first == nil || second == nil {
return first == second
}
return first.bound == second.bound && first.digest == second.digest
}
// Resolve validates one optional scene-description reference into an
// eligibility-only view. An absent slot is represented by the canonical empty
// projection so required generated bindings have a stable sentinel identity.
func Resolve(references contracts.ReferenceSet) (*Registry, error) {
slot, ok := references.Slots[ReferenceSlot]
if !ok {
return &Registry{
digest: semanticDigest([]byte(emptyProjection)),
scenesByID: map[string]sceneEligibility{},
}, nil
}
if len(slot.Items) != 1 {
return nil, fmt.Errorf("reference slot %q must contain exactly one item", ReferenceSlot)
}
item := slot.Items[0]
mediaType, _, err := mime.ParseMediaType(item.MediaType)
if err != nil {
return nil, fmt.Errorf("reference slot %q item media type is invalid", ReferenceSlot)
}
if !strings.EqualFold(mediaType, scenecodec.MediaType) {
return nil, fmt.Errorf("reference slot %q item media type must be %s", ReferenceSlot, scenecodec.MediaType)
}
if len(item.Content) > MaxBytes {
return nil, fmt.Errorf("reference slot %q item is %d bytes, limit %d", ReferenceSlot, len(item.Content), MaxBytes)
}
value, err := scenecodec.New().Decode(item.Content)
if err != nil {
return nil, fmt.Errorf("decode scene eligibility: invalid approved scene JSON")
}
scenesByID := make(map[string]sceneEligibility, len(value.Scenes))
projection := make([]projectedScene, 0, len(value.Scenes))
for _, scene := range value.Scenes {
if _, exists := scenesByID[scene.ID]; exists {
return nil, fmt.Errorf("scene eligibility contains duplicate scene ID %q", scene.ID)
}
eligibility := sceneEligibility{
sourceID: scene.SourceRef.SourceID,
startUnitID: scene.SourceRef.StartUnitID,
endUnitID: scene.SourceRef.EndUnitID,
kind: scene.Kind,
}
scenesByID[scene.ID] = eligibility
projection = append(projection, projectedScene{
ID: scene.ID,
SourceRef: source.SourceRef{SourceID: eligibility.sourceID, StartUnitID: eligibility.startUnitID, EndUnitID: eligibility.endUnitID},
Kind: eligibility.kind,
})
}
content, err := eligibilityProjection(projection)
if err != nil {
return nil, fmt.Errorf("encode scene eligibility projection: %w", err)
}
return &Registry{
bound: true,
digest: semanticDigest(content),
scenesByID: scenesByID,
}, nil
}
// Bound reports whether an approved scene-description reference was supplied.
func (r *Registry) Bound() bool { return r != nil && r.bound }
// Count reports the number of scene records in the eligibility view.
func (r *Registry) Count() int {
if r == nil {
return 0
}
return len(r.scenesByID)
}
// EligibilityDigest returns the semantic SHA-256 digest of the canonical
// eligibility projection, including the canonical unbound projection.
func (r *Registry) EligibilityDigest() string {
if r == nil {
return ""
}
return r.digest
}
// Match classifies a chunk against its scene record. The kind is intentionally
// unavailable unless the ID and complete source range match exactly.
func (r *Registry) Match(chunk *source.Chunk) ChunkMatch {
if r == nil || chunk == nil {
return ChunkMatch{State: MatchMissing}
}
scene, ok := r.scenesByID[chunk.ID]
if !ok {
return ChunkMatch{State: MatchMissing}
}
if scene.sourceID != chunk.Ref.SourceID || scene.startUnitID != chunk.Ref.StartUnitID || scene.endUnitID != chunk.Ref.EndUnitID {
return ChunkMatch{State: MatchMismatched}
}
return ChunkMatch{State: MatchExact, Kind: scene.kind}
}
type projectedScene struct {
ID string `json:"id"`
SourceRef source.SourceRef `json:"source_ref"`
Kind dnd.SceneKind `json:"kind"`
}
type projectedSceneList struct {
Scenes []projectedScene `json:"scenes"`
}
func eligibilityProjection(scenes []projectedScene) ([]byte, error) {
sort.Slice(scenes, func(i, j int) bool { return scenes[i].ID < scenes[j].ID })
return json.Marshal(projectedSceneList{Scenes: scenes})
}
func semanticDigest(content []byte) string {
sum := sha256.Sum256(content)
return "sha256:" + hex.EncodeToString(sum[:])
}

View File

@@ -0,0 +1,270 @@
package registry
import (
"fmt"
"sync"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
scenecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
)
func TestResolveClassifiesExactMissingAndMismatchedChunks(t *testing.T) {
registry := resolveList(t, sceneList(
scene("chunk-combat", "session-alpha", 1, 2, dnd.SceneKindCombat, "Combat title", "Combat summary"),
))
for _, test := range []struct {
name string
chunk *source.Chunk
want ChunkMatch
}{
{"exact", chunk("chunk-combat", "session-alpha", 1, 2), ChunkMatch{State: MatchExact, Kind: dnd.SceneKindCombat}},
{"missing ID", chunk("chunk-missing", "session-alpha", 1, 2), ChunkMatch{State: MatchMissing}},
{"source differs", chunk("chunk-combat", "other-source", 1, 2), ChunkMatch{State: MatchMismatched}},
{"start differs", chunk("chunk-combat", "session-alpha", 2, 2), ChunkMatch{State: MatchMismatched}},
{"end differs", chunk("chunk-combat", "session-alpha", 1, 3), ChunkMatch{State: MatchMismatched}},
} {
t.Run(test.name, func(t *testing.T) {
if got := registry.Match(test.chunk); got != test.want {
t.Fatalf("Match() = %#v, want %#v", got, test.want)
}
})
}
}
func TestResolveRejectsInvalidSceneReferences(t *testing.T) {
valid := encodeList(t, sceneList(scene("chunk-1", "session-alpha", 1, 2, dnd.SceneKindNarrative, "Arrival", "The party arrives.")))
oversized := make([]byte, MaxBytes+1)
for _, test := range []struct {
name string
set contracts.ReferenceSet
}{
{"multiple items", referenceSet(referenceItem(valid), referenceItem(valid))},
{"invalid media type", referenceSet(contracts.ReferenceItem{SlotName: ReferenceSlot, MediaType: "text/plain", Content: valid})},
{"invalid media type syntax", referenceSet(contracts.ReferenceItem{SlotName: ReferenceSlot, MediaType: "not a media type", Content: valid})},
{"oversized", referenceSet(contracts.ReferenceItem{SlotName: ReferenceSlot, MediaType: scenecodec.MediaType, Content: oversized})},
{"invalid approved JSON", referenceSet(contracts.ReferenceItem{SlotName: ReferenceSlot, MediaType: scenecodec.MediaType, Content: []byte(`{"scenes":[]}`)})},
{"duplicate IDs", referenceSet(referenceItem(encodeList(t, sceneList(
scene("chunk-1", "session-alpha", 1, 1, dnd.SceneKindNarrative, "First", "First scene."),
scene("chunk-1", "session-alpha", 2, 2, dnd.SceneKindCombat, "Second", "Second scene."),
))))},
} {
t.Run(test.name, func(t *testing.T) {
if _, err := Resolve(test.set); err == nil {
t.Fatal("Resolve() error = nil")
}
})
}
}
func TestEligibilityDigestTracksOnlyGatingFields(t *testing.T) {
base := sceneList(
scene("chunk-a", "session-alpha", 1, 2, dnd.SceneKindNarrative, "Arrival", "The party arrives."),
scene("chunk-b", "session-alpha", 3, 4, dnd.SceneKindCombat, "Combat", "Combat begins."),
)
baseDigest := resolveList(t, base).EligibilityDigest()
equivalent := sceneList(
scene("chunk-b", "session-alpha", 3, 4, dnd.SceneKindCombat, "Different title", "Different summary."),
scene("chunk-a", "session-alpha", 1, 2, dnd.SceneKindNarrative, "Another title", "Another summary."),
)
if got := resolveList(t, equivalent).EligibilityDigest(); got != baseDigest {
t.Fatalf("equivalent eligibility digest = %q, want %q", got, baseDigest)
}
changes := []struct {
name string
change func(*dnd.SceneDescription)
}{
{"ID", func(value *dnd.SceneDescription) { value.ID = "chunk-other" }},
{"source ID", func(value *dnd.SceneDescription) { value.SourceRef.SourceID = "session-other" }},
{"start unit ID", func(value *dnd.SceneDescription) { value.SourceRef.StartUnitID = 2 }},
{"end unit ID", func(value *dnd.SceneDescription) { value.SourceRef.EndUnitID = 3 }},
{"kind", func(value *dnd.SceneDescription) { value.Kind = dnd.SceneKindRecap }},
}
for _, test := range changes {
t.Run(test.name, func(t *testing.T) {
changed := cloneList(base)
test.change(&changed.Scenes[0])
if got := resolveList(t, changed).EligibilityDigest(); got == baseDigest {
t.Fatal("eligibility digest did not change")
}
})
}
}
func TestResolverUsesGeneratedReferenceWhenPresent(t *testing.T) {
external := sceneList(scene("chunk-1", "session-alpha", 1, 2, dnd.SceneKindNarrative, "External", "External scene."))
generated := sceneList(scene("chunk-1", "session-alpha", 1, 2, dnd.SceneKindCombat, "Generated", "Generated scene."))
resolver, err := NewResolver(referenceSet(referenceItem(encodeList(t, external))))
if err != nil {
t.Fatal(err)
}
if seeded := resolver.Seeded(); !seeded.Bound() || seeded.Count() != 1 || seeded.Match(chunk("chunk-1", "session-alpha", 1, 2)).Kind != dnd.SceneKindNarrative {
t.Fatalf("Seeded() = %#v", seeded)
}
if got, err := resolver.Resolve(contracts.ReferenceSet{}); err != nil || got != resolver.Seeded() {
t.Fatalf("Resolve() = %p, %v; want seeded %p", got, err, resolver.Seeded())
}
resolved, err := resolver.Resolve(referenceSet(referenceItem(encodeList(t, generated))))
if err != nil {
t.Fatal(err)
}
if got := resolved.Match(chunk("chunk-1", "session-alpha", 1, 2)); got != (ChunkMatch{State: MatchExact, Kind: dnd.SceneKindCombat}) {
t.Fatalf("generated Match() = %#v", got)
}
if resolved == resolver.Seeded() {
t.Fatal("generated reference reused a different seeded eligibility view")
}
unbound, err := NewResolver(contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{ReferenceSlot: {}}})
if err != nil {
t.Fatal(err)
}
if seeded := unbound.Seeded(); seeded.Bound() || seeded.Count() != 0 || seeded.EligibilityDigest() != semanticDigest([]byte(emptyProjection)) {
t.Fatalf("unbound Seeded() = %#v", seeded)
}
}
func TestResolverReusesRawAndSemanticallyEquivalentViews(t *testing.T) {
baseContent := encodeList(t, sceneList(
scene("chunk-1", "session-alpha", 1, 2, dnd.SceneKindCombat, "Original title", "Original summary."),
))
baseItem := referenceItem(baseContent)
baseItem.Digest = semanticDigest(baseContent)
resolver, err := NewResolver(contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{ReferenceSlot: {}}})
if err != nil {
t.Fatal(err)
}
first, err := resolver.Resolve(referenceSet(baseItem))
if err != nil {
t.Fatal(err)
}
second, err := resolver.Resolve(referenceSet(baseItem))
if err != nil {
t.Fatal(err)
}
if second != first {
t.Fatal("identical framework reference did not reuse its immutable view")
}
equivalentContent := encodeList(t, sceneList(
scene("chunk-1", "session-alpha", 1, 2, dnd.SceneKindCombat, "Revised title", "Revised summary."),
))
equivalentItem := referenceItem(equivalentContent)
equivalentItem.Digest = semanticDigest(equivalentContent)
equivalent, err := resolver.Resolve(referenceSet(equivalentItem))
if err != nil {
t.Fatal(err)
}
if equivalent != first {
t.Fatal("prose-only change did not reuse the semantic eligibility view")
}
for _, digest := range []string{"", "not-a-sha256-digest"} {
fallbackItem := referenceItem(baseContent)
fallbackItem.Digest = digest
fallback, resolveErr := resolver.Resolve(referenceSet(fallbackItem))
if resolveErr != nil {
t.Fatal(resolveErr)
}
if fallback != first {
t.Fatalf("digest fallback %q did not reuse the content-addressed view", digest)
}
}
}
func TestRegistryIsImmutableAndResolverIsSafeForConcurrentReuse(t *testing.T) {
content := encodeList(t, sceneList(scene("chunk-1", "session-alpha", 1, 2, dnd.SceneKindCombat, "Title", "Summary.")))
references := referenceSet(referenceItem(content))
registry := resolveReferences(t, references)
content[0] = '['
if got := registry.Match(chunk("chunk-1", "session-alpha", 1, 2)); got != (ChunkMatch{State: MatchExact, Kind: dnd.SceneKindCombat}) {
t.Fatalf("Match() after input mutation = %#v", got)
}
resolver, err := NewResolver(contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{ReferenceSlot: {}}})
if err != nil {
t.Fatal(err)
}
const workers = 32
var group sync.WaitGroup
errs := make(chan error, workers)
for range workers {
group.Add(1)
go func() {
defer group.Done()
resolved, err := resolver.Resolve(references)
if err != nil {
errs <- err
return
}
if got := resolved.Match(chunk("chunk-1", "session-alpha", 1, 2)); got != (ChunkMatch{State: MatchExact, Kind: dnd.SceneKindCombat}) {
errs <- fmt.Errorf("Match() = %#v", got)
}
}()
}
group.Wait()
close(errs)
for err := range errs {
t.Error(err)
}
}
func resolveList(t *testing.T, value dnd.SceneDescriptionList) *Registry {
t.Helper()
return resolveReferences(t, referenceSet(referenceItem(encodeList(t, value))))
}
func resolveReferences(t *testing.T, references contracts.ReferenceSet) *Registry {
t.Helper()
registry, err := Resolve(references)
if err != nil {
t.Fatalf("Resolve() error = %v", err)
}
return registry
}
func encodeList(t *testing.T, value dnd.SceneDescriptionList) []byte {
t.Helper()
content, err := scenecodec.New().Encode(value)
if err != nil {
t.Fatalf("Encode() error = %v", err)
}
return content
}
func referenceSet(items ...contracts.ReferenceItem) contracts.ReferenceSet {
return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{ReferenceSlot: {Items: items}}}
}
func referenceItem(content []byte) contracts.ReferenceItem {
return contracts.ReferenceItem{SlotName: ReferenceSlot, MediaType: scenecodec.MediaType, Content: append([]byte(nil), content...)}
}
func scene(id string, sourceID string, startUnitID int, endUnitID int, kind dnd.SceneKind, title string, summary string) dnd.SceneDescription {
return dnd.SceneDescription{
ID: id,
SourceRef: source.SourceRef{SourceID: sourceID, StartUnitID: startUnitID, EndUnitID: endUnitID},
Kind: kind,
Title: title,
Summary: summary,
}
}
func sceneList(scenes ...dnd.SceneDescription) dnd.SceneDescriptionList {
return dnd.SceneDescriptionList{Scenes: scenes}
}
func chunk(id string, sourceID string, startUnitID int, endUnitID int) *source.Chunk {
return &source.Chunk{ID: id, Ref: source.SourceRef{SourceID: sourceID, StartUnitID: startUnitID, EndUnitID: endUnitID}}
}
func cloneList(value dnd.SceneDescriptionList) dnd.SceneDescriptionList {
cloned := dnd.SceneDescriptionList{Scenes: append([]dnd.SceneDescription(nil), value.Scenes...)}
return cloned
}

View File

@@ -16,6 +16,8 @@ const NPCInteractionListKind contracts.ArtifactKind = "dnd/npc-interaction-list"
const SceneDescriptionListKind contracts.ArtifactKind = "dnd/scene-description-list"
const ItemEventListKind contracts.ArtifactKind = "dnd/item-event-list"
type SpellList struct {
SpellCasts []SpellCast `json:"spell_casts"`
}
@@ -97,3 +99,26 @@ type SceneDescription struct {
Title string `json:"title"`
Summary string `json:"summary"`
}
type ItemEventKind string
const (
ItemEventKindDiscovered ItemEventKind = "discovered"
ItemEventKindAcquired ItemEventKind = "acquired"
ItemEventKindLost ItemEventKind = "lost"
ItemEventKindConsumed ItemEventKind = "consumed"
ItemEventKindTransferred ItemEventKind = "transferred"
)
type ItemEventList struct {
Events []ItemEvent `json:"events"`
}
type ItemEvent struct {
Name string `json:"name"`
Kind ItemEventKind `json:"kind"`
Quantity *int `json:"quantity,omitempty"`
From string `json:"from,omitempty"`
To string `json:"to,omitempty"`
SourceRefs []source.SourceRef `json:"source_refs"`
}

View File

@@ -0,0 +1,131 @@
// Package invariants validates normalized D&D item-event artifacts.
package invariants
import (
"context"
"fmt"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
itemeventmodel "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/itemevents"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics"
itemeventshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/shape"
)
const (
Key = "normalize/dnd/item-events/invariants"
ReasonCode = "invalid_normalized_item_event_invariants"
policy = "dnd.item_events.normalize_invariants.v1"
)
type Options struct{}
type Validator struct{}
var _ contracts.TypedValidator[dnd.ItemEventList] = (*Validator)(nil)
var _ pipeline.CheckpointFingerprintProvider = (*Validator)(nil)
func New(Options) *Validator { return &Validator{} }
func (v *Validator) Name() string { return Key }
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *Validator) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
return []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}
}
func (v *Validator) Validate(_ context.Context, req contracts.TypedValidationRequest[dnd.ItemEventList]) (contracts.ValidationResult, error) {
if err := Validate(req.Source, req.Value); err != nil {
return contracts.ValidationResult{Approved: false, ReasonCode: ReasonCode, Message: err.Error()}, nil
}
return contracts.ValidationResult{Approved: true}, nil
}
// Validate checks only invariants introduced by item-event normalization.
// Shape and source-reference failures remain owned by their earlier validators.
func Validate(doc *source.SourceDocument, value dnd.ItemEventList) error {
if itemeventshape.Validate(value) != nil {
return nil
}
index := source.NewDocumentIndex(doc)
if !sourceRefsValid(index, value) {
return nil
}
issues := issuesFor(shared.NewSourceRefOrderFromIndex(index), value)
if len(issues) == 0 {
return nil
}
return fmt.Errorf("%s", diagnostics.Aggregate("invalid normalized item event invariants", issues))
}
func issuesFor(order shared.SourceRefOrder, value dnd.ItemEventList) []string {
issues := make([]string, 0)
seenIdentity := make(map[string]int)
for eventIndex, event := range value.Events {
prefix := fmt.Sprintf("events[%d]", eventIndex)
if event.Name != itemeventmodel.DisplayValue(event.Name) {
issues = append(issues, prefix+".name is not display-normalized")
}
if event.From != itemeventmodel.DisplayValue(event.From) {
issues = append(issues, prefix+".from is not display-normalized")
}
if event.To != itemeventmodel.DisplayValue(event.To) {
issues = append(issues, prefix+".to is not display-normalized")
}
for refIndex := 1; refIndex < len(event.SourceRefs); refIndex++ {
previous := event.SourceRefs[refIndex-1]
current := event.SourceRefs[refIndex]
if order.Less(current, previous) {
issues = append(issues, fmt.Sprintf("%s.source_refs are not in canonical order at index %d", prefix, refIndex))
} else if current == previous {
issues = append(issues, fmt.Sprintf("%s.source_refs[%d] duplicates the previous reference", prefix, refIndex))
}
}
key := itemeventmodel.ExactIdentity(order, event)
if previous, exists := seenIdentity[key]; exists {
issues = append(issues, fmt.Sprintf("%s duplicates item event %d under normalized identity", prefix, previous))
} else {
seenIdentity[key] = eventIndex
}
}
for eventIndex := 1; eventIndex < len(value.Events); eventIndex++ {
if itemeventmodel.Less(order, value.Events[eventIndex], value.Events[eventIndex-1]) {
issues = append(issues, fmt.Sprintf("events[%d] is out of canonical order", eventIndex))
}
}
return issues
}
func sourceRefsValid(index source.DocumentIndex, value dnd.ItemEventList) bool {
for _, event := range value.Events {
if !itemeventmodel.ValidSourceRefs(index, event.SourceRefs) {
return false
}
}
return true
}
func Spec() pipeline.ValidatorSpec {
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
}
func Register(registry *pipeline.ValidatorRegistry) error {
return pipeline.RegisterTypedValidatorBuilder(registry, dnd.ItemEventListKind, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.TypedValidator[dnd.ItemEventList], error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return New(options), nil
})
}
func DecodeOptions(options map[string]any) (Options, error) {
if err := pipeline.RejectUnknownOptions(options); err != nil {
return Options{}, err
}
return Options{}, nil
}
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }

View File

@@ -0,0 +1,129 @@
package invariants
import (
"context"
"reflect"
"strings"
"testing"
"unicode/utf8"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
normalizeitemevents "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/itemevents"
)
func TestValidatorApprovesNormalizerOutput(t *testing.T) {
doc := invariantDocument()
input := dnd.ItemEventList{Events: []dnd.ItemEvent{
{Name: " Coin ", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 20, EndUnitID: 20}}},
{Name: "Arrow", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 10, EndUnitID: 10}}},
}}
normalized, err := normalizeitemevents.New(normalizeitemevents.Options{}).Normalize(context.Background(), contracts.TypedNormalizeRequest[dnd.ItemEventList]{Source: doc, MergeOutput: contracts.MergeArtifact[dnd.ItemEventList]{Value: input}})
if err != nil {
t.Fatal(err)
}
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Source: doc, Value: normalized.Value})
if err != nil || !result.Approved {
t.Fatalf("Validate() = %#v, %v; want approval", result, err)
}
}
func TestValidateRejectsOwnedNormalizedInvariants(t *testing.T) {
tests := []struct {
name string
mutate func(*dnd.ItemEventList)
want string
}{
{name: "name whitespace", mutate: func(value *dnd.ItemEventList) { value.Events[0].Name = " Coin " }, want: ".name is not display-normalized"},
{name: "from whitespace", mutate: func(value *dnd.ItemEventList) {
value.Events[0].Kind = dnd.ItemEventKindLost
value.Events[0].From = " Aria "
}, want: ".from is not display-normalized"},
{name: "to whitespace", mutate: func(value *dnd.ItemEventList) {
value.Events[0].Kind = dnd.ItemEventKindAcquired
value.Events[0].To = " Aria "
}, want: ".to is not display-normalized"},
{name: "reference order", mutate: func(value *dnd.ItemEventList) {
value.Events[0].SourceRefs = []source.SourceRef{{SourceID: "session", StartUnitID: 20, EndUnitID: 20}, {SourceID: "session", StartUnitID: 10, EndUnitID: 10}}
}, want: "not in canonical order"},
{name: "duplicate reference", mutate: func(value *dnd.ItemEventList) {
value.Events[0].SourceRefs = append(value.Events[0].SourceRefs, value.Events[0].SourceRefs[0])
}, want: "duplicates the previous reference"},
{name: "list order", mutate: func(value *dnd.ItemEventList) { value.Events = []dnd.ItemEvent{value.Events[1], value.Events[0]} }, want: "out of canonical order"},
{name: "duplicate event", mutate: func(value *dnd.ItemEventList) { value.Events = append(value.Events, value.Events[0]) }, want: "duplicates item event"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
value := normalizedList()
test.mutate(&value)
err := Validate(invariantDocument(), value)
if err == nil || !strings.Contains(err.Error(), test.want) {
t.Fatalf("Validate() error = %v, want %q", err, test.want)
}
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Source: invariantDocument(), Value: value})
if err != nil || result.Approved || result.ReasonCode != ReasonCode {
t.Fatalf("validator result = %#v, %v", result, err)
}
})
}
}
func TestValidateUsesSourceDocumentOrderAndDefersEarlierFailures(t *testing.T) {
doc := &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 30}, {ID: 10}}}
value := dnd.ItemEventList{Events: []dnd.ItemEvent{{
Name: "Coin", Kind: dnd.ItemEventKindDiscovered,
SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 10, EndUnitID: 10}, {SourceID: doc.ID, StartUnitID: 30, EndUnitID: 30}},
}}}
if err := Validate(doc, value); err == nil || !strings.Contains(err.Error(), "not in canonical order") {
t.Fatalf("Validate() error = %v, want document-order rejection", err)
}
shapeInvalid := normalizedList()
shapeInvalid.Events[0].Name = " "
if err := Validate(invariantDocument(), shapeInvalid); err != nil {
t.Fatalf("shape failure must be deferred, got %v", err)
}
sourceInvalid := normalizedList()
sourceInvalid.Events[0].SourceRefs[0].StartUnitID = 999
if err := Validate(invariantDocument(), sourceInvalid); err != nil {
t.Fatalf("source-reference failure must be deferred, got %v", err)
}
}
func TestValidatorContractAndBoundedDiagnostics(t *testing.T) {
value := normalizedList()
value.Events = make([]dnd.ItemEvent, 30)
for index := range value.Events {
value.Events[index] = normalizedList().Events[0]
value.Events[index].Name = strings.Repeat("火", 300)
}
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Source: invariantDocument(), Value: value})
if err != nil || result.Approved || !utf8.ValidString(result.Message) || len([]byte(result.Message)) > 4096 || !strings.Contains(result.Message, "additional issue(s) omitted") {
t.Fatalf("bounded result = %#v, %v", result, err)
}
if got := New(Options{}).CheckpointFingerprints(); !reflect.DeepEqual(got, []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}) {
t.Fatalf("CheckpointFingerprints() = %#v", got)
}
if spec := Spec(); spec.Key != Key || spec.ExecutionClass != contracts.ExecutionClassDeterministic {
t.Fatalf("Spec() = %#v", spec)
}
registry := pipeline.NewValidatorRegistry()
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v", err)
}
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
t.Fatal("DecodeOptions() accepted unknown option")
}
}
func normalizedList() dnd.ItemEventList {
return dnd.ItemEventList{Events: []dnd.ItemEvent{
{Name: "Arrow", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 10, EndUnitID: 10}}},
{Name: "Coin", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 20, EndUnitID: 20}}},
}}
}
func invariantDocument() *source.SourceDocument {
return &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 10}, {ID: 20}}}
}

View File

@@ -0,0 +1,99 @@
// Package shape validates required D&D item-event candidate fields.
package shape
import (
"context"
"fmt"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/itemevents"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics"
)
const (
Key = "extract/dnd/item-events/shape"
ReasonCode = "invalid_item_event_shape"
policy = "dnd.item_events.shape.v1"
)
type Options struct{}
type Validator struct{}
var _ contracts.TypedValidator[dnd.ItemEventList] = (*Validator)(nil)
var _ pipeline.CheckpointFingerprintProvider = (*Validator)(nil)
func New(Options) *Validator { return &Validator{} }
func (v *Validator) Name() string { return Key }
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *Validator) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
return []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}
}
func (v *Validator) Validate(_ context.Context, req contracts.TypedValidationRequest[dnd.ItemEventList]) (contracts.ValidationResult, error) {
if err := Validate(req.Value); err != nil {
return contracts.ValidationResult{Approved: false, ReasonCode: ReasonCode, Message: err.Error()}, nil
}
return contracts.ValidationResult{Approved: true}, nil
}
// Validate returns one bounded error for every owned item-event shape issue.
func Validate(value dnd.ItemEventList) error {
issues := issuesFor(value)
if len(issues) == 0 {
return nil
}
return fmt.Errorf("%s", diagnostics.Aggregate("invalid item event shape", issues))
}
func issuesFor(value dnd.ItemEventList) []string {
if value.Events == nil {
return []string{"events must be present"}
}
issues := make([]string, 0)
for index, event := range value.Events {
prefix := fmt.Sprintf("events[%d]", index)
if strings.TrimSpace(event.Name) == "" {
issues = append(issues, prefix+".name must not be empty: "+diagnostics.Quote(event.Name))
}
if !itemevents.SupportedKind(event.Kind) {
issues = append(issues, prefix+".kind is unsupported: "+diagnostics.Quote(string(event.Kind)))
} else if !itemevents.ValidHolderCombination(event.Kind, event.From, event.To) {
issues = append(issues, prefix+".from and .to are incompatible with "+diagnostics.Quote(string(event.Kind)))
}
if event.Quantity != nil && *event.Quantity < 1 {
issues = append(issues, prefix+".quantity must be positive when present")
}
if len(event.SourceRefs) == 0 {
issues = append(issues, prefix+".source_refs must contain at least one reference")
}
}
return issues
}
func Spec() pipeline.ValidatorSpec {
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
}
func Register(registry *pipeline.ValidatorRegistry) error {
return pipeline.RegisterTypedValidatorBuilder(registry, dnd.ItemEventListKind, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.TypedValidator[dnd.ItemEventList], error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return New(options), nil
})
}
func DecodeOptions(options map[string]any) (Options, error) {
if err := pipeline.RejectUnknownOptions(options); err != nil {
return Options{}, err
}
return Options{}, nil
}
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }

View File

@@ -0,0 +1,109 @@
package shape
import (
"context"
"reflect"
"strings"
"testing"
"unicode/utf8"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
)
func TestValidatorAcceptsEveryEventRuleAndNormalizableWhitespace(t *testing.T) {
quantity := 1
value := dnd.ItemEventList{Events: []dnd.ItemEvent{
{Name: " Hidden Cache ", Kind: dnd.ItemEventKindDiscovered, From: " ", To: " ", SourceRefs: refs(1, 1)},
{Name: "Gold Pieces", Kind: dnd.ItemEventKindAcquired, Quantity: &quantity, To: " party ", SourceRefs: refs(2, 2)},
{Name: "Torch", Kind: dnd.ItemEventKindLost, From: " party ", SourceRefs: refs(3, 3)},
{Name: "Potion", Kind: dnd.ItemEventKindConsumed, From: " Aria ", SourceRefs: refs(4, 4)},
{Name: "Moonblade", Kind: dnd.ItemEventKindTransferred, From: "Aria", To: "Borin", SourceRefs: refs(5, 5)},
}}
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Value: value})
if err != nil || !result.Approved {
t.Fatalf("Validate() = %#v, %v", result, err)
}
empty, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Value: dnd.ItemEventList{Events: []dnd.ItemEvent{}}})
if err != nil || !empty.Approved {
t.Fatalf("empty list result = %#v, %v", empty, err)
}
}
func TestValidatorRejectsOwnedSemanticBoundaries(t *testing.T) {
valid := validList()
zero := 0
negative := -1
tests := []struct {
name string
value dnd.ItemEventList
want string
}{
{"missing events", dnd.ItemEventList{}, "events must be present"},
{"blank name", listWith(dnd.ItemEvent{Name: " \t", Kind: dnd.ItemEventKindDiscovered, SourceRefs: refs(1, 1)}), "name must not be empty"},
{"unsupported kind", listWith(dnd.ItemEvent{Name: "Ring", Kind: "unknown", SourceRefs: refs(1, 1)}), "kind is unsupported"},
{"discovered holder", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindDiscovered, From: "Aria", SourceRefs: refs(1, 1)}), "incompatible"},
{"acquired without holder", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindAcquired, SourceRefs: refs(1, 1)}), "incompatible"},
{"lost destination", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindLost, From: "Aria", To: "Borin", SourceRefs: refs(1, 1)}), "incompatible"},
{"consumed without holder", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindConsumed, SourceRefs: refs(1, 1)}), "incompatible"},
{"party transfer from", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindTransferred, From: "party", To: "Borin", SourceRefs: refs(1, 1)}), "incompatible"},
{"party transfer to", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindTransferred, From: "Aria", To: " PARTY ", SourceRefs: refs(1, 1)}), "incompatible"},
{"case equivalent transfer", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindTransferred, From: "Aria", To: "aria", SourceRefs: refs(1, 1)}), "incompatible"},
{"unicode equivalent transfer", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindTransferred, From: "Åria", To: "Åria", SourceRefs: refs(1, 1)}), "incompatible"},
{"zero quantity", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindAcquired, Quantity: &zero, To: "party", SourceRefs: refs(1, 1)}), "quantity must be positive"},
{"negative quantity", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindAcquired, Quantity: &negative, To: "party", SourceRefs: refs(1, 1)}), "quantity must be positive"},
{"missing source refs", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindDiscovered}), "source_refs must contain"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Value: test.value})
if err != nil || result.Approved || result.ReasonCode != ReasonCode || !strings.Contains(result.Message, test.want) {
t.Fatalf("Validate() = %#v, %v; want %q", result, err, test.want)
}
})
}
if result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Value: valid}); err != nil || !result.Approved {
t.Fatalf("valid result = %#v, %v", result, err)
}
}
func TestValidatorAggregatesBoundedIndexedDiagnosticsAndRegistration(t *testing.T) {
value := dnd.ItemEventList{Events: make([]dnd.ItemEvent, 24)}
for index := range value.Events {
value.Events[index] = dnd.ItemEvent{Name: " \n", Kind: "unsupported"}
}
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Value: value})
if err != nil || result.Approved || result.ReasonCode != ReasonCode || len([]byte(result.Message)) > 4096 || !utf8.ValidString(result.Message) || !strings.Contains(result.Message, "events[0]") || !strings.Contains(result.Message, "additional issue(s) omitted") {
t.Fatalf("Validate() = %#v, %v", result, err)
}
if value.Events[0].Name != " \n" {
t.Fatal("Validate() mutated input")
}
if got := New(Options{}).CheckpointFingerprints(); !reflect.DeepEqual(got, []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}) {
t.Fatalf("CheckpointFingerprints() = %#v", got)
}
registry := pipeline.NewValidatorRegistry()
if err := Register(registry); err != nil {
t.Fatal(err)
}
if got, ok := registry.Spec(Key); !ok || !reflect.DeepEqual(got, Spec()) {
t.Fatalf("registry spec = %#v, %t", got, ok)
}
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
t.Fatal("DecodeOptions() accepted unknown option")
}
}
func validList() dnd.ItemEventList {
return dnd.ItemEventList{Events: []dnd.ItemEvent{{Name: "Ring", Kind: dnd.ItemEventKindAcquired, To: "party", SourceRefs: refs(1, 1)}}}
}
func listWith(event dnd.ItemEvent) dnd.ItemEventList {
return dnd.ItemEventList{Events: []dnd.ItemEvent{event}}
}
func refs(start, end int) []source.SourceRef {
return []source.SourceRef{{SourceID: "session", StartUnitID: start, EndUnitID: end}}
}

View File

@@ -0,0 +1,121 @@
// Package sourcerefs validates D&D item-event transcript evidence.
package sourcerefs
import (
"context"
"fmt"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics"
itemeventshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/shape"
)
const (
Key = "extract/dnd/item-events/source_refs"
ReasonCode = "invalid_item_event_source_references"
policy = "dnd.item_events.source_refs.v1"
)
type Options struct{}
type Validator struct{}
var _ contracts.TypedValidator[dnd.ItemEventList] = (*Validator)(nil)
var _ pipeline.CheckpointFingerprintProvider = (*Validator)(nil)
func New(Options) *Validator { return &Validator{} }
func (v *Validator) Name() string { return Key }
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *Validator) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
return []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}
}
func (v *Validator) Validate(_ context.Context, req contracts.TypedValidationRequest[dnd.ItemEventList]) (contracts.ValidationResult, error) {
if req.Stage == string(pipeline.StageExtract) && req.Chunk == nil {
return contracts.ValidationResult{}, fmt.Errorf("item event source-reference validator requires the current extraction chunk")
}
if itemeventshape.Validate(req.Value) != nil {
return contracts.ValidationResult{Approved: true}, nil
}
index := source.NewDocumentIndex(req.Source)
var coverage *chunkCoverage
if req.Stage == string(pipeline.StageExtract) {
coverage = newChunkCoverage(req.Chunk)
}
issues := make([]string, 0)
for eventIndex, event := range req.Value.Events {
for refIndex, ref := range event.SourceRefs {
if err := index.ValidateRef(ref); err != nil {
issues = append(issues, fmt.Sprintf("events[%d].source_refs[%d]: %s", eventIndex, refIndex, diagnostics.Truncate(err.Error())))
continue
}
if coverage != nil && !coverage.contains(req.Source, ref) {
issues = append(issues, fmt.Sprintf("events[%d].source_refs[%d]: source reference is outside the current extraction chunk", eventIndex, refIndex))
}
}
}
if len(issues) == 0 {
return contracts.ValidationResult{Approved: true}, nil
}
return contracts.ValidationResult{Approved: false, ReasonCode: ReasonCode, Message: diagnostics.Aggregate("invalid item event source references", issues)}, nil
}
type chunkCoverage struct {
sourceID string
unitIDs map[int]struct{}
}
func newChunkCoverage(chunk *source.Chunk) *chunkCoverage {
coverage := &chunkCoverage{
sourceID: chunk.SourceID,
unitIDs: make(map[int]struct{}, len(chunk.Units)),
}
for _, unit := range chunk.Units {
coverage.unitIDs[unit.ID] = struct{}{}
}
return coverage
}
func (coverage *chunkCoverage) contains(doc *source.SourceDocument, ref source.SourceRef) bool {
if coverage == nil || doc == nil || ref.SourceID != coverage.sourceID {
return false
}
start, startOK := source.UnitIndex(doc, ref.StartUnitID)
end, endOK := source.UnitIndex(doc, ref.EndUnitID)
if !startOK || !endOK || start > end {
return false
}
for position := start; position <= end; position++ {
if _, found := coverage.unitIDs[doc.Units[position].ID]; !found {
return false
}
}
return true
}
func Spec() pipeline.ValidatorSpec {
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
}
func Register(registry *pipeline.ValidatorRegistry) error {
return pipeline.RegisterTypedValidatorBuilder(registry, dnd.ItemEventListKind, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.TypedValidator[dnd.ItemEventList], error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return New(options), nil
})
}
func DecodeOptions(options map[string]any) (Options, error) {
if err := pipeline.RejectUnknownOptions(options); err != nil {
return Options{}, err
}
return Options{}, nil
}
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }

View File

@@ -0,0 +1,121 @@
package sourcerefs
import (
"context"
"reflect"
"strings"
"testing"
"unicode/utf8"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
)
func TestValidatorOwnsSourceAndRangeValidation(t *testing.T) {
value := validList()
result, err := New(Options{}).Validate(context.Background(), request(document(), value))
if err != nil || !result.Approved {
t.Fatalf("valid result = %#v, %v", result, err)
}
value.Events[0].SourceRefs = []source.SourceRef{
{SourceID: "foreign", StartUnitID: 1, EndUnitID: 1},
{SourceID: "session", StartUnitID: 99, EndUnitID: 99},
{SourceID: "session", StartUnitID: 2, EndUnitID: 1},
}
result, err = New(Options{}).Validate(context.Background(), request(document(), value))
if err != nil || result.Approved || result.ReasonCode != ReasonCode {
t.Fatalf("invalid result = %#v, %v", result, err)
}
for _, index := range []string{"events[0].source_refs[0]", "events[0].source_refs[1]", "events[0].source_refs[2]"} {
if !strings.Contains(result.Message, index) {
t.Fatalf("validation message = %q, missing %q", result.Message, index)
}
}
}
func TestValidatorEnforcesChunkOnlyDuringExtraction(t *testing.T) {
doc := document()
chunk := &source.Chunk{ID: "chunk-0", SourceID: doc.ID, Units: append([]source.SourceUnit(nil), doc.Units[:2]...)}
contained := validList()
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Stage: string(pipeline.StageExtract), Source: doc, Chunk: chunk, Value: contained})
if err != nil || !result.Approved {
t.Fatalf("contained evidence = %#v, %v", result, err)
}
crossesChunk := validList()
crossesChunk.Events[0].SourceRefs = []source.SourceRef{{SourceID: doc.ID, StartUnitID: 2, EndUnitID: 3}}
result, err = New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Stage: string(pipeline.StageExtract), Source: doc, Chunk: chunk, Value: crossesChunk})
if err != nil || result.Approved || !strings.Contains(result.Message, "outside the current extraction chunk") {
t.Fatalf("out-of-chunk evidence = %#v, %v", result, err)
}
noncontiguousChunk := &source.Chunk{ID: "chunk-1", SourceID: doc.ID, Units: []source.SourceUnit{doc.Units[0], doc.Units[2]}}
spansMissingUnit := validList()
spansMissingUnit.Events[0].SourceRefs = []source.SourceRef{{SourceID: doc.ID, StartUnitID: 1, EndUnitID: 3}}
result, err = New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Stage: string(pipeline.StageExtract), Source: doc, Chunk: noncontiguousChunk, Value: spansMissingUnit})
if err != nil || result.Approved || !strings.Contains(result.Message, "outside the current extraction chunk") {
t.Fatalf("partially contained evidence = %#v, %v", result, err)
}
multiRange := validList()
multiRange.Events[0].SourceRefs = []source.SourceRef{{SourceID: doc.ID, StartUnitID: 1, EndUnitID: 2}, {SourceID: doc.ID, StartUnitID: 3, EndUnitID: 4}}
result, err = New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Source: doc, Value: multiRange})
if err != nil || !result.Approved {
t.Fatalf("post-merge evidence = %#v, %v", result, err)
}
}
func TestValidatorRequiresChunkDuringExtractionAndDefersShape(t *testing.T) {
_, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Stage: string(pipeline.StageExtract), Source: document(), Value: validList()})
if err == nil || !strings.Contains(err.Error(), "requires the current extraction chunk") {
t.Fatalf("Validate() error = %v", err)
}
malformed := dnd.ItemEventList{Events: []dnd.ItemEvent{{Name: "Ring"}}}
result, err := New(Options{}).Validate(context.Background(), request(document(), malformed))
if err != nil || !result.Approved {
t.Fatalf("shape deferral = %#v, %v", result, err)
}
}
func TestValidatorBoundsDiagnosticsAndRegistration(t *testing.T) {
value := dnd.ItemEventList{Events: make([]dnd.ItemEvent, 24)}
for index := range value.Events {
value.Events[index] = dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "foreign", StartUnitID: index + 1, EndUnitID: index + 1}}}
}
result, err := New(Options{}).Validate(context.Background(), request(document(), value))
if err != nil || result.Approved || result.ReasonCode != ReasonCode || len([]byte(result.Message)) > 4096 || !utf8.ValidString(result.Message) || !strings.Contains(result.Message, "events[0].source_refs[0]") || !strings.Contains(result.Message, "additional issue(s) omitted") {
t.Fatalf("Validate() = %#v, %v", result, err)
}
before := validList()
copy := before
if _, err := New(Options{}).Validate(context.Background(), request(document(), before)); err != nil || !reflect.DeepEqual(before, copy) {
t.Fatal("Validate() mutated input")
}
if got := New(Options{}).CheckpointFingerprints(); !reflect.DeepEqual(got, []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}) {
t.Fatalf("CheckpointFingerprints() = %#v", got)
}
registry := pipeline.NewValidatorRegistry()
if err := Register(registry); err != nil {
t.Fatal(err)
}
if got, ok := registry.Spec(Key); !ok || !reflect.DeepEqual(got, Spec()) {
t.Fatalf("registry spec = %#v, %t", got, ok)
}
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
t.Fatal("DecodeOptions() accepted unknown option")
}
}
func request(doc *source.SourceDocument, value dnd.ItemEventList) contracts.TypedValidationRequest[dnd.ItemEventList] {
return contracts.TypedValidationRequest[dnd.ItemEventList]{Source: doc, Value: value}
}
func document() *source.SourceDocument {
return &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 1}, {ID: 2}, {ID: 3}, {ID: 4}}}
}
func validList() dnd.ItemEventList {
return dnd.ItemEventList{Events: []dnd.ItemEvent{{Name: "Ring", Kind: dnd.ItemEventKindAcquired, To: "party", SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 1, EndUnitID: 2}}}}}
}

View File

@@ -0,0 +1,88 @@
// Package sourcerelatedness warns when item-event evidence does not mention its item.
package sourcerelatedness
import (
"context"
"fmt"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics"
itemeventshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/shape"
)
const (
Key = "extract/dnd/item-events/source_relatedness"
WarningReasonCode = "item_event_source_unrelated"
OmittedReasonCode = "item_event_relatedness_warnings_omitted"
policy = "dnd.item_events.source_relatedness.v1"
)
type Options struct{}
type Validator struct{}
var _ contracts.TypedValidator[dnd.ItemEventList] = (*Validator)(nil)
var _ pipeline.CheckpointFingerprintProvider = (*Validator)(nil)
func New(Options) *Validator { return &Validator{} }
func (v *Validator) Name() string { return Key }
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *Validator) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
return []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}
}
// Validate performs advisory source grounding only. Shape and range failures
// are intentionally left to their blocking owners.
func (v *Validator) Validate(_ context.Context, req contracts.TypedValidationRequest[dnd.ItemEventList]) (contracts.ValidationResult, error) {
if itemeventshape.Validate(req.Value) != nil {
return contracts.ValidationResult{Approved: true}, nil
}
resolver, err := shared.NewCitationResolver(req.Source)
if err != nil {
return contracts.ValidationResult{Approved: true}, nil
}
warnings := make([]contracts.Warning, 0)
for index, event := range req.Value.Events {
citedText, err := resolver.CitedText(event.SourceRefs)
if err != nil || shared.ContainsTokenSequence(citedText, event.Name) {
continue
}
warnings = append(warnings, contracts.Warning{
Scope: fmt.Sprintf("events[%d]", index),
ReasonCode: WarningReasonCode,
Message: fmt.Sprintf("item event name %s was not found in cited source text", diagnostics.Quote(event.Name)),
})
}
return contracts.ValidationResult{
Approved: true,
Warnings: diagnostics.LimitWarnings(warnings, "item_events", OmittedReasonCode),
}, nil
}
func Spec() pipeline.ValidatorSpec {
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
}
func Register(registry *pipeline.ValidatorRegistry) error {
return pipeline.RegisterTypedValidatorBuilder(registry, dnd.ItemEventListKind, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.TypedValidator[dnd.ItemEventList], error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return New(options), nil
})
}
func DecodeOptions(options map[string]any) (Options, error) {
if err := pipeline.RejectUnknownOptions(options); err != nil {
return Options{}, err
}
return Options{}, nil
}
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }

View File

@@ -0,0 +1,74 @@
package sourcerelatedness
import (
"context"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics"
)
func TestValidatorMatchesTokenSequencesAcrossRanges(t *testing.T) {
doc := &source.SourceDocument{ID: "session", Units: []source.SourceUnit{
{ID: 1, Text: "Aria finds a potion,"},
{ID: 2, Text: "of healing in the chest."},
{ID: 3, Text: "The group rests."},
}}
value := dnd.ItemEventList{Events: []dnd.ItemEvent{
{Name: "POTION OF HEALING", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 1, EndUnitID: 1}, {SourceID: doc.ID, StartUnitID: 2, EndUnitID: 2}}},
{Name: "missing-item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 3, EndUnitID: 3}}},
}}
references := contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
"glossary": {Items: []contracts.ReferenceItem{{Content: []byte("missing-item")}}},
}}
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Source: doc, References: references, Value: value})
if err != nil || !result.Approved || len(result.Warnings) != 1 {
t.Fatalf("Validate() = %#v, %v", result, err)
}
if warning := result.Warnings[0]; warning.Scope != "events[1]" || warning.ReasonCode != WarningReasonCode || !strings.Contains(warning.Message, "missing-item") {
t.Fatalf("warning = %#v", warning)
}
}
func TestValidatorDefersMalformedAndUnreadableEvidence(t *testing.T) {
doc := &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 1, Text: "a potion"}}}
for _, value := range []dnd.ItemEventList{
{Events: []dnd.ItemEvent{{Name: "", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 1, EndUnitID: 1}}}}},
{Events: []dnd.ItemEvent{{Name: "potion", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 99, EndUnitID: 99}}}}},
} {
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Source: doc, Value: value})
if err != nil || !result.Approved || len(result.Warnings) != 0 {
t.Fatalf("deferred result = %#v, %v", result, err)
}
}
}
func TestValidatorBoundsWarningsAndRegistersPolicy(t *testing.T) {
count := diagnostics.MaxWarnings + 5
events := make([]dnd.ItemEvent, count)
for index := range events {
events[index] = dnd.ItemEvent{Name: "missing item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 1, EndUnitID: 1}}}
}
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{
Source: &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 1, Text: "nothing useful"}}},
Value: dnd.ItemEventList{Events: events},
})
if err != nil || !result.Approved || len(result.Warnings) != diagnostics.MaxWarnings || result.Warnings[len(result.Warnings)-1].ReasonCode != OmittedReasonCode {
t.Fatalf("Validate() = %#v, %v", result, err)
}
if got := New(Options{}).CheckpointFingerprints(); !reflect.DeepEqual(got, []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}) {
t.Fatalf("CheckpointFingerprints() = %#v", got)
}
registry := pipeline.NewValidatorRegistry()
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v", err)
}
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
t.Fatal("DecodeOptions() accepted unknown option")
}
}

View File

@@ -18,6 +18,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
combatcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/combatturns"
npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs"
scenecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
combatextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
combatnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/combatturns"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity"
@@ -59,6 +60,7 @@ func TestProductionCombatPipelineRetriesMergesNormalizesAndWritesJSON(t *testing
}
fingerprints := prepared.CheckpointFingerprints()
assertCombatFingerprint(t, fingerprints, "extract:combat:"+combatextract.Key+":npc_registry")
assertCombatFingerprint(t, fingerprints, "extract:combat:"+combatextract.Key+":scene_eligibility")
assertCombatFingerprint(t, fingerprints, "normalize:combat:"+combatnormalize.Key+":npc_registry")
if len(fingerprints) == 0 {
t.Fatal("checkpoint fingerprints = empty, want production combat identities")
@@ -99,16 +101,16 @@ func TestProductionCombatPipelineRetriesMergesNormalizesAndWritesJSON(t *testing
if !hasCombatWarning(output.Warnings, combatnormalize.ReasonCodeActorCanonicalized) {
t.Fatalf("warnings = %#v, want registry normalization warning", output.Warnings)
}
if len(output.Manifest.References) != 2 {
t.Fatalf("manifest references = %#v, want separate extract and normalize provenance", output.Manifest.References)
if len(output.Manifest.References) != 3 {
t.Fatalf("manifest references = %#v, want scene eligibility plus extract and normalize NPC provenance", output.Manifest.References)
}
lane := output.Manifest.ArtifactLanes[0]
if lane.ID != "combat" || lane.Extractor != combatextract.Key || lane.Merger != pipeline.DefaultMergeModule || lane.Normalizer != combatnormalize.Key {
t.Fatalf("manifest lane = %#v, want complete combat composition", lane)
}
extractorMetadata, ok := lane.Metadata["extractor"].(map[string]any)
if !ok || extractorMetadata["npc_count"] != 2 || extractorMetadata["npc_registry_digest"] == "" {
t.Fatalf("extractor metadata = %#v, want registry digest and count", lane.Metadata)
if !ok || extractorMetadata["npc_count"] != 2 || extractorMetadata["npc_registry_digest"] == "" || extractorMetadata["scene_description_count"] != 3 || extractorMetadata["scene_eligibility_digest"] == "" || extractorMetadata["scene_gate_policy"] != "dnd.combat_turns.scene_gate.v1" {
t.Fatalf("extractor metadata = %#v, want registry and scene eligibility metadata", lane.Metadata)
}
normalizerMetadata, ok := lane.Metadata["normalizer"].(map[string]any)
if !ok || normalizerMetadata["npc_count"] != 2 || normalizerMetadata["normalization_policy"] != combatnormalize.NormalizationPolicy {
@@ -131,6 +133,13 @@ func TestProductionCombatPipelineAttributesExhaustedInvalidEnumsToShapeValidatio
configValue := combatOnlyConfig()
profile := configValue.Pipelines["dnd-combat-fixture"]
profile.Chunk.Options["max_units"] = 100
profile.References["scene_descriptions"] = pipeline.ExternalReference(combatSceneDescriptionPath(t, []dnd.SceneDescription{{
ID: "chunk-000001",
SourceRef: source.SourceRef{SourceID: "npc-session", StartUnitID: 1, EndUnitID: 5},
Kind: dnd.SceneKindCombat,
Title: "Complete combat encounter",
Summary: "The accepted chunk contains active combat.",
}}))
configValue.Pipelines["dnd-combat-fixture"] = profile
effective, err := configValue.Resolve(config.ResolveInput{
PipelineID: "dnd-combat-fixture",
@@ -144,7 +153,11 @@ func TestProductionCombatPipelineAttributesExhaustedInvalidEnumsToShapeValidatio
combatTestInvalidEnumResponse("invalid"),
combatTestInvalidEnumResponse("unknown"),
}}
prepared, err := pipeline.Prepare(effective.ResolvedPipeline, registries, pipeline.ModuleDependencies{LLM: client})
materialized, _, err := pipeline.MaterializeReferences(effective.ResolvedPipeline, moduleCatalog(registries), pipeline.ReferenceMaterializationOptions{})
if err != nil {
t.Fatalf("MaterializeReferences() error = %v", err)
}
prepared, err := pipeline.Prepare(materialized, registries, pipeline.ModuleDependencies{LLM: client})
if err != nil {
t.Fatalf("Prepare() error = %v, want nil", err)
}
@@ -168,6 +181,56 @@ func TestProductionCombatPipelineAttributesExhaustedInvalidEnumsToShapeValidatio
}
}
func TestCombatPipelinePublishesUnavailableSceneWarningForMismatchedReference(t *testing.T) {
registries := productionNPCRegistries(t)
configValue := combatOnlyConfig()
profile := configValue.Pipelines["dnd-combat"]
profile.Chunk.Options["max_units"] = 100
profile.References["scene_descriptions"] = pipeline.ExternalReference(combatSceneDescriptionPath(t, []dnd.SceneDescription{{
ID: "chunk-000001",
SourceRef: source.SourceRef{SourceID: "npc-session", StartUnitID: 2, EndUnitID: 5},
Kind: dnd.SceneKindCombat,
Title: "Mismatched combat encounter",
Summary: "The classification range does not match the accepted chunk.",
}}))
configValue.Pipelines["dnd-combat"] = profile
catalog := moduleCatalog(registries)
effective, err := configValue.Resolve(config.ResolveInput{PipelineID: "dnd-combat", Catalog: catalog})
if err != nil {
t.Fatalf("Resolve() error = %v", err)
}
materialized, warnings, err := pipeline.MaterializeReferences(effective.ResolvedPipeline, catalog, pipeline.ReferenceMaterializationOptions{})
if err != nil || len(warnings) != 0 {
t.Fatalf("MaterializeReferences() error = %v warnings = %#v", err, warnings)
}
client := &fakeCombatLLMClient{}
prepared, err := pipeline.Prepare(materialized, registries, pipeline.ModuleDependencies{LLM: client})
if err != nil {
t.Fatalf("Prepare() error = %v", err)
}
output, err := pipeline.New().Run(context.Background(), pipeline.RunInput{
Prepared: prepared,
RawInput: readNPCFixture(t),
ExtractWorkers: 1,
})
if err != nil {
t.Fatalf("Run() error = %v", err)
}
if len(client.requests) != 0 || len(output.Rejected) != 0 || !hasCombatWarning(output.Warnings, "scene_classification_unavailable") {
t.Fatalf("requests/rejected/warnings = %#v / %#v / %#v, want accepted unavailable-scene skip without a combat request", client.requests, output.Rejected, output.Warnings)
}
if len(output.NormalizeOutputs) != 1 {
t.Fatalf("normalized outputs = %#v, want accepted empty combat lane", output.NormalizeOutputs)
}
value, err := combatcodec.New().Decode(output.NormalizeOutputs[0].Artifact.Content)
if err != nil {
t.Fatalf("Decode(combat output) error = %v", err)
}
if value.CombatTurns == nil || len(value.CombatTurns) != 0 {
t.Fatalf("mismatched scene combat output = %#v, want non-nil empty turns", value)
}
}
func TestCombatNormalizerRejectsCampaignReferenceBinding(t *testing.T) {
registries := productionNPCRegistries(t)
catalog := moduleCatalog(registries)
@@ -267,11 +330,25 @@ func combatTestNPCPayload(t *testing.T) combatNPCPayload {
return combatNPCPayload{path: path}
}
func combatSceneDescriptionPath(t *testing.T, scenes []dnd.SceneDescription) string {
t.Helper()
content, err := scenecodec.New().Encode(dnd.SceneDescriptionList{Scenes: scenes})
if err != nil {
t.Fatalf("Encode(scene descriptions) error = %v", err)
}
path := filepath.Join(t.TempDir(), "scene-descriptions.json")
if err := os.WriteFile(path, content, 0o600); err != nil {
t.Fatal(err)
}
return path
}
func combatOnlyConfig() config.Config {
cfg := config.Default()
cfg.Pipelines["dnd-combat"] = pipeline.PipelineProfile{
Input: pipeline.Binding(transcript.Key),
Chunk: pipeline.ModuleBinding{Module: pipeline.DefaultChunkModule, Options: map[string]any{"max_units": 2}},
References: map[string]pipeline.ReferenceSource{"scene_descriptions": pipeline.ExternalReference(repositoryPathForIntegration("internal", "modules", "integration", "testdata", "combat_scene_descriptions.json"))},
Artifacts: map[string]pipeline.ArtifactLaneProfile{
"combat": {
Extract: pipeline.ModuleBinding{Module: combatextract.Key, Retries: 2},

View File

@@ -0,0 +1,170 @@
package integration_test
import (
"context"
"encoding/json"
"fmt"
"os"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
itemeventcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/itemevents"
itemeventextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/itemevents"
itemeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/itemevents"
itemeventshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/shape"
itemeventrefs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/source_refs"
"gitea.maximumdirect.net/eric/notarius/internal/modules/seriatim/input/transcript"
)
func TestProductionItemEventPipelineProducesNormalizedDurableArtifact(t *testing.T) {
raw := readItemEventFixture(t)
registries := productionNPCRegistries(t)
resolved := resolveItemEventPipeline(t, registries)
client := &itemEventProductionLLMClient{responses: [][]byte{[]byte(`{"events":[
{"name":"Healing Potion","kind":"transferred","quantity":1,"from":"Aria","to":"Borin","source_refs":[{"start_segment":3,"end_segment":3}]},
{"name":" Gold Pieces ","kind":"acquired","quantity":25,"to":" party ","source_refs":[{"start_segment":2,"end_segment":2}]},
{"name":"Missing Relic","kind":"discovered","source_refs":[{"start_segment":4,"end_segment":4}]},
{"name":"Ancient Coin","kind":"discovered","source_refs":[{"start_segment":1,"end_segment":1}]},
{"name":"Gold Pieces","kind":"acquired","quantity":25,"to":"party","source_refs":[{"start_segment":2,"end_segment":2}]}
]}`)}}
output, err := runPreparedPipeline(t, registries, resolved, client, pipeline.RunInput{RawInput: raw})
if err != nil {
t.Fatalf("Run() error = %v", err)
}
if len(output.Rejected) != 0 || len(output.NormalizeOutputs) != 1 {
t.Fatalf("run output = %#v, rejected = %#v", output.NormalizeOutputs, output.Rejected)
}
serialized := normalizedLane(t, output, "item-events")
if serialized.NormalizerKey != itemeventnormalize.Key || serialized.Artifact.Schema.ID != itemeventcodec.SchemaID || serialized.Artifact.Schema.Version != itemeventcodec.SchemaVersion {
t.Fatalf("serialized item events = %#v", serialized)
}
events, err := itemeventcodec.New().Decode(serialized.Artifact.Content)
if err != nil {
t.Fatalf("Decode(item event output) error = %v", err)
}
if len(events.Events) != 4 {
t.Fatalf("item events = %#v, want one exact duplicate removed", events)
}
first, second, third, fourth := events.Events[0], events.Events[1], events.Events[2], events.Events[3]
if first.Name != "Ancient Coin" || second.Name != "Gold Pieces" || second.Kind != dnd.ItemEventKindAcquired || second.Quantity == nil || *second.Quantity != 25 || second.To != "party" || third.Name != "Healing Potion" || third.Kind != dnd.ItemEventKindTransferred || third.From != "Aria" || third.To != "Borin" || fourth.Name != "Missing Relic" {
t.Fatalf("normalized item event sequence = %#v", events.Events)
}
for _, event := range events.Events {
for _, ref := range event.SourceRefs {
if ref.SourceID != "item-events-session" {
t.Fatalf("source reference = %#v, want current transcript source", ref)
}
}
}
if !hasItemEventWarning(output.Warnings, "item_event_source_unrelated") || !hasItemEventWarning(output.Warnings, "duplicate_item_event_collapsed") {
t.Fatalf("warnings = %#v, want advisory relatedness and duplicate-collapse warnings", output.Warnings)
}
if output.Manifest.ValidationStatus != "approved" || len(output.Manifest.ArtifactLanes) != 1 {
t.Fatalf("manifest = %#v", output.Manifest)
}
lane := output.Manifest.ArtifactLanes[0]
if lane.ID != "item-events" || lane.Extractor != itemeventextract.Key || lane.Merger != pipeline.DefaultMergeModule || lane.Normalizer != itemeventnormalize.Key {
t.Fatalf("manifest lane = %#v", lane)
}
if len(client.requests) != 1 || client.requests[0].PromptID != itemeventextract.PromptID {
t.Fatalf("LLM requests = %#v", client.requests)
}
}
func TestProductionItemEventPipelineRetriesBlockingCandidates(t *testing.T) {
registries := productionNPCRegistries(t)
resolved := resolveItemEventPipeline(t, registries)
for _, test := range []struct {
name string
response string
reasonCode string
validatorName string
}{
{name: "shape", response: `{"events":[{"name":"","kind":"discovered","source_refs":[{"start_segment":1,"end_segment":1}]}]}`, reasonCode: itemeventshape.ReasonCode, validatorName: itemeventshape.Key},
{name: "source references", response: `{"events":[{"name":"Ancient Coin","kind":"discovered","source_refs":[{"start_segment":99,"end_segment":99}]}]}`, reasonCode: itemeventrefs.ReasonCode, validatorName: itemeventrefs.Key},
} {
t.Run(test.name, func(t *testing.T) {
client := &itemEventProductionLLMClient{responses: [][]byte{[]byte(test.response), []byte(test.response), []byte(test.response)}}
output, err := runPreparedPipeline(t, registries, resolved, client, pipeline.RunInput{RawInput: readItemEventFixture(t)})
if err != nil {
t.Fatalf("Run() error = %v; want rejected candidate outcome", err)
}
if len(output.NormalizeOutputs) != 0 || len(output.Rejected) != 1 || len(client.requests) != 3 {
t.Fatalf("output=%#v rejected=%#v requests=%d", output.NormalizeOutputs, output.Rejected, len(client.requests))
}
rejection := output.Rejected[0]
if rejection.ReasonCode != test.reasonCode || rejection.ValidatorName != test.validatorName || rejection.AttemptCount != 3 {
t.Fatalf("rejection = %#v", rejection)
}
})
}
}
func resolveItemEventPipeline(t *testing.T, registries pipeline.Registries) pipeline.ResolvedPipeline {
t.Helper()
configValue := config.Default()
configValue.Pipelines["dnd-item-events-fixture"] = pipeline.PipelineProfile{
Input: pipeline.Binding(transcript.Key),
Chunk: pipeline.ModuleBinding{Module: pipeline.DefaultChunkModule, Options: map[string]any{"max_units": 100}},
Artifacts: map[string]pipeline.ArtifactLaneProfile{
"item-events": {
Extract: pipeline.ModuleBinding{Module: itemeventextract.Key, Retries: 2},
Normalize: pipeline.Binding(itemeventnormalize.Key),
},
},
}
effective, err := configValue.Resolve(config.ResolveInput{PipelineID: "dnd-item-events-fixture", Catalog: moduleCatalog(registries)})
if err != nil {
t.Fatalf("Resolve() error = %v", err)
}
return effective.ResolvedPipeline
}
func readItemEventFixture(t *testing.T) []byte {
t.Helper()
raw, err := os.ReadFile("testdata/seriatim_item_events_session.json")
if err != nil {
t.Fatal(err)
}
return raw
}
func hasItemEventWarning(warnings []contracts.Warning, reasonCode string) bool {
for _, warning := range warnings {
if warning.ReasonCode == reasonCode && strings.HasPrefix(warning.Scope, "events[") {
return true
}
}
return false
}
type itemEventProductionLLMClient struct {
responses [][]byte
requests []contracts.StructuredCompletionRequest
}
func (client *itemEventProductionLLMClient) CompleteStructured(ctx context.Context, request contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
if err := ctx.Err(); err != nil {
return contracts.StructuredCompletionResponse{}, err
}
if request.PromptID != itemeventextract.PromptID {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("unexpected prompt %q", request.PromptID)
}
index := len(client.requests)
if index >= len(client.responses) {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("missing item event response %d", index)
}
content := append([]byte(nil), client.responses[index]...)
if err := json.Unmarshal(content, out); err != nil {
return contracts.StructuredCompletionResponse{}, fmt.Errorf("populate item event structured target: %w", err)
}
client.requests = append(client.requests, request)
return contracts.StructuredCompletionResponse{Content: content, Provider: "test", Model: "item-events-fake"}, nil
}
var _ contracts.StructuredLLMClient = (*itemEventProductionLLMClient)(nil)

View File

@@ -5,6 +5,8 @@ import (
"crypto/sha256"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"sync"
"testing"
@@ -16,9 +18,11 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
combatcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/combatturns"
npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs"
scenecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
spellcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/spells"
combatextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcs"
sceneextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/scenedescriptions"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
)
@@ -26,6 +30,7 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
registries := productionNPCRegistries(t)
catalog := moduleCatalog(registries)
configValue := loadGroundedPipelineConfig(t)
campaignInputs := configureGroundedCampaignReferences(t, &configValue)
effective, err := configValue.Resolve(config.ResolveInput{PipelineID: "dnd-npc-grounded", Catalog: catalog})
if err != nil {
t.Fatalf("Resolve() error = %v", err)
@@ -43,6 +48,7 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
for name, value := range map[string]string{
"extract:npcs:dnd/npcs:mapping_policy": "dnd.npcs.extract_mapping.v2",
"extract:spells:dnd/spells:mapping_policy": "dnd.spells.extract_mapping.v2",
"extract:combat:dnd/combat-turns:scene_gate_policy": "dnd.combat_turns.scene_gate.v1",
} {
assertFingerprintValue(t, prepared.CheckpointFingerprints(), name, value)
}
@@ -54,23 +60,26 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
"extract:spells:dnd/spells:npc_registry",
"extract:combat:dnd/combat-turns:prompt",
"extract:combat:dnd/combat-turns:response_schema",
"extract:combat:dnd/combat-turns:scene_eligibility",
"extract:combat:dnd/combat-turns:npc_registry",
"normalize:combat:dnd/combat-turns:npc_registry",
} {
assertCombatFingerprint(t, prepared.CheckpointFingerprints(), name)
}
checkpoint := newGeneratedReferenceCheckpointLoader()
output, err := pipeline.New().Run(context.Background(), pipeline.RunInput{
Prepared: prepared,
RawInput: readNPCFixture(t),
ExtractWorkers: 1,
Checkpoint: checkpoint,
})
if err != nil {
t.Fatalf("Run() error = %v", err)
}
if len(output.Rejected) != 0 || len(output.NormalizeOutputs) != 3 {
t.Fatalf("run outputs = %#v rejected = %#v, want NPC, spell, and combat outputs", output.NormalizeOutputs, output.Rejected)
if len(output.Rejected) != 0 || len(output.NormalizeOutputs) != 4 {
t.Fatalf("run outputs = %#v rejected = %#v, want NPC, scene, spell, and combat outputs", output.NormalizeOutputs, output.Rejected)
}
wantSchemas := map[string]string{"npcs": npccodec.SchemaID, "spells": spellcodec.SchemaID, "combat": combatcodec.SchemaID}
wantSchemas := map[string]string{"npcs": npccodec.SchemaID, "scene-descriptions": scenecodec.SchemaID, "spells": spellcodec.SchemaID, "combat": combatcodec.SchemaID}
for _, serialized := range output.NormalizeOutputs {
if serialized.Artifact.Schema.ID != wantSchemas[serialized.LaneID] || serialized.Artifact.Schema.Version != "v1" {
t.Fatalf("%s artifact schema = %#v, want minimal v1 identity", serialized.LaneID, serialized.Artifact.Schema)
@@ -78,6 +87,7 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
}
wantExtractorIdentity := map[string]struct{ promptID, schemaID string }{
"npcs": {npcs.PromptID, npcs.ResponseSchemaID},
"scene-descriptions": {sceneextract.PromptID, sceneextract.ResponseSchemaID},
"spells": {spells.PromptID, spells.ResponseSchemaID},
"combat": {combatextract.PromptID, combatextract.ResponseSchemaID},
}
@@ -89,16 +99,26 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
}
}
var npcPayload []byte
var npcPayload, scenePayload []byte
seenSteps := map[string]string{}
for _, serialized := range output.NormalizeOutputs {
if serialized.LaneID == "npcs" {
seenSteps[serialized.LaneID] = serialized.StepID
switch serialized.LaneID {
case "npcs":
npcPayload = append([]byte(nil), serialized.Artifact.Content...)
break
case "scene-descriptions":
scenePayload = append([]byte(nil), serialized.Artifact.Content...)
}
}
if seenSteps["npcs"] != "identify-npcs" || seenSteps["scene-descriptions"] != "identify-npcs" || seenSteps["spells"] != "grounded-events" || seenSteps["combat"] != "grounded-events" {
t.Fatalf("normalized output steps = %#v, want ordered producer and consumer steps", seenSteps)
}
if len(npcPayload) == 0 {
t.Fatal("NPC producer did not publish a canonical payload")
}
if len(scenePayload) == 0 {
t.Fatal("scene producer did not publish a canonical payload")
}
npcValue, err := npccodec.New().Decode(npcPayload)
if err != nil {
t.Fatalf("decode NPC producer payload: %v", err)
@@ -108,6 +128,15 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
t.Fatalf("encode canonical NPC producer payload: %v", err)
}
canonicalDigest := digestBytes(npcPayload)
sceneValue, err := scenecodec.New().Decode(scenePayload)
if err != nil {
t.Fatalf("decode scene producer payload: %v", err)
}
scenePayload, err = scenecodec.New().Encode(sceneValue)
if err != nil {
t.Fatalf("encode canonical scene producer payload: %v", err)
}
sceneDigest := digestBytes(scenePayload)
projection := []byte(`{"npcs":[{"name":"Mira Thorn"},{"name":"Hooded Guard"}]}`)
projectionDigest := digestBytes(projection)
seenConsumers := map[string]bool{}
@@ -119,11 +148,29 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
if input.MediaType != npccodec.MediaType || input.Digest != projectionDigest || string(input.Content) != string(projection) || input.OriginURI != "" {
t.Fatalf("%s NPC prompt input = %#v, want names-only generated registry", request.PromptID, input)
}
if request.PromptID == combatextract.PromptID {
transcript, ok := request.Inputs["transcript"]
if !ok || !strings.Contains(string(transcript.Content), "Mira Thorn asks the party") {
t.Fatalf("combat transcript input = %#v, want configured transcript content", transcript)
}
for name, want := range campaignInputs {
input, ok := request.Inputs[name]
if !ok || string(input.Content) != want {
t.Fatalf("combat %s input = %#v, want configured campaign reference", name, input)
}
}
if _, ok := request.Inputs["scene_descriptions"]; ok {
t.Fatalf("combat prompt inputs = %#v, must not contain scene descriptions", request.Inputs)
}
}
seenConsumers[request.PromptID] = true
}
if !seenConsumers[spells.PromptID] || !seenConsumers[combatextract.PromptID] {
t.Fatalf("consumer prompt IDs = %#v, want spell and combat requests", seenConsumers)
}
if combatCompletionCount(client.requestsSnapshot()) != 1 {
t.Fatalf("combat completion requests = %#v, want exactly one for the exact combat scene", client.requestsSnapshot())
}
provenanceCount := 0
for _, reference := range output.Manifest.References {
@@ -138,19 +185,41 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
if provenanceCount != 3 {
t.Fatalf("NPC generated provenance count = %d, want spell extract plus combat extract/normalize", provenanceCount)
}
sceneProvenanceCount := 0
for _, reference := range output.Manifest.References {
if reference.SlotName == "scene_descriptions" {
sceneProvenanceCount++
if reference.Stage != string(pipeline.StageExtract) || reference.StepID != "grounded-events" || reference.LaneID != "combat" || reference.OriginType != "generated" || reference.Digest != sceneDigest || reference.ArtifactKind != string(dnd.SceneDescriptionListKind) || reference.SchemaID != scenecodec.SchemaID || reference.SchemaName != scenecodec.SchemaName || reference.SchemaVersion != scenecodec.SchemaVersion || reference.MediaType != scenecodec.MediaType || reference.ProducerPipeline != "dnd-npc-grounded" || reference.ProducerStep != "identify-npcs" || reference.ProducerLane != "scene-descriptions" || reference.ProducerModule != "dnd/scene-descriptions" {
t.Fatalf("scene generated provenance = %#v, want canonical combat handoff identity", reference)
}
}
}
if sceneProvenanceCount != 1 {
t.Fatalf("scene generated provenance count = %d, want combat extractor handoff", sceneProvenanceCount)
}
assertGeneratedSceneDependency(t, checkpoint.extractDependencies("combat"))
manifestContent, err := json.Marshal(output.Manifest)
if err != nil {
t.Fatal(err)
}
for _, forbidden := range []string{"Mira Thorn", "Hooded Guard", "For every NPC record", "Extract Dungeons & Dragons"} {
for _, forbidden := range []string{"Mira Thorn", "Hooded Guard", "For every NPC record", "Extract Dungeons & Dragons", "A combat encounter", "The party faces an active encounter.", "Mira Thorn asks the party", `"scenes":`} {
if strings.Contains(string(manifestContent), forbidden) {
t.Fatalf("manifest exposes prompt or reference payload content %q", forbidden)
}
}
for _, lane := range output.Manifest.ArtifactLanes {
componentContent, err := json.Marshal(lane.Metadata)
if err != nil {
t.Fatalf("marshal %s component metadata: %v", lane.ID, err)
}
for _, forbidden := range []string{"A combat encounter", "The party faces an active encounter.", "Mira Thorn asks the party", `"scenes":`} {
if strings.Contains(string(componentContent), forbidden) {
t.Fatalf("%s component metadata exposes scene or transcript content %q", lane.ID, forbidden)
}
}
for _, component := range []string{"extractor", "normalizer"} {
metadata, ok := lane.Metadata[component].(map[string]any)
if ok && metadata["npc_registry_digest"] != nil {
if ok && (metadata["npc_registry_digest"] != nil || metadata["scene_eligibility_digest"] != nil) {
t.Fatalf("%s %s metadata = %#v, want generated identity only in framework provenance", lane.ID, component, metadata)
}
}
@@ -179,6 +248,38 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
assertCurrentEvidence(t, combatValue.CombatTurns[0].SourceRefs)
}
func TestGroundedPipelineSkipsCombatForExactNarrativeScene(t *testing.T) {
registries := productionNPCRegistries(t)
configValue := loadGroundedPipelineConfig(t)
profile := configValue.Pipelines["dnd-npc-grounded"]
combat := profile.Steps[1].Artifacts["combat"]
combat.Extract.Retries = 2
profile.Steps[1].Artifacts["combat"] = combat
configValue.Pipelines["dnd-npc-grounded"] = profile
client := &groundedDNDLLMClient{sceneKind: dnd.SceneKindNarrative}
output := runGroundedPipeline(t, configValue, registries, client, nil)
if combatCompletionCount(client.requestsSnapshot()) != 0 {
t.Fatalf("combat completion requests = %#v, want none for an exact narrative scene", client.requestsSnapshot())
}
if len(output.Rejected) != 0 || hasCombatWarning(output.Warnings, "scene_classification_unavailable") {
t.Fatalf("narrative output rejected/warnings = %#v / %#v, want accepted skip without unavailable warning", output.Rejected, output.Warnings)
}
combatValue := normalizedCombatOutput(t, output)
if combatValue.CombatTurns == nil || len(combatValue.CombatTurns) != 0 {
t.Fatalf("narrative combat output = %#v, want an accepted empty combat list", combatValue)
}
}
func TestGeneratedSceneReferenceChangesCombatCheckpointDependency(t *testing.T) {
registries := productionNPCRegistries(t)
first := generatedSceneDependency(t, registries, "A combat encounter")
second := generatedSceneDependency(t, registries, "A revised combat encounter")
if first == second {
t.Fatalf("generated combat dependencies = %q and %q, want canonical scene artifact change to invalidate reuse", first, second)
}
}
func assertFingerprintValue(t *testing.T, fingerprints []pipeline.CheckpointFingerprint, name, want string) {
t.Helper()
for _, fingerprint := range fingerprints {
@@ -211,9 +312,128 @@ func loadGroundedPipelineConfig(t *testing.T) config.Config {
return cfg
}
func configureGroundedCampaignReferences(t *testing.T, cfg *config.Config) map[string]string {
t.Helper()
values := map[string]string{
"party": "Mira Thorn: ally\n",
"glossary": "Greencloak: Mira Thorn's title\n",
}
profile := cfg.Pipelines["dnd-npc-grounded"]
profile.References = make(map[string]pipeline.ReferenceSource, len(values))
for name, content := range values {
path := filepath.Join(t.TempDir(), name+".txt")
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
t.Fatalf("write %s reference: %v", name, err)
}
profile.References[name] = pipeline.ExternalReference(path)
}
cfg.Pipelines["dnd-npc-grounded"] = profile
return values
}
func runGroundedPipeline(t *testing.T, configValue config.Config, registries pipeline.Registries, client *groundedDNDLLMClient, checkpoint pipeline.CheckpointLoader) pipeline.RunOutput {
t.Helper()
catalog := moduleCatalog(registries)
effective, err := configValue.Resolve(config.ResolveInput{PipelineID: "dnd-npc-grounded", Catalog: catalog})
if err != nil {
t.Fatalf("Resolve() error = %v", err)
}
materialized, warnings, err := pipeline.MaterializeReferences(effective.ResolvedPipeline, catalog, pipeline.ReferenceMaterializationOptions{})
if err != nil || len(warnings) != 0 {
t.Fatalf("MaterializeReferences() error = %v warnings = %#v", err, warnings)
}
prepared, err := pipeline.Prepare(materialized, registries, pipeline.ModuleDependencies{LLM: client})
if err != nil {
t.Fatalf("Prepare() error = %v", err)
}
output, err := pipeline.New().Run(context.Background(), pipeline.RunInput{
Prepared: prepared,
RawInput: readNPCFixture(t),
ExtractWorkers: 1,
Checkpoint: checkpoint,
})
if err != nil {
t.Fatalf("Run() error = %v", err)
}
return output
}
func normalizedCombatOutput(t *testing.T, output pipeline.RunOutput) dnd.CombatTurnList {
t.Helper()
for _, serialized := range output.NormalizeOutputs {
if serialized.LaneID != "combat" {
continue
}
value, err := combatcodec.New().Decode(serialized.Artifact.Content)
if err != nil {
t.Fatalf("decode combat output: %v", err)
}
return value
}
t.Fatalf("normalized outputs = %#v, want combat lane", output.NormalizeOutputs)
return dnd.CombatTurnList{}
}
func combatCompletionCount(requests []contracts.StructuredCompletionRequest) int {
count := 0
for _, request := range requests {
if request.PromptID == combatextract.PromptID {
count++
}
}
return count
}
func generatedSceneDependency(t *testing.T, registries pipeline.Registries, sceneTitle string) string {
t.Helper()
client := &groundedDNDLLMClient{sceneTitle: sceneTitle}
checkpoint := newGeneratedReferenceCheckpointLoader()
runGroundedPipeline(t, loadGroundedPipelineConfig(t), registries, client, checkpoint)
return assertGeneratedSceneDependency(t, checkpoint.extractDependencies("combat"))
}
func assertGeneratedSceneDependency(t *testing.T, dependencies []pipeline.CheckpointFingerprint) string {
t.Helper()
for _, dependency := range dependencies {
if dependency.Name == "generated-reference:scene_descriptions:0" && strings.HasPrefix(dependency.Value, "sha256:") {
return dependency.Value
}
}
t.Fatalf("combat extract checkpoint dependencies = %#v, want generated scene reference fingerprint", dependencies)
return ""
}
type generatedReferenceCheckpointLoader struct {
pipeline.CheckpointLoader
extract map[string][][]pipeline.CheckpointFingerprint
}
func newGeneratedReferenceCheckpointLoader() *generatedReferenceCheckpointLoader {
return &generatedReferenceCheckpointLoader{
CheckpointLoader: pipeline.NoopCheckpointLoader(),
extract: make(map[string][][]pipeline.CheckpointFingerprint),
}
}
func (loader *generatedReferenceCheckpointLoader) Enabled() bool { return true }
func (loader *generatedReferenceCheckpointLoader) Extract(laneID, _ string, dependencies []pipeline.CheckpointFingerprint) (pipeline.ExtractCheckpoint, pipeline.CheckpointDecision) {
loader.extract[laneID] = append(loader.extract[laneID], append([]pipeline.CheckpointFingerprint(nil), dependencies...))
return pipeline.ExtractCheckpoint{}, pipeline.NewCheckpointDecision(pipeline.CheckpointDecisionExecuted, pipeline.CheckpointReasonMissing)
}
func (loader *generatedReferenceCheckpointLoader) extractDependencies(laneID string) []pipeline.CheckpointFingerprint {
if len(loader.extract[laneID]) != 1 {
return nil
}
return loader.extract[laneID][0]
}
type groundedDNDLLMClient struct {
mu sync.Mutex
requests []contracts.StructuredCompletionRequest
sceneKind dnd.SceneKind
sceneTitle string
}
func (client *groundedDNDLLMClient) CompleteStructured(ctx context.Context, request contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
@@ -235,6 +455,16 @@ func (client *groundedDNDLLMClient) CompleteStructured(ctx context.Context, requ
"name": "Hooded Guard", "source_refs": []any{map[string]int{"start_unit_id": 3, "end_unit_id": 3}},
},
}}
case sceneextract.PromptID:
kind := client.sceneKind
if kind == "" {
kind = dnd.SceneKindCombat
}
title := client.sceneTitle
if title == "" {
title = "A combat encounter"
}
payload = map[string]any{"kind": kind, "title": title, "summary": "The party faces an active encounter."}
case spells.PromptID:
payload = map[string]any{"spell_casts": []any{map[string]any{
"caster": "Mira Thorn", "spell": "Cure Wounds",

View File

@@ -0,0 +1,37 @@
{
"scenes": [
{
"id": "chunk-000001",
"source_ref": {
"source_id": "npc-session",
"start_unit_id": 1,
"end_unit_id": 2
},
"kind": "combat",
"title": "First combat exchange",
"summary": "The first accepted chunk contains active combat."
},
{
"id": "chunk-000002",
"source_ref": {
"source_id": "npc-session",
"start_unit_id": 3,
"end_unit_id": 4
},
"kind": "combat",
"title": "Second combat exchange",
"summary": "The second accepted chunk contains active combat."
},
{
"id": "chunk-000003",
"source_ref": {
"source_id": "npc-session",
"start_unit_id": 5,
"end_unit_id": 5
},
"kind": "combat",
"title": "Final combat exchange",
"summary": "The final accepted chunk contains active combat."
}
]
}

View File

@@ -16,12 +16,19 @@ pipelines:
npcs:
extract: dnd/npcs
normalize: dnd/npcs
scene-descriptions:
extract: dnd/scene-descriptions
normalize: dnd/scene-descriptions
- id: grounded-events
references:
npcs:
artifact:
step: identify-npcs
lane: npcs
scene_descriptions:
artifact:
step: identify-npcs
lane: scene-descriptions
artifacts:
spells:
extract: dnd/spells

View File

@@ -0,0 +1,12 @@
{
"metadata": {
"id": "item-events-session",
"title": "Synthetic D&D item-event session"
},
"segments": [
{"id": 1, "start": 0, "end": 4, "speaker": "DM", "text": "An ancient coin rests in the chest."},
{"id": 2, "start": 4, "end": 8, "speaker": "DM", "text": "The party acquires 25 gold pieces."},
{"id": 3, "start": 8, "end": 12, "speaker": "DM", "text": "Aria gives Borin a healing potion."},
{"id": 4, "start": 12, "end": 16, "speaker": "DM", "text": "The party continues down the road."}
]
}