Compare commits
15 Commits
9614469b45
...
927a7beb88
| Author | SHA1 | Date | |
|---|---|---|---|
| 927a7beb88 | |||
| f7059607af | |||
| 63de44c347 | |||
| f0ede9dacc | |||
| 5711f8b9e3 | |||
| da83510234 | |||
| f51b22bea7 | |||
| f320c2fcee | |||
| 4ba1e50a89 | |||
| 2a7e025251 | |||
| 3da20e9d6a | |||
| b1c0faa748 | |||
| 989f2c220b | |||
| 7e35915b3e | |||
| 24238d249e |
13
docs/cli.md
13
docs/cli.md
@@ -103,10 +103,11 @@ go run ./cmd/notarius run dnd-session \
|
||||
--reference spells.extract.glossary=./campaign-glossary.txt
|
||||
```
|
||||
|
||||
For the maintained NPC-grounded workflow, use the explicit ordered pipeline.
|
||||
The first step produces the normalized NPC artifact; the second step receives
|
||||
it in memory and fans it out to spell extraction, combat extraction, and combat
|
||||
normalization:
|
||||
For the maintained ordered D&D workflow, use the explicit pipeline. The first
|
||||
step produces normalized NPC and scene-description artifacts; the second step
|
||||
receives the NPC artifact in memory for spell extraction, combat extraction, and
|
||||
combat normalization, and receives the required scene-description artifact for
|
||||
combat eligibility:
|
||||
|
||||
```sh
|
||||
go run ./cmd/notarius run dnd-session \
|
||||
@@ -118,6 +119,10 @@ go run ./cmd/notarius run dnd-session \
|
||||
The generated NPC content remains contextual grounding, not spell or combat
|
||||
evidence. It is represented in manifests and debug summaries by bounded
|
||||
identity and producer provenance, not by payload content or a filesystem path.
|
||||
The scene-description artifact is control context: combat extraction calls its
|
||||
LLM only for an exact `combat` scene match. See the
|
||||
[D&D combat-turn artifact contract](integrations/dnd-combat-turn-artifacts.md)
|
||||
for the resulting empty-output and warning behavior.
|
||||
|
||||
The same grammar can target chunk, merge, and normalize slots when the configured
|
||||
modules declare them:
|
||||
|
||||
@@ -22,9 +22,9 @@ The explicit-path option is defined in the [CLI reference](cli.md).
|
||||
- [Minimal D&D configuration](../examples/dnd-minimal.config.yml) shows the
|
||||
shortest useful single-step, single-lane pipeline.
|
||||
- [Complete D&D configuration](../examples/dnd-complete.config.yml) shows scene
|
||||
chunking, ordered steps, all five D&D artifact lanes, a generated NPC
|
||||
handoff, campaign and spell-catalog references, checkpoint storage, and
|
||||
chunk-map output.
|
||||
chunking, ordered steps, all six D&D artifact lanes, generated NPC and
|
||||
scene-description handoffs, campaign and spell-catalog references, checkpoint
|
||||
storage, and chunk-map output.
|
||||
|
||||
Both are complete version 3 files. The fragments below illustrate individual
|
||||
fields and are not alternate complete configurations.
|
||||
@@ -168,12 +168,19 @@ pipelines:
|
||||
npcs:
|
||||
extract: dnd/npcs
|
||||
normalize: dnd/npcs
|
||||
scene-descriptions:
|
||||
extract: dnd/scene-descriptions
|
||||
normalize: dnd/scene-descriptions
|
||||
- id: grounded-events
|
||||
references:
|
||||
npcs:
|
||||
artifact:
|
||||
step: identify-npcs
|
||||
lane: npcs
|
||||
scene_descriptions:
|
||||
artifact:
|
||||
step: identify-npcs
|
||||
lane: scene-descriptions
|
||||
artifacts:
|
||||
spells:
|
||||
extract: dnd/spells
|
||||
@@ -374,7 +381,8 @@ production validators do not call the LLM and must not set `llm_profile`.
|
||||
| chunk | `dnd/scenes` | Uses an LLM to split transcript source units into D&D scenes. |
|
||||
| extract | `dnd/spells` | Extracts typed D&D spell-list artifacts. |
|
||||
| extract | `dnd/npcs` | Extracts typed D&D NPC-list artifacts. |
|
||||
| extract | `dnd/combat-turns` | Extracts typed D&D combat-turn-list artifacts. |
|
||||
| extract | `dnd/combat-turns` | Extracts typed D&D combat-turn-list artifacts only for exact combat scene matches. |
|
||||
| extract | `dnd/item-events` | Extracts typed, source-grounded D&D item-event-list artifacts. |
|
||||
| extract | `dnd/npc-interactions` | Extracts typed D&D NPC-interaction-list artifacts from a supplied NPC registry. |
|
||||
| extract | `dnd/scene-descriptions` | Extracts one typed D&D scene description for each accepted chunk. |
|
||||
| merge | `appendorder` | Combines typed artifacts in chunk order. |
|
||||
@@ -382,6 +390,7 @@ production validators do not call the LLM and must not set `llm_profile`.
|
||||
| normalize | `dnd/spells` | Deterministically canonicalizes and de-duplicates typed D&D spell-list artifacts. |
|
||||
| normalize | `dnd/npcs` | Deterministically consolidates typed D&D NPC-list artifacts by canonical name and unions exact evidence. |
|
||||
| normalize | `dnd/combat-turns` | Deterministically canonicalizes, orders, and de-duplicates typed D&D combat-turn artifacts. |
|
||||
| normalize | `dnd/item-events` | Deterministically trims, source-orders, and removes only exact D&D item-event duplicates. |
|
||||
| normalize | `dnd/npc-interactions` | Canonicalizes registry NPC names, orders interaction occurrences, and removes only exact duplicates. |
|
||||
| normalize | `dnd/scene-descriptions` | Validates, orders, trims, and de-duplicates typed D&D scene-description artifacts. |
|
||||
| output | `json` | Produces JSON output files for normalized `application/json` lanes and can opt in to an accepted chunk map. |
|
||||
@@ -406,6 +415,10 @@ production validators do not call the LLM and must not set `llm_profile`.
|
||||
| `extract/dnd/combat-turns/source_refs` | deterministic | Rejects missing or invalid D&D combat-turn source references. |
|
||||
| `extract/dnd/combat-turns/source_relatedness` | deterministic | Emits warnings when an actor is not found near cited source text. |
|
||||
| `normalize/dnd/combat-turns/invariants` | deterministic | Rejects normalized combat-turn identity, evidence-order, and chronology violations. |
|
||||
| `extract/dnd/item-events/shape` | deterministic | Rejects malformed D&D item-event-list artifacts and invalid category, holder, or quantity combinations. |
|
||||
| `extract/dnd/item-events/source_refs` | deterministic | Rejects missing, invalid, or extract-chunk-external D&D item-event source references. |
|
||||
| `extract/dnd/item-events/source_relatedness` | deterministic | Emits bounded warnings when an item-event name is not found in cited source text. |
|
||||
| `normalize/dnd/item-events/invariants` | deterministic | Rejects non-normalized D&D item-event order, source references, or exact duplicates. |
|
||||
| `extract/dnd/npc-interactions/shape` | deterministic | Rejects malformed D&D NPC-interaction-list artifacts. |
|
||||
| `extract/dnd/npc-interactions/registry` | deterministic | Rejects interaction names absent from the supplied NPC registry. |
|
||||
| `extract/dnd/npc-interactions/source_refs` | deterministic | Rejects missing, invalid, or extract-chunk-external D&D interaction source references. |
|
||||
@@ -469,6 +482,26 @@ normalize:
|
||||
- extract/dnd/combat-turns/source_relatedness
|
||||
```
|
||||
|
||||
The production default chains for `dnd/item-events` are:
|
||||
|
||||
```yaml
|
||||
extract:
|
||||
validators:
|
||||
- generic/valid_json
|
||||
- extract/dnd/item-events/shape
|
||||
- extract/dnd/item-events/source_refs
|
||||
- generic/valid_json_schema
|
||||
- extract/dnd/item-events/source_relatedness
|
||||
normalize:
|
||||
validators:
|
||||
- generic/valid_json
|
||||
- extract/dnd/item-events/shape
|
||||
- normalize/dnd/item-events/invariants
|
||||
- extract/dnd/item-events/source_refs
|
||||
- generic/valid_json_schema
|
||||
- extract/dnd/item-events/source_relatedness
|
||||
```
|
||||
|
||||
The production default chains for `dnd/npc-interactions` are:
|
||||
|
||||
```yaml
|
||||
@@ -567,6 +600,13 @@ accepts no references. The
|
||||
normalized output to later spell, combat-turn, and NPC-interaction targets
|
||||
through an explicit ordered step.
|
||||
|
||||
The `dnd/item-events` extractor accepts optional `glossary`, `party`, `players`,
|
||||
and deprecated `roster` campaign slots for disambiguation only. It requires no
|
||||
generated artifact and its normalizer accepts no references, so the complete
|
||||
example keeps this independent lane in the first step. Current transcript
|
||||
references remain the only item-event evidence. Its complete durable behavior
|
||||
is defined in the [D&D item-event artifact contract](integrations/dnd-item-event-artifacts.md).
|
||||
|
||||
The `dnd/scene-descriptions` extractor accepts only the optional `players`,
|
||||
`party`, and `glossary` campaign slots. They can disambiguate terms in the
|
||||
model-facing title and summary but are never source evidence. Its normalizer
|
||||
@@ -574,14 +614,23 @@ accepts no references. No NPC registry or generated artifact is required. See
|
||||
the [D&D scene-description artifact contract](integrations/dnd-scene-description-artifacts.md)
|
||||
and the copyable [complete D&D configuration](../examples/dnd-complete.config.yml).
|
||||
|
||||
The `dnd/combat-turns` extractor declares the optional campaign slots and the
|
||||
structured `npcs` slot. Campaign references guide only the LLM extraction
|
||||
stage. The deterministic normalizer declares only `npcs`, whose operation-time
|
||||
registry supports the same actor canonicalization. Each `npcs` slot
|
||||
accepts exactly one UTF-8 `application/json` artifact no larger than 1 MiB. The
|
||||
registry's source ranges remain provenance for the reference and never become
|
||||
combat evidence. An ordered step binding fans the same generated NPC artifact
|
||||
out to extraction and normalization:
|
||||
The `dnd/combat-turns` extractor declares the optional campaign slots, the
|
||||
optional structured `npcs` slot, and the required `scene_descriptions` slot.
|
||||
Campaign references guide only the LLM extraction stage. The deterministic
|
||||
normalizer declares only `npcs`, whose operation-time registry supports the
|
||||
same actor canonicalization. Each `npcs` slot accepts exactly one UTF-8
|
||||
`application/json` artifact no larger than 1 MiB. The registry's source ranges
|
||||
remain provenance for the reference and never become combat evidence.
|
||||
|
||||
`scene_descriptions` accepts exactly one approved
|
||||
`dnd/scene-description-list` artifact with media type `application/json`, no
|
||||
larger than 1 MiB (1048576 bytes). Pipeline resolution rejects a combat
|
||||
extractor whose required slot is unbound. The scene artifact is eligibility
|
||||
control context only; it is neither prompt input nor combat evidence, and the
|
||||
combat normalizer does not accept it.
|
||||
|
||||
An ordered step binds the generated NPC artifact to extraction and normalization
|
||||
and the generated scene-description artifact to combat extraction:
|
||||
|
||||
```yaml
|
||||
references:
|
||||
@@ -589,14 +638,22 @@ references:
|
||||
artifact:
|
||||
step: identify-npcs
|
||||
lane: npcs
|
||||
scene_descriptions:
|
||||
artifact:
|
||||
step: identify-npcs
|
||||
lane: scene-descriptions
|
||||
```
|
||||
|
||||
When bound, the combat extractor and normalizer receive the generated registry
|
||||
at operation time. Framework provenance and checkpoint dependencies contain its
|
||||
kind, schema identity, media type, canonical digest, size, and bounded producer
|
||||
identity; names, content, and paths are not recorded there. When
|
||||
absent, the combat prompt receives the exact empty registry value
|
||||
`{"npcs":[]}` with its projection digest and no registry provenance.
|
||||
When `npcs` is bound, the combat extractor and normalizer receive the generated
|
||||
NPC registry at operation time. Framework provenance and checkpoint dependencies
|
||||
contain its kind, schema identity, media type, canonical digest, size, and
|
||||
bounded producer identity; names, content, and paths are not recorded there.
|
||||
When absent, the combat prompt receives the exact empty NPC registry value
|
||||
`{"npcs":[]}` with its projection digest and no registry provenance. The
|
||||
generated `scene_descriptions` artifact is supplied only to combat extraction.
|
||||
The complete example uses the same explicit scene binding; an external approved
|
||||
scene-description artifact may be used instead when the workflow crosses a
|
||||
process or session boundary.
|
||||
|
||||
Both `dnd/npc-interactions` stages require the structured `npcs` slot. Bind one
|
||||
accepted normalized NPC artifact from an earlier ordered step; the interaction
|
||||
|
||||
@@ -73,15 +73,29 @@ The standalone extractor uses these identities:
|
||||
| Private response-schema ID | `notarius.dnd.combat_turns.llm` |
|
||||
| Default profile | `gemini-2-flash` |
|
||||
|
||||
It requires `chunks` and `source.transcript`, accepts no options, and makes one
|
||||
structured completion for each supplied chunk. The prompt receives the
|
||||
chunk-scoped transcript plus the existing `players`, `party`, and `glossary`
|
||||
inputs, and optionally the deprecated `roster` reference through the shared
|
||||
party mapping. The optional `npcs` reference is an approved normalized NPC
|
||||
artifact used only for identity grounding; it never supplies combat evidence.
|
||||
An external file is validated during preparation. In an ordered pipeline, the
|
||||
same slot may receive the producer's canonical generated artifact at the step
|
||||
handoff.
|
||||
It requires `chunks` and `source.transcript`, accepts no options, and requires
|
||||
one `scene_descriptions` reference. That reference must be exactly one approved
|
||||
`dnd/scene-description-list` artifact with media type `application/json` and a
|
||||
maximum size of 1 MiB (1048576 bytes). It may be an external file validated
|
||||
during preparation or a canonical generated artifact supplied at an ordered
|
||||
step handoff. An unbound slot is a configuration error.
|
||||
|
||||
The scene artifact controls eligibility, not evidence. The extractor calls the
|
||||
LLM only when exactly one record has the current chunk's ID, source ID, start
|
||||
unit ID, and end unit ID, and that record has `kind: combat`. An exact
|
||||
`narrative`, `recap`, or `meta` record returns an accepted empty
|
||||
`combat_turns` array without an LLM call, warning, or retry attempt. A missing
|
||||
or mismatched exact record returns the same accepted empty result without an
|
||||
LLM call or retry and emits one content-safe
|
||||
`scene_classification_unavailable` warning. The scene artifact, its title and
|
||||
summary, and its source references are never copied into combat turns.
|
||||
|
||||
For an eligible combat chunk, the prompt receives the chunk-scoped transcript
|
||||
plus the existing `players`, `party`, and `glossary` inputs, and optionally the
|
||||
deprecated `roster` reference through the shared party mapping. The optional
|
||||
`npcs` reference is an approved normalized NPC artifact used only for identity
|
||||
grounding; it never supplies combat evidence. `scene_descriptions` is never a
|
||||
combat prompt input.
|
||||
|
||||
The private response envelope has the same turn fields and JSON types as the
|
||||
durable shape except that source references contain only `start_unit_id`
|
||||
@@ -151,13 +165,16 @@ merge validator chain.
|
||||
## Production manifest and references
|
||||
|
||||
The selectable lane uses extractor and normalizer key `dnd/combat-turns`,
|
||||
`appendorder` for the typed merger, and the durable codec above. A bound `npcs`
|
||||
reference contributes raw-file provenance to the run manifest. A generated
|
||||
binding contributes artifact kind, schema identity, media type, canonical
|
||||
digest, size, and bounded producer provenance. Consumer metadata and checkpoint
|
||||
fingerprints contain no registry names, content, paths, or NPC source ranges.
|
||||
The component-local registry fingerprint covers only the names projected to the
|
||||
consumer, while manifest provenance retains the full artifact digest. The
|
||||
normalized lane is emitted as `lanes/<lane-id>.json` by the JSON
|
||||
output module, and warnings and rejection summaries remain in their shared
|
||||
companion files.
|
||||
`appendorder` for the typed merger, and the durable codec above. Bound external
|
||||
references contribute raw-file provenance to the run manifest. Generated
|
||||
bindings contribute artifact kind, schema identity, media type, canonical
|
||||
digest, size, and bounded producer provenance. This includes the generated
|
||||
scene-description artifact for the combat extractor. Consumer metadata and
|
||||
checkpoint fingerprints contain no reference names, content, paths, source
|
||||
ranges, scene titles, or scene summaries. For an external scene artifact,
|
||||
component metadata records the gate policy plus a semantic eligibility digest
|
||||
and record count; generated scene identity remains in framework handoff
|
||||
provenance and dependencies. The eligibility digest changes with scene ID,
|
||||
exact source range, or kind, but not with title or summary. The normalized lane
|
||||
is emitted as `lanes/<lane-id>.json` by the JSON output module, and warnings
|
||||
and rejection summaries remain in their shared companion files.
|
||||
|
||||
133
docs/integrations/dnd-item-event-artifacts.md
Normal file
133
docs/integrations/dnd-item-event-artifacts.md
Normal file
@@ -0,0 +1,133 @@
|
||||
# D&D Item-Event Artifact Contract
|
||||
|
||||
This document defines the durable D&D item-event artifact and its production
|
||||
boundaries. It records source-grounded discoveries and possession changes; it
|
||||
does not maintain an inventory or ledger.
|
||||
|
||||
## Artifact Identity
|
||||
|
||||
| Property | Value |
|
||||
| --- | --- |
|
||||
| Extractor key | `dnd/item-events` |
|
||||
| Extract capability | `dnd.item_events` |
|
||||
| Artifact kind | `dnd/item-event-list` |
|
||||
| Schema ID | `notarius.dnd.item_events` |
|
||||
| Schema name | `notarius_dnd_item_events_v1` |
|
||||
| Schema version | `v1` |
|
||||
| Media type | `application/json` |
|
||||
| Normalizer key | `dnd/item-events` |
|
||||
|
||||
The payload is one strict JSON object containing a required `events` array,
|
||||
which may be empty. Objects reject unknown fields.
|
||||
|
||||
## Event Shape And Categories
|
||||
|
||||
Every event has `name`, `kind`, and a non-empty `source_refs` array. `quantity`
|
||||
is optional and, when present, is a positive integer. `from` and `to` are
|
||||
optional display values whose presence depends on `kind`.
|
||||
|
||||
| Field | Rule |
|
||||
| --- | --- |
|
||||
| `name` | Non-empty, transcript-supported display value. |
|
||||
| `kind` | One of the categories below. |
|
||||
| `quantity` | Optional positive integer; omitted when the source does not establish a count. |
|
||||
| `from` | Holder that loses possession when the category permits it. |
|
||||
| `to` | Holder that gains possession when the category permits it. |
|
||||
| `source_refs` | One or more current-source references. |
|
||||
|
||||
| Kind | Meaning and holder rule |
|
||||
| --- | --- |
|
||||
| `discovered` | The party learns of or encounters an item without established possession; neither holder is present. |
|
||||
| `acquired` | A party member or `party` gains possession; `to` is required and `from` is absent. |
|
||||
| `lost` | A party member or `party` ceases to possess an item without consuming it; `from` is required and `to` is absent. |
|
||||
| `consumed` | Use depletes, expends, or destroys an item; `from` is required and `to` is absent. |
|
||||
| `transferred` | Possession moves between distinct party members; both holders are required and neither may be `party`. |
|
||||
|
||||
`party` is the reserved display holder for collective party possession when an
|
||||
individual holder is not established. Its comparison is case- and
|
||||
Unicode-insensitive. Transfers require distinct normalized holder values.
|
||||
Giving an item to an NPC, spending currency, selling an item, or another move
|
||||
outside party possession is `lost`, not `transferred` or `consumed`. Monetary
|
||||
spending, purchases, and payments are always `lost`. Currency is `consumed` only
|
||||
when the source explicitly describes its physical destruction or expenditure as
|
||||
a non-payment component. Ordinary non-depleting use is not an event.
|
||||
|
||||
Currency is represented as an ordinary event name plus an explicit quantity
|
||||
when the transcript supplies one. Each denomination remains separate. The
|
||||
artifact never converts denominations, infers a missing count, calculates a
|
||||
balance, or sums nearby events.
|
||||
|
||||
## Source Evidence And Normalization
|
||||
|
||||
Each source reference contains `source_id`, `start_unit_id`, and `end_unit_id`.
|
||||
It must identify an ordered range in the current source document. During
|
||||
extraction, every cited range must also be wholly contained in the current
|
||||
accepted chunk. Campaign references may disambiguate names, but never provide
|
||||
event evidence.
|
||||
|
||||
The deterministic normalizer trims only display-edge whitespace in names and
|
||||
holders, canonicalizes source-reference order and exact duplicate references,
|
||||
then orders events by earliest valid source position and stable tie-breakers
|
||||
over name, kind, holders, quantity, and complete evidence. It removes only
|
||||
events with the same normalized fields and complete valid evidence sequence.
|
||||
Invalid evidence is never collapsed. It does not rename, singularize, resolve
|
||||
aliases, infer holders or quantities, or merge nearby events.
|
||||
|
||||
The default extraction validation chain is JSON syntax, item-event shape,
|
||||
source references, durable JSON Schema, then advisory source relatedness. The
|
||||
normalization chain adds normalized invariants after shape validation and before
|
||||
source references and schema validation. Relatedness warnings are advisory so
|
||||
contextual names and currency notation do not reject otherwise valid evidence.
|
||||
|
||||
## Representative JSON
|
||||
|
||||
```json
|
||||
{
|
||||
"events": [
|
||||
{
|
||||
"name": "Hidden Cache",
|
||||
"kind": "discovered",
|
||||
"source_refs": [{"source_id": "session-7", "start_unit_id": 1, "end_unit_id": 1}]
|
||||
},
|
||||
{
|
||||
"name": "Silver Pieces",
|
||||
"kind": "acquired",
|
||||
"quantity": 20,
|
||||
"to": "party",
|
||||
"source_refs": [{"source_id": "session-7", "start_unit_id": 2, "end_unit_id": 2}]
|
||||
},
|
||||
{
|
||||
"name": "Torch",
|
||||
"kind": "lost",
|
||||
"from": "party",
|
||||
"source_refs": [{"source_id": "session-7", "start_unit_id": 3, "end_unit_id": 3}]
|
||||
},
|
||||
{
|
||||
"name": "Healing Potion",
|
||||
"kind": "consumed",
|
||||
"from": "Aria",
|
||||
"source_refs": [{"source_id": "session-7", "start_unit_id": 4, "end_unit_id": 4}]
|
||||
},
|
||||
{
|
||||
"name": "Moonblade",
|
||||
"kind": "transferred",
|
||||
"from": "Aria",
|
||||
"to": "Borin",
|
||||
"source_refs": [{"source_id": "session-7", "start_unit_id": 5, "end_unit_id": 5}]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Production Boundary
|
||||
|
||||
The extractor requires `chunks` and `source.transcript`, and accepts optional
|
||||
`glossary`, `party`, `players`, and deprecated `roster` campaign references for
|
||||
disambiguation only. It has no generated NPC, scene-description, or item-registry
|
||||
dependency. The append-order merger preserves chunk order, and the normalizer
|
||||
has no reference slots.
|
||||
|
||||
The normalized lane is emitted as `lanes/<lane-id>.json` by the JSON output
|
||||
module. See [Configuration](../config.md#implemented-production-modules) for
|
||||
the selectable keys and default chains, and the
|
||||
[JSON output contract](json-output.md) for bundle paths.
|
||||
@@ -109,12 +109,19 @@ steps:
|
||||
npcs:
|
||||
extract: dnd/npcs
|
||||
normalize: dnd/npcs
|
||||
scene-descriptions:
|
||||
extract: dnd/scene-descriptions
|
||||
normalize: dnd/scene-descriptions
|
||||
- id: grounded-events
|
||||
references:
|
||||
npcs:
|
||||
artifact:
|
||||
step: identify-npcs
|
||||
lane: npcs
|
||||
scene_descriptions:
|
||||
artifact:
|
||||
step: identify-npcs
|
||||
lane: scene-descriptions
|
||||
artifacts:
|
||||
spells:
|
||||
extract: dnd/spells
|
||||
@@ -124,13 +131,16 @@ steps:
|
||||
normalize: dnd/combat-turns
|
||||
```
|
||||
|
||||
The framework hands only an accepted normalized artifact across the barrier. It
|
||||
validates the canonical bytes against each consumer slot and clones the
|
||||
operation-time reference for the spell and combat consumers. Generated
|
||||
The framework hands only accepted normalized artifacts across the barrier. It
|
||||
validates the canonical bytes against each consumer slot and clones the NPC
|
||||
operation-time reference for spell and combat consumers. The accompanying
|
||||
scene-description reference is required by the combat extractor for eligibility
|
||||
only; its consumer contract is defined in the
|
||||
[D&D combat-turn artifact contract](dnd-combat-turn-artifacts.md). Generated
|
||||
provenance records the artifact kind, schema identity, media type, canonical
|
||||
digest, size, and producer step/lane/module, but not names, source
|
||||
ranges, or payload bytes. External normalized files remain supported as
|
||||
explicit references and retain their file provenance.
|
||||
digest, size, and producer step/lane/module, but not names, source ranges, or
|
||||
payload bytes. External normalized files remain supported as explicit references
|
||||
and retain their file provenance.
|
||||
|
||||
NPC source references are registry provenance and are never accepted as spell
|
||||
or combat evidence. Current transcript units remain the only event evidence.
|
||||
|
||||
@@ -118,3 +118,13 @@ Use `dnd/scenes` to form accepted scene chunks, then select
|
||||
[complete D&D configuration](../../examples/dnd-complete.config.yml) includes
|
||||
this lane alongside the other D&D artifacts. Selectable keys and default
|
||||
validator chains are defined in [Configuration](../config.md).
|
||||
|
||||
## Downstream combat use
|
||||
|
||||
The combat-turn extractor consumes this approved artifact as required
|
||||
eligibility control context through an explicit ordered reference binding. It
|
||||
uses only an exact chunk ID and source-range match with `kind: combat` to permit
|
||||
combat extraction; titles, summaries, and scene references do not become combat
|
||||
prompt material or evidence. The complete downstream behavior, including empty
|
||||
results and warnings for unavailable coverage, is defined in the
|
||||
[D&D combat-turn artifact contract](dnd-combat-turn-artifacts.md).
|
||||
|
||||
@@ -39,7 +39,7 @@ without exposing Scriptorium types through stage contracts.
|
||||
7. injecting that one shared client into complete pipeline preparation before
|
||||
the source file is read or the runner is invoked.
|
||||
|
||||
The D&D scene chunker and spell, NPC, combat-turn, NPC-interaction, and
|
||||
The D&D scene chunker and spell, NPC, combat-turn, item-event, NPC-interaction, and
|
||||
scene-description extractors retain this
|
||||
injected client and use it for every structured completion. Operation requests
|
||||
do not carry an LLM client.
|
||||
@@ -99,7 +99,7 @@ D&D prompt content. `internal/framework/promptfs` provides the domain-neutral
|
||||
filesystem composition helper used to combine module-owned files with shared
|
||||
domain prompt fragments.
|
||||
|
||||
The D&D scene chunker and spell, NPC, combat-turn, NPC-interaction, and
|
||||
The D&D scene chunker and spell, NPC, combat-turn, item-event, NPC-interaction, and
|
||||
scene-description extractors each declare an
|
||||
ordered prompt asset manifest. The manifest lists the package-owned YAML and
|
||||
Markdown files, then the exact shared fragments rendered by that prompt; the
|
||||
@@ -134,12 +134,13 @@ prompt places its final boundary on its lane-specific instructions, immediately
|
||||
before the transcript. The transcript does not carry cache control because no
|
||||
reusable content follows it.
|
||||
|
||||
Accordingly, the common prefix of the spell, NPC, combat, and interaction
|
||||
Accordingly, the common prefix of the spell, NPC, combat, item-event, and interaction
|
||||
extraction prompts is system,
|
||||
extraction evidence, identity, and campaign references. The NPC prompt then
|
||||
renders task, instructions, and transcript. Spell renders the NPC registry,
|
||||
catalog, task, instructions, and transcript. Combat renders the NPC registry,
|
||||
task, instructions, and transcript. NPC interaction renders the names-only NPC
|
||||
task, instructions, and transcript. Item-event renders task, instructions, and
|
||||
transcript without a generated-artifact input. NPC interaction renders the names-only NPC
|
||||
registry, task, instructions, and transcript. The
|
||||
scene chunker is not an extraction lane: it retains its separate system,
|
||||
transcript, campaign-reference, task, and instruction order and marks its
|
||||
@@ -166,7 +167,7 @@ return defensive copies, and expose a diagnostics map that omits schema bytes.
|
||||
The small framework registry contains only generic test schemas; production
|
||||
schemas remain package-owned.
|
||||
|
||||
The spell, NPC, combat, NPC-interaction, and scene-description extractors'
|
||||
The spell, NPC, combat, item-event, NPC-interaction, and scene-description extractors'
|
||||
package-owned prompts declare their
|
||||
structured JSON inputs and private response schemas. Each private response
|
||||
schema remains separate from its durable artifact codec schema; this work does
|
||||
|
||||
@@ -17,16 +17,16 @@ validator registry. Package-family registrars compose those leaf registrations
|
||||
into the production catalog and own family-level policy such as default
|
||||
validator chains and prompt asset collection.
|
||||
|
||||
Production input, chunk, output, and D&D spell-, NPC-, combat-, interaction-, and scene-description-extract packages
|
||||
Production input, chunk, output, and D&D spell-, NPC-, combat-, item-event-, interaction-, and scene-description-extract packages
|
||||
register strict option decoders and run-local builders. Preparation decodes their options into
|
||||
implementation-owned values and injects dependencies plus the materialized
|
||||
reference set for the selected target. Each builder receives an isolated clone
|
||||
of that set; input and output builders receive no references. The spell, NPC,
|
||||
combat, interaction, and scene-description extractors are typed over the canonical D&D model. D&D validators, merge,
|
||||
combat, item-event, interaction, and scene-description extractors are typed over the canonical D&D model. D&D validators, merge,
|
||||
and normalize use typed variants; JSON representation validators use serialized
|
||||
requests; and unconditional validators expose separate chunk and typed
|
||||
variants. The D&D production registrar registers the canonical typed spell,
|
||||
NPC, combat, interaction, and scene-description implementations, including their kind-specific merge and
|
||||
NPC, combat, item-event, interaction, and scene-description implementations, including their kind-specific merge and
|
||||
normalize behavior.
|
||||
|
||||
For D&D artifact defaults, generic JSON syntax validation runs first. Rejecting
|
||||
@@ -56,7 +56,7 @@ declarations, prompt-input assembly, and source-unit/citation helpers belong in
|
||||
`internal/modules/dnd/shared`, which owns operation-scoped indexed
|
||||
source-reference validation, citation traversal, ordering and canonicalization,
|
||||
plus bounded D&D diagnostics. The
|
||||
D&D scene chunker and spell, NPC, combat-turn, NPC-interaction, and scene-description extractors use ordered
|
||||
D&D scene chunker and spell, NPC, combat-turn, item-event, NPC-interaction, and scene-description extractors use ordered
|
||||
package-local prompt manifests for both rendering and prompt fingerprinting, so
|
||||
only the shared fragments each prompt actually renders participate in either
|
||||
operation. Extraction prompts place stable shared and lane-specific context
|
||||
@@ -67,7 +67,7 @@ contracts expose only Notarius structured-completion types, not Scriptorium
|
||||
public types.
|
||||
|
||||
The shared `PrepareChunkExtraction` helper owns common extraction preflight and
|
||||
transcript material preparation for the spell, NPC, combat-turn,
|
||||
transcript material preparation for the spell, NPC, combat-turn, item-event,
|
||||
NPC-interaction, and scene-description extractors. It validates common request
|
||||
state, clones supplied source metadata, falls back to the materialized chunk
|
||||
when content is absent, checks that content remains chunk-identical, and fills
|
||||
@@ -123,11 +123,38 @@ shared `internal/modules/dnd/codec/candidatejson` package supplies strict typed
|
||||
candidate JSON mechanics; each artifact codec retains its own durable schema
|
||||
and approved-value policy.
|
||||
|
||||
### `internal/modules/dnd/scenedescriptions/registry`
|
||||
|
||||
The scene-description registry owns the required `scene_descriptions` control
|
||||
reference used by combat extraction. It decodes exactly one approved scene-list
|
||||
artifact through the scene-description codec and retains only scene ID, exact
|
||||
source reference, and kind. Titles, summaries, original bytes, paths, and
|
||||
prompt material do not cross this domain boundary.
|
||||
|
||||
An external reference is validated during preparation; an unbound seed is
|
||||
permitted only while a configured generated reference awaits the ordered
|
||||
handoff. At operation time, a generated artifact overrides the seed and is
|
||||
resolved into an immutable view safe for concurrent extract jobs. Matching is
|
||||
strictly exact by chunk ID, source ID, start unit ID, and end unit ID, producing
|
||||
an exact, missing, or mismatched result. Only an exact result exposes kind.
|
||||
|
||||
The registry's semantic eligibility digest is derived from a sorted projection
|
||||
of ID, exact range, and kind. It ignores titles, summaries, and input order;
|
||||
the unbound view has a stable empty projection digest. Combat extractor
|
||||
metadata and checkpoint identity use this semantic boundary for external
|
||||
references, while generated artifact identity and dependencies remain owned by
|
||||
the framework handoff.
|
||||
|
||||
The `internal/modules/dnd/codec/combatturns` package owns the durable
|
||||
`dnd/combat-turn-list` schema and candidate versus approved JSON boundary. It
|
||||
is registered by the production D&D family registrar for the selectable combat
|
||||
lane.
|
||||
|
||||
The `internal/modules/dnd/codec/itemevents` package owns the durable
|
||||
`dnd/item-event-list` schema and candidate versus approved JSON boundary. It is
|
||||
registered by the production D&D family registrar. Its external contract is
|
||||
defined in the [D&D item-event artifact contract](../integrations/dnd-item-event-artifacts.md).
|
||||
|
||||
The `internal/modules/dnd/codec/npcinteractions` package owns the durable
|
||||
`dnd/npc-interaction-list` schema and candidate versus approved JSON boundary.
|
||||
It is registered by the production D&D family registrar for the selectable
|
||||
@@ -293,18 +320,31 @@ checks. The durable contract is defined in the
|
||||
|
||||
### `internal/modules/dnd/extract/combatturns`
|
||||
|
||||
The combat extractor prepares one structured request per supplied chunk using
|
||||
the shared extraction-evidence, identity, campaign-reference, NPC-grounding,
|
||||
and transcript prompt inputs. It
|
||||
maps the private response to `dnd.CombatTurnList`, assigns the current source
|
||||
identity, removes exact duplicate source ranges, and orders turns by valid
|
||||
source-document position while preserving malformed candidate fields for
|
||||
deterministic validators. Its package-owned private response schema enforces
|
||||
only the structural JSON envelope; semantic artifact constraints remain with
|
||||
the validator chain. Its prepared metadata and checkpoint fingerprints contain
|
||||
only prompt/schema/mapping identities plus an optional NPC registry digest.
|
||||
The prompt follows the shared D&D extraction ordering and cache policy
|
||||
documented in
|
||||
The combat extractor requires the `scene_descriptions` reference and resolves
|
||||
it through the immutable scene-description registry before it resolves NPC
|
||||
grounding or constructs prompt inputs. It calls the LLM only for an exact
|
||||
current-chunk match whose kind is `combat`. Exact `narrative`, `recap`, and
|
||||
`meta` matches return an accepted empty `dnd.CombatTurnList`; missing or
|
||||
mismatched coverage returns the same result with one bounded unavailable-
|
||||
classification warning. These deterministic results do not consume retry
|
||||
attempts. Scene descriptions are control context only and are not passed to the
|
||||
combat prompt or copied into combat evidence.
|
||||
|
||||
For eligible chunks, the extractor prepares one structured request using the
|
||||
shared extraction-evidence, identity, campaign-reference, NPC-grounding, and
|
||||
transcript prompt inputs. It maps the private response to
|
||||
`dnd.CombatTurnList`, assigns the current source identity, removes exact
|
||||
duplicate source ranges, and orders turns by valid source-document position
|
||||
while preserving malformed candidate fields for deterministic validators. Its
|
||||
package-owned private response schema enforces only the structural JSON
|
||||
envelope; semantic artifact constraints remain with the validator chain.
|
||||
|
||||
Prepared metadata and checkpoint fingerprints include prompt, response-schema,
|
||||
mapping, and scene-gate identities. An external scene reference additionally
|
||||
reports its semantic eligibility digest and count; generated identity remains
|
||||
framework handoff provenance and dependency state. Neither surface retains
|
||||
scene prose or payload bytes. The prompt follows the shared D&D extraction
|
||||
ordering and cache policy documented in
|
||||
[LLM Runtime](llm.md#dd-extraction-prompt-ordering-and-cache-boundaries). The
|
||||
package exposes typed registration and is included in the production D&D
|
||||
registrar with the default combat extraction chain.
|
||||
@@ -313,6 +353,23 @@ The combat normalizer accepts only the optional structured NPC registry.
|
||||
Campaign references remain extractor-only LLM context and are not materialized
|
||||
for deterministic normalization.
|
||||
|
||||
### `internal/modules/dnd/extract/itemevents`
|
||||
|
||||
The item-event extractor prepares one structured request from the accepted
|
||||
chunk and optional campaign references, then maps private records to
|
||||
`dnd.ItemEventList` with the current source identity. It declares only optional
|
||||
`glossary`, `party`, `players`, and deprecated `roster` reference slots; these
|
||||
can disambiguate names but never supply evidence. It has no NPC,
|
||||
scene-description, or item-registry dependency.
|
||||
|
||||
The private response schema owns structural transport validation. The extractor
|
||||
preserves candidate category, holder, quantity, and source-range values for the
|
||||
deterministic validators, removes exact duplicate ranges, and source-orders
|
||||
events. The source-reference validator requires citations to fit the current
|
||||
accepted chunk. Prompt, response-schema, and mapping identities participate in
|
||||
checkpoint identity. The durable schema is owned separately by
|
||||
`internal/modules/dnd/codec/itemevents`.
|
||||
|
||||
### `internal/modules/dnd/extract/npcinteractions`
|
||||
|
||||
The NPC interaction extractor requires the structured `npcs` registry slot. It
|
||||
@@ -349,7 +406,7 @@ normalization warnings.
|
||||
### `internal/modules/generic/merge/appendorder`
|
||||
|
||||
The merger passes typed values to an injected combine function in framework
|
||||
source-chunk order. The D&D registrar specializes it for all five artifact
|
||||
source-chunk order. The D&D registrar specializes it for all six artifact
|
||||
lists; each append merger preserves collection presence and order while giving
|
||||
the result independently owned nested source-reference slices.
|
||||
|
||||
@@ -397,6 +454,16 @@ digest/count metadata; generated identity is retained in framework provenance
|
||||
and dependency fingerprints. The normalizer is included in the production D&D
|
||||
registrar with the default combat normalization chain.
|
||||
|
||||
### `internal/modules/dnd/normalize/itemevents`
|
||||
|
||||
The item-event normalizer accepts no options or references and makes no LLM
|
||||
calls. It trims display-edge whitespace in names and holders, canonicalizes
|
||||
source references, source-orders events, and collapses only exact duplicates
|
||||
with complete valid evidence. It does not create a ledger, calculate balances,
|
||||
resolve aliases, infer quantities or holders, or reconcile nearby events. Its
|
||||
policy fingerprint and bounded warnings identify deterministic normalization;
|
||||
the matching invariant validator checks the resulting order and duplicate rule.
|
||||
|
||||
### `internal/modules/dnd/normalize/npcinteractions`
|
||||
|
||||
The interaction normalizer requires the same immutable NPC registry. It
|
||||
@@ -506,6 +573,21 @@ expose local policy fingerprints. In the registered defaults, JSON syntax runs
|
||||
first; combat shape, normalized invariants when applicable, and source-reference
|
||||
validation precede JSON Schema validation; warning-only relatedness runs last.
|
||||
|
||||
## D&D Item-Event Validators
|
||||
|
||||
Item-event shape validation owns the required list, non-empty name, supported
|
||||
category, category-and-holder combination, positive optional quantity, and
|
||||
non-empty source-reference collection. Source-reference validation defers
|
||||
malformed shapes, checks current-source identity and ordered ranges, and during
|
||||
extraction requires every citation to fit the accepted chunk. Relatedness is
|
||||
advisory and warning-only: it checks the event name against cited transcript
|
||||
text while deferring malformed candidates and invalid ranges to their blocking
|
||||
owners. The normalized-invariants validator owns display normalization,
|
||||
canonical source-reference order, chronology, and exact duplicate identity.
|
||||
All four validators are deterministic and expose policy fingerprints. The
|
||||
registered chains run syntax and blocking checks before durable JSON Schema;
|
||||
relatedness remains last.
|
||||
|
||||
## D&D NPC Interaction Validators
|
||||
|
||||
Interaction shape validation owns the required list, registry name, supported
|
||||
@@ -538,7 +620,7 @@ complete framework registry set and one LLM asset registry. It invokes
|
||||
`internal/modules/seriatim/register`, and `internal/modules/dnd/register` in
|
||||
that order, then exposes the matching catalog for resolution. The generic and
|
||||
Seriatim registrars own their production leaf registrations. The D&D registrar
|
||||
owns D&D leaf registrations, typed spell, NPC, combat, interaction, and scene-description default-validator
|
||||
owns D&D leaf registrations, typed spell, NPC, combat, item-event, interaction, and scene-description default-validator
|
||||
chains, typed append-order specializations, and D&D prompt/schema asset
|
||||
collection. Its registration helpers group module, validator, prompt-asset, and
|
||||
chain composition while retaining artifact-specific merge and clone behavior in
|
||||
|
||||
@@ -69,9 +69,9 @@ constructs input, chunk, validators, ordered lanes, and output before source
|
||||
parsing. Production modules use strict construction-time option decoding, and
|
||||
LLM-backed modules retain the injected shared client. The D&D family registers
|
||||
the canonical `dnd/spell-list`, `dnd/npc-list`, `dnd/combat-turn-list`,
|
||||
`dnd/npc-interaction-list`, and `dnd/scene-description-list` codecs, typed
|
||||
spell, NPC, combat, interaction, and scene-description extractors and
|
||||
normalizers, validators,
|
||||
`dnd/item-event-list`, `dnd/npc-interaction-list`, and
|
||||
`dnd/scene-description-list` codecs, typed spell, NPC, combat, item-event,
|
||||
interaction, and scene-description extractors and normalizers, validators,
|
||||
plus kind-specific generic merge strategies; generic JSON validators use the
|
||||
serialized-validation contract. The runner executes lanes through
|
||||
private exact-type-checked closures, coordinates extract results independently
|
||||
@@ -90,25 +90,30 @@ Configuration. The implemented module packages are:
|
||||
| `internal/modules/seriatim/input/transcript` | Parses the supported Seriatim transcript format into the generic source model. |
|
||||
| `internal/modules/generic/chunk/units` | Splits ordered source units by unit count and overlap. |
|
||||
| `internal/modules/dnd/chunk/scenes` | Produces contiguous D&D scene chunks from structured model output. |
|
||||
| `internal/modules/dnd` | Owns the canonical D&D spell-list, spell-cast, NPC-list, NPC, combat-turn-list, combat-turn, NPC-interaction-list, and scene-description-list artifact types. |
|
||||
| `internal/modules/dnd` | Owns the canonical D&D spell-list, spell-cast, NPC-list, NPC, combat-turn-list, combat-turn, item-event-list, item-event, NPC-interaction-list, and scene-description-list artifact types. |
|
||||
| `internal/modules/dnd/codec/spells` | Strictly decodes and stably encodes the durable D&D spell-list representation. |
|
||||
| `internal/modules/dnd/codec/npcs` | Strictly decodes and stably encodes the durable D&D NPC-list representation. |
|
||||
| `internal/modules/dnd/codec/combatturns` | Strictly decodes and stably encodes the durable D&D combat-turn-list representation. |
|
||||
| `internal/modules/dnd/codec/itemevents` | Strictly decodes and stably encodes the durable D&D item-event-list representation. |
|
||||
| `internal/modules/dnd/codec/npcinteractions` | Strictly decodes and stably encodes the durable D&D NPC-interaction-list representation. |
|
||||
| `internal/modules/dnd/codec/scenedescriptions` | Strictly decodes and stably encodes the durable D&D scene-description-list representation. |
|
||||
| `internal/modules/dnd/extract/spells` | Maps private structured model output to canonical source-grounded D&D spell lists. |
|
||||
| `internal/modules/dnd/extract/npcs` | Maps private structured model output to canonical source-grounded D&D NPC lists. |
|
||||
| `internal/modules/dnd/extract/combatturns` | Maps private structured model output to source-grounded D&D combat-turn candidates and preserves chronology and invalid candidate values for validators. |
|
||||
| `internal/modules/dnd/extract/combatturns` | Uses exact scene eligibility to select combat chunks, then maps private structured model output to source-grounded D&D combat-turn candidates. |
|
||||
| `internal/modules/dnd/extract/itemevents` | Maps private structured model output to source-grounded D&D item-event candidates. |
|
||||
| `internal/modules/dnd/extract/npcinteractions` | Maps private structured model output to current-source NPC interaction candidates grounded by a required registry. |
|
||||
| `internal/modules/dnd/extract/scenedescriptions` | Maps one private scene description to the current accepted chunk's ID and exact range. |
|
||||
| `internal/modules/dnd/npcinteractions` | Owns interaction occurrence ordering, valid-evidence checks, and exact interaction identity shared by normalization and invariant validation. |
|
||||
| `internal/modules/dnd/normalize/combatturns` | Canonicalizes and orders merged combat turns, applies exact NPC identity matches, and collapses only exact valid-evidence duplicates. |
|
||||
| `internal/modules/dnd/normalize/itemevents` | Trims, source-orders, and removes only exact valid-evidence item-event duplicates. |
|
||||
| `internal/modules/dnd/normalize/npcinteractions` | Canonicalizes required-registry names, orders interaction occurrences, and collapses only exact valid-evidence duplicates. |
|
||||
| `internal/modules/dnd/normalize/scenedescriptions` | Trims, source-orders, and removes only exactly identical scene descriptions while rejecting ID and range conflicts. |
|
||||
| `internal/modules/dnd/validate/combatturns` | Provides deterministic shape, source-reference, source-relatedness, and normalized-invariant validation for the production combat chains. |
|
||||
| `internal/modules/dnd/validate/itemevents` | Provides deterministic shape, source-reference, source-relatedness, and normalized-invariant validation for item-event chains. |
|
||||
| `internal/modules/dnd/validate/npcinteractions` | Provides deterministic shape, registry, source-reference, source-relatedness, and normalized-invariant validation for interaction chains. |
|
||||
| `internal/modules/dnd/validate/scenedescriptions` | Provides deterministic shape, exact extraction attachment, source-relatedness, and normalized-invariant validation for scene-description chains. |
|
||||
| `internal/modules/dnd/npcs/registry` | Resolves validated normalized NPC references into immutable grounding data and exact identity lookup. |
|
||||
| `internal/modules/dnd/scenedescriptions/registry` | Resolves approved scene descriptions into immutable exact-match combat eligibility data without retaining scene prose. |
|
||||
| `internal/modules/dnd/npcs/identity` | Owns Unicode-aware NPC identity, ID derivation, and registry collision validation. |
|
||||
| `internal/modules/dnd/spells/catalog` | Embeds and validates the versioned D&D 5e 2014 SRD catalog, composes optional overlays, and provides immutable effective lookup. |
|
||||
| `internal/modules/generic/merge/appendorder` | Combines accepted extraction results in chunk order. |
|
||||
|
||||
@@ -50,10 +50,13 @@ needed. This is data deletion, not cache cleanup.
|
||||
|
||||
The maintained [complete D&D configuration](../examples/dnd-complete.config.yml)
|
||||
contains one pipeline with two ordered steps. The first step extracts and
|
||||
normalizes NPCs and scene descriptions. Only after those lanes reach accepted
|
||||
terminal results does the second step begin; its generated NPC reference is
|
||||
normalizes independent item events, NPCs, and scene descriptions. Only after
|
||||
the NPC and scene-description lanes reach accepted terminal results does the
|
||||
second step begin; its generated NPC reference is
|
||||
supplied in memory to spell, combat-turn, and NPC-interaction extraction and
|
||||
the applicable normalizers.
|
||||
the applicable normalizers, while its generated scene-description reference is
|
||||
supplied to combat-turn extraction. The item-event lane has no generated
|
||||
reference dependency and retains only current-transcript evidence.
|
||||
|
||||
```sh
|
||||
go run ./cmd/notarius run dnd-session \
|
||||
@@ -70,6 +73,14 @@ exposed through a filesystem path. The same producer artifact may fan out to
|
||||
compatible consumers, while a missing or rejected producer prevents the later
|
||||
step from starting.
|
||||
|
||||
Combat-turn extraction calls its LLM only when a scene record exactly matches
|
||||
the accepted chunk and has kind `combat`. Exact `narrative`, `recap`, and
|
||||
`meta` records produce an accepted empty combat-turn result without an LLM call.
|
||||
Missing or mismatched scene coverage also produces an accepted empty result and
|
||||
a bounded warning. Scene descriptions remain control context rather than combat
|
||||
evidence or prompt material; the complete behavior is defined by the
|
||||
[D&D combat-turn artifact contract](integrations/dnd-combat-turn-artifacts.md).
|
||||
|
||||
Standalone module configurations continue to support external NPC files when a
|
||||
workflow intentionally crosses a process or session boundary. Those files are
|
||||
validated against the consumer slot and must be protected as sensitive
|
||||
|
||||
@@ -18,10 +18,6 @@ not as committed release dates.
|
||||
|
||||
### Use D&D Scene Chunking
|
||||
|
||||
- Allow the combat extractor to no-op for chunks classified as non-combat only
|
||||
after the scene-description artifact can be supplied through an explicit
|
||||
ordered dependency. Do not make generic chunk materialization depend on a D&D
|
||||
classification.
|
||||
- Use ordered pipeline steps whenever a later artifact needs an accepted
|
||||
earlier artifact as context. Keep independent lanes in the same step and do
|
||||
not introduce a general DAG or concurrent cross-lane reconciliation model.
|
||||
|
||||
190
docs/roadmap/scene-aware-combat-extraction.md
Normal file
190
docs/roadmap/scene-aware-combat-extraction.md
Normal file
@@ -0,0 +1,190 @@
|
||||
# Scene-Aware Combat Extraction
|
||||
|
||||
## Status
|
||||
|
||||
Implemented. This document preserves the feature contract and rationale; the
|
||||
current configuration, operational, integration, and internal behavior is
|
||||
maintained in their canonical documentation.
|
||||
|
||||
## Purpose
|
||||
|
||||
Use an accepted D&D scene-description artifact as the eligibility boundary for
|
||||
combat-turn extraction. The combat LLM should run only for a chunk whose exact
|
||||
scene record has `kind: combat`; every other chunk should be ignored by the
|
||||
combat extractor. This work should connect the existing scene-description lane,
|
||||
ordered generated-reference handoff, and combat-turn lane without expanding the
|
||||
minimal combat-turn artifact contract or introducing D&D policy into generic
|
||||
pipeline code.
|
||||
|
||||
## Target Workflow
|
||||
|
||||
The complete D&D pipeline has two ordered steps:
|
||||
|
||||
1. The first step extracts and normalizes NPCs and scene descriptions for the
|
||||
accepted scene chunks.
|
||||
2. At the step barrier, the accepted NPC and scene-description artifacts become
|
||||
generated references.
|
||||
3. The second step supplies the NPC artifact to its existing consumers and the
|
||||
scene-description artifact to combat-turn extraction.
|
||||
4. For each chunk, combat-turn extraction performs its existing LLM extraction
|
||||
only for an exact `combat` classification. Every other chunk produces a
|
||||
deterministic empty artifact without an LLM call.
|
||||
|
||||
The scene-description dependency must remain explicit in pipeline
|
||||
configuration. The combat extractor must not discover another lane implicitly,
|
||||
read output files directly, inspect chunker-private state, or require generic
|
||||
chunk materialization to interpret D&D scene classifications.
|
||||
|
||||
## Scene-Description Reference Contract
|
||||
|
||||
The combat extractor should declare a required structured reference slot named
|
||||
`scene_descriptions`. The slot accepts one approved
|
||||
`dnd/scene-description-list` artifact using the existing durable scene
|
||||
description schema.
|
||||
|
||||
The reference may be supplied as:
|
||||
|
||||
- a generated artifact from an earlier ordered step; or
|
||||
- an external artifact through the existing reference-materialization
|
||||
boundary.
|
||||
|
||||
External artifacts must be decoded and validated before source parsing or LLM
|
||||
execution. Generated artifacts must cross the existing typed step-handoff
|
||||
boundary and be validated before use. A bound artifact that is malformed,
|
||||
incompatible, or internally inconsistent is an error; it must not be treated as
|
||||
though the slot were unbound.
|
||||
|
||||
Pipeline resolution must reject combat extraction when this slot is not bound.
|
||||
In an ordered same-run workflow, failure of the configured scene-description
|
||||
producer to yield an accepted normalized artifact must fail the run before the
|
||||
combat consumer step starts, consistent with existing required generated
|
||||
reference semantics.
|
||||
|
||||
The prepared reference view should be immutable and safe for concurrent
|
||||
extract jobs. Its metadata and checkpoint identity should be content-safe and
|
||||
must not expose scene titles, summaries, paths, or source text.
|
||||
|
||||
## Chunk Matching And Gating Policy
|
||||
|
||||
A scene record authorizes combat extraction for the current chunk only when all
|
||||
of the following are true:
|
||||
|
||||
- the scene ID exactly equals the current accepted chunk ID;
|
||||
- the scene source ID exactly equals the chunk source ID;
|
||||
- the scene start and end unit IDs exactly equal the chunk's inclusive source
|
||||
range; and
|
||||
- the approved artifact contains exactly one such record.
|
||||
|
||||
The normalized scene-description contract already rejects conflicting IDs and
|
||||
ranges. The combat extractor must nevertheless require the exact match above at
|
||||
its own decision boundary rather than relying on array position, range overlap,
|
||||
title, summary, or inferred chronology.
|
||||
|
||||
An exact match with `kind: combat` performs the existing combat-turn LLM
|
||||
extraction. An exact match with `narrative`, `recap`, or `meta` returns a typed
|
||||
`dnd/combat-turn-list` containing an empty `combat_turns` array without making
|
||||
an LLM call.
|
||||
|
||||
A valid artifact with no exact match, incomplete coverage, or a chunk identity
|
||||
or range mismatch also returns the deterministic empty artifact. It must not
|
||||
infer combat eligibility from an overlapping or adjacent scene. Emit a bounded,
|
||||
content-safe warning for missing or mismatched coverage so operators can
|
||||
distinguish an intentional non-combat classification from an unavailable exact
|
||||
classification. The warning must not include scene prose or transcript text.
|
||||
|
||||
This policy relies on the scene contract's mixed-scene precedence: any chunk in
|
||||
which combat is a substantive central activity is classified as `combat`.
|
||||
Scene-aware gating must not add a second classification policy.
|
||||
|
||||
## Extraction, Validation, And Provenance
|
||||
|
||||
The deterministic empty result follows the same typed extractor and validator
|
||||
boundaries as an LLM-produced empty result. It is not a rejection, does not
|
||||
consume retry budget, and continues through merge and normalization normally.
|
||||
The durable combat-turn schema remains unchanged.
|
||||
|
||||
Existing combat extraction behavior—including prompt assets, NPC grounding,
|
||||
candidate mapping, validators, retries, warnings, and normalization—remains
|
||||
unchanged for chunks classified as combat. The scene-description artifact is
|
||||
control context only:
|
||||
|
||||
- it must not be added to the combat prompt;
|
||||
- its title or summary must not become combat evidence;
|
||||
- its source references must not be copied into combat turns; and
|
||||
- it must not create, repair, or classify a combat turn.
|
||||
|
||||
Generated-reference provenance and dependency fingerprints should cover the
|
||||
scene artifact through the existing ordered-handoff machinery. External
|
||||
references should contribute their existing materialization provenance plus a
|
||||
component-local semantic fingerprint sufficient to invalidate combat extract
|
||||
checkpoints when a classification or chunk identity changes. Checkpoint reuse
|
||||
must never preserve a skipped result after the effective scene classification
|
||||
changes.
|
||||
|
||||
Run manifests may report bounded module metadata such as the number of approved
|
||||
scene records. They must not contain scene prose or duplicate the referenced
|
||||
artifact payload.
|
||||
|
||||
## Configuration And Documentation
|
||||
|
||||
The maintained complete D&D example should bind the normalized
|
||||
`scene-descriptions` lane from the first step to the `scene_descriptions` slot
|
||||
in the second step. The minimal example should remain unchanged.
|
||||
|
||||
Current-behavior documentation is maintained in these canonical locations:
|
||||
|
||||
- Configuration owns the new selectable reference slot and binding example.
|
||||
- Operations owns the ordered scene-aware workflow.
|
||||
- The combat-turn integration contract owns externally observable extraction,
|
||||
empty-result, and provenance behavior.
|
||||
- Internal pipeline and module documentation own preparation, handoff, matching,
|
||||
and checkpoint mechanics.
|
||||
- `future.md` should remove the completed scene-aware combat item.
|
||||
|
||||
## Quality Expectations
|
||||
|
||||
Tests should protect behavior and architectural boundaries rather than internal
|
||||
helper shape. Coverage should demonstrate:
|
||||
|
||||
- exact non-combat matches produce accepted empty combat artifacts without an
|
||||
LLM call;
|
||||
- exact combat matches retain the existing LLM path;
|
||||
- an unbound required reference is rejected during pipeline resolution;
|
||||
- missing and mismatched chunk coverage produces an accepted empty combat
|
||||
artifact without an LLM call and emits a bounded warning;
|
||||
- malformed external and generated artifacts fail at the appropriate
|
||||
preparation or handoff boundary;
|
||||
- retries are neither consumed nor attempted for deterministic skips;
|
||||
- scene artifact changes invalidate relevant checkpoint reuse;
|
||||
- no scene prose is exposed through combat prompts, warnings, metadata, or
|
||||
manifests; and
|
||||
- the complete maintained configuration resolves and materializes the intended
|
||||
ordered dependency.
|
||||
|
||||
Model-output fixtures should remain deterministic test doubles. This feature
|
||||
does not require live-provider tests or assertions over probabilistic model
|
||||
quality.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This scope does not:
|
||||
|
||||
- change the durable scene-description or combat-turn schemas;
|
||||
- add scene fields to combat-turn artifacts;
|
||||
- annotate generic chunks with D&D classifications;
|
||||
- make the combat lane depend implicitly on the scene-description lane;
|
||||
- skip spell, NPC, NPC-interaction, or scene-description extraction;
|
||||
- infer combat from scene titles, summaries, overlap, or campaign references;
|
||||
- introduce arbitrary DAG scheduling or concurrent cross-step execution;
|
||||
- add prior-run artifact discovery or new reference syntax; or
|
||||
- add an LLM-backed validator, repair pass, or generic deduplication stage.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
The scope is complete when an explicitly configured ordered D&D pipeline can
|
||||
hand an approved scene-description artifact to combat extraction, exact
|
||||
`combat` scene matches are the only chunks that invoke the combat LLM, every
|
||||
other chunk produces a deterministic empty result, provenance and checkpoint
|
||||
identity remain correct, the complete example demonstrates the workflow, and
|
||||
the canonical current-behavior documentation reflects the implemented
|
||||
contract.
|
||||
@@ -29,9 +29,15 @@ pipelines:
|
||||
options:
|
||||
include_chunk_map: true
|
||||
steps:
|
||||
# Establish session-wide reference artifacts before extracting events.
|
||||
# Establish session-wide reference artifacts alongside independent item events.
|
||||
- id: describe-session
|
||||
artifacts:
|
||||
item-events:
|
||||
extract:
|
||||
module: dnd/item-events
|
||||
retries: 2
|
||||
merge: appendorder
|
||||
normalize: dnd/item-events
|
||||
npcs:
|
||||
extract:
|
||||
module: dnd/npcs
|
||||
@@ -45,13 +51,17 @@ pipelines:
|
||||
merge: appendorder
|
||||
normalize: dnd/scene-descriptions
|
||||
- id: extract-events
|
||||
# The accepted NPC artifact is supplied in memory to every compatible
|
||||
# extractor and normalizer in this step.
|
||||
# Accepted NPC grounding and scene-description eligibility artifacts are
|
||||
# supplied in memory to their compatible consumers in this step.
|
||||
references:
|
||||
npcs:
|
||||
artifact:
|
||||
step: describe-session
|
||||
lane: npcs
|
||||
scene_descriptions:
|
||||
artifact:
|
||||
step: describe-session
|
||||
lane: scene-descriptions
|
||||
artifacts:
|
||||
spells:
|
||||
extract:
|
||||
|
||||
@@ -48,8 +48,12 @@ func TestProductionCombatConfigurationResolvesTypedLane(t *testing.T) {
|
||||
if !ok || codecSpec.Schema.ID != "notarius.dnd.combat_turns" || codecSpec.Schema.Version != "v1" {
|
||||
t.Fatalf("combat codec spec = %#v, want compatible durable schema", codecSpec)
|
||||
}
|
||||
if !hasReferenceSlot(extractSpec.ReferenceSlots, "npcs") || !hasReferenceSlot(normalizeSpec.ReferenceSlots, "npcs") {
|
||||
t.Fatalf("combat reference slots = %#v / %#v, want stage-local NPC slots", extractSpec.ReferenceSlots, normalizeSpec.ReferenceSlots)
|
||||
if !hasReferenceSlot(extractSpec.ReferenceSlots, "npcs") || !hasReferenceSlot(extractSpec.ReferenceSlots, "scene_descriptions") || !hasReferenceSlot(normalizeSpec.ReferenceSlots, "npcs") {
|
||||
t.Fatalf("combat reference slots = %#v / %#v, want extraction scene and NPC slots plus normalization NPC slot", extractSpec.ReferenceSlots, normalizeSpec.ReferenceSlots)
|
||||
}
|
||||
sceneSlot := referenceSlot(extractSpec.ReferenceSlots, "scene_descriptions")
|
||||
if !sceneSlot.Required || !reflect.DeepEqual(sceneSlot.AcceptedMediaTypes, []string{"application/json"}) || !reflect.DeepEqual(sceneSlot.AcceptedArtifactKinds, []contracts.ArtifactKind{dnd.SceneDescriptionListKind}) || sceneSlot.MaxBytes != 1048576 {
|
||||
t.Fatalf("scene description slot = %#v, want required approved scene artifact", sceneSlot)
|
||||
}
|
||||
|
||||
wantExtractChain := []pipeline.ModuleBinding{
|
||||
@@ -89,8 +93,19 @@ func TestProductionCombatConfigurationResolvesTypedLane(t *testing.T) {
|
||||
t.Fatalf("Resolve(bound references) error = %v, want nil", err)
|
||||
}
|
||||
boundLane := bound.ResolvedPipeline.Steps[0].ArtifactLanes[0]
|
||||
if len(boundLane.ExtractReferences.Bindings) != 1 || len(boundLane.NormalizeReferences.Bindings) != 1 || boundLane.ExtractReferences.Bindings[0].SlotName != "npcs" || boundLane.NormalizeReferences.Bindings[0].SlotName != "npcs" {
|
||||
t.Fatalf("bound combat references = %#v / %#v, want one independent NPC binding per stage", boundLane.ExtractReferences, boundLane.NormalizeReferences)
|
||||
if len(boundLane.ExtractReferences.Bindings) != 2 || len(boundLane.NormalizeReferences.Bindings) != 1 || !hasReferenceBinding(boundLane.ExtractReferences.Bindings, "npcs") || !hasReferenceBinding(boundLane.ExtractReferences.Bindings, "scene_descriptions") || !hasReferenceBinding(boundLane.NormalizeReferences.Bindings, "npcs") {
|
||||
t.Fatalf("bound combat references = %#v / %#v, want extraction scene and NPC bindings plus normalization NPC binding", boundLane.ExtractReferences, boundLane.NormalizeReferences)
|
||||
}
|
||||
}
|
||||
|
||||
func TestProductionCombatConfigurationRequiresSceneDescriptions(t *testing.T) {
|
||||
components := productionTestComponents(t)
|
||||
cfg := productionCombatContractConfig()
|
||||
profile := cfg.Pipelines["dnd-combat"]
|
||||
profile.References = nil
|
||||
cfg.Pipelines["dnd-combat"] = profile
|
||||
if _, err := cfg.Resolve(config.ResolveInput{PipelineID: "dnd-combat", Catalog: catalogFromRegistries(components.registries)}); err == nil || !strings.Contains(err.Error(), "scene_descriptions") || !strings.Contains(err.Error(), "required") {
|
||||
t.Fatalf("Resolve() error = %v, want required scene reference failure", err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -154,6 +169,7 @@ func productionCombatContractConfig() config.Config {
|
||||
ID: "dnd-combat",
|
||||
Input: pipeline.Binding("seriatim"),
|
||||
Chunk: pipeline.Binding(pipeline.DefaultChunkModule),
|
||||
References: map[string]pipeline.ReferenceSource{"scene_descriptions": pipeline.ExternalReference("scenes.json")},
|
||||
Artifacts: map[string]pipeline.ArtifactLaneProfile{
|
||||
"combat": {
|
||||
Extract: pipeline.ModuleBinding{Module: combatextract.Key, Retries: 2},
|
||||
@@ -172,3 +188,21 @@ func hasReferenceSlot(slots []contracts.ReferenceSlot, name string) bool {
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func hasReferenceBinding(bindings []pipeline.ReferenceBinding, name string) bool {
|
||||
for _, binding := range bindings {
|
||||
if binding.SlotName == name {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func referenceSlot(slots []contracts.ReferenceSlot, name string) contracts.ReferenceSlot {
|
||||
for _, slot := range slots {
|
||||
if slot.Name == name {
|
||||
return slot
|
||||
}
|
||||
}
|
||||
return contracts.ReferenceSlot{}
|
||||
}
|
||||
|
||||
@@ -34,7 +34,7 @@ func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
|
||||
t.Fatalf("materialize maintained example references for %q: %v", pipelineID, err)
|
||||
}
|
||||
if example.name == "complete" {
|
||||
if got := exampleStepLaneIDs(materialized); strings.Join(got, "|") != "describe-session:npcs,scene-descriptions|extract-events:combat-turns,npc-interactions,spells" {
|
||||
if got := exampleStepLaneIDs(materialized); strings.Join(got, "|") != "describe-session:item-events,npcs,scene-descriptions|extract-events:combat-turns,npc-interactions,spells" {
|
||||
t.Fatalf("complete example steps and lanes = %v, want every D&D extractor in the documented two-step composition", got)
|
||||
}
|
||||
spellLane := referenceContractLane(t, materialized, "spells")
|
||||
@@ -42,6 +42,15 @@ func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
|
||||
len(spellLane.NormalizeReferences.ReferenceSet.Slots["spell_catalog"].Items) != 1 {
|
||||
t.Fatalf("complete example spell catalog reference was not materialized: %#v", spellLane)
|
||||
}
|
||||
itemEventLane := referenceContractLane(t, materialized, "item-events")
|
||||
for _, references := range []pipeline.ResolvedReferenceTarget{itemEventLane.ExtractReferences, itemEventLane.NormalizeReferences} {
|
||||
if _, found := references.ReferenceSet.Slots["npcs"]; found {
|
||||
t.Fatalf("item event lane unexpectedly depends on generated NPCs: %#v", itemEventLane)
|
||||
}
|
||||
if _, found := references.ReferenceSet.Slots["scene_descriptions"]; found {
|
||||
t.Fatalf("item event lane unexpectedly depends on generated scene descriptions: %#v", itemEventLane)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
var stdout, stderr strings.Builder
|
||||
|
||||
@@ -24,10 +24,13 @@ import (
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/chunk/scenes"
|
||||
combatcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/combatturns"
|
||||
itemeventcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/itemevents"
|
||||
spellcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/spells"
|
||||
combatextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
|
||||
itemeventextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/itemevents"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
|
||||
combatnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/combatturns"
|
||||
itemeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/itemevents"
|
||||
spellnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/spells"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/generic/normalize/noop"
|
||||
)
|
||||
@@ -38,9 +41,9 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
||||
|
||||
assertProductionContains(t, "inputs", registries.Inputs.RegisteredKeys(), []string{"seriatim"})
|
||||
assertProductionContains(t, "chunkers", registries.Chunkers.RegisteredKeys(), []string{"dnd/scenes", "generic"})
|
||||
assertProductionContains(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells", "dnd/npcs", combatextract.Key})
|
||||
assertProductionContains(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells", "dnd/npcs", combatextract.Key, itemeventextract.Key})
|
||||
assertProductionContains(t, "mergers", registries.Mergers.RegisteredKeys(), []string{"appendorder"})
|
||||
assertProductionContains(t, "normalizers", registries.Normalizers.RegisteredKeys(), []string{"noop", spellnormalize.Key, "dnd/npcs", combatnormalize.Key})
|
||||
assertProductionContains(t, "normalizers", registries.Normalizers.RegisteredKeys(), []string{"noop", spellnormalize.Key, "dnd/npcs", combatnormalize.Key, itemeventnormalize.Key})
|
||||
assertProductionContains(t, "outputs", registries.Outputs.RegisteredKeys(), []string{"json"})
|
||||
assertProductionContains(t, "validators", registries.Validators.RegisteredKeys(), []string{
|
||||
"extract/dnd/spells/catalog",
|
||||
@@ -51,16 +54,21 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
||||
"extract/dnd/combat-turns/source_refs",
|
||||
"extract/dnd/combat-turns/source_relatedness",
|
||||
"normalize/dnd/combat-turns/invariants",
|
||||
"extract/dnd/item-events/shape",
|
||||
"extract/dnd/item-events/source_refs",
|
||||
"extract/dnd/item-events/source_relatedness",
|
||||
"normalize/dnd/item-events/invariants",
|
||||
"generic/always_accept",
|
||||
"generic/always_reject",
|
||||
"generic/valid_json",
|
||||
"generic/valid_json_schema",
|
||||
})
|
||||
assertProductionContains(t, "artifact codec kinds", registries.ArtifactCodecs.RegisteredKinds(), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind})
|
||||
assertProductionContains(t, "merger variants", registries.Mergers.RegisteredArtifactKinds(pipeline.DefaultMergeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind})
|
||||
assertProductionContains(t, "normalizer variants", registries.Normalizers.RegisteredArtifactKinds(pipeline.DefaultNormalizeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind})
|
||||
assertProductionContains(t, "artifact codec kinds", registries.ArtifactCodecs.RegisteredKinds(), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind})
|
||||
assertProductionContains(t, "merger variants", registries.Mergers.RegisteredArtifactKinds(pipeline.DefaultMergeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind})
|
||||
assertProductionContains(t, "normalizer variants", registries.Normalizers.RegisteredArtifactKinds(pipeline.DefaultNormalizeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind})
|
||||
assertProductionContains(t, "spell normalizer variants", registries.Normalizers.RegisteredArtifactKinds(spellnormalize.Key), []contracts.ArtifactKind{dnd.SpellListKind})
|
||||
assertProductionContains(t, "combat normalizer variants", registries.Normalizers.RegisteredArtifactKinds(combatnormalize.Key), []contracts.ArtifactKind{dnd.CombatTurnListKind})
|
||||
assertProductionContains(t, "item event normalizer variants", registries.Normalizers.RegisteredArtifactKinds(itemeventnormalize.Key), []contracts.ArtifactKind{dnd.ItemEventListKind})
|
||||
|
||||
wantChain := []pipeline.ModuleBinding{
|
||||
pipeline.Binding("generic/valid_json"),
|
||||
@@ -97,6 +105,27 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
||||
if got := registries.ValidatorChains.Validators(pipeline.StageNormalize, combatnormalize.Key); !reflect.DeepEqual(got, combatNormalizeChain) {
|
||||
t.Fatalf("combat normalize validator chain = %#v, want %#v", got, combatNormalizeChain)
|
||||
}
|
||||
itemEventExtractChain := []pipeline.ModuleBinding{
|
||||
pipeline.Binding("generic/valid_json"),
|
||||
pipeline.Binding("extract/dnd/item-events/shape"),
|
||||
pipeline.Binding("extract/dnd/item-events/source_refs"),
|
||||
pipeline.Binding("generic/valid_json_schema"),
|
||||
pipeline.Binding("extract/dnd/item-events/source_relatedness"),
|
||||
}
|
||||
itemEventNormalizeChain := []pipeline.ModuleBinding{
|
||||
pipeline.Binding("generic/valid_json"),
|
||||
pipeline.Binding("extract/dnd/item-events/shape"),
|
||||
pipeline.Binding("normalize/dnd/item-events/invariants"),
|
||||
pipeline.Binding("extract/dnd/item-events/source_refs"),
|
||||
pipeline.Binding("generic/valid_json_schema"),
|
||||
pipeline.Binding("extract/dnd/item-events/source_relatedness"),
|
||||
}
|
||||
if got := registries.ValidatorChains.Validators(pipeline.StageExtract, itemeventextract.Key); !reflect.DeepEqual(got, itemEventExtractChain) {
|
||||
t.Fatalf("item event extract validator chain = %#v, want %#v", got, itemEventExtractChain)
|
||||
}
|
||||
if got := registries.ValidatorChains.Validators(pipeline.StageNormalize, itemeventnormalize.Key); !reflect.DeepEqual(got, itemEventNormalizeChain) {
|
||||
t.Fatalf("item event normalize validator chain = %#v, want %#v", got, itemEventNormalizeChain)
|
||||
}
|
||||
|
||||
assetNames := productionAssetNames(t, components.assets.PromptFS)
|
||||
requiredAssets := []string{
|
||||
@@ -119,6 +148,14 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
||||
"dnd.combat_turns/sharedassets/common-dnd-system.md",
|
||||
"dnd.combat_turns/sharedassets/common-dnd-transcript.md",
|
||||
"dnd.combat_turns/task.md",
|
||||
"dnd.item_events/dnd.item_events.yaml",
|
||||
"dnd.item_events/instructions.md",
|
||||
"dnd.item_events/sharedassets/common-dnd-extraction-evidence.md",
|
||||
"dnd.item_events/sharedassets/common-dnd-identity.md",
|
||||
"dnd.item_events/sharedassets/common-dnd-references.md",
|
||||
"dnd.item_events/sharedassets/common-dnd-system.md",
|
||||
"dnd.item_events/sharedassets/common-dnd-transcript.md",
|
||||
"dnd.item_events/task.md",
|
||||
}
|
||||
assertProductionContains(t, "production prompt assets", assetNames, requiredAssets)
|
||||
|
||||
@@ -135,6 +172,10 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
||||
if !ok || combatCodecSpec.Kind != dnd.CombatTurnListKind || combatCodecSpec.Schema.ID != combatcodec.SchemaID {
|
||||
t.Fatalf("combat codec spec = %#v, ok=%t, want typed D&D combat codec", combatCodecSpec, ok)
|
||||
}
|
||||
itemEventCodecSpec, ok := catalog.ArtifactCodecs.Spec(dnd.ItemEventListKind)
|
||||
if !ok || itemEventCodecSpec.Kind != dnd.ItemEventListKind || itemEventCodecSpec.Schema.ID != itemeventcodec.SchemaID {
|
||||
t.Fatalf("item event codec spec = %#v, ok=%t, want typed D&D item-event codec", itemEventCodecSpec, ok)
|
||||
}
|
||||
if got := catalog.ValidatorChains.Validators(pipeline.StageExtract, spells.Key); !reflect.DeepEqual(got, wantChain) {
|
||||
t.Fatalf("catalog validator chain = %#v, want %#v", got, wantChain)
|
||||
}
|
||||
@@ -520,8 +561,8 @@ func TestProductionSceneRunRecordsAnnotationFreeChunkPlanAndProvenance(t *testin
|
||||
if len(warnings.Warnings) != 0 {
|
||||
t.Fatalf("warnings = %#v, want none", warnings.Warnings)
|
||||
}
|
||||
if len(fake.requestsFor(scenes.PromptID)) != 1 || len(fake.requestsFor(spells.PromptID)) != 1 {
|
||||
t.Fatalf("fake prompt requests = %#v, want one scene and one spell request", fake.requestPrompts())
|
||||
if len(fake.requestsFor(scenes.PromptID)) != 1 || len(fake.requestsFor(spells.PromptID)) != 1 || len(fake.requestsFor(itemeventextract.PromptID)) != 1 {
|
||||
t.Fatalf("fake prompt requests = %#v, want one scene, spell, and item-event request", fake.requestPrompts())
|
||||
}
|
||||
}
|
||||
|
||||
@@ -658,6 +699,8 @@ pipelines:
|
||||
artifacts:
|
||||
spells:
|
||||
extract: dnd/spells
|
||||
item-events:
|
||||
extract: dnd/item-events
|
||||
`, outputRoot, filepath.Join(filepath.Dir(outputRoot), "debug"), chunkModule)
|
||||
}
|
||||
|
||||
@@ -756,6 +799,8 @@ func (client *productionFakeLLMClient) CompleteStructured(ctx context.Context, r
|
||||
} else {
|
||||
content = []byte(`{"spell_casts":[{"caster":"Aria","spell":"Cure Wounds","source_refs":[{"start_unit_id":1,"end_unit_id":1}]}]}`)
|
||||
}
|
||||
case itemeventextract.PromptID:
|
||||
content = []byte(`{"events":[{"name":"Cure Wounds","kind":"acquired","to":"party","source_refs":[{"start_segment":1,"end_segment":1}]}]}`)
|
||||
default:
|
||||
return contracts.StructuredCompletionResponse{}, fmt.Errorf("unexpected prompt %q", req.PromptID)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "notarius.dnd.item_events",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["events"],
|
||||
"properties": {
|
||||
"events": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["name", "kind", "source_refs"],
|
||||
"properties": {
|
||||
"name": {"type": "string", "minLength": 1},
|
||||
"kind": {"type": "string", "enum": ["discovered", "acquired", "lost", "consumed", "transferred"]},
|
||||
"quantity": {"type": "integer", "minimum": 1},
|
||||
"from": {"type": "string", "minLength": 1},
|
||||
"to": {"type": "string", "minLength": 1},
|
||||
"source_refs": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["source_id", "start_unit_id", "end_unit_id"],
|
||||
"properties": {
|
||||
"source_id": {"type": "string", "minLength": 1},
|
||||
"start_unit_id": {"type": "integer", "minimum": 1},
|
||||
"end_unit_id": {"type": "integer", "minimum": 1}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"allOf": [
|
||||
{"if": {"properties": {"kind": {"const": "discovered"}}, "required": ["kind"]}, "then": {"not": {"anyOf": [{"required": ["from"]}, {"required": ["to"]}]}}},
|
||||
{"if": {"properties": {"kind": {"const": "acquired"}}, "required": ["kind"]}, "then": {"required": ["to"], "not": {"required": ["from"]}}},
|
||||
{"if": {"properties": {"kind": {"const": "lost"}}, "required": ["kind"]}, "then": {"required": ["from"], "not": {"required": ["to"]}}},
|
||||
{"if": {"properties": {"kind": {"const": "consumed"}}, "required": ["kind"]}, "then": {"required": ["from"], "not": {"required": ["to"]}}},
|
||||
{"if": {"properties": {"kind": {"const": "transferred"}}, "required": ["kind"]}, "then": {"required": ["from", "to"]}}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
146
internal/modules/dnd/codec/itemevents/codec.go
Normal file
146
internal/modules/dnd/codec/itemevents/codec.go
Normal file
@@ -0,0 +1,146 @@
|
||||
// Package itemevents encodes durable D&D item-event artifacts.
|
||||
package itemevents
|
||||
|
||||
import (
|
||||
"embed"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/candidatejson"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/itemevents"
|
||||
)
|
||||
|
||||
const (
|
||||
SchemaID = "notarius.dnd.item_events"
|
||||
SchemaName = "notarius_dnd_item_events_v1"
|
||||
SchemaVersion = "v1"
|
||||
MediaType = "application/json"
|
||||
)
|
||||
|
||||
//go:embed assets/schemas/dnd_item_events.v1.json
|
||||
var schemaAssets embed.FS
|
||||
|
||||
var _ contracts.ArtifactCodec[dnd.ItemEventList] = (*Codec)(nil)
|
||||
|
||||
type Codec struct{}
|
||||
|
||||
func New() *Codec { return &Codec{} }
|
||||
|
||||
func (c *Codec) Kind() contracts.ArtifactKind { return dnd.ItemEventListKind }
|
||||
|
||||
func (c *Codec) Schema() contracts.ArtifactSchema {
|
||||
raw, err := schemaAssets.ReadFile("assets/schemas/dnd_item_events.v1.json")
|
||||
if err != nil {
|
||||
return contracts.ArtifactSchema{}
|
||||
}
|
||||
return contracts.ArtifactSchema{
|
||||
ID: SchemaID,
|
||||
Name: SchemaName,
|
||||
Version: SchemaVersion,
|
||||
JSONSchema: append([]byte(nil), raw...),
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Codec) MediaType() string { return MediaType }
|
||||
|
||||
func (c *Codec) Metadata(value dnd.ItemEventList) map[string]any {
|
||||
return map[string]any{"event_count": len(value.Events)}
|
||||
}
|
||||
|
||||
func (c *Codec) Encode(value dnd.ItemEventList) ([]byte, error) {
|
||||
if err := validate(value); err != nil {
|
||||
return nil, fmt.Errorf("encode dnd item event list: %w", err)
|
||||
}
|
||||
return c.EncodeCandidate(value)
|
||||
}
|
||||
|
||||
// EncodeCandidate provides the durable representation before semantic
|
||||
// validators have approved a value.
|
||||
func (c *Codec) EncodeCandidate(value dnd.ItemEventList) ([]byte, error) {
|
||||
return candidatejson.EncodeCandidate("dnd item event list", cloneList(value))
|
||||
}
|
||||
|
||||
func (c *Codec) Decode(content []byte) (dnd.ItemEventList, error) {
|
||||
value, err := c.DecodeCandidate(content)
|
||||
if err != nil {
|
||||
return dnd.ItemEventList{}, err
|
||||
}
|
||||
if err := validate(value); err != nil {
|
||||
return dnd.ItemEventList{}, fmt.Errorf("decode dnd item event list: %w", err)
|
||||
}
|
||||
return value, nil
|
||||
}
|
||||
|
||||
// DecodeCandidate reads one strict durable JSON value before semantic
|
||||
// validators have approved it.
|
||||
func (c *Codec) DecodeCandidate(content []byte) (dnd.ItemEventList, error) {
|
||||
value, err := candidatejson.DecodeCandidate[dnd.ItemEventList]("dnd item event list", content)
|
||||
if err != nil {
|
||||
return dnd.ItemEventList{}, err
|
||||
}
|
||||
return cloneList(value), nil
|
||||
}
|
||||
|
||||
func validate(value dnd.ItemEventList) error {
|
||||
if value.Events == nil {
|
||||
return fmt.Errorf("events must be present")
|
||||
}
|
||||
for index, event := range value.Events {
|
||||
prefix := fmt.Sprintf("events[%d]", index)
|
||||
if strings.TrimSpace(event.Name) == "" {
|
||||
return fmt.Errorf("%s.name must not be empty", prefix)
|
||||
}
|
||||
if !itemevents.SupportedKind(event.Kind) {
|
||||
return fmt.Errorf("%s.kind must be supported", prefix)
|
||||
}
|
||||
if event.From != "" && strings.TrimSpace(event.From) == "" {
|
||||
return fmt.Errorf("%s.from must not be empty when present", prefix)
|
||||
}
|
||||
if event.To != "" && strings.TrimSpace(event.To) == "" {
|
||||
return fmt.Errorf("%s.to must not be empty when present", prefix)
|
||||
}
|
||||
if !itemevents.ValidHolderCombination(event.Kind, event.From, event.To) {
|
||||
return fmt.Errorf("%s holders are incompatible with %q", prefix, event.Kind)
|
||||
}
|
||||
if event.Quantity != nil && *event.Quantity < 1 {
|
||||
return fmt.Errorf("%s.quantity must be positive when present", prefix)
|
||||
}
|
||||
if len(event.SourceRefs) == 0 {
|
||||
return fmt.Errorf("%s.source_refs must contain at least one reference", prefix)
|
||||
}
|
||||
for refIndex, ref := range event.SourceRefs {
|
||||
refPrefix := fmt.Sprintf("%s.source_refs[%d]", prefix, refIndex)
|
||||
if strings.TrimSpace(ref.SourceID) == "" {
|
||||
return fmt.Errorf("%s.source_id must not be empty", refPrefix)
|
||||
}
|
||||
if ref.StartUnitID <= 0 {
|
||||
return fmt.Errorf("%s.start_unit_id must be positive", refPrefix)
|
||||
}
|
||||
if ref.EndUnitID <= 0 {
|
||||
return fmt.Errorf("%s.end_unit_id must be positive", refPrefix)
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func cloneList(value dnd.ItemEventList) dnd.ItemEventList {
|
||||
if value.Events == nil {
|
||||
return dnd.ItemEventList{}
|
||||
}
|
||||
cloned := dnd.ItemEventList{Events: make([]dnd.ItemEvent, len(value.Events))}
|
||||
for index, event := range value.Events {
|
||||
cloned.Events[index] = event
|
||||
if event.Quantity != nil {
|
||||
quantity := *event.Quantity
|
||||
cloned.Events[index].Quantity = &quantity
|
||||
}
|
||||
if event.SourceRefs != nil {
|
||||
cloned.Events[index].SourceRefs = append([]source.SourceRef(nil), event.SourceRefs...)
|
||||
}
|
||||
}
|
||||
return cloned
|
||||
}
|
||||
147
internal/modules/dnd/codec/itemevents/codec_test.go
Normal file
147
internal/modules/dnd/codec/itemevents/codec_test.go
Normal file
@@ -0,0 +1,147 @@
|
||||
package itemevents
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
)
|
||||
|
||||
func validList() dnd.ItemEventList {
|
||||
quantity := 12
|
||||
return dnd.ItemEventList{Events: []dnd.ItemEvent{
|
||||
{Name: "Hidden Cache", Kind: dnd.ItemEventKindDiscovered, SourceRefs: refs(1, 1)},
|
||||
{Name: "Gold Pieces", Kind: dnd.ItemEventKindAcquired, Quantity: &quantity, To: "party", SourceRefs: refs(2, 2)},
|
||||
{Name: "Torch", Kind: dnd.ItemEventKindLost, From: "party", SourceRefs: refs(3, 3)},
|
||||
{Name: "Healing Potion", Kind: dnd.ItemEventKindConsumed, From: "party", SourceRefs: refs(4, 4)},
|
||||
{Name: "Moonblade", Kind: dnd.ItemEventKindTransferred, From: "Aria", To: "Borin", SourceRefs: refs(5, 5)},
|
||||
}}
|
||||
}
|
||||
|
||||
func refs(start, end int) []source.SourceRef {
|
||||
return []source.SourceRef{{SourceID: "session", StartUnitID: start, EndUnitID: end}}
|
||||
}
|
||||
|
||||
func TestCodecRoundTripAndIdentities(t *testing.T) {
|
||||
codec := New()
|
||||
value := validList()
|
||||
content, err := codec.Encode(value)
|
||||
if err != nil {
|
||||
t.Fatalf("Encode() error = %v", err)
|
||||
}
|
||||
decoded, err := codec.Decode(content)
|
||||
if err != nil || !reflect.DeepEqual(decoded, value) {
|
||||
t.Fatalf("Decode() = %#v, %v; want %#v", decoded, err, value)
|
||||
}
|
||||
|
||||
schema := codec.Schema()
|
||||
if codec.Kind() != dnd.ItemEventListKind || codec.MediaType() != MediaType || schema.ID != SchemaID || schema.Name != SchemaName || schema.Version != SchemaVersion || !json.Valid(schema.JSONSchema) {
|
||||
t.Fatalf("codec identity/schema = %q/%q %#v", codec.Kind(), codec.MediaType(), schema)
|
||||
}
|
||||
registry := pipeline.NewArtifactCodecRegistry()
|
||||
if err := pipeline.RegisterArtifactCodec(registry, codec); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
spec, ok := registry.Spec(dnd.ItemEventListKind)
|
||||
if !ok || spec.SchemaDigest != contracts.DigestArtifactSchema(schema) {
|
||||
t.Fatalf("registered spec = %#v, %t", spec, ok)
|
||||
}
|
||||
if _, err := registry.Encode(dnd.ItemEventListKind, dnd.NPCList{}); err == nil {
|
||||
t.Fatal("Encode() error = nil, want exact type rejection")
|
||||
} else {
|
||||
var typeErr *pipeline.ArtifactCodecTypeError
|
||||
if !errors.As(err, &typeErr) {
|
||||
t.Fatalf("Encode() error = %T, want ArtifactCodecTypeError", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecSupportsEmptyListAndPreservesInvalidCandidates(t *testing.T) {
|
||||
codec := New()
|
||||
empty := dnd.ItemEventList{Events: []dnd.ItemEvent{}}
|
||||
if content, err := codec.Encode(empty); err != nil || string(content) != `{"events":[]}` {
|
||||
t.Fatalf("Encode() = %s, %v", content, err)
|
||||
}
|
||||
zero := 0
|
||||
candidate := dnd.ItemEventList{Events: []dnd.ItemEvent{{
|
||||
Name: " ", Kind: dnd.ItemEventKindTransferred, Quantity: &zero, From: "party", To: "Party",
|
||||
SourceRefs: []source.SourceRef{{SourceID: "", StartUnitID: 0, EndUnitID: -1}},
|
||||
}}}
|
||||
content, err := codec.EncodeCandidate(candidate)
|
||||
if err != nil || !json.Valid(content) {
|
||||
t.Fatalf("EncodeCandidate() = %s, %v", content, err)
|
||||
}
|
||||
decoded, err := codec.DecodeCandidate(content)
|
||||
if err != nil || !reflect.DeepEqual(decoded, candidate) {
|
||||
t.Fatalf("DecodeCandidate() = %#v, %v; want %#v", decoded, err, candidate)
|
||||
}
|
||||
if _, err := codec.Decode(content); err == nil {
|
||||
t.Fatal("Decode() error = nil, want semantic candidate rejection")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecRejectsStrictJSONAndApprovedBoundaries(t *testing.T) {
|
||||
validJSON := `{"events":[{"name":"Ring","kind":"acquired","to":"party","source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]}]}`
|
||||
tests := []struct {
|
||||
name, raw, want string
|
||||
}{
|
||||
{"malformed", `{`, "decode dnd item event list"},
|
||||
{"unknown top level", `{"events":[],"unexpected":true}`, "unknown field"},
|
||||
{"unknown event field", strings.Replace(validJSON, `"to":"party"`, `"to":"party","unexpected":true`, 1), "unknown field"},
|
||||
{"unknown reference field", strings.Replace(validJSON, `"end_unit_id":1`, `"end_unit_id":1,"unexpected":true`, 1), "unknown field"},
|
||||
{"trailing", `{"events":[]} {}`, "multiple JSON values"},
|
||||
{"missing list", `{}`, "events must be present"},
|
||||
{"zero quantity", strings.Replace(validJSON, `"to":"party"`, `"quantity":0,"to":"party"`, 1), "quantity must be positive"},
|
||||
{"negative quantity", strings.Replace(validJSON, `"to":"party"`, `"quantity":-1,"to":"party"`, 1), "quantity must be positive"},
|
||||
{"party transfer", strings.Replace(validJSON, `"kind":"acquired","to":"party"`, `"kind":"transferred","from":"party","to":"Borin"`, 1), "holders are incompatible"},
|
||||
{"self transfer", strings.Replace(validJSON, `"kind":"acquired","to":"party"`, `"kind":"transferred","from":"Aria","to":"aria"`, 1), "holders are incompatible"},
|
||||
{"missing source refs", strings.Replace(validJSON, `,"source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]`, "", 1), "source_refs must contain"},
|
||||
{"empty source refs", strings.Replace(validJSON, `[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]`, `[]`, 1), "source_refs must contain"},
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if _, err := New().Decode([]byte(test.raw)); err == nil || !strings.Contains(err.Error(), test.want) {
|
||||
t.Fatalf("Decode() error = %v, want %q", err, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecDeepCopiesBoundaryValuesAndMetadata(t *testing.T) {
|
||||
codec := New()
|
||||
value := validList()
|
||||
content, err := codec.EncodeCandidate(value)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
decoded, err := codec.DecodeCandidate(content)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if decoded.Events[1].Quantity == value.Events[1].Quantity || &decoded.Events[1].SourceRefs[0] == &value.Events[1].SourceRefs[0] {
|
||||
t.Fatal("DecodeCandidate() retained caller-owned event fields")
|
||||
}
|
||||
*decoded.Events[1].Quantity = 99
|
||||
decoded.Events[1].SourceRefs[0].SourceID = "changed"
|
||||
if *value.Events[1].Quantity != 12 || value.Events[1].SourceRefs[0].SourceID != "session" {
|
||||
t.Fatal("decoded item event aliases input")
|
||||
}
|
||||
|
||||
first := codec.Schema()
|
||||
first.JSONSchema[0] = '['
|
||||
if second := codec.Schema(); !json.Valid(second.JSONSchema) || second.JSONSchema[0] == '[' {
|
||||
t.Fatal("Schema() returned shared bytes")
|
||||
}
|
||||
metadata := codec.Metadata(value)
|
||||
metadata["payload"] = bytes.Repeat([]byte("x"), 10)
|
||||
if next := codec.Metadata(value); len(next) != 1 || next["event_count"] != len(value.Events) {
|
||||
t.Fatalf("Metadata() = %#v", next)
|
||||
}
|
||||
}
|
||||
@@ -9,17 +9,21 @@ import (
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
npcregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/registry"
|
||||
sceneregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/scenedescriptions/registry"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
const (
|
||||
Key = "dnd/combat-turns"
|
||||
mappingPolicy = "dnd.combat_turns.extract_mapping.v2"
|
||||
sceneGatePolicy = "dnd.combat_turns.scene_gate.v1"
|
||||
)
|
||||
|
||||
const (
|
||||
NPCRegistryReferenceSlot = npcregistry.ReferenceSlot
|
||||
NPCRegistryMaxBytes = npcregistry.MaxBytes
|
||||
SceneDescriptionReferenceSlot = sceneregistry.ReferenceSlot
|
||||
SceneDescriptionMaxBytes = sceneregistry.MaxBytes
|
||||
)
|
||||
|
||||
var requiredCapabilities = []string{
|
||||
@@ -47,6 +51,14 @@ func referenceSlots() []contracts.ReferenceSlot {
|
||||
AcceptedArtifactKinds: []contracts.ArtifactKind{dnd.NPCListKind},
|
||||
MaxBytes: NPCRegistryMaxBytes,
|
||||
})
|
||||
slots = append(slots, contracts.ReferenceSlot{
|
||||
Name: SceneDescriptionReferenceSlot,
|
||||
Description: "Required scene descriptions that gate combat eligibility and are not combat evidence.",
|
||||
Required: true,
|
||||
AcceptedMediaTypes: []string{"application/json"},
|
||||
AcceptedArtifactKinds: []contracts.ArtifactKind{dnd.SceneDescriptionListKind},
|
||||
MaxBytes: SceneDescriptionMaxBytes,
|
||||
})
|
||||
sort.Slice(slots, func(i, j int) bool { return slots[i].Name < slots[j].Name })
|
||||
return slots
|
||||
}
|
||||
@@ -60,6 +72,7 @@ type Options struct{}
|
||||
type Extractor struct {
|
||||
llm contracts.StructuredLLMClient
|
||||
npcResolver *npcregistry.Resolver
|
||||
sceneResolver *sceneregistry.Resolver
|
||||
promptSHA string
|
||||
responseSchemaSHA string
|
||||
}
|
||||
@@ -79,6 +92,10 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
|
||||
if err != nil {
|
||||
return nil, extractorErrorf("prepare NPC registry prompt input: %w", err)
|
||||
}
|
||||
sceneResolver, err := sceneregistry.NewResolver(referenceSet)
|
||||
if err != nil {
|
||||
return nil, extractorErrorf("prepare scene eligibility: %w", err)
|
||||
}
|
||||
promptSHA, err := scriptoriumPromptMetadata()
|
||||
if err != nil {
|
||||
return nil, extractorErrorf("load prompt metadata: %w", err)
|
||||
@@ -90,6 +107,7 @@ func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contr
|
||||
return &Extractor{
|
||||
llm: llmClient,
|
||||
npcResolver: npcResolver,
|
||||
sceneResolver: sceneResolver,
|
||||
promptSHA: promptSHA,
|
||||
responseSchemaSHA: responseSchema.SHA256,
|
||||
}, nil
|
||||
@@ -108,6 +126,7 @@ func (e *Extractor) ManifestMetadata() map[string]any {
|
||||
"prompt_version": SchemaVersion,
|
||||
"prompt_sha256": e.promptSHA,
|
||||
"mapping_policy": mappingPolicy,
|
||||
"scene_gate_policy": sceneGatePolicy,
|
||||
"response_schema_key": string(ResponseSchemaKey),
|
||||
"response_schema_id": ResponseSchemaID,
|
||||
"response_schema_name": ResponseSchemaName,
|
||||
@@ -119,6 +138,11 @@ func (e *Extractor) ManifestMetadata() map[string]any {
|
||||
metadata["npc_registry_digest"] = seeded.Digest()
|
||||
metadata["npc_count"] = seeded.Count()
|
||||
}
|
||||
seededScenes := e.sceneResolver.Seeded()
|
||||
if seededScenes.Bound() {
|
||||
metadata["scene_eligibility_digest"] = seededScenes.EligibilityDigest()
|
||||
metadata["scene_description_count"] = seededScenes.Count()
|
||||
}
|
||||
return metadata
|
||||
}
|
||||
|
||||
@@ -130,9 +154,12 @@ func (e *Extractor) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
|
||||
{Name: "prompt", Value: e.promptSHA},
|
||||
{Name: "response_schema", Value: e.responseSchemaSHA},
|
||||
{Name: "mapping_policy", Value: mappingPolicy},
|
||||
{Name: "scene_gate_policy", Value: sceneGatePolicy},
|
||||
}
|
||||
seeded := e.npcResolver.Seeded()
|
||||
fingerprints = append(fingerprints, pipeline.CheckpointFingerprint{Name: "npc_registry", Value: seeded.ProjectionDigest()})
|
||||
seededScenes := e.sceneResolver.Seeded()
|
||||
fingerprints = append(fingerprints, pipeline.CheckpointFingerprint{Name: "scene_eligibility", Value: seededScenes.EligibilityDigest()})
|
||||
return fingerprints
|
||||
}
|
||||
|
||||
@@ -147,6 +174,31 @@ func (e *Extractor) Extract(ctx context.Context, req contracts.TypedExtractionRe
|
||||
if err != nil {
|
||||
return contracts.TypedExtractionResult[dnd.CombatTurnList]{}, extractorErrorf("%w", err)
|
||||
}
|
||||
if _, ok := req.References.Slots[SceneDescriptionReferenceSlot]; !ok {
|
||||
return contracts.TypedExtractionResult[dnd.CombatTurnList]{}, extractorErrorf("required scene descriptions reference is missing")
|
||||
}
|
||||
scenes, err := e.sceneResolver.Resolve(req.References)
|
||||
if err != nil {
|
||||
return contracts.TypedExtractionResult[dnd.CombatTurnList]{}, extractorErrorf("resolve scene eligibility: %w", err)
|
||||
}
|
||||
match := scenes.Match(req.Chunk)
|
||||
switch match.State {
|
||||
case sceneregistry.MatchExact:
|
||||
if match.Kind != dnd.SceneKindCombat {
|
||||
return contracts.TypedExtractionResult[dnd.CombatTurnList]{Value: dnd.CombatTurnList{CombatTurns: []dnd.CombatTurn{}}}, nil
|
||||
}
|
||||
case sceneregistry.MatchMissing, sceneregistry.MatchMismatched:
|
||||
return contracts.TypedExtractionResult[dnd.CombatTurnList]{
|
||||
Value: dnd.CombatTurnList{CombatTurns: []dnd.CombatTurn{}},
|
||||
Warnings: []contracts.Warning{{
|
||||
Scope: SceneDescriptionReferenceSlot,
|
||||
ReasonCode: "scene_classification_unavailable",
|
||||
Message: "No exact scene classification was available; combat extraction was skipped.",
|
||||
}},
|
||||
}, nil
|
||||
default:
|
||||
return contracts.TypedExtractionResult[dnd.CombatTurnList]{}, extractorErrorf("unsupported scene eligibility match state %q", match.State)
|
||||
}
|
||||
order := shared.NewSourceRefOrder(req.Source)
|
||||
npcRegistry, err := e.npcResolver.Resolve(req.References)
|
||||
if err != nil {
|
||||
|
||||
@@ -13,6 +13,7 @@ import (
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs"
|
||||
scenecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity"
|
||||
)
|
||||
|
||||
@@ -34,7 +35,7 @@ func TestExtractMapsAndOrdersCombatTurnsBySourcePosition(t *testing.T) {
|
||||
},
|
||||
}}}
|
||||
|
||||
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
|
||||
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest(t))
|
||||
if err != nil {
|
||||
t.Fatalf("Extract() error = %v, want nil", err)
|
||||
}
|
||||
@@ -57,7 +58,7 @@ func TestExtractMapsAndOrdersCombatTurnsBySourcePosition(t *testing.T) {
|
||||
t.Fatalf("LLM request identity = %#v", request)
|
||||
}
|
||||
transcript := request.Inputs["transcript"]
|
||||
if transcript.Name != "transcript" || transcript.MediaType != "application/json" || transcript.Digest != "sha256:chunk" || transcript.OriginURI != "file:///session-alpha.json" || !reflect.DeepEqual(transcript.Content, extractionRequest().Chunk.Content) {
|
||||
if transcript.Name != "transcript" || transcript.MediaType != "application/json" || transcript.Digest != "sha256:chunk" || transcript.OriginURI != "file:///session-alpha.json" || !reflect.DeepEqual(transcript.Content, extractionRequest(t).Chunk.Content) {
|
||||
t.Fatalf("transcript input = %#v, want chunk-scoped source input", transcript)
|
||||
}
|
||||
}
|
||||
@@ -69,7 +70,7 @@ func TestExtractPreservesInvalidCandidatesForValidators(t *testing.T) {
|
||||
SourceRefs: []combatSourceRefResponse{{StartUnitID: 99, EndUnitID: 0}},
|
||||
},
|
||||
}}}
|
||||
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
|
||||
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest(t))
|
||||
if err != nil {
|
||||
t.Fatalf("Extract() error = %v, want nil for candidate values", err)
|
||||
}
|
||||
@@ -93,10 +94,11 @@ func TestExtractUsesDocumentOrderForReferencesAndTurns(t *testing.T) {
|
||||
}},
|
||||
{Actor: "Second", TurnKind: "other", SourceRefs: []combatSourceRefResponse{{StartUnitID: 30, EndUnitID: 30}}},
|
||||
}}}
|
||||
req := extractionRequest()
|
||||
req := extractionRequest(t)
|
||||
req.Source.Units = []source.SourceUnit{{ID: 30}, {ID: 10}}
|
||||
req.Chunk.Units = append([]source.SourceUnit(nil), req.Source.Units...)
|
||||
req.Chunk.Ref = source.SourceRef{SourceID: req.Source.ID, StartUnitID: 30, EndUnitID: 10}
|
||||
req.References = sceneReferences(t, req.Chunk, dnd.SceneKindCombat)
|
||||
|
||||
result, err := newExtractor(t, client).Extract(context.Background(), req)
|
||||
if err != nil {
|
||||
@@ -127,7 +129,10 @@ func TestExtractPassesReferencesAndNPCGroundingWithoutUsingItAsEvidence(t *testi
|
||||
"glossary": {Slot: contracts.ReferenceSlot{Name: "glossary"}, Items: []contracts.ReferenceItem{{SlotName: "glossary", Content: []byte("ogre: a large foe")}}},
|
||||
NPCRegistryReferenceSlot: {Slot: contracts.ReferenceSlot{Name: NPCRegistryReferenceSlot}, Items: []contracts.ReferenceItem{{SlotName: NPCRegistryReferenceSlot, MediaType: "application/json", Content: npcRegistryJSON(t)}}},
|
||||
}}
|
||||
req := extractionRequest()
|
||||
req := extractionRequest(t)
|
||||
for name, slot := range req.References.Slots {
|
||||
references.Slots[name] = slot
|
||||
}
|
||||
req.References = references
|
||||
if _, err := newExtractor(t, client, references).Extract(context.Background(), req); err != nil {
|
||||
t.Fatalf("Extract() error = %v, want nil", err)
|
||||
@@ -143,14 +148,20 @@ func TestExtractPassesReferencesAndNPCGroundingWithoutUsingItAsEvidence(t *testi
|
||||
if strings.Contains(string(inputs["transcript"].Content), "Aria: cleric") {
|
||||
t.Fatal("transcript input contains reference content")
|
||||
}
|
||||
if _, ok := inputs[SceneDescriptionReferenceSlot]; ok {
|
||||
t.Fatalf("prompt inputs include scene descriptions: %#v", inputs)
|
||||
}
|
||||
|
||||
metadata := newExtractor(t, &fakeCombatTurnsLLMClient{}, references).ManifestMetadata()
|
||||
if metadata["npc_count"] != 1 || !strings.HasPrefix(metadata["npc_registry_digest"].(string), "sha256:") {
|
||||
t.Fatalf("bound registry metadata = %#v, want digest and count", metadata)
|
||||
}
|
||||
if metadata["scene_description_count"] != 1 || !strings.HasPrefix(metadata["scene_eligibility_digest"].(string), "sha256:") {
|
||||
t.Fatalf("bound scene metadata = %#v, want digest and count", metadata)
|
||||
}
|
||||
fingerprints := newExtractor(t, &fakeCombatTurnsLLMClient{}, references).CheckpointFingerprints()
|
||||
if len(fingerprints) != 4 || fingerprints[3].Name != "npc_registry" {
|
||||
t.Fatalf("bound fingerprints = %#v, want local identities plus NPC registry", fingerprints)
|
||||
if len(fingerprints) != 6 || fingerprints[3].Name != "scene_gate_policy" || fingerprints[4].Name != "npc_registry" || fingerprints[5].Name != "scene_eligibility" {
|
||||
t.Fatalf("bound fingerprints = %#v, want local identities plus scene gate and NPC registry", fingerprints)
|
||||
}
|
||||
encoded, err := json.Marshal(metadata)
|
||||
if err != nil {
|
||||
@@ -164,7 +175,7 @@ func TestExtractPassesReferencesAndNPCGroundingWithoutUsingItAsEvidence(t *testi
|
||||
func TestExtractUnboundRegistryUsesExactEmptyPromptAndOmitsIdentity(t *testing.T) {
|
||||
client := &fakeCombatTurnsLLMClient{response: extractionResponse{CombatTurns: []combatTurnResponse{}}}
|
||||
extractor := newExtractor(t, client)
|
||||
if _, err := extractor.Extract(context.Background(), extractionRequest()); err != nil {
|
||||
if _, err := extractor.Extract(context.Background(), extractionRequest(t)); err != nil {
|
||||
t.Fatalf("Extract() error = %v, want nil", err)
|
||||
}
|
||||
input := client.requests[0].Inputs[NPCRegistryReferenceSlot]
|
||||
@@ -175,7 +186,7 @@ func TestExtractUnboundRegistryUsesExactEmptyPromptAndOmitsIdentity(t *testing.T
|
||||
if _, ok := metadata["npc_registry_digest"]; ok {
|
||||
t.Fatalf("unbound metadata has registry digest: %#v", metadata)
|
||||
}
|
||||
if fingerprints := extractor.CheckpointFingerprints(); len(fingerprints) != 4 || fingerprints[3].Name != "npc_registry" || fingerprints[3].Value != input.Digest {
|
||||
if fingerprints := extractor.CheckpointFingerprints(); len(fingerprints) != 6 || fingerprints[4].Name != "npc_registry" || fingerprints[4].Value != input.Digest || fingerprints[5].Name != "scene_eligibility" || fingerprints[5].Value == "" {
|
||||
t.Fatalf("unbound fingerprints = %#v, want empty-projection identity", fingerprints)
|
||||
}
|
||||
}
|
||||
@@ -184,13 +195,11 @@ func TestExtractorResolvesOperationNPCOverrideWithoutSingletonMetadata(t *testin
|
||||
content := npcRegistryJSON(t)
|
||||
client := &fakeCombatTurnsLLMClient{response: extractionResponse{CombatTurns: []combatTurnResponse{}}}
|
||||
extractor := newExtractor(t, client)
|
||||
request := extractionRequest()
|
||||
request.References = contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
|
||||
NPCRegistryReferenceSlot: {
|
||||
request := extractionRequest(t)
|
||||
request.References.Slots[NPCRegistryReferenceSlot] = contracts.ResolvedReferenceSlot{
|
||||
Slot: contracts.ReferenceSlot{Name: NPCRegistryReferenceSlot},
|
||||
Items: []contracts.ReferenceItem{{SlotName: NPCRegistryReferenceSlot, MediaType: npccodec.MediaType, Content: append([]byte(nil), content...), Origin: contracts.ReferenceOrigin{Type: "generated"}}},
|
||||
},
|
||||
}}
|
||||
}
|
||||
if _, err := extractor.Extract(context.Background(), request); err != nil {
|
||||
t.Fatalf("Extract() error = %v", err)
|
||||
}
|
||||
@@ -198,11 +207,151 @@ func TestExtractorResolvesOperationNPCOverrideWithoutSingletonMetadata(t *testin
|
||||
if input.Digest == "" || string(input.Content) != `{"npcs":[{"name":"Mira Thorn"}]}` || input.OriginURI != "" {
|
||||
t.Fatalf("operation NPC input = %#v, want generated canonical grounding without provenance", input)
|
||||
}
|
||||
if metadata := extractor.ManifestMetadata(); metadata["npc_registry_digest"] != nil || metadata["npc_count"] != nil {
|
||||
if metadata := extractor.ManifestMetadata(); metadata["npc_registry_digest"] != nil || metadata["npc_count"] != nil || metadata["scene_eligibility_digest"] != nil || metadata["scene_description_count"] != nil {
|
||||
t.Fatalf("singleton metadata = %#v, want no operation-varying NPC identity", metadata)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractAppliesSceneEligibilityBeforePromptConstruction(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
scenes []dnd.SceneDescription
|
||||
wantWarning string
|
||||
wantError string
|
||||
wantCalls int
|
||||
wantEmptyList bool
|
||||
}{
|
||||
{
|
||||
name: "exact combat",
|
||||
scenes: []dnd.SceneDescription{sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindCombat)},
|
||||
wantCalls: 1,
|
||||
},
|
||||
{
|
||||
name: "exact narrative",
|
||||
scenes: []dnd.SceneDescription{sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindNarrative)},
|
||||
wantEmptyList: true,
|
||||
},
|
||||
{
|
||||
name: "exact recap",
|
||||
scenes: []dnd.SceneDescription{sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindRecap)},
|
||||
wantEmptyList: true,
|
||||
},
|
||||
{
|
||||
name: "exact meta",
|
||||
scenes: []dnd.SceneDescription{sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindMeta)},
|
||||
wantEmptyList: true,
|
||||
},
|
||||
{
|
||||
name: "missing ID",
|
||||
scenes: []dnd.SceneDescription{func() dnd.SceneDescription {
|
||||
value := sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindCombat)
|
||||
value.ID = "other-chunk"
|
||||
return value
|
||||
}()},
|
||||
wantWarning: "scene_classification_unavailable",
|
||||
wantEmptyList: true,
|
||||
},
|
||||
{
|
||||
name: "source mismatch",
|
||||
scenes: []dnd.SceneDescription{func() dnd.SceneDescription {
|
||||
value := sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindCombat)
|
||||
value.SourceRef.SourceID = "other-source"
|
||||
return value
|
||||
}()},
|
||||
wantWarning: "scene_classification_unavailable",
|
||||
wantEmptyList: true,
|
||||
},
|
||||
{
|
||||
name: "start mismatch",
|
||||
scenes: []dnd.SceneDescription{func() dnd.SceneDescription {
|
||||
value := sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindCombat)
|
||||
value.SourceRef.StartUnitID = 2
|
||||
return value
|
||||
}()},
|
||||
wantWarning: "scene_classification_unavailable",
|
||||
wantEmptyList: true,
|
||||
},
|
||||
{
|
||||
name: "end mismatch",
|
||||
scenes: []dnd.SceneDescription{func() dnd.SceneDescription {
|
||||
value := sceneDescription(extractionRequest(t).Chunk, dnd.SceneKindCombat)
|
||||
value.SourceRef.EndUnitID = 10
|
||||
return value
|
||||
}()},
|
||||
wantWarning: "scene_classification_unavailable",
|
||||
wantEmptyList: true,
|
||||
},
|
||||
{
|
||||
name: "missing required reference",
|
||||
wantError: "required scene descriptions reference is missing",
|
||||
},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
client := &fakeCombatTurnsLLMClient{response: extractionResponse{CombatTurns: []combatTurnResponse{}}}
|
||||
req := extractionRequest(t)
|
||||
if test.wantError != "" {
|
||||
req.References = contracts.ReferenceSet{}
|
||||
} else {
|
||||
req.References = sceneListReferences(t, test.scenes...)
|
||||
}
|
||||
|
||||
result, err := newExtractor(t, client).Extract(context.Background(), req)
|
||||
if test.wantError != "" {
|
||||
if err == nil || !strings.Contains(err.Error(), test.wantError) {
|
||||
t.Fatalf("Extract() error = %v, want %q", err, test.wantError)
|
||||
}
|
||||
} else if err != nil {
|
||||
t.Fatalf("Extract() error = %v", err)
|
||||
}
|
||||
if len(client.requests) != test.wantCalls {
|
||||
t.Fatalf("LLM calls = %d, want %d", len(client.requests), test.wantCalls)
|
||||
}
|
||||
if test.wantEmptyList && result.Value.CombatTurns == nil {
|
||||
t.Fatal("CombatTurns = nil, want accepted non-nil empty list")
|
||||
}
|
||||
if test.wantWarning != "" {
|
||||
if len(result.Warnings) != 1 || result.Warnings[0].Scope != SceneDescriptionReferenceSlot || result.Warnings[0].ReasonCode != test.wantWarning {
|
||||
t.Fatalf("warnings = %#v", result.Warnings)
|
||||
}
|
||||
} else if len(result.Warnings) != 0 {
|
||||
t.Fatalf("warnings = %#v, want none", result.Warnings)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSceneEligibilityMetadataAndFingerprintsTrackGatingValues(t *testing.T) {
|
||||
chunk := extractionRequest(t).Chunk
|
||||
base := sceneDescription(chunk, dnd.SceneKindCombat)
|
||||
baseExtractor := newExtractor(t, &fakeCombatTurnsLLMClient{}, sceneListReferences(t, base))
|
||||
metadata := baseExtractor.ManifestMetadata()
|
||||
if metadata["scene_gate_policy"] != sceneGatePolicy || metadata["scene_description_count"] != 1 {
|
||||
t.Fatalf("metadata = %#v, want scene gate policy and count", metadata)
|
||||
}
|
||||
baseDigest := fingerprintValue(t, baseExtractor.CheckpointFingerprints(), "scene_eligibility")
|
||||
proseChanged := base
|
||||
proseChanged.Title = "Different title"
|
||||
proseChanged.Summary = "Different summary."
|
||||
if got := fingerprintValue(t, newExtractor(t, &fakeCombatTurnsLLMClient{}, sceneListReferences(t, proseChanged)).CheckpointFingerprints(), "scene_eligibility"); got != baseDigest {
|
||||
t.Fatalf("prose-only eligibility digest = %q, want %q", got, baseDigest)
|
||||
}
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
change func(*dnd.SceneDescription)
|
||||
}{
|
||||
{"kind", func(value *dnd.SceneDescription) { value.Kind = dnd.SceneKindNarrative }},
|
||||
{"range", func(value *dnd.SceneDescription) { value.SourceRef.StartUnitID = 2 }},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
changed := base
|
||||
test.change(&changed)
|
||||
if got := fingerprintValue(t, newExtractor(t, &fakeCombatTurnsLLMClient{}, sceneListReferences(t, changed)).CheckpointFingerprints(), "scene_eligibility"); got == baseDigest {
|
||||
t.Fatal("eligibility digest did not change")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewRejectsMalformedNPCRegistryBeforeLLMCallWithoutContent(t *testing.T) {
|
||||
client := &fakeCombatTurnsLLMClient{}
|
||||
references := contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
|
||||
@@ -220,8 +369,25 @@ func TestNewRejectsMalformedNPCRegistryBeforeLLMCallWithoutContent(t *testing.T)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewRejectsInvalidSceneReferenceBeforeLLMCallWithoutContent(t *testing.T) {
|
||||
client := &fakeCombatTurnsLLMClient{}
|
||||
references := contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
|
||||
SceneDescriptionReferenceSlot: {
|
||||
Slot: contracts.ReferenceSlot{Name: SceneDescriptionReferenceSlot},
|
||||
Items: []contracts.ReferenceItem{{SlotName: SceneDescriptionReferenceSlot, MediaType: scenecodec.MediaType, Content: []byte(`{"scenes":[]}`)}},
|
||||
},
|
||||
}}
|
||||
_, err := New(client, Options{}, references)
|
||||
if err == nil || !strings.Contains(err.Error(), "prepare scene eligibility") || strings.Contains(err.Error(), `{"scenes":[]}`) {
|
||||
t.Fatalf("New() error = %v, want bounded content-free scene failure", err)
|
||||
}
|
||||
if len(client.requests) != 0 {
|
||||
t.Fatalf("LLM calls = %d, want none during failed construction", len(client.requests))
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractRejectsInvalidRequestsAndWrapsProviderFailures(t *testing.T) {
|
||||
validReq := extractionRequest()
|
||||
validReq := extractionRequest(t)
|
||||
validExtractor := newExtractor(t, &fakeCombatTurnsLLMClient{response: extractionResponse{CombatTurns: []combatTurnResponse{}}})
|
||||
var nilExtractor *Extractor
|
||||
tests := []struct {
|
||||
@@ -243,7 +409,7 @@ func TestExtractRejectsInvalidRequestsAndWrapsProviderFailures(t *testing.T) {
|
||||
}
|
||||
})
|
||||
}
|
||||
_, err := newExtractor(t, &fakeCombatTurnsLLMClient{err: errors.New("provider unavailable")}).Extract(context.Background(), extractionRequest())
|
||||
_, err := newExtractor(t, &fakeCombatTurnsLLMClient{err: errors.New("provider unavailable")}).Extract(context.Background(), extractionRequest(t))
|
||||
if err == nil || !strings.Contains(err.Error(), "dnd combat turns") || !strings.Contains(err.Error(), "provider unavailable") {
|
||||
t.Fatalf("provider error = %v, want contextual wrapped error", err)
|
||||
}
|
||||
@@ -267,7 +433,7 @@ func TestExtractorManifestMetadataAndFingerprints(t *testing.T) {
|
||||
t.Fatalf("metadata[%q] = %#v, want digest", key, metadata[key])
|
||||
}
|
||||
}
|
||||
wantNames := map[string]struct{}{"prompt": {}, "response_schema": {}, "mapping_policy": {}, "npc_registry": {}}
|
||||
wantNames := map[string]struct{}{"prompt": {}, "response_schema": {}, "mapping_policy": {}, "scene_gate_policy": {}, "npc_registry": {}, "scene_eligibility": {}}
|
||||
for _, fingerprint := range extractor.CheckpointFingerprints() {
|
||||
if _, ok := wantNames[fingerprint.Name]; !ok {
|
||||
t.Fatalf("unexpected fingerprint = %#v", fingerprint)
|
||||
@@ -304,7 +470,8 @@ func TestModuleSpecAndRegistration(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func extractionRequest() contracts.TypedExtractionRequest {
|
||||
func extractionRequest(t *testing.T) contracts.TypedExtractionRequest {
|
||||
t.Helper()
|
||||
doc := combatSourceDocument()
|
||||
chunk := &source.Chunk{
|
||||
ID: "session-alpha:chunk:0",
|
||||
@@ -315,13 +482,15 @@ func extractionRequest() contracts.TypedExtractionRequest {
|
||||
MediaType: "application/json",
|
||||
Units: append([]source.SourceUnit(nil), doc.Units...),
|
||||
}
|
||||
return contracts.TypedExtractionRequest{
|
||||
request := contracts.TypedExtractionRequest{
|
||||
Source: doc,
|
||||
Chunk: chunk,
|
||||
SourceInput: contracts.NewLLMInputMaterial("source", chunk.MediaType, chunk.Content, "sha256:chunk", "file:///session-alpha.json"),
|
||||
SessionID: "session-123",
|
||||
LLMProfile: "profile-combat",
|
||||
}
|
||||
request.References = sceneReferences(t, chunk, dnd.SceneKindCombat)
|
||||
return request
|
||||
}
|
||||
|
||||
func combatSourceDocument() *source.SourceDocument {
|
||||
@@ -358,6 +527,46 @@ func npcRegistryJSON(t *testing.T) []byte {
|
||||
return content
|
||||
}
|
||||
|
||||
func sceneReferences(t *testing.T, chunk *source.Chunk, kind dnd.SceneKind) contracts.ReferenceSet {
|
||||
t.Helper()
|
||||
return sceneListReferences(t, sceneDescription(chunk, kind))
|
||||
}
|
||||
|
||||
func sceneListReferences(t *testing.T, scenes ...dnd.SceneDescription) contracts.ReferenceSet {
|
||||
t.Helper()
|
||||
content, err := scenecodec.New().Encode(dnd.SceneDescriptionList{Scenes: scenes})
|
||||
if err != nil {
|
||||
t.Fatalf("encode scene descriptions: %v", err)
|
||||
}
|
||||
return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
|
||||
SceneDescriptionReferenceSlot: {
|
||||
Slot: contracts.ReferenceSlot{Name: SceneDescriptionReferenceSlot},
|
||||
Items: []contracts.ReferenceItem{{SlotName: SceneDescriptionReferenceSlot, MediaType: scenecodec.MediaType, Content: content}},
|
||||
},
|
||||
}}
|
||||
}
|
||||
|
||||
func sceneDescription(chunk *source.Chunk, kind dnd.SceneKind) dnd.SceneDescription {
|
||||
return dnd.SceneDescription{
|
||||
ID: chunk.ID,
|
||||
SourceRef: chunk.Ref,
|
||||
Kind: kind,
|
||||
Title: "Combat eligibility",
|
||||
Summary: "A valid scene description for combat extraction.",
|
||||
}
|
||||
}
|
||||
|
||||
func fingerprintValue(t *testing.T, fingerprints []pipeline.CheckpointFingerprint, name string) string {
|
||||
t.Helper()
|
||||
for _, fingerprint := range fingerprints {
|
||||
if fingerprint.Name == name {
|
||||
return fingerprint.Value
|
||||
}
|
||||
}
|
||||
t.Fatalf("fingerprints = %#v, missing %q", fingerprints, name)
|
||||
return ""
|
||||
}
|
||||
|
||||
type fakeCombatTurnsLLMClient struct {
|
||||
response extractionResponse
|
||||
err error
|
||||
|
||||
6
internal/modules/dnd/extract/itemevents/assets.go
Normal file
6
internal/modules/dnd/extract/itemevents/assets.go
Normal file
@@ -0,0 +1,6 @@
|
||||
package itemevents
|
||||
|
||||
import "embed"
|
||||
|
||||
//go:embed assets/schemas/dnd_item_events_llm.v1.json assets/prompts/*.yaml assets/prompts/*.md
|
||||
var embeddedAssets embed.FS
|
||||
@@ -0,0 +1,42 @@
|
||||
id: dnd.item_events
|
||||
version: "v1"
|
||||
default_profile: gemini-2-flash
|
||||
inputs:
|
||||
- name: transcript
|
||||
required: true
|
||||
content_type: application/json
|
||||
- name: players
|
||||
required: false
|
||||
content_type: text/plain
|
||||
- name: party
|
||||
required: false
|
||||
content_type: text/plain
|
||||
- name: glossary
|
||||
required: false
|
||||
content_type: text/plain
|
||||
messages:
|
||||
- role: system
|
||||
content_file: ./sharedassets/common-dnd-system.md
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-extraction-evidence.md
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-identity.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-references.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
- role: user
|
||||
content_file: ./task.md
|
||||
- role: user
|
||||
content_file: ./instructions.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-transcript.md
|
||||
output:
|
||||
format: json
|
||||
validation_mode: json_schema
|
||||
schema_path: dnd_item_events_llm.v1.json
|
||||
repair_attempts: 0
|
||||
@@ -0,0 +1,25 @@
|
||||
Return one event only when the transcript establishes a meaningful item or
|
||||
currency occurrence. Use a concise observed item name and preserve the stated
|
||||
currency denomination; include quantity only when the transcript explicitly
|
||||
states it.
|
||||
|
||||
Use `discovered` when the party learns of or encounters an item without
|
||||
establishing possession. Use `acquired` when the party or a party member gains
|
||||
possession. Use `lost` when party possession ends through a gift, sale, payment,
|
||||
theft, abandonment, or destruction not caused by intended use. Use `consumed`
|
||||
when intended use depletes an expendable item. Monetary spending, purchases, and
|
||||
payments are always `lost`, not `consumed`. Classify currency as `consumed` only
|
||||
when the transcript explicitly describes it being physically destroyed or
|
||||
expended as a non-payment component. Use `transferred` only when possession
|
||||
moves between two distinct named party members.
|
||||
|
||||
For `discovered`, omit both holders. For `acquired`, provide only `to`; for
|
||||
`lost` and `consumed`, provide only `from`; and for `transferred`, provide both
|
||||
`from` and `to`. Use `party` only for collective or unresolved party possession,
|
||||
never for either side of a transfer. Do not emit a transfer for a gift, sale, or
|
||||
payment outside the party.
|
||||
|
||||
Ordinary non-depleting use is not an event. Do not infer acquisition from a
|
||||
discovery, or discovery from an acquisition: emit both only when each is
|
||||
independently established. Every event needs at least one narrow transcript
|
||||
range. Return no lore, inventory totals, aliases, or unstated holder details.
|
||||
@@ -0,0 +1,7 @@
|
||||
Extract meaningful Dungeons & Dragons item and currency events established by
|
||||
the transcript. Record only discoveries and changes in party possession, with
|
||||
the transcript ranges that support each event.
|
||||
|
||||
This is an event history, not an inventory or ledger. Do not calculate balances,
|
||||
resolve item identity across records, or infer ownership that the transcript
|
||||
does not establish.
|
||||
@@ -0,0 +1,36 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "notarius.dnd.item_events.llm",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["events"],
|
||||
"properties": {
|
||||
"events": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["name", "kind", "source_refs"],
|
||||
"properties": {
|
||||
"name": {"type": "string"},
|
||||
"kind": {"type": "string"},
|
||||
"quantity": {"type": "integer"},
|
||||
"from": {"type": "string"},
|
||||
"to": {"type": "string"},
|
||||
"source_refs": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["start_segment", "end_segment"],
|
||||
"properties": {
|
||||
"start_segment": {"type": "integer"},
|
||||
"end_segment": {"type": "integer"}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
99
internal/modules/dnd/extract/itemevents/canonicalize.go
Normal file
99
internal/modules/dnd/extract/itemevents/canonicalize.go
Normal file
@@ -0,0 +1,99 @@
|
||||
package itemevents
|
||||
|
||||
import (
|
||||
"sort"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
type orderedItemEventResponse struct {
|
||||
value itemEventResponse
|
||||
earliest int
|
||||
hasEvidence bool
|
||||
}
|
||||
|
||||
func canonicalizeResponse(response *extractionResponse, order shared.SourceRefOrder, sourceID string) {
|
||||
if response == nil {
|
||||
return
|
||||
}
|
||||
ordered := make([]orderedItemEventResponse, len(response.Events))
|
||||
for index := range response.Events {
|
||||
earliest, hasEvidence := canonicalizeItemEvent(&response.Events[index], order, sourceID)
|
||||
ordered[index] = orderedItemEventResponse{
|
||||
value: response.Events[index],
|
||||
earliest: earliest,
|
||||
hasEvidence: hasEvidence,
|
||||
}
|
||||
}
|
||||
sort.SliceStable(ordered, func(left, right int) bool {
|
||||
if ordered[left].hasEvidence != ordered[right].hasEvidence {
|
||||
return ordered[left].hasEvidence
|
||||
}
|
||||
if !ordered[left].hasEvidence {
|
||||
return false
|
||||
}
|
||||
return ordered[left].earliest < ordered[right].earliest
|
||||
})
|
||||
for index := range ordered {
|
||||
response.Events[index] = ordered[index].value
|
||||
}
|
||||
}
|
||||
|
||||
func canonicalizeItemEvent(event *itemEventResponse, order shared.SourceRefOrder, sourceID string) (int, bool) {
|
||||
if event == nil {
|
||||
return 0, false
|
||||
}
|
||||
refs := order.Canonicalize(itemEventSourceRefs(event.SourceRefs, sourceID))
|
||||
event.SourceRefs = itemEventResponseRefs(refs)
|
||||
return order.EarliestValid(refs)
|
||||
}
|
||||
|
||||
func canonicalItemEventList(response extractionResponse, sourceID string) dnd.ItemEventList {
|
||||
if response.Events == nil {
|
||||
return dnd.ItemEventList{}
|
||||
}
|
||||
events := make([]dnd.ItemEvent, len(response.Events))
|
||||
for index, event := range response.Events {
|
||||
events[index] = dnd.ItemEvent{
|
||||
Name: event.Name,
|
||||
Kind: dnd.ItemEventKind(event.Kind),
|
||||
Quantity: cloneQuantity(event.Quantity),
|
||||
From: event.From,
|
||||
To: event.To,
|
||||
SourceRefs: itemEventSourceRefs(event.SourceRefs, sourceID),
|
||||
}
|
||||
}
|
||||
return dnd.ItemEventList{Events: events}
|
||||
}
|
||||
|
||||
func itemEventSourceRefs(refs []itemEventSourceRefResponse, sourceID string) []source.SourceRef {
|
||||
if refs == nil {
|
||||
return nil
|
||||
}
|
||||
values := make([]source.SourceRef, len(refs))
|
||||
for index, ref := range refs {
|
||||
values[index] = source.SourceRef{SourceID: sourceID, StartUnitID: ref.StartSegment, EndUnitID: ref.EndSegment}
|
||||
}
|
||||
return values
|
||||
}
|
||||
|
||||
func itemEventResponseRefs(refs []source.SourceRef) []itemEventSourceRefResponse {
|
||||
if refs == nil {
|
||||
return nil
|
||||
}
|
||||
values := make([]itemEventSourceRefResponse, len(refs))
|
||||
for index, ref := range refs {
|
||||
values[index] = itemEventSourceRefResponse{StartSegment: ref.StartUnitID, EndSegment: ref.EndUnitID}
|
||||
}
|
||||
return values
|
||||
}
|
||||
|
||||
func cloneQuantity(value *int) *int {
|
||||
if value == nil {
|
||||
return nil
|
||||
}
|
||||
quantity := *value
|
||||
return &quantity
|
||||
}
|
||||
163
internal/modules/dnd/extract/itemevents/extractor.go
Normal file
163
internal/modules/dnd/extract/itemevents/extractor.go
Normal file
@@ -0,0 +1,163 @@
|
||||
// Package itemevents extracts source-grounded D&D item events.
|
||||
package itemevents
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
const Key = "dnd/item-events"
|
||||
|
||||
const mappingPolicy = "dnd.item_events.extract_mapping.v1"
|
||||
|
||||
var requiredCapabilities = []string{
|
||||
"chunks",
|
||||
"source.transcript",
|
||||
}
|
||||
|
||||
var providedCapabilities = []string{
|
||||
"dnd.item_events",
|
||||
}
|
||||
|
||||
var referenceSlotDescriptions = shared.ReferenceSlotDescriptions{
|
||||
Glossary: "Optional campaign glossary reference material used only for item and holder disambiguation.",
|
||||
Party: "Optional party roster reference material used only for item and holder disambiguation.",
|
||||
Players: "Optional player list reference material used only for item and holder disambiguation.",
|
||||
Roster: "Deprecated alias for party roster reference material used only for item and holder disambiguation.",
|
||||
}
|
||||
|
||||
func referenceSlots() []contracts.ReferenceSlot {
|
||||
return shared.ReferenceSlots(referenceSlotDescriptions)
|
||||
}
|
||||
|
||||
var _ contracts.Extractor[dnd.ItemEventList] = (*Extractor)(nil)
|
||||
var _ contracts.ManifestMetadataProvider = (*Extractor)(nil)
|
||||
var _ pipeline.CheckpointFingerprintProvider = (*Extractor)(nil)
|
||||
|
||||
type Options struct{}
|
||||
|
||||
type Extractor struct {
|
||||
llm contracts.StructuredLLMClient
|
||||
promptSHA string
|
||||
responseSchemaSHA string
|
||||
}
|
||||
|
||||
func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contracts.ReferenceSet) (*Extractor, error) {
|
||||
if llmClient == nil {
|
||||
return nil, extractorErrorf("LLM client must not be nil")
|
||||
}
|
||||
if len(references) > 1 {
|
||||
return nil, extractorErrorf("at most one reference set may be supplied")
|
||||
}
|
||||
promptSHA, err := scriptoriumPromptMetadata()
|
||||
if err != nil {
|
||||
return nil, extractorErrorf("load prompt metadata: %w", err)
|
||||
}
|
||||
responseSchema, err := loadResponseSchema()
|
||||
if err != nil {
|
||||
return nil, extractorErrorf("load response schema: %w", err)
|
||||
}
|
||||
return &Extractor{llm: llmClient, promptSHA: promptSHA, responseSchemaSHA: responseSchema.SHA256}, nil
|
||||
}
|
||||
|
||||
func (e *Extractor) Key() string { return Key }
|
||||
|
||||
func (e *Extractor) ReferenceSlots() []contracts.ReferenceSlot { return referenceSlots() }
|
||||
|
||||
func (e *Extractor) ManifestMetadata() map[string]any {
|
||||
if e == nil {
|
||||
return nil
|
||||
}
|
||||
return map[string]any{
|
||||
"prompt_id": PromptID,
|
||||
"prompt_version": SchemaVersion,
|
||||
"prompt_sha256": e.promptSHA,
|
||||
"response_schema_key": string(ResponseSchemaKey),
|
||||
"response_schema_id": ResponseSchemaID,
|
||||
"response_schema_name": ResponseSchemaName,
|
||||
"response_schema_version": SchemaVersion,
|
||||
"response_schema_sha256": e.responseSchemaSHA,
|
||||
"mapping_policy": mappingPolicy,
|
||||
}
|
||||
}
|
||||
|
||||
func (e *Extractor) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
|
||||
if e == nil {
|
||||
return nil
|
||||
}
|
||||
return []pipeline.CheckpointFingerprint{
|
||||
{Name: "prompt", Value: e.promptSHA},
|
||||
{Name: "response_schema", Value: e.responseSchemaSHA},
|
||||
{Name: "mapping_policy", Value: mappingPolicy},
|
||||
}
|
||||
}
|
||||
|
||||
func (e *Extractor) Extract(ctx context.Context, req contracts.TypedExtractionRequest) (contracts.TypedExtractionResult[dnd.ItemEventList], error) {
|
||||
if e == nil {
|
||||
return contracts.TypedExtractionResult[dnd.ItemEventList]{}, extractorErrorf("extractor must not be nil")
|
||||
}
|
||||
if e.llm == nil {
|
||||
return contracts.TypedExtractionResult[dnd.ItemEventList]{}, extractorErrorf("LLM client must not be nil")
|
||||
}
|
||||
sourceInput, err := shared.PrepareChunkExtraction(ctx, req)
|
||||
if err != nil {
|
||||
return contracts.TypedExtractionResult[dnd.ItemEventList]{}, extractorErrorf("%w", err)
|
||||
}
|
||||
order := shared.NewSourceRefOrder(req.Source)
|
||||
|
||||
var response extractionResponse
|
||||
if _, err := e.llm.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
|
||||
StageName: Key,
|
||||
PromptID: PromptID,
|
||||
PromptVersion: SchemaVersion,
|
||||
ProfileID: req.LLMProfile,
|
||||
SessionID: req.SessionID,
|
||||
Inputs: shared.PromptInputs(sourceInput, req.References),
|
||||
}, &response); err != nil {
|
||||
return contracts.TypedExtractionResult[dnd.ItemEventList]{}, extractorErrorf("complete structured output: %w", err)
|
||||
}
|
||||
canonicalizeResponse(&response, order, req.Source.ID)
|
||||
return contracts.TypedExtractionResult[dnd.ItemEventList]{Value: canonicalItemEventList(response, req.Source.ID)}, nil
|
||||
}
|
||||
|
||||
func ModuleSpec() pipeline.ModuleSpec {
|
||||
return pipeline.ModuleSpec{
|
||||
Key: Key,
|
||||
Stage: pipeline.StageExtract,
|
||||
Requires: append([]string(nil), requiredCapabilities...),
|
||||
Provides: append([]string(nil), providedCapabilities...),
|
||||
ArtifactKind: dnd.ItemEventListKind,
|
||||
ReferenceSlots: referenceSlots(),
|
||||
}
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ExtractorRegistry) error {
|
||||
return pipeline.RegisterExtractorBuilder(registry, ModuleSpec(), validateOptions, func(request pipeline.BuildRequest) (contracts.Extractor[dnd.ItemEventList], error) {
|
||||
options, err := DecodeOptions(request.Options)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return New(request.Dependencies.LLM, options, request.References)
|
||||
})
|
||||
}
|
||||
|
||||
func validateOptions(options map[string]any) error {
|
||||
_, err := DecodeOptions(options)
|
||||
return err
|
||||
}
|
||||
|
||||
func DecodeOptions(options map[string]any) (Options, error) {
|
||||
if err := pipeline.RejectUnknownOptions(options); err != nil {
|
||||
return Options{}, extractorErrorf("%w", err)
|
||||
}
|
||||
return Options{}, nil
|
||||
}
|
||||
|
||||
func extractorErrorf(format string, args ...any) error {
|
||||
return fmt.Errorf("dnd item events extractor: "+format, args...)
|
||||
}
|
||||
147
internal/modules/dnd/extract/itemevents/extractor_test.go
Normal file
147
internal/modules/dnd/extract/itemevents/extractor_test.go
Normal file
@@ -0,0 +1,147 @@
|
||||
package itemevents
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
)
|
||||
|
||||
func TestExtractMapsAllEventKindsAndCanonicalizesEvidence(t *testing.T) {
|
||||
quantity := 20
|
||||
client := &fakeItemEventsLLMClient{response: extractionResponse{Events: []itemEventResponse{
|
||||
{Name: "Torch", Kind: "lost", From: "party", SourceRefs: responseRefs(5, 5)},
|
||||
{Name: "Moonblade", Kind: "transferred", From: "Aria", To: "Borin", SourceRefs: responseRefs(3, 3)},
|
||||
{Name: "Hidden Cache", Kind: "discovered", SourceRefs: []itemEventSourceRefResponse{{StartSegment: 1, EndSegment: 1}, {StartSegment: 1, EndSegment: 1}}},
|
||||
{Name: "Gold Pieces", Kind: "acquired", Quantity: &quantity, To: "party", SourceRefs: responseRefs(2, 2)},
|
||||
{Name: "Healing Potion", Kind: "consumed", From: "party", SourceRefs: responseRefs(4, 4)},
|
||||
}}}
|
||||
|
||||
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
|
||||
if err != nil {
|
||||
t.Fatalf("Extract() error = %v", err)
|
||||
}
|
||||
if got := []dnd.ItemEventKind{result.Value.Events[0].Kind, result.Value.Events[1].Kind, result.Value.Events[2].Kind, result.Value.Events[3].Kind, result.Value.Events[4].Kind}; !reflect.DeepEqual(got, []dnd.ItemEventKind{
|
||||
dnd.ItemEventKindDiscovered, dnd.ItemEventKindAcquired, dnd.ItemEventKindTransferred, dnd.ItemEventKindConsumed, dnd.ItemEventKindLost,
|
||||
}) {
|
||||
t.Fatalf("event kinds = %#v, want source order", got)
|
||||
}
|
||||
acquired := result.Value.Events[1]
|
||||
if acquired.Name != "Gold Pieces" || acquired.To != "party" || acquired.Quantity == nil || *acquired.Quantity != 20 {
|
||||
t.Fatalf("acquired event = %#v", acquired)
|
||||
}
|
||||
if result.Value.Events[0].Quantity != nil || result.Value.Events[0].From != "" || result.Value.Events[0].To != "" {
|
||||
t.Fatalf("discovered event = %#v, want omitted optional fields", result.Value.Events[0])
|
||||
}
|
||||
if refs := result.Value.Events[0].SourceRefs; len(refs) != 1 || refs[0] != (source.SourceRef{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 1}) {
|
||||
t.Fatalf("source refs = %#v, want source ID attachment and deduplication", refs)
|
||||
}
|
||||
if acquired.Quantity == client.response.Events[3].Quantity {
|
||||
t.Fatal("mapped quantity aliases private response")
|
||||
}
|
||||
if len(client.requests) != 1 {
|
||||
t.Fatalf("LLM calls = %d, want 1", len(client.requests))
|
||||
}
|
||||
request := client.requests[0]
|
||||
if request.StageName != Key || request.PromptID != PromptID || request.PromptVersion != SchemaVersion || request.ProfileID != "profile-item-events" || request.SessionID != "session-123" {
|
||||
t.Fatalf("LLM request identity = %#v", request)
|
||||
}
|
||||
if transcript := request.Inputs["transcript"]; string(transcript.Content) != string(extractionRequest().Chunk.Content) || transcript.Name != "transcript" {
|
||||
t.Fatalf("transcript input = %#v", transcript)
|
||||
}
|
||||
for _, name := range []string{"players", "party", "glossary"} {
|
||||
if input := request.Inputs[name]; string(input.Content) != " " {
|
||||
t.Fatalf("absent %s input = %#v, want retained empty prompt material", name, input)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractPreservesInvalidCandidatesAndEmptyResults(t *testing.T) {
|
||||
client := &fakeItemEventsLLMClient{content: []byte(`{"events":[{"name":"","kind":"transferred","quantity":0,"from":"party","to":"Party","source_refs":[{"start_segment":99,"end_segment":-1}]}]}`)}
|
||||
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
|
||||
if err != nil {
|
||||
t.Fatalf("Extract() error = %v", err)
|
||||
}
|
||||
event := result.Value.Events[0]
|
||||
if event.Name != "" || event.Kind != "transferred" || event.Quantity == nil || *event.Quantity != 0 || event.From != "party" || event.To != "Party" {
|
||||
t.Fatalf("invalid candidate = %#v, want values preserved", event)
|
||||
}
|
||||
if refs := event.SourceRefs; len(refs) != 1 || refs[0] != (source.SourceRef{SourceID: "session-alpha", StartUnitID: 99, EndUnitID: -1}) {
|
||||
t.Fatalf("source refs = %#v, want invalid range preserved", refs)
|
||||
}
|
||||
|
||||
empty, err := newExtractor(t, &fakeItemEventsLLMClient{response: extractionResponse{Events: []itemEventResponse{}}}).Extract(context.Background(), extractionRequest())
|
||||
if err != nil || empty.Value.Events == nil || len(empty.Value.Events) != 0 {
|
||||
t.Fatalf("empty provider result = %#v, %v; want valid empty list", empty.Value, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractUsesSourceDocumentOrderForCandidates(t *testing.T) {
|
||||
client := &fakeItemEventsLLMClient{response: extractionResponse{Events: []itemEventResponse{
|
||||
{Name: "Later", Kind: "discovered", SourceRefs: responseRefs(10, 10)},
|
||||
{Name: "Earlier", Kind: "discovered", SourceRefs: responseRefs(100, 100)},
|
||||
}}}
|
||||
req := extractionRequest()
|
||||
req.Source.Units = []source.SourceUnit{{ID: 100}, {ID: 10}}
|
||||
req.Chunk.Units = append([]source.SourceUnit(nil), req.Source.Units...)
|
||||
req.Chunk.Ref = source.SourceRef{SourceID: req.Source.ID, StartUnitID: 100, EndUnitID: 10}
|
||||
result, err := newExtractor(t, client).Extract(context.Background(), req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := []string{result.Value.Events[0].Name, result.Value.Events[1].Name}; !reflect.DeepEqual(got, []string{"Earlier", "Later"}) {
|
||||
t.Fatalf("event order = %#v, want source document order", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractUsesOnlySupportedPromptInputs(t *testing.T) {
|
||||
client := &fakeItemEventsLLMClient{response: extractionResponse{Events: []itemEventResponse{}}}
|
||||
req := extractionRequest()
|
||||
req.References = contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
|
||||
"players": {Slot: contracts.ReferenceSlot{Name: "players"}, Items: []contracts.ReferenceItem{{SlotName: "players", Content: []byte("Dana: Aria")}}},
|
||||
"party": {Slot: contracts.ReferenceSlot{Name: "party"}, Items: []contracts.ReferenceItem{{SlotName: "party", Content: []byte("Aria: ranger")}}},
|
||||
"glossary": {Slot: contracts.ReferenceSlot{Name: "glossary"}, Items: []contracts.ReferenceItem{{SlotName: "glossary", Content: []byte("Moonblade: heirloom")}}},
|
||||
"npcs": {Slot: contracts.ReferenceSlot{Name: "npcs"}, Items: []contracts.ReferenceItem{{SlotName: "npcs", Content: []byte("must not be used")}}},
|
||||
}}
|
||||
if _, err := newExtractor(t, client).Extract(context.Background(), req); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
inputs := client.requests[0].Inputs
|
||||
if string(inputs["players"].Content) != "Dana: Aria" || string(inputs["party"].Content) != "Aria: ranger" || string(inputs["glossary"].Content) != "Moonblade: heirloom" {
|
||||
t.Fatalf("reference inputs = %#v", inputs)
|
||||
}
|
||||
if _, ok := inputs["npcs"]; ok {
|
||||
t.Fatalf("unexpected generated-lane input: %#v", inputs)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractRetainsLocalErrorContext(t *testing.T) {
|
||||
request := extractionRequest()
|
||||
extractor := newExtractor(t, &fakeItemEventsLLMClient{})
|
||||
var nilExtractor *Extractor
|
||||
request.SourceInput = contracts.NewLLMInputMaterial("source", "application/json", []byte(`{"different":true}`), "sha256:other", "file:///other.json")
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
extractor *Extractor
|
||||
req contracts.TypedExtractionRequest
|
||||
want string
|
||||
}{
|
||||
{"nil extractor", nilExtractor, extractionRequest(), "extractor"},
|
||||
{"nil client", &Extractor{}, extractionRequest(), "LLM client"},
|
||||
{"preflight", extractor, request, "must match chunk"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if _, err := test.extractor.Extract(context.Background(), test.req); err == nil || !strings.Contains(err.Error(), "dnd item events") || !strings.Contains(err.Error(), test.want) {
|
||||
t.Fatalf("Extract() error = %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
if _, err := newExtractor(t, &fakeItemEventsLLMClient{err: errors.New("provider unavailable")}).Extract(context.Background(), extractionRequest()); err == nil || !strings.Contains(err.Error(), "provider unavailable") {
|
||||
t.Fatalf("provider error = %v", err)
|
||||
}
|
||||
}
|
||||
19
internal/modules/dnd/extract/itemevents/model.go
Normal file
19
internal/modules/dnd/extract/itemevents/model.go
Normal file
@@ -0,0 +1,19 @@
|
||||
package itemevents
|
||||
|
||||
type extractionResponse struct {
|
||||
Events []itemEventResponse `json:"events"`
|
||||
}
|
||||
|
||||
type itemEventResponse struct {
|
||||
Name string `json:"name"`
|
||||
Kind string `json:"kind"`
|
||||
Quantity *int `json:"quantity,omitempty"`
|
||||
From string `json:"from,omitempty"`
|
||||
To string `json:"to,omitempty"`
|
||||
SourceRefs []itemEventSourceRefResponse `json:"source_refs"`
|
||||
}
|
||||
|
||||
type itemEventSourceRefResponse struct {
|
||||
StartSegment int `json:"start_segment"`
|
||||
EndSegment int `json:"end_segment"`
|
||||
}
|
||||
65
internal/modules/dnd/extract/itemevents/registry_test.go
Normal file
65
internal/modules/dnd/extract/itemevents/registry_test.go
Normal file
@@ -0,0 +1,65 @@
|
||||
package itemevents
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
)
|
||||
|
||||
func TestConstructorSpecOptionsAndMetadata(t *testing.T) {
|
||||
if _, err := New(nil, Options{}); err == nil || !strings.Contains(err.Error(), "LLM client") {
|
||||
t.Fatalf("New(nil) error = %v", err)
|
||||
}
|
||||
if _, err := New(&fakeItemEventsLLMClient{}, Options{}, contracts.ReferenceSet{}, contracts.ReferenceSet{}); err == nil || !strings.Contains(err.Error(), "at most one reference set") {
|
||||
t.Fatalf("New() error = %v", err)
|
||||
}
|
||||
want := pipeline.ModuleSpec{
|
||||
Key: Key, Stage: pipeline.StageExtract, Requires: []string{"chunks", "source.transcript"}, Provides: []string{"dnd.item_events"}, ArtifactKind: dnd.ItemEventListKind,
|
||||
ReferenceSlots: []contracts.ReferenceSlot{
|
||||
{Name: "glossary", Description: referenceSlotDescriptions.Glossary, AcceptedMediaTypes: []string{"application/json", "application/x-yaml", "application/yaml", "text/markdown", "text/plain"}},
|
||||
{Name: "party", Description: referenceSlotDescriptions.Party, AcceptedMediaTypes: []string{"application/json", "application/x-yaml", "application/yaml", "text/markdown", "text/plain"}},
|
||||
{Name: "players", Description: referenceSlotDescriptions.Players, AcceptedMediaTypes: []string{"application/json", "application/x-yaml", "application/yaml", "text/markdown", "text/plain"}},
|
||||
{Name: "roster", Description: referenceSlotDescriptions.Roster, AcceptedMediaTypes: []string{"application/json", "application/x-yaml", "application/yaml", "text/markdown", "text/plain"}},
|
||||
},
|
||||
}
|
||||
if got := ModuleSpec(); !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("ModuleSpec() = %#v, want %#v", got, want)
|
||||
}
|
||||
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil || !strings.Contains(err.Error(), "unknown option") {
|
||||
t.Fatalf("DecodeOptions() error = %v", err)
|
||||
}
|
||||
registry := pipeline.NewExtractorRegistry()
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got, ok := registry.Spec(Key); !ok || !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("registry spec = %#v, %t", got, ok)
|
||||
}
|
||||
|
||||
extractor := newExtractor(t, &fakeItemEventsLLMClient{})
|
||||
metadata := extractor.ManifestMetadata()
|
||||
for key, want := range map[string]string{
|
||||
"prompt_id": PromptID, "prompt_version": SchemaVersion, "response_schema_key": string(ResponseSchemaKey), "response_schema_id": ResponseSchemaID,
|
||||
"response_schema_name": ResponseSchemaName, "response_schema_version": SchemaVersion, "mapping_policy": mappingPolicy,
|
||||
} {
|
||||
if metadata[key] != want {
|
||||
t.Fatalf("metadata[%q] = %#v, want %q", key, metadata[key], want)
|
||||
}
|
||||
}
|
||||
for _, key := range []string{"prompt_sha256", "response_schema_sha256"} {
|
||||
if value, ok := metadata[key].(string); !ok || !strings.HasPrefix(value, "sha256:") {
|
||||
t.Fatalf("metadata[%q] = %#v", key, metadata[key])
|
||||
}
|
||||
}
|
||||
if got := extractor.CheckpointFingerprints(); !reflect.DeepEqual(got, []pipeline.CheckpointFingerprint{
|
||||
{Name: "prompt", Value: metadata["prompt_sha256"].(string)},
|
||||
{Name: "response_schema", Value: metadata["response_schema_sha256"].(string)},
|
||||
{Name: "mapping_policy", Value: mappingPolicy},
|
||||
}) {
|
||||
t.Fatalf("CheckpointFingerprints() = %#v", got)
|
||||
}
|
||||
}
|
||||
21
internal/modules/dnd/extract/itemevents/schema.go
Normal file
21
internal/modules/dnd/extract/itemevents/schema.go
Normal file
@@ -0,0 +1,21 @@
|
||||
package itemevents
|
||||
|
||||
import "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||
|
||||
const (
|
||||
PromptID = "dnd.item_events"
|
||||
ResponseSchemaKey = llm.ResponseSchemaKey("dnd_item_events_llm")
|
||||
ResponseSchemaID = "notarius.dnd.item_events.llm"
|
||||
ResponseSchemaName = "notarius_dnd_item_events_llm_v1"
|
||||
SchemaVersion = "v1"
|
||||
)
|
||||
|
||||
func loadResponseSchema() (llm.ResponseSchema, error) {
|
||||
return llm.LoadResponseSchema(embeddedAssets, llm.ResponseSchemaDefinition{
|
||||
Key: ResponseSchemaKey,
|
||||
ID: ResponseSchemaID,
|
||||
Version: SchemaVersion,
|
||||
Name: ResponseSchemaName,
|
||||
AssetPath: "assets/schemas/dnd_item_events_llm.v1.json",
|
||||
})
|
||||
}
|
||||
80
internal/modules/dnd/extract/itemevents/schema_test.go
Normal file
80
internal/modules/dnd/extract/itemevents/schema_test.go
Normal file
@@ -0,0 +1,80 @@
|
||||
package itemevents
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/santhosh-tekuri/jsonschema/v6"
|
||||
)
|
||||
|
||||
func TestResponseSchemaIsStrictlyStructuralAndPrivate(t *testing.T) {
|
||||
schema, err := loadResponseSchema()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if schema.Key != ResponseSchemaKey || schema.ID != ResponseSchemaID || schema.Name != ResponseSchemaName || schema.Version != SchemaVersion || !strings.HasPrefix(schema.SHA256, "sha256:") || !json.Valid(schema.JSONSchema) {
|
||||
t.Fatalf("schema = %#v", schema)
|
||||
}
|
||||
valid := map[string]any{"events": []any{map[string]any{
|
||||
"name": "", "kind": "unsupported", "quantity": 0, "from": "party", "to": "Party",
|
||||
"source_refs": []any{map[string]any{"start_segment": 0, "end_segment": -1}},
|
||||
}}}
|
||||
content, err := json.Marshal(valid)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := validateJSONSchema(content, schema.JSONSchema); err != nil {
|
||||
t.Fatalf("private schema rejected validator-owned values: %v", err)
|
||||
}
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
value map[string]any
|
||||
}{
|
||||
{"missing events", map[string]any{}},
|
||||
{"missing event name", map[string]any{"events": []any{map[string]any{"kind": "acquired", "source_refs": []any{}}}}},
|
||||
{"unknown event field", map[string]any{"events": []any{map[string]any{"name": "Ring", "kind": "acquired", "source_refs": []any{}, "extra": true}}}},
|
||||
{"unknown reference field", map[string]any{"events": []any{map[string]any{"name": "Ring", "kind": "acquired", "source_refs": []any{map[string]any{"start_segment": 1, "end_segment": 1, "extra": true}}}}}},
|
||||
{"noninteger range", map[string]any{"events": []any{map[string]any{"name": "Ring", "kind": "acquired", "source_refs": []any{map[string]any{"start_segment": 1.5, "end_segment": 1}}}}}},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
content, err := json.Marshal(test.value)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := validateJSONSchema(content, schema.JSONSchema); err == nil {
|
||||
t.Fatal("private schema accepted invalid structure")
|
||||
}
|
||||
})
|
||||
}
|
||||
first := schema.JSONSchema
|
||||
first[0] = '['
|
||||
second, err := loadResponseSchema()
|
||||
if err != nil || !json.Valid(second.JSONSchema) || bytes.Equal(first, second.JSONSchema) {
|
||||
t.Fatalf("defensive schema copy = %s, %v", second.JSONSchema, err)
|
||||
}
|
||||
if _, ok := second.DiagnosticsMap()["json_schema"]; ok {
|
||||
t.Fatalf("schema diagnostics leaked content: %#v", second.DiagnosticsMap())
|
||||
}
|
||||
}
|
||||
|
||||
func validateJSONSchema(instanceContent, schemaContent []byte) error {
|
||||
instance, err := jsonschema.UnmarshalJSON(bytes.NewReader(instanceContent))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
document, err := jsonschema.UnmarshalJSON(bytes.NewReader(schemaContent))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
compiler := jsonschema.NewCompiler()
|
||||
if err := compiler.AddResource("schema.json", document); err != nil {
|
||||
return err
|
||||
}
|
||||
compiled, err := compiler.Compile("schema.json")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return compiled.Validate(instance)
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
package itemevents
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sync"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/promptfs"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
const scriptoriumPromptRoot = "assets/prompts"
|
||||
|
||||
var promptAssetManifest = shared.PromptAssetManifest{
|
||||
ModuleDir: "dnd.item_events",
|
||||
ModuleFiles: []promptfs.ModulePromptFile{
|
||||
{Name: "dnd.item_events.yaml", Path: "assets/prompts/dnd.item_events.yaml"},
|
||||
{Name: "task.md", Path: "assets/prompts/task.md"},
|
||||
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||
},
|
||||
SharedFiles: []string{
|
||||
"common-dnd-system.md",
|
||||
"common-dnd-extraction-evidence.md",
|
||||
"common-dnd-identity.md",
|
||||
"common-dnd-references.md",
|
||||
"common-dnd-transcript.md",
|
||||
},
|
||||
}
|
||||
|
||||
func RegisterPromptAssets(registry *llm.AssetRegistry) error {
|
||||
promptFS, err := promptAssetManifest.PromptFS(embeddedAssets)
|
||||
if err != nil {
|
||||
return fmt.Errorf("prepare item event prompt assets: %w", err)
|
||||
}
|
||||
if err := registry.RegisterPromptFS(promptFS, scriptoriumPromptRoot); err != nil {
|
||||
return err
|
||||
}
|
||||
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
|
||||
}
|
||||
|
||||
func scriptoriumPromptMetadata() (string, error) {
|
||||
scriptoriumPromptHashOnce.Do(func() {
|
||||
scriptoriumPromptHash, scriptoriumPromptHashErr = promptAssetManifest.Hash(embeddedAssets)
|
||||
})
|
||||
return scriptoriumPromptHash, scriptoriumPromptHashErr
|
||||
}
|
||||
|
||||
var (
|
||||
scriptoriumPromptHashOnce sync.Once
|
||||
scriptoriumPromptHash string
|
||||
scriptoriumPromptHashErr error
|
||||
)
|
||||
@@ -0,0 +1,114 @@
|
||||
package itemevents
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io/fs"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||
"gitea.maximumdirect.net/eric/scriptorium"
|
||||
)
|
||||
|
||||
func TestPromptAssetsUseSharedSequenceAndTranscriptLast(t *testing.T) {
|
||||
registry := llm.NewAssetRegistry()
|
||||
if err := RegisterPromptAssets(registry); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
options, err := registry.ScriptoriumOptions()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
options = append(options, scriptorium.WithProfiles(scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
|
||||
ID: "item-events-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "item-events-test-model",
|
||||
})))
|
||||
engine, err := scriptorium.NewEngine(scriptorium.Config{Timeout: time.Second}, options...)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
|
||||
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "item-events-test-profile",
|
||||
Inputs: map[string]scriptorium.ArtifactRef{
|
||||
"transcript": scriptorium.InlineWithURI("file:///session.json", `{"segments":[1]}`),
|
||||
"players": scriptorium.Inline("Dana: Aria"),
|
||||
"party": scriptorium.Inline("Aria: ranger"),
|
||||
"glossary": scriptorium.Inline("Moonblade: heirloom"),
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_item_events_llm.v1.json" {
|
||||
t.Fatalf("prepared prompt = %#v", prepared)
|
||||
}
|
||||
want := []struct {
|
||||
role string
|
||||
cached bool
|
||||
marker string
|
||||
}{
|
||||
{"system", false, "Dungeons & Dragons gameplay transcripts"},
|
||||
{"user", false, "Transcript units are the only evidence"},
|
||||
{"user", true, "most specific supported in-world"},
|
||||
{"user", true, "Dana: Aria"},
|
||||
{"user", false, "item and currency events"},
|
||||
{"user", true, "Ordinary non-depleting use"},
|
||||
{"user", false, `{"segments":[1]}`},
|
||||
}
|
||||
if len(prepared.Messages) != len(want) {
|
||||
t.Fatalf("prompt messages = %d, want %d", len(prepared.Messages), len(want))
|
||||
}
|
||||
for index, expected := range want {
|
||||
message := prepared.Messages[index]
|
||||
if message.Role != expected.role || !strings.Contains(message.Content, expected.marker) {
|
||||
t.Fatalf("message %d = %#v", index, message)
|
||||
}
|
||||
if (message.CacheControl != nil) != expected.cached {
|
||||
t.Fatalf("message %d cache control = %#v", index, message.CacheControl)
|
||||
}
|
||||
}
|
||||
if strings.Contains(prepared.Messages[len(prepared.Messages)-1].Content, "Moonblade: heirloom") {
|
||||
t.Fatal("transcript message contains optional reference content")
|
||||
}
|
||||
}
|
||||
|
||||
func TestPromptAssetsDoNotLeakIntoMetadata(t *testing.T) {
|
||||
hash, err := scriptoriumPromptMetadata()
|
||||
if err != nil || !strings.HasPrefix(hash, "sha256:") {
|
||||
t.Fatalf("scriptoriumPromptMetadata() = %q, %v", hash, err)
|
||||
}
|
||||
metadata := newExtractor(t, &fakeItemEventsLLMClient{}).ManifestMetadata()
|
||||
payload, err := json.Marshal(metadata)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, forbidden := range []string{"meaningful Dungeons", "common-dnd-system", "start_segment", "dnd_item_events_llm.v1.json"} {
|
||||
if strings.Contains(string(payload), forbidden) {
|
||||
t.Fatalf("metadata leaked raw asset content %q: %s", forbidden, payload)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPromptManifestReusesOnlySharedAssets(t *testing.T) {
|
||||
want := []string{
|
||||
"common-dnd-system.md",
|
||||
"common-dnd-extraction-evidence.md",
|
||||
"common-dnd-identity.md",
|
||||
"common-dnd-references.md",
|
||||
"common-dnd-transcript.md",
|
||||
}
|
||||
if !reflect.DeepEqual(promptAssetManifest.SharedFiles, want) {
|
||||
t.Fatalf("shared assets = %#v, want %#v", promptAssetManifest.SharedFiles, want)
|
||||
}
|
||||
for _, path := range []string{"assets/prompts/task.md", "assets/prompts/instructions.md"} {
|
||||
content, err := fs.ReadFile(embeddedAssets, path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(string(content), "Transcript units are the only evidence") || strings.Contains(string(content), "Dungeons & Dragons gameplay transcripts") {
|
||||
t.Fatalf("module asset %q copied shared prompt text", path)
|
||||
}
|
||||
}
|
||||
}
|
||||
92
internal/modules/dnd/extract/itemevents/test_helpers_test.go
Normal file
92
internal/modules/dnd/extract/itemevents/test_helpers_test.go
Normal file
@@ -0,0 +1,92 @@
|
||||
package itemevents
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
)
|
||||
|
||||
func extractionRequest() contracts.TypedExtractionRequest {
|
||||
doc := sourceDocument()
|
||||
chunk := &source.Chunk{
|
||||
ID: "session-alpha:chunk:0",
|
||||
SourceID: doc.ID,
|
||||
Index: 0,
|
||||
Ref: source.SourceRef{SourceID: doc.ID, StartUnitID: 1, EndUnitID: 5},
|
||||
Content: []byte(`{"segments":[1,2,3,4,5]}`),
|
||||
MediaType: "application/json",
|
||||
Units: append([]source.SourceUnit(nil), doc.Units...),
|
||||
}
|
||||
return contracts.TypedExtractionRequest{
|
||||
Source: doc,
|
||||
Chunk: chunk,
|
||||
SourceInput: contracts.NewLLMInputMaterial("source", chunk.MediaType, chunk.Content, "sha256:chunk", "file:///session-alpha.json"),
|
||||
SessionID: "session-123",
|
||||
LLMProfile: "profile-item-events",
|
||||
}
|
||||
}
|
||||
|
||||
func sourceDocument() *source.SourceDocument {
|
||||
return &source.SourceDocument{
|
||||
ID: "session-alpha", Kind: "transcript", Format: "application/vnd.seriatim.minimal+json", Digest: "sha256:test",
|
||||
Units: []source.SourceUnit{
|
||||
{ID: 1, Kind: "transcript_segment", Text: "The party discovers a hidden cache."},
|
||||
{ID: 2, Kind: "transcript_segment", Text: "They acquire 20 gold pieces."},
|
||||
{ID: 3, Kind: "transcript_segment", Text: "Aria gives Borin the moonblade."},
|
||||
{ID: 4, Kind: "transcript_segment", Text: "The party consumes a healing potion."},
|
||||
{ID: 5, Kind: "transcript_segment", Text: "The party loses a torch."},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func responseRefs(start, end int) []itemEventSourceRefResponse {
|
||||
return []itemEventSourceRefResponse{{StartSegment: start, EndSegment: end}}
|
||||
}
|
||||
|
||||
func newExtractor(t *testing.T, client contracts.StructuredLLMClient, references ...contracts.ReferenceSet) *Extractor {
|
||||
t.Helper()
|
||||
extractor, err := New(client, Options{}, references...)
|
||||
if err != nil {
|
||||
t.Fatalf("New() error = %v", err)
|
||||
}
|
||||
return extractor
|
||||
}
|
||||
|
||||
func cloneStructuredCompletionRequest(req contracts.StructuredCompletionRequest) contracts.StructuredCompletionRequest {
|
||||
req.Inputs = req.Inputs.Clone()
|
||||
return req
|
||||
}
|
||||
|
||||
type fakeItemEventsLLMClient struct {
|
||||
response extractionResponse
|
||||
content []byte
|
||||
err error
|
||||
requests []contracts.StructuredCompletionRequest
|
||||
}
|
||||
|
||||
func (client *fakeItemEventsLLMClient) CompleteStructured(_ context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
|
||||
client.requests = append(client.requests, cloneStructuredCompletionRequest(req))
|
||||
if client.err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, client.err
|
||||
}
|
||||
target, ok := out.(*extractionResponse)
|
||||
if !ok {
|
||||
return contracts.StructuredCompletionResponse{}, errors.New("unexpected output target")
|
||||
}
|
||||
content := append([]byte(nil), client.content...)
|
||||
if len(content) == 0 {
|
||||
var err error
|
||||
content, err = json.Marshal(client.response)
|
||||
if err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, err
|
||||
}
|
||||
}
|
||||
if err := json.Unmarshal(content, target); err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, err
|
||||
}
|
||||
return contracts.StructuredCompletionResponse{Content: content}, nil
|
||||
}
|
||||
209
internal/modules/dnd/itemevents/itemevents.go
Normal file
209
internal/modules/dnd/itemevents/itemevents.go
Normal file
@@ -0,0 +1,209 @@
|
||||
// Package itemevents owns canonical ordering and durable domain rules for D&D
|
||||
// item-event artifacts.
|
||||
package itemevents
|
||||
|
||||
import (
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
const partyHolder = "party"
|
||||
|
||||
// SupportedKind reports whether kind is one of the durable item-event kinds.
|
||||
func SupportedKind(kind dnd.ItemEventKind) bool {
|
||||
switch kind {
|
||||
case dnd.ItemEventKindDiscovered,
|
||||
dnd.ItemEventKindAcquired,
|
||||
dnd.ItemEventKindLost,
|
||||
dnd.ItemEventKindConsumed,
|
||||
dnd.ItemEventKindTransferred:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// DisplayValue returns the durable display form used for item and holder
|
||||
// comparisons without changing the observed internal spelling.
|
||||
func DisplayValue(value string) string { return strings.TrimSpace(value) }
|
||||
|
||||
// ComparisonKey returns the shared D&D Unicode- and case-insensitive key for
|
||||
// a trimmed item or holder display value.
|
||||
func ComparisonKey(value string) string { return identity.ComparisonKey(DisplayValue(value)) }
|
||||
|
||||
// HolderPresent reports whether value is a nonblank optional holder.
|
||||
func HolderPresent(value string) bool { return DisplayValue(value) != "" }
|
||||
|
||||
// IsPartyHolder reports whether value denotes collective party possession.
|
||||
func IsPartyHolder(value string) bool { return ComparisonKey(value) == partyHolder }
|
||||
|
||||
// ValidHolderCombination reports whether the optional holder fields satisfy
|
||||
// the durable rules for kind. Blank holders are treated as absent so callers
|
||||
// can preserve invalid extraction candidates for their owning validators.
|
||||
func ValidHolderCombination(kind dnd.ItemEventKind, from, to string) bool {
|
||||
hasFrom := HolderPresent(from)
|
||||
hasTo := HolderPresent(to)
|
||||
|
||||
switch kind {
|
||||
case dnd.ItemEventKindDiscovered:
|
||||
return !hasFrom && !hasTo
|
||||
case dnd.ItemEventKindAcquired:
|
||||
return !hasFrom && hasTo
|
||||
case dnd.ItemEventKindLost, dnd.ItemEventKindConsumed:
|
||||
return hasFrom && !hasTo
|
||||
case dnd.ItemEventKindTransferred:
|
||||
return hasFrom && hasTo && !IsPartyHolder(from) && !IsPartyHolder(to) && ComparisonKey(from) != ComparisonKey(to)
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// SourceRefsEqual reports whether two source-reference sequences have the
|
||||
// same representation and values, including nil-versus-empty distinction.
|
||||
func SourceRefsEqual(left, right []source.SourceRef) bool {
|
||||
if (left == nil) != (right == nil) || len(left) != len(right) {
|
||||
return false
|
||||
}
|
||||
for index := range left {
|
||||
if left[index] != right[index] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// ValidSourceRefs reports whether refs are non-empty and valid for index.
|
||||
func ValidSourceRefs(index source.DocumentIndex, refs []source.SourceRef) bool {
|
||||
if len(refs) == 0 {
|
||||
return false
|
||||
}
|
||||
for _, ref := range refs {
|
||||
if index.ValidateRef(ref) != nil {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// Less defines the canonical event order. Invalid source references remain
|
||||
// comparable through SourceRefOrder's literal fallback so malformed candidates
|
||||
// are still safe to sort and diagnose.
|
||||
func Less(order shared.SourceRefOrder, left, right dnd.ItemEvent) bool {
|
||||
leftPosition, leftHasEvidence := order.EarliestValid(left.SourceRefs)
|
||||
rightPosition, rightHasEvidence := order.EarliestValid(right.SourceRefs)
|
||||
if leftHasEvidence != rightHasEvidence {
|
||||
return leftHasEvidence
|
||||
}
|
||||
if leftHasEvidence && leftPosition != rightPosition {
|
||||
return leftPosition < rightPosition
|
||||
}
|
||||
if leftKey, rightKey := ComparisonKey(left.Name), ComparisonKey(right.Name); leftKey != rightKey {
|
||||
return leftKey < rightKey
|
||||
}
|
||||
if leftName, rightName := DisplayValue(left.Name), DisplayValue(right.Name); leftName != rightName {
|
||||
return leftName < rightName
|
||||
}
|
||||
if left.Kind != right.Kind {
|
||||
return left.Kind < right.Kind
|
||||
}
|
||||
if less, decided := optionalStringLess(left.From, right.From); decided {
|
||||
return less
|
||||
}
|
||||
if less, decided := optionalStringLess(left.To, right.To); decided {
|
||||
return less
|
||||
}
|
||||
if less, decided := optionalQuantityLess(left.Quantity, right.Quantity); decided {
|
||||
return less
|
||||
}
|
||||
return sourceRefsLess(order, order.Canonicalize(left.SourceRefs), order.Canonicalize(right.SourceRefs))
|
||||
}
|
||||
|
||||
// ExactEqual reports whether events are exact duplicates after their display
|
||||
// fields and evidence have been canonicalized for the supplied source order.
|
||||
func ExactEqual(order shared.SourceRefOrder, left, right dnd.ItemEvent) bool {
|
||||
if DisplayValue(left.Name) != DisplayValue(right.Name) || left.Kind != right.Kind ||
|
||||
DisplayValue(left.From) != DisplayValue(right.From) || DisplayValue(left.To) != DisplayValue(right.To) ||
|
||||
(left.Quantity == nil) != (right.Quantity == nil) {
|
||||
return false
|
||||
}
|
||||
if left.Quantity != nil && *left.Quantity != *right.Quantity {
|
||||
return false
|
||||
}
|
||||
return SourceRefsEqual(order.Canonicalize(left.SourceRefs), order.Canonicalize(right.SourceRefs))
|
||||
}
|
||||
|
||||
// ExactIdentity returns a collision-safe duplicate key after display and
|
||||
// evidence canonicalization. It is intended for callers that have already
|
||||
// decided the event is eligible for duplicate handling.
|
||||
func ExactIdentity(order shared.SourceRefOrder, event dnd.ItemEvent) string {
|
||||
var key strings.Builder
|
||||
writeKeyString(&key, DisplayValue(event.Name))
|
||||
writeKeyString(&key, string(event.Kind))
|
||||
writeKeyString(&key, DisplayValue(event.From))
|
||||
writeKeyString(&key, DisplayValue(event.To))
|
||||
if event.Quantity == nil {
|
||||
key.WriteByte('0')
|
||||
} else {
|
||||
key.WriteByte('1')
|
||||
writeKeyInt(&key, *event.Quantity)
|
||||
}
|
||||
for _, ref := range order.Canonicalize(event.SourceRefs) {
|
||||
writeKeyString(&key, ref.SourceID)
|
||||
writeKeyInt(&key, ref.StartUnitID)
|
||||
writeKeyInt(&key, ref.EndUnitID)
|
||||
}
|
||||
return key.String()
|
||||
}
|
||||
|
||||
func optionalStringLess(left, right string) (bool, bool) {
|
||||
leftPresent, rightPresent := HolderPresent(left), HolderPresent(right)
|
||||
if leftPresent != rightPresent {
|
||||
return !leftPresent, true
|
||||
}
|
||||
if !leftPresent {
|
||||
return false, false
|
||||
}
|
||||
if leftKey, rightKey := ComparisonKey(left), ComparisonKey(right); leftKey != rightKey {
|
||||
return leftKey < rightKey, true
|
||||
}
|
||||
if leftValue, rightValue := DisplayValue(left), DisplayValue(right); leftValue != rightValue {
|
||||
return leftValue < rightValue, true
|
||||
}
|
||||
return false, false
|
||||
}
|
||||
|
||||
func optionalQuantityLess(left, right *int) (bool, bool) {
|
||||
if (left == nil) != (right == nil) {
|
||||
return left == nil, true
|
||||
}
|
||||
if left != nil && *left != *right {
|
||||
return *left < *right, true
|
||||
}
|
||||
return false, false
|
||||
}
|
||||
|
||||
func sourceRefsLess(order shared.SourceRefOrder, left, right []source.SourceRef) bool {
|
||||
for index := 0; index < len(left) && index < len(right); index++ {
|
||||
if left[index] == right[index] {
|
||||
continue
|
||||
}
|
||||
return order.Less(left[index], right[index])
|
||||
}
|
||||
return len(left) < len(right)
|
||||
}
|
||||
|
||||
func writeKeyString(builder *strings.Builder, value string) {
|
||||
builder.WriteString(strconv.Itoa(len(value)))
|
||||
builder.WriteByte(':')
|
||||
builder.WriteString(value)
|
||||
}
|
||||
|
||||
func writeKeyInt(builder *strings.Builder, value int) {
|
||||
builder.WriteString(strconv.Itoa(value))
|
||||
builder.WriteByte(';')
|
||||
}
|
||||
169
internal/modules/dnd/itemevents/itemevents_test.go
Normal file
169
internal/modules/dnd/itemevents/itemevents_test.go
Normal file
@@ -0,0 +1,169 @@
|
||||
package itemevents
|
||||
|
||||
import (
|
||||
"sort"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
func TestValidHolderCombination(t *testing.T) {
|
||||
valid := []struct {
|
||||
kind dnd.ItemEventKind
|
||||
from, to string
|
||||
}{
|
||||
{dnd.ItemEventKindDiscovered, "", ""},
|
||||
{dnd.ItemEventKindAcquired, "", "party"},
|
||||
{dnd.ItemEventKindLost, "party", ""},
|
||||
{dnd.ItemEventKindConsumed, "party", ""},
|
||||
{dnd.ItemEventKindTransferred, "Aria", "Borin"},
|
||||
}
|
||||
for _, test := range valid {
|
||||
if !ValidHolderCombination(test.kind, test.from, test.to) {
|
||||
t.Fatalf("ValidHolderCombination(%q, %q, %q) = false", test.kind, test.from, test.to)
|
||||
}
|
||||
}
|
||||
|
||||
invalid := []struct {
|
||||
kind dnd.ItemEventKind
|
||||
from, to string
|
||||
}{
|
||||
{dnd.ItemEventKindDiscovered, "Aria", ""},
|
||||
{dnd.ItemEventKindAcquired, "", ""},
|
||||
{dnd.ItemEventKindLost, "", ""},
|
||||
{dnd.ItemEventKindConsumed, "", "Borin"},
|
||||
{dnd.ItemEventKindTransferred, "party", "Borin"},
|
||||
{dnd.ItemEventKindTransferred, "Aria", "Party"},
|
||||
{dnd.ItemEventKindTransferred, "Aria", "aria"},
|
||||
{dnd.ItemEventKindTransferred, "Åria", "Åria"},
|
||||
{"unsupported", "", ""},
|
||||
}
|
||||
for _, test := range invalid {
|
||||
if ValidHolderCombination(test.kind, test.from, test.to) {
|
||||
t.Fatalf("ValidHolderCombination(%q, %q, %q) = true", test.kind, test.from, test.to)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestLessUsesEveryCanonicalTieBreaker(t *testing.T) {
|
||||
order := testOrder()
|
||||
ref := func(start, end int) []source.SourceRef {
|
||||
return []source.SourceRef{{SourceID: "session", StartUnitID: start, EndUnitID: end}}
|
||||
}
|
||||
quantity := func(value int) *int { return &value }
|
||||
base := dnd.ItemEvent{Name: "Amulet", Kind: dnd.ItemEventKindAcquired, To: "Borin", SourceRefs: ref(20, 20)}
|
||||
tests := []struct {
|
||||
name string
|
||||
left, right dnd.ItemEvent
|
||||
}{
|
||||
{"earlier evidence", withRefs(base, ref(10, 10)), base},
|
||||
{"valid evidence before malformed", base, withRefs(base, ref(999, 999))},
|
||||
{"normalized name", withName(base, "Amulet"), withName(base, "Blade")},
|
||||
{"exact trimmed name", withName(base, "Amulet"), withName(base, "amulet")},
|
||||
{"kind", withKind(base, dnd.ItemEventKindAcquired), withKind(base, dnd.ItemEventKindLost)},
|
||||
{"from presence", withFrom(base, ""), withFrom(base, "Aria")},
|
||||
{"from normalized value", withFrom(base, "Aria"), withFrom(base, "Borin")},
|
||||
{"from exact value", withFrom(base, "Aria"), withFrom(base, "aria")},
|
||||
{"to presence", withoutTo(base), withTo(base, "Borin")},
|
||||
{"to normalized value", withTo(base, "Aria"), withTo(base, "Borin")},
|
||||
{"to exact value", withTo(base, "Aria"), withTo(base, "aria")},
|
||||
{"quantity presence", withQuantity(base, nil), withQuantity(base, quantity(1))},
|
||||
{"quantity value", withQuantity(base, quantity(1)), withQuantity(base, quantity(2))},
|
||||
{"canonical source reference sequence", withRefs(base, ref(20, 20)), withRefs(base, ref(30, 30))},
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if !Less(order, test.left, test.right) || Less(order, test.right, test.left) {
|
||||
t.Fatalf("Less() did not order %#v before %#v", test.left, test.right)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLessAndExactEqualityCanonicalizeEvidence(t *testing.T) {
|
||||
order := testOrder()
|
||||
first := dnd.ItemEvent{
|
||||
Name: " Silver Coin ", Kind: dnd.ItemEventKindAcquired, To: " party ",
|
||||
SourceRefs: []source.SourceRef{
|
||||
{SourceID: "session", StartUnitID: 30, EndUnitID: 30},
|
||||
{SourceID: "session", StartUnitID: 10, EndUnitID: 10},
|
||||
{SourceID: "session", StartUnitID: 10, EndUnitID: 10},
|
||||
},
|
||||
}
|
||||
second := dnd.ItemEvent{
|
||||
Name: "Silver Coin", Kind: dnd.ItemEventKindAcquired, To: "party",
|
||||
SourceRefs: []source.SourceRef{
|
||||
{SourceID: "session", StartUnitID: 10, EndUnitID: 10},
|
||||
{SourceID: "session", StartUnitID: 30, EndUnitID: 30},
|
||||
},
|
||||
}
|
||||
if !ExactEqual(order, first, second) {
|
||||
t.Fatal("ExactEqual() = false, want canonical duplicate")
|
||||
}
|
||||
if Less(order, first, second) || Less(order, second, first) {
|
||||
t.Fatal("Less() distinguished canonically equal records")
|
||||
}
|
||||
if ExactIdentity(order, first) != ExactIdentity(order, second) {
|
||||
t.Fatal("ExactIdentity() differs for canonical duplicates")
|
||||
}
|
||||
|
||||
quantity := 1
|
||||
differentQuantity := second
|
||||
differentQuantity.Quantity = &quantity
|
||||
differentEvidence := second
|
||||
differentEvidence.SourceRefs = append([]source.SourceRef(nil), second.SourceRefs...)
|
||||
differentEvidence.SourceRefs[1].EndUnitID = 20
|
||||
if ExactEqual(order, second, differentQuantity) || ExactEqual(order, second, differentEvidence) {
|
||||
t.Fatal("ExactEqual() collapsed distinct optional field or evidence values")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceReferenceHelpers(t *testing.T) {
|
||||
refs := []source.SourceRef{{SourceID: "session", StartUnitID: 10, EndUnitID: 20}}
|
||||
if !SourceRefsEqual(refs, append([]source.SourceRef(nil), refs...)) || SourceRefsEqual(nil, []source.SourceRef{}) {
|
||||
t.Fatal("SourceRefsEqual() did not preserve source-reference representation")
|
||||
}
|
||||
doc := &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 10}, {ID: 20}, {ID: 30}}}
|
||||
if !ValidSourceRefs(source.NewDocumentIndex(doc), refs) {
|
||||
t.Fatal("ValidSourceRefs() = false, want valid reference")
|
||||
}
|
||||
if ValidSourceRefs(source.NewDocumentIndex(doc), []source.SourceRef{{SourceID: "other", StartUnitID: 10, EndUnitID: 20}}) {
|
||||
t.Fatal("ValidSourceRefs() = true, want invalid source identifier rejection")
|
||||
}
|
||||
}
|
||||
|
||||
func TestLessSortsMalformedReferencesDeterministically(t *testing.T) {
|
||||
order := testOrder()
|
||||
events := []dnd.ItemEvent{
|
||||
{Name: "A", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 999, EndUnitID: 999}}},
|
||||
{Name: "A", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "other", StartUnitID: 1, EndUnitID: 1}}},
|
||||
{Name: "A", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 10, EndUnitID: 10}}},
|
||||
}
|
||||
sort.SliceStable(events, func(left, right int) bool { return Less(order, events[left], events[right]) })
|
||||
if events[0].SourceRefs[0].StartUnitID != 10 || events[1].SourceRefs[0].SourceID != "other" || events[2].SourceRefs[0].StartUnitID != 999 {
|
||||
t.Fatalf("canonical sort = %#v", events)
|
||||
}
|
||||
}
|
||||
|
||||
func testOrder() shared.SourceRefOrder {
|
||||
return shared.NewSourceRefOrder(&source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 10}, {ID: 20}, {ID: 30}}})
|
||||
}
|
||||
|
||||
func withName(event dnd.ItemEvent, value string) dnd.ItemEvent { event.Name = value; return event }
|
||||
func withKind(event dnd.ItemEvent, value dnd.ItemEventKind) dnd.ItemEvent {
|
||||
event.Kind = value
|
||||
return event
|
||||
}
|
||||
func withFrom(event dnd.ItemEvent, value string) dnd.ItemEvent { event.From = value; return event }
|
||||
func withTo(event dnd.ItemEvent, value string) dnd.ItemEvent { event.To = value; return event }
|
||||
func withoutTo(event dnd.ItemEvent) dnd.ItemEvent { event.To = ""; return event }
|
||||
func withQuantity(event dnd.ItemEvent, value *int) dnd.ItemEvent {
|
||||
event.Quantity = value
|
||||
return event
|
||||
}
|
||||
func withRefs(event dnd.ItemEvent, value []source.SourceRef) dnd.ItemEvent {
|
||||
event.SourceRefs = value
|
||||
return event
|
||||
}
|
||||
254
internal/modules/dnd/normalize/itemevents/normalizer.go
Normal file
254
internal/modules/dnd/normalize/itemevents/normalizer.go
Normal file
@@ -0,0 +1,254 @@
|
||||
// Package itemevents normalizes merged D&D item-event candidates.
|
||||
package itemevents
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"sort"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
itemeventmodel "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/itemevents"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics"
|
||||
)
|
||||
|
||||
const (
|
||||
Key = "dnd/item-events"
|
||||
normalizationPolicy = "dnd.item_events.normalize.v1"
|
||||
NormalizationPolicy = normalizationPolicy
|
||||
|
||||
ReasonCodeDisplayNormalized = "item_event_display_normalized"
|
||||
ReasonCodeSourceRefsNormalized = "source_references_normalized"
|
||||
ReasonCodeEventsReordered = "item_events_reordered"
|
||||
ReasonCodeDuplicateCollapsed = "duplicate_item_event_collapsed"
|
||||
ReasonCodeWarningsOmitted = "item_event_normalization_warnings_omitted"
|
||||
)
|
||||
|
||||
var requiredCapabilities = []string{"merged"}
|
||||
var providedCapabilities = []string{"normalized"}
|
||||
|
||||
var _ contracts.Normalizer[dnd.ItemEventList] = (*Normalizer)(nil)
|
||||
var _ contracts.ManifestMetadataProvider = (*Normalizer)(nil)
|
||||
var _ pipeline.CheckpointFingerprintProvider = (*Normalizer)(nil)
|
||||
|
||||
// Options deliberately has no fields: item-event normalization has no
|
||||
// generated-reference dependency or configurable semantic behavior.
|
||||
type Options struct{}
|
||||
|
||||
type Normalizer struct{}
|
||||
|
||||
func New(Options) *Normalizer { return &Normalizer{} }
|
||||
|
||||
func (n *Normalizer) Key() string { return Key }
|
||||
|
||||
func (n *Normalizer) ReferenceSlots() []contracts.ReferenceSlot { return nil }
|
||||
|
||||
func (n *Normalizer) ManifestMetadata() map[string]any {
|
||||
if n == nil {
|
||||
return nil
|
||||
}
|
||||
return map[string]any{"normalization_policy": normalizationPolicy}
|
||||
}
|
||||
|
||||
func (n *Normalizer) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
|
||||
if n == nil {
|
||||
return nil
|
||||
}
|
||||
return []pipeline.CheckpointFingerprint{{Name: "normalization_policy", Value: normalizationPolicy}}
|
||||
}
|
||||
|
||||
func (n *Normalizer) Normalize(ctx context.Context, req contracts.TypedNormalizeRequest[dnd.ItemEventList]) (contracts.TypedNormalizeResult[dnd.ItemEventList], error) {
|
||||
if n == nil {
|
||||
return contracts.TypedNormalizeResult[dnd.ItemEventList]{}, normalizerErrorf("normalizer must not be nil")
|
||||
}
|
||||
if ctx == nil {
|
||||
return contracts.TypedNormalizeResult[dnd.ItemEventList]{}, normalizerErrorf("context must not be nil")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return contracts.TypedNormalizeResult[dnd.ItemEventList]{}, normalizerErrorf("context error before normalize: %w", err)
|
||||
}
|
||||
|
||||
index := source.NewDocumentIndex(req.Source)
|
||||
value, warnings := normalizeList(req.MergeOutput.Value, index, shared.NewSourceRefOrderFromIndex(index))
|
||||
return contracts.TypedNormalizeResult[dnd.ItemEventList]{Value: value, Warnings: warnings}, nil
|
||||
}
|
||||
|
||||
type normalizedRecord struct {
|
||||
event dnd.ItemEvent
|
||||
inputIndex int
|
||||
}
|
||||
|
||||
func normalizeList(input dnd.ItemEventList, index source.DocumentIndex, order shared.SourceRefOrder) (dnd.ItemEventList, []contracts.Warning) {
|
||||
if input.Events == nil {
|
||||
return dnd.ItemEventList{}, nil
|
||||
}
|
||||
|
||||
records := make([]normalizedRecord, len(input.Events))
|
||||
warnings := make([]contracts.Warning, 0)
|
||||
for index, inputEvent := range input.Events {
|
||||
event, changedFields, refsChanged := normalizeEvent(inputEvent, order)
|
||||
records[index] = normalizedRecord{event: event, inputIndex: index}
|
||||
if len(changedFields) != 0 {
|
||||
warnings = append(warnings, contracts.Warning{
|
||||
Scope: eventScope(index),
|
||||
ReasonCode: ReasonCodeDisplayNormalized,
|
||||
Message: fmt.Sprintf("input index %d: normalized display whitespace in %s", index,
|
||||
diagnostics.Aggregate("fields", changedFields)),
|
||||
})
|
||||
}
|
||||
if refsChanged {
|
||||
warnings = append(warnings, contracts.Warning{
|
||||
Scope: eventScope(index),
|
||||
ReasonCode: ReasonCodeSourceRefsNormalized,
|
||||
Message: fmt.Sprintf("input index %d: source references normalized (original count %d, final count %d)",
|
||||
index, len(inputEvent.SourceRefs), len(event.SourceRefs)),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
sort.SliceStable(records, func(left, right int) bool {
|
||||
return itemeventmodel.Less(order, records[left].event, records[right].event)
|
||||
})
|
||||
for position, record := range records {
|
||||
if position == record.inputIndex {
|
||||
continue
|
||||
}
|
||||
warnings = append(warnings, contracts.Warning{
|
||||
Scope: eventScope(record.inputIndex),
|
||||
ReasonCode: ReasonCodeEventsReordered,
|
||||
Message: fmt.Sprintf("input index %d moved to normalized position %d", record.inputIndex, position),
|
||||
})
|
||||
}
|
||||
|
||||
output, duplicateWarnings := collapseDuplicates(records, index, order)
|
||||
warnings = append(warnings, duplicateWarnings...)
|
||||
return dnd.ItemEventList{Events: output}, diagnostics.LimitWarnings(warnings, "item_events", ReasonCodeWarningsOmitted)
|
||||
}
|
||||
|
||||
func normalizeEvent(input dnd.ItemEvent, order shared.SourceRefOrder) (dnd.ItemEvent, []string, bool) {
|
||||
output := cloneEvent(input)
|
||||
changedFields := make([]string, 0, 3)
|
||||
for _, field := range []struct {
|
||||
name string
|
||||
value *string
|
||||
}{
|
||||
{name: "name", value: &output.Name},
|
||||
{name: "from", value: &output.From},
|
||||
{name: "to", value: &output.To},
|
||||
} {
|
||||
trimmed := itemeventmodel.DisplayValue(*field.value)
|
||||
if *field.value != trimmed {
|
||||
*field.value = trimmed
|
||||
changedFields = append(changedFields, field.name)
|
||||
}
|
||||
}
|
||||
output.SourceRefs = order.Canonicalize(input.SourceRefs)
|
||||
return output, changedFields, !itemeventmodel.SourceRefsEqual(input.SourceRefs, output.SourceRefs)
|
||||
}
|
||||
|
||||
func cloneEvent(input dnd.ItemEvent) dnd.ItemEvent {
|
||||
output := input
|
||||
if input.Quantity != nil {
|
||||
quantity := *input.Quantity
|
||||
output.Quantity = &quantity
|
||||
}
|
||||
if input.SourceRefs != nil {
|
||||
output.SourceRefs = append([]source.SourceRef(nil), input.SourceRefs...)
|
||||
}
|
||||
return output
|
||||
}
|
||||
|
||||
type duplicateGroup struct {
|
||||
retainedIndex int
|
||||
removed []int
|
||||
}
|
||||
|
||||
func collapseDuplicates(records []normalizedRecord, index source.DocumentIndex, order shared.SourceRefOrder) ([]dnd.ItemEvent, []contracts.Warning) {
|
||||
if len(records) == 0 {
|
||||
return make([]dnd.ItemEvent, 0), nil
|
||||
}
|
||||
|
||||
keep := make([]bool, len(records))
|
||||
groups := make([]duplicateGroup, 0)
|
||||
groupByKey := make(map[string]int)
|
||||
for recordIndex, record := range records {
|
||||
if !itemeventmodel.ValidSourceRefs(index, record.event.SourceRefs) {
|
||||
keep[recordIndex] = true
|
||||
continue
|
||||
}
|
||||
key := itemeventmodel.ExactIdentity(order, record.event)
|
||||
groupIndex, exists := groupByKey[key]
|
||||
if !exists {
|
||||
groupByKey[key] = len(groups)
|
||||
groups = append(groups, duplicateGroup{retainedIndex: record.inputIndex})
|
||||
keep[recordIndex] = true
|
||||
continue
|
||||
}
|
||||
groups[groupIndex].removed = append(groups[groupIndex].removed, record.inputIndex)
|
||||
}
|
||||
|
||||
output := make([]dnd.ItemEvent, 0, len(records))
|
||||
for recordIndex, record := range records {
|
||||
if keep[recordIndex] {
|
||||
output = append(output, cloneEvent(record.event))
|
||||
}
|
||||
}
|
||||
warnings := make([]contracts.Warning, 0)
|
||||
for _, group := range groups {
|
||||
if len(group.removed) != 0 {
|
||||
warnings = append(warnings, duplicateWarning(group.retainedIndex, group.removed))
|
||||
}
|
||||
}
|
||||
return output, warnings
|
||||
}
|
||||
|
||||
func duplicateWarning(retainedIndex int, removed []int) contracts.Warning {
|
||||
issues := make([]string, len(removed))
|
||||
for index, removedIndex := range removed {
|
||||
issues[index] = fmt.Sprintf("removed input index %d", removedIndex)
|
||||
}
|
||||
return contracts.Warning{
|
||||
Scope: eventScope(retainedIndex),
|
||||
ReasonCode: ReasonCodeDuplicateCollapsed,
|
||||
Message: diagnostics.Aggregate(
|
||||
fmt.Sprintf("duplicate item event collapsed; retained input index %d", retainedIndex), issues),
|
||||
}
|
||||
}
|
||||
|
||||
func eventScope(index int) string { return fmt.Sprintf("events[%d]", index) }
|
||||
|
||||
func ModuleSpec() pipeline.ModuleSpec {
|
||||
return pipeline.ModuleSpec{
|
||||
Key: Key,
|
||||
Stage: pipeline.StageNormalize,
|
||||
Requires: append([]string(nil), requiredCapabilities...),
|
||||
Provides: append([]string(nil), providedCapabilities...),
|
||||
ArtifactKind: dnd.ItemEventListKind,
|
||||
}
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.NormalizerRegistry) error {
|
||||
return pipeline.RegisterNormalizerBuilder(registry, ModuleSpec(), validateOptions, func(request pipeline.BuildRequest) (contracts.Normalizer[dnd.ItemEventList], error) {
|
||||
options, err := DecodeOptions(request.Options)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return New(options), nil
|
||||
})
|
||||
}
|
||||
|
||||
func DecodeOptions(options map[string]any) (Options, error) {
|
||||
if err := pipeline.RejectUnknownOptions(options); err != nil {
|
||||
return Options{}, normalizerErrorf("%w", err)
|
||||
}
|
||||
return Options{}, nil
|
||||
}
|
||||
|
||||
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }
|
||||
|
||||
func normalizerErrorf(format string, args ...any) error {
|
||||
return fmt.Errorf("dnd item events normalizer: "+format, args...)
|
||||
}
|
||||
194
internal/modules/dnd/normalize/itemevents/normalizer_test.go
Normal file
194
internal/modules/dnd/normalize/itemevents/normalizer_test.go
Normal file
@@ -0,0 +1,194 @@
|
||||
package itemevents
|
||||
|
||||
import (
|
||||
"context"
|
||||
"reflect"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics"
|
||||
)
|
||||
|
||||
func TestNormalizeTrimsCanonicalizesAndClones(t *testing.T) {
|
||||
doc := &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 30}, {ID: 10}}}
|
||||
quantity := 4
|
||||
input := dnd.ItemEventList{Events: []dnd.ItemEvent{{
|
||||
Name: " Gold Pieces ", Kind: dnd.ItemEventKindTransferred, Quantity: &quantity, From: " Borin ", To: " Aria ",
|
||||
SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 10, EndUnitID: 10}, {SourceID: doc.ID, StartUnitID: 30, EndUnitID: 30}, {SourceID: doc.ID, StartUnitID: 30, EndUnitID: 30}},
|
||||
}}}
|
||||
result := normalize(t, doc, input)
|
||||
got := result.Value.Events[0]
|
||||
if got.Name != "Gold Pieces" || got.From != "Borin" || got.To != "Aria" || got.Quantity == input.Events[0].Quantity ||
|
||||
!reflect.DeepEqual(got.SourceRefs, []source.SourceRef{{SourceID: doc.ID, StartUnitID: 30, EndUnitID: 30}, {SourceID: doc.ID, StartUnitID: 10, EndUnitID: 10}}) {
|
||||
t.Fatalf("normalized event = %#v", got)
|
||||
}
|
||||
if !hasWarning(result.Warnings, ReasonCodeDisplayNormalized) || !hasWarning(result.Warnings, ReasonCodeSourceRefsNormalized) {
|
||||
t.Fatalf("warnings = %#v", result.Warnings)
|
||||
}
|
||||
second := normalize(t, doc, result.Value)
|
||||
if !reflect.DeepEqual(second.Value, result.Value) || len(second.Warnings) != 0 {
|
||||
t.Fatalf("second normalization = %#v; want idempotent output without warnings", second)
|
||||
}
|
||||
*got.Quantity = 99
|
||||
got.SourceRefs[0].StartUnitID = 999
|
||||
if quantity != 4 || input.Events[0].SourceRefs[1].StartUnitID != 30 {
|
||||
t.Fatalf("Normalize() aliased input: %#v", input)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizeOrdersAndCollapsesExactDuplicatesOnly(t *testing.T) {
|
||||
doc := &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 30}, {ID: 10}, {ID: 20}}}
|
||||
ref := func(unit int) source.SourceRef {
|
||||
return source.SourceRef{SourceID: doc.ID, StartUnitID: unit, EndUnitID: unit}
|
||||
}
|
||||
quantity := 2
|
||||
duplicate := dnd.ItemEvent{Name: " Potion ", Kind: dnd.ItemEventKindAcquired, Quantity: &quantity, To: " Aria ", SourceRefs: []source.SourceRef{ref(30)}}
|
||||
input := dnd.ItemEventList{Events: []dnd.ItemEvent{
|
||||
{Name: "Later", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(20)}},
|
||||
duplicate,
|
||||
{Name: "Potion", Kind: dnd.ItemEventKindAcquired, Quantity: &quantity, To: "Aria", SourceRefs: []source.SourceRef{ref(30)}},
|
||||
{Name: "Potion", Kind: dnd.ItemEventKindAcquired, Quantity: &quantity, To: "Borin", SourceRefs: []source.SourceRef{ref(30)}},
|
||||
{Name: "Potion", Kind: dnd.ItemEventKindAcquired, Quantity: &quantity, To: "Aria", SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 999, EndUnitID: 999}}},
|
||||
}}
|
||||
result := normalize(t, doc, input)
|
||||
got := result.Value.Events
|
||||
if len(got) != 4 || got[0].Name != "Potion" || got[0].To != "Aria" || got[1].To != "Borin" || got[2].Name != "Later" || got[3].SourceRefs[0].StartUnitID != 999 {
|
||||
t.Fatalf("normalized events = %#v", got)
|
||||
}
|
||||
if !hasWarning(result.Warnings, ReasonCodeEventsReordered) || !hasWarning(result.Warnings, ReasonCodeDuplicateCollapsed) {
|
||||
t.Fatalf("warnings = %#v", result.Warnings)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizeUsesEveryCanonicalTieBreaker(t *testing.T) {
|
||||
doc := testDocument()
|
||||
ref := func(unit int) source.SourceRef {
|
||||
return source.SourceRef{SourceID: doc.ID, StartUnitID: unit, EndUnitID: unit}
|
||||
}
|
||||
quantity := 1
|
||||
tests := []struct {
|
||||
name string
|
||||
earlier dnd.ItemEvent
|
||||
later dnd.ItemEvent
|
||||
}{
|
||||
{"evidence position", dnd.ItemEvent{Name: "Z", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "A", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(20)}}},
|
||||
{"normalized name", dnd.ItemEvent{Name: "A", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "B", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}},
|
||||
{"exact name", dnd.ItemEvent{Name: "A", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "a", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}},
|
||||
{"kind", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindAcquired, To: "Aria", SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}},
|
||||
{"from presence", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, From: "Aria", SourceRefs: []source.SourceRef{ref(10)}}},
|
||||
{"from comparison", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "A", To: "Z", SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "B", To: "Z", SourceRefs: []source.SourceRef{ref(10)}}},
|
||||
{"exact from", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "A", To: "Z", SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "a", To: "Z", SourceRefs: []source.SourceRef{ref(10)}}},
|
||||
{"to presence", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, To: "Aria", SourceRefs: []source.SourceRef{ref(10)}}},
|
||||
{"to comparison", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "A", To: "A", SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "A", To: "B", SourceRefs: []source.SourceRef{ref(10)}}},
|
||||
{"exact to", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "A", To: "A", SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindTransferred, From: "A", To: "a", SourceRefs: []source.SourceRef{ref(10)}}},
|
||||
{"quantity", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, Quantity: &quantity, SourceRefs: []source.SourceRef{ref(10)}}},
|
||||
{"reference sequence", dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10)}}, dnd.ItemEvent{Name: "Item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{ref(10), ref(20)}}},
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
result := normalize(t, doc, dnd.ItemEventList{Events: []dnd.ItemEvent{test.later, test.earlier}})
|
||||
if len(result.Value.Events) != 2 || !reflect.DeepEqual(result.Value.Events[0], test.earlier) || !reflect.DeepEqual(result.Value.Events[1], test.later) {
|
||||
t.Fatalf("normalized events = %#v; want %#v then %#v", result.Value.Events, test.earlier, test.later)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizePreservesEmptyAndMalformedCandidates(t *testing.T) {
|
||||
for _, input := range []dnd.ItemEventList{{}, {Events: []dnd.ItemEvent{}}} {
|
||||
result := normalize(t, nil, input)
|
||||
if (result.Value.Events == nil) != (input.Events == nil) {
|
||||
t.Fatalf("empty representation changed: %#v", result.Value)
|
||||
}
|
||||
}
|
||||
malformed := dnd.ItemEventList{Events: []dnd.ItemEvent{
|
||||
{Name: " Coin ", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 99, EndUnitID: 99}}},
|
||||
{Name: "Coin", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 99, EndUnitID: 99}}},
|
||||
}}
|
||||
result := normalize(t, testDocument(), malformed)
|
||||
if len(result.Value.Events) != 2 || result.Value.Events[0].Name != "Coin" || result.Value.Events[1].Name != "Coin" {
|
||||
t.Fatalf("malformed candidates were not safely retained: %#v", result.Value)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizeDoesNotInferOrConvertInventoryMeaning(t *testing.T) {
|
||||
doc := testDocument()
|
||||
quantity := 25
|
||||
input := dnd.ItemEventList{Events: []dnd.ItemEvent{
|
||||
{Name: " gold pieces ", Kind: dnd.ItemEventKindDiscovered, Quantity: &quantity, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 10, EndUnitID: 10}}},
|
||||
{Name: " gp ", Kind: dnd.ItemEventKindAcquired, Quantity: &quantity, To: " party ", SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 20, EndUnitID: 20}}},
|
||||
{Name: " potion of healing ", Kind: dnd.ItemEventKindTransferred, Quantity: &quantity, From: " Aria ", To: " Borin ", SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 30, EndUnitID: 30}}},
|
||||
}}
|
||||
result := normalize(t, doc, input)
|
||||
got := make(map[string]dnd.ItemEvent, len(result.Value.Events))
|
||||
for _, event := range result.Value.Events {
|
||||
got[event.Name] = event
|
||||
}
|
||||
if event := got["gold pieces"]; event.Kind != dnd.ItemEventKindDiscovered || event.Quantity == nil || *event.Quantity != 25 {
|
||||
t.Fatalf("discovery event was inferred or converted: %#v", event)
|
||||
}
|
||||
if event := got["gp"]; event.Kind != dnd.ItemEventKindAcquired || event.Quantity == nil || *event.Quantity != 25 || event.To != "party" {
|
||||
t.Fatalf("acquisition event was inferred or denomination-converted: %#v", event)
|
||||
}
|
||||
if event := got["potion of healing"]; event.Kind != dnd.ItemEventKindTransferred || event.From != "Aria" || event.To != "Borin" || event.Quantity == nil || *event.Quantity != 25 {
|
||||
t.Fatalf("transfer event was inferred, aliased, singularized, or ledger-adjusted: %#v", event)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizerContractAndBoundedWarnings(t *testing.T) {
|
||||
normalizer := New(Options{})
|
||||
if spec := ModuleSpec(); spec.Key != Key || spec.Stage != pipeline.StageNormalize || spec.ArtifactKind != dnd.ItemEventListKind || len(spec.ReferenceSlots) != 0 ||
|
||||
!reflect.DeepEqual(spec.Requires, []string{"merged"}) || !reflect.DeepEqual(spec.Provides, []string{"normalized"}) {
|
||||
t.Fatalf("ModuleSpec() = %#v", spec)
|
||||
}
|
||||
if metadata := normalizer.ManifestMetadata(); !reflect.DeepEqual(metadata, map[string]any{"normalization_policy": normalizationPolicy}) {
|
||||
t.Fatalf("ManifestMetadata() = %#v", metadata)
|
||||
}
|
||||
if fingerprints := normalizer.CheckpointFingerprints(); !reflect.DeepEqual(fingerprints, []pipeline.CheckpointFingerprint{{Name: "normalization_policy", Value: normalizationPolicy}}) {
|
||||
t.Fatalf("CheckpointFingerprints() = %#v", fingerprints)
|
||||
}
|
||||
registry := pipeline.NewNormalizerRegistry()
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatalf("Register() error = %v", err)
|
||||
}
|
||||
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
|
||||
t.Fatal("DecodeOptions() accepted unknown option")
|
||||
}
|
||||
|
||||
count := diagnostics.MaxWarnings + 5
|
||||
doc := &source.SourceDocument{ID: "session", Units: make([]source.SourceUnit, count)}
|
||||
input := dnd.ItemEventList{Events: make([]dnd.ItemEvent, count)}
|
||||
for index := range input.Events {
|
||||
doc.Units[index].ID = index + 1
|
||||
input.Events[index] = dnd.ItemEvent{Name: " Coin ", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: count - index, EndUnitID: count - index}}}
|
||||
}
|
||||
result := normalize(t, doc, input)
|
||||
if len(result.Warnings) != diagnostics.MaxWarnings || result.Warnings[len(result.Warnings)-1].ReasonCode != ReasonCodeWarningsOmitted {
|
||||
t.Fatalf("warnings = %#v", result.Warnings)
|
||||
}
|
||||
}
|
||||
|
||||
func normalize(t *testing.T, doc *source.SourceDocument, input dnd.ItemEventList) contracts.TypedNormalizeResult[dnd.ItemEventList] {
|
||||
t.Helper()
|
||||
result, err := New(Options{}).Normalize(context.Background(), contracts.TypedNormalizeRequest[dnd.ItemEventList]{Source: doc, MergeOutput: contracts.MergeArtifact[dnd.ItemEventList]{Value: input}})
|
||||
if err != nil {
|
||||
t.Fatalf("Normalize() error = %v", err)
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func testDocument() *source.SourceDocument {
|
||||
return &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 10}, {ID: 20}, {ID: 30}}}
|
||||
}
|
||||
|
||||
func hasWarning(warnings []contracts.Warning, reason string) bool {
|
||||
for _, warning := range warnings {
|
||||
if warning.ReasonCode == reason {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -3,11 +3,13 @@ package register
|
||||
import (
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
combatextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
|
||||
itemeventextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/itemevents"
|
||||
interactionextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcinteractions"
|
||||
npcextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcs"
|
||||
scenedescriptionextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/scenedescriptions"
|
||||
spellextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
|
||||
combatnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/combatturns"
|
||||
itemeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/itemevents"
|
||||
interactionnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/npcinteractions"
|
||||
npcnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/npcs"
|
||||
scenedescriptionnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/scenedescriptions"
|
||||
@@ -16,6 +18,10 @@ import (
|
||||
combatshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/combatturns/shape"
|
||||
combatsourcerefs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/combatturns/source_refs"
|
||||
combatrelatedness "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/combatturns/source_relatedness"
|
||||
itemeventinvariants "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/invariants"
|
||||
itemeventshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/shape"
|
||||
itemeventrefs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/source_refs"
|
||||
itemeventrelatedness "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/source_relatedness"
|
||||
interactioninvariants "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/npcinteractions/invariants"
|
||||
interactionregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/npcinteractions/registry"
|
||||
interactionshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/npcinteractions/shape"
|
||||
@@ -121,6 +127,33 @@ func registerDefaultChains(registry *pipeline.ValidatorChainRegistry) error {
|
||||
},
|
||||
})
|
||||
}},
|
||||
{name: "item events validator chain", register: func() error {
|
||||
return registry.Register(pipeline.ValidatorChainMapping{
|
||||
Stage: pipeline.StageExtract,
|
||||
Module: itemeventextract.Key,
|
||||
Validators: []pipeline.ModuleBinding{
|
||||
pipeline.Binding(validjson.Key),
|
||||
pipeline.Binding(itemeventshape.Key),
|
||||
pipeline.Binding(itemeventrefs.Key),
|
||||
pipeline.Binding(validjsonschema.Key),
|
||||
pipeline.Binding(itemeventrelatedness.Key),
|
||||
},
|
||||
})
|
||||
}},
|
||||
{name: "item events normalize validator chain", register: func() error {
|
||||
return registry.Register(pipeline.ValidatorChainMapping{
|
||||
Stage: pipeline.StageNormalize,
|
||||
Module: itemeventnormalize.Key,
|
||||
Validators: []pipeline.ModuleBinding{
|
||||
pipeline.Binding(validjson.Key),
|
||||
pipeline.Binding(itemeventshape.Key),
|
||||
pipeline.Binding(itemeventinvariants.Key),
|
||||
pipeline.Binding(itemeventrefs.Key),
|
||||
pipeline.Binding(validjsonschema.Key),
|
||||
pipeline.Binding(itemeventrelatedness.Key),
|
||||
},
|
||||
})
|
||||
}},
|
||||
{name: "npc interactions validator chain", register: func() error {
|
||||
return registry.Register(pipeline.ValidatorChainMapping{
|
||||
Stage: pipeline.StageExtract,
|
||||
|
||||
@@ -70,6 +70,27 @@ func appendCombatTurnLists(values []dnd.CombatTurnList) (dnd.CombatTurnList, err
|
||||
return combined, nil
|
||||
}
|
||||
|
||||
func appendItemEventLists(values []dnd.ItemEventList) (dnd.ItemEventList, error) {
|
||||
count := 0
|
||||
present := false
|
||||
for _, value := range values {
|
||||
if value.Events != nil {
|
||||
present = true
|
||||
}
|
||||
count += len(value.Events)
|
||||
}
|
||||
if !present {
|
||||
return dnd.ItemEventList{}, nil
|
||||
}
|
||||
combined := dnd.ItemEventList{Events: make([]dnd.ItemEvent, 0, count)}
|
||||
for _, value := range values {
|
||||
for _, event := range value.Events {
|
||||
combined.Events = append(combined.Events, cloneItemEvent(event))
|
||||
}
|
||||
}
|
||||
return combined, nil
|
||||
}
|
||||
|
||||
func appendNPCInteractionLists(values []dnd.NPCInteractionList) (dnd.NPCInteractionList, error) {
|
||||
count := 0
|
||||
present := false
|
||||
@@ -116,6 +137,16 @@ func cloneCombatTurn(value dnd.CombatTurn) dnd.CombatTurn {
|
||||
return clone
|
||||
}
|
||||
|
||||
func cloneItemEvent(value dnd.ItemEvent) dnd.ItemEvent {
|
||||
clone := value
|
||||
if value.Quantity != nil {
|
||||
quantity := *value.Quantity
|
||||
clone.Quantity = &quantity
|
||||
}
|
||||
clone.SourceRefs = cloneSourceRefs(value.SourceRefs)
|
||||
return clone
|
||||
}
|
||||
|
||||
func cloneSpellCast(value dnd.SpellCast) dnd.SpellCast {
|
||||
clone := value
|
||||
clone.SourceRefs = cloneSourceRefs(value.SourceRefs)
|
||||
|
||||
@@ -6,16 +6,19 @@ import (
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/chunk/scenes"
|
||||
combatcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/combatturns"
|
||||
itemeventcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/itemevents"
|
||||
interactioncodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcinteractions"
|
||||
npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs"
|
||||
scenedescriptioncodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
|
||||
spellcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/spells"
|
||||
combatextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
|
||||
itemeventextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/itemevents"
|
||||
interactionextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcinteractions"
|
||||
npcextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcs"
|
||||
scenedescriptionextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/scenedescriptions"
|
||||
spellextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
|
||||
combatnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/combatturns"
|
||||
itemeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/itemevents"
|
||||
interactionnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/npcinteractions"
|
||||
npcnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/npcs"
|
||||
scenedescriptionnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/scenedescriptions"
|
||||
@@ -30,6 +33,7 @@ func registerModules(registries pipeline.Registries) error {
|
||||
{name: "spells codec", register: func() error { return pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, codec) }},
|
||||
{name: "npcs codec", register: func() error { return pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, npccodec.New()) }},
|
||||
{name: "combat turns codec", register: func() error { return pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, combatcodec.New()) }},
|
||||
{name: "item events codec", register: func() error { return pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, itemeventcodec.New()) }},
|
||||
{name: "npc interactions codec", register: func() error { return pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, interactioncodec.New()) }},
|
||||
{name: "scene descriptions codec", register: func() error {
|
||||
return pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, scenedescriptioncodec.New())
|
||||
@@ -38,6 +42,7 @@ func registerModules(registries pipeline.Registries) error {
|
||||
{name: "spells extractor", register: func() error { return spellextract.Register(registries.Extractors) }},
|
||||
{name: "npcs extractor", register: func() error { return npcextract.Register(registries.Extractors) }},
|
||||
{name: "combat turns extractor", register: func() error { return combatextract.Register(registries.Extractors) }},
|
||||
{name: "item events extractor", register: func() error { return itemeventextract.Register(registries.Extractors) }},
|
||||
{name: "npc interactions extractor", register: func() error { return interactionextract.Register(registries.Extractors) }},
|
||||
{name: "scene descriptions extractor", register: func() error { return scenedescriptionextract.Register(registries.Extractors) }},
|
||||
{name: "spell-list appendorder merger", register: func() error {
|
||||
@@ -49,6 +54,9 @@ func registerModules(registries pipeline.Registries) error {
|
||||
{name: "combat-turn-list appendorder merger", register: func() error {
|
||||
return appendorder.RegisterTyped(registries.Mergers, dnd.CombatTurnListKind, appendCombatTurnLists)
|
||||
}},
|
||||
{name: "item-event-list appendorder merger", register: func() error {
|
||||
return appendorder.RegisterTyped(registries.Mergers, dnd.ItemEventListKind, appendItemEventLists)
|
||||
}},
|
||||
{name: "npc-interaction-list appendorder merger", register: func() error {
|
||||
return appendorder.RegisterTyped(registries.Mergers, dnd.NPCInteractionListKind, appendNPCInteractionLists)
|
||||
}},
|
||||
@@ -58,6 +66,7 @@ func registerModules(registries pipeline.Registries) error {
|
||||
{name: "spells normalizer", register: func() error { return spellnormalize.Register(registries.Normalizers) }},
|
||||
{name: "npcs normalizer", register: func() error { return npcnormalize.Register(registries.Normalizers) }},
|
||||
{name: "combat turns normalizer", register: func() error { return combatnormalize.Register(registries.Normalizers) }},
|
||||
{name: "item events normalizer", register: func() error { return itemeventnormalize.Register(registries.Normalizers) }},
|
||||
{name: "npc interactions normalizer", register: func() error { return interactionnormalize.Register(registries.Normalizers) }},
|
||||
{name: "scene descriptions normalizer", register: func() error { return scenedescriptionnormalize.Register(registries.Normalizers) }},
|
||||
{name: "spell-list noop normalizer", register: func() error {
|
||||
@@ -69,6 +78,9 @@ func registerModules(registries pipeline.Registries) error {
|
||||
{name: "combat-turn-list noop normalizer", register: func() error {
|
||||
return noop.RegisterTyped[dnd.CombatTurnList](registries.Normalizers, dnd.CombatTurnListKind)
|
||||
}},
|
||||
{name: "item-event-list noop normalizer", register: func() error {
|
||||
return noop.RegisterTyped[dnd.ItemEventList](registries.Normalizers, dnd.ItemEventListKind)
|
||||
}},
|
||||
{name: "npc-interaction-list noop normalizer", register: func() error {
|
||||
return noop.RegisterTyped[dnd.NPCInteractionList](registries.Normalizers, dnd.NPCInteractionListKind)
|
||||
}},
|
||||
@@ -84,6 +96,7 @@ func registerPromptAssets(assets *llm.AssetRegistry) error {
|
||||
{name: "spells prompt assets", register: func() error { return spellextract.RegisterPromptAssets(assets) }},
|
||||
{name: "npcs prompt assets", register: func() error { return npcextract.RegisterPromptAssets(assets) }},
|
||||
{name: "combat turns prompt assets", register: func() error { return combatextract.RegisterPromptAssets(assets) }},
|
||||
{name: "item events prompt assets", register: func() error { return itemeventextract.RegisterPromptAssets(assets) }},
|
||||
{name: "npc interactions prompt assets", register: func() error { return interactionextract.RegisterPromptAssets(assets) }},
|
||||
{name: "scene descriptions prompt assets", register: func() error { return scenedescriptionextract.RegisterPromptAssets(assets) }},
|
||||
})
|
||||
|
||||
@@ -13,11 +13,13 @@ import (
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
combatextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
|
||||
itemeventextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/itemevents"
|
||||
interactionextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcinteractions"
|
||||
npcextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcs"
|
||||
scenedescriptionextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/scenedescriptions"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
|
||||
combatnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/combatturns"
|
||||
itemeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/itemevents"
|
||||
interactionnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/npcinteractions"
|
||||
npcnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/npcs"
|
||||
scenedescriptionnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/scenedescriptions"
|
||||
@@ -31,13 +33,14 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
assertContainsKeys(t, "chunkers", registries.Chunkers.RegisteredKeys(), []string{"dnd/scenes"})
|
||||
assertContainsKeys(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells", npcextract.Key, combatextract.Key, interactionextract.Key, scenedescriptionextract.Key})
|
||||
assertContainsKeys(t, "normalizers", registries.Normalizers.RegisteredKeys(), []string{spellnormalize.Key, npcnormalize.Key, combatnormalize.Key, interactionnormalize.Key, scenedescriptionnormalize.Key, pipeline.DefaultNormalizeModule})
|
||||
assertContainsArtifactKinds(t, registries.ArtifactCodecs.RegisteredKinds(), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.NPCInteractionListKind, dnd.SceneDescriptionListKind})
|
||||
assertContainsArtifactKinds(t, registries.Mergers.RegisteredArtifactKinds(pipeline.DefaultMergeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.NPCInteractionListKind, dnd.SceneDescriptionListKind})
|
||||
assertContainsArtifactKinds(t, registries.Normalizers.RegisteredArtifactKinds(pipeline.DefaultNormalizeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.NPCInteractionListKind, dnd.SceneDescriptionListKind})
|
||||
assertContainsKeys(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells", npcextract.Key, combatextract.Key, itemeventextract.Key, interactionextract.Key, scenedescriptionextract.Key})
|
||||
assertContainsKeys(t, "normalizers", registries.Normalizers.RegisteredKeys(), []string{spellnormalize.Key, npcnormalize.Key, combatnormalize.Key, itemeventnormalize.Key, interactionnormalize.Key, scenedescriptionnormalize.Key, pipeline.DefaultNormalizeModule})
|
||||
assertContainsArtifactKinds(t, registries.ArtifactCodecs.RegisteredKinds(), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind, dnd.NPCInteractionListKind, dnd.SceneDescriptionListKind})
|
||||
assertContainsArtifactKinds(t, registries.Mergers.RegisteredArtifactKinds(pipeline.DefaultMergeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind, dnd.NPCInteractionListKind, dnd.SceneDescriptionListKind})
|
||||
assertContainsArtifactKinds(t, registries.Normalizers.RegisteredArtifactKinds(pipeline.DefaultNormalizeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind, dnd.NPCInteractionListKind, dnd.SceneDescriptionListKind})
|
||||
assertContainsArtifactKinds(t, registries.Normalizers.RegisteredArtifactKinds(npcnormalize.Key), []contracts.ArtifactKind{dnd.NPCListKind})
|
||||
assertContainsArtifactKinds(t, registries.Normalizers.RegisteredArtifactKinds(combatnormalize.Key), []contracts.ArtifactKind{dnd.CombatTurnListKind})
|
||||
assertContainsArtifactKinds(t, registries.Normalizers.RegisteredArtifactKinds(itemeventnormalize.Key), []contracts.ArtifactKind{dnd.ItemEventListKind})
|
||||
assertContainsArtifactKinds(t, registries.Normalizers.RegisteredArtifactKinds(interactionnormalize.Key), []contracts.ArtifactKind{dnd.NPCInteractionListKind})
|
||||
assertContainsArtifactKinds(t, registries.Normalizers.RegisteredArtifactKinds(scenedescriptionnormalize.Key), []contracts.ArtifactKind{dnd.SceneDescriptionListKind})
|
||||
assertContainsKeys(t, "validators", registries.Validators.RegisteredKeys(), []string{
|
||||
@@ -53,6 +56,10 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
|
||||
"extract/dnd/combat-turns/source_refs",
|
||||
"extract/dnd/combat-turns/source_relatedness",
|
||||
"normalize/dnd/combat-turns/invariants",
|
||||
"extract/dnd/item-events/shape",
|
||||
"extract/dnd/item-events/source_refs",
|
||||
"extract/dnd/item-events/source_relatedness",
|
||||
"normalize/dnd/item-events/invariants",
|
||||
"extract/dnd/npc-interactions/shape",
|
||||
"extract/dnd/npc-interactions/registry",
|
||||
"extract/dnd/npc-interactions/source_refs",
|
||||
@@ -121,6 +128,27 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
|
||||
if got := registries.ValidatorChains.Validators(pipeline.StageNormalize, combatnormalize.Key); !reflect.DeepEqual(got, combatNormalizeChain) {
|
||||
t.Fatalf("combat normalize validator chain = %#v, want %#v", got, combatNormalizeChain)
|
||||
}
|
||||
itemEventExtractChain := []pipeline.ModuleBinding{
|
||||
pipeline.Binding("generic/valid_json"),
|
||||
pipeline.Binding("extract/dnd/item-events/shape"),
|
||||
pipeline.Binding("extract/dnd/item-events/source_refs"),
|
||||
pipeline.Binding("generic/valid_json_schema"),
|
||||
pipeline.Binding("extract/dnd/item-events/source_relatedness"),
|
||||
}
|
||||
itemEventNormalizeChain := []pipeline.ModuleBinding{
|
||||
pipeline.Binding("generic/valid_json"),
|
||||
pipeline.Binding("extract/dnd/item-events/shape"),
|
||||
pipeline.Binding("normalize/dnd/item-events/invariants"),
|
||||
pipeline.Binding("extract/dnd/item-events/source_refs"),
|
||||
pipeline.Binding("generic/valid_json_schema"),
|
||||
pipeline.Binding("extract/dnd/item-events/source_relatedness"),
|
||||
}
|
||||
if got := registries.ValidatorChains.Validators(pipeline.StageExtract, itemeventextract.Key); !reflect.DeepEqual(got, itemEventExtractChain) {
|
||||
t.Fatalf("item event extract validator chain = %#v, want %#v", got, itemEventExtractChain)
|
||||
}
|
||||
if got := registries.ValidatorChains.Validators(pipeline.StageNormalize, itemeventnormalize.Key); !reflect.DeepEqual(got, itemEventNormalizeChain) {
|
||||
t.Fatalf("item event normalize validator chain = %#v, want %#v", got, itemEventNormalizeChain)
|
||||
}
|
||||
interactionExtractChain := []pipeline.ModuleBinding{
|
||||
pipeline.Binding("generic/valid_json"),
|
||||
pipeline.Binding("extract/dnd/npc-interactions/shape"),
|
||||
@@ -196,6 +224,14 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
|
||||
"dnd.combat_turns/sharedassets/common-dnd-system.md",
|
||||
"dnd.combat_turns/sharedassets/common-dnd-transcript.md",
|
||||
"dnd.combat_turns/task.md",
|
||||
"dnd.item_events/dnd.item_events.yaml",
|
||||
"dnd.item_events/instructions.md",
|
||||
"dnd.item_events/sharedassets/common-dnd-extraction-evidence.md",
|
||||
"dnd.item_events/sharedassets/common-dnd-identity.md",
|
||||
"dnd.item_events/sharedassets/common-dnd-references.md",
|
||||
"dnd.item_events/sharedassets/common-dnd-system.md",
|
||||
"dnd.item_events/sharedassets/common-dnd-transcript.md",
|
||||
"dnd.item_events/task.md",
|
||||
"dnd.npc_interactions/dnd.npc_interactions.yaml",
|
||||
"dnd.npc_interactions/instructions.md",
|
||||
"dnd.npc_interactions/sharedassets/common-dnd-npcs.md",
|
||||
@@ -213,6 +249,7 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
|
||||
"dnd_spells_llm.v1.json",
|
||||
"dnd_npcs_llm.v1.json",
|
||||
"dnd_combat_turns_llm.v1.json",
|
||||
"dnd_item_events_llm.v1.json",
|
||||
"dnd_npc_interactions_llm.v1.json",
|
||||
"dnd_scene_descriptions_llm.v1.json",
|
||||
})
|
||||
@@ -237,6 +274,16 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
|
||||
if spec, ok := registries.Normalizers.Spec(combatnormalize.Key); !ok || spec.ArtifactKind != dnd.CombatTurnListKind || spec.Stage != pipeline.StageNormalize {
|
||||
t.Fatalf("combat normalizer spec = %#v, present = %t; want dnd combat-turn-list artifact", spec, ok)
|
||||
}
|
||||
itemEventExtractSpec, itemEventExtractOK := registries.Extractors.Spec(itemeventextract.Key)
|
||||
itemEventNormalizeSpec, itemEventNormalizeOK := registries.Normalizers.Spec(itemeventnormalize.Key)
|
||||
if !itemEventExtractOK || itemEventExtractSpec.ArtifactKind != dnd.ItemEventListKind || !itemEventNormalizeOK || itemEventNormalizeSpec.ArtifactKind != dnd.ItemEventListKind || itemEventNormalizeSpec.Stage != pipeline.StageNormalize || len(itemEventNormalizeSpec.ReferenceSlots) != 0 {
|
||||
t.Fatalf("item event specs = %#v / %#v, present = %t / %t", itemEventExtractSpec, itemEventNormalizeSpec, itemEventExtractOK, itemEventNormalizeOK)
|
||||
}
|
||||
for _, slot := range itemEventExtractSpec.ReferenceSlots {
|
||||
if slot.Required || len(slot.AcceptedArtifactKinds) != 0 {
|
||||
t.Fatalf("item event extractor has a generated-reference dependency: %#v", slot)
|
||||
}
|
||||
}
|
||||
interactionExtractSpec, extractOK := registries.Extractors.Spec(interactionextract.Key)
|
||||
interactionNormalizeSpec, normalizeOK := registries.Normalizers.Spec(interactionnormalize.Key)
|
||||
if !extractOK || interactionExtractSpec.ArtifactKind != dnd.NPCInteractionListKind || !normalizeOK || interactionNormalizeSpec.ArtifactKind != dnd.NPCInteractionListKind || interactionNormalizeSpec.Stage != pipeline.StageNormalize {
|
||||
@@ -409,6 +456,32 @@ func TestAppendCombatTurnListsPreservesOrderPresenceAndOwnership(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestAppendItemEventListsPreservesOrderPresenceAndOwnership(t *testing.T) {
|
||||
quantity := 3
|
||||
refs := []source.SourceRef{{SourceID: "session", StartUnitID: 1, EndUnitID: 1}}
|
||||
input := []dnd.ItemEventList{
|
||||
{},
|
||||
{Events: []dnd.ItemEvent{}},
|
||||
{Events: []dnd.ItemEvent{{Name: "first", Kind: dnd.ItemEventKindDiscovered, Quantity: &quantity, SourceRefs: refs}}},
|
||||
{Events: []dnd.ItemEvent{{Name: "second", Kind: dnd.ItemEventKindAcquired, To: "party", SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 2, EndUnitID: 2}}}}},
|
||||
}
|
||||
got, err := appendItemEventLists(input)
|
||||
if err != nil || got.Events == nil || !reflect.DeepEqual([]string{got.Events[0].Name, got.Events[1].Name}, []string{"first", "second"}) {
|
||||
t.Fatalf("appendItemEventLists() = %#v, error = %v", got, err)
|
||||
}
|
||||
*got.Events[0].Quantity = 99
|
||||
got.Events[0].SourceRefs[0].StartUnitID = 999
|
||||
if quantity != 3 || input[2].Events[0].SourceRefs[0].StartUnitID != 1 {
|
||||
t.Fatal("merged item events share quantity or source-reference storage")
|
||||
}
|
||||
for _, values := range [][]dnd.ItemEventList{nil, []dnd.ItemEventList{{}, {}}} {
|
||||
result, err := appendItemEventLists(values)
|
||||
if err != nil || result.Events != nil {
|
||||
t.Fatalf("nil-only merge = %#v, %v; want nil events", result, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAppendListsPreserveNestedSourceReferencePresence(t *testing.T) {
|
||||
spells, err := appendSpellLists([]dnd.SpellList{{SpellCasts: []dnd.SpellCast{{SourceRefs: []source.SourceRef{}}}}})
|
||||
if err != nil || spells.SpellCasts[0].SourceRefs == nil {
|
||||
@@ -429,6 +502,11 @@ func TestAppendListsPreserveNestedSourceReferencePresence(t *testing.T) {
|
||||
if err != nil || interactions.Interactions[0].SourceRefs == nil {
|
||||
t.Fatalf("appendNPCInteractionLists() = %#v, %v; want present-empty source refs", interactions, err)
|
||||
}
|
||||
|
||||
events, err := appendItemEventLists([]dnd.ItemEventList{{Events: []dnd.ItemEvent{{SourceRefs: []source.SourceRef{}}}}})
|
||||
if err != nil || events.Events[0].SourceRefs == nil {
|
||||
t.Fatalf("appendItemEventLists() = %#v, %v; want present-empty source refs", events, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRegisterRejectsMissingDNDDependenciesBeforeMutation(t *testing.T) {
|
||||
|
||||
@@ -7,6 +7,10 @@ import (
|
||||
combatshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/combatturns/shape"
|
||||
combatsourcerefs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/combatturns/source_refs"
|
||||
combatrelatedness "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/combatturns/source_relatedness"
|
||||
itemeventinvariants "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/invariants"
|
||||
itemeventshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/shape"
|
||||
itemeventrefs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/source_refs"
|
||||
itemeventrelatedness "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/source_relatedness"
|
||||
interactioninvariants "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/npcinteractions/invariants"
|
||||
interactionregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/npcinteractions/registry"
|
||||
interactionshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/npcinteractions/shape"
|
||||
@@ -42,6 +46,10 @@ func registerValidators(registries pipeline.Registries) error {
|
||||
{name: "combat source references validator", register: func() error { return combatsourcerefs.Register(registries.Validators) }},
|
||||
{name: "combat source relatedness validator", register: func() error { return combatrelatedness.Register(registries.Validators) }},
|
||||
{name: "combat normalized invariants validator", register: func() error { return combatinvariants.Register(registries.Validators) }},
|
||||
{name: "item event shape validator", register: func() error { return itemeventshape.Register(registries.Validators) }},
|
||||
{name: "item event source references validator", register: func() error { return itemeventrefs.Register(registries.Validators) }},
|
||||
{name: "item event source relatedness validator", register: func() error { return itemeventrelatedness.Register(registries.Validators) }},
|
||||
{name: "item event normalized invariants validator", register: func() error { return itemeventinvariants.Register(registries.Validators) }},
|
||||
{name: "npc interaction shape validator", register: func() error { return interactionshape.Register(registries.Validators) }},
|
||||
{name: "npc interaction registry validator", register: func() error { return interactionregistry.Register(registries.Validators) }},
|
||||
{name: "npc interaction source references validator", register: func() error { return interactionrefs.Register(registries.Validators) }},
|
||||
@@ -69,6 +77,12 @@ func registerValidators(registries pipeline.Registries) error {
|
||||
{name: "combat-turn-list always reject validator", register: func() error {
|
||||
return alwaysreject.RegisterTyped[dnd.CombatTurnList](registries.Validators, dnd.CombatTurnListKind)
|
||||
}},
|
||||
{name: "item-event-list always accept validator", register: func() error {
|
||||
return alwaysaccept.RegisterTyped[dnd.ItemEventList](registries.Validators, dnd.ItemEventListKind)
|
||||
}},
|
||||
{name: "item-event-list always reject validator", register: func() error {
|
||||
return alwaysreject.RegisterTyped[dnd.ItemEventList](registries.Validators, dnd.ItemEventListKind)
|
||||
}},
|
||||
{name: "npc-interaction-list always accept validator", register: func() error {
|
||||
return alwaysaccept.RegisterTyped[dnd.NPCInteractionList](registries.Validators, dnd.NPCInteractionListKind)
|
||||
}},
|
||||
|
||||
291
internal/modules/dnd/scenedescriptions/registry/registry.go
Normal file
291
internal/modules/dnd/scenedescriptions/registry/registry.go
Normal file
@@ -0,0 +1,291 @@
|
||||
// Package registry resolves scene-description artifacts into immutable combat
|
||||
// eligibility data.
|
||||
package registry
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"mime"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
scenecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
|
||||
)
|
||||
|
||||
const (
|
||||
ReferenceSlot = "scene_descriptions"
|
||||
MaxBytes = 1048576
|
||||
emptyProjection = `{"scenes":[]}`
|
||||
)
|
||||
|
||||
// MatchState identifies how a chunk relates to the resolved scene records.
|
||||
type MatchState string
|
||||
|
||||
const (
|
||||
MatchExact MatchState = "exact"
|
||||
MatchMissing MatchState = "missing"
|
||||
MatchMismatched MatchState = "mismatched"
|
||||
)
|
||||
|
||||
// ChunkMatch describes whether a scene record exactly covers a chunk. Kind is
|
||||
// populated only for an exact match.
|
||||
type ChunkMatch struct {
|
||||
State MatchState
|
||||
Kind dnd.SceneKind
|
||||
}
|
||||
|
||||
type sceneEligibility struct {
|
||||
sourceID string
|
||||
startUnitID int
|
||||
endUnitID int
|
||||
kind dnd.SceneKind
|
||||
}
|
||||
|
||||
// Registry is an immutable, validated eligibility view. It retains no scene
|
||||
// prose, raw reference bytes, or reference provenance.
|
||||
type Registry struct {
|
||||
bound bool
|
||||
digest string
|
||||
scenesByID map[string]sceneEligibility
|
||||
}
|
||||
|
||||
// Resolver holds an immutable construction-time view and memoizes immutable
|
||||
// operation-time views by raw reference identity and eligibility digest.
|
||||
type Resolver struct {
|
||||
seeded *Registry
|
||||
|
||||
mu sync.Mutex
|
||||
cache map[string]*Registry
|
||||
rawCache map[string]*Registry
|
||||
}
|
||||
|
||||
// NewResolver validates a materialized construction-time scene reference. An
|
||||
// empty slot is permitted because a generated reference is supplied only at
|
||||
// operation time.
|
||||
func NewResolver(references contracts.ReferenceSet) (*Resolver, error) {
|
||||
seeded, err := Resolve(constructionReferences(references))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Resolver{
|
||||
seeded: seeded,
|
||||
cache: make(map[string]*Registry),
|
||||
rawCache: make(map[string]*Registry),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func constructionReferences(references contracts.ReferenceSet) contracts.ReferenceSet {
|
||||
slot, ok := references.Slots[ReferenceSlot]
|
||||
if !ok || len(slot.Items) > 0 {
|
||||
return references
|
||||
}
|
||||
cloned := contracts.ReferenceSet{Slots: make(map[string]contracts.ResolvedReferenceSlot, len(references.Slots))}
|
||||
for name, value := range references.Slots {
|
||||
cloned.Slots[name] = value
|
||||
}
|
||||
delete(cloned.Slots, ReferenceSlot)
|
||||
return cloned
|
||||
}
|
||||
|
||||
// Seeded returns the construction-time eligibility view.
|
||||
func (r *Resolver) Seeded() *Registry {
|
||||
if r == nil {
|
||||
return nil
|
||||
}
|
||||
return r.seeded
|
||||
}
|
||||
|
||||
// Resolve returns the generated operation-time view when the scene slot is
|
||||
// present, otherwise it returns the construction-time view.
|
||||
func (r *Resolver) Resolve(references contracts.ReferenceSet) (*Registry, error) {
|
||||
if r == nil {
|
||||
return Resolve(references)
|
||||
}
|
||||
if _, ok := references.Slots[ReferenceSlot]; !ok {
|
||||
return r.seeded, nil
|
||||
}
|
||||
|
||||
slot := references.Slots[ReferenceSlot]
|
||||
rawKey := ""
|
||||
if len(slot.Items) == 1 {
|
||||
rawKey = rawReferenceKey(slot.Items[0])
|
||||
}
|
||||
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if rawKey != "" {
|
||||
if cached, ok := r.rawCache[rawKey]; ok {
|
||||
return cached, nil
|
||||
}
|
||||
}
|
||||
resolved, err := Resolve(references)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if sameEligibility(r.seeded, resolved) {
|
||||
if rawKey != "" {
|
||||
r.rawCache[rawKey] = r.seeded
|
||||
}
|
||||
return r.seeded, nil
|
||||
}
|
||||
if cached, ok := r.cache[resolved.EligibilityDigest()]; ok {
|
||||
if rawKey != "" {
|
||||
r.rawCache[rawKey] = cached
|
||||
}
|
||||
return cached, nil
|
||||
}
|
||||
r.cache[resolved.EligibilityDigest()] = resolved
|
||||
if rawKey != "" {
|
||||
r.rawCache[rawKey] = resolved
|
||||
}
|
||||
return resolved, nil
|
||||
}
|
||||
|
||||
func rawReferenceKey(item contracts.ReferenceItem) string {
|
||||
digest := strings.ToLower(strings.TrimSpace(item.Digest))
|
||||
if !validSHA256Digest(digest) {
|
||||
digest = semanticDigest(item.Content)
|
||||
}
|
||||
return strings.ToLower(strings.TrimSpace(item.MediaType)) + "\x00" + digest
|
||||
}
|
||||
|
||||
func validSHA256Digest(value string) bool {
|
||||
if len(value) != len("sha256:")+sha256.Size*2 || !strings.HasPrefix(value, "sha256:") {
|
||||
return false
|
||||
}
|
||||
decoded, err := hex.DecodeString(strings.TrimPrefix(value, "sha256:"))
|
||||
return err == nil && len(decoded) == sha256.Size
|
||||
}
|
||||
|
||||
func sameEligibility(first, second *Registry) bool {
|
||||
if first == nil || second == nil {
|
||||
return first == second
|
||||
}
|
||||
return first.bound == second.bound && first.digest == second.digest
|
||||
}
|
||||
|
||||
// Resolve validates one optional scene-description reference into an
|
||||
// eligibility-only view. An absent slot is represented by the canonical empty
|
||||
// projection so required generated bindings have a stable sentinel identity.
|
||||
func Resolve(references contracts.ReferenceSet) (*Registry, error) {
|
||||
slot, ok := references.Slots[ReferenceSlot]
|
||||
if !ok {
|
||||
return &Registry{
|
||||
digest: semanticDigest([]byte(emptyProjection)),
|
||||
scenesByID: map[string]sceneEligibility{},
|
||||
}, nil
|
||||
}
|
||||
if len(slot.Items) != 1 {
|
||||
return nil, fmt.Errorf("reference slot %q must contain exactly one item", ReferenceSlot)
|
||||
}
|
||||
|
||||
item := slot.Items[0]
|
||||
mediaType, _, err := mime.ParseMediaType(item.MediaType)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("reference slot %q item media type is invalid", ReferenceSlot)
|
||||
}
|
||||
if !strings.EqualFold(mediaType, scenecodec.MediaType) {
|
||||
return nil, fmt.Errorf("reference slot %q item media type must be %s", ReferenceSlot, scenecodec.MediaType)
|
||||
}
|
||||
if len(item.Content) > MaxBytes {
|
||||
return nil, fmt.Errorf("reference slot %q item is %d bytes, limit %d", ReferenceSlot, len(item.Content), MaxBytes)
|
||||
}
|
||||
|
||||
value, err := scenecodec.New().Decode(item.Content)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("decode scene eligibility: invalid approved scene JSON")
|
||||
}
|
||||
|
||||
scenesByID := make(map[string]sceneEligibility, len(value.Scenes))
|
||||
projection := make([]projectedScene, 0, len(value.Scenes))
|
||||
for _, scene := range value.Scenes {
|
||||
if _, exists := scenesByID[scene.ID]; exists {
|
||||
return nil, fmt.Errorf("scene eligibility contains duplicate scene ID %q", scene.ID)
|
||||
}
|
||||
eligibility := sceneEligibility{
|
||||
sourceID: scene.SourceRef.SourceID,
|
||||
startUnitID: scene.SourceRef.StartUnitID,
|
||||
endUnitID: scene.SourceRef.EndUnitID,
|
||||
kind: scene.Kind,
|
||||
}
|
||||
scenesByID[scene.ID] = eligibility
|
||||
projection = append(projection, projectedScene{
|
||||
ID: scene.ID,
|
||||
SourceRef: source.SourceRef{SourceID: eligibility.sourceID, StartUnitID: eligibility.startUnitID, EndUnitID: eligibility.endUnitID},
|
||||
Kind: eligibility.kind,
|
||||
})
|
||||
}
|
||||
|
||||
content, err := eligibilityProjection(projection)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("encode scene eligibility projection: %w", err)
|
||||
}
|
||||
return &Registry{
|
||||
bound: true,
|
||||
digest: semanticDigest(content),
|
||||
scenesByID: scenesByID,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Bound reports whether an approved scene-description reference was supplied.
|
||||
func (r *Registry) Bound() bool { return r != nil && r.bound }
|
||||
|
||||
// Count reports the number of scene records in the eligibility view.
|
||||
func (r *Registry) Count() int {
|
||||
if r == nil {
|
||||
return 0
|
||||
}
|
||||
return len(r.scenesByID)
|
||||
}
|
||||
|
||||
// EligibilityDigest returns the semantic SHA-256 digest of the canonical
|
||||
// eligibility projection, including the canonical unbound projection.
|
||||
func (r *Registry) EligibilityDigest() string {
|
||||
if r == nil {
|
||||
return ""
|
||||
}
|
||||
return r.digest
|
||||
}
|
||||
|
||||
// Match classifies a chunk against its scene record. The kind is intentionally
|
||||
// unavailable unless the ID and complete source range match exactly.
|
||||
func (r *Registry) Match(chunk *source.Chunk) ChunkMatch {
|
||||
if r == nil || chunk == nil {
|
||||
return ChunkMatch{State: MatchMissing}
|
||||
}
|
||||
scene, ok := r.scenesByID[chunk.ID]
|
||||
if !ok {
|
||||
return ChunkMatch{State: MatchMissing}
|
||||
}
|
||||
if scene.sourceID != chunk.Ref.SourceID || scene.startUnitID != chunk.Ref.StartUnitID || scene.endUnitID != chunk.Ref.EndUnitID {
|
||||
return ChunkMatch{State: MatchMismatched}
|
||||
}
|
||||
return ChunkMatch{State: MatchExact, Kind: scene.kind}
|
||||
}
|
||||
|
||||
type projectedScene struct {
|
||||
ID string `json:"id"`
|
||||
SourceRef source.SourceRef `json:"source_ref"`
|
||||
Kind dnd.SceneKind `json:"kind"`
|
||||
}
|
||||
|
||||
type projectedSceneList struct {
|
||||
Scenes []projectedScene `json:"scenes"`
|
||||
}
|
||||
|
||||
func eligibilityProjection(scenes []projectedScene) ([]byte, error) {
|
||||
sort.Slice(scenes, func(i, j int) bool { return scenes[i].ID < scenes[j].ID })
|
||||
return json.Marshal(projectedSceneList{Scenes: scenes})
|
||||
}
|
||||
|
||||
func semanticDigest(content []byte) string {
|
||||
sum := sha256.Sum256(content)
|
||||
return "sha256:" + hex.EncodeToString(sum[:])
|
||||
}
|
||||
270
internal/modules/dnd/scenedescriptions/registry/registry_test.go
Normal file
270
internal/modules/dnd/scenedescriptions/registry/registry_test.go
Normal file
@@ -0,0 +1,270 @@
|
||||
package registry
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
scenecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
|
||||
)
|
||||
|
||||
func TestResolveClassifiesExactMissingAndMismatchedChunks(t *testing.T) {
|
||||
registry := resolveList(t, sceneList(
|
||||
scene("chunk-combat", "session-alpha", 1, 2, dnd.SceneKindCombat, "Combat title", "Combat summary"),
|
||||
))
|
||||
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
chunk *source.Chunk
|
||||
want ChunkMatch
|
||||
}{
|
||||
{"exact", chunk("chunk-combat", "session-alpha", 1, 2), ChunkMatch{State: MatchExact, Kind: dnd.SceneKindCombat}},
|
||||
{"missing ID", chunk("chunk-missing", "session-alpha", 1, 2), ChunkMatch{State: MatchMissing}},
|
||||
{"source differs", chunk("chunk-combat", "other-source", 1, 2), ChunkMatch{State: MatchMismatched}},
|
||||
{"start differs", chunk("chunk-combat", "session-alpha", 2, 2), ChunkMatch{State: MatchMismatched}},
|
||||
{"end differs", chunk("chunk-combat", "session-alpha", 1, 3), ChunkMatch{State: MatchMismatched}},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if got := registry.Match(test.chunk); got != test.want {
|
||||
t.Fatalf("Match() = %#v, want %#v", got, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveRejectsInvalidSceneReferences(t *testing.T) {
|
||||
valid := encodeList(t, sceneList(scene("chunk-1", "session-alpha", 1, 2, dnd.SceneKindNarrative, "Arrival", "The party arrives.")))
|
||||
oversized := make([]byte, MaxBytes+1)
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
set contracts.ReferenceSet
|
||||
}{
|
||||
{"multiple items", referenceSet(referenceItem(valid), referenceItem(valid))},
|
||||
{"invalid media type", referenceSet(contracts.ReferenceItem{SlotName: ReferenceSlot, MediaType: "text/plain", Content: valid})},
|
||||
{"invalid media type syntax", referenceSet(contracts.ReferenceItem{SlotName: ReferenceSlot, MediaType: "not a media type", Content: valid})},
|
||||
{"oversized", referenceSet(contracts.ReferenceItem{SlotName: ReferenceSlot, MediaType: scenecodec.MediaType, Content: oversized})},
|
||||
{"invalid approved JSON", referenceSet(contracts.ReferenceItem{SlotName: ReferenceSlot, MediaType: scenecodec.MediaType, Content: []byte(`{"scenes":[]}`)})},
|
||||
{"duplicate IDs", referenceSet(referenceItem(encodeList(t, sceneList(
|
||||
scene("chunk-1", "session-alpha", 1, 1, dnd.SceneKindNarrative, "First", "First scene."),
|
||||
scene("chunk-1", "session-alpha", 2, 2, dnd.SceneKindCombat, "Second", "Second scene."),
|
||||
))))},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if _, err := Resolve(test.set); err == nil {
|
||||
t.Fatal("Resolve() error = nil")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestEligibilityDigestTracksOnlyGatingFields(t *testing.T) {
|
||||
base := sceneList(
|
||||
scene("chunk-a", "session-alpha", 1, 2, dnd.SceneKindNarrative, "Arrival", "The party arrives."),
|
||||
scene("chunk-b", "session-alpha", 3, 4, dnd.SceneKindCombat, "Combat", "Combat begins."),
|
||||
)
|
||||
baseDigest := resolveList(t, base).EligibilityDigest()
|
||||
|
||||
equivalent := sceneList(
|
||||
scene("chunk-b", "session-alpha", 3, 4, dnd.SceneKindCombat, "Different title", "Different summary."),
|
||||
scene("chunk-a", "session-alpha", 1, 2, dnd.SceneKindNarrative, "Another title", "Another summary."),
|
||||
)
|
||||
if got := resolveList(t, equivalent).EligibilityDigest(); got != baseDigest {
|
||||
t.Fatalf("equivalent eligibility digest = %q, want %q", got, baseDigest)
|
||||
}
|
||||
|
||||
changes := []struct {
|
||||
name string
|
||||
change func(*dnd.SceneDescription)
|
||||
}{
|
||||
{"ID", func(value *dnd.SceneDescription) { value.ID = "chunk-other" }},
|
||||
{"source ID", func(value *dnd.SceneDescription) { value.SourceRef.SourceID = "session-other" }},
|
||||
{"start unit ID", func(value *dnd.SceneDescription) { value.SourceRef.StartUnitID = 2 }},
|
||||
{"end unit ID", func(value *dnd.SceneDescription) { value.SourceRef.EndUnitID = 3 }},
|
||||
{"kind", func(value *dnd.SceneDescription) { value.Kind = dnd.SceneKindRecap }},
|
||||
}
|
||||
for _, test := range changes {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
changed := cloneList(base)
|
||||
test.change(&changed.Scenes[0])
|
||||
if got := resolveList(t, changed).EligibilityDigest(); got == baseDigest {
|
||||
t.Fatal("eligibility digest did not change")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolverUsesGeneratedReferenceWhenPresent(t *testing.T) {
|
||||
external := sceneList(scene("chunk-1", "session-alpha", 1, 2, dnd.SceneKindNarrative, "External", "External scene."))
|
||||
generated := sceneList(scene("chunk-1", "session-alpha", 1, 2, dnd.SceneKindCombat, "Generated", "Generated scene."))
|
||||
resolver, err := NewResolver(referenceSet(referenceItem(encodeList(t, external))))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if seeded := resolver.Seeded(); !seeded.Bound() || seeded.Count() != 1 || seeded.Match(chunk("chunk-1", "session-alpha", 1, 2)).Kind != dnd.SceneKindNarrative {
|
||||
t.Fatalf("Seeded() = %#v", seeded)
|
||||
}
|
||||
if got, err := resolver.Resolve(contracts.ReferenceSet{}); err != nil || got != resolver.Seeded() {
|
||||
t.Fatalf("Resolve() = %p, %v; want seeded %p", got, err, resolver.Seeded())
|
||||
}
|
||||
resolved, err := resolver.Resolve(referenceSet(referenceItem(encodeList(t, generated))))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := resolved.Match(chunk("chunk-1", "session-alpha", 1, 2)); got != (ChunkMatch{State: MatchExact, Kind: dnd.SceneKindCombat}) {
|
||||
t.Fatalf("generated Match() = %#v", got)
|
||||
}
|
||||
if resolved == resolver.Seeded() {
|
||||
t.Fatal("generated reference reused a different seeded eligibility view")
|
||||
}
|
||||
|
||||
unbound, err := NewResolver(contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{ReferenceSlot: {}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if seeded := unbound.Seeded(); seeded.Bound() || seeded.Count() != 0 || seeded.EligibilityDigest() != semanticDigest([]byte(emptyProjection)) {
|
||||
t.Fatalf("unbound Seeded() = %#v", seeded)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolverReusesRawAndSemanticallyEquivalentViews(t *testing.T) {
|
||||
baseContent := encodeList(t, sceneList(
|
||||
scene("chunk-1", "session-alpha", 1, 2, dnd.SceneKindCombat, "Original title", "Original summary."),
|
||||
))
|
||||
baseItem := referenceItem(baseContent)
|
||||
baseItem.Digest = semanticDigest(baseContent)
|
||||
resolver, err := NewResolver(contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{ReferenceSlot: {}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
first, err := resolver.Resolve(referenceSet(baseItem))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
second, err := resolver.Resolve(referenceSet(baseItem))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if second != first {
|
||||
t.Fatal("identical framework reference did not reuse its immutable view")
|
||||
}
|
||||
|
||||
equivalentContent := encodeList(t, sceneList(
|
||||
scene("chunk-1", "session-alpha", 1, 2, dnd.SceneKindCombat, "Revised title", "Revised summary."),
|
||||
))
|
||||
equivalentItem := referenceItem(equivalentContent)
|
||||
equivalentItem.Digest = semanticDigest(equivalentContent)
|
||||
equivalent, err := resolver.Resolve(referenceSet(equivalentItem))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if equivalent != first {
|
||||
t.Fatal("prose-only change did not reuse the semantic eligibility view")
|
||||
}
|
||||
|
||||
for _, digest := range []string{"", "not-a-sha256-digest"} {
|
||||
fallbackItem := referenceItem(baseContent)
|
||||
fallbackItem.Digest = digest
|
||||
fallback, resolveErr := resolver.Resolve(referenceSet(fallbackItem))
|
||||
if resolveErr != nil {
|
||||
t.Fatal(resolveErr)
|
||||
}
|
||||
if fallback != first {
|
||||
t.Fatalf("digest fallback %q did not reuse the content-addressed view", digest)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRegistryIsImmutableAndResolverIsSafeForConcurrentReuse(t *testing.T) {
|
||||
content := encodeList(t, sceneList(scene("chunk-1", "session-alpha", 1, 2, dnd.SceneKindCombat, "Title", "Summary.")))
|
||||
references := referenceSet(referenceItem(content))
|
||||
registry := resolveReferences(t, references)
|
||||
content[0] = '['
|
||||
if got := registry.Match(chunk("chunk-1", "session-alpha", 1, 2)); got != (ChunkMatch{State: MatchExact, Kind: dnd.SceneKindCombat}) {
|
||||
t.Fatalf("Match() after input mutation = %#v", got)
|
||||
}
|
||||
|
||||
resolver, err := NewResolver(contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{ReferenceSlot: {}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
const workers = 32
|
||||
var group sync.WaitGroup
|
||||
errs := make(chan error, workers)
|
||||
for range workers {
|
||||
group.Add(1)
|
||||
go func() {
|
||||
defer group.Done()
|
||||
resolved, err := resolver.Resolve(references)
|
||||
if err != nil {
|
||||
errs <- err
|
||||
return
|
||||
}
|
||||
if got := resolved.Match(chunk("chunk-1", "session-alpha", 1, 2)); got != (ChunkMatch{State: MatchExact, Kind: dnd.SceneKindCombat}) {
|
||||
errs <- fmt.Errorf("Match() = %#v", got)
|
||||
}
|
||||
}()
|
||||
}
|
||||
group.Wait()
|
||||
close(errs)
|
||||
for err := range errs {
|
||||
t.Error(err)
|
||||
}
|
||||
}
|
||||
|
||||
func resolveList(t *testing.T, value dnd.SceneDescriptionList) *Registry {
|
||||
t.Helper()
|
||||
return resolveReferences(t, referenceSet(referenceItem(encodeList(t, value))))
|
||||
}
|
||||
|
||||
func resolveReferences(t *testing.T, references contracts.ReferenceSet) *Registry {
|
||||
t.Helper()
|
||||
registry, err := Resolve(references)
|
||||
if err != nil {
|
||||
t.Fatalf("Resolve() error = %v", err)
|
||||
}
|
||||
return registry
|
||||
}
|
||||
|
||||
func encodeList(t *testing.T, value dnd.SceneDescriptionList) []byte {
|
||||
t.Helper()
|
||||
content, err := scenecodec.New().Encode(value)
|
||||
if err != nil {
|
||||
t.Fatalf("Encode() error = %v", err)
|
||||
}
|
||||
return content
|
||||
}
|
||||
|
||||
func referenceSet(items ...contracts.ReferenceItem) contracts.ReferenceSet {
|
||||
return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{ReferenceSlot: {Items: items}}}
|
||||
}
|
||||
|
||||
func referenceItem(content []byte) contracts.ReferenceItem {
|
||||
return contracts.ReferenceItem{SlotName: ReferenceSlot, MediaType: scenecodec.MediaType, Content: append([]byte(nil), content...)}
|
||||
}
|
||||
|
||||
func scene(id string, sourceID string, startUnitID int, endUnitID int, kind dnd.SceneKind, title string, summary string) dnd.SceneDescription {
|
||||
return dnd.SceneDescription{
|
||||
ID: id,
|
||||
SourceRef: source.SourceRef{SourceID: sourceID, StartUnitID: startUnitID, EndUnitID: endUnitID},
|
||||
Kind: kind,
|
||||
Title: title,
|
||||
Summary: summary,
|
||||
}
|
||||
}
|
||||
|
||||
func sceneList(scenes ...dnd.SceneDescription) dnd.SceneDescriptionList {
|
||||
return dnd.SceneDescriptionList{Scenes: scenes}
|
||||
}
|
||||
|
||||
func chunk(id string, sourceID string, startUnitID int, endUnitID int) *source.Chunk {
|
||||
return &source.Chunk{ID: id, Ref: source.SourceRef{SourceID: sourceID, StartUnitID: startUnitID, EndUnitID: endUnitID}}
|
||||
}
|
||||
|
||||
func cloneList(value dnd.SceneDescriptionList) dnd.SceneDescriptionList {
|
||||
cloned := dnd.SceneDescriptionList{Scenes: append([]dnd.SceneDescription(nil), value.Scenes...)}
|
||||
return cloned
|
||||
}
|
||||
@@ -16,6 +16,8 @@ const NPCInteractionListKind contracts.ArtifactKind = "dnd/npc-interaction-list"
|
||||
|
||||
const SceneDescriptionListKind contracts.ArtifactKind = "dnd/scene-description-list"
|
||||
|
||||
const ItemEventListKind contracts.ArtifactKind = "dnd/item-event-list"
|
||||
|
||||
type SpellList struct {
|
||||
SpellCasts []SpellCast `json:"spell_casts"`
|
||||
}
|
||||
@@ -97,3 +99,26 @@ type SceneDescription struct {
|
||||
Title string `json:"title"`
|
||||
Summary string `json:"summary"`
|
||||
}
|
||||
|
||||
type ItemEventKind string
|
||||
|
||||
const (
|
||||
ItemEventKindDiscovered ItemEventKind = "discovered"
|
||||
ItemEventKindAcquired ItemEventKind = "acquired"
|
||||
ItemEventKindLost ItemEventKind = "lost"
|
||||
ItemEventKindConsumed ItemEventKind = "consumed"
|
||||
ItemEventKindTransferred ItemEventKind = "transferred"
|
||||
)
|
||||
|
||||
type ItemEventList struct {
|
||||
Events []ItemEvent `json:"events"`
|
||||
}
|
||||
|
||||
type ItemEvent struct {
|
||||
Name string `json:"name"`
|
||||
Kind ItemEventKind `json:"kind"`
|
||||
Quantity *int `json:"quantity,omitempty"`
|
||||
From string `json:"from,omitempty"`
|
||||
To string `json:"to,omitempty"`
|
||||
SourceRefs []source.SourceRef `json:"source_refs"`
|
||||
}
|
||||
|
||||
131
internal/modules/dnd/validate/itemevents/invariants/validator.go
Normal file
131
internal/modules/dnd/validate/itemevents/invariants/validator.go
Normal file
@@ -0,0 +1,131 @@
|
||||
// Package invariants validates normalized D&D item-event artifacts.
|
||||
package invariants
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
itemeventmodel "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/itemevents"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics"
|
||||
itemeventshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/shape"
|
||||
)
|
||||
|
||||
const (
|
||||
Key = "normalize/dnd/item-events/invariants"
|
||||
ReasonCode = "invalid_normalized_item_event_invariants"
|
||||
policy = "dnd.item_events.normalize_invariants.v1"
|
||||
)
|
||||
|
||||
type Options struct{}
|
||||
type Validator struct{}
|
||||
|
||||
var _ contracts.TypedValidator[dnd.ItemEventList] = (*Validator)(nil)
|
||||
var _ pipeline.CheckpointFingerprintProvider = (*Validator)(nil)
|
||||
|
||||
func New(Options) *Validator { return &Validator{} }
|
||||
func (v *Validator) Name() string { return Key }
|
||||
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
|
||||
return contracts.ExecutionClassDeterministic
|
||||
}
|
||||
func (v *Validator) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
|
||||
return []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}
|
||||
}
|
||||
|
||||
func (v *Validator) Validate(_ context.Context, req contracts.TypedValidationRequest[dnd.ItemEventList]) (contracts.ValidationResult, error) {
|
||||
if err := Validate(req.Source, req.Value); err != nil {
|
||||
return contracts.ValidationResult{Approved: false, ReasonCode: ReasonCode, Message: err.Error()}, nil
|
||||
}
|
||||
return contracts.ValidationResult{Approved: true}, nil
|
||||
}
|
||||
|
||||
// Validate checks only invariants introduced by item-event normalization.
|
||||
// Shape and source-reference failures remain owned by their earlier validators.
|
||||
func Validate(doc *source.SourceDocument, value dnd.ItemEventList) error {
|
||||
if itemeventshape.Validate(value) != nil {
|
||||
return nil
|
||||
}
|
||||
index := source.NewDocumentIndex(doc)
|
||||
if !sourceRefsValid(index, value) {
|
||||
return nil
|
||||
}
|
||||
issues := issuesFor(shared.NewSourceRefOrderFromIndex(index), value)
|
||||
if len(issues) == 0 {
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("%s", diagnostics.Aggregate("invalid normalized item event invariants", issues))
|
||||
}
|
||||
|
||||
func issuesFor(order shared.SourceRefOrder, value dnd.ItemEventList) []string {
|
||||
issues := make([]string, 0)
|
||||
seenIdentity := make(map[string]int)
|
||||
for eventIndex, event := range value.Events {
|
||||
prefix := fmt.Sprintf("events[%d]", eventIndex)
|
||||
if event.Name != itemeventmodel.DisplayValue(event.Name) {
|
||||
issues = append(issues, prefix+".name is not display-normalized")
|
||||
}
|
||||
if event.From != itemeventmodel.DisplayValue(event.From) {
|
||||
issues = append(issues, prefix+".from is not display-normalized")
|
||||
}
|
||||
if event.To != itemeventmodel.DisplayValue(event.To) {
|
||||
issues = append(issues, prefix+".to is not display-normalized")
|
||||
}
|
||||
for refIndex := 1; refIndex < len(event.SourceRefs); refIndex++ {
|
||||
previous := event.SourceRefs[refIndex-1]
|
||||
current := event.SourceRefs[refIndex]
|
||||
if order.Less(current, previous) {
|
||||
issues = append(issues, fmt.Sprintf("%s.source_refs are not in canonical order at index %d", prefix, refIndex))
|
||||
} else if current == previous {
|
||||
issues = append(issues, fmt.Sprintf("%s.source_refs[%d] duplicates the previous reference", prefix, refIndex))
|
||||
}
|
||||
}
|
||||
key := itemeventmodel.ExactIdentity(order, event)
|
||||
if previous, exists := seenIdentity[key]; exists {
|
||||
issues = append(issues, fmt.Sprintf("%s duplicates item event %d under normalized identity", prefix, previous))
|
||||
} else {
|
||||
seenIdentity[key] = eventIndex
|
||||
}
|
||||
}
|
||||
for eventIndex := 1; eventIndex < len(value.Events); eventIndex++ {
|
||||
if itemeventmodel.Less(order, value.Events[eventIndex], value.Events[eventIndex-1]) {
|
||||
issues = append(issues, fmt.Sprintf("events[%d] is out of canonical order", eventIndex))
|
||||
}
|
||||
}
|
||||
return issues
|
||||
}
|
||||
|
||||
func sourceRefsValid(index source.DocumentIndex, value dnd.ItemEventList) bool {
|
||||
for _, event := range value.Events {
|
||||
if !itemeventmodel.ValidSourceRefs(index, event.SourceRefs) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func Spec() pipeline.ValidatorSpec {
|
||||
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ValidatorRegistry) error {
|
||||
return pipeline.RegisterTypedValidatorBuilder(registry, dnd.ItemEventListKind, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.TypedValidator[dnd.ItemEventList], error) {
|
||||
options, err := DecodeOptions(request.Options)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return New(options), nil
|
||||
})
|
||||
}
|
||||
|
||||
func DecodeOptions(options map[string]any) (Options, error) {
|
||||
if err := pipeline.RejectUnknownOptions(options); err != nil {
|
||||
return Options{}, err
|
||||
}
|
||||
return Options{}, nil
|
||||
}
|
||||
|
||||
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }
|
||||
@@ -0,0 +1,129 @@
|
||||
package invariants
|
||||
|
||||
import (
|
||||
"context"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"unicode/utf8"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
normalizeitemevents "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/itemevents"
|
||||
)
|
||||
|
||||
func TestValidatorApprovesNormalizerOutput(t *testing.T) {
|
||||
doc := invariantDocument()
|
||||
input := dnd.ItemEventList{Events: []dnd.ItemEvent{
|
||||
{Name: " Coin ", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 20, EndUnitID: 20}}},
|
||||
{Name: "Arrow", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 10, EndUnitID: 10}}},
|
||||
}}
|
||||
normalized, err := normalizeitemevents.New(normalizeitemevents.Options{}).Normalize(context.Background(), contracts.TypedNormalizeRequest[dnd.ItemEventList]{Source: doc, MergeOutput: contracts.MergeArtifact[dnd.ItemEventList]{Value: input}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Source: doc, Value: normalized.Value})
|
||||
if err != nil || !result.Approved {
|
||||
t.Fatalf("Validate() = %#v, %v; want approval", result, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateRejectsOwnedNormalizedInvariants(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
mutate func(*dnd.ItemEventList)
|
||||
want string
|
||||
}{
|
||||
{name: "name whitespace", mutate: func(value *dnd.ItemEventList) { value.Events[0].Name = " Coin " }, want: ".name is not display-normalized"},
|
||||
{name: "from whitespace", mutate: func(value *dnd.ItemEventList) {
|
||||
value.Events[0].Kind = dnd.ItemEventKindLost
|
||||
value.Events[0].From = " Aria "
|
||||
}, want: ".from is not display-normalized"},
|
||||
{name: "to whitespace", mutate: func(value *dnd.ItemEventList) {
|
||||
value.Events[0].Kind = dnd.ItemEventKindAcquired
|
||||
value.Events[0].To = " Aria "
|
||||
}, want: ".to is not display-normalized"},
|
||||
{name: "reference order", mutate: func(value *dnd.ItemEventList) {
|
||||
value.Events[0].SourceRefs = []source.SourceRef{{SourceID: "session", StartUnitID: 20, EndUnitID: 20}, {SourceID: "session", StartUnitID: 10, EndUnitID: 10}}
|
||||
}, want: "not in canonical order"},
|
||||
{name: "duplicate reference", mutate: func(value *dnd.ItemEventList) {
|
||||
value.Events[0].SourceRefs = append(value.Events[0].SourceRefs, value.Events[0].SourceRefs[0])
|
||||
}, want: "duplicates the previous reference"},
|
||||
{name: "list order", mutate: func(value *dnd.ItemEventList) { value.Events = []dnd.ItemEvent{value.Events[1], value.Events[0]} }, want: "out of canonical order"},
|
||||
{name: "duplicate event", mutate: func(value *dnd.ItemEventList) { value.Events = append(value.Events, value.Events[0]) }, want: "duplicates item event"},
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
value := normalizedList()
|
||||
test.mutate(&value)
|
||||
err := Validate(invariantDocument(), value)
|
||||
if err == nil || !strings.Contains(err.Error(), test.want) {
|
||||
t.Fatalf("Validate() error = %v, want %q", err, test.want)
|
||||
}
|
||||
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Source: invariantDocument(), Value: value})
|
||||
if err != nil || result.Approved || result.ReasonCode != ReasonCode {
|
||||
t.Fatalf("validator result = %#v, %v", result, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateUsesSourceDocumentOrderAndDefersEarlierFailures(t *testing.T) {
|
||||
doc := &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 30}, {ID: 10}}}
|
||||
value := dnd.ItemEventList{Events: []dnd.ItemEvent{{
|
||||
Name: "Coin", Kind: dnd.ItemEventKindDiscovered,
|
||||
SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 10, EndUnitID: 10}, {SourceID: doc.ID, StartUnitID: 30, EndUnitID: 30}},
|
||||
}}}
|
||||
if err := Validate(doc, value); err == nil || !strings.Contains(err.Error(), "not in canonical order") {
|
||||
t.Fatalf("Validate() error = %v, want document-order rejection", err)
|
||||
}
|
||||
shapeInvalid := normalizedList()
|
||||
shapeInvalid.Events[0].Name = " "
|
||||
if err := Validate(invariantDocument(), shapeInvalid); err != nil {
|
||||
t.Fatalf("shape failure must be deferred, got %v", err)
|
||||
}
|
||||
sourceInvalid := normalizedList()
|
||||
sourceInvalid.Events[0].SourceRefs[0].StartUnitID = 999
|
||||
if err := Validate(invariantDocument(), sourceInvalid); err != nil {
|
||||
t.Fatalf("source-reference failure must be deferred, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidatorContractAndBoundedDiagnostics(t *testing.T) {
|
||||
value := normalizedList()
|
||||
value.Events = make([]dnd.ItemEvent, 30)
|
||||
for index := range value.Events {
|
||||
value.Events[index] = normalizedList().Events[0]
|
||||
value.Events[index].Name = strings.Repeat("火", 300)
|
||||
}
|
||||
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Source: invariantDocument(), Value: value})
|
||||
if err != nil || result.Approved || !utf8.ValidString(result.Message) || len([]byte(result.Message)) > 4096 || !strings.Contains(result.Message, "additional issue(s) omitted") {
|
||||
t.Fatalf("bounded result = %#v, %v", result, err)
|
||||
}
|
||||
if got := New(Options{}).CheckpointFingerprints(); !reflect.DeepEqual(got, []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}) {
|
||||
t.Fatalf("CheckpointFingerprints() = %#v", got)
|
||||
}
|
||||
if spec := Spec(); spec.Key != Key || spec.ExecutionClass != contracts.ExecutionClassDeterministic {
|
||||
t.Fatalf("Spec() = %#v", spec)
|
||||
}
|
||||
registry := pipeline.NewValidatorRegistry()
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatalf("Register() error = %v", err)
|
||||
}
|
||||
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
|
||||
t.Fatal("DecodeOptions() accepted unknown option")
|
||||
}
|
||||
}
|
||||
|
||||
func normalizedList() dnd.ItemEventList {
|
||||
return dnd.ItemEventList{Events: []dnd.ItemEvent{
|
||||
{Name: "Arrow", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 10, EndUnitID: 10}}},
|
||||
{Name: "Coin", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 20, EndUnitID: 20}}},
|
||||
}}
|
||||
}
|
||||
|
||||
func invariantDocument() *source.SourceDocument {
|
||||
return &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 10}, {ID: 20}}}
|
||||
}
|
||||
99
internal/modules/dnd/validate/itemevents/shape/validator.go
Normal file
99
internal/modules/dnd/validate/itemevents/shape/validator.go
Normal file
@@ -0,0 +1,99 @@
|
||||
// Package shape validates required D&D item-event candidate fields.
|
||||
package shape
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/itemevents"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics"
|
||||
)
|
||||
|
||||
const (
|
||||
Key = "extract/dnd/item-events/shape"
|
||||
ReasonCode = "invalid_item_event_shape"
|
||||
policy = "dnd.item_events.shape.v1"
|
||||
)
|
||||
|
||||
type Options struct{}
|
||||
type Validator struct{}
|
||||
|
||||
var _ contracts.TypedValidator[dnd.ItemEventList] = (*Validator)(nil)
|
||||
var _ pipeline.CheckpointFingerprintProvider = (*Validator)(nil)
|
||||
|
||||
func New(Options) *Validator { return &Validator{} }
|
||||
func (v *Validator) Name() string { return Key }
|
||||
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
|
||||
return contracts.ExecutionClassDeterministic
|
||||
}
|
||||
func (v *Validator) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
|
||||
return []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}
|
||||
}
|
||||
|
||||
func (v *Validator) Validate(_ context.Context, req contracts.TypedValidationRequest[dnd.ItemEventList]) (contracts.ValidationResult, error) {
|
||||
if err := Validate(req.Value); err != nil {
|
||||
return contracts.ValidationResult{Approved: false, ReasonCode: ReasonCode, Message: err.Error()}, nil
|
||||
}
|
||||
return contracts.ValidationResult{Approved: true}, nil
|
||||
}
|
||||
|
||||
// Validate returns one bounded error for every owned item-event shape issue.
|
||||
func Validate(value dnd.ItemEventList) error {
|
||||
issues := issuesFor(value)
|
||||
if len(issues) == 0 {
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("%s", diagnostics.Aggregate("invalid item event shape", issues))
|
||||
}
|
||||
|
||||
func issuesFor(value dnd.ItemEventList) []string {
|
||||
if value.Events == nil {
|
||||
return []string{"events must be present"}
|
||||
}
|
||||
issues := make([]string, 0)
|
||||
for index, event := range value.Events {
|
||||
prefix := fmt.Sprintf("events[%d]", index)
|
||||
if strings.TrimSpace(event.Name) == "" {
|
||||
issues = append(issues, prefix+".name must not be empty: "+diagnostics.Quote(event.Name))
|
||||
}
|
||||
if !itemevents.SupportedKind(event.Kind) {
|
||||
issues = append(issues, prefix+".kind is unsupported: "+diagnostics.Quote(string(event.Kind)))
|
||||
} else if !itemevents.ValidHolderCombination(event.Kind, event.From, event.To) {
|
||||
issues = append(issues, prefix+".from and .to are incompatible with "+diagnostics.Quote(string(event.Kind)))
|
||||
}
|
||||
if event.Quantity != nil && *event.Quantity < 1 {
|
||||
issues = append(issues, prefix+".quantity must be positive when present")
|
||||
}
|
||||
if len(event.SourceRefs) == 0 {
|
||||
issues = append(issues, prefix+".source_refs must contain at least one reference")
|
||||
}
|
||||
}
|
||||
return issues
|
||||
}
|
||||
|
||||
func Spec() pipeline.ValidatorSpec {
|
||||
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ValidatorRegistry) error {
|
||||
return pipeline.RegisterTypedValidatorBuilder(registry, dnd.ItemEventListKind, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.TypedValidator[dnd.ItemEventList], error) {
|
||||
options, err := DecodeOptions(request.Options)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return New(options), nil
|
||||
})
|
||||
}
|
||||
|
||||
func DecodeOptions(options map[string]any) (Options, error) {
|
||||
if err := pipeline.RejectUnknownOptions(options); err != nil {
|
||||
return Options{}, err
|
||||
}
|
||||
return Options{}, nil
|
||||
}
|
||||
|
||||
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }
|
||||
109
internal/modules/dnd/validate/itemevents/shape/validator_test.go
Normal file
109
internal/modules/dnd/validate/itemevents/shape/validator_test.go
Normal file
@@ -0,0 +1,109 @@
|
||||
package shape
|
||||
|
||||
import (
|
||||
"context"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"unicode/utf8"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
)
|
||||
|
||||
func TestValidatorAcceptsEveryEventRuleAndNormalizableWhitespace(t *testing.T) {
|
||||
quantity := 1
|
||||
value := dnd.ItemEventList{Events: []dnd.ItemEvent{
|
||||
{Name: " Hidden Cache ", Kind: dnd.ItemEventKindDiscovered, From: " ", To: " ", SourceRefs: refs(1, 1)},
|
||||
{Name: "Gold Pieces", Kind: dnd.ItemEventKindAcquired, Quantity: &quantity, To: " party ", SourceRefs: refs(2, 2)},
|
||||
{Name: "Torch", Kind: dnd.ItemEventKindLost, From: " party ", SourceRefs: refs(3, 3)},
|
||||
{Name: "Potion", Kind: dnd.ItemEventKindConsumed, From: " Aria ", SourceRefs: refs(4, 4)},
|
||||
{Name: "Moonblade", Kind: dnd.ItemEventKindTransferred, From: "Aria", To: "Borin", SourceRefs: refs(5, 5)},
|
||||
}}
|
||||
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Value: value})
|
||||
if err != nil || !result.Approved {
|
||||
t.Fatalf("Validate() = %#v, %v", result, err)
|
||||
}
|
||||
empty, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Value: dnd.ItemEventList{Events: []dnd.ItemEvent{}}})
|
||||
if err != nil || !empty.Approved {
|
||||
t.Fatalf("empty list result = %#v, %v", empty, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidatorRejectsOwnedSemanticBoundaries(t *testing.T) {
|
||||
valid := validList()
|
||||
zero := 0
|
||||
negative := -1
|
||||
tests := []struct {
|
||||
name string
|
||||
value dnd.ItemEventList
|
||||
want string
|
||||
}{
|
||||
{"missing events", dnd.ItemEventList{}, "events must be present"},
|
||||
{"blank name", listWith(dnd.ItemEvent{Name: " \t", Kind: dnd.ItemEventKindDiscovered, SourceRefs: refs(1, 1)}), "name must not be empty"},
|
||||
{"unsupported kind", listWith(dnd.ItemEvent{Name: "Ring", Kind: "unknown", SourceRefs: refs(1, 1)}), "kind is unsupported"},
|
||||
{"discovered holder", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindDiscovered, From: "Aria", SourceRefs: refs(1, 1)}), "incompatible"},
|
||||
{"acquired without holder", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindAcquired, SourceRefs: refs(1, 1)}), "incompatible"},
|
||||
{"lost destination", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindLost, From: "Aria", To: "Borin", SourceRefs: refs(1, 1)}), "incompatible"},
|
||||
{"consumed without holder", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindConsumed, SourceRefs: refs(1, 1)}), "incompatible"},
|
||||
{"party transfer from", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindTransferred, From: "party", To: "Borin", SourceRefs: refs(1, 1)}), "incompatible"},
|
||||
{"party transfer to", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindTransferred, From: "Aria", To: " PARTY ", SourceRefs: refs(1, 1)}), "incompatible"},
|
||||
{"case equivalent transfer", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindTransferred, From: "Aria", To: "aria", SourceRefs: refs(1, 1)}), "incompatible"},
|
||||
{"unicode equivalent transfer", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindTransferred, From: "Åria", To: "Åria", SourceRefs: refs(1, 1)}), "incompatible"},
|
||||
{"zero quantity", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindAcquired, Quantity: &zero, To: "party", SourceRefs: refs(1, 1)}), "quantity must be positive"},
|
||||
{"negative quantity", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindAcquired, Quantity: &negative, To: "party", SourceRefs: refs(1, 1)}), "quantity must be positive"},
|
||||
{"missing source refs", listWith(dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindDiscovered}), "source_refs must contain"},
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Value: test.value})
|
||||
if err != nil || result.Approved || result.ReasonCode != ReasonCode || !strings.Contains(result.Message, test.want) {
|
||||
t.Fatalf("Validate() = %#v, %v; want %q", result, err, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
if result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Value: valid}); err != nil || !result.Approved {
|
||||
t.Fatalf("valid result = %#v, %v", result, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidatorAggregatesBoundedIndexedDiagnosticsAndRegistration(t *testing.T) {
|
||||
value := dnd.ItemEventList{Events: make([]dnd.ItemEvent, 24)}
|
||||
for index := range value.Events {
|
||||
value.Events[index] = dnd.ItemEvent{Name: " \n", Kind: "unsupported"}
|
||||
}
|
||||
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Value: value})
|
||||
if err != nil || result.Approved || result.ReasonCode != ReasonCode || len([]byte(result.Message)) > 4096 || !utf8.ValidString(result.Message) || !strings.Contains(result.Message, "events[0]") || !strings.Contains(result.Message, "additional issue(s) omitted") {
|
||||
t.Fatalf("Validate() = %#v, %v", result, err)
|
||||
}
|
||||
if value.Events[0].Name != " \n" {
|
||||
t.Fatal("Validate() mutated input")
|
||||
}
|
||||
if got := New(Options{}).CheckpointFingerprints(); !reflect.DeepEqual(got, []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}) {
|
||||
t.Fatalf("CheckpointFingerprints() = %#v", got)
|
||||
}
|
||||
registry := pipeline.NewValidatorRegistry()
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got, ok := registry.Spec(Key); !ok || !reflect.DeepEqual(got, Spec()) {
|
||||
t.Fatalf("registry spec = %#v, %t", got, ok)
|
||||
}
|
||||
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
|
||||
t.Fatal("DecodeOptions() accepted unknown option")
|
||||
}
|
||||
}
|
||||
|
||||
func validList() dnd.ItemEventList {
|
||||
return dnd.ItemEventList{Events: []dnd.ItemEvent{{Name: "Ring", Kind: dnd.ItemEventKindAcquired, To: "party", SourceRefs: refs(1, 1)}}}
|
||||
}
|
||||
|
||||
func listWith(event dnd.ItemEvent) dnd.ItemEventList {
|
||||
return dnd.ItemEventList{Events: []dnd.ItemEvent{event}}
|
||||
}
|
||||
|
||||
func refs(start, end int) []source.SourceRef {
|
||||
return []source.SourceRef{{SourceID: "session", StartUnitID: start, EndUnitID: end}}
|
||||
}
|
||||
@@ -0,0 +1,121 @@
|
||||
// Package sourcerefs validates D&D item-event transcript evidence.
|
||||
package sourcerefs
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics"
|
||||
itemeventshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/shape"
|
||||
)
|
||||
|
||||
const (
|
||||
Key = "extract/dnd/item-events/source_refs"
|
||||
ReasonCode = "invalid_item_event_source_references"
|
||||
policy = "dnd.item_events.source_refs.v1"
|
||||
)
|
||||
|
||||
type Options struct{}
|
||||
type Validator struct{}
|
||||
|
||||
var _ contracts.TypedValidator[dnd.ItemEventList] = (*Validator)(nil)
|
||||
var _ pipeline.CheckpointFingerprintProvider = (*Validator)(nil)
|
||||
|
||||
func New(Options) *Validator { return &Validator{} }
|
||||
func (v *Validator) Name() string { return Key }
|
||||
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
|
||||
return contracts.ExecutionClassDeterministic
|
||||
}
|
||||
func (v *Validator) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
|
||||
return []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}
|
||||
}
|
||||
|
||||
func (v *Validator) Validate(_ context.Context, req contracts.TypedValidationRequest[dnd.ItemEventList]) (contracts.ValidationResult, error) {
|
||||
if req.Stage == string(pipeline.StageExtract) && req.Chunk == nil {
|
||||
return contracts.ValidationResult{}, fmt.Errorf("item event source-reference validator requires the current extraction chunk")
|
||||
}
|
||||
if itemeventshape.Validate(req.Value) != nil {
|
||||
return contracts.ValidationResult{Approved: true}, nil
|
||||
}
|
||||
index := source.NewDocumentIndex(req.Source)
|
||||
var coverage *chunkCoverage
|
||||
if req.Stage == string(pipeline.StageExtract) {
|
||||
coverage = newChunkCoverage(req.Chunk)
|
||||
}
|
||||
issues := make([]string, 0)
|
||||
for eventIndex, event := range req.Value.Events {
|
||||
for refIndex, ref := range event.SourceRefs {
|
||||
if err := index.ValidateRef(ref); err != nil {
|
||||
issues = append(issues, fmt.Sprintf("events[%d].source_refs[%d]: %s", eventIndex, refIndex, diagnostics.Truncate(err.Error())))
|
||||
continue
|
||||
}
|
||||
if coverage != nil && !coverage.contains(req.Source, ref) {
|
||||
issues = append(issues, fmt.Sprintf("events[%d].source_refs[%d]: source reference is outside the current extraction chunk", eventIndex, refIndex))
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(issues) == 0 {
|
||||
return contracts.ValidationResult{Approved: true}, nil
|
||||
}
|
||||
return contracts.ValidationResult{Approved: false, ReasonCode: ReasonCode, Message: diagnostics.Aggregate("invalid item event source references", issues)}, nil
|
||||
}
|
||||
|
||||
type chunkCoverage struct {
|
||||
sourceID string
|
||||
unitIDs map[int]struct{}
|
||||
}
|
||||
|
||||
func newChunkCoverage(chunk *source.Chunk) *chunkCoverage {
|
||||
coverage := &chunkCoverage{
|
||||
sourceID: chunk.SourceID,
|
||||
unitIDs: make(map[int]struct{}, len(chunk.Units)),
|
||||
}
|
||||
for _, unit := range chunk.Units {
|
||||
coverage.unitIDs[unit.ID] = struct{}{}
|
||||
}
|
||||
return coverage
|
||||
}
|
||||
|
||||
func (coverage *chunkCoverage) contains(doc *source.SourceDocument, ref source.SourceRef) bool {
|
||||
if coverage == nil || doc == nil || ref.SourceID != coverage.sourceID {
|
||||
return false
|
||||
}
|
||||
start, startOK := source.UnitIndex(doc, ref.StartUnitID)
|
||||
end, endOK := source.UnitIndex(doc, ref.EndUnitID)
|
||||
if !startOK || !endOK || start > end {
|
||||
return false
|
||||
}
|
||||
for position := start; position <= end; position++ {
|
||||
if _, found := coverage.unitIDs[doc.Units[position].ID]; !found {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func Spec() pipeline.ValidatorSpec {
|
||||
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ValidatorRegistry) error {
|
||||
return pipeline.RegisterTypedValidatorBuilder(registry, dnd.ItemEventListKind, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.TypedValidator[dnd.ItemEventList], error) {
|
||||
options, err := DecodeOptions(request.Options)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return New(options), nil
|
||||
})
|
||||
}
|
||||
|
||||
func DecodeOptions(options map[string]any) (Options, error) {
|
||||
if err := pipeline.RejectUnknownOptions(options); err != nil {
|
||||
return Options{}, err
|
||||
}
|
||||
return Options{}, nil
|
||||
}
|
||||
|
||||
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }
|
||||
@@ -0,0 +1,121 @@
|
||||
package sourcerefs
|
||||
|
||||
import (
|
||||
"context"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"unicode/utf8"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
)
|
||||
|
||||
func TestValidatorOwnsSourceAndRangeValidation(t *testing.T) {
|
||||
value := validList()
|
||||
result, err := New(Options{}).Validate(context.Background(), request(document(), value))
|
||||
if err != nil || !result.Approved {
|
||||
t.Fatalf("valid result = %#v, %v", result, err)
|
||||
}
|
||||
value.Events[0].SourceRefs = []source.SourceRef{
|
||||
{SourceID: "foreign", StartUnitID: 1, EndUnitID: 1},
|
||||
{SourceID: "session", StartUnitID: 99, EndUnitID: 99},
|
||||
{SourceID: "session", StartUnitID: 2, EndUnitID: 1},
|
||||
}
|
||||
result, err = New(Options{}).Validate(context.Background(), request(document(), value))
|
||||
if err != nil || result.Approved || result.ReasonCode != ReasonCode {
|
||||
t.Fatalf("invalid result = %#v, %v", result, err)
|
||||
}
|
||||
for _, index := range []string{"events[0].source_refs[0]", "events[0].source_refs[1]", "events[0].source_refs[2]"} {
|
||||
if !strings.Contains(result.Message, index) {
|
||||
t.Fatalf("validation message = %q, missing %q", result.Message, index)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidatorEnforcesChunkOnlyDuringExtraction(t *testing.T) {
|
||||
doc := document()
|
||||
chunk := &source.Chunk{ID: "chunk-0", SourceID: doc.ID, Units: append([]source.SourceUnit(nil), doc.Units[:2]...)}
|
||||
contained := validList()
|
||||
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Stage: string(pipeline.StageExtract), Source: doc, Chunk: chunk, Value: contained})
|
||||
if err != nil || !result.Approved {
|
||||
t.Fatalf("contained evidence = %#v, %v", result, err)
|
||||
}
|
||||
|
||||
crossesChunk := validList()
|
||||
crossesChunk.Events[0].SourceRefs = []source.SourceRef{{SourceID: doc.ID, StartUnitID: 2, EndUnitID: 3}}
|
||||
result, err = New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Stage: string(pipeline.StageExtract), Source: doc, Chunk: chunk, Value: crossesChunk})
|
||||
if err != nil || result.Approved || !strings.Contains(result.Message, "outside the current extraction chunk") {
|
||||
t.Fatalf("out-of-chunk evidence = %#v, %v", result, err)
|
||||
}
|
||||
|
||||
noncontiguousChunk := &source.Chunk{ID: "chunk-1", SourceID: doc.ID, Units: []source.SourceUnit{doc.Units[0], doc.Units[2]}}
|
||||
spansMissingUnit := validList()
|
||||
spansMissingUnit.Events[0].SourceRefs = []source.SourceRef{{SourceID: doc.ID, StartUnitID: 1, EndUnitID: 3}}
|
||||
result, err = New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Stage: string(pipeline.StageExtract), Source: doc, Chunk: noncontiguousChunk, Value: spansMissingUnit})
|
||||
if err != nil || result.Approved || !strings.Contains(result.Message, "outside the current extraction chunk") {
|
||||
t.Fatalf("partially contained evidence = %#v, %v", result, err)
|
||||
}
|
||||
|
||||
multiRange := validList()
|
||||
multiRange.Events[0].SourceRefs = []source.SourceRef{{SourceID: doc.ID, StartUnitID: 1, EndUnitID: 2}, {SourceID: doc.ID, StartUnitID: 3, EndUnitID: 4}}
|
||||
result, err = New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Source: doc, Value: multiRange})
|
||||
if err != nil || !result.Approved {
|
||||
t.Fatalf("post-merge evidence = %#v, %v", result, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidatorRequiresChunkDuringExtractionAndDefersShape(t *testing.T) {
|
||||
_, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Stage: string(pipeline.StageExtract), Source: document(), Value: validList()})
|
||||
if err == nil || !strings.Contains(err.Error(), "requires the current extraction chunk") {
|
||||
t.Fatalf("Validate() error = %v", err)
|
||||
}
|
||||
malformed := dnd.ItemEventList{Events: []dnd.ItemEvent{{Name: "Ring"}}}
|
||||
result, err := New(Options{}).Validate(context.Background(), request(document(), malformed))
|
||||
if err != nil || !result.Approved {
|
||||
t.Fatalf("shape deferral = %#v, %v", result, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidatorBoundsDiagnosticsAndRegistration(t *testing.T) {
|
||||
value := dnd.ItemEventList{Events: make([]dnd.ItemEvent, 24)}
|
||||
for index := range value.Events {
|
||||
value.Events[index] = dnd.ItemEvent{Name: "Ring", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "foreign", StartUnitID: index + 1, EndUnitID: index + 1}}}
|
||||
}
|
||||
result, err := New(Options{}).Validate(context.Background(), request(document(), value))
|
||||
if err != nil || result.Approved || result.ReasonCode != ReasonCode || len([]byte(result.Message)) > 4096 || !utf8.ValidString(result.Message) || !strings.Contains(result.Message, "events[0].source_refs[0]") || !strings.Contains(result.Message, "additional issue(s) omitted") {
|
||||
t.Fatalf("Validate() = %#v, %v", result, err)
|
||||
}
|
||||
before := validList()
|
||||
copy := before
|
||||
if _, err := New(Options{}).Validate(context.Background(), request(document(), before)); err != nil || !reflect.DeepEqual(before, copy) {
|
||||
t.Fatal("Validate() mutated input")
|
||||
}
|
||||
if got := New(Options{}).CheckpointFingerprints(); !reflect.DeepEqual(got, []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}) {
|
||||
t.Fatalf("CheckpointFingerprints() = %#v", got)
|
||||
}
|
||||
registry := pipeline.NewValidatorRegistry()
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got, ok := registry.Spec(Key); !ok || !reflect.DeepEqual(got, Spec()) {
|
||||
t.Fatalf("registry spec = %#v, %t", got, ok)
|
||||
}
|
||||
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
|
||||
t.Fatal("DecodeOptions() accepted unknown option")
|
||||
}
|
||||
}
|
||||
|
||||
func request(doc *source.SourceDocument, value dnd.ItemEventList) contracts.TypedValidationRequest[dnd.ItemEventList] {
|
||||
return contracts.TypedValidationRequest[dnd.ItemEventList]{Source: doc, Value: value}
|
||||
}
|
||||
|
||||
func document() *source.SourceDocument {
|
||||
return &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 1}, {ID: 2}, {ID: 3}, {ID: 4}}}
|
||||
}
|
||||
|
||||
func validList() dnd.ItemEventList {
|
||||
return dnd.ItemEventList{Events: []dnd.ItemEvent{{Name: "Ring", Kind: dnd.ItemEventKindAcquired, To: "party", SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 1, EndUnitID: 2}}}}}
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
// Package sourcerelatedness warns when item-event evidence does not mention its item.
|
||||
package sourcerelatedness
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics"
|
||||
itemeventshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/shape"
|
||||
)
|
||||
|
||||
const (
|
||||
Key = "extract/dnd/item-events/source_relatedness"
|
||||
WarningReasonCode = "item_event_source_unrelated"
|
||||
OmittedReasonCode = "item_event_relatedness_warnings_omitted"
|
||||
policy = "dnd.item_events.source_relatedness.v1"
|
||||
)
|
||||
|
||||
type Options struct{}
|
||||
type Validator struct{}
|
||||
|
||||
var _ contracts.TypedValidator[dnd.ItemEventList] = (*Validator)(nil)
|
||||
var _ pipeline.CheckpointFingerprintProvider = (*Validator)(nil)
|
||||
|
||||
func New(Options) *Validator { return &Validator{} }
|
||||
func (v *Validator) Name() string { return Key }
|
||||
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
|
||||
return contracts.ExecutionClassDeterministic
|
||||
}
|
||||
func (v *Validator) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
|
||||
return []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}
|
||||
}
|
||||
|
||||
// Validate performs advisory source grounding only. Shape and range failures
|
||||
// are intentionally left to their blocking owners.
|
||||
func (v *Validator) Validate(_ context.Context, req contracts.TypedValidationRequest[dnd.ItemEventList]) (contracts.ValidationResult, error) {
|
||||
if itemeventshape.Validate(req.Value) != nil {
|
||||
return contracts.ValidationResult{Approved: true}, nil
|
||||
}
|
||||
resolver, err := shared.NewCitationResolver(req.Source)
|
||||
if err != nil {
|
||||
return contracts.ValidationResult{Approved: true}, nil
|
||||
}
|
||||
|
||||
warnings := make([]contracts.Warning, 0)
|
||||
for index, event := range req.Value.Events {
|
||||
citedText, err := resolver.CitedText(event.SourceRefs)
|
||||
if err != nil || shared.ContainsTokenSequence(citedText, event.Name) {
|
||||
continue
|
||||
}
|
||||
warnings = append(warnings, contracts.Warning{
|
||||
Scope: fmt.Sprintf("events[%d]", index),
|
||||
ReasonCode: WarningReasonCode,
|
||||
Message: fmt.Sprintf("item event name %s was not found in cited source text", diagnostics.Quote(event.Name)),
|
||||
})
|
||||
}
|
||||
return contracts.ValidationResult{
|
||||
Approved: true,
|
||||
Warnings: diagnostics.LimitWarnings(warnings, "item_events", OmittedReasonCode),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func Spec() pipeline.ValidatorSpec {
|
||||
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ValidatorRegistry) error {
|
||||
return pipeline.RegisterTypedValidatorBuilder(registry, dnd.ItemEventListKind, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.TypedValidator[dnd.ItemEventList], error) {
|
||||
options, err := DecodeOptions(request.Options)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return New(options), nil
|
||||
})
|
||||
}
|
||||
|
||||
func DecodeOptions(options map[string]any) (Options, error) {
|
||||
if err := pipeline.RejectUnknownOptions(options); err != nil {
|
||||
return Options{}, err
|
||||
}
|
||||
return Options{}, nil
|
||||
}
|
||||
|
||||
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }
|
||||
@@ -0,0 +1,74 @@
|
||||
package sourcerelatedness
|
||||
|
||||
import (
|
||||
"context"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics"
|
||||
)
|
||||
|
||||
func TestValidatorMatchesTokenSequencesAcrossRanges(t *testing.T) {
|
||||
doc := &source.SourceDocument{ID: "session", Units: []source.SourceUnit{
|
||||
{ID: 1, Text: "Aria finds a potion,"},
|
||||
{ID: 2, Text: "of healing in the chest."},
|
||||
{ID: 3, Text: "The group rests."},
|
||||
}}
|
||||
value := dnd.ItemEventList{Events: []dnd.ItemEvent{
|
||||
{Name: "POTION OF HEALING", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 1, EndUnitID: 1}, {SourceID: doc.ID, StartUnitID: 2, EndUnitID: 2}}},
|
||||
{Name: "missing-item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 3, EndUnitID: 3}}},
|
||||
}}
|
||||
references := contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
|
||||
"glossary": {Items: []contracts.ReferenceItem{{Content: []byte("missing-item")}}},
|
||||
}}
|
||||
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Source: doc, References: references, Value: value})
|
||||
if err != nil || !result.Approved || len(result.Warnings) != 1 {
|
||||
t.Fatalf("Validate() = %#v, %v", result, err)
|
||||
}
|
||||
if warning := result.Warnings[0]; warning.Scope != "events[1]" || warning.ReasonCode != WarningReasonCode || !strings.Contains(warning.Message, "missing-item") {
|
||||
t.Fatalf("warning = %#v", warning)
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidatorDefersMalformedAndUnreadableEvidence(t *testing.T) {
|
||||
doc := &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 1, Text: "a potion"}}}
|
||||
for _, value := range []dnd.ItemEventList{
|
||||
{Events: []dnd.ItemEvent{{Name: "", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 1, EndUnitID: 1}}}}},
|
||||
{Events: []dnd.ItemEvent{{Name: "potion", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: doc.ID, StartUnitID: 99, EndUnitID: 99}}}}},
|
||||
} {
|
||||
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{Source: doc, Value: value})
|
||||
if err != nil || !result.Approved || len(result.Warnings) != 0 {
|
||||
t.Fatalf("deferred result = %#v, %v", result, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidatorBoundsWarningsAndRegistersPolicy(t *testing.T) {
|
||||
count := diagnostics.MaxWarnings + 5
|
||||
events := make([]dnd.ItemEvent, count)
|
||||
for index := range events {
|
||||
events[index] = dnd.ItemEvent{Name: "missing item", Kind: dnd.ItemEventKindDiscovered, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 1, EndUnitID: 1}}}
|
||||
}
|
||||
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.ItemEventList]{
|
||||
Source: &source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 1, Text: "nothing useful"}}},
|
||||
Value: dnd.ItemEventList{Events: events},
|
||||
})
|
||||
if err != nil || !result.Approved || len(result.Warnings) != diagnostics.MaxWarnings || result.Warnings[len(result.Warnings)-1].ReasonCode != OmittedReasonCode {
|
||||
t.Fatalf("Validate() = %#v, %v", result, err)
|
||||
}
|
||||
if got := New(Options{}).CheckpointFingerprints(); !reflect.DeepEqual(got, []pipeline.CheckpointFingerprint{{Name: "policy", Value: policy}}) {
|
||||
t.Fatalf("CheckpointFingerprints() = %#v", got)
|
||||
}
|
||||
registry := pipeline.NewValidatorRegistry()
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatalf("Register() error = %v", err)
|
||||
}
|
||||
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
|
||||
t.Fatal("DecodeOptions() accepted unknown option")
|
||||
}
|
||||
}
|
||||
@@ -18,6 +18,7 @@ import (
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
combatcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/combatturns"
|
||||
npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs"
|
||||
scenecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
|
||||
combatextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
|
||||
combatnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/combatturns"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity"
|
||||
@@ -59,6 +60,7 @@ func TestProductionCombatPipelineRetriesMergesNormalizesAndWritesJSON(t *testing
|
||||
}
|
||||
fingerprints := prepared.CheckpointFingerprints()
|
||||
assertCombatFingerprint(t, fingerprints, "extract:combat:"+combatextract.Key+":npc_registry")
|
||||
assertCombatFingerprint(t, fingerprints, "extract:combat:"+combatextract.Key+":scene_eligibility")
|
||||
assertCombatFingerprint(t, fingerprints, "normalize:combat:"+combatnormalize.Key+":npc_registry")
|
||||
if len(fingerprints) == 0 {
|
||||
t.Fatal("checkpoint fingerprints = empty, want production combat identities")
|
||||
@@ -99,16 +101,16 @@ func TestProductionCombatPipelineRetriesMergesNormalizesAndWritesJSON(t *testing
|
||||
if !hasCombatWarning(output.Warnings, combatnormalize.ReasonCodeActorCanonicalized) {
|
||||
t.Fatalf("warnings = %#v, want registry normalization warning", output.Warnings)
|
||||
}
|
||||
if len(output.Manifest.References) != 2 {
|
||||
t.Fatalf("manifest references = %#v, want separate extract and normalize provenance", output.Manifest.References)
|
||||
if len(output.Manifest.References) != 3 {
|
||||
t.Fatalf("manifest references = %#v, want scene eligibility plus extract and normalize NPC provenance", output.Manifest.References)
|
||||
}
|
||||
lane := output.Manifest.ArtifactLanes[0]
|
||||
if lane.ID != "combat" || lane.Extractor != combatextract.Key || lane.Merger != pipeline.DefaultMergeModule || lane.Normalizer != combatnormalize.Key {
|
||||
t.Fatalf("manifest lane = %#v, want complete combat composition", lane)
|
||||
}
|
||||
extractorMetadata, ok := lane.Metadata["extractor"].(map[string]any)
|
||||
if !ok || extractorMetadata["npc_count"] != 2 || extractorMetadata["npc_registry_digest"] == "" {
|
||||
t.Fatalf("extractor metadata = %#v, want registry digest and count", lane.Metadata)
|
||||
if !ok || extractorMetadata["npc_count"] != 2 || extractorMetadata["npc_registry_digest"] == "" || extractorMetadata["scene_description_count"] != 3 || extractorMetadata["scene_eligibility_digest"] == "" || extractorMetadata["scene_gate_policy"] != "dnd.combat_turns.scene_gate.v1" {
|
||||
t.Fatalf("extractor metadata = %#v, want registry and scene eligibility metadata", lane.Metadata)
|
||||
}
|
||||
normalizerMetadata, ok := lane.Metadata["normalizer"].(map[string]any)
|
||||
if !ok || normalizerMetadata["npc_count"] != 2 || normalizerMetadata["normalization_policy"] != combatnormalize.NormalizationPolicy {
|
||||
@@ -131,6 +133,13 @@ func TestProductionCombatPipelineAttributesExhaustedInvalidEnumsToShapeValidatio
|
||||
configValue := combatOnlyConfig()
|
||||
profile := configValue.Pipelines["dnd-combat-fixture"]
|
||||
profile.Chunk.Options["max_units"] = 100
|
||||
profile.References["scene_descriptions"] = pipeline.ExternalReference(combatSceneDescriptionPath(t, []dnd.SceneDescription{{
|
||||
ID: "chunk-000001",
|
||||
SourceRef: source.SourceRef{SourceID: "npc-session", StartUnitID: 1, EndUnitID: 5},
|
||||
Kind: dnd.SceneKindCombat,
|
||||
Title: "Complete combat encounter",
|
||||
Summary: "The accepted chunk contains active combat.",
|
||||
}}))
|
||||
configValue.Pipelines["dnd-combat-fixture"] = profile
|
||||
effective, err := configValue.Resolve(config.ResolveInput{
|
||||
PipelineID: "dnd-combat-fixture",
|
||||
@@ -144,7 +153,11 @@ func TestProductionCombatPipelineAttributesExhaustedInvalidEnumsToShapeValidatio
|
||||
combatTestInvalidEnumResponse("invalid"),
|
||||
combatTestInvalidEnumResponse("unknown"),
|
||||
}}
|
||||
prepared, err := pipeline.Prepare(effective.ResolvedPipeline, registries, pipeline.ModuleDependencies{LLM: client})
|
||||
materialized, _, err := pipeline.MaterializeReferences(effective.ResolvedPipeline, moduleCatalog(registries), pipeline.ReferenceMaterializationOptions{})
|
||||
if err != nil {
|
||||
t.Fatalf("MaterializeReferences() error = %v", err)
|
||||
}
|
||||
prepared, err := pipeline.Prepare(materialized, registries, pipeline.ModuleDependencies{LLM: client})
|
||||
if err != nil {
|
||||
t.Fatalf("Prepare() error = %v, want nil", err)
|
||||
}
|
||||
@@ -168,6 +181,56 @@ func TestProductionCombatPipelineAttributesExhaustedInvalidEnumsToShapeValidatio
|
||||
}
|
||||
}
|
||||
|
||||
func TestCombatPipelinePublishesUnavailableSceneWarningForMismatchedReference(t *testing.T) {
|
||||
registries := productionNPCRegistries(t)
|
||||
configValue := combatOnlyConfig()
|
||||
profile := configValue.Pipelines["dnd-combat"]
|
||||
profile.Chunk.Options["max_units"] = 100
|
||||
profile.References["scene_descriptions"] = pipeline.ExternalReference(combatSceneDescriptionPath(t, []dnd.SceneDescription{{
|
||||
ID: "chunk-000001",
|
||||
SourceRef: source.SourceRef{SourceID: "npc-session", StartUnitID: 2, EndUnitID: 5},
|
||||
Kind: dnd.SceneKindCombat,
|
||||
Title: "Mismatched combat encounter",
|
||||
Summary: "The classification range does not match the accepted chunk.",
|
||||
}}))
|
||||
configValue.Pipelines["dnd-combat"] = profile
|
||||
catalog := moduleCatalog(registries)
|
||||
effective, err := configValue.Resolve(config.ResolveInput{PipelineID: "dnd-combat", Catalog: catalog})
|
||||
if err != nil {
|
||||
t.Fatalf("Resolve() error = %v", err)
|
||||
}
|
||||
materialized, warnings, err := pipeline.MaterializeReferences(effective.ResolvedPipeline, catalog, pipeline.ReferenceMaterializationOptions{})
|
||||
if err != nil || len(warnings) != 0 {
|
||||
t.Fatalf("MaterializeReferences() error = %v warnings = %#v", err, warnings)
|
||||
}
|
||||
client := &fakeCombatLLMClient{}
|
||||
prepared, err := pipeline.Prepare(materialized, registries, pipeline.ModuleDependencies{LLM: client})
|
||||
if err != nil {
|
||||
t.Fatalf("Prepare() error = %v", err)
|
||||
}
|
||||
output, err := pipeline.New().Run(context.Background(), pipeline.RunInput{
|
||||
Prepared: prepared,
|
||||
RawInput: readNPCFixture(t),
|
||||
ExtractWorkers: 1,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
if len(client.requests) != 0 || len(output.Rejected) != 0 || !hasCombatWarning(output.Warnings, "scene_classification_unavailable") {
|
||||
t.Fatalf("requests/rejected/warnings = %#v / %#v / %#v, want accepted unavailable-scene skip without a combat request", client.requests, output.Rejected, output.Warnings)
|
||||
}
|
||||
if len(output.NormalizeOutputs) != 1 {
|
||||
t.Fatalf("normalized outputs = %#v, want accepted empty combat lane", output.NormalizeOutputs)
|
||||
}
|
||||
value, err := combatcodec.New().Decode(output.NormalizeOutputs[0].Artifact.Content)
|
||||
if err != nil {
|
||||
t.Fatalf("Decode(combat output) error = %v", err)
|
||||
}
|
||||
if value.CombatTurns == nil || len(value.CombatTurns) != 0 {
|
||||
t.Fatalf("mismatched scene combat output = %#v, want non-nil empty turns", value)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCombatNormalizerRejectsCampaignReferenceBinding(t *testing.T) {
|
||||
registries := productionNPCRegistries(t)
|
||||
catalog := moduleCatalog(registries)
|
||||
@@ -267,11 +330,25 @@ func combatTestNPCPayload(t *testing.T) combatNPCPayload {
|
||||
return combatNPCPayload{path: path}
|
||||
}
|
||||
|
||||
func combatSceneDescriptionPath(t *testing.T, scenes []dnd.SceneDescription) string {
|
||||
t.Helper()
|
||||
content, err := scenecodec.New().Encode(dnd.SceneDescriptionList{Scenes: scenes})
|
||||
if err != nil {
|
||||
t.Fatalf("Encode(scene descriptions) error = %v", err)
|
||||
}
|
||||
path := filepath.Join(t.TempDir(), "scene-descriptions.json")
|
||||
if err := os.WriteFile(path, content, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return path
|
||||
}
|
||||
|
||||
func combatOnlyConfig() config.Config {
|
||||
cfg := config.Default()
|
||||
cfg.Pipelines["dnd-combat"] = pipeline.PipelineProfile{
|
||||
Input: pipeline.Binding(transcript.Key),
|
||||
Chunk: pipeline.ModuleBinding{Module: pipeline.DefaultChunkModule, Options: map[string]any{"max_units": 2}},
|
||||
References: map[string]pipeline.ReferenceSource{"scene_descriptions": pipeline.ExternalReference(repositoryPathForIntegration("internal", "modules", "integration", "testdata", "combat_scene_descriptions.json"))},
|
||||
Artifacts: map[string]pipeline.ArtifactLaneProfile{
|
||||
"combat": {
|
||||
Extract: pipeline.ModuleBinding{Module: combatextract.Key, Retries: 2},
|
||||
|
||||
170
internal/modules/integration/dnd_itemevents_runner_test.go
Normal file
170
internal/modules/integration/dnd_itemevents_runner_test.go
Normal file
@@ -0,0 +1,170 @@
|
||||
package integration_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
itemeventcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/itemevents"
|
||||
itemeventextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/itemevents"
|
||||
itemeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/itemevents"
|
||||
itemeventshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/shape"
|
||||
itemeventrefs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/itemevents/source_refs"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/seriatim/input/transcript"
|
||||
)
|
||||
|
||||
func TestProductionItemEventPipelineProducesNormalizedDurableArtifact(t *testing.T) {
|
||||
raw := readItemEventFixture(t)
|
||||
registries := productionNPCRegistries(t)
|
||||
resolved := resolveItemEventPipeline(t, registries)
|
||||
client := &itemEventProductionLLMClient{responses: [][]byte{[]byte(`{"events":[
|
||||
{"name":"Healing Potion","kind":"transferred","quantity":1,"from":"Aria","to":"Borin","source_refs":[{"start_segment":3,"end_segment":3}]},
|
||||
{"name":" Gold Pieces ","kind":"acquired","quantity":25,"to":" party ","source_refs":[{"start_segment":2,"end_segment":2}]},
|
||||
{"name":"Missing Relic","kind":"discovered","source_refs":[{"start_segment":4,"end_segment":4}]},
|
||||
{"name":"Ancient Coin","kind":"discovered","source_refs":[{"start_segment":1,"end_segment":1}]},
|
||||
{"name":"Gold Pieces","kind":"acquired","quantity":25,"to":"party","source_refs":[{"start_segment":2,"end_segment":2}]}
|
||||
]}`)}}
|
||||
|
||||
output, err := runPreparedPipeline(t, registries, resolved, client, pipeline.RunInput{RawInput: raw})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
if len(output.Rejected) != 0 || len(output.NormalizeOutputs) != 1 {
|
||||
t.Fatalf("run output = %#v, rejected = %#v", output.NormalizeOutputs, output.Rejected)
|
||||
}
|
||||
serialized := normalizedLane(t, output, "item-events")
|
||||
if serialized.NormalizerKey != itemeventnormalize.Key || serialized.Artifact.Schema.ID != itemeventcodec.SchemaID || serialized.Artifact.Schema.Version != itemeventcodec.SchemaVersion {
|
||||
t.Fatalf("serialized item events = %#v", serialized)
|
||||
}
|
||||
events, err := itemeventcodec.New().Decode(serialized.Artifact.Content)
|
||||
if err != nil {
|
||||
t.Fatalf("Decode(item event output) error = %v", err)
|
||||
}
|
||||
if len(events.Events) != 4 {
|
||||
t.Fatalf("item events = %#v, want one exact duplicate removed", events)
|
||||
}
|
||||
first, second, third, fourth := events.Events[0], events.Events[1], events.Events[2], events.Events[3]
|
||||
if first.Name != "Ancient Coin" || second.Name != "Gold Pieces" || second.Kind != dnd.ItemEventKindAcquired || second.Quantity == nil || *second.Quantity != 25 || second.To != "party" || third.Name != "Healing Potion" || third.Kind != dnd.ItemEventKindTransferred || third.From != "Aria" || third.To != "Borin" || fourth.Name != "Missing Relic" {
|
||||
t.Fatalf("normalized item event sequence = %#v", events.Events)
|
||||
}
|
||||
for _, event := range events.Events {
|
||||
for _, ref := range event.SourceRefs {
|
||||
if ref.SourceID != "item-events-session" {
|
||||
t.Fatalf("source reference = %#v, want current transcript source", ref)
|
||||
}
|
||||
}
|
||||
}
|
||||
if !hasItemEventWarning(output.Warnings, "item_event_source_unrelated") || !hasItemEventWarning(output.Warnings, "duplicate_item_event_collapsed") {
|
||||
t.Fatalf("warnings = %#v, want advisory relatedness and duplicate-collapse warnings", output.Warnings)
|
||||
}
|
||||
if output.Manifest.ValidationStatus != "approved" || len(output.Manifest.ArtifactLanes) != 1 {
|
||||
t.Fatalf("manifest = %#v", output.Manifest)
|
||||
}
|
||||
lane := output.Manifest.ArtifactLanes[0]
|
||||
if lane.ID != "item-events" || lane.Extractor != itemeventextract.Key || lane.Merger != pipeline.DefaultMergeModule || lane.Normalizer != itemeventnormalize.Key {
|
||||
t.Fatalf("manifest lane = %#v", lane)
|
||||
}
|
||||
if len(client.requests) != 1 || client.requests[0].PromptID != itemeventextract.PromptID {
|
||||
t.Fatalf("LLM requests = %#v", client.requests)
|
||||
}
|
||||
}
|
||||
|
||||
func TestProductionItemEventPipelineRetriesBlockingCandidates(t *testing.T) {
|
||||
registries := productionNPCRegistries(t)
|
||||
resolved := resolveItemEventPipeline(t, registries)
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
response string
|
||||
reasonCode string
|
||||
validatorName string
|
||||
}{
|
||||
{name: "shape", response: `{"events":[{"name":"","kind":"discovered","source_refs":[{"start_segment":1,"end_segment":1}]}]}`, reasonCode: itemeventshape.ReasonCode, validatorName: itemeventshape.Key},
|
||||
{name: "source references", response: `{"events":[{"name":"Ancient Coin","kind":"discovered","source_refs":[{"start_segment":99,"end_segment":99}]}]}`, reasonCode: itemeventrefs.ReasonCode, validatorName: itemeventrefs.Key},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
client := &itemEventProductionLLMClient{responses: [][]byte{[]byte(test.response), []byte(test.response), []byte(test.response)}}
|
||||
output, err := runPreparedPipeline(t, registries, resolved, client, pipeline.RunInput{RawInput: readItemEventFixture(t)})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v; want rejected candidate outcome", err)
|
||||
}
|
||||
if len(output.NormalizeOutputs) != 0 || len(output.Rejected) != 1 || len(client.requests) != 3 {
|
||||
t.Fatalf("output=%#v rejected=%#v requests=%d", output.NormalizeOutputs, output.Rejected, len(client.requests))
|
||||
}
|
||||
rejection := output.Rejected[0]
|
||||
if rejection.ReasonCode != test.reasonCode || rejection.ValidatorName != test.validatorName || rejection.AttemptCount != 3 {
|
||||
t.Fatalf("rejection = %#v", rejection)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func resolveItemEventPipeline(t *testing.T, registries pipeline.Registries) pipeline.ResolvedPipeline {
|
||||
t.Helper()
|
||||
configValue := config.Default()
|
||||
configValue.Pipelines["dnd-item-events-fixture"] = pipeline.PipelineProfile{
|
||||
Input: pipeline.Binding(transcript.Key),
|
||||
Chunk: pipeline.ModuleBinding{Module: pipeline.DefaultChunkModule, Options: map[string]any{"max_units": 100}},
|
||||
Artifacts: map[string]pipeline.ArtifactLaneProfile{
|
||||
"item-events": {
|
||||
Extract: pipeline.ModuleBinding{Module: itemeventextract.Key, Retries: 2},
|
||||
Normalize: pipeline.Binding(itemeventnormalize.Key),
|
||||
},
|
||||
},
|
||||
}
|
||||
effective, err := configValue.Resolve(config.ResolveInput{PipelineID: "dnd-item-events-fixture", Catalog: moduleCatalog(registries)})
|
||||
if err != nil {
|
||||
t.Fatalf("Resolve() error = %v", err)
|
||||
}
|
||||
return effective.ResolvedPipeline
|
||||
}
|
||||
|
||||
func readItemEventFixture(t *testing.T) []byte {
|
||||
t.Helper()
|
||||
raw, err := os.ReadFile("testdata/seriatim_item_events_session.json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return raw
|
||||
}
|
||||
|
||||
func hasItemEventWarning(warnings []contracts.Warning, reasonCode string) bool {
|
||||
for _, warning := range warnings {
|
||||
if warning.ReasonCode == reasonCode && strings.HasPrefix(warning.Scope, "events[") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
type itemEventProductionLLMClient struct {
|
||||
responses [][]byte
|
||||
requests []contracts.StructuredCompletionRequest
|
||||
}
|
||||
|
||||
func (client *itemEventProductionLLMClient) CompleteStructured(ctx context.Context, request contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, err
|
||||
}
|
||||
if request.PromptID != itemeventextract.PromptID {
|
||||
return contracts.StructuredCompletionResponse{}, fmt.Errorf("unexpected prompt %q", request.PromptID)
|
||||
}
|
||||
index := len(client.requests)
|
||||
if index >= len(client.responses) {
|
||||
return contracts.StructuredCompletionResponse{}, fmt.Errorf("missing item event response %d", index)
|
||||
}
|
||||
content := append([]byte(nil), client.responses[index]...)
|
||||
if err := json.Unmarshal(content, out); err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, fmt.Errorf("populate item event structured target: %w", err)
|
||||
}
|
||||
client.requests = append(client.requests, request)
|
||||
return contracts.StructuredCompletionResponse{Content: content, Provider: "test", Model: "item-events-fake"}, nil
|
||||
}
|
||||
|
||||
var _ contracts.StructuredLLMClient = (*itemEventProductionLLMClient)(nil)
|
||||
@@ -5,6 +5,8 @@ import (
|
||||
"crypto/sha256"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
@@ -16,9 +18,11 @@ import (
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
combatcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/combatturns"
|
||||
npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs"
|
||||
scenecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
|
||||
spellcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/spells"
|
||||
combatextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcs"
|
||||
sceneextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/scenedescriptions"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
|
||||
)
|
||||
|
||||
@@ -26,6 +30,7 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
|
||||
registries := productionNPCRegistries(t)
|
||||
catalog := moduleCatalog(registries)
|
||||
configValue := loadGroundedPipelineConfig(t)
|
||||
campaignInputs := configureGroundedCampaignReferences(t, &configValue)
|
||||
effective, err := configValue.Resolve(config.ResolveInput{PipelineID: "dnd-npc-grounded", Catalog: catalog})
|
||||
if err != nil {
|
||||
t.Fatalf("Resolve() error = %v", err)
|
||||
@@ -43,6 +48,7 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
|
||||
for name, value := range map[string]string{
|
||||
"extract:npcs:dnd/npcs:mapping_policy": "dnd.npcs.extract_mapping.v2",
|
||||
"extract:spells:dnd/spells:mapping_policy": "dnd.spells.extract_mapping.v2",
|
||||
"extract:combat:dnd/combat-turns:scene_gate_policy": "dnd.combat_turns.scene_gate.v1",
|
||||
} {
|
||||
assertFingerprintValue(t, prepared.CheckpointFingerprints(), name, value)
|
||||
}
|
||||
@@ -54,23 +60,26 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
|
||||
"extract:spells:dnd/spells:npc_registry",
|
||||
"extract:combat:dnd/combat-turns:prompt",
|
||||
"extract:combat:dnd/combat-turns:response_schema",
|
||||
"extract:combat:dnd/combat-turns:scene_eligibility",
|
||||
"extract:combat:dnd/combat-turns:npc_registry",
|
||||
"normalize:combat:dnd/combat-turns:npc_registry",
|
||||
} {
|
||||
assertCombatFingerprint(t, prepared.CheckpointFingerprints(), name)
|
||||
}
|
||||
checkpoint := newGeneratedReferenceCheckpointLoader()
|
||||
output, err := pipeline.New().Run(context.Background(), pipeline.RunInput{
|
||||
Prepared: prepared,
|
||||
RawInput: readNPCFixture(t),
|
||||
ExtractWorkers: 1,
|
||||
Checkpoint: checkpoint,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
if len(output.Rejected) != 0 || len(output.NormalizeOutputs) != 3 {
|
||||
t.Fatalf("run outputs = %#v rejected = %#v, want NPC, spell, and combat outputs", output.NormalizeOutputs, output.Rejected)
|
||||
if len(output.Rejected) != 0 || len(output.NormalizeOutputs) != 4 {
|
||||
t.Fatalf("run outputs = %#v rejected = %#v, want NPC, scene, spell, and combat outputs", output.NormalizeOutputs, output.Rejected)
|
||||
}
|
||||
wantSchemas := map[string]string{"npcs": npccodec.SchemaID, "spells": spellcodec.SchemaID, "combat": combatcodec.SchemaID}
|
||||
wantSchemas := map[string]string{"npcs": npccodec.SchemaID, "scene-descriptions": scenecodec.SchemaID, "spells": spellcodec.SchemaID, "combat": combatcodec.SchemaID}
|
||||
for _, serialized := range output.NormalizeOutputs {
|
||||
if serialized.Artifact.Schema.ID != wantSchemas[serialized.LaneID] || serialized.Artifact.Schema.Version != "v1" {
|
||||
t.Fatalf("%s artifact schema = %#v, want minimal v1 identity", serialized.LaneID, serialized.Artifact.Schema)
|
||||
@@ -78,6 +87,7 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
|
||||
}
|
||||
wantExtractorIdentity := map[string]struct{ promptID, schemaID string }{
|
||||
"npcs": {npcs.PromptID, npcs.ResponseSchemaID},
|
||||
"scene-descriptions": {sceneextract.PromptID, sceneextract.ResponseSchemaID},
|
||||
"spells": {spells.PromptID, spells.ResponseSchemaID},
|
||||
"combat": {combatextract.PromptID, combatextract.ResponseSchemaID},
|
||||
}
|
||||
@@ -89,16 +99,26 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
|
||||
}
|
||||
}
|
||||
|
||||
var npcPayload []byte
|
||||
var npcPayload, scenePayload []byte
|
||||
seenSteps := map[string]string{}
|
||||
for _, serialized := range output.NormalizeOutputs {
|
||||
if serialized.LaneID == "npcs" {
|
||||
seenSteps[serialized.LaneID] = serialized.StepID
|
||||
switch serialized.LaneID {
|
||||
case "npcs":
|
||||
npcPayload = append([]byte(nil), serialized.Artifact.Content...)
|
||||
break
|
||||
case "scene-descriptions":
|
||||
scenePayload = append([]byte(nil), serialized.Artifact.Content...)
|
||||
}
|
||||
}
|
||||
if seenSteps["npcs"] != "identify-npcs" || seenSteps["scene-descriptions"] != "identify-npcs" || seenSteps["spells"] != "grounded-events" || seenSteps["combat"] != "grounded-events" {
|
||||
t.Fatalf("normalized output steps = %#v, want ordered producer and consumer steps", seenSteps)
|
||||
}
|
||||
if len(npcPayload) == 0 {
|
||||
t.Fatal("NPC producer did not publish a canonical payload")
|
||||
}
|
||||
if len(scenePayload) == 0 {
|
||||
t.Fatal("scene producer did not publish a canonical payload")
|
||||
}
|
||||
npcValue, err := npccodec.New().Decode(npcPayload)
|
||||
if err != nil {
|
||||
t.Fatalf("decode NPC producer payload: %v", err)
|
||||
@@ -108,6 +128,15 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
|
||||
t.Fatalf("encode canonical NPC producer payload: %v", err)
|
||||
}
|
||||
canonicalDigest := digestBytes(npcPayload)
|
||||
sceneValue, err := scenecodec.New().Decode(scenePayload)
|
||||
if err != nil {
|
||||
t.Fatalf("decode scene producer payload: %v", err)
|
||||
}
|
||||
scenePayload, err = scenecodec.New().Encode(sceneValue)
|
||||
if err != nil {
|
||||
t.Fatalf("encode canonical scene producer payload: %v", err)
|
||||
}
|
||||
sceneDigest := digestBytes(scenePayload)
|
||||
projection := []byte(`{"npcs":[{"name":"Mira Thorn"},{"name":"Hooded Guard"}]}`)
|
||||
projectionDigest := digestBytes(projection)
|
||||
seenConsumers := map[string]bool{}
|
||||
@@ -119,11 +148,29 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
|
||||
if input.MediaType != npccodec.MediaType || input.Digest != projectionDigest || string(input.Content) != string(projection) || input.OriginURI != "" {
|
||||
t.Fatalf("%s NPC prompt input = %#v, want names-only generated registry", request.PromptID, input)
|
||||
}
|
||||
if request.PromptID == combatextract.PromptID {
|
||||
transcript, ok := request.Inputs["transcript"]
|
||||
if !ok || !strings.Contains(string(transcript.Content), "Mira Thorn asks the party") {
|
||||
t.Fatalf("combat transcript input = %#v, want configured transcript content", transcript)
|
||||
}
|
||||
for name, want := range campaignInputs {
|
||||
input, ok := request.Inputs[name]
|
||||
if !ok || string(input.Content) != want {
|
||||
t.Fatalf("combat %s input = %#v, want configured campaign reference", name, input)
|
||||
}
|
||||
}
|
||||
if _, ok := request.Inputs["scene_descriptions"]; ok {
|
||||
t.Fatalf("combat prompt inputs = %#v, must not contain scene descriptions", request.Inputs)
|
||||
}
|
||||
}
|
||||
seenConsumers[request.PromptID] = true
|
||||
}
|
||||
if !seenConsumers[spells.PromptID] || !seenConsumers[combatextract.PromptID] {
|
||||
t.Fatalf("consumer prompt IDs = %#v, want spell and combat requests", seenConsumers)
|
||||
}
|
||||
if combatCompletionCount(client.requestsSnapshot()) != 1 {
|
||||
t.Fatalf("combat completion requests = %#v, want exactly one for the exact combat scene", client.requestsSnapshot())
|
||||
}
|
||||
|
||||
provenanceCount := 0
|
||||
for _, reference := range output.Manifest.References {
|
||||
@@ -138,19 +185,41 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
|
||||
if provenanceCount != 3 {
|
||||
t.Fatalf("NPC generated provenance count = %d, want spell extract plus combat extract/normalize", provenanceCount)
|
||||
}
|
||||
sceneProvenanceCount := 0
|
||||
for _, reference := range output.Manifest.References {
|
||||
if reference.SlotName == "scene_descriptions" {
|
||||
sceneProvenanceCount++
|
||||
if reference.Stage != string(pipeline.StageExtract) || reference.StepID != "grounded-events" || reference.LaneID != "combat" || reference.OriginType != "generated" || reference.Digest != sceneDigest || reference.ArtifactKind != string(dnd.SceneDescriptionListKind) || reference.SchemaID != scenecodec.SchemaID || reference.SchemaName != scenecodec.SchemaName || reference.SchemaVersion != scenecodec.SchemaVersion || reference.MediaType != scenecodec.MediaType || reference.ProducerPipeline != "dnd-npc-grounded" || reference.ProducerStep != "identify-npcs" || reference.ProducerLane != "scene-descriptions" || reference.ProducerModule != "dnd/scene-descriptions" {
|
||||
t.Fatalf("scene generated provenance = %#v, want canonical combat handoff identity", reference)
|
||||
}
|
||||
}
|
||||
}
|
||||
if sceneProvenanceCount != 1 {
|
||||
t.Fatalf("scene generated provenance count = %d, want combat extractor handoff", sceneProvenanceCount)
|
||||
}
|
||||
assertGeneratedSceneDependency(t, checkpoint.extractDependencies("combat"))
|
||||
manifestContent, err := json.Marshal(output.Manifest)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, forbidden := range []string{"Mira Thorn", "Hooded Guard", "For every NPC record", "Extract Dungeons & Dragons"} {
|
||||
for _, forbidden := range []string{"Mira Thorn", "Hooded Guard", "For every NPC record", "Extract Dungeons & Dragons", "A combat encounter", "The party faces an active encounter.", "Mira Thorn asks the party", `"scenes":`} {
|
||||
if strings.Contains(string(manifestContent), forbidden) {
|
||||
t.Fatalf("manifest exposes prompt or reference payload content %q", forbidden)
|
||||
}
|
||||
}
|
||||
for _, lane := range output.Manifest.ArtifactLanes {
|
||||
componentContent, err := json.Marshal(lane.Metadata)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal %s component metadata: %v", lane.ID, err)
|
||||
}
|
||||
for _, forbidden := range []string{"A combat encounter", "The party faces an active encounter.", "Mira Thorn asks the party", `"scenes":`} {
|
||||
if strings.Contains(string(componentContent), forbidden) {
|
||||
t.Fatalf("%s component metadata exposes scene or transcript content %q", lane.ID, forbidden)
|
||||
}
|
||||
}
|
||||
for _, component := range []string{"extractor", "normalizer"} {
|
||||
metadata, ok := lane.Metadata[component].(map[string]any)
|
||||
if ok && metadata["npc_registry_digest"] != nil {
|
||||
if ok && (metadata["npc_registry_digest"] != nil || metadata["scene_eligibility_digest"] != nil) {
|
||||
t.Fatalf("%s %s metadata = %#v, want generated identity only in framework provenance", lane.ID, component, metadata)
|
||||
}
|
||||
}
|
||||
@@ -179,6 +248,38 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
|
||||
assertCurrentEvidence(t, combatValue.CombatTurns[0].SourceRefs)
|
||||
}
|
||||
|
||||
func TestGroundedPipelineSkipsCombatForExactNarrativeScene(t *testing.T) {
|
||||
registries := productionNPCRegistries(t)
|
||||
configValue := loadGroundedPipelineConfig(t)
|
||||
profile := configValue.Pipelines["dnd-npc-grounded"]
|
||||
combat := profile.Steps[1].Artifacts["combat"]
|
||||
combat.Extract.Retries = 2
|
||||
profile.Steps[1].Artifacts["combat"] = combat
|
||||
configValue.Pipelines["dnd-npc-grounded"] = profile
|
||||
|
||||
client := &groundedDNDLLMClient{sceneKind: dnd.SceneKindNarrative}
|
||||
output := runGroundedPipeline(t, configValue, registries, client, nil)
|
||||
if combatCompletionCount(client.requestsSnapshot()) != 0 {
|
||||
t.Fatalf("combat completion requests = %#v, want none for an exact narrative scene", client.requestsSnapshot())
|
||||
}
|
||||
if len(output.Rejected) != 0 || hasCombatWarning(output.Warnings, "scene_classification_unavailable") {
|
||||
t.Fatalf("narrative output rejected/warnings = %#v / %#v, want accepted skip without unavailable warning", output.Rejected, output.Warnings)
|
||||
}
|
||||
combatValue := normalizedCombatOutput(t, output)
|
||||
if combatValue.CombatTurns == nil || len(combatValue.CombatTurns) != 0 {
|
||||
t.Fatalf("narrative combat output = %#v, want an accepted empty combat list", combatValue)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGeneratedSceneReferenceChangesCombatCheckpointDependency(t *testing.T) {
|
||||
registries := productionNPCRegistries(t)
|
||||
first := generatedSceneDependency(t, registries, "A combat encounter")
|
||||
second := generatedSceneDependency(t, registries, "A revised combat encounter")
|
||||
if first == second {
|
||||
t.Fatalf("generated combat dependencies = %q and %q, want canonical scene artifact change to invalidate reuse", first, second)
|
||||
}
|
||||
}
|
||||
|
||||
func assertFingerprintValue(t *testing.T, fingerprints []pipeline.CheckpointFingerprint, name, want string) {
|
||||
t.Helper()
|
||||
for _, fingerprint := range fingerprints {
|
||||
@@ -211,9 +312,128 @@ func loadGroundedPipelineConfig(t *testing.T) config.Config {
|
||||
return cfg
|
||||
}
|
||||
|
||||
func configureGroundedCampaignReferences(t *testing.T, cfg *config.Config) map[string]string {
|
||||
t.Helper()
|
||||
values := map[string]string{
|
||||
"party": "Mira Thorn: ally\n",
|
||||
"glossary": "Greencloak: Mira Thorn's title\n",
|
||||
}
|
||||
profile := cfg.Pipelines["dnd-npc-grounded"]
|
||||
profile.References = make(map[string]pipeline.ReferenceSource, len(values))
|
||||
for name, content := range values {
|
||||
path := filepath.Join(t.TempDir(), name+".txt")
|
||||
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
|
||||
t.Fatalf("write %s reference: %v", name, err)
|
||||
}
|
||||
profile.References[name] = pipeline.ExternalReference(path)
|
||||
}
|
||||
cfg.Pipelines["dnd-npc-grounded"] = profile
|
||||
return values
|
||||
}
|
||||
|
||||
func runGroundedPipeline(t *testing.T, configValue config.Config, registries pipeline.Registries, client *groundedDNDLLMClient, checkpoint pipeline.CheckpointLoader) pipeline.RunOutput {
|
||||
t.Helper()
|
||||
catalog := moduleCatalog(registries)
|
||||
effective, err := configValue.Resolve(config.ResolveInput{PipelineID: "dnd-npc-grounded", Catalog: catalog})
|
||||
if err != nil {
|
||||
t.Fatalf("Resolve() error = %v", err)
|
||||
}
|
||||
materialized, warnings, err := pipeline.MaterializeReferences(effective.ResolvedPipeline, catalog, pipeline.ReferenceMaterializationOptions{})
|
||||
if err != nil || len(warnings) != 0 {
|
||||
t.Fatalf("MaterializeReferences() error = %v warnings = %#v", err, warnings)
|
||||
}
|
||||
prepared, err := pipeline.Prepare(materialized, registries, pipeline.ModuleDependencies{LLM: client})
|
||||
if err != nil {
|
||||
t.Fatalf("Prepare() error = %v", err)
|
||||
}
|
||||
output, err := pipeline.New().Run(context.Background(), pipeline.RunInput{
|
||||
Prepared: prepared,
|
||||
RawInput: readNPCFixture(t),
|
||||
ExtractWorkers: 1,
|
||||
Checkpoint: checkpoint,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
return output
|
||||
}
|
||||
|
||||
func normalizedCombatOutput(t *testing.T, output pipeline.RunOutput) dnd.CombatTurnList {
|
||||
t.Helper()
|
||||
for _, serialized := range output.NormalizeOutputs {
|
||||
if serialized.LaneID != "combat" {
|
||||
continue
|
||||
}
|
||||
value, err := combatcodec.New().Decode(serialized.Artifact.Content)
|
||||
if err != nil {
|
||||
t.Fatalf("decode combat output: %v", err)
|
||||
}
|
||||
return value
|
||||
}
|
||||
t.Fatalf("normalized outputs = %#v, want combat lane", output.NormalizeOutputs)
|
||||
return dnd.CombatTurnList{}
|
||||
}
|
||||
|
||||
func combatCompletionCount(requests []contracts.StructuredCompletionRequest) int {
|
||||
count := 0
|
||||
for _, request := range requests {
|
||||
if request.PromptID == combatextract.PromptID {
|
||||
count++
|
||||
}
|
||||
}
|
||||
return count
|
||||
}
|
||||
|
||||
func generatedSceneDependency(t *testing.T, registries pipeline.Registries, sceneTitle string) string {
|
||||
t.Helper()
|
||||
client := &groundedDNDLLMClient{sceneTitle: sceneTitle}
|
||||
checkpoint := newGeneratedReferenceCheckpointLoader()
|
||||
runGroundedPipeline(t, loadGroundedPipelineConfig(t), registries, client, checkpoint)
|
||||
return assertGeneratedSceneDependency(t, checkpoint.extractDependencies("combat"))
|
||||
}
|
||||
|
||||
func assertGeneratedSceneDependency(t *testing.T, dependencies []pipeline.CheckpointFingerprint) string {
|
||||
t.Helper()
|
||||
for _, dependency := range dependencies {
|
||||
if dependency.Name == "generated-reference:scene_descriptions:0" && strings.HasPrefix(dependency.Value, "sha256:") {
|
||||
return dependency.Value
|
||||
}
|
||||
}
|
||||
t.Fatalf("combat extract checkpoint dependencies = %#v, want generated scene reference fingerprint", dependencies)
|
||||
return ""
|
||||
}
|
||||
|
||||
type generatedReferenceCheckpointLoader struct {
|
||||
pipeline.CheckpointLoader
|
||||
extract map[string][][]pipeline.CheckpointFingerprint
|
||||
}
|
||||
|
||||
func newGeneratedReferenceCheckpointLoader() *generatedReferenceCheckpointLoader {
|
||||
return &generatedReferenceCheckpointLoader{
|
||||
CheckpointLoader: pipeline.NoopCheckpointLoader(),
|
||||
extract: make(map[string][][]pipeline.CheckpointFingerprint),
|
||||
}
|
||||
}
|
||||
|
||||
func (loader *generatedReferenceCheckpointLoader) Enabled() bool { return true }
|
||||
|
||||
func (loader *generatedReferenceCheckpointLoader) Extract(laneID, _ string, dependencies []pipeline.CheckpointFingerprint) (pipeline.ExtractCheckpoint, pipeline.CheckpointDecision) {
|
||||
loader.extract[laneID] = append(loader.extract[laneID], append([]pipeline.CheckpointFingerprint(nil), dependencies...))
|
||||
return pipeline.ExtractCheckpoint{}, pipeline.NewCheckpointDecision(pipeline.CheckpointDecisionExecuted, pipeline.CheckpointReasonMissing)
|
||||
}
|
||||
|
||||
func (loader *generatedReferenceCheckpointLoader) extractDependencies(laneID string) []pipeline.CheckpointFingerprint {
|
||||
if len(loader.extract[laneID]) != 1 {
|
||||
return nil
|
||||
}
|
||||
return loader.extract[laneID][0]
|
||||
}
|
||||
|
||||
type groundedDNDLLMClient struct {
|
||||
mu sync.Mutex
|
||||
requests []contracts.StructuredCompletionRequest
|
||||
sceneKind dnd.SceneKind
|
||||
sceneTitle string
|
||||
}
|
||||
|
||||
func (client *groundedDNDLLMClient) CompleteStructured(ctx context.Context, request contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
|
||||
@@ -235,6 +455,16 @@ func (client *groundedDNDLLMClient) CompleteStructured(ctx context.Context, requ
|
||||
"name": "Hooded Guard", "source_refs": []any{map[string]int{"start_unit_id": 3, "end_unit_id": 3}},
|
||||
},
|
||||
}}
|
||||
case sceneextract.PromptID:
|
||||
kind := client.sceneKind
|
||||
if kind == "" {
|
||||
kind = dnd.SceneKindCombat
|
||||
}
|
||||
title := client.sceneTitle
|
||||
if title == "" {
|
||||
title = "A combat encounter"
|
||||
}
|
||||
payload = map[string]any{"kind": kind, "title": title, "summary": "The party faces an active encounter."}
|
||||
case spells.PromptID:
|
||||
payload = map[string]any{"spell_casts": []any{map[string]any{
|
||||
"caster": "Mira Thorn", "spell": "Cure Wounds",
|
||||
|
||||
37
internal/modules/integration/testdata/combat_scene_descriptions.json
vendored
Normal file
37
internal/modules/integration/testdata/combat_scene_descriptions.json
vendored
Normal file
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"scenes": [
|
||||
{
|
||||
"id": "chunk-000001",
|
||||
"source_ref": {
|
||||
"source_id": "npc-session",
|
||||
"start_unit_id": 1,
|
||||
"end_unit_id": 2
|
||||
},
|
||||
"kind": "combat",
|
||||
"title": "First combat exchange",
|
||||
"summary": "The first accepted chunk contains active combat."
|
||||
},
|
||||
{
|
||||
"id": "chunk-000002",
|
||||
"source_ref": {
|
||||
"source_id": "npc-session",
|
||||
"start_unit_id": 3,
|
||||
"end_unit_id": 4
|
||||
},
|
||||
"kind": "combat",
|
||||
"title": "Second combat exchange",
|
||||
"summary": "The second accepted chunk contains active combat."
|
||||
},
|
||||
{
|
||||
"id": "chunk-000003",
|
||||
"source_ref": {
|
||||
"source_id": "npc-session",
|
||||
"start_unit_id": 5,
|
||||
"end_unit_id": 5
|
||||
},
|
||||
"kind": "combat",
|
||||
"title": "Final combat exchange",
|
||||
"summary": "The final accepted chunk contains active combat."
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -16,12 +16,19 @@ pipelines:
|
||||
npcs:
|
||||
extract: dnd/npcs
|
||||
normalize: dnd/npcs
|
||||
scene-descriptions:
|
||||
extract: dnd/scene-descriptions
|
||||
normalize: dnd/scene-descriptions
|
||||
- id: grounded-events
|
||||
references:
|
||||
npcs:
|
||||
artifact:
|
||||
step: identify-npcs
|
||||
lane: npcs
|
||||
scene_descriptions:
|
||||
artifact:
|
||||
step: identify-npcs
|
||||
lane: scene-descriptions
|
||||
artifacts:
|
||||
spells:
|
||||
extract: dnd/spells
|
||||
|
||||
12
internal/modules/integration/testdata/seriatim_item_events_session.json
vendored
Normal file
12
internal/modules/integration/testdata/seriatim_item_events_session.json
vendored
Normal file
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"metadata": {
|
||||
"id": "item-events-session",
|
||||
"title": "Synthetic D&D item-event session"
|
||||
},
|
||||
"segments": [
|
||||
{"id": 1, "start": 0, "end": 4, "speaker": "DM", "text": "An ancient coin rests in the chest."},
|
||||
{"id": 2, "start": 4, "end": 8, "speaker": "DM", "text": "The party acquires 25 gold pieces."},
|
||||
{"id": 3, "start": 8, "end": 12, "speaker": "DM", "text": "Aria gives Borin a healing potion."},
|
||||
{"id": 4, "start": 12, "end": 16, "speaker": "DM", "text": "The party continues down the road."}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user