Compare commits
41 Commits
v0.2.0
...
f4c05c34ef
| Author | SHA1 | Date | |
|---|---|---|---|
| f4c05c34ef | |||
| 29fcad6e9b | |||
| f5fd115046 | |||
| 7f28899730 | |||
| 84c0758455 | |||
| 5002864e88 | |||
| 55b188fd84 | |||
| d1f43df88e | |||
| d52387c1f7 | |||
| 9c5e3cff14 | |||
| 811d5b8bd9 | |||
| a168c13b85 | |||
| dd61a4efda | |||
| 228cc6ee83 | |||
| 06170e1f65 | |||
| 7715baa1f6 | |||
| 98506db1a9 | |||
| bb4855f0c6 | |||
| c51934d5c6 | |||
| c3513da880 | |||
| c7d853ea52 | |||
| da7fcdaffd | |||
| b6aad4fa98 | |||
| 9c6af28d02 | |||
| 04eabdfcb9 | |||
| a8a99c1037 | |||
| e15007fffb | |||
| e6b7c61f45 | |||
| 42973215fa | |||
| ba1d112d1f | |||
| b722131d57 | |||
| a92d2c0885 | |||
| 02ec10d66b | |||
| 9dd57dbfa4 | |||
| c164a3fc69 | |||
| 8834df617f | |||
| db2adb52da | |||
| fc3c128171 | |||
| 8a15b083a0 | |||
| d9dae2b639 | |||
| 7a4fd7be7a |
@@ -2,8 +2,9 @@
|
|||||||
|
|
||||||
Notarius is a Go CLI for turning source material into structured artifacts with
|
Notarius is a Go CLI for turning source material into structured artifacts with
|
||||||
configured extraction pipelines. The implemented D&D workflow reads Seriatim
|
configured extraction pipelines. The implemented D&D workflow reads Seriatim
|
||||||
transcript JSON and can produce scene descriptions, item and currency events,
|
transcript JSON and can produce location registries and occurrences, scene
|
||||||
NPC identities, combat turns, NPC interactions, and spell casts.
|
descriptions, item and currency events, NPC identities, combat turns, NPC
|
||||||
|
interactions, enemy events, and spell casts.
|
||||||
|
|
||||||
## Quickstart
|
## Quickstart
|
||||||
|
|
||||||
|
|||||||
12
docs/cli.md
12
docs/cli.md
@@ -40,7 +40,7 @@ pipeline ID and **--input** are required.
|
|||||||
| **--debug-dir path** | Override the debug-bundle root. Requires **--debug**. |
|
| **--debug-dir path** | Override the debug-bundle root. Requires **--debug**. |
|
||||||
| **--only lane-a,lane-b** | Run only the selected comma-separated artifact lanes when that selection is valid for the configured pipeline. |
|
| **--only lane-a,lane-b** | Run only the selected comma-separated artifact lanes when that selection is valid for the configured pipeline. |
|
||||||
| **--llm-profile id** | Highest-precedence configured profile for selected LLM-backed bindings and validators; it replaces binding and [pipeline](config.md#pipelines) defaults. |
|
| **--llm-profile id** | Highest-precedence configured profile for selected LLM-backed bindings and validators; it replaces binding and [pipeline](config.md#pipelines) defaults. |
|
||||||
| **--session-id id** | Supply a non-empty prompt session identifier to LLM-backed module calls. |
|
| **--session-id id** | Override the generated prompt session identifier with a non-empty value for LLM-backed module calls. |
|
||||||
| **--reasoning-effort value** | Replace the selected PromptKit profile's reasoning effort for every LLM-backed call in this run. The value must be non-empty and the flag may be specified only once. |
|
| **--reasoning-effort value** | Replace the selected PromptKit profile's reasoning effort for every LLM-backed call in this run. The value must be non-empty and the flag may be specified only once. |
|
||||||
| **--clear-reasoning-effort** | Clear reasoning effort inherited from the selected PromptKit profile for every LLM-backed call in this run. |
|
| **--clear-reasoning-effort** | Clear reasoning effort inherited from the selected PromptKit profile for every LLM-backed call in this run. |
|
||||||
| **--reference selector=path** | Add or replace a file reference binding. Repeatable. |
|
| **--reference selector=path** | Add or replace a file reference binding. Repeatable. |
|
||||||
@@ -57,6 +57,16 @@ Persistent reasoning settings remain a PromptKit profile concern.
|
|||||||
**--recompute-step** requires **--resume**; checkpoint requirements and reuse
|
**--recompute-step** requires **--resume**; checkpoint requirements and reuse
|
||||||
behavior are documented in [Operations](operations.md).
|
behavior are documented in [Operations](operations.md).
|
||||||
|
|
||||||
|
Every run uses one effective prompt session. Without **--session-id**, Notarius
|
||||||
|
generates a stable `notarius:v1:` identifier from the trimmed resolved input
|
||||||
|
module key and the input file's exact raw bytes. The same module and bytes
|
||||||
|
therefore produce the same identifier, regardless of pipeline, references,
|
||||||
|
profile, retries, or run settings. An explicit non-empty value replaces that
|
||||||
|
default. Session identifiers are visible to providers; they are non-secret
|
||||||
|
correlation identifiers, not credential storage. See
|
||||||
|
[Operations](operations.md#operational-limits) for privacy and workflow
|
||||||
|
guidance.
|
||||||
|
|
||||||
### Reference selectors
|
### Reference selectors
|
||||||
|
|
||||||
Use **--reference** only for a reference slot declared by the selected
|
Use **--reference** only for a reference slot declared by the selected
|
||||||
|
|||||||
@@ -58,7 +58,7 @@ Built-in defaults are:
|
|||||||
|
|
||||||
| Field | Default |
|
| Field | Default |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| **concurrency.total_llm** | 1 |
|
| **concurrency.total_llm** | 16 |
|
||||||
| **concurrency.stage_workers.extract** | Effective **total_llm** |
|
| **concurrency.stage_workers.extract** | Effective **total_llm** |
|
||||||
| **output.directory** | **./notarius-output** |
|
| **output.directory** | **./notarius-output** |
|
||||||
| **cache.chunk_plans.mode** | **auto** |
|
| **cache.chunk_plans.mode** | **auto** |
|
||||||
@@ -370,14 +370,33 @@ selected target declares them:
|
|||||||
| **players** | Optional text player context. |
|
| **players** | Optional text player context. |
|
||||||
| **glossary** | Optional text campaign glossary. |
|
| **glossary** | Optional text campaign glossary. |
|
||||||
| **spell_catalog** | Optional JSON spell-catalog overlay for spell extraction and normalization. See [spell-catalog overlays](integrations/dnd-spell-catalog-overlays.md). |
|
| **spell_catalog** | Optional JSON spell-catalog overlay for spell extraction and normalization. See [spell-catalog overlays](integrations/dnd-spell-catalog-overlays.md). |
|
||||||
| **npcs** | Normalized NPC registry. Optional for spells and combat turns; required for NPC interactions. |
|
| **locations** | Required normalized location registry for location-occurrence extraction and normalization. |
|
||||||
| **scene_descriptions** | Required normalized scene-description artifact for combat-turn extraction. |
|
| **npcs** | Normalized NPC registry. Optional for spells and combat turns; required for NPC interactions and enemy-event extraction and normalization. |
|
||||||
|
| **scene_descriptions** | Required normalized scene-description artifact for combat-turn and enemy-event extraction. |
|
||||||
|
| **combat_turns** | Required normalized combat-turn artifact for enemy-event extraction. |
|
||||||
|
| **npc_interactions** | Required normalized NPC-interaction artifact for enemy-event extraction. |
|
||||||
|
|
||||||
|
Location-occurrence and enemy-event artifact slots have the following exact
|
||||||
|
binding contracts. Durable semantics and wire shapes remain in their
|
||||||
|
[location-occurrence](integrations/dnd-location-occurrence-artifacts.md) and
|
||||||
|
[enemy-event](integrations/dnd-enemy-event-artifacts.md) contracts.
|
||||||
|
|
||||||
|
| Slot | Accepted artifact kind | Media type | Maximum size | Required stage |
|
||||||
|
| --- | --- | --- | --- | --- |
|
||||||
|
| `npcs` | `dnd/npc-list` | `application/json` | 1,048,576 bytes | extract and normalize |
|
||||||
|
| `scene_descriptions` | `dnd/scene-description-list` | `application/json` | 1,048,576 bytes | extract only |
|
||||||
|
| `combat_turns` | `dnd/combat-turn-list` | `application/json` | 1,048,576 bytes | extract only |
|
||||||
|
| `npc_interactions` | `dnd/npc-interaction-list` | `application/json` | 1,048,576 bytes | extract only |
|
||||||
|
| `locations` | `dnd/location-list` | `application/json` | 1,048,576 bytes | location-occurrence extract and normalize |
|
||||||
|
|
||||||
Scene descriptions accept **party**, **players**, and **glossary**, but not
|
Scene descriptions accept **party**, **players**, and **glossary**, but not
|
||||||
**roster**. NPC interactions require **npcs** for both extraction and
|
**roster**. NPC interactions require **npcs** for both extraction and
|
||||||
normalization. Combat turns require **scene_descriptions** for extraction; the
|
normalization. Combat turns require **scene_descriptions** for extraction; the
|
||||||
normalized combat-turn module may use optional **npcs**. The complete example
|
normalized combat-turn module may use optional **npcs**. Location occurrences
|
||||||
shows generated **npcs** and **scene_descriptions** bindings.
|
require **locations** for extraction and normalization. Enemy-event extraction
|
||||||
|
requires all four of its JSON artifact slots; its normalizer requires **npcs**.
|
||||||
|
The [complete example](../examples/dnd-complete.config.yml) shows the ordered
|
||||||
|
generated bindings.
|
||||||
|
|
||||||
## Production Module Keys
|
## Production Module Keys
|
||||||
|
|
||||||
@@ -385,18 +404,27 @@ shows generated **npcs** and **scene_descriptions** bindings.
|
|||||||
| --- | --- |
|
| --- | --- |
|
||||||
| Input | **seriatim** |
|
| Input | **seriatim** |
|
||||||
| Chunk | **generic**, **dnd/scenes** |
|
| Chunk | **generic**, **dnd/scenes** |
|
||||||
| Extract | **dnd/spells**, **dnd/npcs**, **dnd/combat-turns**, **dnd/item-events**, **dnd/npc-interactions**, **dnd/scene-descriptions** |
|
| Extract | **dnd/spells**, **dnd/npcs**, **dnd/combat-turns**, **dnd/item-events**, **dnd/npc-interactions**, **dnd/scene-descriptions**, **dnd/enemy-events**, **dnd/locations**, **dnd/location-occurrences** |
|
||||||
| Merge | **appendorder** |
|
| Merge | **appendorder** |
|
||||||
| Normalize | **noop**, **dnd/spells**, **dnd/npcs**, **dnd/combat-turns**, **dnd/item-events**, **dnd/npc-interactions**, **dnd/scene-descriptions** |
|
| Normalize | **noop**, **dnd/spells**, **dnd/npcs**, **dnd/combat-turns**, **dnd/item-events**, **dnd/npc-interactions**, **dnd/scene-descriptions**, **dnd/enemy-events**, **dnd/locations**, **dnd/location-occurrences** |
|
||||||
| Output | **json** |
|
| Output | **json** |
|
||||||
|
|
||||||
|
`dnd/locations` extraction and normalization are `llm_backed`; location
|
||||||
|
normalization may use the pipeline's selected LLM profile for bounded duplicate
|
||||||
|
proposals. `dnd/location-occurrences` extraction is `llm_backed`, while its
|
||||||
|
normalizer is `deterministic`. The complete example binds the registry in one
|
||||||
|
step and the occurrence lane in the next.
|
||||||
|
|
||||||
The D&D artifact contracts define each emitted schema:
|
The D&D artifact contracts define each emitted schema:
|
||||||
[spells](integrations/dnd-spell-artifacts.md),
|
[spells](integrations/dnd-spell-artifacts.md),
|
||||||
[NPCs](integrations/dnd-npc-artifacts.md),
|
[NPCs](integrations/dnd-npc-artifacts.md),
|
||||||
[NPC interactions](integrations/dnd-npc-interaction-artifacts.md),
|
[NPC interactions](integrations/dnd-npc-interaction-artifacts.md),
|
||||||
[combat turns](integrations/dnd-combat-turn-artifacts.md),
|
[combat turns](integrations/dnd-combat-turn-artifacts.md),
|
||||||
[item events](integrations/dnd-item-event-artifacts.md), and
|
[item events](integrations/dnd-item-event-artifacts.md),
|
||||||
[scene descriptions](integrations/dnd-scene-description-artifacts.md).
|
[scene descriptions](integrations/dnd-scene-description-artifacts.md), and
|
||||||
|
[enemy events](integrations/dnd-enemy-event-artifacts.md),
|
||||||
|
[locations](integrations/dnd-location-artifacts.md), and
|
||||||
|
[location occurrences](integrations/dnd-location-occurrence-artifacts.md).
|
||||||
|
|
||||||
## Production Validator Keys And Default Chains
|
## Production Validator Keys And Default Chains
|
||||||
|
|
||||||
@@ -411,6 +439,9 @@ Available validator keys are:
|
|||||||
| Item events | **extract/dnd/item-events/shape**, **extract/dnd/item-events/source_refs**, **extract/dnd/item-events/source_relatedness**, **normalize/dnd/item-events/invariants** |
|
| Item events | **extract/dnd/item-events/shape**, **extract/dnd/item-events/source_refs**, **extract/dnd/item-events/source_relatedness**, **normalize/dnd/item-events/invariants** |
|
||||||
| NPC interactions | **extract/dnd/npc-interactions/shape**, **extract/dnd/npc-interactions/registry**, **extract/dnd/npc-interactions/source_refs**, **extract/dnd/npc-interactions/source_relatedness**, **normalize/dnd/npc-interactions/invariants** |
|
| NPC interactions | **extract/dnd/npc-interactions/shape**, **extract/dnd/npc-interactions/registry**, **extract/dnd/npc-interactions/source_refs**, **extract/dnd/npc-interactions/source_relatedness**, **normalize/dnd/npc-interactions/invariants** |
|
||||||
| Scene descriptions | **extract/dnd/scene-descriptions/shape**, **extract/dnd/scene-descriptions/source_refs**, **extract/dnd/scene-descriptions/source_relatedness**, **normalize/dnd/scene-descriptions/invariants** |
|
| Scene descriptions | **extract/dnd/scene-descriptions/shape**, **extract/dnd/scene-descriptions/source_refs**, **extract/dnd/scene-descriptions/source_relatedness**, **normalize/dnd/scene-descriptions/invariants** |
|
||||||
|
| Enemy events | **extract/dnd/enemy-events/shape**, **extract/dnd/enemy-events/engagements**, **extract/dnd/enemy-events/source_refs**, **extract/dnd/enemy-events/source_relatedness**, **normalize/dnd/enemy-events/invariants** |
|
||||||
|
| Locations | **extract/dnd/locations/shape**, **extract/dnd/locations/source_refs**, **extract/dnd/locations/source_relatedness**, **normalize/dnd/locations/identity** |
|
||||||
|
| Location occurrences | **extract/dnd/location-occurrences/shape**, **extract/dnd/location-occurrences/registry**, **extract/dnd/location-occurrences/source_refs**, **extract/dnd/location-occurrences/source_relatedness**, **normalize/dnd/location-occurrences/invariants** |
|
||||||
|
|
||||||
When no override is configured, production D&D bindings use the following
|
When no override is configured, production D&D bindings use the following
|
||||||
ordered chains. Each row lists extract then normalize; spell chains are the
|
ordered chains. Each row lists extract then normalize; spell chains are the
|
||||||
@@ -424,6 +455,9 @@ same at both stages.
|
|||||||
| Item events | generic/valid_json, extract/dnd/item-events/shape, extract/dnd/item-events/source_refs, generic/valid_json_schema, extract/dnd/item-events/source_relatedness | generic/valid_json, extract/dnd/item-events/shape, normalize/dnd/item-events/invariants, extract/dnd/item-events/source_refs, generic/valid_json_schema, extract/dnd/item-events/source_relatedness |
|
| Item events | generic/valid_json, extract/dnd/item-events/shape, extract/dnd/item-events/source_refs, generic/valid_json_schema, extract/dnd/item-events/source_relatedness | generic/valid_json, extract/dnd/item-events/shape, normalize/dnd/item-events/invariants, extract/dnd/item-events/source_refs, generic/valid_json_schema, extract/dnd/item-events/source_relatedness |
|
||||||
| NPC interactions | generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness | generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, normalize/dnd/npc-interactions/invariants, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness |
|
| NPC interactions | generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness | generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, normalize/dnd/npc-interactions/invariants, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness |
|
||||||
| Scene descriptions | generic/valid_json, extract/dnd/scene-descriptions/shape, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness | generic/valid_json, extract/dnd/scene-descriptions/shape, normalize/dnd/scene-descriptions/invariants, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness |
|
| Scene descriptions | generic/valid_json, extract/dnd/scene-descriptions/shape, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness | generic/valid_json, extract/dnd/scene-descriptions/shape, normalize/dnd/scene-descriptions/invariants, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness |
|
||||||
|
| Enemy events | generic/valid_json, extract/dnd/enemy-events/shape, extract/dnd/enemy-events/engagements, extract/dnd/enemy-events/source_refs, generic/valid_json_schema, extract/dnd/enemy-events/source_relatedness | generic/valid_json, extract/dnd/enemy-events/shape, normalize/dnd/enemy-events/invariants, extract/dnd/enemy-events/source_refs, generic/valid_json_schema, extract/dnd/enemy-events/source_relatedness |
|
||||||
|
| Locations | generic/valid_json, extract/dnd/locations/shape, extract/dnd/locations/source_refs, generic/valid_json_schema, extract/dnd/locations/source_relatedness | generic/valid_json, extract/dnd/locations/shape, normalize/dnd/locations/identity, extract/dnd/locations/source_refs, generic/valid_json_schema, extract/dnd/locations/source_relatedness |
|
||||||
|
| Location occurrences | generic/valid_json, extract/dnd/location-occurrences/shape, extract/dnd/location-occurrences/registry, extract/dnd/location-occurrences/source_refs, generic/valid_json_schema, extract/dnd/location-occurrences/source_relatedness | generic/valid_json, extract/dnd/location-occurrences/shape, extract/dnd/location-occurrences/registry, normalize/dnd/location-occurrences/invariants, extract/dnd/location-occurrences/source_refs, generic/valid_json_schema, extract/dnd/location-occurrences/source_relatedness |
|
||||||
|
|
||||||
Chains are only registered for the D&D extract and normalize modules shown
|
Chains are only registered for the D&D extract and normalize modules shown
|
||||||
above; select an explicit override when a different compatible chain is
|
above; select an explicit override when a different compatible chain is
|
||||||
|
|||||||
@@ -27,10 +27,13 @@ notarius run pipeline-id \
|
|||||||
```
|
```
|
||||||
|
|
||||||
Use absolute paths for supplied input, configuration, output-root, and
|
Use absolute paths for supplied input, configuration, output-root, and
|
||||||
reference files. When a stable prompt session identifier or references are
|
reference files. Notarius generates a stable prompt session for the resolved
|
||||||
needed, pass the supported CLI flags. Supply credentials through Notarius's
|
input module and exact input bytes. Pass **--session-id** only when intentionally
|
||||||
documented configuration and environment mechanisms, never as command-line
|
grouping different invocations under a different session. Supply credentials
|
||||||
arguments or generated secret-bearing configuration.
|
through Notarius's documented configuration and environment mechanisms, never
|
||||||
|
as command-line arguments or generated secret-bearing configuration. In
|
||||||
|
particular, a session identifier is provider-visible and is not a credential
|
||||||
|
mechanism.
|
||||||
|
|
||||||
Wait for the process before interpreting standard output. Only an exit status
|
Wait for the process before interpreting standard output. Only an exit status
|
||||||
of 0 permits decoding the receipt. On a nonzero exit, retain standard error for
|
of 0 permits decoding the receipt. On a nonzero exit, retain standard error for
|
||||||
|
|||||||
@@ -64,6 +64,8 @@ kind, and complete valid evidence. It does not infer turns, initiative, or
|
|||||||
actions from registry or scene data.
|
actions from registry or scene data.
|
||||||
|
|
||||||
The [NPC-interaction artifact](dnd-npc-interaction-artifacts.md) records
|
The [NPC-interaction artifact](dnd-npc-interaction-artifacts.md) records
|
||||||
broader NPC occurrences. The [JSON output contract](json-output.md) defines
|
broader NPC occurrences. The [enemy-event artifact](dnd-enemy-event-artifacts.md)
|
||||||
publication, and [D&D module internals](../internal/dnd.md) describes routing
|
uses combat turns as grounding only; turns do not establish an enemy event or
|
||||||
and validation mechanics.
|
its outcome. The [JSON output contract](json-output.md) defines publication,
|
||||||
|
and [D&D module internals](../internal/dnd.md) describes routing and validation
|
||||||
|
mechanics.
|
||||||
|
|||||||
114
docs/integrations/dnd-enemy-event-artifacts.md
Normal file
114
docs/integrations/dnd-enemy-event-artifacts.md
Normal file
@@ -0,0 +1,114 @@
|
|||||||
|
# D&D Enemy-Event Artifact
|
||||||
|
|
||||||
|
This contract defines the durable, source-grounded enemy-event occurrence list.
|
||||||
|
It records enemies directly established as opposing the party and explicitly
|
||||||
|
observed combat outcomes. It is an ordered observation artifact from which a
|
||||||
|
consumer may derive a ledger; it is not a ledger, encounter roster, or terminal
|
||||||
|
state model.
|
||||||
|
|
||||||
|
## Identity and compatibility
|
||||||
|
|
||||||
|
| Property | Value |
|
||||||
|
| --- | --- |
|
||||||
|
| Artifact kind | `dnd/enemy-event-list` |
|
||||||
|
| Schema ID | `notarius.dnd.enemy_events` |
|
||||||
|
| Schema name | `notarius_dnd_enemy_events_v1` |
|
||||||
|
| Schema version | `v1` |
|
||||||
|
| Media type | `application/json` |
|
||||||
|
|
||||||
|
`v1` is a strict JSON object with required `events`; the array may be empty.
|
||||||
|
Event and source-reference objects reject unknown fields. An incompatible shape
|
||||||
|
change requires a new schema version.
|
||||||
|
|
||||||
|
## Wire shape
|
||||||
|
|
||||||
|
Every event has these required fields:
|
||||||
|
|
||||||
|
| Field | Contract |
|
||||||
|
| --- | --- |
|
||||||
|
| `name` | Non-empty display name or directly grounded collective subject label. |
|
||||||
|
| `kind` | `engaged`, `killed`, `fled`, `captured`, or `incapacitated`. |
|
||||||
|
| `source_refs` | One or more current-transcript evidence ranges. |
|
||||||
|
|
||||||
|
Each source reference has exactly `source_id`, `start_unit_id`, and
|
||||||
|
`end_unit_id`. It identifies an inclusive current-transcript range; unit IDs
|
||||||
|
are positive and the start may not follow the end.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"events": [
|
||||||
|
{
|
||||||
|
"name": "Ashfang",
|
||||||
|
"kind": "engaged",
|
||||||
|
"source_refs": [
|
||||||
|
{"source_id": "session-7", "start_unit_id": 41, "end_unit_id": 42}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Ashfang",
|
||||||
|
"kind": "fled",
|
||||||
|
"source_refs": [
|
||||||
|
{"source_id": "session-7", "start_unit_id": 57, "end_unit_id": 58}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Event semantics and evidence
|
||||||
|
|
||||||
|
| Kind | Required evidence |
|
||||||
|
| --- | --- |
|
||||||
|
| `engaged` | The subject is directly established as actively opposing the party in combat. At most one engagement is emitted for one subject in one combat scene. |
|
||||||
|
| `killed` | The transcript explicitly establishes that the subject died or was killed. Damage, defeat, disappearance, or combat ending is insufficient. |
|
||||||
|
| `fled` | The subject explicitly escapes, retreats, or otherwise leaves combat to avoid continued engagement. Movement or absence from later turns is insufficient. |
|
||||||
|
| `captured` | The subject is explicitly taken prisoner or secured under the party's control. A grapple or temporary restraint alone is insufficient. |
|
||||||
|
| `incapacitated` | The subject is explicitly rendered unable to continue acting without being established as killed or captured. A missed turn is insufficient. |
|
||||||
|
|
||||||
|
The current transcript is the only event evidence. Campaign context and
|
||||||
|
normalized NPC, scene-description, combat-turn, and NPC-interaction artifacts
|
||||||
|
can ground names or control combat eligibility, but none may supply event
|
||||||
|
evidence. An outcome may share evidence with an engagement, in which case both
|
||||||
|
events are retained.
|
||||||
|
|
||||||
|
Extraction is limited to chunks with an exact combat-scene classification. An
|
||||||
|
exact non-combat classification produces an accepted empty list. Missing or
|
||||||
|
mismatched classification also produces an accepted empty list and a
|
||||||
|
`scene_classification_unavailable` warning.
|
||||||
|
|
||||||
|
## Subjects, normalization, and order
|
||||||
|
|
||||||
|
A subject matching the normalized NPC registry uses that registry's canonical
|
||||||
|
display name. Unmatched hostile creatures, summoned entities, and directly
|
||||||
|
grounded groups remain valid subjects. An unnamed homogeneous group uses the
|
||||||
|
narrowest transcript-grounded label, such as `Orcs`, `One orc`, or `Remaining
|
||||||
|
orcs`; the artifact never invents synthetic member identities or quantities.
|
||||||
|
Party members, allies, neutral observers, mentioned-but-absent enemies, hazards,
|
||||||
|
traps, and environmental effects are excluded.
|
||||||
|
|
||||||
|
Normalization collapses surrounding and repeated internal whitespace in subject
|
||||||
|
display values, canonicalizes recognized registry names, canonicalizes and
|
||||||
|
deduplicates exact source ranges, then orders events by valid evidence
|
||||||
|
chronology, normalized subject identity, display name, kind, and reference
|
||||||
|
sequence. The deterministic kind tie order is `engaged`,
|
||||||
|
`incapacitated`, `captured`, `fled`, then `killed`. Only entries with the same
|
||||||
|
normalized name, kind, and complete canonical evidence sequence are collapsed.
|
||||||
|
Different kinds, evidence, repeated engagement in separate scenes, and later
|
||||||
|
outcomes remain separate. A later engagement for the same named subject is
|
||||||
|
preserved after an earlier outcome because the artifact does not assert an
|
||||||
|
irreversible state transition.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
The artifact has no NPC or scene ID, quantity, confidence, description,
|
||||||
|
rationale, summary, current state, or inferred terminal outcome. It does not
|
||||||
|
emit `active` or `unresolved`; consumers may derive an unresolved ledger view
|
||||||
|
only when an engagement has no later explicit outcome. It never infers an
|
||||||
|
outcome from turn absence, scene termination, initiative order, hit-point
|
||||||
|
guesses, or other artifacts.
|
||||||
|
|
||||||
|
The [JSON output contract](json-output.md) defines publication. Configuration
|
||||||
|
keys, required generated-reference slots, and validator-chain selection are
|
||||||
|
defined in the [configuration reference](../config.md). Implementation and
|
||||||
|
prompt-grounding mechanics are described in the
|
||||||
|
[D&D module internals](../internal/dnd.md).
|
||||||
89
docs/integrations/dnd-location-artifacts.md
Normal file
89
docs/integrations/dnd-location-artifacts.md
Normal file
@@ -0,0 +1,89 @@
|
|||||||
|
# D&D Location Artifact
|
||||||
|
|
||||||
|
This contract defines the durable, source-grounded location registry produced
|
||||||
|
by `dnd/locations`. It records transcript-established physical places for one
|
||||||
|
source document; it is not a map, location hierarchy, campaign-wide world
|
||||||
|
registry, or location description.
|
||||||
|
|
||||||
|
## Identity and compatibility
|
||||||
|
|
||||||
|
| Property | Value |
|
||||||
|
| --- | --- |
|
||||||
|
| Artifact kind | `dnd/location-list` |
|
||||||
|
| Schema ID | `notarius.dnd.locations` |
|
||||||
|
| Schema name | `notarius_dnd_locations_v1` |
|
||||||
|
| Schema version | `v1` |
|
||||||
|
| Media type | `application/json` |
|
||||||
|
| Identity policy | `dnd.locations.identity.v1` |
|
||||||
|
|
||||||
|
`v1` accepts one strict JSON object with required `locations`; the array may be
|
||||||
|
empty. Location and source-reference objects reject unknown fields. An
|
||||||
|
incompatible artifact shape or identity-policy change uses a new version or
|
||||||
|
policy.
|
||||||
|
|
||||||
|
## Wire shape and identity
|
||||||
|
|
||||||
|
Each location has these required fields:
|
||||||
|
|
||||||
|
| Field | Contract |
|
||||||
|
| --- | --- |
|
||||||
|
| `id` | `location:sha256:` followed by 64 lowercase hexadecimal characters. |
|
||||||
|
| `name` | Non-empty transcript-established display name. |
|
||||||
|
| `source_refs` | One or more transcript evidence ranges that identify the place. |
|
||||||
|
|
||||||
|
A source reference has exactly `source_id`, `start_unit_id`, and `end_unit_id`.
|
||||||
|
The source ID identifies the transcript, unit IDs are positive inclusive unit
|
||||||
|
identifiers, and the start may not follow the end.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"locations": [
|
||||||
|
{
|
||||||
|
"id": "location:sha256:5c1a91f15729df0b8c257093865fdf2452b43c215375e8cf2341aa9c37bb99aa",
|
||||||
|
"name": "Moon Gate",
|
||||||
|
"source_refs": [
|
||||||
|
{"source_id": "session-7", "start_unit_id": 4, "end_unit_id": 5}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The ID is deterministic and scoped to the source document. Notarius normalizes
|
||||||
|
the display name for comparison with Unicode NFKC, supported apostrophe
|
||||||
|
normalization, collapsed whitespace, and case folding. It hashes compact JSON
|
||||||
|
for this array, using the earliest canonical source reference as the anchor:
|
||||||
|
|
||||||
|
```text
|
||||||
|
["dnd.locations.identity.v1", comparison_name, source_id, start_unit_id, end_unit_id]
|
||||||
|
```
|
||||||
|
|
||||||
|
The canonical ID is the lowercase SHA-256 digest of those bytes with the
|
||||||
|
`location:sha256:` prefix. Equal display names are allowed when their evidence
|
||||||
|
anchors differ, so a generic name does not force distinct places to collapse.
|
||||||
|
|
||||||
|
## Scope, reconciliation, and evidence
|
||||||
|
|
||||||
|
Locations are physical or spatial places established by the transcript, such
|
||||||
|
as planes, regions, settlements, districts, buildings, rooms, landmarks,
|
||||||
|
routes, and geographic features. A generic label is permitted only when it
|
||||||
|
identifies a specific place in the transcript. Notarius does not infer an
|
||||||
|
unstated place or add hierarchy, coordinates, descriptions, participants, or
|
||||||
|
ownership.
|
||||||
|
|
||||||
|
Normalization first applies deterministic display, evidence, and ID rules. It
|
||||||
|
then may use a bounded LLM-assisted proposal to reconcile semantically duplicate
|
||||||
|
records. The proposal is validated and applied conservatively; invalid or
|
||||||
|
unusable proposals retain the deterministic result with retry or fallback
|
||||||
|
diagnostics. The registry's source references establish registry provenance,
|
||||||
|
not evidence for later artifacts.
|
||||||
|
|
||||||
|
## Consumers and publication
|
||||||
|
|
||||||
|
`dnd/location-occurrences` requires one approved location registry through its
|
||||||
|
`locations` reference slot. Its prompt receives an ordered source-free `{id,
|
||||||
|
name}` projection and must not treat registry references as occurrence
|
||||||
|
evidence. See the [location-occurrence artifact](dnd-location-occurrence-artifacts.md)
|
||||||
|
for that contract, [Configuration](../config.md#references-and-ordered-handoffs)
|
||||||
|
for binding rules, and the [JSON output contract](json-output.md) for
|
||||||
|
publication.
|
||||||
84
docs/integrations/dnd-location-occurrence-artifacts.md
Normal file
84
docs/integrations/dnd-location-occurrence-artifacts.md
Normal file
@@ -0,0 +1,84 @@
|
|||||||
|
# D&D Location-Occurrence Artifact
|
||||||
|
|
||||||
|
This contract defines the durable occurrence list produced by
|
||||||
|
`dnd/location-occurrences`. It records source-grounded ways the party relates
|
||||||
|
to locations in a required normalized location registry; it does not extend
|
||||||
|
that registry or infer a place absent from it.
|
||||||
|
|
||||||
|
## Identity and compatibility
|
||||||
|
|
||||||
|
| Property | Value |
|
||||||
|
| --- | --- |
|
||||||
|
| Artifact kind | `dnd/location-occurrence-list` |
|
||||||
|
| Schema ID | `notarius.dnd.location_occurrences` |
|
||||||
|
| Schema name | `notarius_dnd_location_occurrences_v1` |
|
||||||
|
| Schema version | `v1` |
|
||||||
|
| Media type | `application/json` |
|
||||||
|
|
||||||
|
`v1` accepts one strict JSON object with required `occurrences`; the array may
|
||||||
|
be empty. Occurrence and source-reference objects reject unknown fields. An
|
||||||
|
incompatible shape change requires a new schema version.
|
||||||
|
|
||||||
|
## Wire shape
|
||||||
|
|
||||||
|
Each occurrence has these required fields:
|
||||||
|
|
||||||
|
| Field | Contract |
|
||||||
|
| --- | --- |
|
||||||
|
| `location_id` | Exact ID from the required normalized [location registry](dnd-location-artifacts.md). |
|
||||||
|
| `name` | Exact canonical display name for `location_id` in that registry. |
|
||||||
|
| `kind` | One of `visited`, `planned`, `recalled`, or `mentioned`. |
|
||||||
|
| `source_refs` | One or more current-transcript evidence ranges for this occurrence. |
|
||||||
|
|
||||||
|
A source reference has exactly `source_id`, `start_unit_id`, and `end_unit_id`.
|
||||||
|
It identifies an inclusive range in the current transcript; unit IDs are
|
||||||
|
positive and the start may not follow the end.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"occurrences": [
|
||||||
|
{
|
||||||
|
"location_id": "location:sha256:5c1a91f15729df0b8c257093865fdf2452b43c215375e8cf2341aa9c37bb99aa",
|
||||||
|
"name": "Moon Gate",
|
||||||
|
"kind": "visited",
|
||||||
|
"source_refs": [
|
||||||
|
{"source_id": "session-7", "start_unit_id": 12, "end_unit_id": 13}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Occurrence categories
|
||||||
|
|
||||||
|
| Kind | Meaning |
|
||||||
|
| --- | --- |
|
||||||
|
| `visited` | The transcript establishes physical party presence, including arrival, continuing presence, or departure. |
|
||||||
|
| `planned` | The party explicitly proposes, intends, or agrees to future travel; speculation alone is not enough. |
|
||||||
|
| `recalled` | The transcript explicitly recounts prior party presence before the current live events. |
|
||||||
|
| `mentioned` | The location is explicit but no stronger category applies, including lore, directions, third-party activity, non-actionable speculation, a mere hypothetical reference, or out-of-character discussion. |
|
||||||
|
|
||||||
|
For overlapping evidence, precedence is `visited`, then `planned`, then
|
||||||
|
`recalled`, then `mentioned`. For example, “What if we went to Moon Gate?” is
|
||||||
|
eligible as `mentioned` when its narrow evidence explicitly references that
|
||||||
|
registry location, but it is not `planned` without an actual proposal,
|
||||||
|
intention, or agreement to travel. Inferred, unstated, uncertain, and
|
||||||
|
unsupported places or occurrences are omitted. Normalization
|
||||||
|
canonicalizes the registry name, orders and deduplicates source references, and
|
||||||
|
orders occurrences by source chronology, location ID, name, kind, and reference
|
||||||
|
sequence. It collapses only exact duplicates with the same ID, kind, and
|
||||||
|
complete canonical evidence sequence.
|
||||||
|
|
||||||
|
## Required grounding and evidence
|
||||||
|
|
||||||
|
Both extraction and normalization require exactly one `locations` reference of
|
||||||
|
kind `dnd/location-list`, media type `application/json`, and at most 1 MiB. The
|
||||||
|
registry provides identity grounding only: unknown IDs and mismatched ID/name
|
||||||
|
pairs are rejected rather than guessed or reassigned. The current transcript is
|
||||||
|
the only evidence source for an occurrence; registry evidence and provenance
|
||||||
|
never become occurrence evidence.
|
||||||
|
|
||||||
|
See [Configuration](../config.md#d-d-reference-slots) for the selectable slot
|
||||||
|
and generated-handoff compatibility, [D&D module internals](../internal/dnd.md)
|
||||||
|
for implementation behavior, and the [JSON output contract](json-output.md)
|
||||||
|
for publication.
|
||||||
@@ -55,15 +55,26 @@ with the same canonical identity, retains their earliest position, and merges
|
|||||||
their canonicalized evidence; it does not add aliases, roles, descriptions, or
|
their canonicalized evidence; it does not add aliases, roles, descriptions, or
|
||||||
relationship fields.
|
relationship fields.
|
||||||
|
|
||||||
|
When evidence supports a semantically duplicate group, the canonical display
|
||||||
|
name is one of that group's supplied candidates. A complete, stable proper name
|
||||||
|
is preferred over an abbreviation. An unadorned proper name is preferred over
|
||||||
|
the same name plus a contextual class, role, title, or relationship descriptor
|
||||||
|
unless the transcript establishes that descriptor as part of the person's
|
||||||
|
name. A longer candidate is not preferred solely because it includes such a
|
||||||
|
descriptor.
|
||||||
|
|
||||||
## Scope and consumers
|
## Scope and consumers
|
||||||
|
|
||||||
Only individually identifiable NPC names with transcript evidence belong in
|
Only individually identifiable NPC names with transcript evidence belong in
|
||||||
this artifact. Groups, generic roles, invented labels, and descriptive
|
this artifact. Groups, generic roles, invented labels, and descriptive
|
||||||
enrichment are excluded. Its source references prove registry provenance; they
|
enrichment are excluded. Its source references prove registry provenance; they
|
||||||
do not become evidence for a spell, interaction, or combat occurrence.
|
do not become evidence for a spell, interaction, combat, or enemy-event
|
||||||
|
occurrence.
|
||||||
|
|
||||||
This registry can ground actor or caster names in the [spell](dnd-spell-artifacts.md)
|
This registry can ground actor or caster names in the [spell](dnd-spell-artifacts.md)
|
||||||
and [combat-turn](dnd-combat-turn-artifacts.md) artifacts. It is required to
|
and [combat-turn](dnd-combat-turn-artifacts.md) artifacts. It is required to
|
||||||
resolve the canonical `name` in an [NPC interaction](dnd-npc-interaction-artifacts.md).
|
resolve the canonical `name` in an [NPC interaction](dnd-npc-interaction-artifacts.md).
|
||||||
|
The [enemy-event artifact](dnd-enemy-event-artifacts.md) also uses it only for
|
||||||
|
subject grounding and canonical display names.
|
||||||
The [JSON output contract](json-output.md) defines publication, and
|
The [JSON output contract](json-output.md) defines publication, and
|
||||||
[D&D module internals](../internal/dnd.md) owns pipeline mechanics.
|
[D&D module internals](../internal/dnd.md) owns pipeline mechanics.
|
||||||
|
|||||||
@@ -74,5 +74,8 @@ Only entries with the same canonical name, kind, and complete valid evidence
|
|||||||
sequence are collapsed; distinct categories or evidence remain separate.
|
sequence are collapsed; distinct categories or evidence remain separate.
|
||||||
|
|
||||||
See the [combat-turn artifact](dnd-combat-turn-artifacts.md) for combat-action
|
See the [combat-turn artifact](dnd-combat-turn-artifacts.md) for combat-action
|
||||||
occurrences and the [JSON output contract](json-output.md) for publication.
|
occurrences. The [enemy-event artifact](dnd-enemy-event-artifacts.md) consumes
|
||||||
Pipeline mechanics are described in [D&D module internals](../internal/dnd.md).
|
only `combat_opponent` interactions as grounding; they never establish an enemy
|
||||||
|
event or outcome. The [JSON output contract](json-output.md) defines
|
||||||
|
publication. Pipeline mechanics are described in
|
||||||
|
[D&D module internals](../internal/dnd.md).
|
||||||
|
|||||||
@@ -62,8 +62,9 @@ durable fields, or the same source range with different kind, title, or
|
|||||||
summary, is invalid. It does not merge adjacent ranges, alter prose, or infer
|
summary, is invalid. It does not merge adjacent ranges, alter prose, or infer
|
||||||
missing scenes.
|
missing scenes.
|
||||||
|
|
||||||
The [combat-turn artifact](dnd-combat-turn-artifacts.md) uses an exact matching
|
The [combat-turn artifact](dnd-combat-turn-artifacts.md) and
|
||||||
|
[enemy-event artifact](dnd-enemy-event-artifacts.md) use an exact matching
|
||||||
`combat` scene only as eligibility control; scene title, summary, and source
|
`combat` scene only as eligibility control; scene title, summary, and source
|
||||||
reference never become combat evidence. Publication is defined by the
|
reference never become their evidence. Publication is defined by the
|
||||||
[JSON output contract](json-output.md); implementation details live in
|
[JSON output contract](json-output.md); implementation details live in
|
||||||
[D&D module internals](../internal/dnd.md).
|
[D&D module internals](../internal/dnd.md).
|
||||||
|
|||||||
@@ -74,8 +74,11 @@ than infer a lane schema from its name. The current D&D payload contracts are
|
|||||||
[spells](dnd-spell-artifacts.md), [NPCs](dnd-npc-artifacts.md),
|
[spells](dnd-spell-artifacts.md), [NPCs](dnd-npc-artifacts.md),
|
||||||
[NPC interactions](dnd-npc-interaction-artifacts.md),
|
[NPC interactions](dnd-npc-interaction-artifacts.md),
|
||||||
[combat turns](dnd-combat-turn-artifacts.md),
|
[combat turns](dnd-combat-turn-artifacts.md),
|
||||||
[item events](dnd-item-event-artifacts.md), and
|
[item events](dnd-item-event-artifacts.md),
|
||||||
[scene descriptions](dnd-scene-description-artifacts.md).
|
[scene descriptions](dnd-scene-description-artifacts.md), and
|
||||||
|
[enemy events](dnd-enemy-event-artifacts.md),
|
||||||
|
[locations](dnd-location-artifacts.md), and
|
||||||
|
[location occurrences](dnd-location-occurrence-artifacts.md).
|
||||||
|
|
||||||
## `manifest.json`
|
## `manifest.json`
|
||||||
|
|
||||||
@@ -98,6 +101,11 @@ summarize results without embedding lane payload bytes. A chunk-plan summary is
|
|||||||
provenance for the plan used by this run; cache records, debug artifacts, and
|
provenance for the plan used by this run; cache records, debug artifacts, and
|
||||||
other operational state are not published as bundle files.
|
other operational state are not published as bundle files.
|
||||||
|
|
||||||
|
When present, `metadata.session_id` is the effective non-secret routing
|
||||||
|
correlation identifier used for the run. It can be visible to providers and is
|
||||||
|
not a substitute for a cache or checkpoint identity. Its generation and
|
||||||
|
override behavior are defined by the [CLI reference](../cli.md#run).
|
||||||
|
|
||||||
Each `llm_profiles` entry identifies effective, non-secret LLM execution
|
Each `llm_profiles` entry identifies effective, non-secret LLM execution
|
||||||
provenance:
|
provenance:
|
||||||
|
|
||||||
|
|||||||
@@ -48,11 +48,13 @@ adapter boundary. It also retains responsibility for pipeline retries,
|
|||||||
scheduling, debug persistence, redaction, profile provenance, and conversion
|
scheduling, debug persistence, redaction, profile provenance, and conversion
|
||||||
from private model responses into durable domain artifacts.
|
from private model responses into durable domain artifacts.
|
||||||
|
|
||||||
Notarius sends its trimmed run session through PromptKit's direct session
|
Notarius sends one stable effective session through PromptKit's direct session
|
||||||
field, which is authoritative for provider session behavior. It also retains
|
field, which is authoritative for provider session behavior. It also retains
|
||||||
the same value as the `session_id` prompt variable for maintained prompt
|
the same value as the `session_id` prompt variable for maintained prompt
|
||||||
compatibility. Session IDs are stable, non-secret correlation identifiers and
|
compatibility. The generated identifier is 76 ASCII characters, within
|
||||||
may be exposed to providers and provider observability.
|
PromptKit v0.5.0's 256-code-point session limit. Session IDs are non-secret
|
||||||
|
correlation identifiers and may be exposed to providers and provider
|
||||||
|
observability. The CLI contract owns generation and override behavior.
|
||||||
|
|
||||||
Notarius records PromptKit's selected backend ID and effective reasoning
|
Notarius records PromptKit's selected backend ID and effective reasoning
|
||||||
setting as optional run-manifest provenance. Endpoint-only profiles have no
|
setting as optional run-manifest provenance. Endpoint-only profiles have no
|
||||||
|
|||||||
@@ -89,9 +89,11 @@ handoff:
|
|||||||
profiles;
|
profiles;
|
||||||
4. materialize external or generated references and record redacted invocation
|
4. materialize external or generated references and record redacted invocation
|
||||||
and resolution provenance when debug capture is enabled;
|
and resolution provenance when debug capture is enabled;
|
||||||
5. construct registries, the scheduled LLM client, prepared modules, and the
|
5. construct registries, the scheduled LLM client, and prepared modules;
|
||||||
requested cache/checkpoint collaborators;
|
6. read the source input once, resolve its effective session from the explicit
|
||||||
6. read the source input and invoke the framework runner; and
|
override or resolved input module and raw bytes, then construct requested
|
||||||
|
checkpoint collaborators and invoke the framework runner with that same
|
||||||
|
value; and
|
||||||
7. write the runner's logical output files only after a successful run, then
|
7. write the runner's logical output files only after a successful run, then
|
||||||
complete the command report and user-facing result.
|
complete the command report and user-facing result.
|
||||||
|
|
||||||
@@ -102,6 +104,13 @@ final command result. Detailed state lifecycle, resume handling, and physical
|
|||||||
path confinement are maintained in [Run State Internals](state.md) and
|
path confinement are maintained in [Run State Internals](state.md) and
|
||||||
[Operations](../operations.md).
|
[Operations](../operations.md).
|
||||||
|
|
||||||
|
The CLI owns the versioned generated-session policy and resolves the sole
|
||||||
|
effective value before checkpoint construction. It records that value in the
|
||||||
|
final debug invocation summary when capture is enabled and passes it unchanged
|
||||||
|
to checkpoint identity and `pipeline.RunInput`. The public flag and stability
|
||||||
|
contract are defined by the [CLI reference](../cli.md#run); framework and LLM
|
||||||
|
packages only transport the supplied value.
|
||||||
|
|
||||||
For `run --json`, the CLI constructs and encodes its private run-result receipt
|
For `run --json`, the CLI constructs and encodes its private run-result receipt
|
||||||
after a successful runner result is available, before it publishes logical
|
after a successful runner result is available, before it publishes logical
|
||||||
output files. It writes the prepared receipt to standard output only after
|
output files. It writes the prepared receipt to standard output only after
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ selectable keys, bindings, reference syntax, and default validator chains.
|
|||||||
|
|
||||||
## Durable Artifact Contracts
|
## Durable Artifact Contracts
|
||||||
|
|
||||||
The six lanes have separate durable wire contracts. This guide deliberately
|
The nine lanes have separate durable wire contracts. This guide deliberately
|
||||||
does not repeat their JSON shapes or schemas.
|
does not repeat their JSON shapes or schemas.
|
||||||
|
|
||||||
| Lane | Durable contract |
|
| Lane | Durable contract |
|
||||||
@@ -18,6 +18,9 @@ does not repeat their JSON shapes or schemas.
|
|||||||
| Item events | [item-event artifacts](../integrations/dnd-item-event-artifacts.md) |
|
| Item events | [item-event artifacts](../integrations/dnd-item-event-artifacts.md) |
|
||||||
| NPC interactions | [NPC-interaction artifacts](../integrations/dnd-npc-interaction-artifacts.md) |
|
| NPC interactions | [NPC-interaction artifacts](../integrations/dnd-npc-interaction-artifacts.md) |
|
||||||
| Scene descriptions | [scene-description artifacts](../integrations/dnd-scene-description-artifacts.md) |
|
| Scene descriptions | [scene-description artifacts](../integrations/dnd-scene-description-artifacts.md) |
|
||||||
|
| Enemy events | [enemy-event artifacts](../integrations/dnd-enemy-event-artifacts.md) |
|
||||||
|
| Locations | [location artifacts](../integrations/dnd-location-artifacts.md) |
|
||||||
|
| Location occurrences | [location-occurrence artifacts](../integrations/dnd-location-occurrence-artifacts.md) |
|
||||||
|
|
||||||
## Family Composition
|
## Family Composition
|
||||||
|
|
||||||
@@ -25,9 +28,9 @@ The D&D registrar registers the family’s artifact codecs, extractors, typed
|
|||||||
append-order mergers, normalizers, validators, prompt assets, fallback LLM
|
append-order mergers, normalizers, validators, prompt assets, fallback LLM
|
||||||
profile asset, and default validator chains. Each extractor and normalizer has
|
profile asset, and default validator chains. Each extractor and normalizer has
|
||||||
a stable module spec, explicit execution class, strict option decoding, and a
|
a stable module spec, explicit execution class, strict option decoding, and a
|
||||||
typed builder. Scene chunking, every extractor, and NPC normalization are
|
typed builder. Scene chunking, every extractor, NPC normalization, and location
|
||||||
registered as `llm_backed`; the remaining current D&D mergers and normalizers
|
normalization are registered as `llm_backed`; the remaining current D&D mergers
|
||||||
are `deterministic`. The metadata is available to catalog inspection and
|
and normalizers are `deterministic`. The metadata is available to catalog inspection and
|
||||||
resolved-pipeline debug data and determines which selected bindings inherit the
|
resolved-pipeline debug data and determines which selected bindings inherit the
|
||||||
pipeline profile. Configuration remains the canonical owner of the exact keys,
|
pipeline profile. Configuration remains the canonical owner of the exact keys,
|
||||||
profile precedence, and validator order.
|
profile precedence, and validator order.
|
||||||
@@ -41,10 +44,10 @@ the contracts above define durable data.
|
|||||||
## Prompt Construction
|
## Prompt Construction
|
||||||
|
|
||||||
D&D extractors assemble prompts from an ordered manifest of shared and
|
D&D extractors assemble prompts from an ordered manifest of shared and
|
||||||
module-owned assets. Reuse the shared D&D system, evidence, identity,
|
module-owned assets. The location extractor and occurrence extractor reuse the
|
||||||
reference, and transcript assets instead of copying their text into individual
|
shared D&D system, evidence, identity, reference, and transcript assets instead
|
||||||
modules. A manifest’s declared sequence, including cache-control placement, is
|
of copying their text into individual modules. A manifest’s declared sequence,
|
||||||
part of the prompt behavior.
|
including cache-control placement, is part of the prompt behavior.
|
||||||
|
|
||||||
Every maintained D&D LLM prompt selects `dnd-extraction` as its default
|
Every maintained D&D LLM prompt selects `dnd-extraction` as its default
|
||||||
profile. The D&D registrar embeds that fallback profile with the maintained
|
profile. The D&D registrar embeds that fallback profile with the maintained
|
||||||
@@ -71,7 +74,9 @@ remains stable.
|
|||||||
The other D&D LLM prompts intentionally follow different patterns. Scene
|
The other D&D LLM prompts intentionally follow different patterns. Scene
|
||||||
chunking has no sibling extraction lane with which to share its full transcript,
|
chunking has no sibling extraction lane with which to share its full transcript,
|
||||||
so it renders campaign references before its task and instructions, then places
|
so it renders campaign references before its task and instructions, then places
|
||||||
the cacheable full transcript last. NPC normalization keeps its task and
|
the cacheable full transcript last. NPC and location normalization share the
|
||||||
|
entity-reconciliation response schema and safety boundary while retaining their
|
||||||
|
own task and identity rules. NPC normalization keeps its task and
|
||||||
cacheable instructions before the candidate collection, followed by the
|
cacheable instructions before the candidate collection, followed by the
|
||||||
cacheable transcript windows: candidates must be available before their
|
cacheable transcript windows: candidates must be available before their
|
||||||
supporting evidence is evaluated, and those windows are not a cross-lane
|
supporting evidence is evaluated, and those windows are not a cross-lane
|
||||||
@@ -101,14 +106,20 @@ relatedness validators report advisory evidence concerns. The configured order
|
|||||||
is documented in
|
is documented in
|
||||||
[Configuration](../config.md#production-validator-keys-and-default-chains).
|
[Configuration](../config.md#production-validator-keys-and-default-chains).
|
||||||
|
|
||||||
|
Enemy-event extraction additionally rejects a second `engaged` observation for
|
||||||
|
the same comparison identity within one scene-scoped result. Normalization may
|
||||||
|
combine results from distinct scenes, so it intentionally does not apply that
|
||||||
|
rule. Configuration owns the exact validator key and chain position.
|
||||||
|
|
||||||
Normalizers are deterministic for spells, combat turns, item events, NPC
|
Normalizers are deterministic for spells, combat turns, item events, NPC
|
||||||
interactions, and scene descriptions. They canonicalize display values and
|
interactions, scene descriptions, enemy events, and location occurrences. They canonicalize display
|
||||||
evidence, use source-document order for stable output, and issue bounded
|
values and evidence, use source-document order for stable output, and issue
|
||||||
warnings for changes or collapsed duplicates. The NPC normalizer is the
|
bounded warnings for changes or collapsed duplicates. The NPC and location
|
||||||
intentional exception: it first produces a deterministic candidate set, then
|
normalizers are intentional exceptions: each first produces a deterministic
|
||||||
uses a bounded structured-LLM proposal to reconcile identity groups. Invalid
|
candidate set, then may use a bounded structured-LLM proposal to reconcile
|
||||||
or unusable proposals retain the deterministic result and surface retry or
|
identity groups. Invalid or unusable proposals retain the deterministic result
|
||||||
fallback diagnostics; the model does not directly replace durable records.
|
and surface retry or fallback diagnostics; the model does not directly replace
|
||||||
|
durable records.
|
||||||
|
|
||||||
## Generated References And Grounding
|
## Generated References And Grounding
|
||||||
|
|
||||||
@@ -122,7 +133,13 @@ NPC registries are names-only grounding projections: they may canonicalize
|
|||||||
actors for spells and combat turns and are required for NPC interactions, but
|
actors for spells and combat turns and are required for NPC interactions, but
|
||||||
they do not supply evidence. Scene-description registries are eligibility-only
|
they do not supply evidence. Scene-description registries are eligibility-only
|
||||||
projections: they retain the current chunk’s classification data, not scene
|
projections: they retain the current chunk’s classification data, not scene
|
||||||
prose or evidence, and exist to route combat extraction.
|
prose or evidence, and exist to route combat extraction. Enemy-event extraction
|
||||||
|
also projects combat turns to `actor` and `turn_kind` and filters NPC
|
||||||
|
interactions to `combat_opponent` names and kinds. Location registries project
|
||||||
|
ordered `{id, name}` pairs to location-occurrence extraction and normalization;
|
||||||
|
exact ID/name matching keeps same-name locations distinguishable. These compact
|
||||||
|
projections, like NPC grounding, are source-free guidance and never event
|
||||||
|
evidence.
|
||||||
|
|
||||||
## Lane-Specific Rules
|
## Lane-Specific Rules
|
||||||
|
|
||||||
@@ -137,11 +154,17 @@ shared helper changes.
|
|||||||
| Item events | Uses campaign context for disambiguation but has no NPC-registry or scene-description dependency. |
|
| Item events | Uses campaign context for disambiguation but has no NPC-registry or scene-description dependency. |
|
||||||
| NPC interactions | Requires the normalized NPC registry at extraction and normalization, using it for canonical actor grounding only. |
|
| NPC interactions | Requires the normalized NPC registry at extraction and normalization, using it for canonical actor grounding only. |
|
||||||
| Scene descriptions | Produces the classifications consumed by combat routing; it does not consume an NPC registry or provide evidence for combat artifacts. |
|
| Scene descriptions | Produces the classifications consumed by combat routing; it does not consume an NPC registry or provide evidence for combat artifacts. |
|
||||||
|
| Enemy events | Requires NPC, scene-description, combat-turn, and NPC-interaction artifacts. It calls the LLM only for an exact `combat` classification, records ordered observations rather than terminal state, and normalizes recognized names through the NPC registry while preserving grounded collective labels. |
|
||||||
|
| Locations | Produces a source-anchored, session-scoped registry. Its LLM-assisted reconciliation is proposal-only and never collapses same-name places without validated identity and evidence rules. |
|
||||||
|
| Location occurrences | Requires the normalized location registry for both extraction and normalization. Its [durable occurrence categories](../integrations/dnd-location-occurrence-artifacts.md#occurrence-categories) distinguish explicit speculation from unsupported inference; the deterministic normalizer enforces exact registry grounding and never turns registry provenance into occurrence evidence. |
|
||||||
|
|
||||||
The combat and scene-description contracts describe their exact handoff and
|
The combat and scene-description contracts describe their exact handoff and
|
||||||
empty-result behavior in more detail:
|
empty-result behavior in more detail:
|
||||||
[combat turns](../integrations/dnd-combat-turn-artifacts.md) and
|
[combat turns](../integrations/dnd-combat-turn-artifacts.md) and
|
||||||
[scene descriptions](../integrations/dnd-scene-description-artifacts.md).
|
[scene descriptions](../integrations/dnd-scene-description-artifacts.md).
|
||||||
|
The [enemy-event contract](../integrations/dnd-enemy-event-artifacts.md)
|
||||||
|
defines its durable semantics; [Configuration](../config.md) owns its
|
||||||
|
selectable bindings and validation chains.
|
||||||
|
|
||||||
## Focused Verification
|
## Focused Verification
|
||||||
|
|
||||||
|
|||||||
@@ -26,9 +26,10 @@ durable schemas. Those responsibilities remain with the module and its
|
|||||||
|
|
||||||
`PromptKitClient` validates the request target and prompt identity, maps each
|
`PromptKitClient` validates the request target and prompt identity, maps each
|
||||||
named material to a PromptKit inline artifact while preserving its origin URI,
|
named material to a PromptKit inline artifact while preserving its origin URI,
|
||||||
maps the trimmed request session to PromptKit's direct per-run session field,
|
passes the supplied request session through to PromptKit's direct per-run
|
||||||
retains the same value as the `session_id` prompt variable for maintained
|
session field, retains the same value as the `session_id` prompt variable for
|
||||||
prompt compatibility, and forwards profile selection. It then creates one
|
maintained prompt compatibility, and forwards profile selection. It does not
|
||||||
|
derive or replace session values; the CLI owns that policy. It then creates one
|
||||||
frozen prepared execution, captures its caller-owned credential-redacted
|
frozen prepared execution, captures its caller-owned credential-redacted
|
||||||
details for debug material, and executes that exact snapshot through
|
details for debug material, and executes that exact snapshot through
|
||||||
PromptKit's prepared-execution boundary. The direct field
|
PromptKit's prepared-execution boundary. The direct field
|
||||||
|
|||||||
@@ -10,11 +10,11 @@ own durable output shapes. Concrete production extensions are covered by
|
|||||||
## Boundary
|
## Boundary
|
||||||
|
|
||||||
The pipeline framework accepts a resolved composition, registries, shared
|
The pipeline framework accepts a resolved composition, registries, shared
|
||||||
dependencies, input bytes, and state/debug collaborators. It returns logical
|
dependencies, input bytes, a supplied prompt session, and state/debug
|
||||||
output files, normalized artifacts, recorded rejections and warnings, manifest
|
collaborators. It returns logical output files, normalized artifacts, recorded
|
||||||
provenance, and checkpoint decisions. The CLI owns process arguments,
|
rejections and warnings, manifest provenance, and checkpoint decisions. The
|
||||||
configuration discovery, physical roots, and placement of returned output
|
CLI owns process arguments, configuration discovery, session resolution,
|
||||||
files.
|
physical roots, and placement of returned output files.
|
||||||
|
|
||||||
The framework has one fixed shape:
|
The framework has one fixed shape:
|
||||||
|
|
||||||
@@ -84,6 +84,10 @@ incompatible producer prevents the consumer step from starting.
|
|||||||
|
|
||||||
The runner validates its input, installs no-op state collaborators when none
|
The runner validates its input, installs no-op state collaborators when none
|
||||||
were supplied, and serially performs source parsing and chunk-plan selection.
|
were supplied, and serially performs source parsing and chunk-plan selection.
|
||||||
|
It transports the supplied session unchanged to prompt-facing operations and
|
||||||
|
run-manifest metadata; it neither derives a session nor substitutes a parsed
|
||||||
|
source document identifier. The public session contract is owned by the
|
||||||
|
[CLI reference](../cli.md#run).
|
||||||
An accepted plan is materialized into source-addressed chunks and passes the
|
An accepted plan is materialized into source-addressed chunks and passes the
|
||||||
configured chunk validators before any lane runs. A chunk rejection is a
|
configured chunk validators before any lane runs. A chunk rejection is a
|
||||||
recorded pipeline outcome: lanes do not start, but the output stage can encode
|
recorded pipeline outcome: lanes do not start, but the output stage can encode
|
||||||
|
|||||||
@@ -266,16 +266,18 @@ transport-wide cap. Notarius does not add another timeout around PromptKit.
|
|||||||
The pinned upstream boundary and profile-format links are in
|
The pinned upstream boundary and profile-format links are in
|
||||||
[PromptKit Integration](integrations/pkg-promptkit.md).
|
[PromptKit Integration](integrations/pkg-promptkit.md).
|
||||||
|
|
||||||
Concurrency has two independent layers. Notarius **total_llm** is the
|
Concurrency has two independent layers. Notarius **total_llm** defaults to 16
|
||||||
application-wide provider-call limit shared by all backends, modules, retries,
|
and is the application-wide provider-call limit shared by all backends,
|
||||||
and validators. PromptKit may impose a narrower admission limit for the
|
modules, retries, and validators. PromptKit may impose a narrower admission
|
||||||
selected backend. The effective active-generation bound is the intersection of
|
limit for the selected backend. The effective active-generation bound is the
|
||||||
both limits and can therefore be lower than **total_llm**. Built-in OpenRouter
|
intersection of the Notarius limit, any PromptKit backend limit, and work made
|
||||||
profiles use PromptKit's upstream backend limit; endpoint-only profiles have no
|
available by the pipeline. Built-in OpenRouter profiles use PromptKit's
|
||||||
PromptKit backend limit and remain bounded by Notarius. For the configured
|
upstream backend limit; endpoint-only profiles have no PromptKit backend limit
|
||||||
local backend, a zero **concurrency_limit** leaves only the Notarius scheduler
|
and remain bounded by Notarius. For the configured local backend, a zero
|
||||||
as a call limit. A positive value makes the effective active local-generation
|
**concurrency_limit** leaves only the Notarius scheduler as a call limit. A
|
||||||
bound the smaller of **total_llm** and that local limit.
|
positive value makes the effective active local-generation bound the smaller
|
||||||
|
of **total_llm** and that local limit, so a local limit of four permits no more
|
||||||
|
than four active local generations.
|
||||||
|
|
||||||
For a positive local limit, PromptKit owns its default waiting capacity and
|
For a positive local limit, PromptKit owns its default waiting capacity and
|
||||||
admission behavior. When a PromptKit backend has admitted all active and queued
|
admission behavior. When a PromptKit backend has admitted all active and queued
|
||||||
@@ -289,3 +291,11 @@ under [PromptKit profiles](config.md#promptkit-profiles) and
|
|||||||
limits and actual provider-call limits are independent. Notarius writes local
|
limits and actual provider-call limits are independent. Notarius writes local
|
||||||
filesystem state only; remote storage, archival, and retention automation are
|
filesystem state only; remote storage, archival, and retention automation are
|
||||||
outside the implemented CLI.
|
outside the implemented CLI.
|
||||||
|
|
||||||
|
Every run has an effective prompt session used for provider routing and run
|
||||||
|
provenance. The generated default is stable for the same input module and raw
|
||||||
|
input bytes; use [**--session-id**](cli.md#run) only when intentionally grouping
|
||||||
|
different invocations. Both generated and explicit values can be visible to
|
||||||
|
providers, manifests, checkpoints, and requested debug bundles. Do not put
|
||||||
|
credentials or other secrets in an explicit session identifier; command-line
|
||||||
|
values are not a credential mechanism.
|
||||||
|
|||||||
@@ -7,37 +7,6 @@ not as committed release dates.
|
|||||||
|
|
||||||
## Near-Term D&D Pipeline
|
## Near-Term D&D Pipeline
|
||||||
|
|
||||||
### Combat Enemy Ledger
|
|
||||||
|
|
||||||
- Add a D&D artifact that identifies enemies faced during combat and supports
|
|
||||||
an end-of-session encounter ledger.
|
|
||||||
- Track each enemy's observed state using a small controlled vocabulary such as
|
|
||||||
`active`, `killed`, `fled`, `captured`, or `incapacitated`, while preserving
|
|
||||||
an explicit unresolved state when the transcript does not establish an
|
|
||||||
outcome.
|
|
||||||
- Preserve the evidence for enemy participation and state changes rather than
|
|
||||||
inferring a terminal outcome from combat ending or an enemy disappearing
|
|
||||||
from the conversation.
|
|
||||||
- Define how repeated mentions, groups of unnamed enemies, summoned or allied
|
|
||||||
creatures, and the same enemy appearing in multiple combats affect identity
|
|
||||||
and ledger entries.
|
|
||||||
- Evaluate whether the ledger should be extracted directly, derived from
|
|
||||||
combat-turn artifacts, or use a sequential pipeline that consumes combat
|
|
||||||
turns and the normalized NPC registry as grounding references.
|
|
||||||
|
|
||||||
### Location Extraction
|
|
||||||
|
|
||||||
- Add a D&D artifact for locations visited by the party or otherwise mentioned
|
|
||||||
in the transcript.
|
|
||||||
- Distinguish observed visits from references, plans, recalled places, and
|
|
||||||
uncertain or inferred locations so a mention alone is not reported as a
|
|
||||||
visit.
|
|
||||||
- Preserve transcript evidence for each visit or mention and reconcile aliases,
|
|
||||||
nested places, and repeated appearances without collapsing distinct
|
|
||||||
locations that share a generic name.
|
|
||||||
- Define how the location artifact should ground later narrative reports and
|
|
||||||
whether future event artifacts should retain canonical location identities.
|
|
||||||
|
|
||||||
### Evaluate Spell Extraction And Normalization
|
### Evaluate Spell Extraction And Normalization
|
||||||
|
|
||||||
- Evaluate ordinary extraction retries and the completed normalization path
|
- Evaluate ordinary extraction retries and the completed normalization path
|
||||||
@@ -53,71 +22,6 @@ not as committed release dates.
|
|||||||
spell, combat, interaction, and scene-description lanes after real-world use.
|
spell, combat, interaction, and scene-description lanes after real-world use.
|
||||||
Add more complex chunking only in response to demonstrated failures.
|
Add more complex chunking only in response to demonstrated failures.
|
||||||
|
|
||||||
## Cross-Cutting LLM Runtime
|
|
||||||
|
|
||||||
### Deterministic Prompt Session Identity
|
|
||||||
|
|
||||||
- Replace the source-document-ID default for prompt sessions with one
|
|
||||||
predictable, procedurally generated session ID for the complete
|
|
||||||
source-processing workload.
|
|
||||||
- Preserve an explicit non-empty `--session-id` as the highest-precedence
|
|
||||||
override. Otherwise, derive the default only from the effective input module
|
|
||||||
identity and the exact raw input bytes.
|
|
||||||
- Use a versioned, bounded representation such as
|
|
||||||
`notarius:v1:<sha256(input-module + NUL + raw-input)>`. The exact encoding
|
|
||||||
must fit PromptKit's session length contract and must not embed source
|
|
||||||
content.
|
|
||||||
- Keep the derived session stable across runs, pipelines, selected lanes,
|
|
||||||
ordered steps, retries, resume, recomputation, LLM profiles, reasoning
|
|
||||||
overrides, and output, debug, or cache settings.
|
|
||||||
- Do not include file-backed references, generated references, reference
|
|
||||||
contents, or the composition of a reference bundle in session derivation.
|
|
||||||
References may change between prompt calls within one pipeline without
|
|
||||||
changing routing affinity.
|
|
||||||
- Resolve the authoritative session before checkpoint construction and use the
|
|
||||||
same value for checkpoint runtime identity, every prompt-facing module,
|
|
||||||
PromptKit's direct session field, the compatibility `session_id` prompt
|
|
||||||
variable, run-manifest metadata, and debug metadata.
|
|
||||||
- Keep routing identity separate from cache and checkpoint content identity.
|
|
||||||
Exact prompt prefixes, reference contents, model settings, and other
|
|
||||||
generation-affecting inputs must continue to participate in their existing
|
|
||||||
hashes and checkpoint fingerprints even though they do not change the
|
|
||||||
session.
|
|
||||||
- Treat the generated value as a provider-visible, stable pseudonymous
|
|
||||||
correlation identifier. Do not introduce an installation-specific HMAC or
|
|
||||||
secret unless a concrete multi-tenant or privacy requirement justifies
|
|
||||||
sacrificing deterministic identity across installations.
|
|
||||||
|
|
||||||
### Raise The Default Application-Wide LLM Limit
|
|
||||||
|
|
||||||
- Raise the default `concurrency.total_llm` value from 1 to 16 so ordinary
|
|
||||||
single-backend runs can use PromptKit's expected OpenRouter capacity and
|
|
||||||
lower-capacity local backends without an unnecessarily narrower Notarius
|
|
||||||
limit.
|
|
||||||
- Keep the Notarius application-wide scheduler mandatory and require
|
|
||||||
`total_llm` to remain a positive integer. Do not make the default unlimited:
|
|
||||||
endpoint-only profiles, an unrestricted local backend, injected clients, and
|
|
||||||
aggregate work across several backends may have no narrower PromptKit limit.
|
|
||||||
- Continue defaulting `concurrency.stage_workers.extract` to the effective
|
|
||||||
`total_llm`, making its default 16 as part of the same change. Preserve an
|
|
||||||
explicit lower extract-worker setting when an operator wants less queued or
|
|
||||||
concurrent extraction work.
|
|
||||||
- Define effective provider concurrency as the intersection of the Notarius
|
|
||||||
application-wide limit, the selected PromptKit backend limit when present,
|
|
||||||
and the work made available by stage execution. A Notarius limit of 16 does
|
|
||||||
not narrow a backend already limited to 16, while a local backend limited to
|
|
||||||
4 remains bounded at 4.
|
|
||||||
- Treat the default as an application-wide safety ceiling across profiles,
|
|
||||||
backends, modules, retries, and validators. A run that intentionally needs
|
|
||||||
the combined capacity of several backends may configure a higher
|
|
||||||
`total_llm` and an appropriate extract-worker count explicitly.
|
|
||||||
- Retain the existing configuration and environment override surfaces. Update
|
|
||||||
canonical configuration, operations, and internal documentation together
|
|
||||||
when the default changes.
|
|
||||||
- Reconsider decoupling the extract-worker default from `total_llm` only after
|
|
||||||
mixed-backend workloads demonstrate a need for a high global emergency
|
|
||||||
ceiling with a lower default work-production rate.
|
|
||||||
|
|
||||||
## Shared Normalization And Quality Work
|
## Shared Normalization And Quality Work
|
||||||
|
|
||||||
### Generic LLM-Assisted Deduplication
|
### Generic LLM-Assisted Deduplication
|
||||||
|
|||||||
@@ -1,586 +0,0 @@
|
|||||||
# PromptKit v0.5 Implementation Plan
|
|
||||||
|
|
||||||
## Objective
|
|
||||||
|
|
||||||
Implement the target state in
|
|
||||||
[PromptKit v0.5 Integration And LLM Profile Policy](promptkit.md). Each numbered
|
|
||||||
stage is intended to be one implementation prompt for a GPT-5.6-Terra coding
|
|
||||||
agent. Complete stages in order and leave the repository buildable, tested, and
|
|
||||||
internally coherent after every stage.
|
|
||||||
|
|
||||||
Follow [Architecture](../policy/architecture.md),
|
|
||||||
[Testing Policy](../policy/testing.md), and
|
|
||||||
[Documentation Policy](../policy/documentation.md) throughout. Preserve
|
|
||||||
unrelated user changes. Use `apply_patch` for source and documentation edits,
|
|
||||||
run `gofmt` on changed Go files, and add only tests that protect the behaviors
|
|
||||||
and risks assigned to that stage.
|
|
||||||
|
|
||||||
Do not implement the separate deterministic session-ID or default-concurrency
|
|
||||||
roadmap items as part of this plan. Do not perform paid or credentialed LLM
|
|
||||||
calls.
|
|
||||||
|
|
||||||
## Background Summary
|
|
||||||
|
|
||||||
Notarius currently pins PromptKit v0.3.0, calls `Prepare` and then `Run` for one
|
|
||||||
completion, validates profiles through a synthetic prompt, has no application
|
|
||||||
fallback profile source, and accepts LLM profiles only at individual bindings
|
|
||||||
or through the run-wide CLI override. PromptKit v0.5.0 is source-compatible
|
|
||||||
with the current tree; a temporary v0.5.0 module override has already passed
|
|
||||||
`go test ./...`.
|
|
||||||
|
|
||||||
The implementation must nevertheless treat the upstream optional-parameter
|
|
||||||
change as intentional: unset `temperature`, `max_tokens`, and `top_p` remain
|
|
||||||
unset and are omitted from compatible provider requests. Do not restore the old
|
|
||||||
implicit `top_p: 1` default.
|
|
||||||
|
|
||||||
## Stage 1: Upgrade The PromptKit Dependency
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Establish a clean PromptKit v0.5.0 baseline before adopting its new APIs.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
- Update `go.mod` and `go.sum` from PromptKit v0.3.0 to v0.5.0 and run
|
|
||||||
`go mod tidy`.
|
|
||||||
- Change the PromptKit built-in profile-catalog marker in
|
|
||||||
`internal/framework/llm/promptkit_profile_fingerprint.go` to identify
|
|
||||||
v0.5.0. This deliberately invalidates LLM checkpoints tied to the prior
|
|
||||||
catalog identity.
|
|
||||||
- Review PromptKit-facing compile errors or test failures against the v0.4.0
|
|
||||||
and v0.5.0 release guides. Do not adopt prepared execution, inspection, or
|
|
||||||
fallback profiles in this stage.
|
|
||||||
- Replace the existing test assertion for one exact built-in fingerprint hash
|
|
||||||
with durable assertions that the fingerprint is deterministic, non-empty,
|
|
||||||
non-secret, and changes when a semantic profile source changes. Do not add a
|
|
||||||
new version-constant or exact-hash change detector.
|
|
||||||
- Update `docs/integrations/pkg-promptkit.md` to pin and link v0.5.0 and state
|
|
||||||
the implemented dependency-level behavior: unset optional sampling controls
|
|
||||||
are provider defaults. Do not document later stages as implemented.
|
|
||||||
- Update any other canonical text that explicitly claims the dependency is
|
|
||||||
v0.3.0, but defer descriptions of unimplemented v0.5 APIs.
|
|
||||||
|
|
||||||
### Tests And Validation
|
|
||||||
|
|
||||||
- `go test ./internal/framework/llm ./internal/cli`
|
|
||||||
- `go test ./...`
|
|
||||||
- `go vet ./...`
|
|
||||||
- `go build ./cmd/notarius`
|
|
||||||
- `rg -n 'promptkit v0\.3\.0|promptkit@v0\.3\.0|PromptKit v0\.3\.0' .`
|
|
||||||
- `git diff --check`
|
|
||||||
|
|
||||||
### Completion Criteria
|
|
||||||
|
|
||||||
- The repository directly pins v0.5.0 and all default offline checks pass.
|
|
||||||
- The profile-source fingerprint identifies the new upstream catalog without a
|
|
||||||
brittle literal-hash test.
|
|
||||||
- Current documentation no longer identifies v0.3.0 as the supported version.
|
|
||||||
|
|
||||||
## Stage 2: Execute One Frozen Prepared Snapshot
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Make Notarius debug details and generation use one exact PromptKit preparation.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
- Refactor `PromptKitClient.CompleteStructured` to call
|
|
||||||
`PrepareExecution`, immediately defer `Discard`, obtain a caller-owned
|
|
||||||
`Details` value, and execute with `RunPrepared`.
|
|
||||||
- Preserve the existing Notarius request mapping, cancellation precedence,
|
|
||||||
validation classification, raw structured bytes, response decoding,
|
|
||||||
profile recording, usage reporting, and credential redaction.
|
|
||||||
- Ensure every preparation, execution, validation, empty-result, and decode
|
|
||||||
error retains useful Notarius prompt context without exposing prepared handle
|
|
||||||
state or secrets.
|
|
||||||
- Use `errors.As` to obtain `*promptkit.CapacityError` on admission rejection.
|
|
||||||
Preserve `contracts.ErrLLMCapacityExceeded` as the stable classification and
|
|
||||||
add a nonblank backend ID only to safe application-owned diagnostic context.
|
|
||||||
Do not expose `promptkit.CapacityError` outside the LLM adapter.
|
|
||||||
- Update `docs/internal/llm.md` and the implemented-mechanics portion of
|
|
||||||
`docs/integrations/pkg-promptkit.md` to describe the single frozen execution
|
|
||||||
snapshot and structured capacity adaptation.
|
|
||||||
|
|
||||||
### Tests And Validation
|
|
||||||
|
|
||||||
- Adapt existing PromptKit client tests to the prepared-execution path.
|
|
||||||
- Retain or add one behavioral test proving that the debug prompt details match
|
|
||||||
the request actually passed to generation when a backing prompt source could
|
|
||||||
otherwise change between independent preparations. Test the resulting
|
|
||||||
snapshot consistency, not a private helper call count.
|
|
||||||
- Retain capacity tests proving `errors.Is` reaches
|
|
||||||
`contracts.ErrLLMCapacityExceeded`, the selected backend can appear in safe
|
|
||||||
diagnostic context, and provider calls are not made after rejected
|
|
||||||
admission.
|
|
||||||
- Run `go test ./internal/framework/llm` and
|
|
||||||
`go test -race ./internal/framework/llm`.
|
|
||||||
- Run `go test ./...` and `git diff --check`.
|
|
||||||
|
|
||||||
### Completion Criteria
|
|
||||||
|
|
||||||
- `CompleteStructured` no longer calls independent `Prepare` and `Run`
|
|
||||||
operations for one request.
|
|
||||||
- Debug prompt material and generation result originate from the same frozen
|
|
||||||
PromptKit snapshot.
|
|
||||||
- Capacity remains a provider-neutral Notarius error classification.
|
|
||||||
|
|
||||||
## Stage 3: Replace Synthetic Profile Validation With Inspection
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Validate profiles through PromptKit's exact profile-inspection boundary and
|
|
||||||
centralize engine profile-source construction.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
- Introduce a small provider-adapter-owned profile inspection or validation
|
|
||||||
function in `internal/framework/llm`. Its public internal signature must use
|
|
||||||
Notarius-owned configuration and result/error types rather than returning
|
|
||||||
PromptKit types to the CLI.
|
|
||||||
- Share the code that applies `profile_dir`, `profile_file`, and registered
|
|
||||||
backend options between the production PromptKit engine and the inspection
|
|
||||||
engine. Preserve the mutual-exclusion and local-backend rules.
|
|
||||||
- Change CLI explicit-profile preflight to use `Engine.InspectProfile` through
|
|
||||||
that LLM boundary.
|
|
||||||
- Remove `profileCheckPromptID`, `profileCheckPromptFS`, the `testing/fstest`
|
|
||||||
production dependency, and the synthetic `Prepare` request.
|
|
||||||
- Preserve distinct, useful errors for an absent profile, invalid profile,
|
|
||||||
unknown backend registration, cancellation, and invalid profile source.
|
|
||||||
- Do not require `api_key_env` to be populated during configuration validation.
|
|
||||||
Inspection may report credential requirements internally, but actual
|
|
||||||
preparation remains responsible for credential availability before a model
|
|
||||||
call.
|
|
||||||
- Update current-behavior sections in `docs/internal/cli.md` and
|
|
||||||
`docs/internal/llm.md`. Keep field definitions in `docs/config.md`.
|
|
||||||
|
|
||||||
### Tests And Validation
|
|
||||||
|
|
||||||
- Replace synthetic-prompt tests with profile inspection tests covering:
|
|
||||||
configured local backend success; missing local backend failure; absent
|
|
||||||
profile; malformed profile; and an otherwise valid profile whose credential
|
|
||||||
environment variable is intentionally unset.
|
|
||||||
- Prove validation performs no provider HTTP call and remains offline.
|
|
||||||
- Run `go test ./internal/framework/llm ./internal/cli` and `go test ./...`.
|
|
||||||
- Run `git diff --check`.
|
|
||||||
|
|
||||||
### Completion Criteria
|
|
||||||
|
|
||||||
- No production synthetic profile-check prompt remains.
|
|
||||||
- Profile validation uses the same ordinary profile source and backend
|
|
||||||
registrations as execution.
|
|
||||||
- Configuration validation succeeds for structurally valid profiles without
|
|
||||||
reading credential values.
|
|
||||||
|
|
||||||
## Stage 4: Add Application Fallback Profile Asset Plumbing
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Allow module families to register application-owned fallback profile YAML
|
|
||||||
without placing domain policy in generic LLM code.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
- Extend `internal/framework/llm.AssetRegistry` with a separate fallback
|
|
||||||
profile source collection, registration method, flattened filesystem, and
|
|
||||||
safe content digest.
|
|
||||||
- Reuse the existing asset-source path validation and flattening behavior where
|
|
||||||
appropriate. Reject invalid roots, unreadable assets, and duplicate flattened
|
|
||||||
paths. Do not parse PromptKit profile YAML in Notarius.
|
|
||||||
- Add `promptkit.WithFallbackProfileFS` to production engine options only when
|
|
||||||
at least one fallback profile source is registered.
|
|
||||||
- Supply the identical assembled fallback source to the profile-inspection
|
|
||||||
engine. Adjust CLI composition so pipeline-aware profile validation can use
|
|
||||||
the production LLM asset registry without exposing PromptKit types.
|
|
||||||
- Extend profile-source checkpoint identity to include the exact fallback
|
|
||||||
profile asset digest in addition to the PromptKit catalog marker and operator
|
|
||||||
source. Keep the resulting fingerprint hash-only and path/content/credential
|
|
||||||
free.
|
|
||||||
- Keep operator source precedence owned by PromptKit. Do not implement profile
|
|
||||||
merging or duplicate PromptKit source resolution in Notarius.
|
|
||||||
- Update `docs/internal/llm.md` only for the new implemented generic asset and
|
|
||||||
fingerprint mechanics. No domain fallback exists until Stage 5.
|
|
||||||
|
|
||||||
### Tests And Validation
|
|
||||||
|
|
||||||
- Add focused AssetRegistry tests for successful flattening, invalid roots,
|
|
||||||
duplicate paths, and hash changes when fallback bytes change.
|
|
||||||
- Add adapter-level tests showing that the fallback filesystem reaches both
|
|
||||||
execution construction and inspection construction.
|
|
||||||
- Extend checkpoint tests to prove fallback content changes profile-source
|
|
||||||
identity without exposing raw YAML or paths. Use relational comparisons, not
|
|
||||||
a fixed hash literal.
|
|
||||||
- Run `go test ./internal/framework/llm ./internal/cli` and `go test ./...`.
|
|
||||||
- Run `git diff --check`.
|
|
||||||
|
|
||||||
### Completion Criteria
|
|
||||||
|
|
||||||
- Generic plumbing can carry application fallback profiles while remaining
|
|
||||||
unaware of D&D IDs or model settings.
|
|
||||||
- Inspection, execution, and checkpoint identity use the same fallback asset
|
|
||||||
source.
|
|
||||||
|
|
||||||
## Stage 5: Adopt The D&D `dnd-extraction` Fallback
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Give the D&D module family one stable embedded workload profile that operators
|
|
||||||
can replace.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
- Add a D&D-owned embedded PromptKit profile asset with ID `dnd-extraction`
|
|
||||||
under `internal/modules/dnd`. Use the exact baseline defined in
|
|
||||||
`promptkit.md`: OpenRouter, `openai/gpt-5.6-luna`, no explicit reasoning
|
|
||||||
effort, a 240-second timeout, flex service tier, and no selected temperature,
|
|
||||||
token limit, or `top_p`. The omitted reasoning value intentionally allows
|
|
||||||
OpenAI's backend to apply its `medium` default.
|
|
||||||
- Register the profile filesystem from the D&D registrar through the generic
|
|
||||||
fallback profile asset boundary. Keep D&D policy out of
|
|
||||||
`internal/framework/llm` and the CLI composition root.
|
|
||||||
- Change every maintained D&D LLM prompt definition—including scene chunking,
|
|
||||||
all D&D extractors, and NPC normalization—from the model-named default to
|
|
||||||
`default_profile: dnd-extraction`.
|
|
||||||
- Add an integration-level profile-resolution test proving that:
|
|
||||||
- the fallback resolves when no operator source defines the ID;
|
|
||||||
- a valid operator profile with the same ID wins completely; and
|
|
||||||
- an invalid matching operator profile fails rather than falling through.
|
|
||||||
- Test through Notarius's assembled production assets and PromptKit boundary;
|
|
||||||
do not duplicate every upstream source-precedence case.
|
|
||||||
- Update the implemented profile ownership and prompt-default behavior in
|
|
||||||
`docs/internal/dnd.md`, `docs/internal/llm.md`, and
|
|
||||||
`docs/integrations/pkg-promptkit.md`. Defer the complete operator walkthrough
|
|
||||||
and examples to Stage 10.
|
|
||||||
|
|
||||||
### Tests And Validation
|
|
||||||
|
|
||||||
- Run focused D&D prompt preparation tests and the production composition
|
|
||||||
tests.
|
|
||||||
- Run `go test ./internal/modules/dnd/... ./internal/framework/llm
|
|
||||||
./internal/cli`.
|
|
||||||
- Run `go test ./...`.
|
|
||||||
- Verify `rg -n 'default_profile: gemini-2-flash' internal/modules/dnd`
|
|
||||||
returns no matches.
|
|
||||||
- Run `git diff --check`.
|
|
||||||
|
|
||||||
### Completion Criteria
|
|
||||||
|
|
||||||
- All maintained D&D prompts use the application-owned logical profile ID.
|
|
||||||
- The fallback works without an operator profile and remains authoritatively
|
|
||||||
overridable by a matching valid operator definition.
|
|
||||||
|
|
||||||
## Stage 6: Introduce Module Execution-Class Metadata
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Make each production module's ability to use an LLM statically discoverable
|
|
||||||
without yet changing profile inheritance.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
- Add `ExecutionClass contracts.ExecutionClass` to `pipeline.ModuleSpec` and
|
|
||||||
preserve it through normalization, cloning, catalogs, registries, JSON/debug
|
|
||||||
views, and lookup helpers.
|
|
||||||
- In this transitional stage only, allow an omitted execution class to
|
|
||||||
normalize to deterministic so existing test-only fixtures can be migrated in
|
|
||||||
Stage 7 without breaking the repository midway.
|
|
||||||
- Explicitly classify every production module:
|
|
||||||
- D&D scene chunking, every D&D extractor, and D&D NPC normalization as
|
|
||||||
`llm_backed`;
|
|
||||||
- all other current production input, chunk, merge, normalize, and output
|
|
||||||
modules as `deterministic`.
|
|
||||||
- Update production module specification tests and production catalog tests to
|
|
||||||
assert the semantic class alongside stage, artifact kind, and capabilities.
|
|
||||||
- Add catalog lookup support needed by later resolution to retrieve a selected
|
|
||||||
module's execution class by stage and key without constructing it.
|
|
||||||
- Do not implement pipeline-level profile inheritance or reject deterministic
|
|
||||||
profiles yet.
|
|
||||||
- Update `docs/internal/modules.md` and `docs/internal/dnd.md` to identify
|
|
||||||
execution class as registered module metadata, while noting only implemented
|
|
||||||
uses.
|
|
||||||
|
|
||||||
### Tests And Validation
|
|
||||||
|
|
||||||
- Run module registration/spec tests across generic, Seriatim, and D&D
|
|
||||||
families.
|
|
||||||
- Run `go test ./internal/framework/pipeline ./internal/modules/...`.
|
|
||||||
- Run `go test ./...` and `git diff --check`.
|
|
||||||
|
|
||||||
### Completion Criteria
|
|
||||||
|
|
||||||
- Every production module has an explicit correct execution class.
|
|
||||||
- Catalog consumers can retrieve that class without a concrete module
|
|
||||||
instance.
|
|
||||||
- Test-only omitted classes remain the only temporary compatibility behavior.
|
|
||||||
|
|
||||||
## Stage 7: Enforce Execution Metadata And Remove Runtime Probing
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Finish the execution-class contract so missing metadata cannot cause future
|
|
||||||
profile drift.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
- Update every framework, CLI, and integration test module specification to
|
|
||||||
declare an explicit execution class appropriate to the fake behavior.
|
|
||||||
- Change module-spec validation so an empty or unsupported execution class is a
|
|
||||||
registration error. Remove the transitional deterministic default from
|
|
||||||
Stage 6.
|
|
||||||
- Replace the chunk runner's special `ChunkExecutionClassProvider` probe with
|
|
||||||
specification-derived behavior. Remove the now-redundant provider interface,
|
|
||||||
implementation methods, and tests when they have no remaining consumer.
|
|
||||||
- Ensure chunk producer provenance remains unchanged: it records a non-empty
|
|
||||||
effective binding profile for an LLM-backed chunker, while a deterministic
|
|
||||||
chunker records no profile. A profile selected only through the prompt
|
|
||||||
default remains represented by PromptKit's actual-profile manifest rather
|
|
||||||
than being invented as an explicit chunk binding.
|
|
||||||
- Review helper constructors and fixtures for opportunities to set execution
|
|
||||||
class once without obscuring the class under test. Do not introduce an
|
|
||||||
elaborate test-spec framework.
|
|
||||||
- Update internal documentation if the removal changes any described runtime
|
|
||||||
mechanics.
|
|
||||||
|
|
||||||
### Tests And Validation
|
|
||||||
|
|
||||||
- Add or retain focused registration tests for missing and invalid execution
|
|
||||||
classes.
|
|
||||||
- Retain chunk-plan provenance tests for LLM-backed and deterministic
|
|
||||||
chunkers.
|
|
||||||
- Run `go test ./internal/framework/pipeline ./internal/modules/...`.
|
|
||||||
- Run `go test ./...`, `go vet ./...`, and `git diff --check`.
|
|
||||||
|
|
||||||
### Completion Criteria
|
|
||||||
|
|
||||||
- No registered module specification relies on an implicit execution class.
|
|
||||||
- Pipeline metadata, not a concrete runtime type assertion, owns module
|
|
||||||
execution classification.
|
|
||||||
|
|
||||||
## Stage 8: Resolve Programmatic Pipeline Profile Defaults
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Implement profile inheritance and precedence inside the pipeline resolver
|
|
||||||
before exposing the field through YAML configuration.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
- Add an optional trimmed `LLMProfile` field to
|
|
||||||
`pipeline.PipelineProfile`. Add a non-empty runtime override field to
|
|
||||||
`pipeline.ResolveOptions` so all precedence decisions occur in the resolver
|
|
||||||
rather than through pre-resolution mutation.
|
|
||||||
- After module selection, `--only` filtering, default validator-chain
|
|
||||||
selection, and validator compatibility resolution, apply effective profiles
|
|
||||||
to every selected input, chunk, extract, merge, normalize, output, and
|
|
||||||
validator binding according to the precedence in `promptkit.md`.
|
|
||||||
- Apply profiles only when the selected module or validator execution class is
|
|
||||||
`llm_backed`.
|
|
||||||
- Reject a binding-specific `llm_profile` on any deterministic module or
|
|
||||||
validator. Do not reject or inspect an unused pipeline default when no
|
|
||||||
selected LLM-backed binding consumes it.
|
|
||||||
- Leave an LLM-backed binding empty when no CLI, binding, or pipeline profile is
|
|
||||||
selected so PromptKit can use the prompt's `default_profile`.
|
|
||||||
- Store the effective values on resolved bindings before digest construction.
|
|
||||||
Do not add a second inheritance decision to execution.
|
|
||||||
- Ensure semantically equivalent repeated binding profiles and one inherited
|
|
||||||
default produce the same resolved pipeline digest. Ensure any changed
|
|
||||||
effective profile changes the digest.
|
|
||||||
- Do not modify file configuration or CLI parsing in this stage.
|
|
||||||
|
|
||||||
### Tests And Validation
|
|
||||||
|
|
||||||
- Add pipeline package tests for the complete precedence matrix:
|
|
||||||
runtime override; binding-specific exception; pipeline default; prompt
|
|
||||||
fallback; and deterministic bindings.
|
|
||||||
- Cover default and explicitly configured validator chains, all relevant stage
|
|
||||||
categories, `--only` lane selection, unused defaults, deterministic-profile
|
|
||||||
rejection, and semantic digest equivalence.
|
|
||||||
- Prefer table-driven package-level tests over assertions on private traversal
|
|
||||||
helpers.
|
|
||||||
- Run `go test ./internal/framework/pipeline` and `go test ./...`.
|
|
||||||
- Run `git diff --check`.
|
|
||||||
|
|
||||||
### Completion Criteria
|
|
||||||
|
|
||||||
- Programmatic pipelines resolve one canonical effective profile policy.
|
|
||||||
- Only LLM-backed resolved bindings can contain a profile.
|
|
||||||
- Runtime override, binding, pipeline, and prompt precedence is unambiguous and
|
|
||||||
digest-stable.
|
|
||||||
|
|
||||||
## Stage 9: Expose Pipeline Defaults Through Configuration And CLI
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Make the profile-default workflow available to operators while preserving
|
|
||||||
validation and override behavior.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
- Add optional `pipelines.<id>.llm_profile` support to the version 4 file
|
|
||||||
configuration model. Use presence-aware decoding so an explicitly set blank
|
|
||||||
value is rejected, while omission remains valid.
|
|
||||||
- Preserve the field through file application, configuration cloning,
|
|
||||||
effective configuration, and programmatic profile copies without aliasing or
|
|
||||||
trimming drift.
|
|
||||||
- Remove `applyLLMProfileOverride`. Pass the CLI override through the resolver's
|
|
||||||
runtime-override input so deterministic bindings are never populated.
|
|
||||||
- Update effective profile-ID collection to cover every selected LLM-backed
|
|
||||||
module stage and LLM-backed validator, including future LLM-backed input and
|
|
||||||
output modules. Do not inspect deterministic or unselected profiles.
|
|
||||||
- Ensure `run`, `config validate --pipeline`, resume/checkpoint identity, and
|
|
||||||
relevant dry preflight paths all use the same resolved effective profiles.
|
|
||||||
- Preserve `--llm-profile` as the highest-precedence non-empty run-wide
|
|
||||||
override and preserve binding-specific profiles as exceptions when no CLI
|
|
||||||
override is present.
|
|
||||||
- Do not increment the configuration version.
|
|
||||||
- Update current configuration and CLI contracts in `docs/config.md` and
|
|
||||||
`docs/cli.md` in the same stage. Link to operations for the deployment
|
|
||||||
workflow rather than duplicating it prematurely.
|
|
||||||
|
|
||||||
### Tests And Validation
|
|
||||||
|
|
||||||
- Add file-config tests for omission, trimming, explicit blank rejection,
|
|
||||||
unknown-key behavior, cloning, and round-trip application.
|
|
||||||
- Add effective-config and CLI contract tests for precedence, LLM-only
|
|
||||||
application, inherited-profile inspection failure before factory execution,
|
|
||||||
`--only`, and digest changes.
|
|
||||||
- Retain offline operation and do not require credentials for
|
|
||||||
`config validate --pipeline`.
|
|
||||||
- Run `go test ./internal/core/config ./internal/framework/pipeline
|
|
||||||
./internal/cli`.
|
|
||||||
- Run `go test ./...`, `go vet ./...`, and `git diff --check`.
|
|
||||||
|
|
||||||
### Completion Criteria
|
|
||||||
|
|
||||||
- Operators can select `dnd-extraction` once per pipeline.
|
|
||||||
- Configuration and CLI paths share the resolver's precedence policy.
|
|
||||||
- Unknown effective profiles fail preflight, while deterministic and unused
|
|
||||||
profiles do not cause spurious inspection.
|
|
||||||
|
|
||||||
## Stage 10: Complete Operator Documentation, Examples, And Decision Record
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Make the implemented workflow understandable, copyable, and maintainable
|
|
||||||
without duplicating canonical facts.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
- Create an ADR using the next sequential number for the durable decision to
|
|
||||||
use workload-oriented pipeline defaults with operator-overridable application
|
|
||||||
fallback profiles. Record context, decision, alternatives, and consequences;
|
|
||||||
do not turn the ADR into a field reference or implementation log.
|
|
||||||
- Complete `docs/config.md` as the canonical owner of profile-source fields,
|
|
||||||
`pipelines.<id>.llm_profile`, validation, and precedence.
|
|
||||||
- Complete `docs/operations.md` with an operator workflow that distinguishes
|
|
||||||
Notarius embedded prompts, Notarius fallback profiles, PromptKit built-ins,
|
|
||||||
and deployment filesystem profiles. Include production/development/local use
|
|
||||||
of the same `dnd-extraction` ID, credential handling, absolute-path guidance,
|
|
||||||
and the fact that current relative profile paths use the process working
|
|
||||||
directory rather than the configuration file's directory.
|
|
||||||
- Complete `docs/integrations/pkg-promptkit.md` with the v0.5.0 boundary,
|
|
||||||
prepared execution, inspection, fallback and ordinary source precedence,
|
|
||||||
optional provider controls, capacity adaptation, and compatibility policy.
|
|
||||||
- Update `docs/internal/configuration.md`, `docs/internal/pipeline.md`,
|
|
||||||
`docs/internal/cli.md`, `docs/internal/llm.md`, `docs/internal/modules.md`, and
|
|
||||||
`docs/internal/dnd.md` only for their owned implementation details. Link to
|
|
||||||
canonical configuration, operations, and upstream format contracts rather
|
|
||||||
than restating them.
|
|
||||||
- Keep exactly the existing two D&D configuration examples. Add
|
|
||||||
`llm_profile: dnd-extraction` to the minimal and complete pipelines and remove
|
|
||||||
the now-redundant model-named binding override from the complete example.
|
|
||||||
- Add one secret-free maintained operator profile at
|
|
||||||
`examples/profiles/dnd-extraction.yml`. It should be a complete valid profile
|
|
||||||
for the same logical ID and may mirror the embedded baseline; its purpose is
|
|
||||||
to demonstrate file ownership and format, not claim automatic environment
|
|
||||||
detection. Link it from the configuration and operations documentation.
|
|
||||||
- If the complete example selects the external profile file, use a path that
|
|
||||||
is valid for the documented repository-root invocation and explicitly note
|
|
||||||
the working-directory rule. Keep the minimal example dependent only on the
|
|
||||||
embedded fallback.
|
|
||||||
- Add or extend maintained-example validation so both configuration examples
|
|
||||||
and the profile YAML are checked without generation or credentials.
|
|
||||||
- Remove the now-implemented `Pipeline-Level LLM Profile Defaults` section from
|
|
||||||
`docs/roadmap/future.md`. Preserve the unrelated deterministic session and
|
|
||||||
concurrency items.
|
|
||||||
- Do not delete `promptkit.md` or this implementation plan during the feature
|
|
||||||
implementation; retire them only after post-implementation review.
|
|
||||||
|
|
||||||
### Tests And Validation
|
|
||||||
|
|
||||||
- Run maintained example/configuration tests and relevant CLI help/parser
|
|
||||||
tests.
|
|
||||||
- Run `go test ./...`.
|
|
||||||
- Run `rg -n 'gemini-2-flash' examples docs` and review every remaining match
|
|
||||||
for intentional model-policy or historical context.
|
|
||||||
- Run `rg -n 'v0\.3\.0|profileCheckPrompt|applyLLMProfileOverride' .` and resolve
|
|
||||||
stale production or current-documentation matches.
|
|
||||||
- Verify all new links and `git diff --check`.
|
|
||||||
|
|
||||||
### Completion Criteria
|
|
||||||
|
|
||||||
- Every current fact has one canonical documentation owner.
|
|
||||||
- Operators can distinguish and deploy all profile layers without reading Go
|
|
||||||
source.
|
|
||||||
- Both maintained configurations and the maintained external profile are valid,
|
|
||||||
secret-free, and tested offline.
|
|
||||||
- Implemented profile work no longer remains in `future.md`.
|
|
||||||
|
|
||||||
## Stage 11: Final Verification And Quality Review
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Verify the complete migration as one integrated change and correct only defects
|
|
||||||
or omissions found during that review.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
- Review the final diff against every acceptance criterion in `promptkit.md`.
|
|
||||||
- Confirm provider-specific PromptKit types remain inside the LLM integration
|
|
||||||
boundary and D&D policy remains inside the D&D module family.
|
|
||||||
- Confirm execution and inspection receive identical ordinary, fallback, and
|
|
||||||
backend configuration.
|
|
||||||
- Confirm no paths, profile YAML, endpoints, credentials, or prepared handle
|
|
||||||
state leak into fingerprints or ordinary diagnostics.
|
|
||||||
- Confirm all production module specs have explicit correct execution classes
|
|
||||||
and every resolved deterministic binding is profile-free.
|
|
||||||
- Confirm prompt default, pipeline default, binding override, and CLI override
|
|
||||||
behavior through representative assembled configurations.
|
|
||||||
- Review tests for redundancy and remove obsolete synthetic-prompt,
|
|
||||||
runtime-probe, exact-hash, or duplicated upstream-behavior tests superseded by
|
|
||||||
stronger contract tests.
|
|
||||||
- Perform an optional manual D&D quality comparison if credentials and an
|
|
||||||
evaluation transcript are deliberately supplied. Record no private input or
|
|
||||||
credential material, and do not make this comparison a completion gate.
|
|
||||||
|
|
||||||
### Validation Commands
|
|
||||||
|
|
||||||
```sh
|
|
||||||
gofmt -w <changed-go-files>
|
|
||||||
go test ./...
|
|
||||||
go test -race ./internal/framework/llm ./internal/core/config ./internal/framework/pipeline ./internal/cli
|
|
||||||
go vet ./...
|
|
||||||
go build ./cmd/notarius
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
Also run focused stale-contract searches:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
rg -n 'gitea.maximumdirect.net/eric/promptkit v0\.3\.0|PromptKit v0\.3\.0' .
|
|
||||||
rg -n 'default_profile: gemini-2-flash|profileCheckPrompt|applyLLMProfileOverride' internal docs examples
|
|
||||||
```
|
|
||||||
|
|
||||||
Review any matches rather than deleting intentional historical references
|
|
||||||
blindly.
|
|
||||||
|
|
||||||
### Completion Criteria
|
|
||||||
|
|
||||||
- All automated checks pass offline and without real credentials.
|
|
||||||
- The implemented behavior matches `promptkit.md` with no known architecture,
|
|
||||||
provenance, checkpoint, profile-precedence, or documentation gap.
|
|
||||||
- Any optional live evaluation is clearly separate from correctness testing.
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None. The roadmap decisions are sufficient to implement every stage without an
|
|
||||||
additional product or architecture choice.
|
|
||||||
@@ -1,318 +0,0 @@
|
|||||||
# PromptKit v0.5 Integration And LLM Profile Policy
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
This roadmap defines the target state for upgrading Notarius from PromptKit
|
|
||||||
v0.3.0 to v0.5.0 and adopting the upstream runtime and profile facilities that
|
|
||||||
directly improve Notarius. It also defines the application policy for stable,
|
|
||||||
domain-oriented LLM profile names, operator overrides, pipeline inheritance,
|
|
||||||
profile validation, provider defaults, checkpoint identity, and documentation.
|
|
||||||
|
|
||||||
The ordered work needed to reach this state belongs in
|
|
||||||
[the implementation plan](implementation.md). Current behavior remains defined
|
|
||||||
by the canonical documentation outside `docs/roadmap/` until the corresponding
|
|
||||||
work is implemented.
|
|
||||||
|
|
||||||
## Background
|
|
||||||
|
|
||||||
Notarius currently pins PromptKit v0.3.0. Its adapter prepares a request once
|
|
||||||
for debug material and then independently runs the original request, causing
|
|
||||||
PromptKit to prepare the same logical call a second time. The CLI validates an
|
|
||||||
explicit profile by preparing a synthetic prompt. PromptKit profile selection
|
|
||||||
can be repeated on individual module bindings or replaced for one invocation
|
|
||||||
with `--llm-profile`, but a configured pipeline cannot yet declare one inherited
|
|
||||||
profile policy.
|
|
||||||
|
|
||||||
PromptKit v0.4.0 and v0.5.0 add the upstream boundaries needed to improve these
|
|
||||||
areas:
|
|
||||||
|
|
||||||
- [v0.4.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.5.0/docs/releases/v0.4.0.md)
|
|
||||||
adds opaque prepared executions, exact profile and prompt inspection, and a
|
|
||||||
typed backend-capacity error;
|
|
||||||
- [v0.5.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.5.0/docs/releases/v0.5.0.md)
|
|
||||||
adds application fallback profile filesystems and stops sending unset
|
|
||||||
optional sampling controls as framework-selected provider values; and
|
|
||||||
- the [v0.5.0 format contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.5.0/docs/formats.md)
|
|
||||||
defines the resulting profile-source and execution-setting precedence.
|
|
||||||
|
|
||||||
A source-compatibility test of the current Notarius repository against
|
|
||||||
PromptKit v0.5.0 completed successfully. The work is therefore primarily an
|
|
||||||
intentional runtime and configuration migration rather than a repair for a
|
|
||||||
breaking Go API change.
|
|
||||||
|
|
||||||
## Goals
|
|
||||||
|
|
||||||
- Pin and document PromptKit v0.5.0 as Notarius's supported upstream contract.
|
|
||||||
- Execute the exact prepared request snapshot whose safe details are recorded
|
|
||||||
in Notarius debug material.
|
|
||||||
- Validate configured PromptKit profiles through the upstream inspection API
|
|
||||||
without synthetic prompts, provider calls, or credential-value access.
|
|
||||||
- Give Notarius an application-owned, operator-overridable
|
|
||||||
`dnd-extraction` profile fallback.
|
|
||||||
- Let a pipeline choose one default LLM profile without repeating that ID on
|
|
||||||
every LLM-backed binding.
|
|
||||||
- Apply profile inheritance and run-wide overrides only where the resolved
|
|
||||||
module or validator can use an LLM.
|
|
||||||
- Preserve accurate checkpoint invalidation, effective profile provenance,
|
|
||||||
redaction, cancellation, concurrency, and provider-neutral module contracts.
|
|
||||||
- Provide operators with one clear deployment pattern for production,
|
|
||||||
development, and local profile definitions.
|
|
||||||
|
|
||||||
## Target End State
|
|
||||||
|
|
||||||
### PromptKit Runtime Boundary
|
|
||||||
|
|
||||||
Notarius depends on PromptKit v0.5.0 and uses its public APIs rather than
|
|
||||||
reimplementing source or execution resolution.
|
|
||||||
|
|
||||||
For each structured completion, the adapter:
|
|
||||||
|
|
||||||
1. builds one PromptKit run request from the provider-neutral Notarius request;
|
|
||||||
2. calls `PrepareExecution` once;
|
|
||||||
3. immediately arranges an idempotent `Discard` for every unexecuted handle;
|
|
||||||
4. obtains credential-redacted `Details` for debug and response metadata; and
|
|
||||||
5. calls `RunPrepared` so generation uses that exact frozen snapshot.
|
|
||||||
|
|
||||||
The debug prompt and successful result therefore describe the same selected
|
|
||||||
profile, rendered messages, input bytes, session, output contract, and effective
|
|
||||||
settings even when a filesystem-backed source changes concurrently. PromptKit
|
|
||||||
handle types remain private to `internal/framework/llm`.
|
|
||||||
|
|
||||||
PromptKit admission failures continue to match Notarius's provider-neutral
|
|
||||||
`ErrLLMCapacityExceeded` contract. When PromptKit supplies a `CapacityError`,
|
|
||||||
the adapter obtains the normalized backend ID through `errors.As` and may add it
|
|
||||||
to safe application-owned diagnostics without parsing upstream error wording.
|
|
||||||
The backend ID does not become a provider-specific module contract.
|
|
||||||
|
|
||||||
### Optional Provider Controls
|
|
||||||
|
|
||||||
Notarius accepts PromptKit v0.5.0's new behavior for `temperature`,
|
|
||||||
`max_tokens`, and `top_p`: an unset setting is omitted from compatible provider
|
|
||||||
requests and the provider chooses its own default. Notarius does not restore
|
|
||||||
PromptKit's former implicit `top_p: 1` value globally.
|
|
||||||
|
|
||||||
An operator who requires a particular value specifies it in the selected
|
|
||||||
PromptKit profile. The application fallback described below intentionally
|
|
||||||
leaves these controls unset. A human-reviewed D&D extraction comparison should
|
|
||||||
be performed after the upgrade, but paid or nondeterministic model output is
|
|
||||||
not part of the default automated test suite.
|
|
||||||
|
|
||||||
### Profile Inspection
|
|
||||||
|
|
||||||
Pipeline-aware configuration validation uses `Engine.InspectProfile` for every
|
|
||||||
effective explicit profile ID. It verifies that the profile exists, parses and
|
|
||||||
validates, resolves its backend and target, and is compatible with the engine's
|
|
||||||
registered backends. It does not create a synthetic prompt, load prompt inputs,
|
|
||||||
contact a provider, or require credential values to exist in the validation
|
|
||||||
process environment.
|
|
||||||
|
|
||||||
Credential availability is execution-time state. PromptKit preparation still
|
|
||||||
enforces the selected profile's credential contract before generation. This
|
|
||||||
keeps `notarius config validate` useful in build and deployment validation
|
|
||||||
environments where secrets are deliberately absent.
|
|
||||||
|
|
||||||
PromptKit construction for inspection and execution uses one shared internal
|
|
||||||
profile-source and backend-option path. The CLI does not expose PromptKit public
|
|
||||||
types across the Notarius LLM boundary merely to perform inspection.
|
|
||||||
|
|
||||||
`InspectPrompt` is not adopted merely because it exists. It remains available
|
|
||||||
for a later, separately defined module-to-prompt interface preflight if a
|
|
||||||
concrete validation requirement justifies that additional contract.
|
|
||||||
|
|
||||||
### Application And Operator Profile Sources
|
|
||||||
|
|
||||||
Notarius embeds one ordinary PromptKit YAML profile with the stable ID
|
|
||||||
`dnd-extraction`. It is an application fallback registered through
|
|
||||||
`WithFallbackProfileFS`, is owned by the D&D module family, and initially
|
|
||||||
preserves the current effective D&D baseline:
|
|
||||||
|
|
||||||
- backend: PromptKit's built-in `openrouter` backend;
|
|
||||||
- model: `openai/gpt-5.6-luna`;
|
|
||||||
- reasoning effort: unset, allowing OpenAI's backend to apply its default of
|
|
||||||
`medium`;
|
|
||||||
- generation timeout: 240 seconds;
|
|
||||||
- service tier: `flex`; and
|
|
||||||
- no application-selected `temperature`, `max_tokens`, or `top_p`.
|
|
||||||
|
|
||||||
All maintained D&D LLM prompt definitions use `dnd-extraction` as their
|
|
||||||
`default_profile`. The ID communicates workload intent rather than a provider,
|
|
||||||
model, or environment. Changing the embedded fallback is an intentional
|
|
||||||
Notarius execution-policy change and participates in checkpoint identity.
|
|
||||||
|
|
||||||
Effective profile definitions resolve in PromptKit's order:
|
|
||||||
|
|
||||||
1. programmatic in-memory profiles used by tests or explicit consumers;
|
|
||||||
2. the operator source configured by `promptkit.profile_file` or
|
|
||||||
`promptkit.profile_dir`;
|
|
||||||
3. the Notarius application fallback source; and
|
|
||||||
4. PromptKit's embedded built-in catalog.
|
|
||||||
|
|
||||||
Only an absent ID falls through to the next source. A matching profile is a
|
|
||||||
complete definition: fields are not merged with a lower-precedence definition,
|
|
||||||
and a malformed matching operator profile fails rather than silently selecting
|
|
||||||
the application fallback.
|
|
||||||
|
|
||||||
Production, development, and local deployments should normally provide
|
|
||||||
different complete definitions for the same `dnd-extraction` ID. An operator
|
|
||||||
source is optional because the application fallback keeps the maintained D&D
|
|
||||||
workflow usable, but a deployment that needs an intentional model or backend
|
|
||||||
policy should configure its own definition.
|
|
||||||
|
|
||||||
### Domain Ownership And Asset Assembly
|
|
||||||
|
|
||||||
The D&D fallback profile remains under `internal/modules/dnd` and is registered
|
|
||||||
by the D&D registrar, consistent with ADR-0004. Generic LLM plumbing knows how
|
|
||||||
to collect and flatten application fallback profile filesystems but contains no
|
|
||||||
D&D model or policy knowledge.
|
|
||||||
|
|
||||||
The shared asset registry detects invalid roots, unreadable sources, and
|
|
||||||
duplicate flattened paths. PromptKit remains responsible for strict profile
|
|
||||||
YAML parsing, duplicate profile-ID detection, source precedence, and effective
|
|
||||||
target resolution. The same assembled fallback source is supplied to runtime
|
|
||||||
execution and CLI profile inspection.
|
|
||||||
|
|
||||||
### Explicit Module Execution Metadata
|
|
||||||
|
|
||||||
Every registered input, chunk, extract, merge, normalize, and output module
|
|
||||||
declares one required execution class: `deterministic` or `llm_backed`.
|
|
||||||
Validator registrations continue to declare the same distinction through their
|
|
||||||
validator specifications.
|
|
||||||
|
|
||||||
The registered specification is authoritative for configuration resolution.
|
|
||||||
Current production classifications are:
|
|
||||||
|
|
||||||
- the D&D scene chunker, all D&D extractors, and the D&D NPC normalizer are
|
|
||||||
LLM-backed;
|
|
||||||
- the Seriatim input adapter, generic chunker, all current mergers, all other
|
|
||||||
current normalizers, and the JSON output encoder are deterministic; and
|
|
||||||
- current validators retain their declared classifications.
|
|
||||||
|
|
||||||
Missing or unsupported execution metadata is a registration error. Explicitly
|
|
||||||
assigning `llm_profile` to a deterministic module or validator is a pipeline
|
|
||||||
resolution error. The framework does not infer execution class by inspecting
|
|
||||||
domain package names or concrete implementation types at runtime.
|
|
||||||
|
|
||||||
The module specification replaces the chunk runner's special runtime
|
|
||||||
execution-class probe. Effective resolved bindings already express the result:
|
|
||||||
only LLM-backed bindings may retain a non-empty profile.
|
|
||||||
|
|
||||||
### Pipeline-Level Profile Default
|
|
||||||
|
|
||||||
Configuration version 4 gains one optional non-empty pipeline field:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
pipelines:
|
|
||||||
dnd-session:
|
|
||||||
llm_profile: dnd-extraction
|
|
||||||
```
|
|
||||||
|
|
||||||
No configuration-version increment is required because the field is additive
|
|
||||||
and existing files remain valid. An explicitly present blank value is invalid.
|
|
||||||
|
|
||||||
For every selected LLM-backed module and validator, the effective profile uses
|
|
||||||
this precedence:
|
|
||||||
|
|
||||||
1. non-empty run-wide `--llm-profile` override;
|
|
||||||
2. binding-specific `llm_profile`;
|
|
||||||
3. pipeline-level `llm_profile`; and
|
|
||||||
4. the prompt definition's `default_profile`, represented by an empty effective
|
|
||||||
Notarius binding profile.
|
|
||||||
|
|
||||||
The run-wide override and inherited pipeline default never attach to a
|
|
||||||
deterministic binding. Binding-specific exceptions remain available when one
|
|
||||||
operation needs a different cost, latency, quality, backend, or reasoning
|
|
||||||
policy.
|
|
||||||
|
|
||||||
Inheritance is resolved after module and validator selection, including
|
|
||||||
`--only` lane filtering, but before effective-pipeline validation, digest
|
|
||||||
construction, explicit-profile inspection, checkpoint construction,
|
|
||||||
preparation, execution, or provenance capture. Only profiles used by selected
|
|
||||||
LLM-backed bindings are inspected. An unused pipeline default in a pipeline
|
|
||||||
with no selected LLM-backed work does not require an otherwise unused profile
|
|
||||||
to exist.
|
|
||||||
|
|
||||||
The resolved pipeline contains effective binding profiles rather than a second
|
|
||||||
runtime inheritance mechanism. Two pipelines that differ only by spelling the
|
|
||||||
same effective policy once as a pipeline default and once on every LLM-backed
|
|
||||||
binding have the same semantic resolved digest. Changing an effective profile
|
|
||||||
changes the digest and applicable checkpoint identity.
|
|
||||||
|
|
||||||
### Provenance And Checkpoints
|
|
||||||
|
|
||||||
The PromptKit profile-source checkpoint fingerprint covers:
|
|
||||||
|
|
||||||
- the PromptKit v0.5.0 built-in profile catalog identity;
|
|
||||||
- exact application fallback profile asset content; and
|
|
||||||
- exact configured operator profile YAML content, when present.
|
|
||||||
|
|
||||||
The existing local-backend target fingerprint remains separate and continues
|
|
||||||
to exclude scheduling-only concurrency limits. Fingerprints contain hashes and
|
|
||||||
stable markers, not profile contents, filesystem paths, endpoints, credentials,
|
|
||||||
or other secrets.
|
|
||||||
|
|
||||||
Changing the PromptKit version, application fallback, operator profile, or
|
|
||||||
effective pipeline profile makes incompatible LLM checkpoints ineligible for
|
|
||||||
reuse. The dependency upgrade is expected to invalidate checkpoints produced
|
|
||||||
under v0.3.0.
|
|
||||||
|
|
||||||
Successful run manifests continue to record only profiles actually selected by
|
|
||||||
PromptKit, including their effective model, backend, and reasoning metadata.
|
|
||||||
Debug output reports the same effective execution snapshot used for generation.
|
|
||||||
|
|
||||||
### Operator Documentation And Examples
|
|
||||||
|
|
||||||
Canonical documentation clearly distinguishes:
|
|
||||||
|
|
||||||
- Notarius prompt and schema assets embedded in the application;
|
|
||||||
- Notarius application fallback profiles embedded in the application;
|
|
||||||
- PromptKit's own embedded built-in profiles; and
|
|
||||||
- operator profile files on the deployment filesystem.
|
|
||||||
|
|
||||||
The configuration reference owns the pipeline field, profile-source fields,
|
|
||||||
validation rules, and precedence. Operations owns deployment layout, working
|
|
||||||
directory behavior, credentials, and environment-specific profile management.
|
|
||||||
The PromptKit integration document owns the pinned upstream contract and
|
|
||||||
source-precedence boundary. Internal documents describe asset registration,
|
|
||||||
resolution, inspection, prepared execution, fingerprinting, and tests without
|
|
||||||
duplicating user-facing field definitions.
|
|
||||||
|
|
||||||
The maintained examples continue to include only the minimal and complete D&D
|
|
||||||
configurations. They use the stable `dnd-extraction` policy, and one maintained
|
|
||||||
PromptKit profile file under `examples/` demonstrates an operator override.
|
|
||||||
Examples remain secret-free and are validated without live provider calls.
|
|
||||||
|
|
||||||
## Out Of Scope
|
|
||||||
|
|
||||||
- Implementing the separate deterministic prompt-session identity roadmap
|
|
||||||
item.
|
|
||||||
- Changing the default `concurrency.total_llm` value; PromptKit's retained
|
|
||||||
OpenRouter capacity of 16 remains relevant to that separate item.
|
|
||||||
- Adding model evaluation as a deterministic or CI correctness gate.
|
|
||||||
- Automatically selecting production, development, or local environments.
|
|
||||||
Deployment configuration chooses the operator profile source.
|
|
||||||
- Profile inheritance, partial profile merging, or cross-profile aliases.
|
|
||||||
- Exposing PromptKit types to modules, validators, durable output contracts, or
|
|
||||||
public configuration structures.
|
|
||||||
- Adopting `InspectPrompt` without a separately justified prompt-interface
|
|
||||||
validation contract.
|
|
||||||
|
|
||||||
## Acceptance Criteria
|
|
||||||
|
|
||||||
- Notarius builds and its offline test suite passes with PromptKit v0.5.0.
|
|
||||||
- Every structured completion executes the exact snapshot used for safe debug
|
|
||||||
prompt details.
|
|
||||||
- Profile preflight uses profile inspection and no synthetic prompt.
|
|
||||||
- The embedded `dnd-extraction` fallback resolves without an operator source,
|
|
||||||
and a matching valid operator profile replaces it completely.
|
|
||||||
- Every production module has explicit, correct execution metadata.
|
|
||||||
- Pipeline, binding, CLI, and prompt-default precedence behaves as defined for
|
|
||||||
modules and validators, while deterministic bindings remain profile-free.
|
|
||||||
- Effective profiles participate in pipeline digests, profile inspection,
|
|
||||||
checkpoint identity, debug records, and run provenance at the appropriate
|
|
||||||
boundaries.
|
|
||||||
- The dependency and application fallback changes invalidate incompatible old
|
|
||||||
checkpoints without exposing profile or credential content.
|
|
||||||
- Canonical documentation and maintained examples accurately describe and
|
|
||||||
exercise the implemented operator workflow.
|
|
||||||
- Default tests remain deterministic, offline, credential-free, and focused on
|
|
||||||
Notarius-owned behavior rather than duplicating PromptKit's upstream suite.
|
|
||||||
@@ -36,10 +36,13 @@ pipelines:
|
|||||||
window_units: 3
|
window_units: 3
|
||||||
lanes:
|
lanes:
|
||||||
- item-events
|
- item-events
|
||||||
|
- locations
|
||||||
|
- location-occurrences
|
||||||
- npcs
|
- npcs
|
||||||
- spells
|
- spells
|
||||||
- combat-turns
|
- combat-turns
|
||||||
- npc-interactions
|
- npc-interactions
|
||||||
|
- enemy-events
|
||||||
steps:
|
steps:
|
||||||
# Establish session-wide reference artifacts alongside independent item events.
|
# Establish session-wide reference artifacts alongside independent item events.
|
||||||
- id: describe-session
|
- id: describe-session
|
||||||
@@ -58,6 +61,14 @@ pipelines:
|
|||||||
normalize:
|
normalize:
|
||||||
module: dnd/npcs
|
module: dnd/npcs
|
||||||
retries: 2
|
retries: 2
|
||||||
|
locations:
|
||||||
|
extract:
|
||||||
|
module: dnd/locations
|
||||||
|
retries: 2
|
||||||
|
merge: appendorder
|
||||||
|
normalize:
|
||||||
|
module: dnd/locations
|
||||||
|
retries: 2
|
||||||
scene-descriptions:
|
scene-descriptions:
|
||||||
extract:
|
extract:
|
||||||
module: dnd/scene-descriptions
|
module: dnd/scene-descriptions
|
||||||
@@ -65,9 +76,13 @@ pipelines:
|
|||||||
merge: appendorder
|
merge: appendorder
|
||||||
normalize: dnd/scene-descriptions
|
normalize: dnd/scene-descriptions
|
||||||
- id: extract-events
|
- id: extract-events
|
||||||
# Accepted NPC grounding and scene-description eligibility artifacts are
|
# Accepted registry artifacts and scene-description eligibility artifacts
|
||||||
# supplied in memory to their compatible consumers in this step.
|
# are supplied in memory to their compatible consumers in this step.
|
||||||
references:
|
references:
|
||||||
|
locations:
|
||||||
|
artifact:
|
||||||
|
step: describe-session
|
||||||
|
lane: locations
|
||||||
npcs:
|
npcs:
|
||||||
artifact:
|
artifact:
|
||||||
step: describe-session
|
step: describe-session
|
||||||
@@ -101,3 +116,34 @@ pipelines:
|
|||||||
retries: 2
|
retries: 2
|
||||||
merge: appendorder
|
merge: appendorder
|
||||||
normalize: dnd/npc-interactions
|
normalize: dnd/npc-interactions
|
||||||
|
location-occurrences:
|
||||||
|
extract:
|
||||||
|
module: dnd/location-occurrences
|
||||||
|
retries: 2
|
||||||
|
merge: appendorder
|
||||||
|
normalize: dnd/location-occurrences
|
||||||
|
- id: track-enemies
|
||||||
|
references:
|
||||||
|
npcs:
|
||||||
|
artifact:
|
||||||
|
step: describe-session
|
||||||
|
lane: npcs
|
||||||
|
scene_descriptions:
|
||||||
|
artifact:
|
||||||
|
step: describe-session
|
||||||
|
lane: scene-descriptions
|
||||||
|
combat_turns:
|
||||||
|
artifact:
|
||||||
|
step: extract-events
|
||||||
|
lane: combat-turns
|
||||||
|
npc_interactions:
|
||||||
|
artifact:
|
||||||
|
step: extract-events
|
||||||
|
lane: npc-interactions
|
||||||
|
artifacts:
|
||||||
|
enemy-events:
|
||||||
|
extract:
|
||||||
|
module: dnd/enemy-events
|
||||||
|
retries: 2
|
||||||
|
merge: appendorder
|
||||||
|
normalize: dnd/enemy-events
|
||||||
|
|||||||
@@ -249,10 +249,24 @@ func TestRunAutoReusesPlanWhenRunInputsChange(t *testing.T) {
|
|||||||
}
|
}
|
||||||
harness.mu.Lock()
|
harness.mu.Lock()
|
||||||
chunkCalls := harness.chunkCalls
|
chunkCalls := harness.chunkCalls
|
||||||
|
sessions := append([]string(nil), harness.sessionIDs...)
|
||||||
harness.mu.Unlock()
|
harness.mu.Unlock()
|
||||||
if chunkCalls != 1 {
|
if chunkCalls != 1 {
|
||||||
t.Fatalf("chunk calls across changed run inputs = %d, want 1", chunkCalls)
|
t.Fatalf("chunk calls across changed run inputs = %d, want 1", chunkCalls)
|
||||||
}
|
}
|
||||||
|
rawInput, err := os.ReadFile(roots.input)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
wantSessionID, err := resolvePromptSessionID("", "test/input", rawInput)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, sessionID := range sessions {
|
||||||
|
if sessionID != wantSessionID {
|
||||||
|
t.Fatalf("session IDs across reference changes = %#v, want %q", sessions, wantSessionID)
|
||||||
|
}
|
||||||
|
}
|
||||||
assertFile(t, filepath.Join(roots.plans, strings.TrimPrefix(stateTestDigest, "sha256:"), "plan.json"))
|
assertFile(t, filepath.Join(roots.plans, strings.TrimPrefix(stateTestDigest, "sha256:"), "plan.json"))
|
||||||
assertAnyFile(t, roots.output)
|
assertAnyFile(t, roots.output)
|
||||||
}
|
}
|
||||||
|
|||||||
340
internal/cli/dnd_enemy_events_contract_test.go
Normal file
340
internal/cli/dnd_enemy_events_contract_test.go
Normal file
@@ -0,0 +1,340 @@
|
|||||||
|
package cli
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/evidencecontext"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/chunk/scenes"
|
||||||
|
locationoccurrencecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/locationoccurrences"
|
||||||
|
locationcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/locations"
|
||||||
|
combat "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
|
||||||
|
enemyevents "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/enemyevents"
|
||||||
|
itemevents "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/itemevents"
|
||||||
|
locationoccurrences "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/locationoccurrences"
|
||||||
|
locations "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/locations"
|
||||||
|
npcinteractions "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcinteractions"
|
||||||
|
npcs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcs"
|
||||||
|
scenedescriptions "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/scenedescriptions"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
|
||||||
|
enemyeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/enemyevents"
|
||||||
|
locationnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/locations"
|
||||||
|
npcnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/npcs"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestProductionEnemyEventConfigurationResolvesGeneratedHandoffs(t *testing.T) {
|
||||||
|
components := productionTestComponents(t)
|
||||||
|
cfg := loadMaintainedExample(t, repositoryPath("examples", "dnd-complete.config.yml"))
|
||||||
|
effective, err := cfg.Resolve(resolveInputForMaintainedExample(components, "dnd-session"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Resolve() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
materialized, _, err := pipeline.MaterializeReferences(effective.ResolvedPipeline, catalogFromRegistries(components.registries), pipeline.ReferenceMaterializationOptions{
|
||||||
|
ConfigPath: repositoryPath("examples", "dnd-complete.config.yml"),
|
||||||
|
WorkingDir: repositoryPath("examples"),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("MaterializeReferences() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
lane := referenceContractLane(t, materialized, "enemy-events")
|
||||||
|
if lane.ArtifactKind != dnd.EnemyEventListKind || lane.Extract.Module != enemyevents.Key || lane.Extract.Retries != 2 || lane.Merge.Module != pipeline.DefaultMergeModule || lane.Normalize.Module != enemyeventnormalize.Key {
|
||||||
|
t.Fatalf("enemy event lane = %#v, want typed production composition", lane)
|
||||||
|
}
|
||||||
|
for slot, want := range map[string]struct{ step, lane string }{
|
||||||
|
"npcs": {step: "describe-session", lane: "npcs"},
|
||||||
|
"scene_descriptions": {step: "describe-session", lane: "scene-descriptions"},
|
||||||
|
"combat_turns": {step: "extract-events", lane: "combat-turns"},
|
||||||
|
"npc_interactions": {step: "extract-events", lane: "npc-interactions"},
|
||||||
|
} {
|
||||||
|
binding, found := generatedReferenceBinding(lane.ExtractReferences.Bindings, slot)
|
||||||
|
if !found || binding.Artifact.Step != want.step || binding.Artifact.Lane != want.lane {
|
||||||
|
t.Fatalf("enemy event %s reference = %#v, want generated %s/%s artifact", slot, binding, want.step, want.lane)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if binding, found := generatedReferenceBinding(lane.NormalizeReferences.Bindings, "npcs"); !found || binding.Artifact.Step != "describe-session" || binding.Artifact.Lane != "npcs" {
|
||||||
|
t.Fatalf("enemy event normalizer NPC reference = %#v, want generated NPC artifact", binding)
|
||||||
|
}
|
||||||
|
|
||||||
|
catalog := catalogFromRegistries(components.registries)
|
||||||
|
extractSpec, ok := catalog.Extractors.Spec(enemyevents.Key)
|
||||||
|
if !ok || !reflect.DeepEqual(extractSpec.Requires, []string{"chunks", "source.transcript"}) || !reflect.DeepEqual(extractSpec.Provides, []string{"dnd.enemy_events"}) {
|
||||||
|
t.Fatalf("enemy event extractor spec = %#v, want source and artifact capabilities", extractSpec)
|
||||||
|
}
|
||||||
|
normalizeSpec, ok := catalog.Normalizers.SpecForArtifact(enemyeventnormalize.Key, dnd.EnemyEventListKind)
|
||||||
|
if !ok || !reflect.DeepEqual(normalizeSpec.Requires, []string{"merged"}) || !reflect.DeepEqual(normalizeSpec.Provides, []string{"normalized"}) {
|
||||||
|
t.Fatalf("enemy event normalizer spec = %#v, want merged/normalized capabilities", normalizeSpec)
|
||||||
|
}
|
||||||
|
for _, slot := range []string{"npcs", "scene_descriptions", "combat_turns", "npc_interactions"} {
|
||||||
|
if !hasReferenceSlot(extractSpec.ReferenceSlots, slot) {
|
||||||
|
t.Fatalf("enemy event extractor slots = %#v, want %q", extractSpec.ReferenceSlots, slot)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !hasReferenceSlot(normalizeSpec.ReferenceSlots, "npcs") {
|
||||||
|
t.Fatalf("enemy event normalizer slots = %#v, want NPC registry", normalizeSpec.ReferenceSlots)
|
||||||
|
}
|
||||||
|
|
||||||
|
profile := cfg.Pipelines["dnd-session"]
|
||||||
|
profile.Steps[2].References["npcs"] = pipeline.GeneratedReference("track-enemies", "enemy-events")
|
||||||
|
cfg.Pipelines["dnd-session"] = profile
|
||||||
|
if _, err := cfg.Resolve(resolveInputForMaintainedExample(components, "dnd-session")); err == nil || !strings.Contains(err.Error(), "earlier step") {
|
||||||
|
t.Fatalf("Resolve() error = %v, want future generated-reference rejection", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMaintainedCompleteExampleProducesEnemyEventsThroughGeneratedHandoffs(t *testing.T) {
|
||||||
|
t.Chdir(repositoryPath())
|
||||||
|
outputRoot := filepath.Join(t.TempDir(), "output")
|
||||||
|
configPath := completeExampleConfigWithTemporaryCache(t)
|
||||||
|
client := &enemyEventLLMClient{}
|
||||||
|
options := productionCLIOptions(t)
|
||||||
|
options.Now = func() time.Time { return time.Unix(1700000000, 0).UTC() }
|
||||||
|
options.RunIDGenerator = func(time.Time) (string, error) { return productionRunID, nil }
|
||||||
|
options.UserCacheDir = func() (string, error) { return "", errors.New("user cache must not be used") }
|
||||||
|
options.LLMClientFactory = func(context.Context, config.Config, string, LLMRuntimeOverrides) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error) {
|
||||||
|
return client, nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var stdout, stderr strings.Builder
|
||||||
|
code := RunWithOptions([]string{
|
||||||
|
"run", "dnd-session",
|
||||||
|
"--config", configPath,
|
||||||
|
"--input", repositoryPath("examples", "dnd-complete-transcript.json"),
|
||||||
|
"--chunk_cache", "bypass", "--output-dir", outputRoot, "--session-id", "enemy-event-session",
|
||||||
|
}, &stdout, &stderr, options)
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatalf("code=%d stdout=%q stderr=%q", code, stdout.String(), stderr.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
runRoot := filepath.Join(outputRoot, productionRunID)
|
||||||
|
index := readProductionJSON[exampleOutputIndex](t, filepath.Join(runRoot, "index.json"))
|
||||||
|
var enemyOutput exampleOutputIndexEntry
|
||||||
|
var locationOutput, occurrenceOutput exampleOutputIndexEntry
|
||||||
|
for _, entry := range index.OutputFiles {
|
||||||
|
switch entry.LaneID {
|
||||||
|
case "enemy-events":
|
||||||
|
enemyOutput = entry
|
||||||
|
case "locations":
|
||||||
|
locationOutput = entry
|
||||||
|
case "location-occurrences":
|
||||||
|
occurrenceOutput = entry
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if enemyOutput.File != "lanes/enemy-events.json" || enemyOutput.SchemaID != "notarius.dnd.enemy_events" || enemyOutput.SchemaVersion != "v1" {
|
||||||
|
t.Fatalf("enemy event output = %#v, want typed enemy-event JSON", enemyOutput)
|
||||||
|
}
|
||||||
|
value := readProductionJSON[dnd.EnemyEventList](t, filepath.Join(runRoot, enemyOutput.File))
|
||||||
|
if len(value.Events) != 1 || value.Events[0].Name != "Kesh" || value.Events[0].Kind != dnd.EnemyEventKindFled || len(value.Events[0].SourceRefs) != 1 || value.Events[0].SourceRefs[0].SourceID != "session-ravenfall" || value.Events[0].SourceRefs[0].StartUnitID != 10 {
|
||||||
|
t.Fatalf("enemy event artifact = %#v, want source-linked Kesh fleeing event", value)
|
||||||
|
}
|
||||||
|
if locationOutput.File != "lanes/locations.json" || locationOutput.SchemaID != locationcodec.SchemaID || locationOutput.SchemaVersion != locationcodec.SchemaVersion {
|
||||||
|
t.Fatalf("location output = %#v, want typed location registry JSON", locationOutput)
|
||||||
|
}
|
||||||
|
locationsValue := readProductionJSON[dnd.LocationList](t, filepath.Join(runRoot, locationOutput.File))
|
||||||
|
if len(locationsValue.Locations) != 2 || locationsValue.Locations[0].Name != "Moon Gate" || locationsValue.Locations[1].Name != "Moon Gate" || locationsValue.Locations[0].ID == locationsValue.Locations[1].ID {
|
||||||
|
t.Fatalf("location registry = %#v, want distinct source-grounded identities for same-name locations", locationsValue)
|
||||||
|
}
|
||||||
|
if occurrenceOutput.File != "lanes/location-occurrences.json" || occurrenceOutput.SchemaID != locationoccurrencecodec.SchemaID || occurrenceOutput.SchemaVersion != locationoccurrencecodec.SchemaVersion {
|
||||||
|
t.Fatalf("location occurrence output = %#v, want typed occurrence JSON", occurrenceOutput)
|
||||||
|
}
|
||||||
|
occurrencesValue := readProductionJSON[dnd.LocationOccurrenceList](t, filepath.Join(runRoot, occurrenceOutput.File))
|
||||||
|
if len(occurrencesValue.Occurrences) != 2 || occurrencesValue.Occurrences[0].LocationID == occurrencesValue.Occurrences[1].LocationID || occurrencesValue.Occurrences[0].Name != "Moon Gate" || occurrencesValue.Occurrences[1].Name != "Moon Gate" {
|
||||||
|
t.Fatalf("location occurrences = %#v, want source-grounded references to distinct registry identities", occurrencesValue)
|
||||||
|
}
|
||||||
|
|
||||||
|
evidence := readProductionJSON[evidencecontext.Document](t, filepath.Join(runRoot, "evidence-context.json"))
|
||||||
|
for _, laneID := range []string{"enemy-events", "locations", "location-occurrences"} {
|
||||||
|
if !containsString(evidence.SelectedLanes, laneID) || !evidenceHasLane(evidence, laneID) {
|
||||||
|
t.Fatalf("evidence context = %#v, want direct %s evidence", evidence, laneID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
requests := client.requestsFor(enemyevents.PromptID)
|
||||||
|
if len(requests) != 1 {
|
||||||
|
t.Fatalf("enemy event requests = %#v, want only the combat scene request", requests)
|
||||||
|
}
|
||||||
|
request := requests[0]
|
||||||
|
if request.SessionID != "enemy-event-session" {
|
||||||
|
t.Fatalf("enemy event session = %q, want shared session", request.SessionID)
|
||||||
|
}
|
||||||
|
for slot, required := range map[string]string{
|
||||||
|
"npcs": "Kesh",
|
||||||
|
"combat_turns": "Kesh",
|
||||||
|
"npc_interactions": "Kesh",
|
||||||
|
} {
|
||||||
|
input, ok := request.Inputs[slot]
|
||||||
|
if !ok || !strings.Contains(string(input.Content), required) || strings.Contains(string(input.Content), "source_refs") || strings.Contains(string(input.Content), "start_unit_id") {
|
||||||
|
t.Fatalf("enemy event %s prompt input = %q, want compact source-free grounding", slot, input.Content)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
locationRequests := client.requestsFor(locationoccurrences.PromptID)
|
||||||
|
if len(locationRequests) != 2 {
|
||||||
|
t.Fatalf("location occurrence requests = %#v, want one request per scene", locationRequests)
|
||||||
|
}
|
||||||
|
for _, request := range locationRequests {
|
||||||
|
registryInput := request.Inputs["locations"]
|
||||||
|
if !strings.Contains(string(registryInput.Content), "Moon Gate") || !strings.Contains(string(registryInput.Content), `"id"`) || strings.Contains(string(registryInput.Content), "source_refs") {
|
||||||
|
t.Fatalf("location occurrence registry input = %q, want source-free ID grounding", registryInput.Content)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func completeExampleConfigWithTemporaryCache(t *testing.T) string {
|
||||||
|
t.Helper()
|
||||||
|
content, err := os.ReadFile(repositoryPath("examples", "dnd-complete.config.yml"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cacheRoot := t.TempDir()
|
||||||
|
updated := strings.Replace(string(content), "directory: ./notarius-cache/chunk-plans", fmt.Sprintf("directory: %q", filepath.Join(cacheRoot, "chunk-plans")), 1)
|
||||||
|
updated = strings.Replace(updated, "directory: ./notarius-cache/checkpoints", fmt.Sprintf("directory: %q", filepath.Join(cacheRoot, "checkpoints")), 1)
|
||||||
|
for relative, absolute := range map[string]string{
|
||||||
|
"./dnd-party.txt": repositoryPath("examples", "dnd-party.txt"),
|
||||||
|
"./dnd-glossary.txt": repositoryPath("examples", "dnd-glossary.txt"),
|
||||||
|
"./dnd-spell-catalog.json": repositoryPath("examples", "dnd-spell-catalog.json"),
|
||||||
|
} {
|
||||||
|
updated = strings.ReplaceAll(updated, relative, fmt.Sprintf("%q", absolute))
|
||||||
|
}
|
||||||
|
path := filepath.Join(t.TempDir(), "dnd-complete.config.yml")
|
||||||
|
if err := os.WriteFile(path, []byte(updated), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return path
|
||||||
|
}
|
||||||
|
|
||||||
|
type enemyEventLLMClient struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
requests []contracts.StructuredCompletionRequest
|
||||||
|
}
|
||||||
|
|
||||||
|
func (client *enemyEventLLMClient) CompleteStructured(ctx context.Context, request contracts.StructuredCompletionRequest, output any) (contracts.StructuredCompletionResponse, error) {
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return contracts.StructuredCompletionResponse{}, err
|
||||||
|
}
|
||||||
|
combatScene := strings.Contains(string(request.Inputs["transcript"].Content), "Roll initiative")
|
||||||
|
var content []byte
|
||||||
|
switch request.PromptID {
|
||||||
|
case scenes.PromptID:
|
||||||
|
content = []byte(`{"scenes":[{"start_unit_id":1,"end_unit_id":6},{"start_unit_id":7,"end_unit_id":11}]}`)
|
||||||
|
case npcs.PromptID:
|
||||||
|
if combatScene {
|
||||||
|
content = []byte(`{"npcs":[{"name":"Kesh","source_refs":[{"start_unit_id":7,"end_unit_id":7}]}]}`)
|
||||||
|
} else {
|
||||||
|
content = []byte(`{"npcs":[]}`)
|
||||||
|
}
|
||||||
|
case npcnormalize.PromptID:
|
||||||
|
content = []byte(`{"duplicate_groups":[]}`)
|
||||||
|
case scenedescriptions.PromptID:
|
||||||
|
kind, title := "narrative", "Arrival"
|
||||||
|
if combatScene {
|
||||||
|
kind, title = "combat", "Raiders attack"
|
||||||
|
}
|
||||||
|
content = []byte(fmt.Sprintf(`{"kind":%q,"title":%q,"summary":"session scene"}`, kind, title))
|
||||||
|
case locations.PromptID:
|
||||||
|
unitID := 1
|
||||||
|
if combatScene {
|
||||||
|
unitID = 7
|
||||||
|
}
|
||||||
|
content = []byte(fmt.Sprintf(`{"locations":[{"name":"Moon Gate","source_refs":[{"start_unit_id":%d,"end_unit_id":%d}]}]}`, unitID, unitID))
|
||||||
|
case locationnormalize.PromptID:
|
||||||
|
content = []byte(`{"duplicate_groups":[]}`)
|
||||||
|
case spells.PromptID:
|
||||||
|
content = []byte(`{"spell_casts":[]}`)
|
||||||
|
case itemevents.PromptID:
|
||||||
|
content = []byte(`{"events":[]}`)
|
||||||
|
case combat.PromptID:
|
||||||
|
content = []byte(`{"combat_turns":[{"actor":"Kesh","turn_kind":"turn","source_refs":[{"start_unit_id":8,"end_unit_id":8}]}]}`)
|
||||||
|
case npcinteractions.PromptID:
|
||||||
|
if combatScene {
|
||||||
|
content = []byte(`{"interactions":[{"name":"Kesh","kind":"combat_opponent","source_refs":[{"start_unit_id":7,"end_unit_id":7}]}]}`)
|
||||||
|
} else {
|
||||||
|
content = []byte(`{"interactions":[]}`)
|
||||||
|
}
|
||||||
|
case locationoccurrences.PromptID:
|
||||||
|
var registry struct {
|
||||||
|
Locations []struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
} `json:"locations"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(request.Inputs["locations"].Content, ®istry); err != nil {
|
||||||
|
return contracts.StructuredCompletionResponse{}, fmt.Errorf("decode generated location registry: %w", err)
|
||||||
|
}
|
||||||
|
if len(registry.Locations) == 0 {
|
||||||
|
return contracts.StructuredCompletionResponse{}, fmt.Errorf("generated location registry has no locations")
|
||||||
|
}
|
||||||
|
unitID := 1
|
||||||
|
locationID := registry.Locations[0].ID
|
||||||
|
if combatScene {
|
||||||
|
unitID = 7
|
||||||
|
if len(registry.Locations) > 1 {
|
||||||
|
locationID = registry.Locations[1].ID
|
||||||
|
}
|
||||||
|
}
|
||||||
|
content = []byte(fmt.Sprintf(`{"occurrences":[{"location_id":%q,"name":"Moon Gate","kind":"visited","source_refs":[{"start_unit_id":%d,"end_unit_id":%d}]}]}`, locationID, unitID, unitID))
|
||||||
|
case enemyevents.PromptID:
|
||||||
|
content = []byte(`{"events":[{"name":"Kesh","kind":"fled","source_refs":[{"start_unit_id":10,"end_unit_id":10}]}]}`)
|
||||||
|
default:
|
||||||
|
return contracts.StructuredCompletionResponse{}, fmt.Errorf("unexpected prompt %q", request.PromptID)
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(content, output); err != nil {
|
||||||
|
return contracts.StructuredCompletionResponse{}, fmt.Errorf("populate fake structured target: %w", err)
|
||||||
|
}
|
||||||
|
client.mu.Lock()
|
||||||
|
client.requests = append(client.requests, request)
|
||||||
|
client.mu.Unlock()
|
||||||
|
return contracts.StructuredCompletionResponse{Content: content, Provider: "test", Model: "deterministic", ProfileID: request.ProfileID}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (client *enemyEventLLMClient) requestsFor(promptID string) []contracts.StructuredCompletionRequest {
|
||||||
|
client.mu.Lock()
|
||||||
|
defer client.mu.Unlock()
|
||||||
|
var requests []contracts.StructuredCompletionRequest
|
||||||
|
for _, request := range client.requests {
|
||||||
|
if request.PromptID == promptID {
|
||||||
|
requests = append(requests, request)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return requests
|
||||||
|
}
|
||||||
|
|
||||||
|
func containsString(values []string, want string) bool {
|
||||||
|
for _, value := range values {
|
||||||
|
if value == want {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
func evidenceHasLane(value evidencecontext.Document, laneID string) bool {
|
||||||
|
for _, context := range value.Contexts {
|
||||||
|
for _, reference := range context.EvidenceRefs {
|
||||||
|
if reference.LaneID == laneID {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
func generatedReferenceBinding(bindings []pipeline.ReferenceBinding, slotName string) (pipeline.ReferenceBinding, bool) {
|
||||||
|
for _, binding := range bindings {
|
||||||
|
if binding.SlotName == slotName && binding.Artifact != nil {
|
||||||
|
return binding, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return pipeline.ReferenceBinding{}, false
|
||||||
|
}
|
||||||
@@ -15,6 +15,10 @@ import (
|
|||||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
locationoccurrenceextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/locationoccurrences"
|
||||||
|
locationextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/locations"
|
||||||
|
locationoccurrencenormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/locationoccurrences"
|
||||||
|
locationnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/locations"
|
||||||
spellnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/spells"
|
spellnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/spells"
|
||||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/seriatim/input/transcript"
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/seriatim/input/transcript"
|
||||||
)
|
)
|
||||||
@@ -24,6 +28,9 @@ func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
|
|||||||
for _, example := range maintainedExampleFiles(t) {
|
for _, example := range maintainedExampleFiles(t) {
|
||||||
t.Run(example.name, func(t *testing.T) {
|
t.Run(example.name, func(t *testing.T) {
|
||||||
cfg := loadMaintainedExample(t, example.path)
|
cfg := loadMaintainedExample(t, example.path)
|
||||||
|
if example.name == "complete" && (cfg.Concurrency.TotalLLM != 2 || cfg.Concurrency.StageWorkers["extract"] != 2) {
|
||||||
|
t.Fatalf("complete example concurrency = %#v, want explicit limits of 2", cfg.Concurrency)
|
||||||
|
}
|
||||||
raw, err := os.ReadFile(example.transcriptPath)
|
raw, err := os.ReadFile(example.transcriptPath)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("read maintained transcript %q: %v", example.transcriptPath, err)
|
t.Fatalf("read maintained transcript %q: %v", example.transcriptPath, err)
|
||||||
@@ -51,8 +58,30 @@ func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
|
|||||||
t.Fatalf("materialize maintained example references for %q: %v", pipelineID, err)
|
t.Fatalf("materialize maintained example references for %q: %v", pipelineID, err)
|
||||||
}
|
}
|
||||||
if example.name == "complete" {
|
if example.name == "complete" {
|
||||||
if got := exampleStepLaneIDs(materialized); strings.Join(got, "|") != "describe-session:item-events,npcs,scene-descriptions|extract-events:combat-turns,npc-interactions,spells" {
|
if got := exampleStepLaneIDs(materialized); strings.Join(got, "|") != "describe-session:item-events,locations,npcs,scene-descriptions|extract-events:combat-turns,location-occurrences,npc-interactions,spells|track-enemies:enemy-events" {
|
||||||
t.Fatalf("complete example steps and lanes = %v, want every D&D extractor in the documented two-step composition", got)
|
t.Fatalf("complete example steps and lanes = %v, want the documented D&D extractor composition", got)
|
||||||
|
}
|
||||||
|
locationLane := referenceContractLane(t, materialized, "locations")
|
||||||
|
if locationLane.ArtifactKind != dnd.LocationListKind || locationLane.Extract.Module != locationextract.Key || locationLane.Extract.Retries != 2 || locationLane.Merge.Module != pipeline.DefaultMergeModule || locationLane.Normalize.Module != locationnormalize.Key || locationLane.Normalize.Retries != 2 {
|
||||||
|
t.Fatalf("location lane = %#v, want typed registry composition", locationLane)
|
||||||
|
}
|
||||||
|
occurrenceLane := referenceContractLane(t, materialized, "location-occurrences")
|
||||||
|
if occurrenceLane.ArtifactKind != dnd.LocationOccurrenceListKind || occurrenceLane.Extract.Module != locationoccurrenceextract.Key || occurrenceLane.Extract.Retries != 2 || occurrenceLane.Merge.Module != pipeline.DefaultMergeModule || occurrenceLane.Normalize.Module != locationoccurrencenormalize.Key {
|
||||||
|
t.Fatalf("location occurrence lane = %#v, want typed occurrence composition", occurrenceLane)
|
||||||
|
}
|
||||||
|
for _, target := range []pipeline.ResolvedReferenceTarget{occurrenceLane.ExtractReferences, occurrenceLane.NormalizeReferences} {
|
||||||
|
binding, found := generatedReferenceBinding(target.Bindings, "locations")
|
||||||
|
if !found || binding.Artifact.Step != "describe-session" || binding.Artifact.Lane != "locations" {
|
||||||
|
t.Fatalf("location occurrence %s reference = %#v, want generated location registry", target.Stage, binding)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, slot := range []string{"party", "glossary"} {
|
||||||
|
if len(occurrenceLane.ExtractReferences.ReferenceSet.Slots[slot].Items) != 1 {
|
||||||
|
t.Fatalf("location occurrence extractor %s reference was not materialized: %#v", slot, occurrenceLane.ExtractReferences)
|
||||||
|
}
|
||||||
|
if _, found := occurrenceLane.NormalizeReferences.ReferenceSet.Slots[slot]; found {
|
||||||
|
t.Fatalf("location occurrence normalizer unexpectedly consumes %s: %#v", slot, occurrenceLane.NormalizeReferences)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
spellLane := referenceContractLane(t, materialized, "spells")
|
spellLane := referenceContractLane(t, materialized, "spells")
|
||||||
if len(spellLane.ExtractReferences.ReferenceSet.Slots["spell_catalog"].Items) != 1 ||
|
if len(spellLane.ExtractReferences.ReferenceSet.Slots["spell_catalog"].Items) != 1 ||
|
||||||
@@ -68,6 +97,18 @@ func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
|
|||||||
t.Fatalf("item event lane unexpectedly depends on generated scene descriptions: %#v", itemEventLane)
|
t.Fatalf("item event lane unexpectedly depends on generated scene descriptions: %#v", itemEventLane)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
enemyEventLane := referenceContractLane(t, materialized, "enemy-events")
|
||||||
|
for slot, want := range map[string]struct{ step, lane string }{
|
||||||
|
"npcs": {step: "describe-session", lane: "npcs"},
|
||||||
|
"scene_descriptions": {step: "describe-session", lane: "scene-descriptions"},
|
||||||
|
"combat_turns": {step: "extract-events", lane: "combat-turns"},
|
||||||
|
"npc_interactions": {step: "extract-events", lane: "npc-interactions"},
|
||||||
|
} {
|
||||||
|
binding, found := generatedReferenceBinding(enemyEventLane.ExtractReferences.Bindings, slot)
|
||||||
|
if !found || binding.Artifact.Step != want.step || binding.Artifact.Lane != want.lane {
|
||||||
|
t.Fatalf("enemy event %s reference = %#v, want generated %s/%s artifact", slot, binding, want.step, want.lane)
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
var stdout, stderr strings.Builder
|
var stdout, stderr strings.Builder
|
||||||
@@ -242,6 +283,12 @@ func TestMaintainedMalformedInputOnlyRecordsDebugFailureWhenRequested(t *testing
|
|||||||
if report.Succeeded || report.PipelineID != "dnd-session" {
|
if report.Succeeded || report.PipelineID != "dnd-session" {
|
||||||
t.Fatalf("failure report = %#v, want failed dnd-session report", report)
|
t.Fatalf("failure report = %#v, want failed dnd-session report", report)
|
||||||
}
|
}
|
||||||
|
invocation := readProductionJSON[debugbundle.Invocation](t, filepath.Join(bundle, "summary", "invocation.json"))
|
||||||
|
manifest := readProductionJSON[artifacts.RunManifest](t, filepath.Join(bundle, "summary", "run-manifest.json"))
|
||||||
|
manifestSession, found := manifest.Metadata["session_id"]
|
||||||
|
if invocation.SessionID == "" || !found || manifestSession != invocation.SessionID {
|
||||||
|
t.Fatalf("failed run sessions: invocation=%q manifest=%#v metadata=%#v", invocation.SessionID, manifestSession, manifest.Metadata)
|
||||||
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -29,12 +29,15 @@ import (
|
|||||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/chunk/scenes"
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/chunk/scenes"
|
||||||
combatcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/combatturns"
|
combatcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/combatturns"
|
||||||
|
enemyeventcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/enemyevents"
|
||||||
itemeventcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/itemevents"
|
itemeventcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/itemevents"
|
||||||
spellcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/spells"
|
spellcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/spells"
|
||||||
combatextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
|
combatextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
|
||||||
|
enemyeventextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/enemyevents"
|
||||||
itemeventextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/itemevents"
|
itemeventextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/itemevents"
|
||||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
|
||||||
combatnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/combatturns"
|
combatnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/combatturns"
|
||||||
|
enemyeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/enemyevents"
|
||||||
itemeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/itemevents"
|
itemeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/itemevents"
|
||||||
spellnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/spells"
|
spellnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/spells"
|
||||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/generic/normalize/noop"
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/generic/normalize/noop"
|
||||||
@@ -47,9 +50,9 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
|||||||
|
|
||||||
assertProductionContains(t, "inputs", registries.Inputs.RegisteredKeys(), []string{"seriatim"})
|
assertProductionContains(t, "inputs", registries.Inputs.RegisteredKeys(), []string{"seriatim"})
|
||||||
assertProductionContains(t, "chunkers", registries.Chunkers.RegisteredKeys(), []string{"dnd/scenes", "generic"})
|
assertProductionContains(t, "chunkers", registries.Chunkers.RegisteredKeys(), []string{"dnd/scenes", "generic"})
|
||||||
assertProductionContains(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells", "dnd/npcs", combatextract.Key, itemeventextract.Key})
|
assertProductionContains(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells", "dnd/npcs", combatextract.Key, itemeventextract.Key, enemyeventextract.Key})
|
||||||
assertProductionContains(t, "mergers", registries.Mergers.RegisteredKeys(), []string{"appendorder"})
|
assertProductionContains(t, "mergers", registries.Mergers.RegisteredKeys(), []string{"appendorder"})
|
||||||
assertProductionContains(t, "normalizers", registries.Normalizers.RegisteredKeys(), []string{"noop", spellnormalize.Key, "dnd/npcs", combatnormalize.Key, itemeventnormalize.Key})
|
assertProductionContains(t, "normalizers", registries.Normalizers.RegisteredKeys(), []string{"noop", spellnormalize.Key, "dnd/npcs", combatnormalize.Key, itemeventnormalize.Key, enemyeventnormalize.Key})
|
||||||
assertProductionContains(t, "outputs", registries.Outputs.RegisteredKeys(), []string{"json"})
|
assertProductionContains(t, "outputs", registries.Outputs.RegisteredKeys(), []string{"json"})
|
||||||
assertProductionContains(t, "validators", registries.Validators.RegisteredKeys(), []string{
|
assertProductionContains(t, "validators", registries.Validators.RegisteredKeys(), []string{
|
||||||
"extract/dnd/spells/catalog",
|
"extract/dnd/spells/catalog",
|
||||||
@@ -64,17 +67,23 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
|||||||
"extract/dnd/item-events/source_refs",
|
"extract/dnd/item-events/source_refs",
|
||||||
"extract/dnd/item-events/source_relatedness",
|
"extract/dnd/item-events/source_relatedness",
|
||||||
"normalize/dnd/item-events/invariants",
|
"normalize/dnd/item-events/invariants",
|
||||||
|
"extract/dnd/enemy-events/shape",
|
||||||
|
"extract/dnd/enemy-events/engagements",
|
||||||
|
"extract/dnd/enemy-events/source_refs",
|
||||||
|
"extract/dnd/enemy-events/source_relatedness",
|
||||||
|
"normalize/dnd/enemy-events/invariants",
|
||||||
"generic/always_accept",
|
"generic/always_accept",
|
||||||
"generic/always_reject",
|
"generic/always_reject",
|
||||||
"generic/valid_json",
|
"generic/valid_json",
|
||||||
"generic/valid_json_schema",
|
"generic/valid_json_schema",
|
||||||
})
|
})
|
||||||
assertProductionContains(t, "artifact codec kinds", registries.ArtifactCodecs.RegisteredKinds(), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind})
|
assertProductionContains(t, "artifact codec kinds", registries.ArtifactCodecs.RegisteredKinds(), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind, dnd.EnemyEventListKind})
|
||||||
assertProductionContains(t, "merger variants", registries.Mergers.RegisteredArtifactKinds(pipeline.DefaultMergeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind})
|
assertProductionContains(t, "merger variants", registries.Mergers.RegisteredArtifactKinds(pipeline.DefaultMergeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind, dnd.EnemyEventListKind})
|
||||||
assertProductionContains(t, "normalizer variants", registries.Normalizers.RegisteredArtifactKinds(pipeline.DefaultNormalizeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind})
|
assertProductionContains(t, "normalizer variants", registries.Normalizers.RegisteredArtifactKinds(pipeline.DefaultNormalizeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind, dnd.EnemyEventListKind})
|
||||||
assertProductionContains(t, "spell normalizer variants", registries.Normalizers.RegisteredArtifactKinds(spellnormalize.Key), []contracts.ArtifactKind{dnd.SpellListKind})
|
assertProductionContains(t, "spell normalizer variants", registries.Normalizers.RegisteredArtifactKinds(spellnormalize.Key), []contracts.ArtifactKind{dnd.SpellListKind})
|
||||||
assertProductionContains(t, "combat normalizer variants", registries.Normalizers.RegisteredArtifactKinds(combatnormalize.Key), []contracts.ArtifactKind{dnd.CombatTurnListKind})
|
assertProductionContains(t, "combat normalizer variants", registries.Normalizers.RegisteredArtifactKinds(combatnormalize.Key), []contracts.ArtifactKind{dnd.CombatTurnListKind})
|
||||||
assertProductionContains(t, "item event normalizer variants", registries.Normalizers.RegisteredArtifactKinds(itemeventnormalize.Key), []contracts.ArtifactKind{dnd.ItemEventListKind})
|
assertProductionContains(t, "item event normalizer variants", registries.Normalizers.RegisteredArtifactKinds(itemeventnormalize.Key), []contracts.ArtifactKind{dnd.ItemEventListKind})
|
||||||
|
assertProductionContains(t, "enemy event normalizer variants", registries.Normalizers.RegisteredArtifactKinds(enemyeventnormalize.Key), []contracts.ArtifactKind{dnd.EnemyEventListKind})
|
||||||
|
|
||||||
wantChain := []pipeline.ModuleBinding{
|
wantChain := []pipeline.ModuleBinding{
|
||||||
pipeline.Binding("generic/valid_json"),
|
pipeline.Binding("generic/valid_json"),
|
||||||
@@ -132,6 +141,17 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
|||||||
if got := registries.ValidatorChains.Validators(pipeline.StageNormalize, itemeventnormalize.Key); !reflect.DeepEqual(got, itemEventNormalizeChain) {
|
if got := registries.ValidatorChains.Validators(pipeline.StageNormalize, itemeventnormalize.Key); !reflect.DeepEqual(got, itemEventNormalizeChain) {
|
||||||
t.Fatalf("item event normalize validator chain = %#v, want %#v", got, itemEventNormalizeChain)
|
t.Fatalf("item event normalize validator chain = %#v, want %#v", got, itemEventNormalizeChain)
|
||||||
}
|
}
|
||||||
|
enemyEventExtractChain := []pipeline.ModuleBinding{
|
||||||
|
pipeline.Binding("generic/valid_json"),
|
||||||
|
pipeline.Binding("extract/dnd/enemy-events/shape"),
|
||||||
|
pipeline.Binding("extract/dnd/enemy-events/engagements"),
|
||||||
|
pipeline.Binding("extract/dnd/enemy-events/source_refs"),
|
||||||
|
pipeline.Binding("generic/valid_json_schema"),
|
||||||
|
pipeline.Binding("extract/dnd/enemy-events/source_relatedness"),
|
||||||
|
}
|
||||||
|
if got := registries.ValidatorChains.Validators(pipeline.StageExtract, enemyeventextract.Key); !reflect.DeepEqual(got, enemyEventExtractChain) {
|
||||||
|
t.Fatalf("enemy event extract validator chain = %#v, want %#v", got, enemyEventExtractChain)
|
||||||
|
}
|
||||||
|
|
||||||
assetNames := productionAssetNames(t, components.assets.PromptFS)
|
assetNames := productionAssetNames(t, components.assets.PromptFS)
|
||||||
requiredAssets := []string{
|
requiredAssets := []string{
|
||||||
@@ -162,6 +182,10 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
|||||||
"dnd.item_events/sharedassets/common-dnd-system.md",
|
"dnd.item_events/sharedassets/common-dnd-system.md",
|
||||||
"dnd.item_events/sharedassets/common-dnd-transcript.md",
|
"dnd.item_events/sharedassets/common-dnd-transcript.md",
|
||||||
"dnd.item_events/task.md",
|
"dnd.item_events/task.md",
|
||||||
|
"dnd.enemy_events/dnd.enemy_events.yaml",
|
||||||
|
"dnd.enemy_events/grounding.md",
|
||||||
|
"dnd.enemy_events/instructions.md",
|
||||||
|
"dnd.enemy_events/task.md",
|
||||||
}
|
}
|
||||||
assertProductionContains(t, "production prompt assets", assetNames, requiredAssets)
|
assertProductionContains(t, "production prompt assets", assetNames, requiredAssets)
|
||||||
|
|
||||||
@@ -180,6 +204,7 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
|||||||
{stage: pipeline.StageExtract, key: "dnd/item-events", want: contracts.ExecutionClassLLMBacked},
|
{stage: pipeline.StageExtract, key: "dnd/item-events", want: contracts.ExecutionClassLLMBacked},
|
||||||
{stage: pipeline.StageExtract, key: "dnd/npc-interactions", want: contracts.ExecutionClassLLMBacked},
|
{stage: pipeline.StageExtract, key: "dnd/npc-interactions", want: contracts.ExecutionClassLLMBacked},
|
||||||
{stage: pipeline.StageExtract, key: "dnd/scene-descriptions", want: contracts.ExecutionClassLLMBacked},
|
{stage: pipeline.StageExtract, key: "dnd/scene-descriptions", want: contracts.ExecutionClassLLMBacked},
|
||||||
|
{stage: pipeline.StageExtract, key: enemyeventextract.Key, want: contracts.ExecutionClassLLMBacked},
|
||||||
{stage: pipeline.StageMerge, key: "appendorder", want: contracts.ExecutionClassDeterministic},
|
{stage: pipeline.StageMerge, key: "appendorder", want: contracts.ExecutionClassDeterministic},
|
||||||
{stage: pipeline.StageNormalize, key: "noop", want: contracts.ExecutionClassDeterministic},
|
{stage: pipeline.StageNormalize, key: "noop", want: contracts.ExecutionClassDeterministic},
|
||||||
{stage: pipeline.StageNormalize, key: "dnd/spells", want: contracts.ExecutionClassDeterministic},
|
{stage: pipeline.StageNormalize, key: "dnd/spells", want: contracts.ExecutionClassDeterministic},
|
||||||
@@ -188,6 +213,7 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
|||||||
{stage: pipeline.StageNormalize, key: "dnd/item-events", want: contracts.ExecutionClassDeterministic},
|
{stage: pipeline.StageNormalize, key: "dnd/item-events", want: contracts.ExecutionClassDeterministic},
|
||||||
{stage: pipeline.StageNormalize, key: "dnd/npc-interactions", want: contracts.ExecutionClassDeterministic},
|
{stage: pipeline.StageNormalize, key: "dnd/npc-interactions", want: contracts.ExecutionClassDeterministic},
|
||||||
{stage: pipeline.StageNormalize, key: "dnd/scene-descriptions", want: contracts.ExecutionClassDeterministic},
|
{stage: pipeline.StageNormalize, key: "dnd/scene-descriptions", want: contracts.ExecutionClassDeterministic},
|
||||||
|
{stage: pipeline.StageNormalize, key: enemyeventnormalize.Key, want: contracts.ExecutionClassDeterministic},
|
||||||
{stage: pipeline.StageOutput, key: "json", want: contracts.ExecutionClassDeterministic},
|
{stage: pipeline.StageOutput, key: "json", want: contracts.ExecutionClassDeterministic},
|
||||||
} {
|
} {
|
||||||
got, ok := catalog.ExecutionClass(test.stage, test.key)
|
got, ok := catalog.ExecutionClass(test.stage, test.key)
|
||||||
@@ -211,6 +237,10 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
|||||||
if !ok || itemEventCodecSpec.Kind != dnd.ItemEventListKind || itemEventCodecSpec.Schema.ID != itemeventcodec.SchemaID {
|
if !ok || itemEventCodecSpec.Kind != dnd.ItemEventListKind || itemEventCodecSpec.Schema.ID != itemeventcodec.SchemaID {
|
||||||
t.Fatalf("item event codec spec = %#v, ok=%t, want typed D&D item-event codec", itemEventCodecSpec, ok)
|
t.Fatalf("item event codec spec = %#v, ok=%t, want typed D&D item-event codec", itemEventCodecSpec, ok)
|
||||||
}
|
}
|
||||||
|
enemyEventCodecSpec, ok := catalog.ArtifactCodecs.Spec(dnd.EnemyEventListKind)
|
||||||
|
if !ok || enemyEventCodecSpec.Kind != dnd.EnemyEventListKind || enemyEventCodecSpec.Schema.ID != enemyeventcodec.SchemaID {
|
||||||
|
t.Fatalf("enemy event codec spec = %#v, ok=%t, want typed D&D enemy-event codec", enemyEventCodecSpec, ok)
|
||||||
|
}
|
||||||
if got := catalog.ValidatorChains.Validators(pipeline.StageExtract, spells.Key); !reflect.DeepEqual(got, wantChain) {
|
if got := catalog.ValidatorChains.Validators(pipeline.StageExtract, spells.Key); !reflect.DeepEqual(got, wantChain) {
|
||||||
t.Fatalf("catalog validator chain = %#v, want %#v", got, wantChain)
|
t.Fatalf("catalog validator chain = %#v, want %#v", got, wantChain)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -153,10 +153,10 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
|
|||||||
resume := fs.Bool("resume", false, "reuse compatible recorded checkpoints")
|
resume := fs.Bool("resume", false, "reuse compatible recorded checkpoints")
|
||||||
recomputeStep := singleValueFlag{name: "--recompute-step"}
|
recomputeStep := singleValueFlag{name: "--recompute-step"}
|
||||||
chunkCache := chunkCacheFlag{}
|
chunkCache := chunkCacheFlag{}
|
||||||
sessionID := sessionIDFlag{}
|
requestedSessionID := sessionIDFlag{}
|
||||||
referenceFlags := stringListFlag{}
|
referenceFlags := stringListFlag{}
|
||||||
withoutReferenceFlags := stringListFlag{}
|
withoutReferenceFlags := stringListFlag{}
|
||||||
fs.Var(&sessionID, "session-id", "prompt session identifier")
|
fs.Var(&requestedSessionID, "session-id", "prompt session identifier")
|
||||||
fs.Var(&reasoningEffort, "reasoning-effort", "reasoning effort override")
|
fs.Var(&reasoningEffort, "reasoning-effort", "reasoning effort override")
|
||||||
fs.Var(&chunkCache, "chunk_cache", "chunk plan cache mode: auto, bypass, or refresh")
|
fs.Var(&chunkCache, "chunk_cache", "chunk plan cache mode: auto, bypass, or refresh")
|
||||||
fs.Var(&referenceFlags, "reference", "reference binding, as slot=path, chunk.slot=path, merge.slot=path, lane.slot=path, lane.extract.slot=path, lane.merge.slot=path, or lane.normalize.slot=path")
|
fs.Var(&referenceFlags, "reference", "reference binding, as slot=path, chunk.slot=path, merge.slot=path, lane.slot=path, lane.extract.slot=path, lane.merge.slot=path, or lane.normalize.slot=path")
|
||||||
@@ -199,7 +199,7 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
|
|||||||
fmt.Fprintln(stderr, "notarius: --debug-dir must not be empty")
|
fmt.Fprintln(stderr, "notarius: --debug-dir must not be empty")
|
||||||
return 2
|
return 2
|
||||||
}
|
}
|
||||||
if sessionID.set && strings.TrimSpace(sessionID.value) == "" {
|
if requestedSessionID.set && strings.TrimSpace(requestedSessionID.value) == "" {
|
||||||
fmt.Fprintln(stderr, "notarius: --session-id must not be empty")
|
fmt.Fprintln(stderr, "notarius: --session-id must not be empty")
|
||||||
return 2
|
return 2
|
||||||
}
|
}
|
||||||
@@ -414,11 +414,19 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("read input %q: %w", strings.TrimSpace(*inputPath), err))
|
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("read input %q: %w", strings.TrimSpace(*inputPath), err))
|
||||||
}
|
}
|
||||||
|
effectiveSessionID, err := resolvePromptSessionID(requestedSessionID.value, effective.ResolvedPipeline.Input.Module, rawInput)
|
||||||
|
if err != nil {
|
||||||
|
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
||||||
|
}
|
||||||
|
invocation.SessionID = effectiveSessionID
|
||||||
|
if err := writeSummary(summary, func() error { return summary.WriteInvocation(invocation) }); err != nil {
|
||||||
|
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("write debug invocation metadata: %w", err))
|
||||||
|
}
|
||||||
chunkPlans, err := chunkPlanStoreForRun(effective.Config.Cache.ChunkPlans, opts)
|
chunkPlans, err := chunkPlanStoreForRun(effective.Config.Cache.ChunkPlans, opts)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
||||||
}
|
}
|
||||||
checkpointRecorder, checkpointLoader, err := checkpointHandlersForRun(effective.Config.Cache.Checkpoints, opts, effective.ResolvedPipeline, prepared.CheckpointFingerprints(), llmFingerprints, rawInput, only, llmProfiles, strings.TrimSpace(*llmProfile), strings.TrimSpace(sessionID.value), runtimeOverrides, *resume)
|
checkpointRecorder, checkpointLoader, err := checkpointHandlersForRun(effective.Config.Cache.Checkpoints, opts, effective.ResolvedPipeline, prepared.CheckpointFingerprints(), llmFingerprints, rawInput, only, llmProfiles, strings.TrimSpace(*llmProfile), effectiveSessionID, runtimeOverrides, *resume)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
||||||
}
|
}
|
||||||
@@ -427,7 +435,7 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
|
|||||||
Prepared: prepared,
|
Prepared: prepared,
|
||||||
Path: strings.TrimSpace(*inputPath),
|
Path: strings.TrimSpace(*inputPath),
|
||||||
RawInput: rawInput,
|
RawInput: rawInput,
|
||||||
SessionID: strings.TrimSpace(sessionID.value),
|
SessionID: effectiveSessionID,
|
||||||
RunID: runID,
|
RunID: runID,
|
||||||
StartedAt: startedAt,
|
StartedAt: startedAt,
|
||||||
LLMProfiles: llmProfiles,
|
LLMProfiles: llmProfiles,
|
||||||
|
|||||||
@@ -492,19 +492,30 @@ func TestEffectiveLLMProfileIDsAreSortedDeduplicatedAndLLMOnly(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRunSessionIDUsesExplicitValueOrSourceDocumentID(t *testing.T) {
|
func TestRunSessionIDUsesEffectiveValueForPromptRequests(t *testing.T) {
|
||||||
for _, tt := range []struct {
|
for _, tt := range []struct {
|
||||||
name string
|
name string
|
||||||
args []string
|
args []string
|
||||||
want string
|
want string
|
||||||
}{
|
}{
|
||||||
{name: "source document", want: "source"},
|
{name: "derived default"},
|
||||||
{name: "explicit trimmed value", args: []string{"--session-id", " explicit-session "}, want: "explicit-session"},
|
{name: "explicit trimmed value", args: []string{"--session-id", " explicit-session "}, want: "explicit-session"},
|
||||||
} {
|
} {
|
||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
roots := newStateTestRoots(t)
|
roots := newStateTestRoots(t)
|
||||||
|
want := tt.want
|
||||||
|
if want == "" {
|
||||||
|
rawInput, err := os.ReadFile(roots.input)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
want, err = resolvePromptSessionID("", "test/input", rawInput)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
harness := newStateTestHarness()
|
harness := newStateTestHarness()
|
||||||
args := append([]string{"run", "sample", "--config", roots.config, "--input", roots.input, "--chunk_cache", "bypass"}, tt.args...)
|
args := append([]string{"run", "sample", "--config", roots.config, "--input", roots.input, "--chunk_cache", "bypass", "--debug"}, tt.args...)
|
||||||
var stdout, stderr bytes.Buffer
|
var stdout, stderr bytes.Buffer
|
||||||
code := RunWithOptions(args, &stdout, &stderr, harness.options())
|
code := RunWithOptions(args, &stdout, &stderr, harness.options())
|
||||||
if code != 0 || stderr.Len() != 0 {
|
if code != 0 || stderr.Len() != 0 {
|
||||||
@@ -517,10 +528,15 @@ func TestRunSessionIDUsesExplicitValueOrSourceDocumentID(t *testing.T) {
|
|||||||
t.Fatalf("session IDs = %#v, want all prompt-facing module requests", sessions)
|
t.Fatalf("session IDs = %#v, want all prompt-facing module requests", sessions)
|
||||||
}
|
}
|
||||||
for _, session := range sessions {
|
for _, session := range sessions {
|
||||||
if session != tt.want {
|
if session != want {
|
||||||
t.Fatalf("session IDs = %#v, want %q", sessions, tt.want)
|
t.Fatalf("session IDs = %#v, want %q", sessions, want)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
var manifest artifacts.RunManifest
|
||||||
|
readStateTestSummaryJSON(t, onlyChildDir(t, roots.debug), "run-manifest.json", &manifest)
|
||||||
|
if session, ok := manifest.Metadata["session_id"]; !ok || session != want {
|
||||||
|
t.Fatalf("manifest session = %#v, want %q; metadata = %#v", session, want, manifest.Metadata)
|
||||||
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
26
internal/cli/session.go
Normal file
26
internal/cli/session.go
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
package cli
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
const generatedSessionIDPrefix = "notarius:v1:"
|
||||||
|
|
||||||
|
func resolvePromptSessionID(explicitSessionID, inputModule string, rawInput []byte) (string, error) {
|
||||||
|
inputModule = strings.TrimSpace(inputModule)
|
||||||
|
if inputModule == "" {
|
||||||
|
return "", fmt.Errorf("resolve prompt session: input module key must not be empty")
|
||||||
|
}
|
||||||
|
if sessionID := strings.TrimSpace(explicitSessionID); sessionID != "" {
|
||||||
|
return sessionID, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
hasher := sha256.New()
|
||||||
|
_, _ = hasher.Write([]byte(inputModule))
|
||||||
|
_, _ = hasher.Write([]byte{0})
|
||||||
|
_, _ = hasher.Write(rawInput)
|
||||||
|
return generatedSessionIDPrefix + hex.EncodeToString(hasher.Sum(nil)), nil
|
||||||
|
}
|
||||||
61
internal/cli/session_test.go
Normal file
61
internal/cli/session_test.go
Normal file
@@ -0,0 +1,61 @@
|
|||||||
|
package cli
|
||||||
|
|
||||||
|
import "testing"
|
||||||
|
|
||||||
|
func TestResolvePromptSessionIDUsesVersionedInputIdentity(t *testing.T) {
|
||||||
|
got, err := resolvePromptSessionID("", " seriatim/input/transcript ", []byte("{\"entries\":[\"one\"]}\n"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
const want = "notarius:v1:e15fefdca48653e73248b4157900547be1fd34250c0138f8d3d1f2e89b43bb25"
|
||||||
|
if got != want {
|
||||||
|
t.Fatalf("resolved session = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolvePromptSessionIDStabilityAndOverride(t *testing.T) {
|
||||||
|
rawInput := []byte("same input")
|
||||||
|
baseline, err := resolvePromptSessionID("", "input/transcript", rawInput)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
repeated, err := resolvePromptSessionID("", "input/transcript", rawInput)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if baseline != repeated {
|
||||||
|
t.Fatalf("resolved sessions = %q and %q, want stable value", baseline, repeated)
|
||||||
|
}
|
||||||
|
|
||||||
|
differentModule, err := resolvePromptSessionID("", "input/other", rawInput)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if baseline == differentModule {
|
||||||
|
t.Fatalf("resolved sessions = %q for distinct input modules", baseline)
|
||||||
|
}
|
||||||
|
|
||||||
|
differentInput, err := resolvePromptSessionID("", "input/transcript", []byte("same inpuu"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if baseline == differentInput {
|
||||||
|
t.Fatalf("resolved sessions = %q for distinct input bytes", baseline)
|
||||||
|
}
|
||||||
|
|
||||||
|
override, err := resolvePromptSessionID(" explicit-session ", "input/other", []byte("different input"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if override != "explicit-session" {
|
||||||
|
t.Fatalf("resolved override = %q, want %q", override, "explicit-session")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolvePromptSessionIDRejectsEmptyInputModule(t *testing.T) {
|
||||||
|
for _, explicitSessionID := range []string{"", "explicit-session"} {
|
||||||
|
if _, err := resolvePromptSessionID(explicitSessionID, " \t", []byte("input")); err == nil {
|
||||||
|
t.Fatalf("resolvePromptSessionID(%q) error = nil, want empty module failure", explicitSessionID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -329,11 +329,25 @@ func TestRunUsesOneInjectedIdentityForDebugOutputAndManifest(t *testing.T) {
|
|||||||
if manifest.StartedAt == nil || !manifest.StartedAt.Equal(wantStartedAt) {
|
if manifest.StartedAt == nil || !manifest.StartedAt.Equal(wantStartedAt) {
|
||||||
t.Fatalf("manifest started at = %v, want %v", manifest.StartedAt, wantStartedAt)
|
t.Fatalf("manifest started at = %v, want %v", manifest.StartedAt, wantStartedAt)
|
||||||
}
|
}
|
||||||
|
rawInput, err := os.ReadFile(roots.input)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
wantSessionID, err := resolvePromptSessionID("", "test/input", rawInput)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if sessionID, ok := manifest.Metadata["session_id"]; !ok || sessionID != wantSessionID {
|
||||||
|
t.Fatalf("manifest session = %#v, want %q; metadata = %#v", sessionID, wantSessionID, manifest.Metadata)
|
||||||
|
}
|
||||||
var invocation debugbundle.Invocation
|
var invocation debugbundle.Invocation
|
||||||
readStateTestSummaryJSON(t, debugPath, "invocation.json", &invocation)
|
readStateTestSummaryJSON(t, debugPath, "invocation.json", &invocation)
|
||||||
if invocation.RunID != runID || !invocation.StartedAt.Equal(wantStartedAt) {
|
if invocation.RunID != runID || !invocation.StartedAt.Equal(wantStartedAt) {
|
||||||
t.Fatalf("debug invocation identity = %#v, want run %q at %v", invocation, runID, wantStartedAt)
|
t.Fatalf("debug invocation identity = %#v, want run %q at %v", invocation, runID, wantStartedAt)
|
||||||
}
|
}
|
||||||
|
if invocation.SessionID != wantSessionID {
|
||||||
|
t.Fatalf("debug invocation session = %q, want %q", invocation.SessionID, wantSessionID)
|
||||||
|
}
|
||||||
report := readStateTestRunReport(t, debugPath)
|
report := readStateTestRunReport(t, debugPath)
|
||||||
if !report.Succeeded || report.RunID != runID || report.PipelineID != "sample" || report.OutputPath != outputPath || report.DebugPath != debugPath || report.OutputCount != 1 || report.RejectedCount != 0 || report.WarningCount != 0 || report.ValidationStatus != "approved" {
|
if !report.Succeeded || report.RunID != runID || report.PipelineID != "sample" || report.OutputPath != outputPath || report.DebugPath != debugPath || report.OutputCount != 1 || report.RejectedCount != 0 || report.WarningCount != 0 || report.ValidationStatus != "approved" {
|
||||||
t.Fatalf("success report = %#v", report)
|
t.Fatalf("success report = %#v", report)
|
||||||
@@ -343,6 +357,45 @@ func TestRunUsesOneInjectedIdentityForDebugOutputAndManifest(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestRunCheckpointReuseRequiresSameEffectiveSession(t *testing.T) {
|
||||||
|
roots := newStateTestRoots(t)
|
||||||
|
harness := newStateTestHarness()
|
||||||
|
run := func(extra ...string) stateTestResult {
|
||||||
|
args := []string{"run", "sample", "--config", roots.config, "--input", roots.input, "--chunk_cache", "bypass"}
|
||||||
|
args = append(args, extra...)
|
||||||
|
var stdout, stderr bytes.Buffer
|
||||||
|
return stateTestResult{code: RunWithOptions(args, &stdout, &stderr, harness.options()), stdout: stdout.String(), stderr: stderr.String()}
|
||||||
|
}
|
||||||
|
|
||||||
|
if result := run(); result.code != 0 {
|
||||||
|
t.Fatalf("initial run code=%d stdout=%q stderr=%q", result.code, result.stdout, result.stderr)
|
||||||
|
}
|
||||||
|
harness.mu.Lock()
|
||||||
|
initialExtractCalls := harness.extractCalls
|
||||||
|
harness.mu.Unlock()
|
||||||
|
if initialExtractCalls != 1 {
|
||||||
|
t.Fatalf("initial extract calls = %d, want 1", initialExtractCalls)
|
||||||
|
}
|
||||||
|
if result := run("--resume"); result.code != 0 {
|
||||||
|
t.Fatalf("same-session resume code=%d stdout=%q stderr=%q", result.code, result.stdout, result.stderr)
|
||||||
|
}
|
||||||
|
harness.mu.Lock()
|
||||||
|
reusedExtractCalls := harness.extractCalls
|
||||||
|
harness.mu.Unlock()
|
||||||
|
if reusedExtractCalls != initialExtractCalls {
|
||||||
|
t.Fatalf("extract calls after same-session resume = %d, want %d", reusedExtractCalls, initialExtractCalls)
|
||||||
|
}
|
||||||
|
if result := run("--resume", "--session-id", "different-session"); result.code != 0 {
|
||||||
|
t.Fatalf("different-session resume code=%d stdout=%q stderr=%q", result.code, result.stdout, result.stderr)
|
||||||
|
}
|
||||||
|
harness.mu.Lock()
|
||||||
|
differentSessionExtractCalls := harness.extractCalls
|
||||||
|
harness.mu.Unlock()
|
||||||
|
if differentSessionExtractCalls != initialExtractCalls+1 {
|
||||||
|
t.Fatalf("extract calls after different-session resume = %d, want %d", differentSessionExtractCalls, initialExtractCalls+1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestRunWritesTerminalArtifactsForResolutionPipelineAndOutputFailures(t *testing.T) {
|
func TestRunWritesTerminalArtifactsForResolutionPipelineAndOutputFailures(t *testing.T) {
|
||||||
for _, tc := range []struct {
|
for _, tc := range []struct {
|
||||||
name string
|
name string
|
||||||
|
|||||||
@@ -34,6 +34,8 @@ type ConcurrencyConfig struct {
|
|||||||
defaultedExtractWorkers int
|
defaultedExtractWorkers int
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const defaultLLMConcurrency = 16
|
||||||
|
|
||||||
type OutputConfig struct {
|
type OutputConfig struct {
|
||||||
Directory string `json:"directory"`
|
Directory string `json:"directory"`
|
||||||
}
|
}
|
||||||
@@ -60,9 +62,9 @@ func Default() Config {
|
|||||||
return Config{
|
return Config{
|
||||||
Pipelines: map[string]pipeline.PipelineProfile{},
|
Pipelines: map[string]pipeline.PipelineProfile{},
|
||||||
Concurrency: ConcurrencyConfig{
|
Concurrency: ConcurrencyConfig{
|
||||||
TotalLLM: 1,
|
TotalLLM: defaultLLMConcurrency,
|
||||||
StageWorkers: map[string]int{"extract": 1},
|
StageWorkers: map[string]int{"extract": defaultLLMConcurrency},
|
||||||
defaultedExtractWorkers: 1,
|
defaultedExtractWorkers: defaultLLMConcurrency,
|
||||||
},
|
},
|
||||||
Output: OutputConfig{Directory: "./notarius-output"},
|
Output: OutputConfig{Directory: "./notarius-output"},
|
||||||
Cache: CacheConfig{ChunkPlans: ChunkPlanCacheConfig{Mode: pipeline.ChunkCacheAuto}},
|
Cache: CacheConfig{ChunkPlans: ChunkPlanCacheConfig{Mode: pipeline.ChunkCacheAuto}},
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ import (
|
|||||||
|
|
||||||
func TestDefaultReturnsDocumentedValuesAndIndependentMaps(t *testing.T) {
|
func TestDefaultReturnsDocumentedValuesAndIndependentMaps(t *testing.T) {
|
||||||
first := Default()
|
first := Default()
|
||||||
if first.Concurrency.TotalLLM != 1 || first.Concurrency.StageWorkers["extract"] != 1 {
|
if first.Concurrency.TotalLLM != 16 || first.Concurrency.StageWorkers["extract"] != 16 {
|
||||||
t.Fatalf("concurrency defaults = %#v", first.Concurrency)
|
t.Fatalf("concurrency defaults = %#v", first.Concurrency)
|
||||||
}
|
}
|
||||||
if first.Output.Directory != "./notarius-output" || first.Debug.Directory != "./notarius-debug" {
|
if first.Output.Directory != "./notarius-output" || first.Debug.Directory != "./notarius-debug" {
|
||||||
@@ -31,7 +31,7 @@ func TestDefaultReturnsDocumentedValuesAndIndependentMaps(t *testing.T) {
|
|||||||
first.Concurrency.StageWorkers["other"] = 100
|
first.Concurrency.StageWorkers["other"] = 100
|
||||||
first.Pipelines["changed"] = pipeline.PipelineProfile{}
|
first.Pipelines["changed"] = pipeline.PipelineProfile{}
|
||||||
second := Default()
|
second := Default()
|
||||||
if second.Concurrency.StageWorkers["extract"] != 1 || len(second.Concurrency.StageWorkers) != 1 || len(second.Pipelines) != 0 {
|
if second.Concurrency.StageWorkers["extract"] != 16 || len(second.Concurrency.StageWorkers) != 1 || len(second.Pipelines) != 0 {
|
||||||
t.Fatalf("Default() returned state shared with an earlier result: %#v", second)
|
t.Fatalf("Default() returned state shared with an earlier result: %#v", second)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -45,7 +45,7 @@ func TestFileConfigMinimalVersion4AppliesOverDefaults(t *testing.T) {
|
|||||||
if cfg.Output.Directory != "./notarius-output" || cfg.Debug.Directory != "./notarius-debug" || cfg.Cache.ChunkPlans.Mode != pipeline.ChunkCacheAuto {
|
if cfg.Output.Directory != "./notarius-output" || cfg.Debug.Directory != "./notarius-debug" || cfg.Cache.ChunkPlans.Mode != pipeline.ChunkCacheAuto {
|
||||||
t.Fatalf("minimal file changed unrelated defaults: %#v", cfg)
|
t.Fatalf("minimal file changed unrelated defaults: %#v", cfg)
|
||||||
}
|
}
|
||||||
if cfg.Concurrency.TotalLLM != 1 || cfg.Concurrency.StageWorkers["extract"] != 1 || len(cfg.Pipelines) != 0 {
|
if cfg.Concurrency.TotalLLM != 16 || cfg.Concurrency.StageWorkers["extract"] != 16 || len(cfg.Pipelines) != 0 {
|
||||||
t.Fatalf("minimal file did not retain defaults: %#v", cfg)
|
t.Fatalf("minimal file did not retain defaults: %#v", cfg)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -179,6 +179,30 @@ func TestWriteInvocationPreservesReasoningEffortOverrideStates(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestWriteInvocationOmitsEmptySessionID(t *testing.T) {
|
||||||
|
for _, sessionID := range []string{"", "session-123"} {
|
||||||
|
bundle, err := Allocate(t.TempDir(), testBundleRunID, time.Unix(0, 42))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := bundle.Summary().WriteInvocation(Invocation{Operation: "run", SessionID: sessionID}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
data, err := os.ReadFile(filepath.Join(bundle.SummaryRoot(), ArtifactInvocationMetadata))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var payload map[string]any
|
||||||
|
if err := json.Unmarshal(data, &payload); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
value, found := payload["session_id"]
|
||||||
|
if found != (sessionID != "") || (found && value != sessionID) {
|
||||||
|
t.Fatalf("session found=%t value=%#v, want found=%t value=%q; JSON=%s", found, value, sessionID != "", sessionID, data)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
func TestSummaryWriterInternalWritesConfineArtifacts(t *testing.T) {
|
func TestSummaryWriterInternalWritesConfineArtifacts(t *testing.T) {
|
||||||
bundle, err := Allocate(t.TempDir(), testBundleRunID, time.Unix(0, 42))
|
bundle, err := Allocate(t.TempDir(), testBundleRunID, time.Unix(0, 42))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -40,6 +40,7 @@ type Invocation struct {
|
|||||||
OnlyLanes []string `json:"only_lanes,omitempty"`
|
OnlyLanes []string `json:"only_lanes,omitempty"`
|
||||||
ChunkCacheOverride string `json:"chunk_cache_override,omitempty"`
|
ChunkCacheOverride string `json:"chunk_cache_override,omitempty"`
|
||||||
ReasoningEffortOverride *string `json:"reasoning_effort_override,omitempty"`
|
ReasoningEffortOverride *string `json:"reasoning_effort_override,omitempty"`
|
||||||
|
SessionID string `json:"session_id,omitempty"`
|
||||||
RunID string `json:"run_id"`
|
RunID string `json:"run_id"`
|
||||||
StartedAt time.Time `json:"started_at"`
|
StartedAt time.Time `json:"started_at"`
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -91,6 +91,11 @@ func (r *Runner) Run(ctx context.Context, input RunInput) (output RunOutput, err
|
|||||||
input.llmClient = input.Prepared.dependencies.LLM
|
input.llmClient = input.Prepared.dependencies.LLM
|
||||||
|
|
||||||
output.Manifest = manifestFromPipeline(input)
|
output.Manifest = manifestFromPipeline(input)
|
||||||
|
sessionID := strings.TrimSpace(input.SessionID)
|
||||||
|
output.Manifest.Metadata, err = manifestMetadataWithSessionID(output.Manifest.Metadata, sessionID)
|
||||||
|
if err != nil {
|
||||||
|
return failOutput(output), err
|
||||||
|
}
|
||||||
output.ChunkPlan = &artifacts.ChunkPlanSummary{
|
output.ChunkPlan = &artifacts.ChunkPlanSummary{
|
||||||
Mode: string(effectiveChunkCacheMode(input.ChunkCacheMode)), RequestedModule: input.pipeline.Chunk.Module,
|
Mode: string(effectiveChunkCacheMode(input.ChunkCacheMode)), RequestedModule: input.pipeline.Chunk.Module,
|
||||||
LookupStatus: "skipped", LookupReason: "chunk plan lookup skipped",
|
LookupStatus: "skipped", LookupReason: "chunk plan lookup skipped",
|
||||||
@@ -191,11 +196,6 @@ func (r *Runner) Run(ctx context.Context, input RunInput) (output RunOutput, err
|
|||||||
return failOutput(output), fmt.Errorf("write source debug artifact: %w", err)
|
return failOutput(output), fmt.Errorf("write source debug artifact: %w", err)
|
||||||
}
|
}
|
||||||
sourceInput := sourceInputMaterial(input.Path, input.RawInput)
|
sourceInput := sourceInputMaterial(input.Path, input.RawInput)
|
||||||
sessionID := resolvedSessionID(input.SessionID, doc.ID)
|
|
||||||
output.Manifest.Metadata, err = manifestMetadataWithSessionID(output.Manifest.Metadata, sessionID)
|
|
||||||
if err != nil {
|
|
||||||
return failOutput(output), err
|
|
||||||
}
|
|
||||||
output.Manifest.SourceDigests = []string{doc.Digest}
|
output.Manifest.SourceDigests = []string{doc.Digest}
|
||||||
|
|
||||||
chunker := input.Prepared.chunker
|
chunker := input.Prepared.chunker
|
||||||
@@ -896,13 +896,6 @@ func sourceInputOriginURI(inputPath string) string {
|
|||||||
return fileURI(inputPath)
|
return fileURI(inputPath)
|
||||||
}
|
}
|
||||||
|
|
||||||
func resolvedSessionID(explicit string, sourceDocumentID string) string {
|
|
||||||
if trimmed := strings.TrimSpace(explicit); trimmed != "" {
|
|
||||||
return trimmed
|
|
||||||
}
|
|
||||||
return strings.TrimSpace(sourceDocumentID)
|
|
||||||
}
|
|
||||||
|
|
||||||
func manifestMetadataWithSessionID(metadata map[string]any, sessionID string) (map[string]any, error) {
|
func manifestMetadataWithSessionID(metadata map[string]any, sessionID string) (map[string]any, error) {
|
||||||
out, err := cloneMetadata(metadata)
|
out, err := cloneMetadata(metadata)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
37
internal/framework/pipeline/runner_session_test.go
Normal file
37
internal/framework/pipeline/runner_session_test.go
Normal file
@@ -0,0 +1,37 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRunnerUsesProvidedSessionWithoutSourceFallback(t *testing.T) {
|
||||||
|
for _, test := range []struct {
|
||||||
|
name string
|
||||||
|
session string
|
||||||
|
wantSet bool
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{name: "provided", session: " supplied-session ", wantSet: true, want: "supplied-session"},
|
||||||
|
{name: "empty", wantSet: false},
|
||||||
|
} {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
prepared, _ := preparedTerminalDebugPipeline(t)
|
||||||
|
output, err := New().Run(context.Background(), RunInput{
|
||||||
|
Prepared: prepared,
|
||||||
|
RawInput: []byte("input"),
|
||||||
|
SessionID: test.session,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
value, found := output.Manifest.Metadata["session_id"]
|
||||||
|
if found != test.wantSet {
|
||||||
|
t.Fatalf("manifest session presence = %t, want %t; metadata = %#v", found, test.wantSet, output.Manifest.Metadata)
|
||||||
|
}
|
||||||
|
if found && value != test.want {
|
||||||
|
t.Fatalf("manifest session = %#v, want %q", value, test.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"$id": "notarius.dnd.enemy_events",
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["events"],
|
||||||
|
"properties": {
|
||||||
|
"events": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["name", "kind", "source_refs"],
|
||||||
|
"properties": {
|
||||||
|
"name": {"type": "string", "minLength": 1},
|
||||||
|
"kind": {"type": "string", "enum": ["engaged", "killed", "fled", "captured", "incapacitated"]},
|
||||||
|
"source_refs": {
|
||||||
|
"type": "array",
|
||||||
|
"minItems": 1,
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["source_id", "start_unit_id", "end_unit_id"],
|
||||||
|
"properties": {
|
||||||
|
"source_id": {"type": "string", "minLength": 1},
|
||||||
|
"start_unit_id": {"type": "integer", "minimum": 1},
|
||||||
|
"end_unit_id": {"type": "integer", "minimum": 1}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
182
internal/modules/dnd/codec/enemyevents/codec.go
Normal file
182
internal/modules/dnd/codec/enemyevents/codec.go
Normal file
@@ -0,0 +1,182 @@
|
|||||||
|
// Package enemyevents encodes durable D&D enemy-event artifacts.
|
||||||
|
package enemyevents
|
||||||
|
|
||||||
|
import (
|
||||||
|
"embed"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/candidatejson"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
SchemaID = "notarius.dnd.enemy_events"
|
||||||
|
SchemaName = "notarius_dnd_enemy_events_v1"
|
||||||
|
SchemaVersion = "v1"
|
||||||
|
MediaType = "application/json"
|
||||||
|
)
|
||||||
|
|
||||||
|
//go:embed assets/schemas/dnd_enemy_events.v1.json
|
||||||
|
var schemaAssets embed.FS
|
||||||
|
|
||||||
|
var _ contracts.ArtifactCodec[dnd.EnemyEventList] = (*Codec)(nil)
|
||||||
|
|
||||||
|
type Codec struct{}
|
||||||
|
|
||||||
|
func New() *Codec { return &Codec{} }
|
||||||
|
|
||||||
|
func (c *Codec) Kind() contracts.ArtifactKind { return dnd.EnemyEventListKind }
|
||||||
|
|
||||||
|
func (c *Codec) Schema() contracts.ArtifactSchema {
|
||||||
|
raw, err := schemaAssets.ReadFile("assets/schemas/dnd_enemy_events.v1.json")
|
||||||
|
if err != nil {
|
||||||
|
return contracts.ArtifactSchema{}
|
||||||
|
}
|
||||||
|
return contracts.ArtifactSchema{
|
||||||
|
ID: SchemaID,
|
||||||
|
Name: SchemaName,
|
||||||
|
Version: SchemaVersion,
|
||||||
|
JSONSchema: append([]byte(nil), raw...),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Codec) MediaType() string { return MediaType }
|
||||||
|
|
||||||
|
func (c *Codec) Metadata(value dnd.EnemyEventList) map[string]any {
|
||||||
|
return map[string]any{"event_count": len(value.Events)}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Codec) Encode(value dnd.EnemyEventList) ([]byte, error) {
|
||||||
|
if err := validateRequiredValueFields(value); err != nil {
|
||||||
|
return nil, fmt.Errorf("encode dnd enemy event list: %w", err)
|
||||||
|
}
|
||||||
|
return c.EncodeCandidate(value)
|
||||||
|
}
|
||||||
|
|
||||||
|
// EncodeCandidate provides the durable representation before semantic
|
||||||
|
// validators have approved a value.
|
||||||
|
func (c *Codec) EncodeCandidate(value dnd.EnemyEventList) ([]byte, error) {
|
||||||
|
return candidatejson.EncodeCandidate("dnd enemy event list", cloneList(value))
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Codec) Decode(content []byte) (dnd.EnemyEventList, error) {
|
||||||
|
value, err := c.DecodeCandidate(content)
|
||||||
|
if err != nil {
|
||||||
|
return dnd.EnemyEventList{}, err
|
||||||
|
}
|
||||||
|
if err := validateRequiredJSONFields(content); err != nil {
|
||||||
|
return dnd.EnemyEventList{}, fmt.Errorf("decode dnd enemy event list: %w", err)
|
||||||
|
}
|
||||||
|
return value, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DecodeCandidate reads one strict durable JSON value before semantic
|
||||||
|
// validators have approved it.
|
||||||
|
func (c *Codec) DecodeCandidate(content []byte) (dnd.EnemyEventList, error) {
|
||||||
|
value, err := candidatejson.DecodeCandidate[dnd.EnemyEventList]("dnd enemy event list", content)
|
||||||
|
if err != nil {
|
||||||
|
return dnd.EnemyEventList{}, err
|
||||||
|
}
|
||||||
|
return cloneList(value), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateRequiredValueFields(value dnd.EnemyEventList) error {
|
||||||
|
if value.Events == nil {
|
||||||
|
return fmt.Errorf("events must be present")
|
||||||
|
}
|
||||||
|
for index, event := range value.Events {
|
||||||
|
if event.SourceRefs == nil {
|
||||||
|
return fmt.Errorf("events[%d].source_refs must be present", index)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateRequiredJSONFields(content []byte) error {
|
||||||
|
var root map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal(content, &root); err != nil || root == nil {
|
||||||
|
return fmt.Errorf("must be a JSON object")
|
||||||
|
}
|
||||||
|
events, err := requiredArray(root, "events", "")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for eventIndex, rawEvent := range events {
|
||||||
|
path := fmt.Sprintf("events[%d]", eventIndex)
|
||||||
|
var event map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal(rawEvent, &event); err != nil || event == nil {
|
||||||
|
return fmt.Errorf("%s must be an object", path)
|
||||||
|
}
|
||||||
|
for _, field := range []string{"name", "kind"} {
|
||||||
|
if err := requireField(event, field, path); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
refs, err := requiredArray(event, "source_refs", path)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for refIndex, rawRef := range refs {
|
||||||
|
refPath := fmt.Sprintf("%s.source_refs[%d]", path, refIndex)
|
||||||
|
var ref map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal(rawRef, &ref); err != nil || ref == nil {
|
||||||
|
return fmt.Errorf("%s must be an object", refPath)
|
||||||
|
}
|
||||||
|
for _, field := range []string{"source_id", "start_unit_id", "end_unit_id"} {
|
||||||
|
if err := requireField(ref, field, refPath); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func requiredArray(object map[string]json.RawMessage, field, path string) ([]json.RawMessage, error) {
|
||||||
|
raw, err := requiredField(object, field, path)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var values []json.RawMessage
|
||||||
|
if err := json.Unmarshal(raw, &values); err != nil {
|
||||||
|
return nil, fmt.Errorf("%s must be an array", fieldPath(path, field))
|
||||||
|
}
|
||||||
|
return values, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func requireField(object map[string]json.RawMessage, field, path string) error {
|
||||||
|
_, err := requiredField(object, field, path)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func requiredField(object map[string]json.RawMessage, field, path string) (json.RawMessage, error) {
|
||||||
|
raw, ok := object[field]
|
||||||
|
if !ok || string(raw) == "null" {
|
||||||
|
return nil, fmt.Errorf("%s must be present", fieldPath(path, field))
|
||||||
|
}
|
||||||
|
return raw, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func fieldPath(path, field string) string {
|
||||||
|
if path == "" {
|
||||||
|
return field
|
||||||
|
}
|
||||||
|
return path + "." + field
|
||||||
|
}
|
||||||
|
|
||||||
|
func cloneList(value dnd.EnemyEventList) dnd.EnemyEventList {
|
||||||
|
if value.Events == nil {
|
||||||
|
return dnd.EnemyEventList{}
|
||||||
|
}
|
||||||
|
cloned := dnd.EnemyEventList{Events: make([]dnd.EnemyEvent, len(value.Events))}
|
||||||
|
for index, event := range value.Events {
|
||||||
|
cloned.Events[index] = event
|
||||||
|
if event.SourceRefs != nil {
|
||||||
|
cloned.Events[index].SourceRefs = append([]source.SourceRef(nil), event.SourceRefs...)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return cloned
|
||||||
|
}
|
||||||
99
internal/modules/dnd/codec/enemyevents/codec_test.go
Normal file
99
internal/modules/dnd/codec/enemyevents/codec_test.go
Normal file
@@ -0,0 +1,99 @@
|
|||||||
|
package enemyevents
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
)
|
||||||
|
|
||||||
|
func validList() dnd.EnemyEventList {
|
||||||
|
return dnd.EnemyEventList{Events: []dnd.EnemyEvent{
|
||||||
|
{Name: "Ashfang", Kind: dnd.EnemyEventKindEngaged, SourceRefs: refs(1, 2)},
|
||||||
|
{Name: "Ashfang", Kind: dnd.EnemyEventKindFled, SourceRefs: refs(5, 6)},
|
||||||
|
}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func refs(start, end int) []source.SourceRef {
|
||||||
|
return []source.SourceRef{{SourceID: "session", StartUnitID: start, EndUnitID: end}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCodecRoundTripAndIdentity(t *testing.T) {
|
||||||
|
codec := New()
|
||||||
|
value := validList()
|
||||||
|
content, err := codec.Encode(value)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Encode() error = %v", err)
|
||||||
|
}
|
||||||
|
decoded, err := codec.Decode(content)
|
||||||
|
if err != nil || !reflect.DeepEqual(decoded, value) {
|
||||||
|
t.Fatalf("Decode() = %#v, %v; want %#v", decoded, err, value)
|
||||||
|
}
|
||||||
|
|
||||||
|
schema := codec.Schema()
|
||||||
|
if codec.Kind() != dnd.EnemyEventListKind || codec.MediaType() != MediaType || schema.ID != SchemaID || schema.Name != SchemaName || schema.Version != SchemaVersion || !json.Valid(schema.JSONSchema) {
|
||||||
|
t.Fatalf("codec identity/schema = %q/%q %#v", codec.Kind(), codec.MediaType(), schema)
|
||||||
|
}
|
||||||
|
registry := pipeline.NewArtifactCodecRegistry()
|
||||||
|
if err := pipeline.RegisterArtifactCodec(registry, codec); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
spec, ok := registry.Spec(dnd.EnemyEventListKind)
|
||||||
|
if !ok || spec.SchemaDigest != contracts.DigestArtifactSchema(schema) {
|
||||||
|
t.Fatalf("registered spec = %#v, %t", spec, ok)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCodecRejectsStrictJSONAndMissingRequiredFields(t *testing.T) {
|
||||||
|
validJSON := `{"events":[{"name":"Ashfang","kind":"engaged","source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]}]}`
|
||||||
|
tests := []struct {
|
||||||
|
name, raw, want string
|
||||||
|
}{
|
||||||
|
{"invalid JSON", `{`, "decode dnd enemy event list"},
|
||||||
|
{"unknown root field", `{"events":[],"unexpected":true}`, "unknown field"},
|
||||||
|
{"unknown event field", strings.Replace(validJSON, `"kind":"engaged"`, `"kind":"engaged","unexpected":true`, 1), "unknown field"},
|
||||||
|
{"missing events", `{}`, "events must be present"},
|
||||||
|
{"missing name", strings.Replace(validJSON, `"name":"Ashfang",`, "", 1), "events[0].name must be present"},
|
||||||
|
{"missing kind", strings.Replace(validJSON, `"kind":"engaged",`, "", 1), "events[0].kind must be present"},
|
||||||
|
{"missing refs", strings.Replace(validJSON, `,"source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]`, "", 1), "events[0].source_refs must be present"},
|
||||||
|
{"missing source ID", strings.Replace(validJSON, `"source_id":"session",`, "", 1), "source_refs[0].source_id must be present"},
|
||||||
|
{"trailing JSON", `{"events":[]} {}`, "multiple JSON values"},
|
||||||
|
}
|
||||||
|
for _, test := range tests {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
if _, err := New().Decode([]byte(test.raw)); err == nil || !strings.Contains(err.Error(), test.want) {
|
||||||
|
t.Fatalf("Decode() error = %v, want %q", err, test.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCodecDefensivelyOwnsValuesAndDefersSemanticValidation(t *testing.T) {
|
||||||
|
codec := New()
|
||||||
|
candidate := dnd.EnemyEventList{Events: []dnd.EnemyEvent{{
|
||||||
|
Name: " ", Kind: "unsupported", SourceRefs: []source.SourceRef{{SourceID: "", StartUnitID: 0, EndUnitID: -1}},
|
||||||
|
}}}
|
||||||
|
content, err := codec.EncodeCandidate(candidate)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
decoded, err := codec.Decode(content)
|
||||||
|
if err != nil || !reflect.DeepEqual(decoded, candidate) {
|
||||||
|
t.Fatalf("Decode() = %#v, %v; want semantic candidate preservation", decoded, err)
|
||||||
|
}
|
||||||
|
decoded.Events[0].SourceRefs[0].SourceID = "changed"
|
||||||
|
if candidate.Events[0].SourceRefs[0].SourceID != "" {
|
||||||
|
t.Fatal("Decode() retained caller-owned source references")
|
||||||
|
}
|
||||||
|
|
||||||
|
first := codec.Schema()
|
||||||
|
first.JSONSchema[0] = '['
|
||||||
|
if second := codec.Schema(); !json.Valid(second.JSONSchema) || second.JSONSchema[0] == '[' {
|
||||||
|
t.Fatal("Schema() returned shared bytes")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"$id": "notarius.dnd.location_occurrences",
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["occurrences"],
|
||||||
|
"properties": {
|
||||||
|
"occurrences": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["location_id", "name", "kind", "source_refs"],
|
||||||
|
"properties": {
|
||||||
|
"location_id": {
|
||||||
|
"type": "string",
|
||||||
|
"pattern": "^location:sha256:[0-9a-f]{64}$"
|
||||||
|
},
|
||||||
|
"name": {
|
||||||
|
"type": "string",
|
||||||
|
"minLength": 1
|
||||||
|
},
|
||||||
|
"kind": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": ["visited", "planned", "recalled", "mentioned"]
|
||||||
|
},
|
||||||
|
"source_refs": {
|
||||||
|
"type": "array",
|
||||||
|
"minItems": 1,
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["source_id", "start_unit_id", "end_unit_id"],
|
||||||
|
"properties": {
|
||||||
|
"source_id": {
|
||||||
|
"type": "string",
|
||||||
|
"minLength": 1
|
||||||
|
},
|
||||||
|
"start_unit_id": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1
|
||||||
|
},
|
||||||
|
"end_unit_id": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
126
internal/modules/dnd/codec/locationoccurrences/codec.go
Normal file
126
internal/modules/dnd/codec/locationoccurrences/codec.go
Normal file
@@ -0,0 +1,126 @@
|
|||||||
|
// Package locationoccurrences encodes durable D&D location occurrence artifacts.
|
||||||
|
package locationoccurrences
|
||||||
|
|
||||||
|
import (
|
||||||
|
"embed"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/candidatejson"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
SchemaID = "notarius.dnd.location_occurrences"
|
||||||
|
SchemaName = "notarius_dnd_location_occurrences_v1"
|
||||||
|
SchemaVersion = "v1"
|
||||||
|
MediaType = "application/json"
|
||||||
|
)
|
||||||
|
|
||||||
|
//go:embed assets/schemas/dnd_location_occurrences.v1.json
|
||||||
|
var schemaAssets embed.FS
|
||||||
|
|
||||||
|
var _ contracts.ArtifactCodec[dnd.LocationOccurrenceList] = (*Codec)(nil)
|
||||||
|
|
||||||
|
type Codec struct{}
|
||||||
|
|
||||||
|
func New() *Codec { return &Codec{} }
|
||||||
|
|
||||||
|
func (c *Codec) Kind() contracts.ArtifactKind { return dnd.LocationOccurrenceListKind }
|
||||||
|
|
||||||
|
func (c *Codec) Schema() contracts.ArtifactSchema {
|
||||||
|
raw, err := schemaAssets.ReadFile("assets/schemas/dnd_location_occurrences.v1.json")
|
||||||
|
if err != nil {
|
||||||
|
return contracts.ArtifactSchema{}
|
||||||
|
}
|
||||||
|
return contracts.ArtifactSchema{
|
||||||
|
ID: SchemaID,
|
||||||
|
Name: SchemaName,
|
||||||
|
Version: SchemaVersion,
|
||||||
|
JSONSchema: append([]byte(nil), raw...),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Codec) MediaType() string { return MediaType }
|
||||||
|
|
||||||
|
func (c *Codec) Metadata(value dnd.LocationOccurrenceList) map[string]any {
|
||||||
|
return map[string]any{"occurrence_count": len(value.Occurrences)}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Codec) Encode(value dnd.LocationOccurrenceList) ([]byte, error) {
|
||||||
|
if err := validate(value); err != nil {
|
||||||
|
return nil, fmt.Errorf("encode dnd location occurrence list: %w", err)
|
||||||
|
}
|
||||||
|
return c.EncodeCandidate(value)
|
||||||
|
}
|
||||||
|
|
||||||
|
// EncodeCandidate provides the durable representation before semantic
|
||||||
|
// validators have approved a value.
|
||||||
|
func (c *Codec) EncodeCandidate(value dnd.LocationOccurrenceList) ([]byte, error) {
|
||||||
|
return candidatejson.EncodeCandidate("dnd location occurrence list", value)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Codec) Decode(content []byte) (dnd.LocationOccurrenceList, error) {
|
||||||
|
value, err := c.DecodeCandidate(content)
|
||||||
|
if err != nil {
|
||||||
|
return dnd.LocationOccurrenceList{}, err
|
||||||
|
}
|
||||||
|
if err := validate(value); err != nil {
|
||||||
|
return dnd.LocationOccurrenceList{}, fmt.Errorf("decode dnd location occurrence list: %w", err)
|
||||||
|
}
|
||||||
|
return value, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DecodeCandidate reads one strict durable JSON value before semantic
|
||||||
|
// validators have approved it.
|
||||||
|
func (c *Codec) DecodeCandidate(content []byte) (dnd.LocationOccurrenceList, error) {
|
||||||
|
return candidatejson.DecodeCandidate[dnd.LocationOccurrenceList]("dnd location occurrence list", content)
|
||||||
|
}
|
||||||
|
|
||||||
|
func validate(value dnd.LocationOccurrenceList) error {
|
||||||
|
if value.Occurrences == nil {
|
||||||
|
return fmt.Errorf("occurrences must be present")
|
||||||
|
}
|
||||||
|
for index, occurrence := range value.Occurrences {
|
||||||
|
prefix := fmt.Sprintf("occurrences[%d]", index)
|
||||||
|
if !identity.IsValidID(occurrence.LocationID) {
|
||||||
|
return fmt.Errorf("%s.location_id must match location ID pattern", prefix)
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(occurrence.Name) == "" {
|
||||||
|
return fmt.Errorf("%s.name must not be empty", prefix)
|
||||||
|
}
|
||||||
|
if !validOccurrenceKind(occurrence.Kind) {
|
||||||
|
return fmt.Errorf("%s.kind must be supported", prefix)
|
||||||
|
}
|
||||||
|
if len(occurrence.SourceRefs) == 0 {
|
||||||
|
return fmt.Errorf("%s.source_refs must contain at least one reference", prefix)
|
||||||
|
}
|
||||||
|
for refIndex, ref := range occurrence.SourceRefs {
|
||||||
|
refPrefix := fmt.Sprintf("%s.source_refs[%d]", prefix, refIndex)
|
||||||
|
if strings.TrimSpace(ref.SourceID) == "" {
|
||||||
|
return fmt.Errorf("%s.source_id must not be empty", refPrefix)
|
||||||
|
}
|
||||||
|
if ref.StartUnitID <= 0 {
|
||||||
|
return fmt.Errorf("%s.start_unit_id must be positive", refPrefix)
|
||||||
|
}
|
||||||
|
if ref.EndUnitID <= 0 {
|
||||||
|
return fmt.Errorf("%s.end_unit_id must be positive", refPrefix)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validOccurrenceKind(value dnd.LocationOccurrenceKind) bool {
|
||||||
|
switch value {
|
||||||
|
case dnd.LocationOccurrenceKindVisited,
|
||||||
|
dnd.LocationOccurrenceKindPlanned,
|
||||||
|
dnd.LocationOccurrenceKindRecalled,
|
||||||
|
dnd.LocationOccurrenceKindMentioned:
|
||||||
|
return true
|
||||||
|
default:
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
163
internal/modules/dnd/codec/locationoccurrences/codec_test.go
Normal file
163
internal/modules/dnd/codec/locationoccurrences/codec_test.go
Normal file
@@ -0,0 +1,163 @@
|
|||||||
|
package locationoccurrences
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"os"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||||
|
)
|
||||||
|
|
||||||
|
func validList() dnd.LocationOccurrenceList {
|
||||||
|
refs := []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 2}}
|
||||||
|
return dnd.LocationOccurrenceList{Occurrences: []dnd.LocationOccurrence{{
|
||||||
|
LocationID: identity.DeriveID("The Old Tavern", refs),
|
||||||
|
Name: "The Old Tavern",
|
||||||
|
Kind: dnd.LocationOccurrenceKindVisited,
|
||||||
|
SourceRefs: refs,
|
||||||
|
}}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCodecMatchesMaintainedDurableFixture(t *testing.T) {
|
||||||
|
raw, err := os.ReadFile("testdata/dnd_location_occurrences.v1.json")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read durable fixture: %v", err)
|
||||||
|
}
|
||||||
|
codec := New()
|
||||||
|
value, err := codec.Decode(raw)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Decode() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if want := validList(); !reflect.DeepEqual(value, want) {
|
||||||
|
t.Fatalf("Decode() = %#v, want %#v", value, want)
|
||||||
|
}
|
||||||
|
encoded, err := codec.Encode(value)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Encode() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
var compact bytes.Buffer
|
||||||
|
if err := json.Compact(&compact, raw); err != nil {
|
||||||
|
t.Fatalf("compact durable fixture: %v", err)
|
||||||
|
}
|
||||||
|
if !bytes.Equal(encoded, compact.Bytes()) {
|
||||||
|
t.Fatalf("Encode() = %s, want %s", encoded, compact.Bytes())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCodecOwnsDurableSchemaAndMetadata(t *testing.T) {
|
||||||
|
codec := New()
|
||||||
|
schema := codec.Schema()
|
||||||
|
if codec.Kind() != dnd.LocationOccurrenceListKind || codec.MediaType() != MediaType {
|
||||||
|
t.Fatalf("codec identity = %q/%q", codec.Kind(), codec.MediaType())
|
||||||
|
}
|
||||||
|
if schema.ID != SchemaID || schema.Name != SchemaName || schema.Version != SchemaVersion || !json.Valid(schema.JSONSchema) {
|
||||||
|
t.Fatalf("schema = %#v, want durable location occurrence schema", schema)
|
||||||
|
}
|
||||||
|
var document map[string]any
|
||||||
|
if err := json.Unmarshal(schema.JSONSchema, &document); err != nil || document["$id"] != SchemaID {
|
||||||
|
t.Fatalf("durable schema document = %#v, %v", document, err)
|
||||||
|
}
|
||||||
|
registry := pipeline.NewArtifactCodecRegistry()
|
||||||
|
if err := pipeline.RegisterArtifactCodec(registry, codec); err != nil {
|
||||||
|
t.Fatalf("RegisterArtifactCodec() error = %v", err)
|
||||||
|
}
|
||||||
|
if spec, ok := registry.Spec(dnd.LocationOccurrenceListKind); !ok || spec.SchemaDigest != contracts.DigestArtifactSchema(schema) {
|
||||||
|
t.Fatalf("registered spec = %#v, %t", spec, ok)
|
||||||
|
}
|
||||||
|
first := schema.JSONSchema
|
||||||
|
first[0] = '['
|
||||||
|
if next := codec.Schema().JSONSchema; !json.Valid(next) || next[0] == '[' {
|
||||||
|
t.Fatal("Schema() returned shared bytes")
|
||||||
|
}
|
||||||
|
metadata := codec.Metadata(validList())
|
||||||
|
metadata["other"] = true
|
||||||
|
if next := codec.Metadata(validList()); len(next) != 1 || next["occurrence_count"] != 1 {
|
||||||
|
t.Fatalf("Metadata() = %#v", next)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCodecSupportsEmptyListsAndCandidateSemanticFailures(t *testing.T) {
|
||||||
|
codec := New()
|
||||||
|
empty := dnd.LocationOccurrenceList{Occurrences: []dnd.LocationOccurrence{}}
|
||||||
|
if content, err := codec.Encode(empty); err != nil || string(content) != `{"occurrences":[]}` {
|
||||||
|
t.Fatalf("Encode() = %s, %v", content, err)
|
||||||
|
}
|
||||||
|
for _, candidate := range []dnd.LocationOccurrenceList{
|
||||||
|
{},
|
||||||
|
empty,
|
||||||
|
{Occurrences: []dnd.LocationOccurrence{{LocationID: "bad", Name: " ", Kind: "unsupported", SourceRefs: nil}}},
|
||||||
|
{Occurrences: []dnd.LocationOccurrence{{LocationID: "bad", Name: " ", Kind: "unsupported", SourceRefs: []source.SourceRef{}}}},
|
||||||
|
{Occurrences: []dnd.LocationOccurrence{{LocationID: "bad", Name: " ", Kind: "unsupported", SourceRefs: []source.SourceRef{{StartUnitID: 0, EndUnitID: -1}}}}},
|
||||||
|
} {
|
||||||
|
content, err := codec.EncodeCandidate(candidate)
|
||||||
|
if err != nil || !json.Valid(content) {
|
||||||
|
t.Fatalf("EncodeCandidate() = %s, %v", content, err)
|
||||||
|
}
|
||||||
|
decoded, err := codec.DecodeCandidate(content)
|
||||||
|
if err != nil || !reflect.DeepEqual(decoded, candidate) {
|
||||||
|
t.Fatalf("DecodeCandidate() = %#v, %v, want %#v", decoded, err, candidate)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCodecRejectsStructuralJSONBeforeSemanticApproval(t *testing.T) {
|
||||||
|
valid := `{"occurrences":[{"location_id":"location:sha256:0000000000000000000000000000000000000000000000000000000000000000","name":"The Tavern","kind":"visited","source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]}]}`
|
||||||
|
for _, test := range []struct{ name, raw, want string }{
|
||||||
|
{"malformed", `{`, "decode dnd location occurrence list"},
|
||||||
|
{"unknown top-level", `{"occurrences":[],"unexpected":true}`, "unknown field"},
|
||||||
|
{"unknown occurrence field", strings.Replace(valid, `"kind":"visited"`, `"kind":"visited","unexpected":true`, 1), "unknown field"},
|
||||||
|
{"unknown source reference field", strings.Replace(valid, `"end_unit_id":1`, `"end_unit_id":1,"unexpected":true`, 1), "unknown field"},
|
||||||
|
{"invalid type", `{"occurrences":"not-an-array"}`, "cannot unmarshal string"},
|
||||||
|
{"trailing", `{"occurrences":[]} {}`, "multiple JSON values"},
|
||||||
|
} {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
if _, err := New().DecodeCandidate([]byte(test.raw)); err == nil || !strings.Contains(err.Error(), test.want) {
|
||||||
|
t.Fatalf("DecodeCandidate() error = %v, want %q", err, test.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCodecRejectsApprovedShapeEnumAndReferenceBoundaries(t *testing.T) {
|
||||||
|
base := validList().Occurrences[0]
|
||||||
|
for _, test := range []struct {
|
||||||
|
name string
|
||||||
|
value dnd.LocationOccurrenceList
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{"nil occurrences", dnd.LocationOccurrenceList{}, "occurrences must be present"},
|
||||||
|
{"invalid ID", dnd.LocationOccurrenceList{Occurrences: []dnd.LocationOccurrence{{LocationID: "bad", Name: base.Name, Kind: base.Kind, SourceRefs: base.SourceRefs}}}, "location_id must match location ID pattern"},
|
||||||
|
{"blank name", dnd.LocationOccurrenceList{Occurrences: []dnd.LocationOccurrence{{LocationID: base.LocationID, Name: " ", Kind: base.Kind, SourceRefs: base.SourceRefs}}}, "name must not be empty"},
|
||||||
|
{"invalid kind", dnd.LocationOccurrenceList{Occurrences: []dnd.LocationOccurrence{{LocationID: base.LocationID, Name: base.Name, Kind: "unsupported", SourceRefs: base.SourceRefs}}}, "kind must be supported"},
|
||||||
|
{"empty source refs", dnd.LocationOccurrenceList{Occurrences: []dnd.LocationOccurrence{{LocationID: base.LocationID, Name: base.Name, Kind: base.Kind}}}, "source_refs must contain"},
|
||||||
|
{"malformed source reference", dnd.LocationOccurrenceList{Occurrences: []dnd.LocationOccurrence{{LocationID: base.LocationID, Name: base.Name, Kind: base.Kind, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 1, EndUnitID: 0}}}}}, "end_unit_id must be positive"},
|
||||||
|
} {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
if _, err := New().Encode(test.value); err == nil || !strings.Contains(err.Error(), test.want) {
|
||||||
|
t.Fatalf("Encode() error = %v, want %q", err, test.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCodecAcceptsEveryOccurrenceKind(t *testing.T) {
|
||||||
|
for _, kind := range []dnd.LocationOccurrenceKind{
|
||||||
|
dnd.LocationOccurrenceKindVisited,
|
||||||
|
dnd.LocationOccurrenceKindPlanned,
|
||||||
|
dnd.LocationOccurrenceKindRecalled,
|
||||||
|
dnd.LocationOccurrenceKindMentioned,
|
||||||
|
} {
|
||||||
|
value := validList()
|
||||||
|
value.Occurrences[0].Kind = kind
|
||||||
|
if _, err := New().Encode(value); err != nil {
|
||||||
|
t.Fatalf("Encode(%q) error = %v", kind, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
12
internal/modules/dnd/codec/locationoccurrences/testdata/dnd_location_occurrences.v1.json
vendored
Normal file
12
internal/modules/dnd/codec/locationoccurrences/testdata/dnd_location_occurrences.v1.json
vendored
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
{
|
||||||
|
"occurrences": [
|
||||||
|
{
|
||||||
|
"location_id": "location:sha256:cdd2b57615b56a4e92d506cf053cd3985e11d4df4895c5383b30ec2ce9f39ed0",
|
||||||
|
"name": "The Old Tavern",
|
||||||
|
"kind": "visited",
|
||||||
|
"source_refs": [
|
||||||
|
{"source_id": "session-alpha", "start_unit_id": 1, "end_unit_id": 2}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"$id": "notarius.dnd.locations",
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["locations"],
|
||||||
|
"properties": {
|
||||||
|
"locations": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["id", "name", "source_refs"],
|
||||||
|
"properties": {
|
||||||
|
"id": {
|
||||||
|
"type": "string",
|
||||||
|
"pattern": "^location:sha256:[0-9a-f]{64}$"
|
||||||
|
},
|
||||||
|
"name": {
|
||||||
|
"type": "string",
|
||||||
|
"minLength": 1
|
||||||
|
},
|
||||||
|
"source_refs": {
|
||||||
|
"type": "array",
|
||||||
|
"minItems": 1,
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["source_id", "start_unit_id", "end_unit_id"],
|
||||||
|
"properties": {
|
||||||
|
"source_id": {
|
||||||
|
"type": "string",
|
||||||
|
"minLength": 1
|
||||||
|
},
|
||||||
|
"start_unit_id": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1
|
||||||
|
},
|
||||||
|
"end_unit_id": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
111
internal/modules/dnd/codec/locations/codec.go
Normal file
111
internal/modules/dnd/codec/locations/codec.go
Normal file
@@ -0,0 +1,111 @@
|
|||||||
|
// Package locations encodes durable D&D location artifacts.
|
||||||
|
package locations
|
||||||
|
|
||||||
|
import (
|
||||||
|
"embed"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/candidatejson"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
SchemaID = "notarius.dnd.locations"
|
||||||
|
SchemaName = "notarius_dnd_locations_v1"
|
||||||
|
SchemaVersion = "v1"
|
||||||
|
MediaType = "application/json"
|
||||||
|
)
|
||||||
|
|
||||||
|
//go:embed assets/schemas/dnd_locations.v1.json
|
||||||
|
var schemaAssets embed.FS
|
||||||
|
|
||||||
|
var _ contracts.ArtifactCodec[dnd.LocationList] = (*Codec)(nil)
|
||||||
|
|
||||||
|
type Codec struct{}
|
||||||
|
|
||||||
|
func New() *Codec { return &Codec{} }
|
||||||
|
|
||||||
|
func (c *Codec) Kind() contracts.ArtifactKind { return dnd.LocationListKind }
|
||||||
|
|
||||||
|
func (c *Codec) Schema() contracts.ArtifactSchema {
|
||||||
|
raw, err := schemaAssets.ReadFile("assets/schemas/dnd_locations.v1.json")
|
||||||
|
if err != nil {
|
||||||
|
return contracts.ArtifactSchema{}
|
||||||
|
}
|
||||||
|
return contracts.ArtifactSchema{
|
||||||
|
ID: SchemaID,
|
||||||
|
Name: SchemaName,
|
||||||
|
Version: SchemaVersion,
|
||||||
|
JSONSchema: append([]byte(nil), raw...),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Codec) MediaType() string { return MediaType }
|
||||||
|
|
||||||
|
func (c *Codec) Metadata(value dnd.LocationList) map[string]any {
|
||||||
|
return map[string]any{"location_count": len(value.Locations)}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Codec) Encode(value dnd.LocationList) ([]byte, error) {
|
||||||
|
if err := validate(value); err != nil {
|
||||||
|
return nil, fmt.Errorf("encode dnd location list: %w", err)
|
||||||
|
}
|
||||||
|
return c.EncodeCandidate(value)
|
||||||
|
}
|
||||||
|
|
||||||
|
// EncodeCandidate provides the durable representation before semantic
|
||||||
|
// validators have approved a value.
|
||||||
|
func (c *Codec) EncodeCandidate(value dnd.LocationList) ([]byte, error) {
|
||||||
|
return candidatejson.EncodeCandidate("dnd location list", value)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Codec) Decode(content []byte) (dnd.LocationList, error) {
|
||||||
|
value, err := c.DecodeCandidate(content)
|
||||||
|
if err != nil {
|
||||||
|
return dnd.LocationList{}, err
|
||||||
|
}
|
||||||
|
if err := validate(value); err != nil {
|
||||||
|
return dnd.LocationList{}, fmt.Errorf("decode dnd location list: %w", err)
|
||||||
|
}
|
||||||
|
return value, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DecodeCandidate reads one strict durable JSON value before semantic
|
||||||
|
// validators have approved it.
|
||||||
|
func (c *Codec) DecodeCandidate(content []byte) (dnd.LocationList, error) {
|
||||||
|
return candidatejson.DecodeCandidate[dnd.LocationList]("dnd location list", content)
|
||||||
|
}
|
||||||
|
|
||||||
|
func validate(value dnd.LocationList) error {
|
||||||
|
if value.Locations == nil {
|
||||||
|
return fmt.Errorf("locations must be present")
|
||||||
|
}
|
||||||
|
for index, location := range value.Locations {
|
||||||
|
prefix := fmt.Sprintf("locations[%d]", index)
|
||||||
|
if !identity.IsValidID(location.ID) {
|
||||||
|
return fmt.Errorf("%s.id must match location ID pattern", prefix)
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(location.Name) == "" {
|
||||||
|
return fmt.Errorf("%s.name must not be empty", prefix)
|
||||||
|
}
|
||||||
|
if len(location.SourceRefs) == 0 {
|
||||||
|
return fmt.Errorf("%s.source_refs must contain at least one reference", prefix)
|
||||||
|
}
|
||||||
|
for refIndex, ref := range location.SourceRefs {
|
||||||
|
refPrefix := fmt.Sprintf("%s.source_refs[%d]", prefix, refIndex)
|
||||||
|
if strings.TrimSpace(ref.SourceID) == "" {
|
||||||
|
return fmt.Errorf("%s.source_id must not be empty", refPrefix)
|
||||||
|
}
|
||||||
|
if ref.StartUnitID <= 0 {
|
||||||
|
return fmt.Errorf("%s.start_unit_id must be positive", refPrefix)
|
||||||
|
}
|
||||||
|
if ref.EndUnitID <= 0 {
|
||||||
|
return fmt.Errorf("%s.end_unit_id must be positive", refPrefix)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
146
internal/modules/dnd/codec/locations/codec_test.go
Normal file
146
internal/modules/dnd/codec/locations/codec_test.go
Normal file
@@ -0,0 +1,146 @@
|
|||||||
|
package locations
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"os"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||||
|
)
|
||||||
|
|
||||||
|
func validList() dnd.LocationList {
|
||||||
|
refs := []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 2}}
|
||||||
|
return dnd.LocationList{Locations: []dnd.Location{{
|
||||||
|
ID: identity.DeriveID("The Old Tavern", refs),
|
||||||
|
Name: "The Old Tavern",
|
||||||
|
SourceRefs: refs,
|
||||||
|
}}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCodecMatchesMaintainedDurableFixture(t *testing.T) {
|
||||||
|
raw, err := os.ReadFile("testdata/dnd_locations.v1.json")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read durable fixture: %v", err)
|
||||||
|
}
|
||||||
|
codec := New()
|
||||||
|
value, err := codec.Decode(raw)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Decode() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
if want := validList(); !reflect.DeepEqual(value, want) {
|
||||||
|
t.Fatalf("Decode() = %#v, want %#v", value, want)
|
||||||
|
}
|
||||||
|
encoded, err := codec.Encode(value)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Encode() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
var compact bytes.Buffer
|
||||||
|
if err := json.Compact(&compact, raw); err != nil {
|
||||||
|
t.Fatalf("compact durable fixture: %v", err)
|
||||||
|
}
|
||||||
|
if !bytes.Equal(encoded, compact.Bytes()) {
|
||||||
|
t.Fatalf("Encode() = %s, want %s", encoded, compact.Bytes())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCodecOwnsDurableSchemaAndMetadata(t *testing.T) {
|
||||||
|
codec := New()
|
||||||
|
schema := codec.Schema()
|
||||||
|
if codec.Kind() != dnd.LocationListKind || codec.MediaType() != MediaType {
|
||||||
|
t.Fatalf("codec identity = %q/%q", codec.Kind(), codec.MediaType())
|
||||||
|
}
|
||||||
|
if schema.ID != SchemaID || schema.Name != SchemaName || schema.Version != SchemaVersion || !json.Valid(schema.JSONSchema) {
|
||||||
|
t.Fatalf("schema = %#v, want durable location schema", schema)
|
||||||
|
}
|
||||||
|
var document map[string]any
|
||||||
|
if err := json.Unmarshal(schema.JSONSchema, &document); err != nil || document["$id"] != SchemaID {
|
||||||
|
t.Fatalf("durable schema document = %#v, %v", document, err)
|
||||||
|
}
|
||||||
|
registry := pipeline.NewArtifactCodecRegistry()
|
||||||
|
if err := pipeline.RegisterArtifactCodec(registry, codec); err != nil {
|
||||||
|
t.Fatalf("RegisterArtifactCodec() error = %v", err)
|
||||||
|
}
|
||||||
|
if spec, ok := registry.Spec(dnd.LocationListKind); !ok || spec.SchemaDigest != contracts.DigestArtifactSchema(schema) {
|
||||||
|
t.Fatalf("registered spec = %#v, %t", spec, ok)
|
||||||
|
}
|
||||||
|
first := schema.JSONSchema
|
||||||
|
first[0] = '['
|
||||||
|
if next := codec.Schema().JSONSchema; !json.Valid(next) || next[0] == '[' {
|
||||||
|
t.Fatal("Schema() returned shared bytes")
|
||||||
|
}
|
||||||
|
metadata := codec.Metadata(validList())
|
||||||
|
metadata["other"] = true
|
||||||
|
if next := codec.Metadata(validList()); len(next) != 1 || next["location_count"] != 1 {
|
||||||
|
t.Fatalf("Metadata() = %#v", next)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCodecSupportsEmptyListsAndCandidateSemanticFailures(t *testing.T) {
|
||||||
|
codec := New()
|
||||||
|
empty := dnd.LocationList{Locations: []dnd.Location{}}
|
||||||
|
if content, err := codec.Encode(empty); err != nil || string(content) != `{"locations":[]}` {
|
||||||
|
t.Fatalf("Encode() = %s, %v", content, err)
|
||||||
|
}
|
||||||
|
for _, candidate := range []dnd.LocationList{
|
||||||
|
{},
|
||||||
|
empty,
|
||||||
|
{Locations: []dnd.Location{{ID: "bad", Name: " ", SourceRefs: nil}}},
|
||||||
|
{Locations: []dnd.Location{{ID: "bad", Name: " ", SourceRefs: []source.SourceRef{}}}},
|
||||||
|
{Locations: []dnd.Location{{ID: "bad", Name: " ", SourceRefs: []source.SourceRef{{StartUnitID: 0, EndUnitID: -1}}}}},
|
||||||
|
} {
|
||||||
|
content, err := codec.EncodeCandidate(candidate)
|
||||||
|
if err != nil || !json.Valid(content) {
|
||||||
|
t.Fatalf("EncodeCandidate() = %s, %v", content, err)
|
||||||
|
}
|
||||||
|
decoded, err := codec.DecodeCandidate(content)
|
||||||
|
if err != nil || !reflect.DeepEqual(decoded, candidate) {
|
||||||
|
t.Fatalf("DecodeCandidate() = %#v, %v, want %#v", decoded, err, candidate)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCodecRejectsStructuralJSONBeforeSemanticApproval(t *testing.T) {
|
||||||
|
valid := `{"locations":[{"id":"location:sha256:0000000000000000000000000000000000000000000000000000000000000000","name":"The Tavern","source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]}]}`
|
||||||
|
for _, test := range []struct{ name, raw, want string }{
|
||||||
|
{"malformed", `{`, "decode dnd location list"},
|
||||||
|
{"unknown top-level", `{"locations":[],"unexpected":true}`, "unknown field"},
|
||||||
|
{"unknown location field", strings.Replace(valid, `"name":"The Tavern"`, `"name":"The Tavern","unexpected":true`, 1), "unknown field"},
|
||||||
|
{"unknown source reference field", strings.Replace(valid, `"end_unit_id":1`, `"end_unit_id":1,"unexpected":true`, 1), "unknown field"},
|
||||||
|
{"invalid type", `{"locations":"not-an-array"}`, "cannot unmarshal string"},
|
||||||
|
{"trailing", `{"locations":[]} {}`, "multiple JSON values"},
|
||||||
|
} {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
if _, err := New().DecodeCandidate([]byte(test.raw)); err == nil || !strings.Contains(err.Error(), test.want) {
|
||||||
|
t.Fatalf("DecodeCandidate() error = %v, want %q", err, test.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCodecRejectsApprovedShapeBoundaries(t *testing.T) {
|
||||||
|
base := validList().Locations[0]
|
||||||
|
for _, test := range []struct {
|
||||||
|
name string
|
||||||
|
value dnd.LocationList
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{"nil locations", dnd.LocationList{}, "locations must be present"},
|
||||||
|
{"invalid ID", dnd.LocationList{Locations: []dnd.Location{{ID: "bad", Name: base.Name, SourceRefs: base.SourceRefs}}}, "id must match location ID pattern"},
|
||||||
|
{"blank name", dnd.LocationList{Locations: []dnd.Location{{ID: base.ID, Name: " ", SourceRefs: base.SourceRefs}}}, "name must not be empty"},
|
||||||
|
{"empty source refs", dnd.LocationList{Locations: []dnd.Location{{ID: base.ID, Name: base.Name, SourceRefs: nil}}}, "source_refs must contain"},
|
||||||
|
{"malformed source reference", dnd.LocationList{Locations: []dnd.Location{{ID: base.ID, Name: base.Name, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 0, EndUnitID: 1}}}}}, "start_unit_id must be positive"},
|
||||||
|
} {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
if _, err := New().Encode(test.value); err == nil || !strings.Contains(err.Error(), test.want) {
|
||||||
|
t.Fatalf("Encode() error = %v, want %q", err, test.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
11
internal/modules/dnd/codec/locations/testdata/dnd_locations.v1.json
vendored
Normal file
11
internal/modules/dnd/codec/locations/testdata/dnd_locations.v1.json
vendored
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"locations": [
|
||||||
|
{
|
||||||
|
"id": "location:sha256:cdd2b57615b56a4e92d506cf053cd3985e11d4df4895c5383b30ec2ce9f39ed0",
|
||||||
|
"name": "The Old Tavern",
|
||||||
|
"source_refs": [
|
||||||
|
{"source_id": "session-alpha", "start_unit_id": 1, "end_unit_id": 2}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
107
internal/modules/dnd/enemyevents/enemyevents.go
Normal file
107
internal/modules/dnd/enemyevents/enemyevents.go
Normal file
@@ -0,0 +1,107 @@
|
|||||||
|
// Package enemyevents owns canonical ordering and identity policy for D&D
|
||||||
|
// enemy-event artifacts.
|
||||||
|
package enemyevents
|
||||||
|
|
||||||
|
import (
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||||
|
)
|
||||||
|
|
||||||
|
// NormalizeDisplay returns the durable display form for an enemy subject.
|
||||||
|
func NormalizeDisplay(value string) string { return identity.NormalizeDisplay(value) }
|
||||||
|
|
||||||
|
// ComparisonKey returns the shared D&D identity key for an enemy subject.
|
||||||
|
func ComparisonKey(value string) string { return identity.ComparisonKey(NormalizeDisplay(value)) }
|
||||||
|
|
||||||
|
// SupportedKind reports whether kind belongs to the durable enemy-event vocabulary.
|
||||||
|
func SupportedKind(kind dnd.EnemyEventKind) bool {
|
||||||
|
_, ok := KindRank(kind)
|
||||||
|
return ok
|
||||||
|
}
|
||||||
|
|
||||||
|
// KindRank returns the explicit chronological tie-break order for event kinds.
|
||||||
|
func KindRank(kind dnd.EnemyEventKind) (int, bool) {
|
||||||
|
switch kind {
|
||||||
|
case dnd.EnemyEventKindEngaged:
|
||||||
|
return 0, true
|
||||||
|
case dnd.EnemyEventKindIncapacitated:
|
||||||
|
return 1, true
|
||||||
|
case dnd.EnemyEventKindCaptured:
|
||||||
|
return 2, true
|
||||||
|
case dnd.EnemyEventKindFled:
|
||||||
|
return 3, true
|
||||||
|
case dnd.EnemyEventKindKilled:
|
||||||
|
return 4, true
|
||||||
|
default:
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// SourceRefsEqual reports whether two reference sequences are equal after
|
||||||
|
// canonical ordering and exact duplicate removal.
|
||||||
|
func SourceRefsEqual(order shared.SourceRefOrder, left, right []source.SourceRef) bool {
|
||||||
|
left = order.Canonicalize(left)
|
||||||
|
right = order.Canonicalize(right)
|
||||||
|
if (left == nil) != (right == nil) || len(left) != len(right) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for index := range left {
|
||||||
|
if left[index] != right[index] {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// ExactEqual reports whether events have the same subject identity, kind, and
|
||||||
|
// complete canonical evidence sequence.
|
||||||
|
func ExactEqual(order shared.SourceRefOrder, left, right dnd.EnemyEvent) bool {
|
||||||
|
return ComparisonKey(left.Name) == ComparisonKey(right.Name) &&
|
||||||
|
left.Kind == right.Kind &&
|
||||||
|
SourceRefsEqual(order, left.SourceRefs, right.SourceRefs)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Less defines the canonical stable event order. Invalid source references
|
||||||
|
// remain comparable through SourceRefOrder's literal fallback so malformed
|
||||||
|
// candidates can still be sorted for later validation.
|
||||||
|
func Less(order shared.SourceRefOrder, left, right dnd.EnemyEvent) bool {
|
||||||
|
leftPosition, leftHasEvidence := order.EarliestValid(left.SourceRefs)
|
||||||
|
rightPosition, rightHasEvidence := order.EarliestValid(right.SourceRefs)
|
||||||
|
if leftHasEvidence != rightHasEvidence {
|
||||||
|
return leftHasEvidence
|
||||||
|
}
|
||||||
|
if leftHasEvidence && leftPosition != rightPosition {
|
||||||
|
return leftPosition < rightPosition
|
||||||
|
}
|
||||||
|
if leftKey, rightKey := ComparisonKey(left.Name), ComparisonKey(right.Name); leftKey != rightKey {
|
||||||
|
return leftKey < rightKey
|
||||||
|
}
|
||||||
|
if leftName, rightName := NormalizeDisplay(left.Name), NormalizeDisplay(right.Name); leftName != rightName {
|
||||||
|
return leftName < rightName
|
||||||
|
}
|
||||||
|
if leftRank, leftKnown := KindRank(left.Kind); leftKnown {
|
||||||
|
if rightRank, rightKnown := KindRank(right.Kind); rightKnown && leftRank != rightRank {
|
||||||
|
return leftRank < rightRank
|
||||||
|
} else if !rightKnown {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
} else if _, rightKnown := KindRank(right.Kind); rightKnown {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if left.Kind != right.Kind {
|
||||||
|
return left.Kind < right.Kind
|
||||||
|
}
|
||||||
|
return sourceRefsLess(order, order.Canonicalize(left.SourceRefs), order.Canonicalize(right.SourceRefs))
|
||||||
|
}
|
||||||
|
|
||||||
|
func sourceRefsLess(order shared.SourceRefOrder, left, right []source.SourceRef) bool {
|
||||||
|
for index := 0; index < len(left) && index < len(right); index++ {
|
||||||
|
if left[index] == right[index] {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
return order.Less(left[index], right[index])
|
||||||
|
}
|
||||||
|
return len(left) < len(right)
|
||||||
|
}
|
||||||
90
internal/modules/dnd/enemyevents/enemyevents_test.go
Normal file
90
internal/modules/dnd/enemyevents/enemyevents_test.go
Normal file
@@ -0,0 +1,90 @@
|
|||||||
|
package enemyevents
|
||||||
|
|
||||||
|
import (
|
||||||
|
"sort"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestSupportedKindAndRank(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
kind dnd.EnemyEventKind
|
||||||
|
rank int
|
||||||
|
}{
|
||||||
|
{dnd.EnemyEventKindEngaged, 0},
|
||||||
|
{dnd.EnemyEventKindIncapacitated, 1},
|
||||||
|
{dnd.EnemyEventKindCaptured, 2},
|
||||||
|
{dnd.EnemyEventKindFled, 3},
|
||||||
|
{dnd.EnemyEventKindKilled, 4},
|
||||||
|
}
|
||||||
|
for _, test := range tests {
|
||||||
|
if rank, ok := KindRank(test.kind); !ok || rank != test.rank || !SupportedKind(test.kind) {
|
||||||
|
t.Fatalf("kind %q = (%d, %t), supported %t", test.kind, rank, ok, SupportedKind(test.kind))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, ok := KindRank("unknown"); ok || SupportedKind("unknown") {
|
||||||
|
t.Fatal("unsupported kind was accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLessOrdersChronologyThenKind(t *testing.T) {
|
||||||
|
order := testOrder()
|
||||||
|
events := []dnd.EnemyEvent{
|
||||||
|
{Name: "Ashfang", Kind: dnd.EnemyEventKindKilled, SourceRefs: refs(20)},
|
||||||
|
{Name: "Ashfang", Kind: dnd.EnemyEventKindFled, SourceRefs: refs(20)},
|
||||||
|
{Name: "Ashfang", Kind: dnd.EnemyEventKindCaptured, SourceRefs: refs(20)},
|
||||||
|
{Name: "Ashfang", Kind: dnd.EnemyEventKindIncapacitated, SourceRefs: refs(20)},
|
||||||
|
{Name: "Ashfang", Kind: dnd.EnemyEventKindEngaged, SourceRefs: refs(20)},
|
||||||
|
{Name: "Later", Kind: dnd.EnemyEventKindEngaged, SourceRefs: refs(30)},
|
||||||
|
{Name: "Earlier", Kind: dnd.EnemyEventKindKilled, SourceRefs: refs(10)},
|
||||||
|
}
|
||||||
|
sort.SliceStable(events, func(left, right int) bool { return Less(order, events[left], events[right]) })
|
||||||
|
want := []dnd.EnemyEventKind{
|
||||||
|
dnd.EnemyEventKindKilled,
|
||||||
|
dnd.EnemyEventKindEngaged,
|
||||||
|
dnd.EnemyEventKindIncapacitated,
|
||||||
|
dnd.EnemyEventKindCaptured,
|
||||||
|
dnd.EnemyEventKindFled,
|
||||||
|
dnd.EnemyEventKindKilled,
|
||||||
|
dnd.EnemyEventKindEngaged,
|
||||||
|
}
|
||||||
|
for index, kind := range want {
|
||||||
|
if events[index].Kind != kind {
|
||||||
|
t.Fatalf("event %d kind = %q, want %q", index, events[index].Kind, kind)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExactEqualUsesSubjectIdentityAndCanonicalEvidence(t *testing.T) {
|
||||||
|
order := testOrder()
|
||||||
|
first := dnd.EnemyEvent{Name: " ASHFANG ", Kind: dnd.EnemyEventKindFled, SourceRefs: []source.SourceRef{
|
||||||
|
{SourceID: "session", StartUnitID: 30, EndUnitID: 30},
|
||||||
|
{SourceID: "session", StartUnitID: 10, EndUnitID: 10},
|
||||||
|
{SourceID: "session", StartUnitID: 10, EndUnitID: 10},
|
||||||
|
}}
|
||||||
|
second := dnd.EnemyEvent{Name: "Ashfang", Kind: dnd.EnemyEventKindFled, SourceRefs: []source.SourceRef{
|
||||||
|
{SourceID: "session", StartUnitID: 10, EndUnitID: 10},
|
||||||
|
{SourceID: "session", StartUnitID: 30, EndUnitID: 30},
|
||||||
|
}}
|
||||||
|
if !ExactEqual(order, first, second) || !SourceRefsEqual(order, first.SourceRefs, second.SourceRefs) {
|
||||||
|
t.Fatal("canonical duplicate identity was not recognized")
|
||||||
|
}
|
||||||
|
differentKind := second
|
||||||
|
differentKind.Kind = dnd.EnemyEventKindKilled
|
||||||
|
differentEvidence := second
|
||||||
|
differentEvidence.SourceRefs = []source.SourceRef{{SourceID: "session", StartUnitID: 20, EndUnitID: 20}}
|
||||||
|
if ExactEqual(order, first, differentKind) || ExactEqual(order, first, differentEvidence) {
|
||||||
|
t.Fatal("distinct event was treated as an exact duplicate")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func refs(unitID int) []source.SourceRef {
|
||||||
|
return []source.SourceRef{{SourceID: "session", StartUnitID: unitID, EndUnitID: unitID}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func testOrder() shared.SourceRefOrder {
|
||||||
|
return shared.NewSourceRefOrder(&source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 10}, {ID: 20}, {ID: 30}}})
|
||||||
|
}
|
||||||
6
internal/modules/dnd/extract/enemyevents/assets.go
Normal file
6
internal/modules/dnd/extract/enemyevents/assets.go
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
package enemyevents
|
||||||
|
|
||||||
|
import "embed"
|
||||||
|
|
||||||
|
//go:embed assets/schemas/dnd_enemy_events_llm.v1.json assets/prompts/*.yaml assets/prompts/*.md
|
||||||
|
var embeddedAssets embed.FS
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
id: dnd.enemy_events
|
||||||
|
version: "v1"
|
||||||
|
default_profile: dnd-extraction
|
||||||
|
inputs:
|
||||||
|
- name: transcript
|
||||||
|
required: true
|
||||||
|
content_type: application/json
|
||||||
|
- name: players
|
||||||
|
required: false
|
||||||
|
content_type: text/plain
|
||||||
|
- name: party
|
||||||
|
required: false
|
||||||
|
content_type: text/plain
|
||||||
|
- name: glossary
|
||||||
|
required: false
|
||||||
|
content_type: text/plain
|
||||||
|
- name: npcs
|
||||||
|
required: true
|
||||||
|
content_type: application/json
|
||||||
|
- name: combat_turns
|
||||||
|
required: true
|
||||||
|
content_type: application/json
|
||||||
|
- name: npc_interactions
|
||||||
|
required: true
|
||||||
|
content_type: application/json
|
||||||
|
messages:
|
||||||
|
- role: system
|
||||||
|
content_file: ./sharedassets/common-dnd-system.md
|
||||||
|
- role: user
|
||||||
|
content_file: ./sharedassets/common-dnd-identity.md
|
||||||
|
- role: user
|
||||||
|
content_file: ./sharedassets/common-dnd-references.md
|
||||||
|
cache_control:
|
||||||
|
type: ephemeral
|
||||||
|
- role: user
|
||||||
|
content_file: ./sharedassets/common-dnd-transcript.md
|
||||||
|
cache_control:
|
||||||
|
type: ephemeral
|
||||||
|
- role: user
|
||||||
|
content_file: ./sharedassets/common-dnd-extraction-evidence.md
|
||||||
|
- role: user
|
||||||
|
content_file: ./sharedassets/common-dnd-npcs.md
|
||||||
|
- role: user
|
||||||
|
content_file: ./grounding.md
|
||||||
|
- role: user
|
||||||
|
content_file: ./task.md
|
||||||
|
- role: user
|
||||||
|
content_file: ./instructions.md
|
||||||
|
cache_control:
|
||||||
|
type: ephemeral
|
||||||
|
output:
|
||||||
|
format: json
|
||||||
|
validation_mode: json_schema
|
||||||
|
schema_path: dnd_enemy_events_llm.v1.json
|
||||||
|
repair_attempts: 0
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
Compact combat grounding is supplied below. It can guide attention and
|
||||||
|
disambiguation, but it is not evidence. Do not derive an event, subject,
|
||||||
|
outcome, or source range from either list. The current transcript alone must
|
||||||
|
directly establish every returned event.
|
||||||
|
|
||||||
|
Combat-turn grounding:
|
||||||
|
|
||||||
|
{{ input "combat_turns" }}
|
||||||
|
|
||||||
|
Named combat-opponent grounding:
|
||||||
|
|
||||||
|
{{ input "npc_interactions" }}
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
Exclude party members, allies, neutral observers, mentioned-but-absent enemies,
|
||||||
|
hazards, traps, environmental effects, uncertain allegiance, table talk,
|
||||||
|
planning, hypotheses, recaps outside this passage, and downstream inference.
|
||||||
|
|
||||||
|
Do not infer an engagement or outcome from initiative, turn absence, damage,
|
||||||
|
defeat, movement, a scene ending, combat-opponent grounding, or any auxiliary
|
||||||
|
artifact. Auxiliary inputs can guide attention but cannot prove or supply an
|
||||||
|
event. Cite only narrow current-transcript ranges that establish each event.
|
||||||
|
|
||||||
|
Return the `events` array even when no enemy event is established. Every event
|
||||||
|
must contain only `name`, `kind`, and `source_refs`. Use exactly one kind:
|
||||||
|
`engaged`, `killed`, `fled`, `captured`, or `incapacitated`. Each source range
|
||||||
|
uses integer `start_unit_id` and `end_unit_id`; omit `source_id` because
|
||||||
|
Notarius assigns the current source identity.
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
Extract Dungeons & Dragons enemy events from the supplied combat transcript.
|
||||||
|
|
||||||
|
Return an `engaged` event only when the transcript directly establishes that a
|
||||||
|
subject is actively opposing the party in combat. Return `killed`, `fled`,
|
||||||
|
`captured`, or `incapacitated` only when the transcript explicitly establishes
|
||||||
|
that outcome. An outcome may share evidence with an engagement, and a later
|
||||||
|
engagement or outcome for the same subject remains a separate observation.
|
||||||
|
Emit at most one engagement for the same subject in this combat scene.
|
||||||
|
|
||||||
|
For `killed`, direct death or killing is required. For `fled`, the subject must
|
||||||
|
explicitly escape, retreat, or leave combat to avoid continued engagement. For
|
||||||
|
`captured`, the subject must be explicitly taken prisoner or secured under the
|
||||||
|
party's control. For `incapacitated`, the subject must be explicitly unable to
|
||||||
|
continue acting without being established as killed or captured.
|
||||||
|
|
||||||
|
Use a normalized NPC registry spelling when the transcript identifies that
|
||||||
|
named NPC. A hostile creature without a registry entry is allowed. For unnamed
|
||||||
|
individuals or groups, use only the narrowest transcript-grounded label, such
|
||||||
|
as `Orcs`, `One orc`, or `Remaining orcs`; never invent member names, IDs, or
|
||||||
|
quantities.
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"$id": "notarius.dnd.enemy_events.llm",
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["events"],
|
||||||
|
"properties": {
|
||||||
|
"events": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["name", "kind", "source_refs"],
|
||||||
|
"properties": {
|
||||||
|
"name": {"type": "string"},
|
||||||
|
"kind": {"type": "string"},
|
||||||
|
"source_refs": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["start_unit_id", "end_unit_id"],
|
||||||
|
"properties": {
|
||||||
|
"start_unit_id": {"type": "integer"},
|
||||||
|
"end_unit_id": {"type": "integer"}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
210
internal/modules/dnd/extract/enemyevents/extractor.go
Normal file
210
internal/modules/dnd/extract/enemyevents/extractor.go
Normal file
@@ -0,0 +1,210 @@
|
|||||||
|
package enemyevents
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
sceneregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/scenedescriptions/registry"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
Key = "dnd/enemy-events"
|
||||||
|
mappingPolicy = "dnd.enemy_events.extract_mapping.v1"
|
||||||
|
sceneGatePolicy = "dnd.enemy_events.scene_gate.v1"
|
||||||
|
)
|
||||||
|
|
||||||
|
var requiredCapabilities = []string{
|
||||||
|
"chunks",
|
||||||
|
"source.transcript",
|
||||||
|
}
|
||||||
|
|
||||||
|
var providedCapabilities = []string{
|
||||||
|
"dnd.enemy_events",
|
||||||
|
}
|
||||||
|
|
||||||
|
var _ contracts.Extractor[dnd.EnemyEventList] = (*Extractor)(nil)
|
||||||
|
var _ contracts.ManifestMetadataProvider = (*Extractor)(nil)
|
||||||
|
var _ pipeline.CheckpointFingerprintProvider = (*Extractor)(nil)
|
||||||
|
|
||||||
|
type Options struct{}
|
||||||
|
|
||||||
|
type Extractor struct {
|
||||||
|
llm contracts.StructuredLLMClient
|
||||||
|
grounding *groundingResolver
|
||||||
|
|
||||||
|
promptSHA string
|
||||||
|
responseSchemaSHA string
|
||||||
|
}
|
||||||
|
|
||||||
|
func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contracts.ReferenceSet) (*Extractor, error) {
|
||||||
|
if llmClient == nil {
|
||||||
|
return nil, extractorErrorf("LLM client must not be nil")
|
||||||
|
}
|
||||||
|
if len(references) > 1 {
|
||||||
|
return nil, extractorErrorf("at most one reference set may be supplied")
|
||||||
|
}
|
||||||
|
var referenceSet contracts.ReferenceSet
|
||||||
|
if len(references) == 1 {
|
||||||
|
referenceSet = references[0]
|
||||||
|
}
|
||||||
|
grounding, err := newGroundingResolver(referenceSet)
|
||||||
|
if err != nil {
|
||||||
|
return nil, extractorErrorf("prepare grounding: %w", err)
|
||||||
|
}
|
||||||
|
promptSHA, err := promptAssetMetadata()
|
||||||
|
if err != nil {
|
||||||
|
return nil, extractorErrorf("load prompt metadata: %w", err)
|
||||||
|
}
|
||||||
|
responseSchema, err := loadResponseSchema()
|
||||||
|
if err != nil {
|
||||||
|
return nil, extractorErrorf("load response schema: %w", err)
|
||||||
|
}
|
||||||
|
return &Extractor{
|
||||||
|
llm: llmClient,
|
||||||
|
grounding: grounding,
|
||||||
|
promptSHA: promptSHA,
|
||||||
|
responseSchemaSHA: responseSchema.SHA256,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Extractor) Key() string { return Key }
|
||||||
|
|
||||||
|
func (e *Extractor) ReferenceSlots() []contracts.ReferenceSlot { return referenceSlots() }
|
||||||
|
|
||||||
|
func (e *Extractor) ManifestMetadata() map[string]any {
|
||||||
|
if e == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return map[string]any{
|
||||||
|
"prompt_id": PromptID,
|
||||||
|
"prompt_version": SchemaVersion,
|
||||||
|
"prompt_sha256": e.promptSHA,
|
||||||
|
"mapping_policy": mappingPolicy,
|
||||||
|
"scene_gate_policy": sceneGatePolicy,
|
||||||
|
"response_schema_key": string(ResponseSchemaKey),
|
||||||
|
"response_schema_id": ResponseSchemaID,
|
||||||
|
"response_schema_name": ResponseSchemaName,
|
||||||
|
"response_schema_version": SchemaVersion,
|
||||||
|
"response_schema_sha256": e.responseSchemaSHA,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Extractor) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
|
||||||
|
if e == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return []pipeline.CheckpointFingerprint{
|
||||||
|
{Name: "prompt", Value: e.promptSHA},
|
||||||
|
{Name: "response_schema", Value: e.responseSchemaSHA},
|
||||||
|
{Name: "mapping_policy", Value: mappingPolicy},
|
||||||
|
{Name: "scene_gate_policy", Value: sceneGatePolicy},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Extractor) Extract(ctx context.Context, req contracts.TypedExtractionRequest) (contracts.TypedExtractionResult[dnd.EnemyEventList], error) {
|
||||||
|
if e == nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("extractor must not be nil")
|
||||||
|
}
|
||||||
|
if e.llm == nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("LLM client must not be nil")
|
||||||
|
}
|
||||||
|
if e.grounding == nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("grounding resolver must not be nil")
|
||||||
|
}
|
||||||
|
sourceInput, err := shared.PrepareChunkExtraction(ctx, req)
|
||||||
|
if err != nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("%w", err)
|
||||||
|
}
|
||||||
|
match, err := e.grounding.SceneMatch(req.References, req.Chunk)
|
||||||
|
if err != nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("resolve scene eligibility: %w", err)
|
||||||
|
}
|
||||||
|
switch match.State {
|
||||||
|
case sceneregistry.MatchExact:
|
||||||
|
if match.Kind != dnd.SceneKindCombat {
|
||||||
|
return emptyResult(), nil
|
||||||
|
}
|
||||||
|
case sceneregistry.MatchMissing, sceneregistry.MatchMismatched:
|
||||||
|
return unavailableSceneResult(), nil
|
||||||
|
default:
|
||||||
|
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("unsupported scene eligibility match state %q", match.State)
|
||||||
|
}
|
||||||
|
grounding, err := e.grounding.Resolve(req.References)
|
||||||
|
if err != nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("resolve enemy-event grounding: %w", err)
|
||||||
|
}
|
||||||
|
inputs := shared.PromptInputs(sourceInput, req.References)
|
||||||
|
for name, input := range grounding.PromptInputs() {
|
||||||
|
inputs[name] = input
|
||||||
|
}
|
||||||
|
var response extractionResponse
|
||||||
|
if _, err := e.llm.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
|
||||||
|
StageName: Key,
|
||||||
|
PromptID: PromptID,
|
||||||
|
PromptVersion: SchemaVersion,
|
||||||
|
ProfileID: req.LLMProfile,
|
||||||
|
SessionID: req.SessionID,
|
||||||
|
Inputs: inputs,
|
||||||
|
}, &response); err != nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("complete structured output: %w", err)
|
||||||
|
}
|
||||||
|
return contracts.TypedExtractionResult[dnd.EnemyEventList]{
|
||||||
|
Value: canonicalEnemyEventList(response, shared.NewSourceRefOrder(req.Source), req.Source.ID),
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func emptyResult() contracts.TypedExtractionResult[dnd.EnemyEventList] {
|
||||||
|
return contracts.TypedExtractionResult[dnd.EnemyEventList]{Value: dnd.EnemyEventList{Events: []dnd.EnemyEvent{}}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func unavailableSceneResult() contracts.TypedExtractionResult[dnd.EnemyEventList] {
|
||||||
|
result := emptyResult()
|
||||||
|
result.Warnings = []contracts.Warning{{
|
||||||
|
Scope: SceneDescriptionReferenceSlot,
|
||||||
|
ReasonCode: "scene_classification_unavailable",
|
||||||
|
Message: "No exact scene classification was available; enemy-event extraction was skipped.",
|
||||||
|
}}
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
func ModuleSpec() pipeline.ModuleSpec {
|
||||||
|
return pipeline.ModuleSpec{
|
||||||
|
Key: Key,
|
||||||
|
Stage: pipeline.StageExtract,
|
||||||
|
ExecutionClass: contracts.ExecutionClassLLMBacked,
|
||||||
|
Requires: append([]string(nil), requiredCapabilities...),
|
||||||
|
Provides: append([]string(nil), providedCapabilities...),
|
||||||
|
ArtifactKind: dnd.EnemyEventListKind,
|
||||||
|
ReferenceSlots: referenceSlots(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func Register(registry *pipeline.ExtractorRegistry) error {
|
||||||
|
return pipeline.RegisterExtractorBuilder(registry, ModuleSpec(), validateOptions, func(request pipeline.BuildRequest) (contracts.Extractor[dnd.EnemyEventList], error) {
|
||||||
|
options, err := DecodeOptions(request.Options)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return New(request.Dependencies.LLM, options, request.References)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateOptions(options map[string]any) error {
|
||||||
|
_, err := DecodeOptions(options)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func DecodeOptions(options map[string]any) (Options, error) {
|
||||||
|
if err := pipeline.RejectUnknownOptions(options); err != nil {
|
||||||
|
return Options{}, extractorErrorf("%w", err)
|
||||||
|
}
|
||||||
|
return Options{}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func extractorErrorf(format string, args ...any) error {
|
||||||
|
return fmt.Errorf("dnd enemy events extractor: "+format, args...)
|
||||||
|
}
|
||||||
229
internal/modules/dnd/extract/enemyevents/extractor_test.go
Normal file
229
internal/modules/dnd/extract/enemyevents/extractor_test.go
Normal file
@@ -0,0 +1,229 @@
|
|||||||
|
package enemyevents
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestExtractMapsEnemyEventsInSourceOrder(t *testing.T) {
|
||||||
|
client := &fakeEnemyEventsLLMClient{response: extractionResponse{Events: []enemyEventResponse{
|
||||||
|
{Name: "Ashfang", Kind: "killed", SourceRefs: []enemySourceRefResponse{{StartUnitID: 4, EndUnitID: 4}}},
|
||||||
|
{Name: "Ashfang", Kind: "engaged", SourceRefs: []enemySourceRefResponse{{StartUnitID: 1, EndUnitID: 1}, {StartUnitID: 1, EndUnitID: 1}}},
|
||||||
|
{Name: "Orcs", Kind: "fled", SourceRefs: []enemySourceRefResponse{{StartUnitID: 3, EndUnitID: 3}}},
|
||||||
|
{Name: "One orc", Kind: "captured", SourceRefs: []enemySourceRefResponse{{StartUnitID: 2, EndUnitID: 2}}},
|
||||||
|
{Name: "Ashfang", Kind: "incapacitated", SourceRefs: []enemySourceRefResponse{{StartUnitID: 2, EndUnitID: 2}}},
|
||||||
|
}}}
|
||||||
|
|
||||||
|
result, err := newEnemyExtractor(t, client).Extract(context.Background(), enemyExtractionRequest(t))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got := []dnd.EnemyEventKind{result.Value.Events[0].Kind, result.Value.Events[1].Kind, result.Value.Events[2].Kind, result.Value.Events[3].Kind, result.Value.Events[4].Kind}; !reflect.DeepEqual(got, []dnd.EnemyEventKind{
|
||||||
|
dnd.EnemyEventKindEngaged,
|
||||||
|
dnd.EnemyEventKindCaptured,
|
||||||
|
dnd.EnemyEventKindIncapacitated,
|
||||||
|
dnd.EnemyEventKindFled,
|
||||||
|
dnd.EnemyEventKindKilled,
|
||||||
|
}) {
|
||||||
|
t.Fatalf("event order = %#v", got)
|
||||||
|
}
|
||||||
|
if refs := result.Value.Events[0].SourceRefs; !reflect.DeepEqual(refs, []source.SourceRef{{SourceID: "combat-session", StartUnitID: 1, EndUnitID: 1}}) {
|
||||||
|
t.Fatalf("canonical source refs = %#v", refs)
|
||||||
|
}
|
||||||
|
if len(client.requests) != 1 {
|
||||||
|
t.Fatalf("LLM calls = %d, want 1", len(client.requests))
|
||||||
|
}
|
||||||
|
request := client.requests[0]
|
||||||
|
if request.StageName != Key || request.PromptID != PromptID || request.PromptVersion != SchemaVersion || request.ProfileID != "enemy-profile" || request.SessionID != "session-123" {
|
||||||
|
t.Fatalf("LLM request identity = %#v", request)
|
||||||
|
}
|
||||||
|
if got := string(request.Inputs[CombatTurnReferenceSlot].Content); !strings.Contains(got, `"actor":"Ashfang"`) || strings.Contains(got, "source_ref") {
|
||||||
|
t.Fatalf("combat grounding = %s", got)
|
||||||
|
}
|
||||||
|
if got := string(request.Inputs[NPCInteractionReferenceSlot].Content); !strings.Contains(got, `"kind":"combat_opponent"`) || strings.Contains(got, "Aria") {
|
||||||
|
t.Fatalf("interaction grounding = %s", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractPreservesSemanticCandidatesAndResponseOwnership(t *testing.T) {
|
||||||
|
client := &fakeEnemyEventsLLMClient{response: extractionResponse{Events: []enemyEventResponse{{
|
||||||
|
Name: " ", Kind: "unsupported", SourceRefs: []enemySourceRefResponse{{StartUnitID: 99, EndUnitID: 0}},
|
||||||
|
}}}}
|
||||||
|
result, err := newEnemyExtractor(t, client).Extract(context.Background(), enemyExtractionRequest(t))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
event := result.Value.Events[0]
|
||||||
|
if event.Name != " " || event.Kind != "unsupported" || event.SourceRefs[0] != (source.SourceRef{SourceID: "combat-session", StartUnitID: 99}) {
|
||||||
|
t.Fatalf("semantic candidate = %#v", event)
|
||||||
|
}
|
||||||
|
result.Value.Events[0].SourceRefs[0].StartUnitID = 7
|
||||||
|
if client.response.Events[0].SourceRefs[0].StartUnitID != 99 {
|
||||||
|
t.Fatal("mapped event aliases model-owned source references")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractSkipsModelForIneligibleScenes(t *testing.T) {
|
||||||
|
for _, test := range []struct {
|
||||||
|
name string
|
||||||
|
kind dnd.SceneKind
|
||||||
|
change func(*source.Chunk)
|
||||||
|
wantWarning bool
|
||||||
|
}{
|
||||||
|
{name: "non-combat", kind: dnd.SceneKindNarrative},
|
||||||
|
{name: "missing", kind: dnd.SceneKindCombat, change: func(chunk *source.Chunk) { chunk.ID = "other" }, wantWarning: true},
|
||||||
|
{name: "mismatched", kind: dnd.SceneKindCombat, change: func(chunk *source.Chunk) { chunk.Ref.EndUnitID++ }, wantWarning: true},
|
||||||
|
} {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
client := &fakeEnemyEventsLLMClient{}
|
||||||
|
request := enemyExtractionRequest(t)
|
||||||
|
request.References = groundingReferences(t, "Ashfang", test.kind)
|
||||||
|
if test.change != nil {
|
||||||
|
test.change(request.Chunk)
|
||||||
|
}
|
||||||
|
result, err := newEnemyExtractor(t, client).Extract(context.Background(), request)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(client.requests) != 0 || result.Value.Events == nil || len(result.Value.Events) != 0 {
|
||||||
|
t.Fatalf("result = %#v, calls = %d", result, len(client.requests))
|
||||||
|
}
|
||||||
|
if test.wantWarning {
|
||||||
|
if len(result.Warnings) != 1 || result.Warnings[0].ReasonCode != "scene_classification_unavailable" || result.Warnings[0].Scope != SceneDescriptionReferenceSlot {
|
||||||
|
t.Fatalf("warnings = %#v", result.Warnings)
|
||||||
|
}
|
||||||
|
} else if len(result.Warnings) != 0 {
|
||||||
|
t.Fatalf("warnings = %#v", result.Warnings)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractReportsRequiredGroundingAndProviderFailures(t *testing.T) {
|
||||||
|
client := &fakeEnemyEventsLLMClient{}
|
||||||
|
request := enemyExtractionRequest(t)
|
||||||
|
request.References = withoutSlot(request.References, NPCRegistryReferenceSlot)
|
||||||
|
if _, err := (&Extractor{llm: client, grounding: mustGroundingResolver(t, request.References)}).Extract(context.Background(), request); err == nil || !strings.Contains(err.Error(), "NPC registry") {
|
||||||
|
t.Fatalf("Extract() error = %v, want required grounding context", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
provider := &fakeEnemyEventsLLMClient{err: errors.New("provider unavailable")}
|
||||||
|
if _, err := newEnemyExtractor(t, provider).Extract(context.Background(), enemyExtractionRequest(t)); err == nil || !strings.Contains(err.Error(), "complete structured output") || !strings.Contains(err.Error(), "provider unavailable") {
|
||||||
|
t.Fatalf("Extract() error = %v, want provider context", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestConstructorSpecOptionsAndSafeMetadata(t *testing.T) {
|
||||||
|
if _, err := New(nil, Options{}); err == nil || !strings.Contains(err.Error(), "LLM client") {
|
||||||
|
t.Fatalf("New(nil) error = %v", err)
|
||||||
|
}
|
||||||
|
if _, err := New(&fakeEnemyEventsLLMClient{}, Options{}, contracts.ReferenceSet{}, contracts.ReferenceSet{}); err == nil || !strings.Contains(err.Error(), "at most one") {
|
||||||
|
t.Fatalf("New() error = %v", err)
|
||||||
|
}
|
||||||
|
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil || !strings.Contains(err.Error(), "unknown option") {
|
||||||
|
t.Fatalf("DecodeOptions() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
first := ModuleSpec()
|
||||||
|
first.Requires[0] = "changed"
|
||||||
|
first.ReferenceSlots[0].AcceptedMediaTypes[0] = "changed"
|
||||||
|
second := ModuleSpec()
|
||||||
|
if second.Requires[0] != "chunks" || second.ArtifactKind != dnd.EnemyEventListKind || second.ExecutionClass != contracts.ExecutionClassLLMBacked || second.ReferenceSlots[0].AcceptedMediaTypes[0] == "changed" {
|
||||||
|
t.Fatalf("ModuleSpec() reused mutable state: %#v", second)
|
||||||
|
}
|
||||||
|
registry := pipeline.NewExtractorRegistry()
|
||||||
|
if err := Register(registry); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if spec, ok := registry.Spec(Key); !ok || spec.Key != Key || spec.ArtifactKind != dnd.EnemyEventListKind {
|
||||||
|
t.Fatalf("registered spec = %#v, %t", spec, ok)
|
||||||
|
}
|
||||||
|
|
||||||
|
extractor := newEnemyExtractor(t, &fakeEnemyEventsLLMClient{}, groundingReferences(t, "Ashfang", dnd.SceneKindCombat))
|
||||||
|
metadata := extractor.ManifestMetadata()
|
||||||
|
encoded, err := json.Marshal(metadata)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if strings.Contains(string(encoded), "Ashfang") || metadata["mapping_policy"] != mappingPolicy || metadata["scene_gate_policy"] != sceneGatePolicy {
|
||||||
|
t.Fatalf("unsafe or incomplete metadata = %s", encoded)
|
||||||
|
}
|
||||||
|
fingerprints := extractor.CheckpointFingerprints()
|
||||||
|
if len(fingerprints) != 4 || fingerprints[0].Value != metadata["prompt_sha256"] || fingerprints[1].Value != metadata["response_schema_sha256"] || fingerprints[2].Value != mappingPolicy || fingerprints[3].Value != sceneGatePolicy {
|
||||||
|
t.Fatalf("fingerprints = %#v", fingerprints)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func newEnemyExtractor(t *testing.T, client contracts.StructuredLLMClient, references ...contracts.ReferenceSet) *Extractor {
|
||||||
|
t.Helper()
|
||||||
|
if len(references) == 0 {
|
||||||
|
references = []contracts.ReferenceSet{groundingReferences(t, "Ashfang", dnd.SceneKindCombat)}
|
||||||
|
}
|
||||||
|
extractor, err := New(client, Options{}, references...)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return extractor
|
||||||
|
}
|
||||||
|
|
||||||
|
func mustGroundingResolver(t *testing.T, references contracts.ReferenceSet) *groundingResolver {
|
||||||
|
t.Helper()
|
||||||
|
resolver, err := newGroundingResolver(references)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return resolver
|
||||||
|
}
|
||||||
|
|
||||||
|
func enemyExtractionRequest(t *testing.T) contracts.TypedExtractionRequest {
|
||||||
|
t.Helper()
|
||||||
|
chunk := combatChunk()
|
||||||
|
chunk.Units = []source.SourceUnit{{ID: 1}, {ID: 2}, {ID: 3}, {ID: 4}}
|
||||||
|
chunk.MediaType = "application/json"
|
||||||
|
chunk.Content = []byte(`{"id":"combat-scene","units":[1,2,3,4]}`)
|
||||||
|
return contracts.TypedExtractionRequest{
|
||||||
|
Source: &source.SourceDocument{ID: "combat-session", Units: append([]source.SourceUnit(nil), chunk.Units...)},
|
||||||
|
Chunk: chunk,
|
||||||
|
SourceInput: contracts.NewLLMInputMaterial("source", "application/json", chunk.Content, digest(chunk.Content), "file:///combat-session.json"),
|
||||||
|
References: groundingReferences(t, "Ashfang", dnd.SceneKindCombat),
|
||||||
|
LLMProfile: "enemy-profile",
|
||||||
|
SessionID: "session-123",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type fakeEnemyEventsLLMClient struct {
|
||||||
|
response extractionResponse
|
||||||
|
err error
|
||||||
|
requests []contracts.StructuredCompletionRequest
|
||||||
|
}
|
||||||
|
|
||||||
|
func (client *fakeEnemyEventsLLMClient) CompleteStructured(_ context.Context, request contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
|
||||||
|
client.requests = append(client.requests, cloneEnemyRequest(request))
|
||||||
|
if client.err != nil {
|
||||||
|
return contracts.StructuredCompletionResponse{}, client.err
|
||||||
|
}
|
||||||
|
target, ok := out.(*extractionResponse)
|
||||||
|
if !ok {
|
||||||
|
return contracts.StructuredCompletionResponse{}, errors.New("unexpected output target")
|
||||||
|
}
|
||||||
|
*target = client.response
|
||||||
|
content, err := json.Marshal(client.response)
|
||||||
|
if err != nil {
|
||||||
|
return contracts.StructuredCompletionResponse{}, err
|
||||||
|
}
|
||||||
|
return contracts.StructuredCompletionResponse{Content: content}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func cloneEnemyRequest(request contracts.StructuredCompletionRequest) contracts.StructuredCompletionRequest {
|
||||||
|
request.Inputs = request.Inputs.Clone()
|
||||||
|
return request
|
||||||
|
}
|
||||||
293
internal/modules/dnd/extract/enemyevents/grounding.go
Normal file
293
internal/modules/dnd/extract/enemyevents/grounding.go
Normal file
@@ -0,0 +1,293 @@
|
|||||||
|
// Package enemyevents prepares validated D&D combat grounding for enemy-event
|
||||||
|
// extraction.
|
||||||
|
package enemyevents
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"mime"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
combatturncodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/combatturns"
|
||||||
|
interactioncodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcinteractions"
|
||||||
|
npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs"
|
||||||
|
scenecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
|
||||||
|
npcregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/registry"
|
||||||
|
sceneregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/scenedescriptions/registry"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
NPCRegistryReferenceSlot = npcregistry.ReferenceSlot
|
||||||
|
SceneDescriptionReferenceSlot = sceneregistry.ReferenceSlot
|
||||||
|
CombatTurnReferenceSlot = "combat_turns"
|
||||||
|
NPCInteractionReferenceSlot = "npc_interactions"
|
||||||
|
ReferenceMaxBytes = 1048576
|
||||||
|
promptProjectionMediaType = "application/json"
|
||||||
|
)
|
||||||
|
|
||||||
|
var referenceSlotDescriptions = shared.ReferenceSlotDescriptions{
|
||||||
|
Glossary: "Optional campaign glossary reference material used only for disambiguation.",
|
||||||
|
Party: "Optional party roster reference material used only for disambiguation.",
|
||||||
|
Players: "Optional player list reference material used only for disambiguation.",
|
||||||
|
Roster: "Deprecated alias for party roster reference material used only for disambiguation.",
|
||||||
|
}
|
||||||
|
|
||||||
|
func referenceSlots() []contracts.ReferenceSlot {
|
||||||
|
slots := shared.ReferenceSlots(referenceSlotDescriptions)
|
||||||
|
slots = append(slots,
|
||||||
|
contracts.ReferenceSlot{
|
||||||
|
Name: NPCRegistryReferenceSlot,
|
||||||
|
Description: "Required normalized NPC registry used only for enemy-subject grounding, never as event evidence.",
|
||||||
|
Required: true,
|
||||||
|
AcceptedMediaTypes: []string{npccodec.MediaType},
|
||||||
|
AcceptedArtifactKinds: []contracts.ArtifactKind{dnd.NPCListKind},
|
||||||
|
MaxBytes: ReferenceMaxBytes,
|
||||||
|
},
|
||||||
|
contracts.ReferenceSlot{
|
||||||
|
Name: SceneDescriptionReferenceSlot,
|
||||||
|
Description: "Required scene descriptions used only to determine exact combat eligibility, never as event evidence.",
|
||||||
|
Required: true,
|
||||||
|
AcceptedMediaTypes: []string{scenecodec.MediaType},
|
||||||
|
AcceptedArtifactKinds: []contracts.ArtifactKind{dnd.SceneDescriptionListKind},
|
||||||
|
MaxBytes: ReferenceMaxBytes,
|
||||||
|
},
|
||||||
|
contracts.ReferenceSlot{
|
||||||
|
Name: CombatTurnReferenceSlot,
|
||||||
|
Description: "Required combat-turn artifact used only as source-free enemy-event grounding.",
|
||||||
|
Required: true,
|
||||||
|
AcceptedMediaTypes: []string{combatturncodec.MediaType},
|
||||||
|
AcceptedArtifactKinds: []contracts.ArtifactKind{dnd.CombatTurnListKind},
|
||||||
|
MaxBytes: ReferenceMaxBytes,
|
||||||
|
},
|
||||||
|
contracts.ReferenceSlot{
|
||||||
|
Name: NPCInteractionReferenceSlot,
|
||||||
|
Description: "Required NPC-interaction artifact used only as source-free enemy-event grounding.",
|
||||||
|
Required: true,
|
||||||
|
AcceptedMediaTypes: []string{interactioncodec.MediaType},
|
||||||
|
AcceptedArtifactKinds: []contracts.ArtifactKind{dnd.NPCInteractionListKind},
|
||||||
|
MaxBytes: ReferenceMaxBytes,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
sort.Slice(slots, func(left, right int) bool { return slots[left].Name < slots[right].Name })
|
||||||
|
return contracts.CloneReferenceSlots(slots)
|
||||||
|
}
|
||||||
|
|
||||||
|
// groundingResolver retains only validated, compact construction-time views.
|
||||||
|
// Per-operation generated references are decoded when supplied and never
|
||||||
|
// become static metadata.
|
||||||
|
type groundingResolver struct {
|
||||||
|
npcs *npcregistry.Resolver
|
||||||
|
scenes *sceneregistry.Resolver
|
||||||
|
|
||||||
|
combatTurns *contracts.LLMInputMaterial
|
||||||
|
npcInteractions *contracts.LLMInputMaterial
|
||||||
|
}
|
||||||
|
|
||||||
|
type grounding struct {
|
||||||
|
npcInput contracts.LLMInputMaterial
|
||||||
|
combatTurnInput contracts.LLMInputMaterial
|
||||||
|
npcInteractionInput contracts.LLMInputMaterial
|
||||||
|
}
|
||||||
|
|
||||||
|
func newGroundingResolver(references contracts.ReferenceSet) (*groundingResolver, error) {
|
||||||
|
npcs, err := npcregistry.NewResolver(references)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("prepare NPC registry grounding: %w", err)
|
||||||
|
}
|
||||||
|
scenes, err := sceneregistry.NewResolver(references)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("prepare scene eligibility: %w", err)
|
||||||
|
}
|
||||||
|
combatTurns, err := prepareCombatTurnInput(references)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
npcInteractions, err := prepareNPCInteractionInput(references)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &groundingResolver{
|
||||||
|
npcs: npcs,
|
||||||
|
scenes: scenes,
|
||||||
|
combatTurns: combatTurns,
|
||||||
|
npcInteractions: npcInteractions,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *groundingResolver) Resolve(references contracts.ReferenceSet) (grounding, error) {
|
||||||
|
if r == nil {
|
||||||
|
return grounding{}, fmt.Errorf("grounding resolver must not be nil")
|
||||||
|
}
|
||||||
|
npcs, err := r.npcs.Resolve(references)
|
||||||
|
if err != nil {
|
||||||
|
return grounding{}, fmt.Errorf("resolve NPC registry grounding: %w", err)
|
||||||
|
}
|
||||||
|
if !npcs.Bound() {
|
||||||
|
return grounding{}, fmt.Errorf("NPC registry reference is required")
|
||||||
|
}
|
||||||
|
combatTurns, err := resolveInput(references, CombatTurnReferenceSlot, r.combatTurns, prepareCombatTurnInput)
|
||||||
|
if err != nil {
|
||||||
|
return grounding{}, err
|
||||||
|
}
|
||||||
|
npcInteractions, err := resolveInput(references, NPCInteractionReferenceSlot, r.npcInteractions, prepareNPCInteractionInput)
|
||||||
|
if err != nil {
|
||||||
|
return grounding{}, err
|
||||||
|
}
|
||||||
|
return grounding{
|
||||||
|
npcInput: npcs.PromptInput(),
|
||||||
|
combatTurnInput: combatTurns,
|
||||||
|
npcInteractionInput: npcInteractions,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *groundingResolver) SceneMatch(references contracts.ReferenceSet, chunk *source.Chunk) (sceneregistry.ChunkMatch, error) {
|
||||||
|
if r == nil {
|
||||||
|
return sceneregistry.ChunkMatch{}, fmt.Errorf("grounding resolver must not be nil")
|
||||||
|
}
|
||||||
|
scenes, err := r.resolveScenes(references)
|
||||||
|
if err != nil {
|
||||||
|
return sceneregistry.ChunkMatch{}, err
|
||||||
|
}
|
||||||
|
return scenes.Match(chunk), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *groundingResolver) resolveScenes(references contracts.ReferenceSet) (*sceneregistry.Registry, error) {
|
||||||
|
scenes, err := r.scenes.Resolve(references)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("resolve scene eligibility: %w", err)
|
||||||
|
}
|
||||||
|
if !scenes.Bound() {
|
||||||
|
return nil, fmt.Errorf("scene descriptions reference is required")
|
||||||
|
}
|
||||||
|
return scenes, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveInput(references contracts.ReferenceSet, slot string, seeded *contracts.LLMInputMaterial, prepare func(contracts.ReferenceSet) (*contracts.LLMInputMaterial, error)) (contracts.LLMInputMaterial, error) {
|
||||||
|
if _, ok := references.Slots[slot]; ok {
|
||||||
|
value, err := prepare(references)
|
||||||
|
if err != nil {
|
||||||
|
return contracts.LLMInputMaterial{}, err
|
||||||
|
}
|
||||||
|
if value == nil {
|
||||||
|
return contracts.LLMInputMaterial{}, fmt.Errorf("%s reference is required", slot)
|
||||||
|
}
|
||||||
|
return value.Clone(), nil
|
||||||
|
}
|
||||||
|
if seeded == nil {
|
||||||
|
return contracts.LLMInputMaterial{}, fmt.Errorf("%s reference is required", slot)
|
||||||
|
}
|
||||||
|
return seeded.Clone(), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (g grounding) PromptInputs() contracts.LLMInputSet {
|
||||||
|
return contracts.LLMInputSet{
|
||||||
|
NPCRegistryReferenceSlot: g.npcInput.Clone(),
|
||||||
|
CombatTurnReferenceSlot: g.combatTurnInput.Clone(),
|
||||||
|
NPCInteractionReferenceSlot: g.npcInteractionInput.Clone(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func prepareCombatTurnInput(references contracts.ReferenceSet) (*contracts.LLMInputMaterial, error) {
|
||||||
|
item, ok, err := referenceItem(references, CombatTurnReferenceSlot, combatturncodec.MediaType)
|
||||||
|
if err != nil || !ok {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
value, err := combatturncodec.New().Decode(item.Content)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("decode combat-turn grounding: invalid approved combat-turn JSON")
|
||||||
|
}
|
||||||
|
content, err := json.Marshal(struct {
|
||||||
|
CombatTurns []combatTurnProjection `json:"combat_turns"`
|
||||||
|
}{CombatTurns: projectCombatTurns(value.CombatTurns)})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("encode combat-turn grounding: %w", err)
|
||||||
|
}
|
||||||
|
return newPromptInput(CombatTurnReferenceSlot, content), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func prepareNPCInteractionInput(references contracts.ReferenceSet) (*contracts.LLMInputMaterial, error) {
|
||||||
|
item, ok, err := referenceItem(references, NPCInteractionReferenceSlot, interactioncodec.MediaType)
|
||||||
|
if err != nil || !ok {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
value, err := interactioncodec.New().Decode(item.Content)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("decode NPC-interaction grounding: invalid approved NPC-interaction JSON")
|
||||||
|
}
|
||||||
|
content, err := json.Marshal(struct {
|
||||||
|
Interactions []npcInteractionProjection `json:"npc_interactions"`
|
||||||
|
}{Interactions: projectNPCInteractions(value.Interactions)})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("encode NPC-interaction grounding: %w", err)
|
||||||
|
}
|
||||||
|
return newPromptInput(NPCInteractionReferenceSlot, content), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func referenceItem(references contracts.ReferenceSet, slotName, expectedMediaType string) (contracts.ReferenceItem, bool, error) {
|
||||||
|
slot, ok := references.Slots[slotName]
|
||||||
|
if !ok {
|
||||||
|
return contracts.ReferenceItem{}, false, nil
|
||||||
|
}
|
||||||
|
if len(slot.Items) == 0 {
|
||||||
|
return contracts.ReferenceItem{}, false, nil
|
||||||
|
}
|
||||||
|
if len(slot.Items) != 1 {
|
||||||
|
return contracts.ReferenceItem{}, false, fmt.Errorf("reference slot %q must contain exactly one item", slotName)
|
||||||
|
}
|
||||||
|
item := slot.Items[0]
|
||||||
|
mediaType, _, err := mime.ParseMediaType(item.MediaType)
|
||||||
|
if err != nil {
|
||||||
|
return contracts.ReferenceItem{}, false, fmt.Errorf("reference slot %q item media type is invalid", slotName)
|
||||||
|
}
|
||||||
|
if !strings.EqualFold(mediaType, expectedMediaType) {
|
||||||
|
return contracts.ReferenceItem{}, false, fmt.Errorf("reference slot %q item media type must be %s", slotName, expectedMediaType)
|
||||||
|
}
|
||||||
|
if len(item.Content) > ReferenceMaxBytes {
|
||||||
|
return contracts.ReferenceItem{}, false, fmt.Errorf("reference slot %q item is %d bytes, limit %d", slotName, len(item.Content), ReferenceMaxBytes)
|
||||||
|
}
|
||||||
|
return item, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type combatTurnProjection struct {
|
||||||
|
Actor string `json:"actor"`
|
||||||
|
TurnKind dnd.CombatTurnKind `json:"turn_kind"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func projectCombatTurns(turns []dnd.CombatTurn) []combatTurnProjection {
|
||||||
|
if turns == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
projection := make([]combatTurnProjection, len(turns))
|
||||||
|
for index, turn := range turns {
|
||||||
|
projection[index] = combatTurnProjection{Actor: turn.Actor, TurnKind: turn.TurnKind}
|
||||||
|
}
|
||||||
|
return projection
|
||||||
|
}
|
||||||
|
|
||||||
|
type npcInteractionProjection struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Kind dnd.NPCInteractionKind `json:"kind"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func projectNPCInteractions(interactions []dnd.NPCInteraction) []npcInteractionProjection {
|
||||||
|
projection := make([]npcInteractionProjection, 0, len(interactions))
|
||||||
|
for _, interaction := range interactions {
|
||||||
|
if interaction.Kind == dnd.NPCInteractionKindCombatOpponent {
|
||||||
|
projection = append(projection, npcInteractionProjection{Name: interaction.Name, Kind: interaction.Kind})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return projection
|
||||||
|
}
|
||||||
|
|
||||||
|
func newPromptInput(name string, content []byte) *contracts.LLMInputMaterial {
|
||||||
|
sum := sha256.Sum256(content)
|
||||||
|
material := contracts.NewLLMInputMaterial(name, promptProjectionMediaType, content, "sha256:"+hex.EncodeToString(sum[:]), "")
|
||||||
|
return &material
|
||||||
|
}
|
||||||
279
internal/modules/dnd/extract/enemyevents/grounding_test.go
Normal file
279
internal/modules/dnd/extract/enemyevents/grounding_test.go
Normal file
@@ -0,0 +1,279 @@
|
|||||||
|
package enemyevents
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
combatturncodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/combatturns"
|
||||||
|
interactioncodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcinteractions"
|
||||||
|
npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs"
|
||||||
|
scenecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity"
|
||||||
|
sceneregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/scenedescriptions/registry"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestReferenceSlotsDescribeRequiredTypedArtifacts(t *testing.T) {
|
||||||
|
slots := referenceSlots()
|
||||||
|
if len(slots) != 8 {
|
||||||
|
t.Fatalf("ReferenceSlots() count = %d, want 8", len(slots))
|
||||||
|
}
|
||||||
|
byName := make(map[string]contracts.ReferenceSlot, len(slots))
|
||||||
|
for index, slot := range slots {
|
||||||
|
if index > 0 && slots[index-1].Name > slot.Name {
|
||||||
|
t.Fatalf("ReferenceSlots() is not sorted: %#v", slots)
|
||||||
|
}
|
||||||
|
byName[slot.Name] = slot
|
||||||
|
}
|
||||||
|
for _, want := range []struct {
|
||||||
|
name string
|
||||||
|
kind contracts.ArtifactKind
|
||||||
|
}{
|
||||||
|
{NPCRegistryReferenceSlot, dnd.NPCListKind},
|
||||||
|
{SceneDescriptionReferenceSlot, dnd.SceneDescriptionListKind},
|
||||||
|
{CombatTurnReferenceSlot, dnd.CombatTurnListKind},
|
||||||
|
{NPCInteractionReferenceSlot, dnd.NPCInteractionListKind},
|
||||||
|
} {
|
||||||
|
slot, ok := byName[want.name]
|
||||||
|
if !ok || !slot.Required || slot.MaxBytes != ReferenceMaxBytes || len(slot.AcceptedMediaTypes) != 1 || slot.AcceptedMediaTypes[0] != "application/json" || len(slot.AcceptedArtifactKinds) != 1 || slot.AcceptedArtifactKinds[0] != want.kind {
|
||||||
|
t.Fatalf("slot %q = %#v", want.name, slot)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
slots[0].AcceptedMediaTypes[0] = "changed"
|
||||||
|
if referenceSlots()[0].AcceptedMediaTypes[0] == "changed" {
|
||||||
|
t.Fatal("ReferenceSlots() returned caller-owned storage")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGroundingProducesExactSourceFreePromptInputs(t *testing.T) {
|
||||||
|
resolver, err := newGroundingResolver(groundingReferences(t, "Ashfang", dnd.SceneKindCombat))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
resolved, err := resolver.Resolve(contracts.ReferenceSet{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
inputs := resolved.PromptInputs()
|
||||||
|
want := map[string]string{
|
||||||
|
NPCRegistryReferenceSlot: `{"npcs":[{"name":"Ashfang"}]}`,
|
||||||
|
CombatTurnReferenceSlot: `{"combat_turns":[{"actor":"Ashfang","turn_kind":"turn"},{"actor":"Aria","turn_kind":"reaction"}]}`,
|
||||||
|
NPCInteractionReferenceSlot: `{"npc_interactions":[{"name":"Ashfang","kind":"combat_opponent"}]}`,
|
||||||
|
}
|
||||||
|
if len(inputs) != len(want) {
|
||||||
|
t.Fatalf("PromptInputs() = %#v", inputs)
|
||||||
|
}
|
||||||
|
for name, content := range want {
|
||||||
|
input, ok := inputs[name]
|
||||||
|
if !ok || string(input.Content) != content || input.Digest != digest([]byte(content)) || input.OriginURI != "" || input.MediaType != "application/json" {
|
||||||
|
t.Fatalf("PromptInputs()[%q] = %#v, want %q", name, input, content)
|
||||||
|
}
|
||||||
|
for _, forbidden := range []string{"source_ref", "npc-ashfang", "origin", "summary", "combat-session"} {
|
||||||
|
if strings.Contains(string(input.Content), forbidden) {
|
||||||
|
t.Fatalf("PromptInputs()[%q] leaked %q: %s", name, forbidden, input.Content)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, ok := inputs[SceneDescriptionReferenceSlot]; ok {
|
||||||
|
t.Fatal("scene descriptions were rendered as prompt grounding")
|
||||||
|
}
|
||||||
|
|
||||||
|
inputs[CombatTurnReferenceSlot] = contracts.NewLLMInputMaterial("changed", "text/plain", []byte("changed"), "", "")
|
||||||
|
if next := resolved.PromptInputs()[CombatTurnReferenceSlot]; string(next.Content) != want[CombatTurnReferenceSlot] {
|
||||||
|
t.Fatal("PromptInputs() did not return a defensive copy")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGroundingResolvesGeneratedReferencesAndSceneEligibility(t *testing.T) {
|
||||||
|
prepared := groundingReferences(t, "Ashfang", dnd.SceneKindCombat)
|
||||||
|
resolver, err := newGroundingResolver(prepared)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
generated := groundingReferences(t, "Grimjaw", dnd.SceneKindNarrative)
|
||||||
|
resolved, err := resolver.Resolve(generated)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got := string(resolved.PromptInputs()[NPCRegistryReferenceSlot].Content); got != `{"npcs":[{"name":"Grimjaw"}]}` {
|
||||||
|
t.Fatalf("generated NPC projection = %s", got)
|
||||||
|
}
|
||||||
|
match, err := resolver.SceneMatch(generated, combatChunk())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if match != (sceneregistry.ChunkMatch{State: sceneregistry.MatchExact, Kind: dnd.SceneKindNarrative}) {
|
||||||
|
t.Fatalf("generated scene match = %#v", match)
|
||||||
|
}
|
||||||
|
match, err = resolver.SceneMatch(generated, &source.Chunk{ID: "other", Ref: combatChunk().Ref})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if match.State != sceneregistry.MatchMissing {
|
||||||
|
t.Fatal("missing scene was not reported")
|
||||||
|
}
|
||||||
|
mismatched := combatChunk()
|
||||||
|
mismatched.Ref.EndUnitID++
|
||||||
|
match, err = resolver.SceneMatch(generated, mismatched)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if match.State != sceneregistry.MatchMismatched {
|
||||||
|
t.Fatal("mismatched scene was not reported")
|
||||||
|
}
|
||||||
|
|
||||||
|
static, err := resolver.Resolve(contracts.ReferenceSet{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got := string(static.PromptInputs()[NPCRegistryReferenceSlot].Content); got != `{"npcs":[{"name":"Ashfang"}]}` {
|
||||||
|
t.Fatalf("static NPC projection changed after generated resolution: %s", got)
|
||||||
|
}
|
||||||
|
match, err = resolver.SceneMatch(contracts.ReferenceSet{}, combatChunk())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if match != (sceneregistry.ChunkMatch{State: sceneregistry.MatchExact, Kind: dnd.SceneKindCombat}) {
|
||||||
|
t.Fatalf("static scene match = %#v", match)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGroundingRejectsMissingAndInvalidReferences(t *testing.T) {
|
||||||
|
valid := groundingReferences(t, "Ashfang", dnd.SceneKindCombat)
|
||||||
|
for _, test := range []struct {
|
||||||
|
slot string
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{NPCRegistryReferenceSlot, "NPC registry"},
|
||||||
|
{SceneDescriptionReferenceSlot, "scene descriptions"},
|
||||||
|
{CombatTurnReferenceSlot, CombatTurnReferenceSlot},
|
||||||
|
{NPCInteractionReferenceSlot, NPCInteractionReferenceSlot},
|
||||||
|
} {
|
||||||
|
t.Run("missing "+test.slot, func(t *testing.T) {
|
||||||
|
resolver, err := newGroundingResolver(withoutSlot(valid, test.slot))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if test.slot == SceneDescriptionReferenceSlot {
|
||||||
|
_, err = resolver.SceneMatch(contracts.ReferenceSet{}, combatChunk())
|
||||||
|
} else {
|
||||||
|
_, err = resolver.Resolve(contracts.ReferenceSet{})
|
||||||
|
}
|
||||||
|
if err == nil || !strings.Contains(err.Error(), test.want) {
|
||||||
|
t.Fatalf("grounding error = %v, want missing %q reference", err, test.slot)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
set contracts.ReferenceSet
|
||||||
|
}{
|
||||||
|
{"multiple items", replaceSlot(valid, CombatTurnReferenceSlot, contracts.ResolvedReferenceSlot{Items: []contracts.ReferenceItem{{MediaType: "application/json"}, {MediaType: "application/json"}}})},
|
||||||
|
{"malformed durable content", replaceItem(valid, NPCInteractionReferenceSlot, contracts.ReferenceItem{MediaType: "application/json", Content: []byte(`{}`)})},
|
||||||
|
{"wrong media type", replaceItem(valid, CombatTurnReferenceSlot, contracts.ReferenceItem{MediaType: "text/plain", Content: valid.Slots[CombatTurnReferenceSlot].Items[0].Content})},
|
||||||
|
{"oversize", replaceItem(valid, CombatTurnReferenceSlot, contracts.ReferenceItem{MediaType: "application/json", Content: make([]byte, ReferenceMaxBytes+1)})},
|
||||||
|
}
|
||||||
|
resolver, err := newGroundingResolver(valid)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, test := range tests {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
if _, err := resolver.Resolve(test.set); err == nil {
|
||||||
|
t.Fatal("Resolve() error = nil")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func groundingReferences(t *testing.T, enemy string, sceneKind dnd.SceneKind) contracts.ReferenceSet {
|
||||||
|
t.Helper()
|
||||||
|
npcContent, err := npccodec.New().Encode(dnd.NPCList{NPCs: []dnd.NPC{{
|
||||||
|
ID: identity.DeriveID(enemy),
|
||||||
|
Name: enemy,
|
||||||
|
SourceRefs: []source.SourceRef{{SourceID: "combat-session", StartUnitID: 1, EndUnitID: 1}},
|
||||||
|
}}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
sceneContent, err := scenecodec.New().Encode(dnd.SceneDescriptionList{Scenes: []dnd.SceneDescription{{
|
||||||
|
ID: "combat-scene",
|
||||||
|
SourceRef: source.SourceRef{SourceID: "combat-session", StartUnitID: 1, EndUnitID: 4},
|
||||||
|
Kind: sceneKind,
|
||||||
|
Title: "Combat session",
|
||||||
|
Summary: "A detailed scene summary that must not reach the prompt.",
|
||||||
|
}}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
turnContent, err := combatturncodec.New().Encode(dnd.CombatTurnList{CombatTurns: []dnd.CombatTurn{
|
||||||
|
{Actor: enemy, TurnKind: dnd.CombatTurnKindTurn, SourceRefs: []source.SourceRef{{SourceID: "combat-session", StartUnitID: 1, EndUnitID: 1}}},
|
||||||
|
{Actor: "Aria", TurnKind: dnd.CombatTurnKindReaction, SourceRefs: []source.SourceRef{{SourceID: "combat-session", StartUnitID: 2, EndUnitID: 2}}},
|
||||||
|
}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
interactionContent, err := interactioncodec.New().Encode(dnd.NPCInteractionList{Interactions: []dnd.NPCInteraction{
|
||||||
|
{Name: enemy, Kind: dnd.NPCInteractionKindCombatOpponent, SourceRefs: []source.SourceRef{{SourceID: "combat-session", StartUnitID: 1, EndUnitID: 1}}},
|
||||||
|
{Name: "Aria", Kind: dnd.NPCInteractionKindCombatAlly, SourceRefs: []source.SourceRef{{SourceID: "combat-session", StartUnitID: 2, EndUnitID: 2}}},
|
||||||
|
}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
|
||||||
|
NPCRegistryReferenceSlot: {Items: []contracts.ReferenceItem{newReferenceItem(NPCRegistryReferenceSlot, npcContent)}},
|
||||||
|
SceneDescriptionReferenceSlot: {Items: []contracts.ReferenceItem{newReferenceItem(SceneDescriptionReferenceSlot, sceneContent)}},
|
||||||
|
CombatTurnReferenceSlot: {Items: []contracts.ReferenceItem{newReferenceItem(CombatTurnReferenceSlot, turnContent)}},
|
||||||
|
NPCInteractionReferenceSlot: {Items: []contracts.ReferenceItem{newReferenceItem(NPCInteractionReferenceSlot, interactionContent)}},
|
||||||
|
}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func newReferenceItem(slot string, content []byte) contracts.ReferenceItem {
|
||||||
|
return contracts.ReferenceItem{
|
||||||
|
SlotName: slot,
|
||||||
|
MediaType: "application/json",
|
||||||
|
Content: append([]byte(nil), content...),
|
||||||
|
Digest: digest(content),
|
||||||
|
Origin: contracts.ReferenceOrigin{Type: "file", URI: "file:///private/reference.json"},
|
||||||
|
Producer: contracts.ReferenceProducer{PipelineID: "prior", StepID: "extract", LaneID: "lane", ModuleKey: "dnd/example"},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func withoutSlot(set contracts.ReferenceSet, name string) contracts.ReferenceSet {
|
||||||
|
cloned := contracts.ReferenceSet{Slots: make(map[string]contracts.ResolvedReferenceSlot, len(set.Slots)-1)}
|
||||||
|
for slotName, slot := range set.Slots {
|
||||||
|
if slotName != name {
|
||||||
|
cloned.Slots[slotName] = slot
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return cloned
|
||||||
|
}
|
||||||
|
|
||||||
|
func replaceSlot(set contracts.ReferenceSet, name string, value contracts.ResolvedReferenceSlot) contracts.ReferenceSet {
|
||||||
|
cloned := contracts.ReferenceSet{Slots: make(map[string]contracts.ResolvedReferenceSlot, len(set.Slots))}
|
||||||
|
for slotName, slot := range set.Slots {
|
||||||
|
cloned.Slots[slotName] = slot
|
||||||
|
}
|
||||||
|
cloned.Slots[name] = value
|
||||||
|
return cloned
|
||||||
|
}
|
||||||
|
|
||||||
|
func replaceItem(set contracts.ReferenceSet, name string, item contracts.ReferenceItem) contracts.ReferenceSet {
|
||||||
|
return replaceSlot(set, name, contracts.ResolvedReferenceSlot{Items: []contracts.ReferenceItem{item}})
|
||||||
|
}
|
||||||
|
|
||||||
|
func combatChunk() *source.Chunk {
|
||||||
|
return &source.Chunk{ID: "combat-scene", Ref: source.SourceRef{SourceID: "combat-session", StartUnitID: 1, EndUnitID: 4}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func digest(content []byte) string {
|
||||||
|
sum := sha256.Sum256(content)
|
||||||
|
return "sha256:" + hex.EncodeToString(sum[:])
|
||||||
|
}
|
||||||
60
internal/modules/dnd/extract/enemyevents/mapping.go
Normal file
60
internal/modules/dnd/extract/enemyevents/mapping.go
Normal file
@@ -0,0 +1,60 @@
|
|||||||
|
package enemyevents
|
||||||
|
|
||||||
|
import (
|
||||||
|
"sort"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||||
|
)
|
||||||
|
|
||||||
|
type orderedEnemyEvent struct {
|
||||||
|
value dnd.EnemyEvent
|
||||||
|
earliest int
|
||||||
|
hasEvidence bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func canonicalEnemyEventList(response extractionResponse, order shared.SourceRefOrder, sourceID string) dnd.EnemyEventList {
|
||||||
|
if response.Events == nil {
|
||||||
|
return dnd.EnemyEventList{}
|
||||||
|
}
|
||||||
|
ordered := make([]orderedEnemyEvent, len(response.Events))
|
||||||
|
for index, event := range response.Events {
|
||||||
|
refs := order.Canonicalize(sourceRefs(event.SourceRefs, sourceID))
|
||||||
|
earliest, hasEvidence := order.EarliestValid(refs)
|
||||||
|
ordered[index] = orderedEnemyEvent{
|
||||||
|
value: dnd.EnemyEvent{
|
||||||
|
Name: event.Name,
|
||||||
|
Kind: dnd.EnemyEventKind(event.Kind),
|
||||||
|
SourceRefs: refs,
|
||||||
|
},
|
||||||
|
earliest: earliest,
|
||||||
|
hasEvidence: hasEvidence,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.SliceStable(ordered, func(left, right int) bool {
|
||||||
|
if ordered[left].hasEvidence != ordered[right].hasEvidence {
|
||||||
|
return ordered[left].hasEvidence
|
||||||
|
}
|
||||||
|
if !ordered[left].hasEvidence {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return ordered[left].earliest < ordered[right].earliest
|
||||||
|
})
|
||||||
|
events := make([]dnd.EnemyEvent, len(ordered))
|
||||||
|
for index := range ordered {
|
||||||
|
events[index] = ordered[index].value
|
||||||
|
}
|
||||||
|
return dnd.EnemyEventList{Events: events}
|
||||||
|
}
|
||||||
|
|
||||||
|
func sourceRefs(refs []enemySourceRefResponse, sourceID string) []source.SourceRef {
|
||||||
|
if refs == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
values := make([]source.SourceRef, len(refs))
|
||||||
|
for index, ref := range refs {
|
||||||
|
values[index] = source.SourceRef{SourceID: sourceID, StartUnitID: ref.StartUnitID, EndUnitID: ref.EndUnitID}
|
||||||
|
}
|
||||||
|
return values
|
||||||
|
}
|
||||||
16
internal/modules/dnd/extract/enemyevents/model.go
Normal file
16
internal/modules/dnd/extract/enemyevents/model.go
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
package enemyevents
|
||||||
|
|
||||||
|
type extractionResponse struct {
|
||||||
|
Events []enemyEventResponse `json:"events"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type enemyEventResponse struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Kind string `json:"kind"`
|
||||||
|
SourceRefs []enemySourceRefResponse `json:"source_refs"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type enemySourceRefResponse struct {
|
||||||
|
StartUnitID int `json:"start_unit_id"`
|
||||||
|
EndUnitID int `json:"end_unit_id"`
|
||||||
|
}
|
||||||
54
internal/modules/dnd/extract/enemyevents/prompt_assets.go
Normal file
54
internal/modules/dnd/extract/enemyevents/prompt_assets.go
Normal file
@@ -0,0 +1,54 @@
|
|||||||
|
package enemyevents
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"sync"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/promptfs"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||||
|
)
|
||||||
|
|
||||||
|
const promptAssetRoot = "assets/prompts"
|
||||||
|
|
||||||
|
var promptAssetManifest = shared.PromptAssetManifest{
|
||||||
|
ModuleDir: "dnd.enemy_events",
|
||||||
|
ModuleFiles: []promptfs.ModulePromptFile{
|
||||||
|
{Name: "dnd.enemy_events.yaml", Path: "assets/prompts/dnd.enemy_events.yaml"},
|
||||||
|
{Name: "grounding.md", Path: "assets/prompts/grounding.md"},
|
||||||
|
{Name: "task.md", Path: "assets/prompts/task.md"},
|
||||||
|
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||||
|
},
|
||||||
|
SharedFiles: []string{
|
||||||
|
"common-dnd-system.md",
|
||||||
|
"common-dnd-extraction-evidence.md",
|
||||||
|
"common-dnd-identity.md",
|
||||||
|
"common-dnd-transcript.md",
|
||||||
|
"common-dnd-references.md",
|
||||||
|
"common-dnd-npcs.md",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
func RegisterPromptAssets(registry *llm.AssetRegistry) error {
|
||||||
|
promptFS, err := promptAssetManifest.PromptFS(embeddedAssets)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("prepare enemy-event prompt assets: %w", err)
|
||||||
|
}
|
||||||
|
if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
|
||||||
|
}
|
||||||
|
|
||||||
|
func promptAssetMetadata() (string, error) {
|
||||||
|
promptAssetHashOnce.Do(func() {
|
||||||
|
promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
|
||||||
|
})
|
||||||
|
return promptAssetHash, promptAssetHashErr
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
promptAssetHashOnce sync.Once
|
||||||
|
promptAssetHash string
|
||||||
|
promptAssetHashErr error
|
||||||
|
)
|
||||||
123
internal/modules/dnd/extract/enemyevents/prompt_assets_test.go
Normal file
123
internal/modules/dnd/extract/enemyevents/prompt_assets_test.go
Normal file
@@ -0,0 +1,123 @@
|
|||||||
|
package enemyevents
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"io/fs"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||||
|
"gitea.maximumdirect.net/eric/promptkit"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRegisterPromptAssetsAndPrepareEnemyEventPrompt(t *testing.T) {
|
||||||
|
registry := llm.NewAssetRegistry()
|
||||||
|
if err := RegisterPromptAssets(registry); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
schemaFS, err := registry.SchemaFS()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := fs.ReadFile(schemaFS, "dnd_enemy_events_llm.v1.json"); err != nil {
|
||||||
|
t.Fatalf("response schema asset: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
prepared := prepareEnemyEventPrompt(t)
|
||||||
|
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_enemy_events_llm.v1.json" || prepared.SelectedProfileID != "dnd-extraction" {
|
||||||
|
t.Fatalf("prepared prompt = %#v", prepared)
|
||||||
|
}
|
||||||
|
transcriptIndex := renderedMessageIndex(t, prepared.Messages, "enemy-transcript")
|
||||||
|
for _, sentinel := range []string{"enemy-npc", "enemy-turn", "enemy-opponent", "Extract Dungeons & Dragons enemy events"} {
|
||||||
|
if index := renderedMessageIndex(t, prepared.Messages, sentinel); index <= transcriptIndex {
|
||||||
|
t.Fatalf("message containing %q has index %d, want after transcript index %d", sentinel, index, transcriptIndex)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
instructionIndex := renderedMessageIndex(t, prepared.Messages, "Return the `events` array")
|
||||||
|
if instructionIndex <= transcriptIndex {
|
||||||
|
t.Fatalf("instruction index = %d, want after transcript index %d", instructionIndex, transcriptIndex)
|
||||||
|
}
|
||||||
|
if instructionIndex != len(prepared.Messages)-1 {
|
||||||
|
t.Fatalf("instruction message index = %d, want final message", instructionIndex)
|
||||||
|
}
|
||||||
|
if cache := prepared.Messages[instructionIndex].CacheControl; cache == nil || cache.Type != promptkit.CacheControlEphemeral {
|
||||||
|
t.Fatalf("final instruction cache control = %#v", cache)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnemyEventPromptRequiresGroundingInputs(t *testing.T) {
|
||||||
|
engine := newEnemyEventPromptEngine(t)
|
||||||
|
for _, inputName := range []string{"npcs", "combat_turns", "npc_interactions"} {
|
||||||
|
t.Run(inputName, func(t *testing.T) {
|
||||||
|
inputs := enemyEventPromptInputs()
|
||||||
|
delete(inputs, inputName)
|
||||||
|
_, err := engine.Prepare(context.Background(), promptkit.RunRequest{
|
||||||
|
PromptID: PromptID, PromptVersion: SchemaVersion, Inputs: inputs,
|
||||||
|
})
|
||||||
|
if err == nil || !strings.Contains(err.Error(), inputName) {
|
||||||
|
t.Fatalf("Prepare() error = %v, want required %q input", err, inputName)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func prepareEnemyEventPrompt(t *testing.T) *promptkit.PreparedRun {
|
||||||
|
t.Helper()
|
||||||
|
prepared, err := newEnemyEventPromptEngine(t).Prepare(context.Background(), promptkit.RunRequest{
|
||||||
|
PromptID: PromptID, PromptVersion: SchemaVersion, Inputs: enemyEventPromptInputs(),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return prepared
|
||||||
|
}
|
||||||
|
|
||||||
|
func newEnemyEventPromptEngine(t *testing.T) *promptkit.Engine {
|
||||||
|
t.Helper()
|
||||||
|
registry := llm.NewAssetRegistry()
|
||||||
|
if err := RegisterPromptAssets(registry); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
options, err := registry.PromptKitOptions()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
|
||||||
|
ID: "dnd-extraction", Endpoint: "http://127.0.0.1:1/v1", Model: "enemy-test-model",
|
||||||
|
})))
|
||||||
|
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return engine
|
||||||
|
}
|
||||||
|
|
||||||
|
func enemyEventPromptInputs() map[string]promptkit.ArtifactRef {
|
||||||
|
return map[string]promptkit.ArtifactRef{
|
||||||
|
"transcript": promptkit.Inline(`{"value":"enemy-transcript"}`),
|
||||||
|
"players": promptkit.Inline("enemy-player"),
|
||||||
|
"party": promptkit.Inline("enemy-party"),
|
||||||
|
"glossary": promptkit.Inline("enemy-glossary"),
|
||||||
|
"npcs": promptkit.Inline(`{"npcs":[{"name":"enemy-npc"}]}`),
|
||||||
|
"combat_turns": promptkit.Inline(`{"combat_turns":[{"actor":"enemy-turn","turn_kind":"turn"}]}`),
|
||||||
|
"npc_interactions": promptkit.Inline(`{"npc_interactions":[{"name":"enemy-opponent","kind":"combat_opponent"}]}`),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func renderedMessageIndex(t *testing.T, messages []promptkit.RenderedMessage, sentinel string) int {
|
||||||
|
t.Helper()
|
||||||
|
index := -1
|
||||||
|
occurrences := 0
|
||||||
|
for messageIndex, message := range messages {
|
||||||
|
count := strings.Count(message.Content, sentinel)
|
||||||
|
if count > 0 {
|
||||||
|
index = messageIndex
|
||||||
|
occurrences += count
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if occurrences != 1 {
|
||||||
|
t.Fatalf("message sentinel %q rendered %d times, want exactly once", sentinel, occurrences)
|
||||||
|
}
|
||||||
|
return index
|
||||||
|
}
|
||||||
@@ -1,12 +1,12 @@
|
|||||||
package npcs
|
package enemyevents
|
||||||
|
|
||||||
import "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
import "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||||
|
|
||||||
const (
|
const (
|
||||||
PromptID = "dnd.npcs.normalize"
|
PromptID = "dnd.enemy_events"
|
||||||
ResponseSchemaKey = llm.ResponseSchemaKey("dnd_npcs_normalize_llm")
|
ResponseSchemaKey = llm.ResponseSchemaKey("dnd_enemy_events_llm")
|
||||||
ResponseSchemaID = "notarius.dnd.npcs.normalize.llm"
|
ResponseSchemaID = "notarius.dnd.enemy_events.llm"
|
||||||
ResponseSchemaName = "notarius_dnd_npcs_normalize_llm_v1"
|
ResponseSchemaName = "notarius_dnd_enemy_events_llm_v1"
|
||||||
SchemaVersion = "v1"
|
SchemaVersion = "v1"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -16,6 +16,6 @@ func loadResponseSchema() (llm.ResponseSchema, error) {
|
|||||||
ID: ResponseSchemaID,
|
ID: ResponseSchemaID,
|
||||||
Version: SchemaVersion,
|
Version: SchemaVersion,
|
||||||
Name: ResponseSchemaName,
|
Name: ResponseSchemaName,
|
||||||
AssetPath: "assets/schemas/dnd_npcs_normalize_llm.v1.json",
|
AssetPath: "assets/schemas/dnd_enemy_events_llm.v1.json",
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
97
internal/modules/dnd/extract/enemyevents/schema_test.go
Normal file
97
internal/modules/dnd/extract/enemyevents/schema_test.go
Normal file
@@ -0,0 +1,97 @@
|
|||||||
|
package enemyevents
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/santhosh-tekuri/jsonschema/v6"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestResponseSchemaDefinesPrivateStructuralBoundary(t *testing.T) {
|
||||||
|
schema, err := loadResponseSchema()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if schema.Key != ResponseSchemaKey || schema.ID != ResponseSchemaID || schema.Name != ResponseSchemaName || schema.Version != SchemaVersion || !strings.HasPrefix(schema.SHA256, "sha256:") || !json.Valid(schema.JSONSchema) {
|
||||||
|
t.Fatalf("schema = %#v", schema)
|
||||||
|
}
|
||||||
|
valid := validEnemyResponse()
|
||||||
|
content, err := json.Marshal(valid)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := validateEnemySchema(content, schema.JSONSchema); err != nil {
|
||||||
|
t.Fatalf("valid response rejected: %v", err)
|
||||||
|
}
|
||||||
|
semantic := validEnemyResponse()
|
||||||
|
event := semantic["events"].([]any)[0].(map[string]any)
|
||||||
|
event["name"] = ""
|
||||||
|
event["kind"] = "unsupported"
|
||||||
|
ref := event["source_refs"].([]any)[0].(map[string]any)
|
||||||
|
ref["start_unit_id"] = 0
|
||||||
|
ref["end_unit_id"] = -1
|
||||||
|
content, err = json.Marshal(semantic)
|
||||||
|
if err != nil || validateEnemySchema(content, schema.JSONSchema) != nil {
|
||||||
|
t.Fatalf("validator-owned semantics were rejected: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResponseSchemaRejectsInvalidStructure(t *testing.T) {
|
||||||
|
schema, err := loadResponseSchema()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, mutate := range []func(map[string]any){
|
||||||
|
func(event map[string]any) { delete(event, "name") },
|
||||||
|
func(event map[string]any) { event["kind"] = 1 },
|
||||||
|
func(event map[string]any) { event["unexpected"] = true },
|
||||||
|
func(event map[string]any) { event["source_refs"].([]any)[0].(map[string]any)["source_id"] = "session" },
|
||||||
|
} {
|
||||||
|
candidate := validEnemyResponse()
|
||||||
|
mutate(candidate["events"].([]any)[0].(map[string]any))
|
||||||
|
content, err := json.Marshal(candidate)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := validateEnemySchema(content, schema.JSONSchema); err == nil {
|
||||||
|
t.Fatal("private schema accepted structurally invalid response")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
first, err := loadResponseSchema()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
first.JSONSchema[0] = '['
|
||||||
|
second, err := loadResponseSchema()
|
||||||
|
if err != nil || !json.Valid(second.JSONSchema) || bytes.Equal(first.JSONSchema, second.JSONSchema) {
|
||||||
|
t.Fatalf("schema defensive copy = %s, %v", second.JSONSchema, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func validEnemyResponse() map[string]any {
|
||||||
|
return map[string]any{"events": []any{map[string]any{
|
||||||
|
"name": "Ashfang", "kind": "engaged", "source_refs": []any{map[string]any{"start_unit_id": 1, "end_unit_id": 1}},
|
||||||
|
}}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateEnemySchema(instanceContent, schemaContent []byte) error {
|
||||||
|
instance, err := jsonschema.UnmarshalJSON(bytes.NewReader(instanceContent))
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
document, err := jsonschema.UnmarshalJSON(bytes.NewReader(schemaContent))
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
compiler := jsonschema.NewCompiler()
|
||||||
|
if err := compiler.AddResource("schema.json", document); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
schema, err := compiler.Compile("schema.json")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return schema.Validate(instance)
|
||||||
|
}
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
Return one event only when the transcript establishes a meaningful item or
|
Return one event only when the transcript establishes a meaningful item or
|
||||||
currency occurrence. Use a concise observed item name and preserve the stated
|
currency occurrence. Use a concise observed item name and preserve the stated
|
||||||
currency denomination; include quantity only when the transcript explicitly
|
currency denomination. Set `quantity` to the explicitly stated integer, or to
|
||||||
states it.
|
`null` when the transcript does not state one.
|
||||||
|
|
||||||
Use `discovered` when the party learns of or encounters an item without
|
Use `discovered` when the party learns of or encounters an item without
|
||||||
establishing possession. Use `acquired` when the party or a party member gains
|
establishing possession. Use `acquired` when the party or a party member gains
|
||||||
@@ -13,11 +13,12 @@ when the transcript explicitly describes it being physically destroyed or
|
|||||||
expended as a non-payment component. Use `transferred` only when possession
|
expended as a non-payment component. Use `transferred` only when possession
|
||||||
moves between two distinct named party members.
|
moves between two distinct named party members.
|
||||||
|
|
||||||
For `discovered`, omit both holders. For `acquired`, provide only `to`; for
|
Return both `from` and `to` for every event, using `null` when a holder does not
|
||||||
`lost` and `consumed`, provide only `from`; and for `transferred`, provide both
|
apply. For `discovered`, set both holders to `null`. For `acquired`, set `from`
|
||||||
`from` and `to`. Use `party` only for collective or unresolved party possession,
|
to `null` and provide `to`; for `lost` and `consumed`, provide `from` and set
|
||||||
never for either side of a transfer. Do not emit a transfer for a gift, sale, or
|
`to` to `null`; and for `transferred`, provide both holders. Use `party` only
|
||||||
payment outside the party.
|
for collective or unresolved party possession, never for either side of a
|
||||||
|
transfer. Do not emit a transfer for a gift, sale, or payment outside the party.
|
||||||
|
|
||||||
Ordinary non-depleting use is not an event. Do not infer acquisition from a
|
Ordinary non-depleting use is not an event. Do not infer acquisition from a
|
||||||
discovery, or discovery from an acquisition: emit both only when each is
|
discovery, or discovery from an acquisition: emit both only when each is
|
||||||
|
|||||||
@@ -10,13 +10,13 @@
|
|||||||
"items": {
|
"items": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"additionalProperties": false,
|
"additionalProperties": false,
|
||||||
"required": ["name", "kind", "source_refs"],
|
"required": ["name", "kind", "quantity", "from", "to", "source_refs"],
|
||||||
"properties": {
|
"properties": {
|
||||||
"name": {"type": "string"},
|
"name": {"type": "string"},
|
||||||
"kind": {"type": "string"},
|
"kind": {"type": "string"},
|
||||||
"quantity": {"type": "integer"},
|
"quantity": {"type": ["integer", "null"]},
|
||||||
"from": {"type": "string"},
|
"from": {"type": ["string", "null"]},
|
||||||
"to": {"type": "string"},
|
"to": {"type": ["string", "null"]},
|
||||||
"source_refs": {
|
"source_refs": {
|
||||||
"type": "array",
|
"type": "array",
|
||||||
"items": {
|
"items": {
|
||||||
|
|||||||
@@ -81,6 +81,21 @@ func TestExtractPreservesInvalidCandidatesAndEmptyResults(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestExtractMapsNullableResponseFieldsToAbsentArtifactFields(t *testing.T) {
|
||||||
|
client := &fakeItemEventsLLMClient{content: []byte(`{"events":[{"name":"Hidden Cache","kind":"discovered","quantity":null,"from":null,"to":null,"source_refs":[{"start_segment":1,"end_segment":1}]}]}`)}
|
||||||
|
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(result.Value.Events) != 1 {
|
||||||
|
t.Fatalf("events = %#v, want one", result.Value.Events)
|
||||||
|
}
|
||||||
|
event := result.Value.Events[0]
|
||||||
|
if event.Quantity != nil || event.From != "" || event.To != "" {
|
||||||
|
t.Fatalf("nullable response fields mapped to artifact values: %#v", event)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestExtractUsesSourceDocumentOrderForCandidates(t *testing.T) {
|
func TestExtractUsesSourceDocumentOrderForCandidates(t *testing.T) {
|
||||||
client := &fakeItemEventsLLMClient{response: extractionResponse{Events: []itemEventResponse{
|
client := &fakeItemEventsLLMClient{response: extractionResponse{Events: []itemEventResponse{
|
||||||
{Name: "Later", Kind: "discovered", SourceRefs: responseRefs(10, 10)},
|
{Name: "Later", Kind: "discovered", SourceRefs: responseRefs(10, 10)},
|
||||||
|
|||||||
@@ -17,10 +17,16 @@ func TestResponseSchemaIsStrictlyStructuralAndPrivate(t *testing.T) {
|
|||||||
if schema.Key != ResponseSchemaKey || schema.ID != ResponseSchemaID || schema.Name != ResponseSchemaName || schema.Version != SchemaVersion || !strings.HasPrefix(schema.SHA256, "sha256:") || !json.Valid(schema.JSONSchema) {
|
if schema.Key != ResponseSchemaKey || schema.ID != ResponseSchemaID || schema.Name != ResponseSchemaName || schema.Version != SchemaVersion || !strings.HasPrefix(schema.SHA256, "sha256:") || !json.Valid(schema.JSONSchema) {
|
||||||
t.Fatalf("schema = %#v", schema)
|
t.Fatalf("schema = %#v", schema)
|
||||||
}
|
}
|
||||||
valid := map[string]any{"events": []any{map[string]any{
|
valid := map[string]any{"events": []any{
|
||||||
|
map[string]any{
|
||||||
"name": "", "kind": "unsupported", "quantity": 0, "from": "party", "to": "Party",
|
"name": "", "kind": "unsupported", "quantity": 0, "from": "party", "to": "Party",
|
||||||
"source_refs": []any{map[string]any{"start_segment": 0, "end_segment": -1}},
|
"source_refs": []any{map[string]any{"start_segment": 0, "end_segment": -1}},
|
||||||
}}}
|
},
|
||||||
|
map[string]any{
|
||||||
|
"name": "Hidden Cache", "kind": "discovered", "quantity": nil, "from": nil, "to": nil,
|
||||||
|
"source_refs": []any{map[string]any{"start_segment": 1, "end_segment": 1}},
|
||||||
|
},
|
||||||
|
}}
|
||||||
content, err := json.Marshal(valid)
|
content, err := json.Marshal(valid)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
@@ -33,10 +39,11 @@ func TestResponseSchemaIsStrictlyStructuralAndPrivate(t *testing.T) {
|
|||||||
value map[string]any
|
value map[string]any
|
||||||
}{
|
}{
|
||||||
{"missing events", map[string]any{}},
|
{"missing events", map[string]any{}},
|
||||||
{"missing event name", map[string]any{"events": []any{map[string]any{"kind": "acquired", "source_refs": []any{}}}}},
|
{"missing event name", map[string]any{"events": []any{withoutField(responseEvent(), "name")}}},
|
||||||
{"unknown event field", map[string]any{"events": []any{map[string]any{"name": "Ring", "kind": "acquired", "source_refs": []any{}, "extra": true}}}},
|
{"missing nullable field", map[string]any{"events": []any{withoutField(responseEvent(), "quantity")}}},
|
||||||
{"unknown reference field", map[string]any{"events": []any{map[string]any{"name": "Ring", "kind": "acquired", "source_refs": []any{map[string]any{"start_segment": 1, "end_segment": 1, "extra": true}}}}}},
|
{"unknown event field", map[string]any{"events": []any{withField(responseEvent(), "extra", true)}}},
|
||||||
{"noninteger range", map[string]any{"events": []any{map[string]any{"name": "Ring", "kind": "acquired", "source_refs": []any{map[string]any{"start_segment": 1.5, "end_segment": 1}}}}}},
|
{"unknown reference field", map[string]any{"events": []any{withField(responseEvent(), "source_refs", []any{map[string]any{"start_segment": 1, "end_segment": 1, "extra": true}})}}},
|
||||||
|
{"noninteger range", map[string]any{"events": []any{withField(responseEvent(), "source_refs", []any{map[string]any{"start_segment": 1.5, "end_segment": 1}})}}},
|
||||||
} {
|
} {
|
||||||
t.Run(test.name, func(t *testing.T) {
|
t.Run(test.name, func(t *testing.T) {
|
||||||
content, err := json.Marshal(test.value)
|
content, err := json.Marshal(test.value)
|
||||||
@@ -59,6 +66,22 @@ func TestResponseSchemaIsStrictlyStructuralAndPrivate(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func responseEvent() map[string]any {
|
||||||
|
return map[string]any{
|
||||||
|
"name": "Ring", "kind": "acquired", "quantity": nil, "from": nil, "to": "party", "source_refs": []any{},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func withoutField(value map[string]any, name string) map[string]any {
|
||||||
|
delete(value, name)
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
|
||||||
|
func withField(value map[string]any, name string, fieldValue any) map[string]any {
|
||||||
|
value[name] = fieldValue
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
|
||||||
func validateJSONSchema(instanceContent, schemaContent []byte) error {
|
func validateJSONSchema(instanceContent, schemaContent []byte) error {
|
||||||
instance, err := jsonschema.UnmarshalJSON(bytes.NewReader(instanceContent))
|
instance, err := jsonschema.UnmarshalJSON(bytes.NewReader(instanceContent))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -0,0 +1,6 @@
|
|||||||
|
package locationoccurrences
|
||||||
|
|
||||||
|
import "embed"
|
||||||
|
|
||||||
|
//go:embed assets/prompts/*.yaml assets/prompts/*.md assets/schemas/*.json
|
||||||
|
var embeddedAssets embed.FS
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
id: dnd.location_occurrences
|
||||||
|
version: "v1"
|
||||||
|
default_profile: dnd-extraction
|
||||||
|
inputs:
|
||||||
|
- name: transcript
|
||||||
|
required: true
|
||||||
|
content_type: application/json
|
||||||
|
- name: players
|
||||||
|
required: false
|
||||||
|
content_type: text/plain
|
||||||
|
- name: party
|
||||||
|
required: false
|
||||||
|
content_type: text/plain
|
||||||
|
- name: glossary
|
||||||
|
required: false
|
||||||
|
content_type: text/plain
|
||||||
|
- name: locations
|
||||||
|
required: true
|
||||||
|
content_type: application/json
|
||||||
|
messages:
|
||||||
|
- role: system
|
||||||
|
content_file: ./sharedassets/common-dnd-system.md
|
||||||
|
- role: user
|
||||||
|
content_file: ./sharedassets/common-dnd-identity.md
|
||||||
|
- role: user
|
||||||
|
content_file: ./sharedassets/common-dnd-references.md
|
||||||
|
cache_control:
|
||||||
|
type: ephemeral
|
||||||
|
- role: user
|
||||||
|
content_file: ./sharedassets/common-dnd-transcript.md
|
||||||
|
cache_control:
|
||||||
|
type: ephemeral
|
||||||
|
- role: user
|
||||||
|
content_file: ./sharedassets/common-dnd-extraction-evidence.md
|
||||||
|
- role: user
|
||||||
|
content_file: ./locations.md
|
||||||
|
- role: user
|
||||||
|
content_file: ./task.md
|
||||||
|
- role: user
|
||||||
|
content_file: ./instructions.md
|
||||||
|
cache_control:
|
||||||
|
type: ephemeral
|
||||||
|
output:
|
||||||
|
format: json
|
||||||
|
validation_mode: json_schema
|
||||||
|
schema_path: dnd_location_occurrences_llm.v1.json
|
||||||
|
repair_attempts: 0
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
Return the occurrences array even when no occurrence is established. Every
|
||||||
|
record must contain location_id, name, kind, and source_refs. Copy location_id
|
||||||
|
and name from one supplied registry record, and cite only narrow transcript
|
||||||
|
ranges that support both that location and its classified occurrence.
|
||||||
|
|
||||||
|
Do not summarize location descriptions, infer a missing registry record, or
|
||||||
|
use registry context as evidence. Omit source_id; Notarius assigns the current
|
||||||
|
transcript source identity.
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
A normalized location registry is provided below for identity grounding. It may
|
||||||
|
be empty. Each record contains the exact location ID and canonical display name
|
||||||
|
to copy when the transcript establishes an occurrence of that place.
|
||||||
|
|
||||||
|
Registry content is context, not occurrence evidence. Do not derive an
|
||||||
|
occurrence or a source range from the registry, and do not infer a location
|
||||||
|
that is absent from it.
|
||||||
|
|
||||||
|
{{ input "locations" }}
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
Extract Dungeons & Dragons location occurrences from the supplied transcript.
|
||||||
|
|
||||||
|
Include an occurrence only when the transcript establishes one supplied
|
||||||
|
location, one occurrence kind, and a coherent passage supporting both. Use
|
||||||
|
only the exact ID and name pair from the supplied location registry. Return an
|
||||||
|
empty occurrences array when no supplied location has an evidenced occurrence
|
||||||
|
in this transcript passage.
|
||||||
|
|
||||||
|
Use exactly one kind per occurrence:
|
||||||
|
|
||||||
|
- visited: party members are physically present, arrive, remain, or depart;
|
||||||
|
- planned: the party explicitly proposes, intends, or agrees to future travel;
|
||||||
|
- recalled: the transcript explicitly recounts an earlier party visit; or
|
||||||
|
- mentioned: the location is explicitly referenced without stronger support,
|
||||||
|
including non-actionable speculation or a mere hypothetical reference.
|
||||||
|
|
||||||
|
A mere hypothetical or speculative reference is not planned unless the
|
||||||
|
transcript also establishes an actual proposal, intention, or agreement to
|
||||||
|
travel. When the hypothetical itself explicitly names a supplied registry
|
||||||
|
location, it may be mentioned using the narrow passage that supports that
|
||||||
|
reference.
|
||||||
|
|
||||||
|
For overlapping support, visited outranks planned, recalled, and mentioned;
|
||||||
|
planned outranks recalled and mentioned; recalled outranks mentioned. A passage
|
||||||
|
may produce multiple records when it independently establishes separate facts,
|
||||||
|
such as recalling an earlier visit while planning a return. Omit inferred,
|
||||||
|
unstated, uncertain, or unsupported places and occurrences. Do not infer a
|
||||||
|
location or occurrence from surrounding events when the transcript does not
|
||||||
|
state it.
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"$id": "notarius.dnd.location_occurrences.llm",
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["occurrences"],
|
||||||
|
"properties": {
|
||||||
|
"occurrences": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["location_id", "name", "kind", "source_refs"],
|
||||||
|
"properties": {
|
||||||
|
"location_id": {"type": "string"},
|
||||||
|
"name": {"type": "string"},
|
||||||
|
"kind": {"enum": ["visited", "planned", "recalled", "mentioned"]},
|
||||||
|
"source_refs": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["start_unit_id", "end_unit_id"],
|
||||||
|
"properties": {
|
||||||
|
"start_unit_id": {"type": "integer"},
|
||||||
|
"end_unit_id": {"type": "integer"}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
106
internal/modules/dnd/extract/locationoccurrences/canonicalize.go
Normal file
106
internal/modules/dnd/extract/locationoccurrences/canonicalize.go
Normal file
@@ -0,0 +1,106 @@
|
|||||||
|
package locationoccurrences
|
||||||
|
|
||||||
|
import (
|
||||||
|
"reflect"
|
||||||
|
"sort"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||||
|
)
|
||||||
|
|
||||||
|
type orderedOccurrence struct {
|
||||||
|
value dnd.LocationOccurrence
|
||||||
|
earliest int
|
||||||
|
hasEvidence bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func canonicalOccurrenceList(response extractionResponse, order shared.SourceRefOrder, sourceID string) dnd.LocationOccurrenceList {
|
||||||
|
if response.Occurrences == nil {
|
||||||
|
return dnd.LocationOccurrenceList{}
|
||||||
|
}
|
||||||
|
ordered := make([]orderedOccurrence, len(response.Occurrences))
|
||||||
|
for index, occurrence := range response.Occurrences {
|
||||||
|
refs := order.Canonicalize(canonicalSourceRefs(occurrence.SourceRefs, sourceID))
|
||||||
|
earliest, hasEvidence := order.EarliestValid(refs)
|
||||||
|
ordered[index] = orderedOccurrence{value: dnd.LocationOccurrence{
|
||||||
|
LocationID: occurrence.LocationID,
|
||||||
|
Name: occurrence.Name,
|
||||||
|
Kind: dnd.LocationOccurrenceKind(occurrence.Kind),
|
||||||
|
SourceRefs: refs,
|
||||||
|
}, earliest: earliest, hasEvidence: hasEvidence}
|
||||||
|
}
|
||||||
|
sort.SliceStable(ordered, func(left, right int) bool {
|
||||||
|
return lessOccurrence(ordered[left], ordered[right], order)
|
||||||
|
})
|
||||||
|
occurrences := make([]dnd.LocationOccurrence, 0, len(ordered))
|
||||||
|
for _, occurrence := range ordered {
|
||||||
|
if len(occurrences) == 0 || !sameOccurrence(occurrences[len(occurrences)-1], occurrence.value) {
|
||||||
|
occurrences = append(occurrences, occurrence.value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return dnd.LocationOccurrenceList{Occurrences: occurrences}
|
||||||
|
}
|
||||||
|
|
||||||
|
func lessOccurrence(left, right orderedOccurrence, order shared.SourceRefOrder) bool {
|
||||||
|
if left.hasEvidence != right.hasEvidence {
|
||||||
|
return left.hasEvidence
|
||||||
|
}
|
||||||
|
if left.hasEvidence && left.earliest != right.earliest {
|
||||||
|
return left.earliest < right.earliest
|
||||||
|
}
|
||||||
|
if left.value.LocationID != right.value.LocationID {
|
||||||
|
return left.value.LocationID < right.value.LocationID
|
||||||
|
}
|
||||||
|
if left.value.Name != right.value.Name {
|
||||||
|
return left.value.Name < right.value.Name
|
||||||
|
}
|
||||||
|
if kindOrder(left.value.Kind) != kindOrder(right.value.Kind) {
|
||||||
|
return kindOrder(left.value.Kind) < kindOrder(right.value.Kind)
|
||||||
|
}
|
||||||
|
return lessReferences(left.value.SourceRefs, right.value.SourceRefs, order)
|
||||||
|
}
|
||||||
|
|
||||||
|
func kindOrder(kind dnd.LocationOccurrenceKind) int {
|
||||||
|
switch kind {
|
||||||
|
case dnd.LocationOccurrenceKindVisited:
|
||||||
|
return 0
|
||||||
|
case dnd.LocationOccurrenceKindPlanned:
|
||||||
|
return 1
|
||||||
|
case dnd.LocationOccurrenceKindRecalled:
|
||||||
|
return 2
|
||||||
|
case dnd.LocationOccurrenceKindMentioned:
|
||||||
|
return 3
|
||||||
|
default:
|
||||||
|
return 4
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func lessReferences(left, right []source.SourceRef, order shared.SourceRefOrder) bool {
|
||||||
|
limit := len(left)
|
||||||
|
if len(right) < limit {
|
||||||
|
limit = len(right)
|
||||||
|
}
|
||||||
|
for index := 0; index < limit; index++ {
|
||||||
|
if left[index] == right[index] {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
return order.Less(left[index], right[index])
|
||||||
|
}
|
||||||
|
return len(left) < len(right)
|
||||||
|
}
|
||||||
|
|
||||||
|
func sameOccurrence(left, right dnd.LocationOccurrence) bool {
|
||||||
|
return left.LocationID == right.LocationID && left.Name == right.Name && left.Kind == right.Kind && reflect.DeepEqual(left.SourceRefs, right.SourceRefs)
|
||||||
|
}
|
||||||
|
|
||||||
|
func canonicalSourceRefs(values []occurrenceSourceRefResponse, sourceID string) []source.SourceRef {
|
||||||
|
if values == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
refs := make([]source.SourceRef, len(values))
|
||||||
|
for index, value := range values {
|
||||||
|
refs[index] = source.SourceRef{SourceID: sourceID, StartUnitID: value.StartUnitID, EndUnitID: value.EndUnitID}
|
||||||
|
}
|
||||||
|
return refs
|
||||||
|
}
|
||||||
188
internal/modules/dnd/extract/locationoccurrences/extractor.go
Normal file
188
internal/modules/dnd/extract/locationoccurrences/extractor.go
Normal file
@@ -0,0 +1,188 @@
|
|||||||
|
// Package locationoccurrences extracts source-grounded D&D location occurrences.
|
||||||
|
package locationoccurrences
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
locationregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/registry"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
Key = "dnd/location-occurrences"
|
||||||
|
mappingPolicy = "dnd.location_occurrences.extract_mapping.v1"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
LocationRegistryReferenceSlot = locationregistry.ReferenceSlot
|
||||||
|
LocationRegistryMaxBytes = locationregistry.MaxBytes
|
||||||
|
)
|
||||||
|
|
||||||
|
var requiredCapabilities = []string{"chunks", "source.transcript"}
|
||||||
|
var providedCapabilities = []string{"dnd.location_occurrences"}
|
||||||
|
|
||||||
|
var referenceSlotDescriptions = shared.ReferenceSlotDescriptions{
|
||||||
|
Glossary: "Optional campaign glossary reference material used only for location-occurrence disambiguation.",
|
||||||
|
Party: "Optional party roster reference material used only for location-occurrence disambiguation.",
|
||||||
|
Players: "Optional player list reference material used only for location-occurrence disambiguation.",
|
||||||
|
Roster: "Deprecated alias for party roster reference material used only for location-occurrence disambiguation.",
|
||||||
|
}
|
||||||
|
|
||||||
|
func referenceSlots() []contracts.ReferenceSlot {
|
||||||
|
slots := shared.ReferenceSlots(referenceSlotDescriptions)
|
||||||
|
slots = append(slots, contracts.ReferenceSlot{
|
||||||
|
Name: LocationRegistryReferenceSlot,
|
||||||
|
Description: "Required normalized location registry used only for location identity grounding, never as occurrence evidence.",
|
||||||
|
Required: true,
|
||||||
|
AcceptedMediaTypes: []string{"application/json"},
|
||||||
|
AcceptedArtifactKinds: []contracts.ArtifactKind{dnd.LocationListKind},
|
||||||
|
MaxBytes: LocationRegistryMaxBytes,
|
||||||
|
})
|
||||||
|
sort.Slice(slots, func(left, right int) bool { return slots[left].Name < slots[right].Name })
|
||||||
|
return slots
|
||||||
|
}
|
||||||
|
|
||||||
|
var _ contracts.Extractor[dnd.LocationOccurrenceList] = (*Extractor)(nil)
|
||||||
|
var _ contracts.ManifestMetadataProvider = (*Extractor)(nil)
|
||||||
|
var _ pipeline.CheckpointFingerprintProvider = (*Extractor)(nil)
|
||||||
|
|
||||||
|
type Options struct{}
|
||||||
|
|
||||||
|
type Extractor struct {
|
||||||
|
llm contracts.StructuredLLMClient
|
||||||
|
locationResolver *locationregistry.Resolver
|
||||||
|
promptSHA string
|
||||||
|
responseSchemaSHA string
|
||||||
|
}
|
||||||
|
|
||||||
|
func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contracts.ReferenceSet) (*Extractor, error) {
|
||||||
|
if llmClient == nil {
|
||||||
|
return nil, extractorErrorf("LLM client must not be nil")
|
||||||
|
}
|
||||||
|
if len(references) > 1 {
|
||||||
|
return nil, extractorErrorf("at most one reference set may be supplied")
|
||||||
|
}
|
||||||
|
var referenceSet contracts.ReferenceSet
|
||||||
|
if len(references) == 1 {
|
||||||
|
referenceSet = references[0]
|
||||||
|
}
|
||||||
|
locationResolver, err := locationregistry.NewResolver(referenceSet)
|
||||||
|
if err != nil {
|
||||||
|
return nil, extractorErrorf("prepare location registry prompt input: %w", err)
|
||||||
|
}
|
||||||
|
promptSHA, err := promptAssetMetadata()
|
||||||
|
if err != nil {
|
||||||
|
return nil, extractorErrorf("load prompt metadata: %w", err)
|
||||||
|
}
|
||||||
|
responseSchema, err := loadResponseSchema()
|
||||||
|
if err != nil {
|
||||||
|
return nil, extractorErrorf("load response schema: %w", err)
|
||||||
|
}
|
||||||
|
return &Extractor{llm: llmClient, locationResolver: locationResolver, promptSHA: promptSHA, responseSchemaSHA: responseSchema.SHA256}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Extractor) Key() string { return Key }
|
||||||
|
|
||||||
|
func (e *Extractor) ReferenceSlots() []contracts.ReferenceSlot { return referenceSlots() }
|
||||||
|
|
||||||
|
func (e *Extractor) ManifestMetadata() map[string]any {
|
||||||
|
if e == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
metadata := map[string]any{
|
||||||
|
"prompt_id": PromptID,
|
||||||
|
"prompt_version": SchemaVersion,
|
||||||
|
"prompt_sha256": e.promptSHA,
|
||||||
|
"mapping_policy": mappingPolicy,
|
||||||
|
"response_schema_key": string(ResponseSchemaKey),
|
||||||
|
"response_schema_id": ResponseSchemaID,
|
||||||
|
"response_schema_name": ResponseSchemaName,
|
||||||
|
"response_schema_version": SchemaVersion,
|
||||||
|
"response_schema_sha256": e.responseSchemaSHA,
|
||||||
|
}
|
||||||
|
seeded := e.locationResolver.Seeded()
|
||||||
|
if seeded.Bound() {
|
||||||
|
metadata["location_registry_digest"] = seeded.Digest()
|
||||||
|
metadata["location_count"] = seeded.Count()
|
||||||
|
}
|
||||||
|
return metadata
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Extractor) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
|
||||||
|
if e == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return []pipeline.CheckpointFingerprint{
|
||||||
|
{Name: "prompt", Value: e.promptSHA},
|
||||||
|
{Name: "response_schema", Value: e.responseSchemaSHA},
|
||||||
|
{Name: "mapping_policy", Value: mappingPolicy},
|
||||||
|
{Name: "location_registry", Value: e.locationResolver.Seeded().ProjectionDigest()},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Extractor) Extract(ctx context.Context, req contracts.TypedExtractionRequest) (contracts.TypedExtractionResult[dnd.LocationOccurrenceList], error) {
|
||||||
|
if e == nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.LocationOccurrenceList]{}, extractorErrorf("extractor must not be nil")
|
||||||
|
}
|
||||||
|
if e.llm == nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.LocationOccurrenceList]{}, extractorErrorf("LLM client must not be nil")
|
||||||
|
}
|
||||||
|
sourceInput, err := shared.PrepareChunkExtraction(ctx, req)
|
||||||
|
if err != nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.LocationOccurrenceList]{}, extractorErrorf("%w", err)
|
||||||
|
}
|
||||||
|
registry, err := e.locationResolver.Resolve(req.References)
|
||||||
|
if err != nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.LocationOccurrenceList]{}, extractorErrorf("resolve location registry: %w", err)
|
||||||
|
}
|
||||||
|
if !registry.Bound() {
|
||||||
|
return contracts.TypedExtractionResult[dnd.LocationOccurrenceList]{}, extractorErrorf("location registry reference is required")
|
||||||
|
}
|
||||||
|
|
||||||
|
var response extractionResponse
|
||||||
|
inputs := shared.PromptInputs(sourceInput, req.References)
|
||||||
|
inputs[LocationRegistryReferenceSlot] = registry.PromptInput()
|
||||||
|
if _, err := e.llm.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
|
||||||
|
StageName: Key, PromptID: PromptID, PromptVersion: SchemaVersion,
|
||||||
|
ProfileID: req.LLMProfile, SessionID: req.SessionID, Inputs: inputs,
|
||||||
|
}, &response); err != nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.LocationOccurrenceList]{}, extractorErrorf("complete structured output: %w", err)
|
||||||
|
}
|
||||||
|
return contracts.TypedExtractionResult[dnd.LocationOccurrenceList]{Value: canonicalOccurrenceList(response, shared.NewSourceRefOrder(req.Source), req.Source.ID)}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func ModuleSpec() pipeline.ModuleSpec {
|
||||||
|
return pipeline.ModuleSpec{
|
||||||
|
Key: Key, Stage: pipeline.StageExtract, ExecutionClass: contracts.ExecutionClassLLMBacked,
|
||||||
|
Requires: append([]string(nil), requiredCapabilities...), Provides: append([]string(nil), providedCapabilities...),
|
||||||
|
ArtifactKind: dnd.LocationOccurrenceListKind, ReferenceSlots: referenceSlots(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func Register(registry *pipeline.ExtractorRegistry) error {
|
||||||
|
return pipeline.RegisterExtractorBuilder(registry, ModuleSpec(), validateOptions, func(request pipeline.BuildRequest) (contracts.Extractor[dnd.LocationOccurrenceList], error) {
|
||||||
|
options, err := DecodeOptions(request.Options)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return New(request.Dependencies.LLM, options, request.References)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }
|
||||||
|
|
||||||
|
func DecodeOptions(options map[string]any) (Options, error) {
|
||||||
|
if err := pipeline.RejectUnknownOptions(options); err != nil {
|
||||||
|
return Options{}, extractorErrorf("%w", err)
|
||||||
|
}
|
||||||
|
return Options{}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func extractorErrorf(format string, args ...any) error {
|
||||||
|
return fmt.Errorf("dnd location occurrences extractor: "+format, args...)
|
||||||
|
}
|
||||||
@@ -0,0 +1,243 @@
|
|||||||
|
package locationoccurrences
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
locationcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/locations"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestExtractMapsKindsOrdersOccurrencesAndPreservesIndependentFacts(t *testing.T) {
|
||||||
|
locations := locationRegistry(t, "The Tavern", "The Tavern")
|
||||||
|
first, second := locations.Locations[0], locations.Locations[1]
|
||||||
|
client := &fakeOccurrencesLLMClient{response: extractionResponse{Occurrences: []occurrenceResponse{
|
||||||
|
{LocationID: second.ID, Name: second.Name, Kind: "mentioned", SourceRefs: occurrenceRefs(30, 30)},
|
||||||
|
{LocationID: first.ID, Name: first.Name, Kind: "mentioned", SourceRefs: occurrenceRefs(10, 10)},
|
||||||
|
{LocationID: first.ID, Name: first.Name, Kind: "recalled", SourceRefs: occurrenceRefs(10, 10)},
|
||||||
|
{LocationID: first.ID, Name: first.Name, Kind: "planned", SourceRefs: occurrenceRefs(10, 10)},
|
||||||
|
{LocationID: first.ID, Name: first.Name, Kind: "visited", SourceRefs: append(occurrenceRefs(10, 10), occurrenceRefs(10, 10)...)},
|
||||||
|
{LocationID: first.ID, Name: first.Name, Kind: "visited", SourceRefs: occurrenceRefs(20, 20)},
|
||||||
|
{LocationID: first.ID, Name: first.Name, Kind: "visited", SourceRefs: occurrenceRefs(10, 10)},
|
||||||
|
}}}
|
||||||
|
references := registryReferences(t, locations)
|
||||||
|
req := extractionRequest()
|
||||||
|
req.References = references
|
||||||
|
result, err := newExtractor(t, client, references).Extract(context.Background(), req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(result.Value.Occurrences) != 6 {
|
||||||
|
t.Fatalf("occurrences = %#v, want exact duplicate removed", result.Value.Occurrences)
|
||||||
|
}
|
||||||
|
got := result.Value.Occurrences
|
||||||
|
if kinds := []dnd.LocationOccurrenceKind{got[0].Kind, got[1].Kind, got[2].Kind, got[3].Kind}; !reflect.DeepEqual(kinds, []dnd.LocationOccurrenceKind{dnd.LocationOccurrenceKindVisited, dnd.LocationOccurrenceKindPlanned, dnd.LocationOccurrenceKindRecalled, dnd.LocationOccurrenceKindMentioned}) {
|
||||||
|
t.Fatalf("same-evidence kind order = %#v", kinds)
|
||||||
|
}
|
||||||
|
if got[4].Kind != dnd.LocationOccurrenceKindVisited || got[4].SourceRefs[0].StartUnitID != 20 || got[5].LocationID != second.ID || got[5].SourceRefs[0].StartUnitID != 30 {
|
||||||
|
t.Fatalf("occurrence order = %#v", got)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(got[0].SourceRefs, []source.SourceRef{{SourceID: req.Source.ID, StartUnitID: 10, EndUnitID: 10}}) {
|
||||||
|
t.Fatalf("canonical evidence = %#v", got[0].SourceRefs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractUsesIDsNamesAndCurrentTranscriptEvidenceOnly(t *testing.T) {
|
||||||
|
locations := locationRegistry(t, "The Tavern", "The Tavern")
|
||||||
|
first, second := locations.Locations[0], locations.Locations[1]
|
||||||
|
client := &fakeOccurrencesLLMClient{response: extractionResponse{Occurrences: []occurrenceResponse{{
|
||||||
|
LocationID: second.ID, Name: second.Name, Kind: "visited", SourceRefs: occurrenceRefs(10, 10),
|
||||||
|
}}}}
|
||||||
|
references := registryReferences(t, locations)
|
||||||
|
req := extractionRequest()
|
||||||
|
req.References = references
|
||||||
|
result, err := newExtractor(t, client, references).Extract(context.Background(), req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if occurrence := result.Value.Occurrences[0]; occurrence.LocationID != second.ID || occurrence.Name != second.Name || occurrence.SourceRefs[0].SourceID != req.Source.ID {
|
||||||
|
t.Fatalf("occurrence = %#v", occurrence)
|
||||||
|
}
|
||||||
|
input := client.requests[0].Inputs[LocationRegistryReferenceSlot]
|
||||||
|
if input.Name != LocationRegistryReferenceSlot || !strings.Contains(string(input.Content), first.ID) || !strings.Contains(string(input.Content), second.ID) {
|
||||||
|
t.Fatalf("location prompt input = %#v", input)
|
||||||
|
}
|
||||||
|
for _, forbidden := range []string{"source_refs", "source_id", "other-session"} {
|
||||||
|
if strings.Contains(string(input.Content), forbidden) {
|
||||||
|
t.Fatalf("location prompt leaked %q: %s", forbidden, input.Content)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.Contains(string(client.requests[0].Inputs["transcript"].Content), "other-session") {
|
||||||
|
t.Fatal("transcript input contains registry evidence")
|
||||||
|
}
|
||||||
|
metadata, err := json.Marshal(newExtractor(t, &fakeOccurrencesLLMClient{}, references).ManifestMetadata())
|
||||||
|
if err != nil || strings.Contains(string(metadata), "other-session") || strings.Contains(string(metadata), first.ID) {
|
||||||
|
t.Fatalf("manifest metadata = %s, %v", metadata, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractPreservesUnknownOrMismatchedGroundingForValidators(t *testing.T) {
|
||||||
|
locations := locationRegistry(t, "The Mill")
|
||||||
|
known := locations.Locations[0]
|
||||||
|
client := &fakeOccurrencesLLMClient{response: extractionResponse{Occurrences: []occurrenceResponse{
|
||||||
|
{LocationID: "location:sha256:unknown", Name: "The Mill", Kind: "mentioned", SourceRefs: occurrenceRefs(10, 10)},
|
||||||
|
{LocationID: known.ID, Name: "A Different Mill", Kind: "mentioned", SourceRefs: occurrenceRefs(20, 20)},
|
||||||
|
}}}
|
||||||
|
references := registryReferences(t, locations)
|
||||||
|
req := extractionRequest()
|
||||||
|
req.References = references
|
||||||
|
result, err := newExtractor(t, client, references).Extract(context.Background(), req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if result.Value.Occurrences[0].LocationID != "location:sha256:unknown" || result.Value.Occurrences[1].Name != "A Different Mill" {
|
||||||
|
t.Fatalf("extractor repaired validator-owned grounding errors: %#v", result.Value.Occurrences)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractRequiresRegistryAndAcceptsEmptyRegistryWithNoOccurrences(t *testing.T) {
|
||||||
|
client := &fakeOccurrencesLLMClient{response: extractionResponse{Occurrences: []occurrenceResponse{}}}
|
||||||
|
if _, err := newExtractor(t, client).Extract(context.Background(), extractionRequest()); err == nil || !strings.Contains(err.Error(), "location registry reference is required") {
|
||||||
|
t.Fatalf("Extract() error = %v", err)
|
||||||
|
}
|
||||||
|
if len(client.requests) != 0 {
|
||||||
|
t.Fatalf("LLM calls = %d", len(client.requests))
|
||||||
|
}
|
||||||
|
empty := dnd.LocationList{Locations: []dnd.Location{}}
|
||||||
|
references := registryReferences(t, empty)
|
||||||
|
req := extractionRequest()
|
||||||
|
req.References = references
|
||||||
|
result, err := newExtractor(t, client, references).Extract(context.Background(), req)
|
||||||
|
if err != nil || result.Value.Occurrences == nil || len(result.Value.Occurrences) != 0 {
|
||||||
|
t.Fatalf("empty registry result = %#v, %v", result, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractResolvesGeneratedRegistryAtOperationTimeAndDoesNotMutateResponse(t *testing.T) {
|
||||||
|
locations := locationRegistry(t, "The Mill")
|
||||||
|
location := locations.Locations[0]
|
||||||
|
client := &fakeOccurrencesLLMClient{response: extractionResponse{Occurrences: []occurrenceResponse{{
|
||||||
|
LocationID: location.ID, Name: location.Name, Kind: "mentioned", SourceRefs: occurrenceRefs(30, 30),
|
||||||
|
}}}}
|
||||||
|
references := registryReferences(t, locations)
|
||||||
|
req := extractionRequest()
|
||||||
|
req.References = references
|
||||||
|
extractor := newExtractor(t, client)
|
||||||
|
result, err := extractor.Extract(context.Background(), req)
|
||||||
|
if err != nil || result.Value.Occurrences[0].Name != "The Mill" {
|
||||||
|
t.Fatalf("Extract() = %#v, %v", result, err)
|
||||||
|
}
|
||||||
|
if input := client.requests[0].Inputs[LocationRegistryReferenceSlot]; !strings.Contains(string(input.Content), location.ID) || input.OriginURI != "" {
|
||||||
|
t.Fatalf("generated registry prompt input = %#v", input)
|
||||||
|
}
|
||||||
|
if _, ok := extractor.ManifestMetadata()["location_registry_digest"]; ok {
|
||||||
|
t.Fatalf("operation registry leaked into static metadata: %#v", extractor.ManifestMetadata())
|
||||||
|
}
|
||||||
|
if client.response.Occurrences[0].SourceRefs[0].StartUnitID != 30 {
|
||||||
|
t.Fatalf("model response mutated: %#v", client.response)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractorContractsMetadataAndFailures(t *testing.T) {
|
||||||
|
if _, err := New(nil, Options{}); err == nil || !strings.Contains(err.Error(), "LLM client") {
|
||||||
|
t.Fatalf("New(nil) error = %v", err)
|
||||||
|
}
|
||||||
|
if _, err := New(&fakeOccurrencesLLMClient{}, Options{}, contracts.ReferenceSet{}, contracts.ReferenceSet{}); err == nil || !strings.Contains(err.Error(), "at most one reference set") {
|
||||||
|
t.Fatalf("New() error = %v", err)
|
||||||
|
}
|
||||||
|
malformed := contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{LocationRegistryReferenceSlot: {
|
||||||
|
Items: []contracts.ReferenceItem{{SlotName: LocationRegistryReferenceSlot, MediaType: "application/json", Content: []byte(`{"secret":"registry evidence"}`)}},
|
||||||
|
}}}
|
||||||
|
if _, err := New(&fakeOccurrencesLLMClient{}, Options{}, malformed); err == nil || !strings.Contains(err.Error(), "prepare location registry") || strings.Contains(err.Error(), "registry evidence") {
|
||||||
|
t.Fatalf("New() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
locations := locationRegistry(t, "The Mill")
|
||||||
|
references := registryReferences(t, locations)
|
||||||
|
req := extractionRequest()
|
||||||
|
req.References = references
|
||||||
|
extractor := newExtractor(t, &fakeOccurrencesLLMClient{}, references)
|
||||||
|
var nilExtractor *Extractor
|
||||||
|
for _, test := range []struct {
|
||||||
|
name string
|
||||||
|
extractor *Extractor
|
||||||
|
req contracts.TypedExtractionRequest
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{"nil extractor", nilExtractor, req, "extractor"},
|
||||||
|
{"nil client", &Extractor{}, req, "LLM client"},
|
||||||
|
{"invalid request", extractor, mismatchedSourceInputRequest(req), "must match chunk"},
|
||||||
|
} {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
if _, err := test.extractor.Extract(context.Background(), test.req); err == nil || !strings.Contains(err.Error(), test.want) {
|
||||||
|
t.Fatalf("Extract() error = %v", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
if _, err := newExtractor(t, &fakeOccurrencesLLMClient{err: errors.New("provider unavailable")}, references).Extract(context.Background(), req); err == nil || !strings.Contains(err.Error(), "provider unavailable") {
|
||||||
|
t.Fatalf("provider error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
spec := ModuleSpec()
|
||||||
|
if spec.Key != Key || spec.Stage != pipeline.StageExtract || spec.ExecutionClass != contracts.ExecutionClassLLMBacked || spec.ArtifactKind != dnd.LocationOccurrenceListKind {
|
||||||
|
t.Fatalf("ModuleSpec() = %#v", spec)
|
||||||
|
}
|
||||||
|
var slot contracts.ReferenceSlot
|
||||||
|
for _, candidate := range spec.ReferenceSlots {
|
||||||
|
if candidate.Name == LocationRegistryReferenceSlot {
|
||||||
|
slot = candidate
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !slot.Required || !reflect.DeepEqual(slot.AcceptedArtifactKinds, []contracts.ArtifactKind{dnd.LocationListKind}) || slot.MaxBytes != LocationRegistryMaxBytes {
|
||||||
|
t.Fatalf("location registry slot = %#v", slot)
|
||||||
|
}
|
||||||
|
registry := pipeline.NewExtractorRegistry()
|
||||||
|
if err := Register(registry); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, ok := registry.Spec(Key); !ok {
|
||||||
|
t.Fatalf("registration missing %q", Key)
|
||||||
|
}
|
||||||
|
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
|
||||||
|
t.Fatal("DecodeOptions() accepted unknown options")
|
||||||
|
}
|
||||||
|
metadata := newExtractor(t, &fakeOccurrencesLLMClient{}, references).ManifestMetadata()
|
||||||
|
for _, key := range []string{"prompt_sha256", "response_schema_sha256", "location_registry_digest"} {
|
||||||
|
if value, ok := metadata[key].(string); !ok || !strings.HasPrefix(value, "sha256:") {
|
||||||
|
t.Fatalf("metadata[%q] = %#v", key, metadata[key])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got := newExtractor(t, &fakeOccurrencesLLMClient{}, references).CheckpointFingerprints(); len(got) != 4 || got[3].Name != "location_registry" {
|
||||||
|
t.Fatalf("fingerprints = %#v", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func locationRegistry(t *testing.T, names ...string) dnd.LocationList {
|
||||||
|
t.Helper()
|
||||||
|
locations := make([]dnd.Location, len(names))
|
||||||
|
for index, name := range names {
|
||||||
|
refs := []source.SourceRef{{SourceID: "other-session", StartUnitID: index + 1, EndUnitID: index + 1}}
|
||||||
|
locations[index] = dnd.Location{ID: identity.DeriveID(name, refs), Name: name, SourceRefs: refs}
|
||||||
|
}
|
||||||
|
return dnd.LocationList{Locations: locations}
|
||||||
|
}
|
||||||
|
|
||||||
|
func registryReferences(t *testing.T, locations dnd.LocationList) contracts.ReferenceSet {
|
||||||
|
t.Helper()
|
||||||
|
content, err := locationcodec.New().Encode(locations)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{LocationRegistryReferenceSlot: {
|
||||||
|
Slot: contracts.ReferenceSlot{Name: LocationRegistryReferenceSlot},
|
||||||
|
Items: []contracts.ReferenceItem{{SlotName: LocationRegistryReferenceSlot, MediaType: locationcodec.MediaType, Content: content, Origin: contracts.ReferenceOrigin{Type: "generated"}}},
|
||||||
|
}}}
|
||||||
|
}
|
||||||
17
internal/modules/dnd/extract/locationoccurrences/model.go
Normal file
17
internal/modules/dnd/extract/locationoccurrences/model.go
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
package locationoccurrences
|
||||||
|
|
||||||
|
type extractionResponse struct {
|
||||||
|
Occurrences []occurrenceResponse `json:"occurrences"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type occurrenceResponse struct {
|
||||||
|
LocationID string `json:"location_id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Kind string `json:"kind"`
|
||||||
|
SourceRefs []occurrenceSourceRefResponse `json:"source_refs"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type occurrenceSourceRefResponse struct {
|
||||||
|
StartUnitID int `json:"start_unit_id"`
|
||||||
|
EndUnitID int `json:"end_unit_id"`
|
||||||
|
}
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
package locationoccurrences
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"sync"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/promptfs"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||||
|
)
|
||||||
|
|
||||||
|
const promptAssetRoot = "assets/prompts"
|
||||||
|
|
||||||
|
var promptAssetManifest = shared.PromptAssetManifest{
|
||||||
|
ModuleDir: PromptID,
|
||||||
|
ModuleFiles: []promptfs.ModulePromptFile{
|
||||||
|
{Name: "dnd.location_occurrences.yaml", Path: "assets/prompts/dnd.location_occurrences.yaml"},
|
||||||
|
{Name: "locations.md", Path: "assets/prompts/locations.md"},
|
||||||
|
{Name: "task.md", Path: "assets/prompts/task.md"},
|
||||||
|
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||||
|
},
|
||||||
|
SharedFiles: []string{
|
||||||
|
"common-dnd-system.md",
|
||||||
|
"common-dnd-extraction-evidence.md",
|
||||||
|
"common-dnd-identity.md",
|
||||||
|
"common-dnd-transcript.md",
|
||||||
|
"common-dnd-references.md",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
func RegisterPromptAssets(registry *llm.AssetRegistry) error {
|
||||||
|
promptFS, err := promptAssetManifest.PromptFS(embeddedAssets)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("prepare location-occurrence prompt assets: %w", err)
|
||||||
|
}
|
||||||
|
if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
|
||||||
|
}
|
||||||
|
|
||||||
|
func promptAssetMetadata() (string, error) {
|
||||||
|
promptAssetHashOnce.Do(func() {
|
||||||
|
promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
|
||||||
|
})
|
||||||
|
return promptAssetHash, promptAssetHashErr
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
promptAssetHashOnce sync.Once
|
||||||
|
promptAssetHash string
|
||||||
|
promptAssetHashErr error
|
||||||
|
)
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
package locationoccurrences
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||||
|
"gitea.maximumdirect.net/eric/promptkit"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRegisterPromptAssetsPreparesLocationOccurrencePrompt(t *testing.T) {
|
||||||
|
registry := llm.NewAssetRegistry()
|
||||||
|
if err := RegisterPromptAssets(registry); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
options, err := registry.PromptKitOptions()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
|
||||||
|
ID: "location-occurrences-test", Endpoint: "http://127.0.0.1:1/v1", Model: "test",
|
||||||
|
})))
|
||||||
|
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
|
||||||
|
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "location-occurrences-test",
|
||||||
|
Inputs: map[string]promptkit.ArtifactRef{
|
||||||
|
"transcript": promptkit.Inline(`{"units":[1]}`), "players": promptkit.Inline(" "), "party": promptkit.Inline(" "), "glossary": promptkit.Inline(" "),
|
||||||
|
"locations": promptkit.Inline(`{"locations":[{"id":"location:sha256:test","name":"The Mill"}]}`),
|
||||||
|
},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_location_occurrences_llm.v1.json" {
|
||||||
|
t.Fatalf("prepared prompt = %#v", prepared)
|
||||||
|
}
|
||||||
|
var registryMessage string
|
||||||
|
registryIndex := -1
|
||||||
|
evidenceIndex := -1
|
||||||
|
taskIndex := -1
|
||||||
|
for index, message := range prepared.Messages {
|
||||||
|
if strings.Contains(message.Content, "normalized location registry") {
|
||||||
|
registryMessage = message.Content
|
||||||
|
registryIndex = index
|
||||||
|
}
|
||||||
|
if strings.Contains(message.Content, "Transcript units are the only evidence") {
|
||||||
|
evidenceIndex = index
|
||||||
|
}
|
||||||
|
if strings.Contains(message.Content, "Extract Dungeons & Dragons location occurrences") {
|
||||||
|
taskIndex = index
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !strings.Contains(registryMessage, "location:sha256:test") || !strings.Contains(registryMessage, "The Mill") || strings.Contains(registryMessage, "source_refs") {
|
||||||
|
t.Fatalf("rendered prompt did not preserve source-free registry grounding: %s", registryMessage)
|
||||||
|
}
|
||||||
|
if evidenceIndex < 0 || taskIndex < 0 || registryIndex <= evidenceIndex || registryIndex >= taskIndex {
|
||||||
|
t.Fatalf("registry prompt placement = evidence %d, registry %d, task %d", evidenceIndex, registryIndex, taskIndex)
|
||||||
|
}
|
||||||
|
}
|
||||||
21
internal/modules/dnd/extract/locationoccurrences/schema.go
Normal file
21
internal/modules/dnd/extract/locationoccurrences/schema.go
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
package locationoccurrences
|
||||||
|
|
||||||
|
import "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||||
|
|
||||||
|
const (
|
||||||
|
PromptID = "dnd.location_occurrences"
|
||||||
|
ResponseSchemaKey = llm.ResponseSchemaKey("dnd_location_occurrences_llm")
|
||||||
|
ResponseSchemaID = "notarius.dnd.location_occurrences.llm"
|
||||||
|
ResponseSchemaName = "notarius_dnd_location_occurrences_llm_v1"
|
||||||
|
SchemaVersion = "v1"
|
||||||
|
)
|
||||||
|
|
||||||
|
func loadResponseSchema() (llm.ResponseSchema, error) {
|
||||||
|
return llm.LoadResponseSchema(embeddedAssets, llm.ResponseSchemaDefinition{
|
||||||
|
Key: ResponseSchemaKey,
|
||||||
|
ID: ResponseSchemaID,
|
||||||
|
Version: SchemaVersion,
|
||||||
|
Name: ResponseSchemaName,
|
||||||
|
AssetPath: "assets/schemas/dnd_location_occurrences_llm.v1.json",
|
||||||
|
})
|
||||||
|
}
|
||||||
@@ -0,0 +1,96 @@
|
|||||||
|
package locationoccurrences
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/santhosh-tekuri/jsonschema/v6"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestResponseSchemaRestrictsPrivateOccurrenceStructureAndKinds(t *testing.T) {
|
||||||
|
schema, err := loadResponseSchema()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if schema.Key != ResponseSchemaKey || schema.ID != ResponseSchemaID || schema.Name != ResponseSchemaName || schema.Version != SchemaVersion || !strings.HasPrefix(schema.SHA256, "sha256:") || !json.Valid(schema.JSONSchema) {
|
||||||
|
t.Fatalf("schema = %#v", schema)
|
||||||
|
}
|
||||||
|
valid := map[string]any{"occurrences": []any{map[string]any{
|
||||||
|
"location_id": "location:sha256:test", "name": "The Mill", "kind": "visited",
|
||||||
|
"source_refs": []any{map[string]any{"start_unit_id": 1, "end_unit_id": 2}},
|
||||||
|
}}}
|
||||||
|
if err := validateSchema(t, valid, schema.JSONSchema); err != nil {
|
||||||
|
t.Fatalf("valid response rejected: %v", err)
|
||||||
|
}
|
||||||
|
for _, mutate := range []func(map[string]any){
|
||||||
|
func(value map[string]any) { delete(value, "location_id") },
|
||||||
|
func(value map[string]any) { value["kind"] = "other" },
|
||||||
|
func(value map[string]any) { value["unexpected"] = true },
|
||||||
|
func(value map[string]any) {
|
||||||
|
value["source_refs"].([]any)[0].(map[string]any)["source_id"] = "assigned later"
|
||||||
|
},
|
||||||
|
} {
|
||||||
|
candidate := cloneCandidate(t, valid)
|
||||||
|
mutate(candidate["occurrences"].([]any)[0].(map[string]any))
|
||||||
|
if err := validateSchema(t, candidate, schema.JSONSchema); err == nil {
|
||||||
|
t.Fatal("schema accepted structurally invalid response")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResponseSchemaIsDefensiveAndContentSafe(t *testing.T) {
|
||||||
|
first, err := loadResponseSchema()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
first.JSONSchema[0] = '['
|
||||||
|
second, err := loadResponseSchema()
|
||||||
|
if err != nil || !json.Valid(second.JSONSchema) || bytes.Equal(first.JSONSchema, second.JSONSchema) {
|
||||||
|
t.Fatalf("schema = %s, %v", second.JSONSchema, err)
|
||||||
|
}
|
||||||
|
if diagnostics := second.DiagnosticsMap(); diagnostics["key"] != ResponseSchemaKey || diagnostics["id"] != ResponseSchemaID {
|
||||||
|
t.Fatalf("diagnostics = %#v", diagnostics)
|
||||||
|
} else if _, ok := diagnostics["json_schema"]; ok {
|
||||||
|
t.Fatalf("diagnostics leaked schema content: %#v", diagnostics)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func cloneCandidate(t *testing.T, value map[string]any) map[string]any {
|
||||||
|
t.Helper()
|
||||||
|
content, err := json.Marshal(value)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var clone map[string]any
|
||||||
|
if err := json.Unmarshal(content, &clone); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return clone
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateSchema(t *testing.T, value map[string]any, schemaContent []byte) error {
|
||||||
|
t.Helper()
|
||||||
|
content, err := json.Marshal(value)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
instance, err := jsonschema.UnmarshalJSON(bytes.NewReader(content))
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
document, err := jsonschema.UnmarshalJSON(bytes.NewReader(schemaContent))
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
compiler := jsonschema.NewCompiler()
|
||||||
|
if err := compiler.AddResource("schema.json", document); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
schema, err := compiler.Compile("schema.json")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return schema.Validate(instance)
|
||||||
|
}
|
||||||
@@ -0,0 +1,82 @@
|
|||||||
|
package locationoccurrences
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
)
|
||||||
|
|
||||||
|
func extractionRequest() contracts.TypedExtractionRequest {
|
||||||
|
doc := sourceDocument()
|
||||||
|
chunk := &source.Chunk{
|
||||||
|
ID: "session-occurrences:chunk:0", SourceID: doc.ID, Index: 0,
|
||||||
|
Ref: source.SourceRef{SourceID: doc.ID, StartUnitID: 10, EndUnitID: 30},
|
||||||
|
Content: []byte(`{"units":[10,20,30]}`), MediaType: "application/json",
|
||||||
|
Units: append([]source.SourceUnit(nil), doc.Units...),
|
||||||
|
}
|
||||||
|
return contracts.TypedExtractionRequest{
|
||||||
|
Source: doc, Chunk: chunk,
|
||||||
|
SourceInput: contracts.NewLLMInputMaterial("source", chunk.MediaType, chunk.Content, "sha256:chunk", "file:///session-occurrences.json"),
|
||||||
|
SessionID: "occurrence-session", LLMProfile: "occurrence-profile",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func sourceDocument() *source.SourceDocument {
|
||||||
|
return &source.SourceDocument{ID: "session-occurrences", Kind: "transcript", Format: "application/json", Digest: "sha256:test", Units: []source.SourceUnit{
|
||||||
|
{ID: 10, Kind: "transcript_segment", Text: "The party returns to the tavern."},
|
||||||
|
{ID: 20, Kind: "transcript_segment", Text: "They plan to travel to the tavern tomorrow."},
|
||||||
|
{ID: 30, Kind: "transcript_segment", Text: "They recall their first visit to the tavern."},
|
||||||
|
}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func occurrenceRefs(start, end int) []occurrenceSourceRefResponse {
|
||||||
|
return []occurrenceSourceRefResponse{{StartUnitID: start, EndUnitID: end}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func newExtractor(t *testing.T, client contracts.StructuredLLMClient, references ...contracts.ReferenceSet) *Extractor {
|
||||||
|
t.Helper()
|
||||||
|
extractor, err := New(client, Options{}, references...)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("New() error = %v", err)
|
||||||
|
}
|
||||||
|
return extractor
|
||||||
|
}
|
||||||
|
|
||||||
|
func mismatchedSourceInputRequest(req contracts.TypedExtractionRequest) contracts.TypedExtractionRequest {
|
||||||
|
req.SourceInput = contracts.NewLLMInputMaterial("source", "application/json", []byte(`{"different":true}`), "sha256:other", "")
|
||||||
|
return req
|
||||||
|
}
|
||||||
|
|
||||||
|
type fakeOccurrencesLLMClient struct {
|
||||||
|
response extractionResponse
|
||||||
|
err error
|
||||||
|
requests []contracts.StructuredCompletionRequest
|
||||||
|
}
|
||||||
|
|
||||||
|
func (client *fakeOccurrencesLLMClient) CompleteStructured(_ context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
|
||||||
|
client.requests = append(client.requests, cloneStructuredCompletionRequest(req))
|
||||||
|
if client.err != nil {
|
||||||
|
return contracts.StructuredCompletionResponse{}, client.err
|
||||||
|
}
|
||||||
|
target, ok := out.(*extractionResponse)
|
||||||
|
if !ok {
|
||||||
|
return contracts.StructuredCompletionResponse{}, errors.New("unexpected output target")
|
||||||
|
}
|
||||||
|
content, err := json.Marshal(client.response)
|
||||||
|
if err != nil {
|
||||||
|
return contracts.StructuredCompletionResponse{}, err
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(content, target); err != nil {
|
||||||
|
return contracts.StructuredCompletionResponse{}, err
|
||||||
|
}
|
||||||
|
return contracts.StructuredCompletionResponse{Content: content}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func cloneStructuredCompletionRequest(req contracts.StructuredCompletionRequest) contracts.StructuredCompletionRequest {
|
||||||
|
req.Inputs = req.Inputs.Clone()
|
||||||
|
return req
|
||||||
|
}
|
||||||
6
internal/modules/dnd/extract/locations/assets.go
Normal file
6
internal/modules/dnd/extract/locations/assets.go
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
package locations
|
||||||
|
|
||||||
|
import "embed"
|
||||||
|
|
||||||
|
//go:embed assets/prompts/*.yaml assets/prompts/*.md assets/schemas/*.json
|
||||||
|
var embeddedAssets embed.FS
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
id: dnd.locations
|
||||||
|
version: "v1"
|
||||||
|
default_profile: dnd-extraction
|
||||||
|
inputs:
|
||||||
|
- name: transcript
|
||||||
|
required: true
|
||||||
|
content_type: application/json
|
||||||
|
- name: players
|
||||||
|
required: false
|
||||||
|
content_type: text/plain
|
||||||
|
- name: party
|
||||||
|
required: false
|
||||||
|
content_type: text/plain
|
||||||
|
- name: glossary
|
||||||
|
required: false
|
||||||
|
content_type: text/plain
|
||||||
|
messages:
|
||||||
|
- role: system
|
||||||
|
content_file: ./sharedassets/common-dnd-system.md
|
||||||
|
- role: user
|
||||||
|
content_file: ./sharedassets/common-dnd-identity.md
|
||||||
|
- role: user
|
||||||
|
content_file: ./sharedassets/common-dnd-references.md
|
||||||
|
cache_control:
|
||||||
|
type: ephemeral
|
||||||
|
- role: user
|
||||||
|
content_file: ./sharedassets/common-dnd-transcript.md
|
||||||
|
cache_control:
|
||||||
|
type: ephemeral
|
||||||
|
- role: user
|
||||||
|
content_file: ./sharedassets/common-dnd-extraction-evidence.md
|
||||||
|
- role: user
|
||||||
|
content_file: ./task.md
|
||||||
|
- role: user
|
||||||
|
content_file: ./instructions.md
|
||||||
|
cache_control:
|
||||||
|
type: ephemeral
|
||||||
|
output:
|
||||||
|
format: json
|
||||||
|
validation_mode: json_schema
|
||||||
|
schema_path: dnd_locations_llm.v1.json
|
||||||
|
repair_attempts: 0
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
Return only observed location display names and narrow transcript source ranges.
|
||||||
|
|
||||||
|
Exclude people, creatures, objects, organizations, abstract concepts, and
|
||||||
|
places merely inferred from an event. Omit uncertain or unsupported places.
|
||||||
|
Campaign references may clarify terms already present in the transcript, but
|
||||||
|
they are not evidence and must never supply a source range.
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
Extract physical places established by the provided Dungeons & Dragons
|
||||||
|
transcript and cite where each place is identified.
|
||||||
|
|
||||||
|
Include planes, regions, settlements, districts, buildings, rooms, landmarks,
|
||||||
|
routes, and geographic features. A generic label such as "the tavern" is
|
||||||
|
allowed only when the transcript uses it for a specific place. Keep aliases and
|
||||||
|
nested places when the transcript identifies them; do not merge or invent
|
||||||
|
qualifiers for similarly named places.
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"$id": "notarius.dnd.locations.llm",
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["locations"],
|
||||||
|
"properties": {
|
||||||
|
"locations": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["name", "source_refs"],
|
||||||
|
"properties": {
|
||||||
|
"name": {"type": "string"},
|
||||||
|
"source_refs": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["start_unit_id", "end_unit_id"],
|
||||||
|
"properties": {
|
||||||
|
"start_unit_id": {"type": "integer"},
|
||||||
|
"end_unit_id": {"type": "integer"}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
86
internal/modules/dnd/extract/locations/canonicalize.go
Normal file
86
internal/modules/dnd/extract/locations/canonicalize.go
Normal file
@@ -0,0 +1,86 @@
|
|||||||
|
package locations
|
||||||
|
|
||||||
|
import (
|
||||||
|
"sort"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||||
|
)
|
||||||
|
|
||||||
|
type orderedLocationResponse struct {
|
||||||
|
value locationResponse
|
||||||
|
earliest int
|
||||||
|
hasEvidence bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func canonicalizeResponse(response *extractionResponse, order shared.SourceRefOrder, sourceID string) {
|
||||||
|
if response == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ordered := make([]orderedLocationResponse, len(response.Locations))
|
||||||
|
for index := range response.Locations {
|
||||||
|
earliest, hasEvidence := canonicalizeLocation(&response.Locations[index], order, sourceID)
|
||||||
|
ordered[index] = orderedLocationResponse{value: response.Locations[index], earliest: earliest, hasEvidence: hasEvidence}
|
||||||
|
}
|
||||||
|
sort.SliceStable(ordered, func(left, right int) bool {
|
||||||
|
if ordered[left].hasEvidence != ordered[right].hasEvidence {
|
||||||
|
return ordered[left].hasEvidence
|
||||||
|
}
|
||||||
|
if !ordered[left].hasEvidence {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return ordered[left].earliest < ordered[right].earliest
|
||||||
|
})
|
||||||
|
for index := range ordered {
|
||||||
|
response.Locations[index] = ordered[index].value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func canonicalizeLocation(location *locationResponse, order shared.SourceRefOrder, sourceID string) (int, bool) {
|
||||||
|
if location == nil {
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
refs := order.Canonicalize(canonicalSourceRefs(location.SourceRefs, sourceID))
|
||||||
|
location.SourceRefs = locationResponseRefs(refs)
|
||||||
|
return order.EarliestValid(refs)
|
||||||
|
}
|
||||||
|
|
||||||
|
func canonicalLocationList(response extractionResponse, sourceID string) dnd.LocationList {
|
||||||
|
if response.Locations == nil {
|
||||||
|
return dnd.LocationList{Locations: nil}
|
||||||
|
}
|
||||||
|
locations := make([]dnd.Location, len(response.Locations))
|
||||||
|
for index, location := range response.Locations {
|
||||||
|
refs := canonicalSourceRefs(location.SourceRefs, sourceID)
|
||||||
|
locations[index] = dnd.Location{
|
||||||
|
ID: identity.DeriveID(location.Name, refs),
|
||||||
|
Name: location.Name,
|
||||||
|
SourceRefs: refs,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return dnd.LocationList{Locations: locations}
|
||||||
|
}
|
||||||
|
|
||||||
|
func canonicalSourceRefs(values []locationSourceRefResponse, sourceID string) []source.SourceRef {
|
||||||
|
if values == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
refs := make([]source.SourceRef, len(values))
|
||||||
|
for index, value := range values {
|
||||||
|
refs[index] = source.SourceRef{SourceID: sourceID, StartUnitID: value.StartUnitID, EndUnitID: value.EndUnitID}
|
||||||
|
}
|
||||||
|
return refs
|
||||||
|
}
|
||||||
|
|
||||||
|
func locationResponseRefs(values []source.SourceRef) []locationSourceRefResponse {
|
||||||
|
if values == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
refs := make([]locationSourceRefResponse, len(values))
|
||||||
|
for index, value := range values {
|
||||||
|
refs[index] = locationSourceRefResponse{StartUnitID: value.StartUnitID, EndUnitID: value.EndUnitID}
|
||||||
|
}
|
||||||
|
return refs
|
||||||
|
}
|
||||||
152
internal/modules/dnd/extract/locations/extractor.go
Normal file
152
internal/modules/dnd/extract/locations/extractor.go
Normal file
@@ -0,0 +1,152 @@
|
|||||||
|
// Package locations extracts source-grounded D&D physical location candidates.
|
||||||
|
package locations
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
Key = "dnd/locations"
|
||||||
|
mappingPolicy = "dnd.locations.extract_mapping.v1"
|
||||||
|
)
|
||||||
|
|
||||||
|
var requiredCapabilities = []string{"chunks", "source.transcript"}
|
||||||
|
var providedCapabilities = []string{"dnd.locations"}
|
||||||
|
|
||||||
|
var referenceSlotDescriptions = shared.ReferenceSlotDescriptions{
|
||||||
|
Glossary: "Optional campaign glossary reference material used only for location disambiguation.",
|
||||||
|
Party: "Optional party roster reference material used only for location disambiguation.",
|
||||||
|
Players: "Optional player list reference material used only for location disambiguation.",
|
||||||
|
Roster: "Deprecated alias for party roster reference material used only for location disambiguation.",
|
||||||
|
}
|
||||||
|
|
||||||
|
func referenceSlots() []contracts.ReferenceSlot {
|
||||||
|
return shared.ReferenceSlots(referenceSlotDescriptions)
|
||||||
|
}
|
||||||
|
|
||||||
|
var _ contracts.Extractor[dnd.LocationList] = (*Extractor)(nil)
|
||||||
|
var _ contracts.ManifestMetadataProvider = (*Extractor)(nil)
|
||||||
|
var _ pipeline.CheckpointFingerprintProvider = (*Extractor)(nil)
|
||||||
|
|
||||||
|
type Options struct{}
|
||||||
|
|
||||||
|
type Extractor struct {
|
||||||
|
llm contracts.StructuredLLMClient
|
||||||
|
promptSHA string
|
||||||
|
responseSchemaSHA string
|
||||||
|
}
|
||||||
|
|
||||||
|
func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contracts.ReferenceSet) (*Extractor, error) {
|
||||||
|
if llmClient == nil {
|
||||||
|
return nil, extractorErrorf("LLM client must not be nil")
|
||||||
|
}
|
||||||
|
if len(references) > 1 {
|
||||||
|
return nil, extractorErrorf("at most one reference set may be supplied")
|
||||||
|
}
|
||||||
|
promptSHA, err := promptAssetMetadata()
|
||||||
|
if err != nil {
|
||||||
|
return nil, extractorErrorf("load prompt metadata: %w", err)
|
||||||
|
}
|
||||||
|
responseSchema, err := loadResponseSchema()
|
||||||
|
if err != nil {
|
||||||
|
return nil, extractorErrorf("load response schema: %w", err)
|
||||||
|
}
|
||||||
|
return &Extractor{llm: llmClient, promptSHA: promptSHA, responseSchemaSHA: responseSchema.SHA256}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Extractor) Key() string { return Key }
|
||||||
|
|
||||||
|
func (e *Extractor) ReferenceSlots() []contracts.ReferenceSlot { return referenceSlots() }
|
||||||
|
|
||||||
|
func (e *Extractor) ManifestMetadata() map[string]any {
|
||||||
|
if e == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return map[string]any{
|
||||||
|
"prompt_id": PromptID,
|
||||||
|
"prompt_version": SchemaVersion,
|
||||||
|
"prompt_sha256": e.promptSHA,
|
||||||
|
"response_schema_key": string(ResponseSchemaKey),
|
||||||
|
"response_schema_id": ResponseSchemaID,
|
||||||
|
"response_schema_name": ResponseSchemaName,
|
||||||
|
"response_schema_version": SchemaVersion,
|
||||||
|
"response_schema_sha256": e.responseSchemaSHA,
|
||||||
|
"identity_policy": identity.Policy,
|
||||||
|
"mapping_policy": mappingPolicy,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Extractor) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
|
||||||
|
if e == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return []pipeline.CheckpointFingerprint{
|
||||||
|
{Name: "prompt", Value: e.promptSHA},
|
||||||
|
{Name: "response_schema", Value: e.responseSchemaSHA},
|
||||||
|
{Name: "identity_policy", Value: identity.Policy},
|
||||||
|
{Name: "mapping_policy", Value: mappingPolicy},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Extractor) Extract(ctx context.Context, req contracts.TypedExtractionRequest) (contracts.TypedExtractionResult[dnd.LocationList], error) {
|
||||||
|
if e == nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.LocationList]{}, extractorErrorf("extractor must not be nil")
|
||||||
|
}
|
||||||
|
if e.llm == nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.LocationList]{}, extractorErrorf("LLM client must not be nil")
|
||||||
|
}
|
||||||
|
sourceInput, err := shared.PrepareChunkExtraction(ctx, req)
|
||||||
|
if err != nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.LocationList]{}, extractorErrorf("%w", err)
|
||||||
|
}
|
||||||
|
order := shared.NewSourceRefOrder(req.Source)
|
||||||
|
|
||||||
|
var response extractionResponse
|
||||||
|
if _, err := e.llm.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
|
||||||
|
StageName: Key, PromptID: PromptID, PromptVersion: SchemaVersion,
|
||||||
|
ProfileID: req.LLMProfile, SessionID: req.SessionID,
|
||||||
|
Inputs: shared.PromptInputs(sourceInput, req.References),
|
||||||
|
}, &response); err != nil {
|
||||||
|
return contracts.TypedExtractionResult[dnd.LocationList]{}, extractorErrorf("complete structured output: %w", err)
|
||||||
|
}
|
||||||
|
canonicalizeResponse(&response, order, req.Source.ID)
|
||||||
|
return contracts.TypedExtractionResult[dnd.LocationList]{Value: canonicalLocationList(response, req.Source.ID)}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func ModuleSpec() pipeline.ModuleSpec {
|
||||||
|
return pipeline.ModuleSpec{
|
||||||
|
Key: Key, Stage: pipeline.StageExtract, ExecutionClass: contracts.ExecutionClassLLMBacked,
|
||||||
|
Requires: append([]string(nil), requiredCapabilities...), Provides: append([]string(nil), providedCapabilities...),
|
||||||
|
ArtifactKind: dnd.LocationListKind, ReferenceSlots: referenceSlots(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func Register(registry *pipeline.ExtractorRegistry) error {
|
||||||
|
return pipeline.RegisterExtractorBuilder(registry, ModuleSpec(), validateOptions, func(request pipeline.BuildRequest) (contracts.Extractor[dnd.LocationList], error) {
|
||||||
|
options, err := DecodeOptions(request.Options)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return New(request.Dependencies.LLM, options, request.References)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }
|
||||||
|
|
||||||
|
func DecodeOptions(options map[string]any) (Options, error) {
|
||||||
|
if err := pipeline.RejectUnknownOptions(options); err != nil {
|
||||||
|
return Options{}, extractorErrorf("%w", err)
|
||||||
|
}
|
||||||
|
return Options{}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func extractorErrorf(format string, args ...any) error {
|
||||||
|
return fmt.Errorf("dnd locations extractor: "+format, args...)
|
||||||
|
}
|
||||||
125
internal/modules/dnd/extract/locations/extractor_test.go
Normal file
125
internal/modules/dnd/extract/locations/extractor_test.go
Normal file
@@ -0,0 +1,125 @@
|
|||||||
|
package locations
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestExtractMapsLocationsWithOwnedEvidenceAndDeterministicOrder(t *testing.T) {
|
||||||
|
client := &fakeLocationsLLMClient{response: extractionResponse{Locations: []locationResponse{
|
||||||
|
{Name: "The Tavern", SourceRefs: responseSourceRefs(3, 3)},
|
||||||
|
{Name: "Old Mill", SourceRefs: []locationSourceRefResponse{{StartUnitID: 2, EndUnitID: 2}, {StartUnitID: 1, EndUnitID: 1}, {StartUnitID: 1, EndUnitID: 1}}},
|
||||||
|
}}}
|
||||||
|
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Extract() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
refs := []source.SourceRef{{SourceID: "session-locations", StartUnitID: 1, EndUnitID: 1}, {SourceID: "session-locations", StartUnitID: 2, EndUnitID: 2}}
|
||||||
|
want := dnd.LocationList{Locations: []dnd.Location{
|
||||||
|
{ID: identity.DeriveID("Old Mill", refs), Name: "Old Mill", SourceRefs: refs},
|
||||||
|
{ID: identity.DeriveID("The Tavern", []source.SourceRef{{SourceID: "session-locations", StartUnitID: 3, EndUnitID: 3}}), Name: "The Tavern", SourceRefs: []source.SourceRef{{SourceID: "session-locations", StartUnitID: 3, EndUnitID: 3}}},
|
||||||
|
}}
|
||||||
|
if !reflect.DeepEqual(result.Value, want) {
|
||||||
|
t.Fatalf("Value = %#v, want %#v", result.Value, want)
|
||||||
|
}
|
||||||
|
result.Value.Locations[0].SourceRefs[0].StartUnitID = 99
|
||||||
|
for _, location := range client.response.Locations {
|
||||||
|
for _, ref := range location.SourceRefs {
|
||||||
|
if ref.StartUnitID == 99 {
|
||||||
|
t.Fatal("result source references alias the model response")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractRetainsSameNameLocationsAtDifferentAnchors(t *testing.T) {
|
||||||
|
client := &fakeLocationsLLMClient{response: extractionResponse{Locations: []locationResponse{
|
||||||
|
{Name: "the tavern", SourceRefs: responseSourceRefs(1, 1)},
|
||||||
|
{Name: "the tavern", SourceRefs: responseSourceRefs(3, 3)},
|
||||||
|
}}}
|
||||||
|
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
|
||||||
|
if err != nil || len(result.Value.Locations) != 2 {
|
||||||
|
t.Fatalf("Extract() = %#v, %v; want both same-name candidates", result, err)
|
||||||
|
}
|
||||||
|
if result.Value.Locations[0].ID == result.Value.Locations[1].ID || result.Value.Locations[0].Name != result.Value.Locations[1].Name {
|
||||||
|
t.Fatalf("locations = %#v, want distinct evidence-anchored IDs", result.Value.Locations)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractPreservesInvalidCandidatesForValidators(t *testing.T) {
|
||||||
|
client := &fakeLocationsLLMClient{content: []byte(`{"locations":[{"name":"","source_refs":[{"start_unit_id":0,"end_unit_id":-1}]}]}`)}
|
||||||
|
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Extract() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
location := result.Value.Locations[0]
|
||||||
|
if location.ID != "" || location.Name != "" || !reflect.DeepEqual(location.SourceRefs, []source.SourceRef{{SourceID: "session-locations", StartUnitID: 0, EndUnitID: -1}}) {
|
||||||
|
t.Fatalf("location = %#v, want invalid candidate preserved", location)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractPassesReferencesWithoutTreatingThemAsEvidence(t *testing.T) {
|
||||||
|
client := &fakeLocationsLLMClient{response: extractionResponse{Locations: []locationResponse{}}}
|
||||||
|
req := extractionRequest()
|
||||||
|
req.References = contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
|
||||||
|
"glossary": {Slot: contracts.ReferenceSlot{Name: "glossary"}, Items: []contracts.ReferenceItem{{SlotName: "glossary", Content: []byte("Old Mill: abandoned granary")}}},
|
||||||
|
}}
|
||||||
|
if _, err := newExtractor(t, client).Extract(context.Background(), req); err != nil {
|
||||||
|
t.Fatalf("Extract() error = %v", err)
|
||||||
|
}
|
||||||
|
inputs := client.requests[0].Inputs
|
||||||
|
if string(inputs["glossary"].Content) != "Old Mill: abandoned granary" || strings.Contains(string(inputs["transcript"].Content), "abandoned granary") {
|
||||||
|
t.Fatalf("prompt inputs = %#v, want separated reference material", inputs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractDoesNotMutateRequestMaterials(t *testing.T) {
|
||||||
|
client := &fakeLocationsLLMClient{response: extractionResponse{Locations: []locationResponse{{Name: "Old Mill", SourceRefs: responseSourceRefs(1, 1)}}}}
|
||||||
|
req := extractionRequest()
|
||||||
|
beforeUnits := append([]source.SourceUnit(nil), req.Source.Units...)
|
||||||
|
beforeChunkUnits := append([]source.SourceUnit(nil), req.Chunk.Units...)
|
||||||
|
beforeContent := append([]byte(nil), req.Chunk.Content...)
|
||||||
|
if _, err := newExtractor(t, client).Extract(context.Background(), req); err != nil {
|
||||||
|
t.Fatalf("Extract() error = %v", err)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(req.Source.Units, beforeUnits) || !reflect.DeepEqual(req.Chunk.Units, beforeChunkUnits) || !reflect.DeepEqual(req.Chunk.Content, beforeContent) {
|
||||||
|
t.Fatalf("Extract() mutated request: %#v", req)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractHandlesEmptyOutputAndLocalFailures(t *testing.T) {
|
||||||
|
empty, err := newExtractor(t, &fakeLocationsLLMClient{response: extractionResponse{Locations: []locationResponse{}}}).Extract(context.Background(), extractionRequest())
|
||||||
|
if err != nil || empty.Value.Locations == nil || len(empty.Value.Locations) != 0 {
|
||||||
|
t.Fatalf("empty Extract() = %#v, %v; want empty list", empty, err)
|
||||||
|
}
|
||||||
|
request := extractionRequest()
|
||||||
|
var nilExtractor *Extractor
|
||||||
|
for _, test := range []struct {
|
||||||
|
name string
|
||||||
|
extractor *Extractor
|
||||||
|
req contracts.TypedExtractionRequest
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{name: "nil extractor", extractor: nilExtractor, req: request, want: "extractor"},
|
||||||
|
{name: "nil client", extractor: &Extractor{}, req: request, want: "LLM client"},
|
||||||
|
{name: "preflight", extractor: newExtractor(t, &fakeLocationsLLMClient{}), req: mismatchedSourceInputRequest(request), want: "must match chunk"},
|
||||||
|
} {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
if _, err := test.extractor.Extract(context.Background(), test.req); err == nil || !strings.Contains(err.Error(), "dnd locations") || !strings.Contains(err.Error(), test.want) {
|
||||||
|
t.Fatalf("Extract() error = %v, want local context", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
_, err = newExtractor(t, &fakeLocationsLLMClient{err: errors.New("provider unavailable")}).Extract(context.Background(), request)
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "dnd locations") || !strings.Contains(err.Error(), "provider unavailable") {
|
||||||
|
t.Fatalf("provider error = %v, want contextual provider error", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
15
internal/modules/dnd/extract/locations/model.go
Normal file
15
internal/modules/dnd/extract/locations/model.go
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
package locations
|
||||||
|
|
||||||
|
type extractionResponse struct {
|
||||||
|
Locations []locationResponse `json:"locations"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type locationResponse struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
SourceRefs []locationSourceRefResponse `json:"source_refs"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type locationSourceRefResponse struct {
|
||||||
|
StartUnitID int `json:"start_unit_id"`
|
||||||
|
EndUnitID int `json:"end_unit_id"`
|
||||||
|
}
|
||||||
52
internal/modules/dnd/extract/locations/prompt_assets.go
Normal file
52
internal/modules/dnd/extract/locations/prompt_assets.go
Normal file
@@ -0,0 +1,52 @@
|
|||||||
|
package locations
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"sync"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/promptfs"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||||
|
)
|
||||||
|
|
||||||
|
const promptAssetRoot = "assets/prompts"
|
||||||
|
|
||||||
|
var promptAssetManifest = shared.PromptAssetManifest{
|
||||||
|
ModuleDir: PromptID,
|
||||||
|
ModuleFiles: []promptfs.ModulePromptFile{
|
||||||
|
{Name: "dnd.locations.yaml", Path: "assets/prompts/dnd.locations.yaml"},
|
||||||
|
{Name: "task.md", Path: "assets/prompts/task.md"},
|
||||||
|
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||||
|
},
|
||||||
|
SharedFiles: []string{
|
||||||
|
"common-dnd-system.md",
|
||||||
|
"common-dnd-extraction-evidence.md",
|
||||||
|
"common-dnd-identity.md",
|
||||||
|
"common-dnd-transcript.md",
|
||||||
|
"common-dnd-references.md",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
func RegisterPromptAssets(registry *llm.AssetRegistry) error {
|
||||||
|
promptFS, err := promptAssetManifest.PromptFS(embeddedAssets)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("prepare location prompt assets: %w", err)
|
||||||
|
}
|
||||||
|
if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
|
||||||
|
}
|
||||||
|
|
||||||
|
func promptAssetMetadata() (string, error) {
|
||||||
|
promptAssetHashOnce.Do(func() {
|
||||||
|
promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
|
||||||
|
})
|
||||||
|
return promptAssetHash, promptAssetHashErr
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
promptAssetHashOnce sync.Once
|
||||||
|
promptAssetHash string
|
||||||
|
promptAssetHashErr error
|
||||||
|
)
|
||||||
50
internal/modules/dnd/extract/locations/prompt_assets_test.go
Normal file
50
internal/modules/dnd/extract/locations/prompt_assets_test.go
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
package locations
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"slices"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||||
|
"gitea.maximumdirect.net/eric/promptkit"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRegisterPromptAssetsPreparesLocationPrompt(t *testing.T) {
|
||||||
|
for _, name := range []string{"common-dnd-system.md", "common-dnd-identity.md", "common-dnd-references.md", "common-dnd-transcript.md", "common-dnd-extraction-evidence.md"} {
|
||||||
|
if !slices.Contains(promptAssetManifest.SharedFiles, name) {
|
||||||
|
t.Fatalf("shared prompt assets = %#v, missing %q", promptAssetManifest.SharedFiles, name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
registry := llm.NewAssetRegistry()
|
||||||
|
if err := RegisterPromptAssets(registry); err != nil {
|
||||||
|
t.Fatalf("RegisterPromptAssets() error = %v", err)
|
||||||
|
}
|
||||||
|
options, err := registry.PromptKitOptions()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("PromptKitOptions() error = %v", err)
|
||||||
|
}
|
||||||
|
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{ID: "location-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "location-test-model"})))
|
||||||
|
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("NewEngine() error = %v", err)
|
||||||
|
}
|
||||||
|
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "location-test-profile", Inputs: map[string]promptkit.ArtifactRef{
|
||||||
|
"transcript": promptkit.Inline(`{"units":[1]}`), "players": promptkit.Inline(" "), "party": promptkit.Inline(" "), "glossary": promptkit.Inline(" "),
|
||||||
|
}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Prepare() error = %v", err)
|
||||||
|
}
|
||||||
|
if prepared.OutputContract.SchemaPath != "dnd_locations_llm.v1.json" {
|
||||||
|
t.Fatalf("output contract = %#v", prepared.OutputContract)
|
||||||
|
}
|
||||||
|
if len(prepared.Messages) != 7 || !strings.Contains(prepared.Messages[3].Content, `"units"`) || strings.Contains(prepared.Messages[3].Content, "location-test") {
|
||||||
|
t.Fatalf("prepared messages = %#v, want rendered transcript only in transcript message", prepared.Messages)
|
||||||
|
}
|
||||||
|
for _, index := range []int{2, 3, 6} {
|
||||||
|
if prepared.Messages[index].CacheControl == nil || prepared.Messages[index].CacheControl.Type != promptkit.CacheControlEphemeral {
|
||||||
|
t.Fatalf("message %d cache control = %#v, want ephemeral", index, prepared.Messages[index].CacheControl)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
53
internal/modules/dnd/extract/locations/registry_test.go
Normal file
53
internal/modules/dnd/extract/locations/registry_test.go
Normal file
@@ -0,0 +1,53 @@
|
|||||||
|
package locations
|
||||||
|
|
||||||
|
import (
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestModuleRegistrationAndMetadata(t *testing.T) {
|
||||||
|
if _, err := New(nil, Options{}); err == nil || !strings.Contains(err.Error(), "LLM client") {
|
||||||
|
t.Fatalf("New(nil) error = %v, want client rejection", err)
|
||||||
|
}
|
||||||
|
if _, err := New(&fakeLocationsLLMClient{}, Options{}, contracts.ReferenceSet{}, contracts.ReferenceSet{}); err == nil || !strings.Contains(err.Error(), "at most one") {
|
||||||
|
t.Fatalf("New() error = %v, want reference-set rejection", err)
|
||||||
|
}
|
||||||
|
want := pipeline.ModuleSpec{Key: Key, Stage: pipeline.StageExtract, ExecutionClass: contracts.ExecutionClassLLMBacked, Requires: []string{"chunks", "source.transcript"}, Provides: []string{"dnd.locations"}, ArtifactKind: dnd.LocationListKind, ReferenceSlots: referenceSlots()}
|
||||||
|
if got := ModuleSpec(); !reflect.DeepEqual(got, want) {
|
||||||
|
t.Fatalf("ModuleSpec() = %#v, want %#v", got, want)
|
||||||
|
}
|
||||||
|
registry := pipeline.NewExtractorRegistry()
|
||||||
|
if err := Register(registry); err != nil {
|
||||||
|
t.Fatalf("Register() error = %v", err)
|
||||||
|
}
|
||||||
|
if got, ok := registry.Spec(Key); !ok || !reflect.DeepEqual(got, want) {
|
||||||
|
t.Fatalf("registry spec = %#v, present = %t", got, ok)
|
||||||
|
}
|
||||||
|
if err := Register(nil); err == nil || !strings.Contains(err.Error(), "extractor registry") {
|
||||||
|
t.Fatalf("Register(nil) error = %v, want registry rejection", err)
|
||||||
|
}
|
||||||
|
if _, err := DecodeOptions(map[string]any{"unknown": true}); err == nil {
|
||||||
|
t.Fatal("DecodeOptions() accepted unknown option")
|
||||||
|
}
|
||||||
|
extractor := newExtractor(t, &fakeLocationsLLMClient{})
|
||||||
|
metadata := extractor.ManifestMetadata()
|
||||||
|
for key, value := range map[string]string{
|
||||||
|
"prompt_id": PromptID, "prompt_version": SchemaVersion,
|
||||||
|
"response_schema_key": string(ResponseSchemaKey), "response_schema_id": ResponseSchemaID,
|
||||||
|
"response_schema_name": ResponseSchemaName, "response_schema_version": SchemaVersion,
|
||||||
|
"identity_policy": identity.Policy, "mapping_policy": mappingPolicy,
|
||||||
|
} {
|
||||||
|
if metadata[key] != value {
|
||||||
|
t.Fatalf("metadata[%q] = %#v, want %q", key, metadata[key], value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got := extractor.CheckpointFingerprints(); len(got) != 4 || got[0].Name != "prompt" || got[1].Name != "response_schema" || got[2].Value != identity.Policy || got[3].Value != mappingPolicy {
|
||||||
|
t.Fatalf("CheckpointFingerprints() = %#v", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
21
internal/modules/dnd/extract/locations/schema.go
Normal file
21
internal/modules/dnd/extract/locations/schema.go
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
package locations
|
||||||
|
|
||||||
|
import "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||||
|
|
||||||
|
const (
|
||||||
|
PromptID = "dnd.locations"
|
||||||
|
ResponseSchemaKey = llm.ResponseSchemaKey("dnd_locations_llm")
|
||||||
|
ResponseSchemaID = "notarius.dnd.locations.llm"
|
||||||
|
ResponseSchemaName = "notarius_dnd_locations_llm_v1"
|
||||||
|
SchemaVersion = "v1"
|
||||||
|
)
|
||||||
|
|
||||||
|
func loadResponseSchema() (llm.ResponseSchema, error) {
|
||||||
|
return llm.LoadResponseSchema(embeddedAssets, llm.ResponseSchemaDefinition{
|
||||||
|
Key: ResponseSchemaKey,
|
||||||
|
ID: ResponseSchemaID,
|
||||||
|
Version: SchemaVersion,
|
||||||
|
Name: ResponseSchemaName,
|
||||||
|
AssetPath: "assets/schemas/dnd_locations_llm.v1.json",
|
||||||
|
})
|
||||||
|
}
|
||||||
25
internal/modules/dnd/extract/locations/schema_test.go
Normal file
25
internal/modules/dnd/extract/locations/schema_test.go
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
package locations
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestLocationResponseSchemaIsPrivateAndStructural(t *testing.T) {
|
||||||
|
schema, err := loadResponseSchema()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("loadResponseSchema() error = %v", err)
|
||||||
|
}
|
||||||
|
if schema.Key != ResponseSchemaKey || schema.ID != ResponseSchemaID || schema.Name != ResponseSchemaName || schema.Version != SchemaVersion || !strings.HasPrefix(schema.SHA256, "sha256:") || !json.Valid(schema.JSONSchema) {
|
||||||
|
t.Fatalf("schema = %#v, want location response schema", schema)
|
||||||
|
}
|
||||||
|
var document map[string]any
|
||||||
|
if err := json.Unmarshal(schema.JSONSchema, &document); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
encoded, err := json.Marshal(document)
|
||||||
|
if err != nil || strings.Contains(string(encoded), `"id"`) {
|
||||||
|
t.Fatalf("schema = %s, want no durable ID field", encoded)
|
||||||
|
}
|
||||||
|
}
|
||||||
89
internal/modules/dnd/extract/locations/test_helpers_test.go
Normal file
89
internal/modules/dnd/extract/locations/test_helpers_test.go
Normal file
@@ -0,0 +1,89 @@
|
|||||||
|
package locations
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||||
|
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||||
|
)
|
||||||
|
|
||||||
|
func extractionRequest() contracts.TypedExtractionRequest {
|
||||||
|
doc := sourceDocument()
|
||||||
|
chunk := &source.Chunk{
|
||||||
|
ID: "session-locations:chunk:0", SourceID: doc.ID, Index: 0,
|
||||||
|
Ref: source.SourceRef{SourceID: doc.ID, StartUnitID: 1, EndUnitID: 3},
|
||||||
|
Content: []byte(`{"units":[1,2,3]}`), MediaType: "application/json",
|
||||||
|
Units: append([]source.SourceUnit(nil), doc.Units...),
|
||||||
|
}
|
||||||
|
return contracts.TypedExtractionRequest{
|
||||||
|
Source: doc, Chunk: chunk,
|
||||||
|
SourceInput: contracts.NewLLMInputMaterial("source", chunk.MediaType, chunk.Content, "sha256:chunk", "file:///session-locations.json"),
|
||||||
|
SessionID: "location-session", LLMProfile: "location-profile",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func sourceDocument() *source.SourceDocument {
|
||||||
|
return &source.SourceDocument{ID: "session-locations", Kind: "transcript", Format: "application/json", Digest: "sha256:test", Units: []source.SourceUnit{
|
||||||
|
{ID: 1, Kind: "transcript_segment", Text: "The party enters the Old Mill."},
|
||||||
|
{ID: 2, Kind: "transcript_segment", Text: "They leave the old road behind."},
|
||||||
|
{ID: 3, Kind: "transcript_segment", Text: "The tavern is quiet."},
|
||||||
|
}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func responseSourceRefs(startUnitID, endUnitID int) []locationSourceRefResponse {
|
||||||
|
return []locationSourceRefResponse{{StartUnitID: startUnitID, EndUnitID: endUnitID}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func newExtractor(t *testing.T, client contracts.StructuredLLMClient, references ...contracts.ReferenceSet) *Extractor {
|
||||||
|
t.Helper()
|
||||||
|
extractor, err := New(client, Options{}, references...)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("New() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
return extractor
|
||||||
|
}
|
||||||
|
|
||||||
|
func mismatchedSourceInputRequest(req contracts.TypedExtractionRequest) contracts.TypedExtractionRequest {
|
||||||
|
req.SourceInput = contracts.NewLLMInputMaterial("source", "application/json", []byte(`{"different":true}`), "sha256:other", "file:///other.json")
|
||||||
|
return req
|
||||||
|
}
|
||||||
|
|
||||||
|
type fakeLocationsLLMClient struct {
|
||||||
|
response extractionResponse
|
||||||
|
content []byte
|
||||||
|
err error
|
||||||
|
requests []contracts.StructuredCompletionRequest
|
||||||
|
}
|
||||||
|
|
||||||
|
func (client *fakeLocationsLLMClient) CompleteStructured(_ context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
|
||||||
|
client.requests = append(client.requests, cloneStructuredCompletionRequest(req))
|
||||||
|
if client.err != nil {
|
||||||
|
return contracts.StructuredCompletionResponse{}, client.err
|
||||||
|
}
|
||||||
|
target, ok := out.(*extractionResponse)
|
||||||
|
if !ok {
|
||||||
|
return contracts.StructuredCompletionResponse{}, errors.New("unexpected output target")
|
||||||
|
}
|
||||||
|
content := append([]byte(nil), client.content...)
|
||||||
|
if len(content) != 0 {
|
||||||
|
if err := json.Unmarshal(content, target); err != nil {
|
||||||
|
return contracts.StructuredCompletionResponse{}, err
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
*target = client.response
|
||||||
|
var err error
|
||||||
|
content, err = json.Marshal(client.response)
|
||||||
|
if err != nil {
|
||||||
|
return contracts.StructuredCompletionResponse{}, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return contracts.StructuredCompletionResponse{Content: content}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func cloneStructuredCompletionRequest(req contracts.StructuredCompletionRequest) contracts.StructuredCompletionRequest {
|
||||||
|
req.Inputs = req.Inputs.Clone()
|
||||||
|
return req
|
||||||
|
}
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user