Compare commits
40 Commits
v0.2.0
...
29fcad6e9b
| Author | SHA1 | Date | |
|---|---|---|---|
| 29fcad6e9b | |||
| f5fd115046 | |||
| 7f28899730 | |||
| 84c0758455 | |||
| 5002864e88 | |||
| 55b188fd84 | |||
| d1f43df88e | |||
| d52387c1f7 | |||
| 9c5e3cff14 | |||
| 811d5b8bd9 | |||
| a168c13b85 | |||
| dd61a4efda | |||
| 228cc6ee83 | |||
| 06170e1f65 | |||
| 7715baa1f6 | |||
| 98506db1a9 | |||
| bb4855f0c6 | |||
| c51934d5c6 | |||
| c3513da880 | |||
| c7d853ea52 | |||
| da7fcdaffd | |||
| b6aad4fa98 | |||
| 9c6af28d02 | |||
| 04eabdfcb9 | |||
| a8a99c1037 | |||
| e15007fffb | |||
| e6b7c61f45 | |||
| 42973215fa | |||
| ba1d112d1f | |||
| b722131d57 | |||
| a92d2c0885 | |||
| 02ec10d66b | |||
| 9dd57dbfa4 | |||
| c164a3fc69 | |||
| 8834df617f | |||
| db2adb52da | |||
| fc3c128171 | |||
| 8a15b083a0 | |||
| d9dae2b639 | |||
| 7a4fd7be7a |
@@ -2,8 +2,9 @@
|
||||
|
||||
Notarius is a Go CLI for turning source material into structured artifacts with
|
||||
configured extraction pipelines. The implemented D&D workflow reads Seriatim
|
||||
transcript JSON and can produce scene descriptions, item and currency events,
|
||||
NPC identities, combat turns, NPC interactions, and spell casts.
|
||||
transcript JSON and can produce location registries and occurrences, scene
|
||||
descriptions, item and currency events, NPC identities, combat turns, NPC
|
||||
interactions, enemy events, and spell casts.
|
||||
|
||||
## Quickstart
|
||||
|
||||
|
||||
12
docs/cli.md
12
docs/cli.md
@@ -40,7 +40,7 @@ pipeline ID and **--input** are required.
|
||||
| **--debug-dir path** | Override the debug-bundle root. Requires **--debug**. |
|
||||
| **--only lane-a,lane-b** | Run only the selected comma-separated artifact lanes when that selection is valid for the configured pipeline. |
|
||||
| **--llm-profile id** | Highest-precedence configured profile for selected LLM-backed bindings and validators; it replaces binding and [pipeline](config.md#pipelines) defaults. |
|
||||
| **--session-id id** | Supply a non-empty prompt session identifier to LLM-backed module calls. |
|
||||
| **--session-id id** | Override the generated prompt session identifier with a non-empty value for LLM-backed module calls. |
|
||||
| **--reasoning-effort value** | Replace the selected PromptKit profile's reasoning effort for every LLM-backed call in this run. The value must be non-empty and the flag may be specified only once. |
|
||||
| **--clear-reasoning-effort** | Clear reasoning effort inherited from the selected PromptKit profile for every LLM-backed call in this run. |
|
||||
| **--reference selector=path** | Add or replace a file reference binding. Repeatable. |
|
||||
@@ -57,6 +57,16 @@ Persistent reasoning settings remain a PromptKit profile concern.
|
||||
**--recompute-step** requires **--resume**; checkpoint requirements and reuse
|
||||
behavior are documented in [Operations](operations.md).
|
||||
|
||||
Every run uses one effective prompt session. Without **--session-id**, Notarius
|
||||
generates a stable `notarius:v1:` identifier from the trimmed resolved input
|
||||
module key and the input file's exact raw bytes. The same module and bytes
|
||||
therefore produce the same identifier, regardless of pipeline, references,
|
||||
profile, retries, or run settings. An explicit non-empty value replaces that
|
||||
default. Session identifiers are visible to providers; they are non-secret
|
||||
correlation identifiers, not credential storage. See
|
||||
[Operations](operations.md#operational-limits) for privacy and workflow
|
||||
guidance.
|
||||
|
||||
### Reference selectors
|
||||
|
||||
Use **--reference** only for a reference slot declared by the selected
|
||||
|
||||
@@ -58,7 +58,7 @@ Built-in defaults are:
|
||||
|
||||
| Field | Default |
|
||||
| --- | --- |
|
||||
| **concurrency.total_llm** | 1 |
|
||||
| **concurrency.total_llm** | 16 |
|
||||
| **concurrency.stage_workers.extract** | Effective **total_llm** |
|
||||
| **output.directory** | **./notarius-output** |
|
||||
| **cache.chunk_plans.mode** | **auto** |
|
||||
@@ -370,14 +370,33 @@ selected target declares them:
|
||||
| **players** | Optional text player context. |
|
||||
| **glossary** | Optional text campaign glossary. |
|
||||
| **spell_catalog** | Optional JSON spell-catalog overlay for spell extraction and normalization. See [spell-catalog overlays](integrations/dnd-spell-catalog-overlays.md). |
|
||||
| **npcs** | Normalized NPC registry. Optional for spells and combat turns; required for NPC interactions. |
|
||||
| **scene_descriptions** | Required normalized scene-description artifact for combat-turn extraction. |
|
||||
| **locations** | Required normalized location registry for location-occurrence extraction and normalization. |
|
||||
| **npcs** | Normalized NPC registry. Optional for spells and combat turns; required for NPC interactions and enemy-event extraction and normalization. |
|
||||
| **scene_descriptions** | Required normalized scene-description artifact for combat-turn and enemy-event extraction. |
|
||||
| **combat_turns** | Required normalized combat-turn artifact for enemy-event extraction. |
|
||||
| **npc_interactions** | Required normalized NPC-interaction artifact for enemy-event extraction. |
|
||||
|
||||
Location-occurrence and enemy-event artifact slots have the following exact
|
||||
binding contracts. Durable semantics and wire shapes remain in their
|
||||
[location-occurrence](integrations/dnd-location-occurrence-artifacts.md) and
|
||||
[enemy-event](integrations/dnd-enemy-event-artifacts.md) contracts.
|
||||
|
||||
| Slot | Accepted artifact kind | Media type | Maximum size | Required stage |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `npcs` | `dnd/npc-list` | `application/json` | 1,048,576 bytes | extract and normalize |
|
||||
| `scene_descriptions` | `dnd/scene-description-list` | `application/json` | 1,048,576 bytes | extract only |
|
||||
| `combat_turns` | `dnd/combat-turn-list` | `application/json` | 1,048,576 bytes | extract only |
|
||||
| `npc_interactions` | `dnd/npc-interaction-list` | `application/json` | 1,048,576 bytes | extract only |
|
||||
| `locations` | `dnd/location-list` | `application/json` | 1,048,576 bytes | location-occurrence extract and normalize |
|
||||
|
||||
Scene descriptions accept **party**, **players**, and **glossary**, but not
|
||||
**roster**. NPC interactions require **npcs** for both extraction and
|
||||
normalization. Combat turns require **scene_descriptions** for extraction; the
|
||||
normalized combat-turn module may use optional **npcs**. The complete example
|
||||
shows generated **npcs** and **scene_descriptions** bindings.
|
||||
normalized combat-turn module may use optional **npcs**. Location occurrences
|
||||
require **locations** for extraction and normalization. Enemy-event extraction
|
||||
requires all four of its JSON artifact slots; its normalizer requires **npcs**.
|
||||
The [complete example](../examples/dnd-complete.config.yml) shows the ordered
|
||||
generated bindings.
|
||||
|
||||
## Production Module Keys
|
||||
|
||||
@@ -385,18 +404,27 @@ shows generated **npcs** and **scene_descriptions** bindings.
|
||||
| --- | --- |
|
||||
| Input | **seriatim** |
|
||||
| Chunk | **generic**, **dnd/scenes** |
|
||||
| Extract | **dnd/spells**, **dnd/npcs**, **dnd/combat-turns**, **dnd/item-events**, **dnd/npc-interactions**, **dnd/scene-descriptions** |
|
||||
| Extract | **dnd/spells**, **dnd/npcs**, **dnd/combat-turns**, **dnd/item-events**, **dnd/npc-interactions**, **dnd/scene-descriptions**, **dnd/enemy-events**, **dnd/locations**, **dnd/location-occurrences** |
|
||||
| Merge | **appendorder** |
|
||||
| Normalize | **noop**, **dnd/spells**, **dnd/npcs**, **dnd/combat-turns**, **dnd/item-events**, **dnd/npc-interactions**, **dnd/scene-descriptions** |
|
||||
| Normalize | **noop**, **dnd/spells**, **dnd/npcs**, **dnd/combat-turns**, **dnd/item-events**, **dnd/npc-interactions**, **dnd/scene-descriptions**, **dnd/enemy-events**, **dnd/locations**, **dnd/location-occurrences** |
|
||||
| Output | **json** |
|
||||
|
||||
`dnd/locations` extraction and normalization are `llm_backed`; location
|
||||
normalization may use the pipeline's selected LLM profile for bounded duplicate
|
||||
proposals. `dnd/location-occurrences` extraction is `llm_backed`, while its
|
||||
normalizer is `deterministic`. The complete example binds the registry in one
|
||||
step and the occurrence lane in the next.
|
||||
|
||||
The D&D artifact contracts define each emitted schema:
|
||||
[spells](integrations/dnd-spell-artifacts.md),
|
||||
[NPCs](integrations/dnd-npc-artifacts.md),
|
||||
[NPC interactions](integrations/dnd-npc-interaction-artifacts.md),
|
||||
[combat turns](integrations/dnd-combat-turn-artifacts.md),
|
||||
[item events](integrations/dnd-item-event-artifacts.md), and
|
||||
[scene descriptions](integrations/dnd-scene-description-artifacts.md).
|
||||
[item events](integrations/dnd-item-event-artifacts.md),
|
||||
[scene descriptions](integrations/dnd-scene-description-artifacts.md), and
|
||||
[enemy events](integrations/dnd-enemy-event-artifacts.md),
|
||||
[locations](integrations/dnd-location-artifacts.md), and
|
||||
[location occurrences](integrations/dnd-location-occurrence-artifacts.md).
|
||||
|
||||
## Production Validator Keys And Default Chains
|
||||
|
||||
@@ -411,6 +439,9 @@ Available validator keys are:
|
||||
| Item events | **extract/dnd/item-events/shape**, **extract/dnd/item-events/source_refs**, **extract/dnd/item-events/source_relatedness**, **normalize/dnd/item-events/invariants** |
|
||||
| NPC interactions | **extract/dnd/npc-interactions/shape**, **extract/dnd/npc-interactions/registry**, **extract/dnd/npc-interactions/source_refs**, **extract/dnd/npc-interactions/source_relatedness**, **normalize/dnd/npc-interactions/invariants** |
|
||||
| Scene descriptions | **extract/dnd/scene-descriptions/shape**, **extract/dnd/scene-descriptions/source_refs**, **extract/dnd/scene-descriptions/source_relatedness**, **normalize/dnd/scene-descriptions/invariants** |
|
||||
| Enemy events | **extract/dnd/enemy-events/shape**, **extract/dnd/enemy-events/engagements**, **extract/dnd/enemy-events/source_refs**, **extract/dnd/enemy-events/source_relatedness**, **normalize/dnd/enemy-events/invariants** |
|
||||
| Locations | **extract/dnd/locations/shape**, **extract/dnd/locations/source_refs**, **extract/dnd/locations/source_relatedness**, **normalize/dnd/locations/identity** |
|
||||
| Location occurrences | **extract/dnd/location-occurrences/shape**, **extract/dnd/location-occurrences/registry**, **extract/dnd/location-occurrences/source_refs**, **extract/dnd/location-occurrences/source_relatedness**, **normalize/dnd/location-occurrences/invariants** |
|
||||
|
||||
When no override is configured, production D&D bindings use the following
|
||||
ordered chains. Each row lists extract then normalize; spell chains are the
|
||||
@@ -424,6 +455,9 @@ same at both stages.
|
||||
| Item events | generic/valid_json, extract/dnd/item-events/shape, extract/dnd/item-events/source_refs, generic/valid_json_schema, extract/dnd/item-events/source_relatedness | generic/valid_json, extract/dnd/item-events/shape, normalize/dnd/item-events/invariants, extract/dnd/item-events/source_refs, generic/valid_json_schema, extract/dnd/item-events/source_relatedness |
|
||||
| NPC interactions | generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness | generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, normalize/dnd/npc-interactions/invariants, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness |
|
||||
| Scene descriptions | generic/valid_json, extract/dnd/scene-descriptions/shape, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness | generic/valid_json, extract/dnd/scene-descriptions/shape, normalize/dnd/scene-descriptions/invariants, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness |
|
||||
| Enemy events | generic/valid_json, extract/dnd/enemy-events/shape, extract/dnd/enemy-events/engagements, extract/dnd/enemy-events/source_refs, generic/valid_json_schema, extract/dnd/enemy-events/source_relatedness | generic/valid_json, extract/dnd/enemy-events/shape, normalize/dnd/enemy-events/invariants, extract/dnd/enemy-events/source_refs, generic/valid_json_schema, extract/dnd/enemy-events/source_relatedness |
|
||||
| Locations | generic/valid_json, extract/dnd/locations/shape, extract/dnd/locations/source_refs, generic/valid_json_schema, extract/dnd/locations/source_relatedness | generic/valid_json, extract/dnd/locations/shape, normalize/dnd/locations/identity, extract/dnd/locations/source_refs, generic/valid_json_schema, extract/dnd/locations/source_relatedness |
|
||||
| Location occurrences | generic/valid_json, extract/dnd/location-occurrences/shape, extract/dnd/location-occurrences/registry, extract/dnd/location-occurrences/source_refs, generic/valid_json_schema, extract/dnd/location-occurrences/source_relatedness | generic/valid_json, extract/dnd/location-occurrences/shape, extract/dnd/location-occurrences/registry, normalize/dnd/location-occurrences/invariants, extract/dnd/location-occurrences/source_refs, generic/valid_json_schema, extract/dnd/location-occurrences/source_relatedness |
|
||||
|
||||
Chains are only registered for the D&D extract and normalize modules shown
|
||||
above; select an explicit override when a different compatible chain is
|
||||
|
||||
@@ -27,10 +27,13 @@ notarius run pipeline-id \
|
||||
```
|
||||
|
||||
Use absolute paths for supplied input, configuration, output-root, and
|
||||
reference files. When a stable prompt session identifier or references are
|
||||
needed, pass the supported CLI flags. Supply credentials through Notarius's
|
||||
documented configuration and environment mechanisms, never as command-line
|
||||
arguments or generated secret-bearing configuration.
|
||||
reference files. Notarius generates a stable prompt session for the resolved
|
||||
input module and exact input bytes. Pass **--session-id** only when intentionally
|
||||
grouping different invocations under a different session. Supply credentials
|
||||
through Notarius's documented configuration and environment mechanisms, never
|
||||
as command-line arguments or generated secret-bearing configuration. In
|
||||
particular, a session identifier is provider-visible and is not a credential
|
||||
mechanism.
|
||||
|
||||
Wait for the process before interpreting standard output. Only an exit status
|
||||
of 0 permits decoding the receipt. On a nonzero exit, retain standard error for
|
||||
|
||||
@@ -64,6 +64,8 @@ kind, and complete valid evidence. It does not infer turns, initiative, or
|
||||
actions from registry or scene data.
|
||||
|
||||
The [NPC-interaction artifact](dnd-npc-interaction-artifacts.md) records
|
||||
broader NPC occurrences. The [JSON output contract](json-output.md) defines
|
||||
publication, and [D&D module internals](../internal/dnd.md) describes routing
|
||||
and validation mechanics.
|
||||
broader NPC occurrences. The [enemy-event artifact](dnd-enemy-event-artifacts.md)
|
||||
uses combat turns as grounding only; turns do not establish an enemy event or
|
||||
its outcome. The [JSON output contract](json-output.md) defines publication,
|
||||
and [D&D module internals](../internal/dnd.md) describes routing and validation
|
||||
mechanics.
|
||||
|
||||
114
docs/integrations/dnd-enemy-event-artifacts.md
Normal file
114
docs/integrations/dnd-enemy-event-artifacts.md
Normal file
@@ -0,0 +1,114 @@
|
||||
# D&D Enemy-Event Artifact
|
||||
|
||||
This contract defines the durable, source-grounded enemy-event occurrence list.
|
||||
It records enemies directly established as opposing the party and explicitly
|
||||
observed combat outcomes. It is an ordered observation artifact from which a
|
||||
consumer may derive a ledger; it is not a ledger, encounter roster, or terminal
|
||||
state model.
|
||||
|
||||
## Identity and compatibility
|
||||
|
||||
| Property | Value |
|
||||
| --- | --- |
|
||||
| Artifact kind | `dnd/enemy-event-list` |
|
||||
| Schema ID | `notarius.dnd.enemy_events` |
|
||||
| Schema name | `notarius_dnd_enemy_events_v1` |
|
||||
| Schema version | `v1` |
|
||||
| Media type | `application/json` |
|
||||
|
||||
`v1` is a strict JSON object with required `events`; the array may be empty.
|
||||
Event and source-reference objects reject unknown fields. An incompatible shape
|
||||
change requires a new schema version.
|
||||
|
||||
## Wire shape
|
||||
|
||||
Every event has these required fields:
|
||||
|
||||
| Field | Contract |
|
||||
| --- | --- |
|
||||
| `name` | Non-empty display name or directly grounded collective subject label. |
|
||||
| `kind` | `engaged`, `killed`, `fled`, `captured`, or `incapacitated`. |
|
||||
| `source_refs` | One or more current-transcript evidence ranges. |
|
||||
|
||||
Each source reference has exactly `source_id`, `start_unit_id`, and
|
||||
`end_unit_id`. It identifies an inclusive current-transcript range; unit IDs
|
||||
are positive and the start may not follow the end.
|
||||
|
||||
```json
|
||||
{
|
||||
"events": [
|
||||
{
|
||||
"name": "Ashfang",
|
||||
"kind": "engaged",
|
||||
"source_refs": [
|
||||
{"source_id": "session-7", "start_unit_id": 41, "end_unit_id": 42}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "Ashfang",
|
||||
"kind": "fled",
|
||||
"source_refs": [
|
||||
{"source_id": "session-7", "start_unit_id": 57, "end_unit_id": 58}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Event semantics and evidence
|
||||
|
||||
| Kind | Required evidence |
|
||||
| --- | --- |
|
||||
| `engaged` | The subject is directly established as actively opposing the party in combat. At most one engagement is emitted for one subject in one combat scene. |
|
||||
| `killed` | The transcript explicitly establishes that the subject died or was killed. Damage, defeat, disappearance, or combat ending is insufficient. |
|
||||
| `fled` | The subject explicitly escapes, retreats, or otherwise leaves combat to avoid continued engagement. Movement or absence from later turns is insufficient. |
|
||||
| `captured` | The subject is explicitly taken prisoner or secured under the party's control. A grapple or temporary restraint alone is insufficient. |
|
||||
| `incapacitated` | The subject is explicitly rendered unable to continue acting without being established as killed or captured. A missed turn is insufficient. |
|
||||
|
||||
The current transcript is the only event evidence. Campaign context and
|
||||
normalized NPC, scene-description, combat-turn, and NPC-interaction artifacts
|
||||
can ground names or control combat eligibility, but none may supply event
|
||||
evidence. An outcome may share evidence with an engagement, in which case both
|
||||
events are retained.
|
||||
|
||||
Extraction is limited to chunks with an exact combat-scene classification. An
|
||||
exact non-combat classification produces an accepted empty list. Missing or
|
||||
mismatched classification also produces an accepted empty list and a
|
||||
`scene_classification_unavailable` warning.
|
||||
|
||||
## Subjects, normalization, and order
|
||||
|
||||
A subject matching the normalized NPC registry uses that registry's canonical
|
||||
display name. Unmatched hostile creatures, summoned entities, and directly
|
||||
grounded groups remain valid subjects. An unnamed homogeneous group uses the
|
||||
narrowest transcript-grounded label, such as `Orcs`, `One orc`, or `Remaining
|
||||
orcs`; the artifact never invents synthetic member identities or quantities.
|
||||
Party members, allies, neutral observers, mentioned-but-absent enemies, hazards,
|
||||
traps, and environmental effects are excluded.
|
||||
|
||||
Normalization collapses surrounding and repeated internal whitespace in subject
|
||||
display values, canonicalizes recognized registry names, canonicalizes and
|
||||
deduplicates exact source ranges, then orders events by valid evidence
|
||||
chronology, normalized subject identity, display name, kind, and reference
|
||||
sequence. The deterministic kind tie order is `engaged`,
|
||||
`incapacitated`, `captured`, `fled`, then `killed`. Only entries with the same
|
||||
normalized name, kind, and complete canonical evidence sequence are collapsed.
|
||||
Different kinds, evidence, repeated engagement in separate scenes, and later
|
||||
outcomes remain separate. A later engagement for the same named subject is
|
||||
preserved after an earlier outcome because the artifact does not assert an
|
||||
irreversible state transition.
|
||||
|
||||
## Non-goals
|
||||
|
||||
The artifact has no NPC or scene ID, quantity, confidence, description,
|
||||
rationale, summary, current state, or inferred terminal outcome. It does not
|
||||
emit `active` or `unresolved`; consumers may derive an unresolved ledger view
|
||||
only when an engagement has no later explicit outcome. It never infers an
|
||||
outcome from turn absence, scene termination, initiative order, hit-point
|
||||
guesses, or other artifacts.
|
||||
|
||||
The [JSON output contract](json-output.md) defines publication. Configuration
|
||||
keys, required generated-reference slots, and validator-chain selection are
|
||||
defined in the [configuration reference](../config.md). Implementation and
|
||||
prompt-grounding mechanics are described in the
|
||||
[D&D module internals](../internal/dnd.md).
|
||||
89
docs/integrations/dnd-location-artifacts.md
Normal file
89
docs/integrations/dnd-location-artifacts.md
Normal file
@@ -0,0 +1,89 @@
|
||||
# D&D Location Artifact
|
||||
|
||||
This contract defines the durable, source-grounded location registry produced
|
||||
by `dnd/locations`. It records transcript-established physical places for one
|
||||
source document; it is not a map, location hierarchy, campaign-wide world
|
||||
registry, or location description.
|
||||
|
||||
## Identity and compatibility
|
||||
|
||||
| Property | Value |
|
||||
| --- | --- |
|
||||
| Artifact kind | `dnd/location-list` |
|
||||
| Schema ID | `notarius.dnd.locations` |
|
||||
| Schema name | `notarius_dnd_locations_v1` |
|
||||
| Schema version | `v1` |
|
||||
| Media type | `application/json` |
|
||||
| Identity policy | `dnd.locations.identity.v1` |
|
||||
|
||||
`v1` accepts one strict JSON object with required `locations`; the array may be
|
||||
empty. Location and source-reference objects reject unknown fields. An
|
||||
incompatible artifact shape or identity-policy change uses a new version or
|
||||
policy.
|
||||
|
||||
## Wire shape and identity
|
||||
|
||||
Each location has these required fields:
|
||||
|
||||
| Field | Contract |
|
||||
| --- | --- |
|
||||
| `id` | `location:sha256:` followed by 64 lowercase hexadecimal characters. |
|
||||
| `name` | Non-empty transcript-established display name. |
|
||||
| `source_refs` | One or more transcript evidence ranges that identify the place. |
|
||||
|
||||
A source reference has exactly `source_id`, `start_unit_id`, and `end_unit_id`.
|
||||
The source ID identifies the transcript, unit IDs are positive inclusive unit
|
||||
identifiers, and the start may not follow the end.
|
||||
|
||||
```json
|
||||
{
|
||||
"locations": [
|
||||
{
|
||||
"id": "location:sha256:5c1a91f15729df0b8c257093865fdf2452b43c215375e8cf2341aa9c37bb99aa",
|
||||
"name": "Moon Gate",
|
||||
"source_refs": [
|
||||
{"source_id": "session-7", "start_unit_id": 4, "end_unit_id": 5}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
The ID is deterministic and scoped to the source document. Notarius normalizes
|
||||
the display name for comparison with Unicode NFKC, supported apostrophe
|
||||
normalization, collapsed whitespace, and case folding. It hashes compact JSON
|
||||
for this array, using the earliest canonical source reference as the anchor:
|
||||
|
||||
```text
|
||||
["dnd.locations.identity.v1", comparison_name, source_id, start_unit_id, end_unit_id]
|
||||
```
|
||||
|
||||
The canonical ID is the lowercase SHA-256 digest of those bytes with the
|
||||
`location:sha256:` prefix. Equal display names are allowed when their evidence
|
||||
anchors differ, so a generic name does not force distinct places to collapse.
|
||||
|
||||
## Scope, reconciliation, and evidence
|
||||
|
||||
Locations are physical or spatial places established by the transcript, such
|
||||
as planes, regions, settlements, districts, buildings, rooms, landmarks,
|
||||
routes, and geographic features. A generic label is permitted only when it
|
||||
identifies a specific place in the transcript. Notarius does not infer an
|
||||
unstated place or add hierarchy, coordinates, descriptions, participants, or
|
||||
ownership.
|
||||
|
||||
Normalization first applies deterministic display, evidence, and ID rules. It
|
||||
then may use a bounded LLM-assisted proposal to reconcile semantically duplicate
|
||||
records. The proposal is validated and applied conservatively; invalid or
|
||||
unusable proposals retain the deterministic result with retry or fallback
|
||||
diagnostics. The registry's source references establish registry provenance,
|
||||
not evidence for later artifacts.
|
||||
|
||||
## Consumers and publication
|
||||
|
||||
`dnd/location-occurrences` requires one approved location registry through its
|
||||
`locations` reference slot. Its prompt receives an ordered source-free `{id,
|
||||
name}` projection and must not treat registry references as occurrence
|
||||
evidence. See the [location-occurrence artifact](dnd-location-occurrence-artifacts.md)
|
||||
for that contract, [Configuration](../config.md#references-and-ordered-handoffs)
|
||||
for binding rules, and the [JSON output contract](json-output.md) for
|
||||
publication.
|
||||
84
docs/integrations/dnd-location-occurrence-artifacts.md
Normal file
84
docs/integrations/dnd-location-occurrence-artifacts.md
Normal file
@@ -0,0 +1,84 @@
|
||||
# D&D Location-Occurrence Artifact
|
||||
|
||||
This contract defines the durable occurrence list produced by
|
||||
`dnd/location-occurrences`. It records source-grounded ways the party relates
|
||||
to locations in a required normalized location registry; it does not extend
|
||||
that registry or infer a place absent from it.
|
||||
|
||||
## Identity and compatibility
|
||||
|
||||
| Property | Value |
|
||||
| --- | --- |
|
||||
| Artifact kind | `dnd/location-occurrence-list` |
|
||||
| Schema ID | `notarius.dnd.location_occurrences` |
|
||||
| Schema name | `notarius_dnd_location_occurrences_v1` |
|
||||
| Schema version | `v1` |
|
||||
| Media type | `application/json` |
|
||||
|
||||
`v1` accepts one strict JSON object with required `occurrences`; the array may
|
||||
be empty. Occurrence and source-reference objects reject unknown fields. An
|
||||
incompatible shape change requires a new schema version.
|
||||
|
||||
## Wire shape
|
||||
|
||||
Each occurrence has these required fields:
|
||||
|
||||
| Field | Contract |
|
||||
| --- | --- |
|
||||
| `location_id` | Exact ID from the required normalized [location registry](dnd-location-artifacts.md). |
|
||||
| `name` | Exact canonical display name for `location_id` in that registry. |
|
||||
| `kind` | One of `visited`, `planned`, `recalled`, or `mentioned`. |
|
||||
| `source_refs` | One or more current-transcript evidence ranges for this occurrence. |
|
||||
|
||||
A source reference has exactly `source_id`, `start_unit_id`, and `end_unit_id`.
|
||||
It identifies an inclusive range in the current transcript; unit IDs are
|
||||
positive and the start may not follow the end.
|
||||
|
||||
```json
|
||||
{
|
||||
"occurrences": [
|
||||
{
|
||||
"location_id": "location:sha256:5c1a91f15729df0b8c257093865fdf2452b43c215375e8cf2341aa9c37bb99aa",
|
||||
"name": "Moon Gate",
|
||||
"kind": "visited",
|
||||
"source_refs": [
|
||||
{"source_id": "session-7", "start_unit_id": 12, "end_unit_id": 13}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Occurrence categories
|
||||
|
||||
| Kind | Meaning |
|
||||
| --- | --- |
|
||||
| `visited` | The transcript establishes physical party presence, including arrival, continuing presence, or departure. |
|
||||
| `planned` | The party explicitly proposes, intends, or agrees to future travel; speculation alone is not enough. |
|
||||
| `recalled` | The transcript explicitly recounts prior party presence before the current live events. |
|
||||
| `mentioned` | The location is explicit but no stronger category applies, including lore, directions, third-party activity, non-actionable speculation, a mere hypothetical reference, or out-of-character discussion. |
|
||||
|
||||
For overlapping evidence, precedence is `visited`, then `planned`, then
|
||||
`recalled`, then `mentioned`. For example, “What if we went to Moon Gate?” is
|
||||
eligible as `mentioned` when its narrow evidence explicitly references that
|
||||
registry location, but it is not `planned` without an actual proposal,
|
||||
intention, or agreement to travel. Inferred, unstated, uncertain, and
|
||||
unsupported places or occurrences are omitted. Normalization
|
||||
canonicalizes the registry name, orders and deduplicates source references, and
|
||||
orders occurrences by source chronology, location ID, name, kind, and reference
|
||||
sequence. It collapses only exact duplicates with the same ID, kind, and
|
||||
complete canonical evidence sequence.
|
||||
|
||||
## Required grounding and evidence
|
||||
|
||||
Both extraction and normalization require exactly one `locations` reference of
|
||||
kind `dnd/location-list`, media type `application/json`, and at most 1 MiB. The
|
||||
registry provides identity grounding only: unknown IDs and mismatched ID/name
|
||||
pairs are rejected rather than guessed or reassigned. The current transcript is
|
||||
the only evidence source for an occurrence; registry evidence and provenance
|
||||
never become occurrence evidence.
|
||||
|
||||
See [Configuration](../config.md#d-d-reference-slots) for the selectable slot
|
||||
and generated-handoff compatibility, [D&D module internals](../internal/dnd.md)
|
||||
for implementation behavior, and the [JSON output contract](json-output.md)
|
||||
for publication.
|
||||
@@ -55,15 +55,26 @@ with the same canonical identity, retains their earliest position, and merges
|
||||
their canonicalized evidence; it does not add aliases, roles, descriptions, or
|
||||
relationship fields.
|
||||
|
||||
When evidence supports a semantically duplicate group, the canonical display
|
||||
name is one of that group's supplied candidates. A complete, stable proper name
|
||||
is preferred over an abbreviation. An unadorned proper name is preferred over
|
||||
the same name plus a contextual class, role, title, or relationship descriptor
|
||||
unless the transcript establishes that descriptor as part of the person's
|
||||
name. A longer candidate is not preferred solely because it includes such a
|
||||
descriptor.
|
||||
|
||||
## Scope and consumers
|
||||
|
||||
Only individually identifiable NPC names with transcript evidence belong in
|
||||
this artifact. Groups, generic roles, invented labels, and descriptive
|
||||
enrichment are excluded. Its source references prove registry provenance; they
|
||||
do not become evidence for a spell, interaction, or combat occurrence.
|
||||
do not become evidence for a spell, interaction, combat, or enemy-event
|
||||
occurrence.
|
||||
|
||||
This registry can ground actor or caster names in the [spell](dnd-spell-artifacts.md)
|
||||
and [combat-turn](dnd-combat-turn-artifacts.md) artifacts. It is required to
|
||||
resolve the canonical `name` in an [NPC interaction](dnd-npc-interaction-artifacts.md).
|
||||
The [enemy-event artifact](dnd-enemy-event-artifacts.md) also uses it only for
|
||||
subject grounding and canonical display names.
|
||||
The [JSON output contract](json-output.md) defines publication, and
|
||||
[D&D module internals](../internal/dnd.md) owns pipeline mechanics.
|
||||
|
||||
@@ -74,5 +74,8 @@ Only entries with the same canonical name, kind, and complete valid evidence
|
||||
sequence are collapsed; distinct categories or evidence remain separate.
|
||||
|
||||
See the [combat-turn artifact](dnd-combat-turn-artifacts.md) for combat-action
|
||||
occurrences and the [JSON output contract](json-output.md) for publication.
|
||||
Pipeline mechanics are described in [D&D module internals](../internal/dnd.md).
|
||||
occurrences. The [enemy-event artifact](dnd-enemy-event-artifacts.md) consumes
|
||||
only `combat_opponent` interactions as grounding; they never establish an enemy
|
||||
event or outcome. The [JSON output contract](json-output.md) defines
|
||||
publication. Pipeline mechanics are described in
|
||||
[D&D module internals](../internal/dnd.md).
|
||||
|
||||
@@ -62,8 +62,9 @@ durable fields, or the same source range with different kind, title, or
|
||||
summary, is invalid. It does not merge adjacent ranges, alter prose, or infer
|
||||
missing scenes.
|
||||
|
||||
The [combat-turn artifact](dnd-combat-turn-artifacts.md) uses an exact matching
|
||||
The [combat-turn artifact](dnd-combat-turn-artifacts.md) and
|
||||
[enemy-event artifact](dnd-enemy-event-artifacts.md) use an exact matching
|
||||
`combat` scene only as eligibility control; scene title, summary, and source
|
||||
reference never become combat evidence. Publication is defined by the
|
||||
reference never become their evidence. Publication is defined by the
|
||||
[JSON output contract](json-output.md); implementation details live in
|
||||
[D&D module internals](../internal/dnd.md).
|
||||
|
||||
@@ -74,8 +74,11 @@ than infer a lane schema from its name. The current D&D payload contracts are
|
||||
[spells](dnd-spell-artifacts.md), [NPCs](dnd-npc-artifacts.md),
|
||||
[NPC interactions](dnd-npc-interaction-artifacts.md),
|
||||
[combat turns](dnd-combat-turn-artifacts.md),
|
||||
[item events](dnd-item-event-artifacts.md), and
|
||||
[scene descriptions](dnd-scene-description-artifacts.md).
|
||||
[item events](dnd-item-event-artifacts.md),
|
||||
[scene descriptions](dnd-scene-description-artifacts.md), and
|
||||
[enemy events](dnd-enemy-event-artifacts.md),
|
||||
[locations](dnd-location-artifacts.md), and
|
||||
[location occurrences](dnd-location-occurrence-artifacts.md).
|
||||
|
||||
## `manifest.json`
|
||||
|
||||
@@ -98,6 +101,11 @@ summarize results without embedding lane payload bytes. A chunk-plan summary is
|
||||
provenance for the plan used by this run; cache records, debug artifacts, and
|
||||
other operational state are not published as bundle files.
|
||||
|
||||
When present, `metadata.session_id` is the effective non-secret routing
|
||||
correlation identifier used for the run. It can be visible to providers and is
|
||||
not a substitute for a cache or checkpoint identity. Its generation and
|
||||
override behavior are defined by the [CLI reference](../cli.md#run).
|
||||
|
||||
Each `llm_profiles` entry identifies effective, non-secret LLM execution
|
||||
provenance:
|
||||
|
||||
|
||||
@@ -48,11 +48,13 @@ adapter boundary. It also retains responsibility for pipeline retries,
|
||||
scheduling, debug persistence, redaction, profile provenance, and conversion
|
||||
from private model responses into durable domain artifacts.
|
||||
|
||||
Notarius sends its trimmed run session through PromptKit's direct session
|
||||
Notarius sends one stable effective session through PromptKit's direct session
|
||||
field, which is authoritative for provider session behavior. It also retains
|
||||
the same value as the `session_id` prompt variable for maintained prompt
|
||||
compatibility. Session IDs are stable, non-secret correlation identifiers and
|
||||
may be exposed to providers and provider observability.
|
||||
compatibility. The generated identifier is 76 ASCII characters, within
|
||||
PromptKit v0.5.0's 256-code-point session limit. Session IDs are non-secret
|
||||
correlation identifiers and may be exposed to providers and provider
|
||||
observability. The CLI contract owns generation and override behavior.
|
||||
|
||||
Notarius records PromptKit's selected backend ID and effective reasoning
|
||||
setting as optional run-manifest provenance. Endpoint-only profiles have no
|
||||
|
||||
@@ -89,9 +89,11 @@ handoff:
|
||||
profiles;
|
||||
4. materialize external or generated references and record redacted invocation
|
||||
and resolution provenance when debug capture is enabled;
|
||||
5. construct registries, the scheduled LLM client, prepared modules, and the
|
||||
requested cache/checkpoint collaborators;
|
||||
6. read the source input and invoke the framework runner; and
|
||||
5. construct registries, the scheduled LLM client, and prepared modules;
|
||||
6. read the source input once, resolve its effective session from the explicit
|
||||
override or resolved input module and raw bytes, then construct requested
|
||||
checkpoint collaborators and invoke the framework runner with that same
|
||||
value; and
|
||||
7. write the runner's logical output files only after a successful run, then
|
||||
complete the command report and user-facing result.
|
||||
|
||||
@@ -102,6 +104,13 @@ final command result. Detailed state lifecycle, resume handling, and physical
|
||||
path confinement are maintained in [Run State Internals](state.md) and
|
||||
[Operations](../operations.md).
|
||||
|
||||
The CLI owns the versioned generated-session policy and resolves the sole
|
||||
effective value before checkpoint construction. It records that value in the
|
||||
final debug invocation summary when capture is enabled and passes it unchanged
|
||||
to checkpoint identity and `pipeline.RunInput`. The public flag and stability
|
||||
contract are defined by the [CLI reference](../cli.md#run); framework and LLM
|
||||
packages only transport the supplied value.
|
||||
|
||||
For `run --json`, the CLI constructs and encodes its private run-result receipt
|
||||
after a successful runner result is available, before it publishes logical
|
||||
output files. It writes the prepared receipt to standard output only after
|
||||
|
||||
@@ -7,7 +7,7 @@ selectable keys, bindings, reference syntax, and default validator chains.
|
||||
|
||||
## Durable Artifact Contracts
|
||||
|
||||
The six lanes have separate durable wire contracts. This guide deliberately
|
||||
The nine lanes have separate durable wire contracts. This guide deliberately
|
||||
does not repeat their JSON shapes or schemas.
|
||||
|
||||
| Lane | Durable contract |
|
||||
@@ -18,6 +18,9 @@ does not repeat their JSON shapes or schemas.
|
||||
| Item events | [item-event artifacts](../integrations/dnd-item-event-artifacts.md) |
|
||||
| NPC interactions | [NPC-interaction artifacts](../integrations/dnd-npc-interaction-artifacts.md) |
|
||||
| Scene descriptions | [scene-description artifacts](../integrations/dnd-scene-description-artifacts.md) |
|
||||
| Enemy events | [enemy-event artifacts](../integrations/dnd-enemy-event-artifacts.md) |
|
||||
| Locations | [location artifacts](../integrations/dnd-location-artifacts.md) |
|
||||
| Location occurrences | [location-occurrence artifacts](../integrations/dnd-location-occurrence-artifacts.md) |
|
||||
|
||||
## Family Composition
|
||||
|
||||
@@ -25,9 +28,9 @@ The D&D registrar registers the family’s artifact codecs, extractors, typed
|
||||
append-order mergers, normalizers, validators, prompt assets, fallback LLM
|
||||
profile asset, and default validator chains. Each extractor and normalizer has
|
||||
a stable module spec, explicit execution class, strict option decoding, and a
|
||||
typed builder. Scene chunking, every extractor, and NPC normalization are
|
||||
registered as `llm_backed`; the remaining current D&D mergers and normalizers
|
||||
are `deterministic`. The metadata is available to catalog inspection and
|
||||
typed builder. Scene chunking, every extractor, NPC normalization, and location
|
||||
normalization are registered as `llm_backed`; the remaining current D&D mergers
|
||||
and normalizers are `deterministic`. The metadata is available to catalog inspection and
|
||||
resolved-pipeline debug data and determines which selected bindings inherit the
|
||||
pipeline profile. Configuration remains the canonical owner of the exact keys,
|
||||
profile precedence, and validator order.
|
||||
@@ -41,10 +44,10 @@ the contracts above define durable data.
|
||||
## Prompt Construction
|
||||
|
||||
D&D extractors assemble prompts from an ordered manifest of shared and
|
||||
module-owned assets. Reuse the shared D&D system, evidence, identity,
|
||||
reference, and transcript assets instead of copying their text into individual
|
||||
modules. A manifest’s declared sequence, including cache-control placement, is
|
||||
part of the prompt behavior.
|
||||
module-owned assets. The location extractor and occurrence extractor reuse the
|
||||
shared D&D system, evidence, identity, reference, and transcript assets instead
|
||||
of copying their text into individual modules. A manifest’s declared sequence,
|
||||
including cache-control placement, is part of the prompt behavior.
|
||||
|
||||
Every maintained D&D LLM prompt selects `dnd-extraction` as its default
|
||||
profile. The D&D registrar embeds that fallback profile with the maintained
|
||||
@@ -71,7 +74,9 @@ remains stable.
|
||||
The other D&D LLM prompts intentionally follow different patterns. Scene
|
||||
chunking has no sibling extraction lane with which to share its full transcript,
|
||||
so it renders campaign references before its task and instructions, then places
|
||||
the cacheable full transcript last. NPC normalization keeps its task and
|
||||
the cacheable full transcript last. NPC and location normalization share the
|
||||
entity-reconciliation response schema and safety boundary while retaining their
|
||||
own task and identity rules. NPC normalization keeps its task and
|
||||
cacheable instructions before the candidate collection, followed by the
|
||||
cacheable transcript windows: candidates must be available before their
|
||||
supporting evidence is evaluated, and those windows are not a cross-lane
|
||||
@@ -101,14 +106,20 @@ relatedness validators report advisory evidence concerns. The configured order
|
||||
is documented in
|
||||
[Configuration](../config.md#production-validator-keys-and-default-chains).
|
||||
|
||||
Enemy-event extraction additionally rejects a second `engaged` observation for
|
||||
the same comparison identity within one scene-scoped result. Normalization may
|
||||
combine results from distinct scenes, so it intentionally does not apply that
|
||||
rule. Configuration owns the exact validator key and chain position.
|
||||
|
||||
Normalizers are deterministic for spells, combat turns, item events, NPC
|
||||
interactions, and scene descriptions. They canonicalize display values and
|
||||
evidence, use source-document order for stable output, and issue bounded
|
||||
warnings for changes or collapsed duplicates. The NPC normalizer is the
|
||||
intentional exception: it first produces a deterministic candidate set, then
|
||||
uses a bounded structured-LLM proposal to reconcile identity groups. Invalid
|
||||
or unusable proposals retain the deterministic result and surface retry or
|
||||
fallback diagnostics; the model does not directly replace durable records.
|
||||
interactions, scene descriptions, enemy events, and location occurrences. They canonicalize display
|
||||
values and evidence, use source-document order for stable output, and issue
|
||||
bounded warnings for changes or collapsed duplicates. The NPC and location
|
||||
normalizers are intentional exceptions: each first produces a deterministic
|
||||
candidate set, then may use a bounded structured-LLM proposal to reconcile
|
||||
identity groups. Invalid or unusable proposals retain the deterministic result
|
||||
and surface retry or fallback diagnostics; the model does not directly replace
|
||||
durable records.
|
||||
|
||||
## Generated References And Grounding
|
||||
|
||||
@@ -122,7 +133,13 @@ NPC registries are names-only grounding projections: they may canonicalize
|
||||
actors for spells and combat turns and are required for NPC interactions, but
|
||||
they do not supply evidence. Scene-description registries are eligibility-only
|
||||
projections: they retain the current chunk’s classification data, not scene
|
||||
prose or evidence, and exist to route combat extraction.
|
||||
prose or evidence, and exist to route combat extraction. Enemy-event extraction
|
||||
also projects combat turns to `actor` and `turn_kind` and filters NPC
|
||||
interactions to `combat_opponent` names and kinds. Location registries project
|
||||
ordered `{id, name}` pairs to location-occurrence extraction and normalization;
|
||||
exact ID/name matching keeps same-name locations distinguishable. These compact
|
||||
projections, like NPC grounding, are source-free guidance and never event
|
||||
evidence.
|
||||
|
||||
## Lane-Specific Rules
|
||||
|
||||
@@ -137,11 +154,17 @@ shared helper changes.
|
||||
| Item events | Uses campaign context for disambiguation but has no NPC-registry or scene-description dependency. |
|
||||
| NPC interactions | Requires the normalized NPC registry at extraction and normalization, using it for canonical actor grounding only. |
|
||||
| Scene descriptions | Produces the classifications consumed by combat routing; it does not consume an NPC registry or provide evidence for combat artifacts. |
|
||||
| Enemy events | Requires NPC, scene-description, combat-turn, and NPC-interaction artifacts. It calls the LLM only for an exact `combat` classification, records ordered observations rather than terminal state, and normalizes recognized names through the NPC registry while preserving grounded collective labels. |
|
||||
| Locations | Produces a source-anchored, session-scoped registry. Its LLM-assisted reconciliation is proposal-only and never collapses same-name places without validated identity and evidence rules. |
|
||||
| Location occurrences | Requires the normalized location registry for both extraction and normalization. Its [durable occurrence categories](../integrations/dnd-location-occurrence-artifacts.md#occurrence-categories) distinguish explicit speculation from unsupported inference; the deterministic normalizer enforces exact registry grounding and never turns registry provenance into occurrence evidence. |
|
||||
|
||||
The combat and scene-description contracts describe their exact handoff and
|
||||
empty-result behavior in more detail:
|
||||
[combat turns](../integrations/dnd-combat-turn-artifacts.md) and
|
||||
[scene descriptions](../integrations/dnd-scene-description-artifacts.md).
|
||||
The [enemy-event contract](../integrations/dnd-enemy-event-artifacts.md)
|
||||
defines its durable semantics; [Configuration](../config.md) owns its
|
||||
selectable bindings and validation chains.
|
||||
|
||||
## Focused Verification
|
||||
|
||||
|
||||
@@ -26,9 +26,10 @@ durable schemas. Those responsibilities remain with the module and its
|
||||
|
||||
`PromptKitClient` validates the request target and prompt identity, maps each
|
||||
named material to a PromptKit inline artifact while preserving its origin URI,
|
||||
maps the trimmed request session to PromptKit's direct per-run session field,
|
||||
retains the same value as the `session_id` prompt variable for maintained
|
||||
prompt compatibility, and forwards profile selection. It then creates one
|
||||
passes the supplied request session through to PromptKit's direct per-run
|
||||
session field, retains the same value as the `session_id` prompt variable for
|
||||
maintained prompt compatibility, and forwards profile selection. It does not
|
||||
derive or replace session values; the CLI owns that policy. It then creates one
|
||||
frozen prepared execution, captures its caller-owned credential-redacted
|
||||
details for debug material, and executes that exact snapshot through
|
||||
PromptKit's prepared-execution boundary. The direct field
|
||||
|
||||
@@ -10,11 +10,11 @@ own durable output shapes. Concrete production extensions are covered by
|
||||
## Boundary
|
||||
|
||||
The pipeline framework accepts a resolved composition, registries, shared
|
||||
dependencies, input bytes, and state/debug collaborators. It returns logical
|
||||
output files, normalized artifacts, recorded rejections and warnings, manifest
|
||||
provenance, and checkpoint decisions. The CLI owns process arguments,
|
||||
configuration discovery, physical roots, and placement of returned output
|
||||
files.
|
||||
dependencies, input bytes, a supplied prompt session, and state/debug
|
||||
collaborators. It returns logical output files, normalized artifacts, recorded
|
||||
rejections and warnings, manifest provenance, and checkpoint decisions. The
|
||||
CLI owns process arguments, configuration discovery, session resolution,
|
||||
physical roots, and placement of returned output files.
|
||||
|
||||
The framework has one fixed shape:
|
||||
|
||||
@@ -84,6 +84,10 @@ incompatible producer prevents the consumer step from starting.
|
||||
|
||||
The runner validates its input, installs no-op state collaborators when none
|
||||
were supplied, and serially performs source parsing and chunk-plan selection.
|
||||
It transports the supplied session unchanged to prompt-facing operations and
|
||||
run-manifest metadata; it neither derives a session nor substitutes a parsed
|
||||
source document identifier. The public session contract is owned by the
|
||||
[CLI reference](../cli.md#run).
|
||||
An accepted plan is materialized into source-addressed chunks and passes the
|
||||
configured chunk validators before any lane runs. A chunk rejection is a
|
||||
recorded pipeline outcome: lanes do not start, but the output stage can encode
|
||||
|
||||
@@ -266,16 +266,18 @@ transport-wide cap. Notarius does not add another timeout around PromptKit.
|
||||
The pinned upstream boundary and profile-format links are in
|
||||
[PromptKit Integration](integrations/pkg-promptkit.md).
|
||||
|
||||
Concurrency has two independent layers. Notarius **total_llm** is the
|
||||
application-wide provider-call limit shared by all backends, modules, retries,
|
||||
and validators. PromptKit may impose a narrower admission limit for the
|
||||
selected backend. The effective active-generation bound is the intersection of
|
||||
both limits and can therefore be lower than **total_llm**. Built-in OpenRouter
|
||||
profiles use PromptKit's upstream backend limit; endpoint-only profiles have no
|
||||
PromptKit backend limit and remain bounded by Notarius. For the configured
|
||||
local backend, a zero **concurrency_limit** leaves only the Notarius scheduler
|
||||
as a call limit. A positive value makes the effective active local-generation
|
||||
bound the smaller of **total_llm** and that local limit.
|
||||
Concurrency has two independent layers. Notarius **total_llm** defaults to 16
|
||||
and is the application-wide provider-call limit shared by all backends,
|
||||
modules, retries, and validators. PromptKit may impose a narrower admission
|
||||
limit for the selected backend. The effective active-generation bound is the
|
||||
intersection of the Notarius limit, any PromptKit backend limit, and work made
|
||||
available by the pipeline. Built-in OpenRouter profiles use PromptKit's
|
||||
upstream backend limit; endpoint-only profiles have no PromptKit backend limit
|
||||
and remain bounded by Notarius. For the configured local backend, a zero
|
||||
**concurrency_limit** leaves only the Notarius scheduler as a call limit. A
|
||||
positive value makes the effective active local-generation bound the smaller
|
||||
of **total_llm** and that local limit, so a local limit of four permits no more
|
||||
than four active local generations.
|
||||
|
||||
For a positive local limit, PromptKit owns its default waiting capacity and
|
||||
admission behavior. When a PromptKit backend has admitted all active and queued
|
||||
@@ -289,3 +291,11 @@ under [PromptKit profiles](config.md#promptkit-profiles) and
|
||||
limits and actual provider-call limits are independent. Notarius writes local
|
||||
filesystem state only; remote storage, archival, and retention automation are
|
||||
outside the implemented CLI.
|
||||
|
||||
Every run has an effective prompt session used for provider routing and run
|
||||
provenance. The generated default is stable for the same input module and raw
|
||||
input bytes; use [**--session-id**](cli.md#run) only when intentionally grouping
|
||||
different invocations. Both generated and explicit values can be visible to
|
||||
providers, manifests, checkpoints, and requested debug bundles. Do not put
|
||||
credentials or other secrets in an explicit session identifier; command-line
|
||||
values are not a credential mechanism.
|
||||
|
||||
@@ -7,37 +7,6 @@ not as committed release dates.
|
||||
|
||||
## Near-Term D&D Pipeline
|
||||
|
||||
### Combat Enemy Ledger
|
||||
|
||||
- Add a D&D artifact that identifies enemies faced during combat and supports
|
||||
an end-of-session encounter ledger.
|
||||
- Track each enemy's observed state using a small controlled vocabulary such as
|
||||
`active`, `killed`, `fled`, `captured`, or `incapacitated`, while preserving
|
||||
an explicit unresolved state when the transcript does not establish an
|
||||
outcome.
|
||||
- Preserve the evidence for enemy participation and state changes rather than
|
||||
inferring a terminal outcome from combat ending or an enemy disappearing
|
||||
from the conversation.
|
||||
- Define how repeated mentions, groups of unnamed enemies, summoned or allied
|
||||
creatures, and the same enemy appearing in multiple combats affect identity
|
||||
and ledger entries.
|
||||
- Evaluate whether the ledger should be extracted directly, derived from
|
||||
combat-turn artifacts, or use a sequential pipeline that consumes combat
|
||||
turns and the normalized NPC registry as grounding references.
|
||||
|
||||
### Location Extraction
|
||||
|
||||
- Add a D&D artifact for locations visited by the party or otherwise mentioned
|
||||
in the transcript.
|
||||
- Distinguish observed visits from references, plans, recalled places, and
|
||||
uncertain or inferred locations so a mention alone is not reported as a
|
||||
visit.
|
||||
- Preserve transcript evidence for each visit or mention and reconcile aliases,
|
||||
nested places, and repeated appearances without collapsing distinct
|
||||
locations that share a generic name.
|
||||
- Define how the location artifact should ground later narrative reports and
|
||||
whether future event artifacts should retain canonical location identities.
|
||||
|
||||
### Evaluate Spell Extraction And Normalization
|
||||
|
||||
- Evaluate ordinary extraction retries and the completed normalization path
|
||||
@@ -53,71 +22,6 @@ not as committed release dates.
|
||||
spell, combat, interaction, and scene-description lanes after real-world use.
|
||||
Add more complex chunking only in response to demonstrated failures.
|
||||
|
||||
## Cross-Cutting LLM Runtime
|
||||
|
||||
### Deterministic Prompt Session Identity
|
||||
|
||||
- Replace the source-document-ID default for prompt sessions with one
|
||||
predictable, procedurally generated session ID for the complete
|
||||
source-processing workload.
|
||||
- Preserve an explicit non-empty `--session-id` as the highest-precedence
|
||||
override. Otherwise, derive the default only from the effective input module
|
||||
identity and the exact raw input bytes.
|
||||
- Use a versioned, bounded representation such as
|
||||
`notarius:v1:<sha256(input-module + NUL + raw-input)>`. The exact encoding
|
||||
must fit PromptKit's session length contract and must not embed source
|
||||
content.
|
||||
- Keep the derived session stable across runs, pipelines, selected lanes,
|
||||
ordered steps, retries, resume, recomputation, LLM profiles, reasoning
|
||||
overrides, and output, debug, or cache settings.
|
||||
- Do not include file-backed references, generated references, reference
|
||||
contents, or the composition of a reference bundle in session derivation.
|
||||
References may change between prompt calls within one pipeline without
|
||||
changing routing affinity.
|
||||
- Resolve the authoritative session before checkpoint construction and use the
|
||||
same value for checkpoint runtime identity, every prompt-facing module,
|
||||
PromptKit's direct session field, the compatibility `session_id` prompt
|
||||
variable, run-manifest metadata, and debug metadata.
|
||||
- Keep routing identity separate from cache and checkpoint content identity.
|
||||
Exact prompt prefixes, reference contents, model settings, and other
|
||||
generation-affecting inputs must continue to participate in their existing
|
||||
hashes and checkpoint fingerprints even though they do not change the
|
||||
session.
|
||||
- Treat the generated value as a provider-visible, stable pseudonymous
|
||||
correlation identifier. Do not introduce an installation-specific HMAC or
|
||||
secret unless a concrete multi-tenant or privacy requirement justifies
|
||||
sacrificing deterministic identity across installations.
|
||||
|
||||
### Raise The Default Application-Wide LLM Limit
|
||||
|
||||
- Raise the default `concurrency.total_llm` value from 1 to 16 so ordinary
|
||||
single-backend runs can use PromptKit's expected OpenRouter capacity and
|
||||
lower-capacity local backends without an unnecessarily narrower Notarius
|
||||
limit.
|
||||
- Keep the Notarius application-wide scheduler mandatory and require
|
||||
`total_llm` to remain a positive integer. Do not make the default unlimited:
|
||||
endpoint-only profiles, an unrestricted local backend, injected clients, and
|
||||
aggregate work across several backends may have no narrower PromptKit limit.
|
||||
- Continue defaulting `concurrency.stage_workers.extract` to the effective
|
||||
`total_llm`, making its default 16 as part of the same change. Preserve an
|
||||
explicit lower extract-worker setting when an operator wants less queued or
|
||||
concurrent extraction work.
|
||||
- Define effective provider concurrency as the intersection of the Notarius
|
||||
application-wide limit, the selected PromptKit backend limit when present,
|
||||
and the work made available by stage execution. A Notarius limit of 16 does
|
||||
not narrow a backend already limited to 16, while a local backend limited to
|
||||
4 remains bounded at 4.
|
||||
- Treat the default as an application-wide safety ceiling across profiles,
|
||||
backends, modules, retries, and validators. A run that intentionally needs
|
||||
the combined capacity of several backends may configure a higher
|
||||
`total_llm` and an appropriate extract-worker count explicitly.
|
||||
- Retain the existing configuration and environment override surfaces. Update
|
||||
canonical configuration, operations, and internal documentation together
|
||||
when the default changes.
|
||||
- Reconsider decoupling the extract-worker default from `total_llm` only after
|
||||
mixed-backend workloads demonstrate a need for a high global emergency
|
||||
ceiling with a lower default work-production rate.
|
||||
|
||||
## Shared Normalization And Quality Work
|
||||
|
||||
### Generic LLM-Assisted Deduplication
|
||||
|
||||
@@ -1,586 +0,0 @@
|
||||
# PromptKit v0.5 Implementation Plan
|
||||
|
||||
## Objective
|
||||
|
||||
Implement the target state in
|
||||
[PromptKit v0.5 Integration And LLM Profile Policy](promptkit.md). Each numbered
|
||||
stage is intended to be one implementation prompt for a GPT-5.6-Terra coding
|
||||
agent. Complete stages in order and leave the repository buildable, tested, and
|
||||
internally coherent after every stage.
|
||||
|
||||
Follow [Architecture](../policy/architecture.md),
|
||||
[Testing Policy](../policy/testing.md), and
|
||||
[Documentation Policy](../policy/documentation.md) throughout. Preserve
|
||||
unrelated user changes. Use `apply_patch` for source and documentation edits,
|
||||
run `gofmt` on changed Go files, and add only tests that protect the behaviors
|
||||
and risks assigned to that stage.
|
||||
|
||||
Do not implement the separate deterministic session-ID or default-concurrency
|
||||
roadmap items as part of this plan. Do not perform paid or credentialed LLM
|
||||
calls.
|
||||
|
||||
## Background Summary
|
||||
|
||||
Notarius currently pins PromptKit v0.3.0, calls `Prepare` and then `Run` for one
|
||||
completion, validates profiles through a synthetic prompt, has no application
|
||||
fallback profile source, and accepts LLM profiles only at individual bindings
|
||||
or through the run-wide CLI override. PromptKit v0.5.0 is source-compatible
|
||||
with the current tree; a temporary v0.5.0 module override has already passed
|
||||
`go test ./...`.
|
||||
|
||||
The implementation must nevertheless treat the upstream optional-parameter
|
||||
change as intentional: unset `temperature`, `max_tokens`, and `top_p` remain
|
||||
unset and are omitted from compatible provider requests. Do not restore the old
|
||||
implicit `top_p: 1` default.
|
||||
|
||||
## Stage 1: Upgrade The PromptKit Dependency
|
||||
|
||||
### Goal
|
||||
|
||||
Establish a clean PromptKit v0.5.0 baseline before adopting its new APIs.
|
||||
|
||||
### Work
|
||||
|
||||
- Update `go.mod` and `go.sum` from PromptKit v0.3.0 to v0.5.0 and run
|
||||
`go mod tidy`.
|
||||
- Change the PromptKit built-in profile-catalog marker in
|
||||
`internal/framework/llm/promptkit_profile_fingerprint.go` to identify
|
||||
v0.5.0. This deliberately invalidates LLM checkpoints tied to the prior
|
||||
catalog identity.
|
||||
- Review PromptKit-facing compile errors or test failures against the v0.4.0
|
||||
and v0.5.0 release guides. Do not adopt prepared execution, inspection, or
|
||||
fallback profiles in this stage.
|
||||
- Replace the existing test assertion for one exact built-in fingerprint hash
|
||||
with durable assertions that the fingerprint is deterministic, non-empty,
|
||||
non-secret, and changes when a semantic profile source changes. Do not add a
|
||||
new version-constant or exact-hash change detector.
|
||||
- Update `docs/integrations/pkg-promptkit.md` to pin and link v0.5.0 and state
|
||||
the implemented dependency-level behavior: unset optional sampling controls
|
||||
are provider defaults. Do not document later stages as implemented.
|
||||
- Update any other canonical text that explicitly claims the dependency is
|
||||
v0.3.0, but defer descriptions of unimplemented v0.5 APIs.
|
||||
|
||||
### Tests And Validation
|
||||
|
||||
- `go test ./internal/framework/llm ./internal/cli`
|
||||
- `go test ./...`
|
||||
- `go vet ./...`
|
||||
- `go build ./cmd/notarius`
|
||||
- `rg -n 'promptkit v0\.3\.0|promptkit@v0\.3\.0|PromptKit v0\.3\.0' .`
|
||||
- `git diff --check`
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- The repository directly pins v0.5.0 and all default offline checks pass.
|
||||
- The profile-source fingerprint identifies the new upstream catalog without a
|
||||
brittle literal-hash test.
|
||||
- Current documentation no longer identifies v0.3.0 as the supported version.
|
||||
|
||||
## Stage 2: Execute One Frozen Prepared Snapshot
|
||||
|
||||
### Goal
|
||||
|
||||
Make Notarius debug details and generation use one exact PromptKit preparation.
|
||||
|
||||
### Work
|
||||
|
||||
- Refactor `PromptKitClient.CompleteStructured` to call
|
||||
`PrepareExecution`, immediately defer `Discard`, obtain a caller-owned
|
||||
`Details` value, and execute with `RunPrepared`.
|
||||
- Preserve the existing Notarius request mapping, cancellation precedence,
|
||||
validation classification, raw structured bytes, response decoding,
|
||||
profile recording, usage reporting, and credential redaction.
|
||||
- Ensure every preparation, execution, validation, empty-result, and decode
|
||||
error retains useful Notarius prompt context without exposing prepared handle
|
||||
state or secrets.
|
||||
- Use `errors.As` to obtain `*promptkit.CapacityError` on admission rejection.
|
||||
Preserve `contracts.ErrLLMCapacityExceeded` as the stable classification and
|
||||
add a nonblank backend ID only to safe application-owned diagnostic context.
|
||||
Do not expose `promptkit.CapacityError` outside the LLM adapter.
|
||||
- Update `docs/internal/llm.md` and the implemented-mechanics portion of
|
||||
`docs/integrations/pkg-promptkit.md` to describe the single frozen execution
|
||||
snapshot and structured capacity adaptation.
|
||||
|
||||
### Tests And Validation
|
||||
|
||||
- Adapt existing PromptKit client tests to the prepared-execution path.
|
||||
- Retain or add one behavioral test proving that the debug prompt details match
|
||||
the request actually passed to generation when a backing prompt source could
|
||||
otherwise change between independent preparations. Test the resulting
|
||||
snapshot consistency, not a private helper call count.
|
||||
- Retain capacity tests proving `errors.Is` reaches
|
||||
`contracts.ErrLLMCapacityExceeded`, the selected backend can appear in safe
|
||||
diagnostic context, and provider calls are not made after rejected
|
||||
admission.
|
||||
- Run `go test ./internal/framework/llm` and
|
||||
`go test -race ./internal/framework/llm`.
|
||||
- Run `go test ./...` and `git diff --check`.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- `CompleteStructured` no longer calls independent `Prepare` and `Run`
|
||||
operations for one request.
|
||||
- Debug prompt material and generation result originate from the same frozen
|
||||
PromptKit snapshot.
|
||||
- Capacity remains a provider-neutral Notarius error classification.
|
||||
|
||||
## Stage 3: Replace Synthetic Profile Validation With Inspection
|
||||
|
||||
### Goal
|
||||
|
||||
Validate profiles through PromptKit's exact profile-inspection boundary and
|
||||
centralize engine profile-source construction.
|
||||
|
||||
### Work
|
||||
|
||||
- Introduce a small provider-adapter-owned profile inspection or validation
|
||||
function in `internal/framework/llm`. Its public internal signature must use
|
||||
Notarius-owned configuration and result/error types rather than returning
|
||||
PromptKit types to the CLI.
|
||||
- Share the code that applies `profile_dir`, `profile_file`, and registered
|
||||
backend options between the production PromptKit engine and the inspection
|
||||
engine. Preserve the mutual-exclusion and local-backend rules.
|
||||
- Change CLI explicit-profile preflight to use `Engine.InspectProfile` through
|
||||
that LLM boundary.
|
||||
- Remove `profileCheckPromptID`, `profileCheckPromptFS`, the `testing/fstest`
|
||||
production dependency, and the synthetic `Prepare` request.
|
||||
- Preserve distinct, useful errors for an absent profile, invalid profile,
|
||||
unknown backend registration, cancellation, and invalid profile source.
|
||||
- Do not require `api_key_env` to be populated during configuration validation.
|
||||
Inspection may report credential requirements internally, but actual
|
||||
preparation remains responsible for credential availability before a model
|
||||
call.
|
||||
- Update current-behavior sections in `docs/internal/cli.md` and
|
||||
`docs/internal/llm.md`. Keep field definitions in `docs/config.md`.
|
||||
|
||||
### Tests And Validation
|
||||
|
||||
- Replace synthetic-prompt tests with profile inspection tests covering:
|
||||
configured local backend success; missing local backend failure; absent
|
||||
profile; malformed profile; and an otherwise valid profile whose credential
|
||||
environment variable is intentionally unset.
|
||||
- Prove validation performs no provider HTTP call and remains offline.
|
||||
- Run `go test ./internal/framework/llm ./internal/cli` and `go test ./...`.
|
||||
- Run `git diff --check`.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- No production synthetic profile-check prompt remains.
|
||||
- Profile validation uses the same ordinary profile source and backend
|
||||
registrations as execution.
|
||||
- Configuration validation succeeds for structurally valid profiles without
|
||||
reading credential values.
|
||||
|
||||
## Stage 4: Add Application Fallback Profile Asset Plumbing
|
||||
|
||||
### Goal
|
||||
|
||||
Allow module families to register application-owned fallback profile YAML
|
||||
without placing domain policy in generic LLM code.
|
||||
|
||||
### Work
|
||||
|
||||
- Extend `internal/framework/llm.AssetRegistry` with a separate fallback
|
||||
profile source collection, registration method, flattened filesystem, and
|
||||
safe content digest.
|
||||
- Reuse the existing asset-source path validation and flattening behavior where
|
||||
appropriate. Reject invalid roots, unreadable assets, and duplicate flattened
|
||||
paths. Do not parse PromptKit profile YAML in Notarius.
|
||||
- Add `promptkit.WithFallbackProfileFS` to production engine options only when
|
||||
at least one fallback profile source is registered.
|
||||
- Supply the identical assembled fallback source to the profile-inspection
|
||||
engine. Adjust CLI composition so pipeline-aware profile validation can use
|
||||
the production LLM asset registry without exposing PromptKit types.
|
||||
- Extend profile-source checkpoint identity to include the exact fallback
|
||||
profile asset digest in addition to the PromptKit catalog marker and operator
|
||||
source. Keep the resulting fingerprint hash-only and path/content/credential
|
||||
free.
|
||||
- Keep operator source precedence owned by PromptKit. Do not implement profile
|
||||
merging or duplicate PromptKit source resolution in Notarius.
|
||||
- Update `docs/internal/llm.md` only for the new implemented generic asset and
|
||||
fingerprint mechanics. No domain fallback exists until Stage 5.
|
||||
|
||||
### Tests And Validation
|
||||
|
||||
- Add focused AssetRegistry tests for successful flattening, invalid roots,
|
||||
duplicate paths, and hash changes when fallback bytes change.
|
||||
- Add adapter-level tests showing that the fallback filesystem reaches both
|
||||
execution construction and inspection construction.
|
||||
- Extend checkpoint tests to prove fallback content changes profile-source
|
||||
identity without exposing raw YAML or paths. Use relational comparisons, not
|
||||
a fixed hash literal.
|
||||
- Run `go test ./internal/framework/llm ./internal/cli` and `go test ./...`.
|
||||
- Run `git diff --check`.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- Generic plumbing can carry application fallback profiles while remaining
|
||||
unaware of D&D IDs or model settings.
|
||||
- Inspection, execution, and checkpoint identity use the same fallback asset
|
||||
source.
|
||||
|
||||
## Stage 5: Adopt The D&D `dnd-extraction` Fallback
|
||||
|
||||
### Goal
|
||||
|
||||
Give the D&D module family one stable embedded workload profile that operators
|
||||
can replace.
|
||||
|
||||
### Work
|
||||
|
||||
- Add a D&D-owned embedded PromptKit profile asset with ID `dnd-extraction`
|
||||
under `internal/modules/dnd`. Use the exact baseline defined in
|
||||
`promptkit.md`: OpenRouter, `openai/gpt-5.6-luna`, no explicit reasoning
|
||||
effort, a 240-second timeout, flex service tier, and no selected temperature,
|
||||
token limit, or `top_p`. The omitted reasoning value intentionally allows
|
||||
OpenAI's backend to apply its `medium` default.
|
||||
- Register the profile filesystem from the D&D registrar through the generic
|
||||
fallback profile asset boundary. Keep D&D policy out of
|
||||
`internal/framework/llm` and the CLI composition root.
|
||||
- Change every maintained D&D LLM prompt definition—including scene chunking,
|
||||
all D&D extractors, and NPC normalization—from the model-named default to
|
||||
`default_profile: dnd-extraction`.
|
||||
- Add an integration-level profile-resolution test proving that:
|
||||
- the fallback resolves when no operator source defines the ID;
|
||||
- a valid operator profile with the same ID wins completely; and
|
||||
- an invalid matching operator profile fails rather than falling through.
|
||||
- Test through Notarius's assembled production assets and PromptKit boundary;
|
||||
do not duplicate every upstream source-precedence case.
|
||||
- Update the implemented profile ownership and prompt-default behavior in
|
||||
`docs/internal/dnd.md`, `docs/internal/llm.md`, and
|
||||
`docs/integrations/pkg-promptkit.md`. Defer the complete operator walkthrough
|
||||
and examples to Stage 10.
|
||||
|
||||
### Tests And Validation
|
||||
|
||||
- Run focused D&D prompt preparation tests and the production composition
|
||||
tests.
|
||||
- Run `go test ./internal/modules/dnd/... ./internal/framework/llm
|
||||
./internal/cli`.
|
||||
- Run `go test ./...`.
|
||||
- Verify `rg -n 'default_profile: gemini-2-flash' internal/modules/dnd`
|
||||
returns no matches.
|
||||
- Run `git diff --check`.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- All maintained D&D prompts use the application-owned logical profile ID.
|
||||
- The fallback works without an operator profile and remains authoritatively
|
||||
overridable by a matching valid operator definition.
|
||||
|
||||
## Stage 6: Introduce Module Execution-Class Metadata
|
||||
|
||||
### Goal
|
||||
|
||||
Make each production module's ability to use an LLM statically discoverable
|
||||
without yet changing profile inheritance.
|
||||
|
||||
### Work
|
||||
|
||||
- Add `ExecutionClass contracts.ExecutionClass` to `pipeline.ModuleSpec` and
|
||||
preserve it through normalization, cloning, catalogs, registries, JSON/debug
|
||||
views, and lookup helpers.
|
||||
- In this transitional stage only, allow an omitted execution class to
|
||||
normalize to deterministic so existing test-only fixtures can be migrated in
|
||||
Stage 7 without breaking the repository midway.
|
||||
- Explicitly classify every production module:
|
||||
- D&D scene chunking, every D&D extractor, and D&D NPC normalization as
|
||||
`llm_backed`;
|
||||
- all other current production input, chunk, merge, normalize, and output
|
||||
modules as `deterministic`.
|
||||
- Update production module specification tests and production catalog tests to
|
||||
assert the semantic class alongside stage, artifact kind, and capabilities.
|
||||
- Add catalog lookup support needed by later resolution to retrieve a selected
|
||||
module's execution class by stage and key without constructing it.
|
||||
- Do not implement pipeline-level profile inheritance or reject deterministic
|
||||
profiles yet.
|
||||
- Update `docs/internal/modules.md` and `docs/internal/dnd.md` to identify
|
||||
execution class as registered module metadata, while noting only implemented
|
||||
uses.
|
||||
|
||||
### Tests And Validation
|
||||
|
||||
- Run module registration/spec tests across generic, Seriatim, and D&D
|
||||
families.
|
||||
- Run `go test ./internal/framework/pipeline ./internal/modules/...`.
|
||||
- Run `go test ./...` and `git diff --check`.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- Every production module has an explicit correct execution class.
|
||||
- Catalog consumers can retrieve that class without a concrete module
|
||||
instance.
|
||||
- Test-only omitted classes remain the only temporary compatibility behavior.
|
||||
|
||||
## Stage 7: Enforce Execution Metadata And Remove Runtime Probing
|
||||
|
||||
### Goal
|
||||
|
||||
Finish the execution-class contract so missing metadata cannot cause future
|
||||
profile drift.
|
||||
|
||||
### Work
|
||||
|
||||
- Update every framework, CLI, and integration test module specification to
|
||||
declare an explicit execution class appropriate to the fake behavior.
|
||||
- Change module-spec validation so an empty or unsupported execution class is a
|
||||
registration error. Remove the transitional deterministic default from
|
||||
Stage 6.
|
||||
- Replace the chunk runner's special `ChunkExecutionClassProvider` probe with
|
||||
specification-derived behavior. Remove the now-redundant provider interface,
|
||||
implementation methods, and tests when they have no remaining consumer.
|
||||
- Ensure chunk producer provenance remains unchanged: it records a non-empty
|
||||
effective binding profile for an LLM-backed chunker, while a deterministic
|
||||
chunker records no profile. A profile selected only through the prompt
|
||||
default remains represented by PromptKit's actual-profile manifest rather
|
||||
than being invented as an explicit chunk binding.
|
||||
- Review helper constructors and fixtures for opportunities to set execution
|
||||
class once without obscuring the class under test. Do not introduce an
|
||||
elaborate test-spec framework.
|
||||
- Update internal documentation if the removal changes any described runtime
|
||||
mechanics.
|
||||
|
||||
### Tests And Validation
|
||||
|
||||
- Add or retain focused registration tests for missing and invalid execution
|
||||
classes.
|
||||
- Retain chunk-plan provenance tests for LLM-backed and deterministic
|
||||
chunkers.
|
||||
- Run `go test ./internal/framework/pipeline ./internal/modules/...`.
|
||||
- Run `go test ./...`, `go vet ./...`, and `git diff --check`.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- No registered module specification relies on an implicit execution class.
|
||||
- Pipeline metadata, not a concrete runtime type assertion, owns module
|
||||
execution classification.
|
||||
|
||||
## Stage 8: Resolve Programmatic Pipeline Profile Defaults
|
||||
|
||||
### Goal
|
||||
|
||||
Implement profile inheritance and precedence inside the pipeline resolver
|
||||
before exposing the field through YAML configuration.
|
||||
|
||||
### Work
|
||||
|
||||
- Add an optional trimmed `LLMProfile` field to
|
||||
`pipeline.PipelineProfile`. Add a non-empty runtime override field to
|
||||
`pipeline.ResolveOptions` so all precedence decisions occur in the resolver
|
||||
rather than through pre-resolution mutation.
|
||||
- After module selection, `--only` filtering, default validator-chain
|
||||
selection, and validator compatibility resolution, apply effective profiles
|
||||
to every selected input, chunk, extract, merge, normalize, output, and
|
||||
validator binding according to the precedence in `promptkit.md`.
|
||||
- Apply profiles only when the selected module or validator execution class is
|
||||
`llm_backed`.
|
||||
- Reject a binding-specific `llm_profile` on any deterministic module or
|
||||
validator. Do not reject or inspect an unused pipeline default when no
|
||||
selected LLM-backed binding consumes it.
|
||||
- Leave an LLM-backed binding empty when no CLI, binding, or pipeline profile is
|
||||
selected so PromptKit can use the prompt's `default_profile`.
|
||||
- Store the effective values on resolved bindings before digest construction.
|
||||
Do not add a second inheritance decision to execution.
|
||||
- Ensure semantically equivalent repeated binding profiles and one inherited
|
||||
default produce the same resolved pipeline digest. Ensure any changed
|
||||
effective profile changes the digest.
|
||||
- Do not modify file configuration or CLI parsing in this stage.
|
||||
|
||||
### Tests And Validation
|
||||
|
||||
- Add pipeline package tests for the complete precedence matrix:
|
||||
runtime override; binding-specific exception; pipeline default; prompt
|
||||
fallback; and deterministic bindings.
|
||||
- Cover default and explicitly configured validator chains, all relevant stage
|
||||
categories, `--only` lane selection, unused defaults, deterministic-profile
|
||||
rejection, and semantic digest equivalence.
|
||||
- Prefer table-driven package-level tests over assertions on private traversal
|
||||
helpers.
|
||||
- Run `go test ./internal/framework/pipeline` and `go test ./...`.
|
||||
- Run `git diff --check`.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- Programmatic pipelines resolve one canonical effective profile policy.
|
||||
- Only LLM-backed resolved bindings can contain a profile.
|
||||
- Runtime override, binding, pipeline, and prompt precedence is unambiguous and
|
||||
digest-stable.
|
||||
|
||||
## Stage 9: Expose Pipeline Defaults Through Configuration And CLI
|
||||
|
||||
### Goal
|
||||
|
||||
Make the profile-default workflow available to operators while preserving
|
||||
validation and override behavior.
|
||||
|
||||
### Work
|
||||
|
||||
- Add optional `pipelines.<id>.llm_profile` support to the version 4 file
|
||||
configuration model. Use presence-aware decoding so an explicitly set blank
|
||||
value is rejected, while omission remains valid.
|
||||
- Preserve the field through file application, configuration cloning,
|
||||
effective configuration, and programmatic profile copies without aliasing or
|
||||
trimming drift.
|
||||
- Remove `applyLLMProfileOverride`. Pass the CLI override through the resolver's
|
||||
runtime-override input so deterministic bindings are never populated.
|
||||
- Update effective profile-ID collection to cover every selected LLM-backed
|
||||
module stage and LLM-backed validator, including future LLM-backed input and
|
||||
output modules. Do not inspect deterministic or unselected profiles.
|
||||
- Ensure `run`, `config validate --pipeline`, resume/checkpoint identity, and
|
||||
relevant dry preflight paths all use the same resolved effective profiles.
|
||||
- Preserve `--llm-profile` as the highest-precedence non-empty run-wide
|
||||
override and preserve binding-specific profiles as exceptions when no CLI
|
||||
override is present.
|
||||
- Do not increment the configuration version.
|
||||
- Update current configuration and CLI contracts in `docs/config.md` and
|
||||
`docs/cli.md` in the same stage. Link to operations for the deployment
|
||||
workflow rather than duplicating it prematurely.
|
||||
|
||||
### Tests And Validation
|
||||
|
||||
- Add file-config tests for omission, trimming, explicit blank rejection,
|
||||
unknown-key behavior, cloning, and round-trip application.
|
||||
- Add effective-config and CLI contract tests for precedence, LLM-only
|
||||
application, inherited-profile inspection failure before factory execution,
|
||||
`--only`, and digest changes.
|
||||
- Retain offline operation and do not require credentials for
|
||||
`config validate --pipeline`.
|
||||
- Run `go test ./internal/core/config ./internal/framework/pipeline
|
||||
./internal/cli`.
|
||||
- Run `go test ./...`, `go vet ./...`, and `git diff --check`.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- Operators can select `dnd-extraction` once per pipeline.
|
||||
- Configuration and CLI paths share the resolver's precedence policy.
|
||||
- Unknown effective profiles fail preflight, while deterministic and unused
|
||||
profiles do not cause spurious inspection.
|
||||
|
||||
## Stage 10: Complete Operator Documentation, Examples, And Decision Record
|
||||
|
||||
### Goal
|
||||
|
||||
Make the implemented workflow understandable, copyable, and maintainable
|
||||
without duplicating canonical facts.
|
||||
|
||||
### Work
|
||||
|
||||
- Create an ADR using the next sequential number for the durable decision to
|
||||
use workload-oriented pipeline defaults with operator-overridable application
|
||||
fallback profiles. Record context, decision, alternatives, and consequences;
|
||||
do not turn the ADR into a field reference or implementation log.
|
||||
- Complete `docs/config.md` as the canonical owner of profile-source fields,
|
||||
`pipelines.<id>.llm_profile`, validation, and precedence.
|
||||
- Complete `docs/operations.md` with an operator workflow that distinguishes
|
||||
Notarius embedded prompts, Notarius fallback profiles, PromptKit built-ins,
|
||||
and deployment filesystem profiles. Include production/development/local use
|
||||
of the same `dnd-extraction` ID, credential handling, absolute-path guidance,
|
||||
and the fact that current relative profile paths use the process working
|
||||
directory rather than the configuration file's directory.
|
||||
- Complete `docs/integrations/pkg-promptkit.md` with the v0.5.0 boundary,
|
||||
prepared execution, inspection, fallback and ordinary source precedence,
|
||||
optional provider controls, capacity adaptation, and compatibility policy.
|
||||
- Update `docs/internal/configuration.md`, `docs/internal/pipeline.md`,
|
||||
`docs/internal/cli.md`, `docs/internal/llm.md`, `docs/internal/modules.md`, and
|
||||
`docs/internal/dnd.md` only for their owned implementation details. Link to
|
||||
canonical configuration, operations, and upstream format contracts rather
|
||||
than restating them.
|
||||
- Keep exactly the existing two D&D configuration examples. Add
|
||||
`llm_profile: dnd-extraction` to the minimal and complete pipelines and remove
|
||||
the now-redundant model-named binding override from the complete example.
|
||||
- Add one secret-free maintained operator profile at
|
||||
`examples/profiles/dnd-extraction.yml`. It should be a complete valid profile
|
||||
for the same logical ID and may mirror the embedded baseline; its purpose is
|
||||
to demonstrate file ownership and format, not claim automatic environment
|
||||
detection. Link it from the configuration and operations documentation.
|
||||
- If the complete example selects the external profile file, use a path that
|
||||
is valid for the documented repository-root invocation and explicitly note
|
||||
the working-directory rule. Keep the minimal example dependent only on the
|
||||
embedded fallback.
|
||||
- Add or extend maintained-example validation so both configuration examples
|
||||
and the profile YAML are checked without generation or credentials.
|
||||
- Remove the now-implemented `Pipeline-Level LLM Profile Defaults` section from
|
||||
`docs/roadmap/future.md`. Preserve the unrelated deterministic session and
|
||||
concurrency items.
|
||||
- Do not delete `promptkit.md` or this implementation plan during the feature
|
||||
implementation; retire them only after post-implementation review.
|
||||
|
||||
### Tests And Validation
|
||||
|
||||
- Run maintained example/configuration tests and relevant CLI help/parser
|
||||
tests.
|
||||
- Run `go test ./...`.
|
||||
- Run `rg -n 'gemini-2-flash' examples docs` and review every remaining match
|
||||
for intentional model-policy or historical context.
|
||||
- Run `rg -n 'v0\.3\.0|profileCheckPrompt|applyLLMProfileOverride' .` and resolve
|
||||
stale production or current-documentation matches.
|
||||
- Verify all new links and `git diff --check`.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- Every current fact has one canonical documentation owner.
|
||||
- Operators can distinguish and deploy all profile layers without reading Go
|
||||
source.
|
||||
- Both maintained configurations and the maintained external profile are valid,
|
||||
secret-free, and tested offline.
|
||||
- Implemented profile work no longer remains in `future.md`.
|
||||
|
||||
## Stage 11: Final Verification And Quality Review
|
||||
|
||||
### Goal
|
||||
|
||||
Verify the complete migration as one integrated change and correct only defects
|
||||
or omissions found during that review.
|
||||
|
||||
### Work
|
||||
|
||||
- Review the final diff against every acceptance criterion in `promptkit.md`.
|
||||
- Confirm provider-specific PromptKit types remain inside the LLM integration
|
||||
boundary and D&D policy remains inside the D&D module family.
|
||||
- Confirm execution and inspection receive identical ordinary, fallback, and
|
||||
backend configuration.
|
||||
- Confirm no paths, profile YAML, endpoints, credentials, or prepared handle
|
||||
state leak into fingerprints or ordinary diagnostics.
|
||||
- Confirm all production module specs have explicit correct execution classes
|
||||
and every resolved deterministic binding is profile-free.
|
||||
- Confirm prompt default, pipeline default, binding override, and CLI override
|
||||
behavior through representative assembled configurations.
|
||||
- Review tests for redundancy and remove obsolete synthetic-prompt,
|
||||
runtime-probe, exact-hash, or duplicated upstream-behavior tests superseded by
|
||||
stronger contract tests.
|
||||
- Perform an optional manual D&D quality comparison if credentials and an
|
||||
evaluation transcript are deliberately supplied. Record no private input or
|
||||
credential material, and do not make this comparison a completion gate.
|
||||
|
||||
### Validation Commands
|
||||
|
||||
```sh
|
||||
gofmt -w <changed-go-files>
|
||||
go test ./...
|
||||
go test -race ./internal/framework/llm ./internal/core/config ./internal/framework/pipeline ./internal/cli
|
||||
go vet ./...
|
||||
go build ./cmd/notarius
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Also run focused stale-contract searches:
|
||||
|
||||
```sh
|
||||
rg -n 'gitea.maximumdirect.net/eric/promptkit v0\.3\.0|PromptKit v0\.3\.0' .
|
||||
rg -n 'default_profile: gemini-2-flash|profileCheckPrompt|applyLLMProfileOverride' internal docs examples
|
||||
```
|
||||
|
||||
Review any matches rather than deleting intentional historical references
|
||||
blindly.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- All automated checks pass offline and without real credentials.
|
||||
- The implemented behavior matches `promptkit.md` with no known architecture,
|
||||
provenance, checkpoint, profile-precedence, or documentation gap.
|
||||
- Any optional live evaluation is clearly separate from correctness testing.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None. The roadmap decisions are sufficient to implement every stage without an
|
||||
additional product or architecture choice.
|
||||
@@ -1,318 +0,0 @@
|
||||
# PromptKit v0.5 Integration And LLM Profile Policy
|
||||
|
||||
## Purpose
|
||||
|
||||
This roadmap defines the target state for upgrading Notarius from PromptKit
|
||||
v0.3.0 to v0.5.0 and adopting the upstream runtime and profile facilities that
|
||||
directly improve Notarius. It also defines the application policy for stable,
|
||||
domain-oriented LLM profile names, operator overrides, pipeline inheritance,
|
||||
profile validation, provider defaults, checkpoint identity, and documentation.
|
||||
|
||||
The ordered work needed to reach this state belongs in
|
||||
[the implementation plan](implementation.md). Current behavior remains defined
|
||||
by the canonical documentation outside `docs/roadmap/` until the corresponding
|
||||
work is implemented.
|
||||
|
||||
## Background
|
||||
|
||||
Notarius currently pins PromptKit v0.3.0. Its adapter prepares a request once
|
||||
for debug material and then independently runs the original request, causing
|
||||
PromptKit to prepare the same logical call a second time. The CLI validates an
|
||||
explicit profile by preparing a synthetic prompt. PromptKit profile selection
|
||||
can be repeated on individual module bindings or replaced for one invocation
|
||||
with `--llm-profile`, but a configured pipeline cannot yet declare one inherited
|
||||
profile policy.
|
||||
|
||||
PromptKit v0.4.0 and v0.5.0 add the upstream boundaries needed to improve these
|
||||
areas:
|
||||
|
||||
- [v0.4.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.5.0/docs/releases/v0.4.0.md)
|
||||
adds opaque prepared executions, exact profile and prompt inspection, and a
|
||||
typed backend-capacity error;
|
||||
- [v0.5.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.5.0/docs/releases/v0.5.0.md)
|
||||
adds application fallback profile filesystems and stops sending unset
|
||||
optional sampling controls as framework-selected provider values; and
|
||||
- the [v0.5.0 format contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.5.0/docs/formats.md)
|
||||
defines the resulting profile-source and execution-setting precedence.
|
||||
|
||||
A source-compatibility test of the current Notarius repository against
|
||||
PromptKit v0.5.0 completed successfully. The work is therefore primarily an
|
||||
intentional runtime and configuration migration rather than a repair for a
|
||||
breaking Go API change.
|
||||
|
||||
## Goals
|
||||
|
||||
- Pin and document PromptKit v0.5.0 as Notarius's supported upstream contract.
|
||||
- Execute the exact prepared request snapshot whose safe details are recorded
|
||||
in Notarius debug material.
|
||||
- Validate configured PromptKit profiles through the upstream inspection API
|
||||
without synthetic prompts, provider calls, or credential-value access.
|
||||
- Give Notarius an application-owned, operator-overridable
|
||||
`dnd-extraction` profile fallback.
|
||||
- Let a pipeline choose one default LLM profile without repeating that ID on
|
||||
every LLM-backed binding.
|
||||
- Apply profile inheritance and run-wide overrides only where the resolved
|
||||
module or validator can use an LLM.
|
||||
- Preserve accurate checkpoint invalidation, effective profile provenance,
|
||||
redaction, cancellation, concurrency, and provider-neutral module contracts.
|
||||
- Provide operators with one clear deployment pattern for production,
|
||||
development, and local profile definitions.
|
||||
|
||||
## Target End State
|
||||
|
||||
### PromptKit Runtime Boundary
|
||||
|
||||
Notarius depends on PromptKit v0.5.0 and uses its public APIs rather than
|
||||
reimplementing source or execution resolution.
|
||||
|
||||
For each structured completion, the adapter:
|
||||
|
||||
1. builds one PromptKit run request from the provider-neutral Notarius request;
|
||||
2. calls `PrepareExecution` once;
|
||||
3. immediately arranges an idempotent `Discard` for every unexecuted handle;
|
||||
4. obtains credential-redacted `Details` for debug and response metadata; and
|
||||
5. calls `RunPrepared` so generation uses that exact frozen snapshot.
|
||||
|
||||
The debug prompt and successful result therefore describe the same selected
|
||||
profile, rendered messages, input bytes, session, output contract, and effective
|
||||
settings even when a filesystem-backed source changes concurrently. PromptKit
|
||||
handle types remain private to `internal/framework/llm`.
|
||||
|
||||
PromptKit admission failures continue to match Notarius's provider-neutral
|
||||
`ErrLLMCapacityExceeded` contract. When PromptKit supplies a `CapacityError`,
|
||||
the adapter obtains the normalized backend ID through `errors.As` and may add it
|
||||
to safe application-owned diagnostics without parsing upstream error wording.
|
||||
The backend ID does not become a provider-specific module contract.
|
||||
|
||||
### Optional Provider Controls
|
||||
|
||||
Notarius accepts PromptKit v0.5.0's new behavior for `temperature`,
|
||||
`max_tokens`, and `top_p`: an unset setting is omitted from compatible provider
|
||||
requests and the provider chooses its own default. Notarius does not restore
|
||||
PromptKit's former implicit `top_p: 1` value globally.
|
||||
|
||||
An operator who requires a particular value specifies it in the selected
|
||||
PromptKit profile. The application fallback described below intentionally
|
||||
leaves these controls unset. A human-reviewed D&D extraction comparison should
|
||||
be performed after the upgrade, but paid or nondeterministic model output is
|
||||
not part of the default automated test suite.
|
||||
|
||||
### Profile Inspection
|
||||
|
||||
Pipeline-aware configuration validation uses `Engine.InspectProfile` for every
|
||||
effective explicit profile ID. It verifies that the profile exists, parses and
|
||||
validates, resolves its backend and target, and is compatible with the engine's
|
||||
registered backends. It does not create a synthetic prompt, load prompt inputs,
|
||||
contact a provider, or require credential values to exist in the validation
|
||||
process environment.
|
||||
|
||||
Credential availability is execution-time state. PromptKit preparation still
|
||||
enforces the selected profile's credential contract before generation. This
|
||||
keeps `notarius config validate` useful in build and deployment validation
|
||||
environments where secrets are deliberately absent.
|
||||
|
||||
PromptKit construction for inspection and execution uses one shared internal
|
||||
profile-source and backend-option path. The CLI does not expose PromptKit public
|
||||
types across the Notarius LLM boundary merely to perform inspection.
|
||||
|
||||
`InspectPrompt` is not adopted merely because it exists. It remains available
|
||||
for a later, separately defined module-to-prompt interface preflight if a
|
||||
concrete validation requirement justifies that additional contract.
|
||||
|
||||
### Application And Operator Profile Sources
|
||||
|
||||
Notarius embeds one ordinary PromptKit YAML profile with the stable ID
|
||||
`dnd-extraction`. It is an application fallback registered through
|
||||
`WithFallbackProfileFS`, is owned by the D&D module family, and initially
|
||||
preserves the current effective D&D baseline:
|
||||
|
||||
- backend: PromptKit's built-in `openrouter` backend;
|
||||
- model: `openai/gpt-5.6-luna`;
|
||||
- reasoning effort: unset, allowing OpenAI's backend to apply its default of
|
||||
`medium`;
|
||||
- generation timeout: 240 seconds;
|
||||
- service tier: `flex`; and
|
||||
- no application-selected `temperature`, `max_tokens`, or `top_p`.
|
||||
|
||||
All maintained D&D LLM prompt definitions use `dnd-extraction` as their
|
||||
`default_profile`. The ID communicates workload intent rather than a provider,
|
||||
model, or environment. Changing the embedded fallback is an intentional
|
||||
Notarius execution-policy change and participates in checkpoint identity.
|
||||
|
||||
Effective profile definitions resolve in PromptKit's order:
|
||||
|
||||
1. programmatic in-memory profiles used by tests or explicit consumers;
|
||||
2. the operator source configured by `promptkit.profile_file` or
|
||||
`promptkit.profile_dir`;
|
||||
3. the Notarius application fallback source; and
|
||||
4. PromptKit's embedded built-in catalog.
|
||||
|
||||
Only an absent ID falls through to the next source. A matching profile is a
|
||||
complete definition: fields are not merged with a lower-precedence definition,
|
||||
and a malformed matching operator profile fails rather than silently selecting
|
||||
the application fallback.
|
||||
|
||||
Production, development, and local deployments should normally provide
|
||||
different complete definitions for the same `dnd-extraction` ID. An operator
|
||||
source is optional because the application fallback keeps the maintained D&D
|
||||
workflow usable, but a deployment that needs an intentional model or backend
|
||||
policy should configure its own definition.
|
||||
|
||||
### Domain Ownership And Asset Assembly
|
||||
|
||||
The D&D fallback profile remains under `internal/modules/dnd` and is registered
|
||||
by the D&D registrar, consistent with ADR-0004. Generic LLM plumbing knows how
|
||||
to collect and flatten application fallback profile filesystems but contains no
|
||||
D&D model or policy knowledge.
|
||||
|
||||
The shared asset registry detects invalid roots, unreadable sources, and
|
||||
duplicate flattened paths. PromptKit remains responsible for strict profile
|
||||
YAML parsing, duplicate profile-ID detection, source precedence, and effective
|
||||
target resolution. The same assembled fallback source is supplied to runtime
|
||||
execution and CLI profile inspection.
|
||||
|
||||
### Explicit Module Execution Metadata
|
||||
|
||||
Every registered input, chunk, extract, merge, normalize, and output module
|
||||
declares one required execution class: `deterministic` or `llm_backed`.
|
||||
Validator registrations continue to declare the same distinction through their
|
||||
validator specifications.
|
||||
|
||||
The registered specification is authoritative for configuration resolution.
|
||||
Current production classifications are:
|
||||
|
||||
- the D&D scene chunker, all D&D extractors, and the D&D NPC normalizer are
|
||||
LLM-backed;
|
||||
- the Seriatim input adapter, generic chunker, all current mergers, all other
|
||||
current normalizers, and the JSON output encoder are deterministic; and
|
||||
- current validators retain their declared classifications.
|
||||
|
||||
Missing or unsupported execution metadata is a registration error. Explicitly
|
||||
assigning `llm_profile` to a deterministic module or validator is a pipeline
|
||||
resolution error. The framework does not infer execution class by inspecting
|
||||
domain package names or concrete implementation types at runtime.
|
||||
|
||||
The module specification replaces the chunk runner's special runtime
|
||||
execution-class probe. Effective resolved bindings already express the result:
|
||||
only LLM-backed bindings may retain a non-empty profile.
|
||||
|
||||
### Pipeline-Level Profile Default
|
||||
|
||||
Configuration version 4 gains one optional non-empty pipeline field:
|
||||
|
||||
```yaml
|
||||
pipelines:
|
||||
dnd-session:
|
||||
llm_profile: dnd-extraction
|
||||
```
|
||||
|
||||
No configuration-version increment is required because the field is additive
|
||||
and existing files remain valid. An explicitly present blank value is invalid.
|
||||
|
||||
For every selected LLM-backed module and validator, the effective profile uses
|
||||
this precedence:
|
||||
|
||||
1. non-empty run-wide `--llm-profile` override;
|
||||
2. binding-specific `llm_profile`;
|
||||
3. pipeline-level `llm_profile`; and
|
||||
4. the prompt definition's `default_profile`, represented by an empty effective
|
||||
Notarius binding profile.
|
||||
|
||||
The run-wide override and inherited pipeline default never attach to a
|
||||
deterministic binding. Binding-specific exceptions remain available when one
|
||||
operation needs a different cost, latency, quality, backend, or reasoning
|
||||
policy.
|
||||
|
||||
Inheritance is resolved after module and validator selection, including
|
||||
`--only` lane filtering, but before effective-pipeline validation, digest
|
||||
construction, explicit-profile inspection, checkpoint construction,
|
||||
preparation, execution, or provenance capture. Only profiles used by selected
|
||||
LLM-backed bindings are inspected. An unused pipeline default in a pipeline
|
||||
with no selected LLM-backed work does not require an otherwise unused profile
|
||||
to exist.
|
||||
|
||||
The resolved pipeline contains effective binding profiles rather than a second
|
||||
runtime inheritance mechanism. Two pipelines that differ only by spelling the
|
||||
same effective policy once as a pipeline default and once on every LLM-backed
|
||||
binding have the same semantic resolved digest. Changing an effective profile
|
||||
changes the digest and applicable checkpoint identity.
|
||||
|
||||
### Provenance And Checkpoints
|
||||
|
||||
The PromptKit profile-source checkpoint fingerprint covers:
|
||||
|
||||
- the PromptKit v0.5.0 built-in profile catalog identity;
|
||||
- exact application fallback profile asset content; and
|
||||
- exact configured operator profile YAML content, when present.
|
||||
|
||||
The existing local-backend target fingerprint remains separate and continues
|
||||
to exclude scheduling-only concurrency limits. Fingerprints contain hashes and
|
||||
stable markers, not profile contents, filesystem paths, endpoints, credentials,
|
||||
or other secrets.
|
||||
|
||||
Changing the PromptKit version, application fallback, operator profile, or
|
||||
effective pipeline profile makes incompatible LLM checkpoints ineligible for
|
||||
reuse. The dependency upgrade is expected to invalidate checkpoints produced
|
||||
under v0.3.0.
|
||||
|
||||
Successful run manifests continue to record only profiles actually selected by
|
||||
PromptKit, including their effective model, backend, and reasoning metadata.
|
||||
Debug output reports the same effective execution snapshot used for generation.
|
||||
|
||||
### Operator Documentation And Examples
|
||||
|
||||
Canonical documentation clearly distinguishes:
|
||||
|
||||
- Notarius prompt and schema assets embedded in the application;
|
||||
- Notarius application fallback profiles embedded in the application;
|
||||
- PromptKit's own embedded built-in profiles; and
|
||||
- operator profile files on the deployment filesystem.
|
||||
|
||||
The configuration reference owns the pipeline field, profile-source fields,
|
||||
validation rules, and precedence. Operations owns deployment layout, working
|
||||
directory behavior, credentials, and environment-specific profile management.
|
||||
The PromptKit integration document owns the pinned upstream contract and
|
||||
source-precedence boundary. Internal documents describe asset registration,
|
||||
resolution, inspection, prepared execution, fingerprinting, and tests without
|
||||
duplicating user-facing field definitions.
|
||||
|
||||
The maintained examples continue to include only the minimal and complete D&D
|
||||
configurations. They use the stable `dnd-extraction` policy, and one maintained
|
||||
PromptKit profile file under `examples/` demonstrates an operator override.
|
||||
Examples remain secret-free and are validated without live provider calls.
|
||||
|
||||
## Out Of Scope
|
||||
|
||||
- Implementing the separate deterministic prompt-session identity roadmap
|
||||
item.
|
||||
- Changing the default `concurrency.total_llm` value; PromptKit's retained
|
||||
OpenRouter capacity of 16 remains relevant to that separate item.
|
||||
- Adding model evaluation as a deterministic or CI correctness gate.
|
||||
- Automatically selecting production, development, or local environments.
|
||||
Deployment configuration chooses the operator profile source.
|
||||
- Profile inheritance, partial profile merging, or cross-profile aliases.
|
||||
- Exposing PromptKit types to modules, validators, durable output contracts, or
|
||||
public configuration structures.
|
||||
- Adopting `InspectPrompt` without a separately justified prompt-interface
|
||||
validation contract.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Notarius builds and its offline test suite passes with PromptKit v0.5.0.
|
||||
- Every structured completion executes the exact snapshot used for safe debug
|
||||
prompt details.
|
||||
- Profile preflight uses profile inspection and no synthetic prompt.
|
||||
- The embedded `dnd-extraction` fallback resolves without an operator source,
|
||||
and a matching valid operator profile replaces it completely.
|
||||
- Every production module has explicit, correct execution metadata.
|
||||
- Pipeline, binding, CLI, and prompt-default precedence behaves as defined for
|
||||
modules and validators, while deterministic bindings remain profile-free.
|
||||
- Effective profiles participate in pipeline digests, profile inspection,
|
||||
checkpoint identity, debug records, and run provenance at the appropriate
|
||||
boundaries.
|
||||
- The dependency and application fallback changes invalidate incompatible old
|
||||
checkpoints without exposing profile or credential content.
|
||||
- Canonical documentation and maintained examples accurately describe and
|
||||
exercise the implemented operator workflow.
|
||||
- Default tests remain deterministic, offline, credential-free, and focused on
|
||||
Notarius-owned behavior rather than duplicating PromptKit's upstream suite.
|
||||
@@ -36,10 +36,13 @@ pipelines:
|
||||
window_units: 3
|
||||
lanes:
|
||||
- item-events
|
||||
- locations
|
||||
- location-occurrences
|
||||
- npcs
|
||||
- spells
|
||||
- combat-turns
|
||||
- npc-interactions
|
||||
- enemy-events
|
||||
steps:
|
||||
# Establish session-wide reference artifacts alongside independent item events.
|
||||
- id: describe-session
|
||||
@@ -58,6 +61,14 @@ pipelines:
|
||||
normalize:
|
||||
module: dnd/npcs
|
||||
retries: 2
|
||||
locations:
|
||||
extract:
|
||||
module: dnd/locations
|
||||
retries: 2
|
||||
merge: appendorder
|
||||
normalize:
|
||||
module: dnd/locations
|
||||
retries: 2
|
||||
scene-descriptions:
|
||||
extract:
|
||||
module: dnd/scene-descriptions
|
||||
@@ -65,9 +76,13 @@ pipelines:
|
||||
merge: appendorder
|
||||
normalize: dnd/scene-descriptions
|
||||
- id: extract-events
|
||||
# Accepted NPC grounding and scene-description eligibility artifacts are
|
||||
# supplied in memory to their compatible consumers in this step.
|
||||
# Accepted registry artifacts and scene-description eligibility artifacts
|
||||
# are supplied in memory to their compatible consumers in this step.
|
||||
references:
|
||||
locations:
|
||||
artifact:
|
||||
step: describe-session
|
||||
lane: locations
|
||||
npcs:
|
||||
artifact:
|
||||
step: describe-session
|
||||
@@ -101,3 +116,34 @@ pipelines:
|
||||
retries: 2
|
||||
merge: appendorder
|
||||
normalize: dnd/npc-interactions
|
||||
location-occurrences:
|
||||
extract:
|
||||
module: dnd/location-occurrences
|
||||
retries: 2
|
||||
merge: appendorder
|
||||
normalize: dnd/location-occurrences
|
||||
- id: track-enemies
|
||||
references:
|
||||
npcs:
|
||||
artifact:
|
||||
step: describe-session
|
||||
lane: npcs
|
||||
scene_descriptions:
|
||||
artifact:
|
||||
step: describe-session
|
||||
lane: scene-descriptions
|
||||
combat_turns:
|
||||
artifact:
|
||||
step: extract-events
|
||||
lane: combat-turns
|
||||
npc_interactions:
|
||||
artifact:
|
||||
step: extract-events
|
||||
lane: npc-interactions
|
||||
artifacts:
|
||||
enemy-events:
|
||||
extract:
|
||||
module: dnd/enemy-events
|
||||
retries: 2
|
||||
merge: appendorder
|
||||
normalize: dnd/enemy-events
|
||||
|
||||
@@ -249,10 +249,24 @@ func TestRunAutoReusesPlanWhenRunInputsChange(t *testing.T) {
|
||||
}
|
||||
harness.mu.Lock()
|
||||
chunkCalls := harness.chunkCalls
|
||||
sessions := append([]string(nil), harness.sessionIDs...)
|
||||
harness.mu.Unlock()
|
||||
if chunkCalls != 1 {
|
||||
t.Fatalf("chunk calls across changed run inputs = %d, want 1", chunkCalls)
|
||||
}
|
||||
rawInput, err := os.ReadFile(roots.input)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wantSessionID, err := resolvePromptSessionID("", "test/input", rawInput)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, sessionID := range sessions {
|
||||
if sessionID != wantSessionID {
|
||||
t.Fatalf("session IDs across reference changes = %#v, want %q", sessions, wantSessionID)
|
||||
}
|
||||
}
|
||||
assertFile(t, filepath.Join(roots.plans, strings.TrimPrefix(stateTestDigest, "sha256:"), "plan.json"))
|
||||
assertAnyFile(t, roots.output)
|
||||
}
|
||||
|
||||
340
internal/cli/dnd_enemy_events_contract_test.go
Normal file
340
internal/cli/dnd_enemy_events_contract_test.go
Normal file
@@ -0,0 +1,340 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/artifacts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/config"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/evidencecontext"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/chunk/scenes"
|
||||
locationoccurrencecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/locationoccurrences"
|
||||
locationcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/locations"
|
||||
combat "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
|
||||
enemyevents "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/enemyevents"
|
||||
itemevents "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/itemevents"
|
||||
locationoccurrences "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/locationoccurrences"
|
||||
locations "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/locations"
|
||||
npcinteractions "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcinteractions"
|
||||
npcs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/npcs"
|
||||
scenedescriptions "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/scenedescriptions"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
|
||||
enemyeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/enemyevents"
|
||||
locationnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/locations"
|
||||
npcnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/npcs"
|
||||
)
|
||||
|
||||
func TestProductionEnemyEventConfigurationResolvesGeneratedHandoffs(t *testing.T) {
|
||||
components := productionTestComponents(t)
|
||||
cfg := loadMaintainedExample(t, repositoryPath("examples", "dnd-complete.config.yml"))
|
||||
effective, err := cfg.Resolve(resolveInputForMaintainedExample(components, "dnd-session"))
|
||||
if err != nil {
|
||||
t.Fatalf("Resolve() error = %v, want nil", err)
|
||||
}
|
||||
materialized, _, err := pipeline.MaterializeReferences(effective.ResolvedPipeline, catalogFromRegistries(components.registries), pipeline.ReferenceMaterializationOptions{
|
||||
ConfigPath: repositoryPath("examples", "dnd-complete.config.yml"),
|
||||
WorkingDir: repositoryPath("examples"),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("MaterializeReferences() error = %v, want nil", err)
|
||||
}
|
||||
lane := referenceContractLane(t, materialized, "enemy-events")
|
||||
if lane.ArtifactKind != dnd.EnemyEventListKind || lane.Extract.Module != enemyevents.Key || lane.Extract.Retries != 2 || lane.Merge.Module != pipeline.DefaultMergeModule || lane.Normalize.Module != enemyeventnormalize.Key {
|
||||
t.Fatalf("enemy event lane = %#v, want typed production composition", lane)
|
||||
}
|
||||
for slot, want := range map[string]struct{ step, lane string }{
|
||||
"npcs": {step: "describe-session", lane: "npcs"},
|
||||
"scene_descriptions": {step: "describe-session", lane: "scene-descriptions"},
|
||||
"combat_turns": {step: "extract-events", lane: "combat-turns"},
|
||||
"npc_interactions": {step: "extract-events", lane: "npc-interactions"},
|
||||
} {
|
||||
binding, found := generatedReferenceBinding(lane.ExtractReferences.Bindings, slot)
|
||||
if !found || binding.Artifact.Step != want.step || binding.Artifact.Lane != want.lane {
|
||||
t.Fatalf("enemy event %s reference = %#v, want generated %s/%s artifact", slot, binding, want.step, want.lane)
|
||||
}
|
||||
}
|
||||
if binding, found := generatedReferenceBinding(lane.NormalizeReferences.Bindings, "npcs"); !found || binding.Artifact.Step != "describe-session" || binding.Artifact.Lane != "npcs" {
|
||||
t.Fatalf("enemy event normalizer NPC reference = %#v, want generated NPC artifact", binding)
|
||||
}
|
||||
|
||||
catalog := catalogFromRegistries(components.registries)
|
||||
extractSpec, ok := catalog.Extractors.Spec(enemyevents.Key)
|
||||
if !ok || !reflect.DeepEqual(extractSpec.Requires, []string{"chunks", "source.transcript"}) || !reflect.DeepEqual(extractSpec.Provides, []string{"dnd.enemy_events"}) {
|
||||
t.Fatalf("enemy event extractor spec = %#v, want source and artifact capabilities", extractSpec)
|
||||
}
|
||||
normalizeSpec, ok := catalog.Normalizers.SpecForArtifact(enemyeventnormalize.Key, dnd.EnemyEventListKind)
|
||||
if !ok || !reflect.DeepEqual(normalizeSpec.Requires, []string{"merged"}) || !reflect.DeepEqual(normalizeSpec.Provides, []string{"normalized"}) {
|
||||
t.Fatalf("enemy event normalizer spec = %#v, want merged/normalized capabilities", normalizeSpec)
|
||||
}
|
||||
for _, slot := range []string{"npcs", "scene_descriptions", "combat_turns", "npc_interactions"} {
|
||||
if !hasReferenceSlot(extractSpec.ReferenceSlots, slot) {
|
||||
t.Fatalf("enemy event extractor slots = %#v, want %q", extractSpec.ReferenceSlots, slot)
|
||||
}
|
||||
}
|
||||
if !hasReferenceSlot(normalizeSpec.ReferenceSlots, "npcs") {
|
||||
t.Fatalf("enemy event normalizer slots = %#v, want NPC registry", normalizeSpec.ReferenceSlots)
|
||||
}
|
||||
|
||||
profile := cfg.Pipelines["dnd-session"]
|
||||
profile.Steps[2].References["npcs"] = pipeline.GeneratedReference("track-enemies", "enemy-events")
|
||||
cfg.Pipelines["dnd-session"] = profile
|
||||
if _, err := cfg.Resolve(resolveInputForMaintainedExample(components, "dnd-session")); err == nil || !strings.Contains(err.Error(), "earlier step") {
|
||||
t.Fatalf("Resolve() error = %v, want future generated-reference rejection", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMaintainedCompleteExampleProducesEnemyEventsThroughGeneratedHandoffs(t *testing.T) {
|
||||
t.Chdir(repositoryPath())
|
||||
outputRoot := filepath.Join(t.TempDir(), "output")
|
||||
configPath := completeExampleConfigWithTemporaryCache(t)
|
||||
client := &enemyEventLLMClient{}
|
||||
options := productionCLIOptions(t)
|
||||
options.Now = func() time.Time { return time.Unix(1700000000, 0).UTC() }
|
||||
options.RunIDGenerator = func(time.Time) (string, error) { return productionRunID, nil }
|
||||
options.UserCacheDir = func() (string, error) { return "", errors.New("user cache must not be used") }
|
||||
options.LLMClientFactory = func(context.Context, config.Config, string, LLMRuntimeOverrides) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error) {
|
||||
return client, nil, nil
|
||||
}
|
||||
|
||||
var stdout, stderr strings.Builder
|
||||
code := RunWithOptions([]string{
|
||||
"run", "dnd-session",
|
||||
"--config", configPath,
|
||||
"--input", repositoryPath("examples", "dnd-complete-transcript.json"),
|
||||
"--chunk_cache", "bypass", "--output-dir", outputRoot, "--session-id", "enemy-event-session",
|
||||
}, &stdout, &stderr, options)
|
||||
if code != 0 {
|
||||
t.Fatalf("code=%d stdout=%q stderr=%q", code, stdout.String(), stderr.String())
|
||||
}
|
||||
|
||||
runRoot := filepath.Join(outputRoot, productionRunID)
|
||||
index := readProductionJSON[exampleOutputIndex](t, filepath.Join(runRoot, "index.json"))
|
||||
var enemyOutput exampleOutputIndexEntry
|
||||
var locationOutput, occurrenceOutput exampleOutputIndexEntry
|
||||
for _, entry := range index.OutputFiles {
|
||||
switch entry.LaneID {
|
||||
case "enemy-events":
|
||||
enemyOutput = entry
|
||||
case "locations":
|
||||
locationOutput = entry
|
||||
case "location-occurrences":
|
||||
occurrenceOutput = entry
|
||||
}
|
||||
}
|
||||
if enemyOutput.File != "lanes/enemy-events.json" || enemyOutput.SchemaID != "notarius.dnd.enemy_events" || enemyOutput.SchemaVersion != "v1" {
|
||||
t.Fatalf("enemy event output = %#v, want typed enemy-event JSON", enemyOutput)
|
||||
}
|
||||
value := readProductionJSON[dnd.EnemyEventList](t, filepath.Join(runRoot, enemyOutput.File))
|
||||
if len(value.Events) != 1 || value.Events[0].Name != "Kesh" || value.Events[0].Kind != dnd.EnemyEventKindFled || len(value.Events[0].SourceRefs) != 1 || value.Events[0].SourceRefs[0].SourceID != "session-ravenfall" || value.Events[0].SourceRefs[0].StartUnitID != 10 {
|
||||
t.Fatalf("enemy event artifact = %#v, want source-linked Kesh fleeing event", value)
|
||||
}
|
||||
if locationOutput.File != "lanes/locations.json" || locationOutput.SchemaID != locationcodec.SchemaID || locationOutput.SchemaVersion != locationcodec.SchemaVersion {
|
||||
t.Fatalf("location output = %#v, want typed location registry JSON", locationOutput)
|
||||
}
|
||||
locationsValue := readProductionJSON[dnd.LocationList](t, filepath.Join(runRoot, locationOutput.File))
|
||||
if len(locationsValue.Locations) != 2 || locationsValue.Locations[0].Name != "Moon Gate" || locationsValue.Locations[1].Name != "Moon Gate" || locationsValue.Locations[0].ID == locationsValue.Locations[1].ID {
|
||||
t.Fatalf("location registry = %#v, want distinct source-grounded identities for same-name locations", locationsValue)
|
||||
}
|
||||
if occurrenceOutput.File != "lanes/location-occurrences.json" || occurrenceOutput.SchemaID != locationoccurrencecodec.SchemaID || occurrenceOutput.SchemaVersion != locationoccurrencecodec.SchemaVersion {
|
||||
t.Fatalf("location occurrence output = %#v, want typed occurrence JSON", occurrenceOutput)
|
||||
}
|
||||
occurrencesValue := readProductionJSON[dnd.LocationOccurrenceList](t, filepath.Join(runRoot, occurrenceOutput.File))
|
||||
if len(occurrencesValue.Occurrences) != 2 || occurrencesValue.Occurrences[0].LocationID == occurrencesValue.Occurrences[1].LocationID || occurrencesValue.Occurrences[0].Name != "Moon Gate" || occurrencesValue.Occurrences[1].Name != "Moon Gate" {
|
||||
t.Fatalf("location occurrences = %#v, want source-grounded references to distinct registry identities", occurrencesValue)
|
||||
}
|
||||
|
||||
evidence := readProductionJSON[evidencecontext.Document](t, filepath.Join(runRoot, "evidence-context.json"))
|
||||
for _, laneID := range []string{"enemy-events", "locations", "location-occurrences"} {
|
||||
if !containsString(evidence.SelectedLanes, laneID) || !evidenceHasLane(evidence, laneID) {
|
||||
t.Fatalf("evidence context = %#v, want direct %s evidence", evidence, laneID)
|
||||
}
|
||||
}
|
||||
|
||||
requests := client.requestsFor(enemyevents.PromptID)
|
||||
if len(requests) != 1 {
|
||||
t.Fatalf("enemy event requests = %#v, want only the combat scene request", requests)
|
||||
}
|
||||
request := requests[0]
|
||||
if request.SessionID != "enemy-event-session" {
|
||||
t.Fatalf("enemy event session = %q, want shared session", request.SessionID)
|
||||
}
|
||||
for slot, required := range map[string]string{
|
||||
"npcs": "Kesh",
|
||||
"combat_turns": "Kesh",
|
||||
"npc_interactions": "Kesh",
|
||||
} {
|
||||
input, ok := request.Inputs[slot]
|
||||
if !ok || !strings.Contains(string(input.Content), required) || strings.Contains(string(input.Content), "source_refs") || strings.Contains(string(input.Content), "start_unit_id") {
|
||||
t.Fatalf("enemy event %s prompt input = %q, want compact source-free grounding", slot, input.Content)
|
||||
}
|
||||
}
|
||||
locationRequests := client.requestsFor(locationoccurrences.PromptID)
|
||||
if len(locationRequests) != 2 {
|
||||
t.Fatalf("location occurrence requests = %#v, want one request per scene", locationRequests)
|
||||
}
|
||||
for _, request := range locationRequests {
|
||||
registryInput := request.Inputs["locations"]
|
||||
if !strings.Contains(string(registryInput.Content), "Moon Gate") || !strings.Contains(string(registryInput.Content), `"id"`) || strings.Contains(string(registryInput.Content), "source_refs") {
|
||||
t.Fatalf("location occurrence registry input = %q, want source-free ID grounding", registryInput.Content)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func completeExampleConfigWithTemporaryCache(t *testing.T) string {
|
||||
t.Helper()
|
||||
content, err := os.ReadFile(repositoryPath("examples", "dnd-complete.config.yml"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cacheRoot := t.TempDir()
|
||||
updated := strings.Replace(string(content), "directory: ./notarius-cache/chunk-plans", fmt.Sprintf("directory: %q", filepath.Join(cacheRoot, "chunk-plans")), 1)
|
||||
updated = strings.Replace(updated, "directory: ./notarius-cache/checkpoints", fmt.Sprintf("directory: %q", filepath.Join(cacheRoot, "checkpoints")), 1)
|
||||
for relative, absolute := range map[string]string{
|
||||
"./dnd-party.txt": repositoryPath("examples", "dnd-party.txt"),
|
||||
"./dnd-glossary.txt": repositoryPath("examples", "dnd-glossary.txt"),
|
||||
"./dnd-spell-catalog.json": repositoryPath("examples", "dnd-spell-catalog.json"),
|
||||
} {
|
||||
updated = strings.ReplaceAll(updated, relative, fmt.Sprintf("%q", absolute))
|
||||
}
|
||||
path := filepath.Join(t.TempDir(), "dnd-complete.config.yml")
|
||||
if err := os.WriteFile(path, []byte(updated), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return path
|
||||
}
|
||||
|
||||
type enemyEventLLMClient struct {
|
||||
mu sync.Mutex
|
||||
requests []contracts.StructuredCompletionRequest
|
||||
}
|
||||
|
||||
func (client *enemyEventLLMClient) CompleteStructured(ctx context.Context, request contracts.StructuredCompletionRequest, output any) (contracts.StructuredCompletionResponse, error) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, err
|
||||
}
|
||||
combatScene := strings.Contains(string(request.Inputs["transcript"].Content), "Roll initiative")
|
||||
var content []byte
|
||||
switch request.PromptID {
|
||||
case scenes.PromptID:
|
||||
content = []byte(`{"scenes":[{"start_unit_id":1,"end_unit_id":6},{"start_unit_id":7,"end_unit_id":11}]}`)
|
||||
case npcs.PromptID:
|
||||
if combatScene {
|
||||
content = []byte(`{"npcs":[{"name":"Kesh","source_refs":[{"start_unit_id":7,"end_unit_id":7}]}]}`)
|
||||
} else {
|
||||
content = []byte(`{"npcs":[]}`)
|
||||
}
|
||||
case npcnormalize.PromptID:
|
||||
content = []byte(`{"duplicate_groups":[]}`)
|
||||
case scenedescriptions.PromptID:
|
||||
kind, title := "narrative", "Arrival"
|
||||
if combatScene {
|
||||
kind, title = "combat", "Raiders attack"
|
||||
}
|
||||
content = []byte(fmt.Sprintf(`{"kind":%q,"title":%q,"summary":"session scene"}`, kind, title))
|
||||
case locations.PromptID:
|
||||
unitID := 1
|
||||
if combatScene {
|
||||
unitID = 7
|
||||
}
|
||||
content = []byte(fmt.Sprintf(`{"locations":[{"name":"Moon Gate","source_refs":[{"start_unit_id":%d,"end_unit_id":%d}]}]}`, unitID, unitID))
|
||||
case locationnormalize.PromptID:
|
||||
content = []byte(`{"duplicate_groups":[]}`)
|
||||
case spells.PromptID:
|
||||
content = []byte(`{"spell_casts":[]}`)
|
||||
case itemevents.PromptID:
|
||||
content = []byte(`{"events":[]}`)
|
||||
case combat.PromptID:
|
||||
content = []byte(`{"combat_turns":[{"actor":"Kesh","turn_kind":"turn","source_refs":[{"start_unit_id":8,"end_unit_id":8}]}]}`)
|
||||
case npcinteractions.PromptID:
|
||||
if combatScene {
|
||||
content = []byte(`{"interactions":[{"name":"Kesh","kind":"combat_opponent","source_refs":[{"start_unit_id":7,"end_unit_id":7}]}]}`)
|
||||
} else {
|
||||
content = []byte(`{"interactions":[]}`)
|
||||
}
|
||||
case locationoccurrences.PromptID:
|
||||
var registry struct {
|
||||
Locations []struct {
|
||||
ID string `json:"id"`
|
||||
} `json:"locations"`
|
||||
}
|
||||
if err := json.Unmarshal(request.Inputs["locations"].Content, ®istry); err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, fmt.Errorf("decode generated location registry: %w", err)
|
||||
}
|
||||
if len(registry.Locations) == 0 {
|
||||
return contracts.StructuredCompletionResponse{}, fmt.Errorf("generated location registry has no locations")
|
||||
}
|
||||
unitID := 1
|
||||
locationID := registry.Locations[0].ID
|
||||
if combatScene {
|
||||
unitID = 7
|
||||
if len(registry.Locations) > 1 {
|
||||
locationID = registry.Locations[1].ID
|
||||
}
|
||||
}
|
||||
content = []byte(fmt.Sprintf(`{"occurrences":[{"location_id":%q,"name":"Moon Gate","kind":"visited","source_refs":[{"start_unit_id":%d,"end_unit_id":%d}]}]}`, locationID, unitID, unitID))
|
||||
case enemyevents.PromptID:
|
||||
content = []byte(`{"events":[{"name":"Kesh","kind":"fled","source_refs":[{"start_unit_id":10,"end_unit_id":10}]}]}`)
|
||||
default:
|
||||
return contracts.StructuredCompletionResponse{}, fmt.Errorf("unexpected prompt %q", request.PromptID)
|
||||
}
|
||||
if err := json.Unmarshal(content, output); err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, fmt.Errorf("populate fake structured target: %w", err)
|
||||
}
|
||||
client.mu.Lock()
|
||||
client.requests = append(client.requests, request)
|
||||
client.mu.Unlock()
|
||||
return contracts.StructuredCompletionResponse{Content: content, Provider: "test", Model: "deterministic", ProfileID: request.ProfileID}, nil
|
||||
}
|
||||
|
||||
func (client *enemyEventLLMClient) requestsFor(promptID string) []contracts.StructuredCompletionRequest {
|
||||
client.mu.Lock()
|
||||
defer client.mu.Unlock()
|
||||
var requests []contracts.StructuredCompletionRequest
|
||||
for _, request := range client.requests {
|
||||
if request.PromptID == promptID {
|
||||
requests = append(requests, request)
|
||||
}
|
||||
}
|
||||
return requests
|
||||
}
|
||||
|
||||
func containsString(values []string, want string) bool {
|
||||
for _, value := range values {
|
||||
if value == want {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func evidenceHasLane(value evidencecontext.Document, laneID string) bool {
|
||||
for _, context := range value.Contexts {
|
||||
for _, reference := range context.EvidenceRefs {
|
||||
if reference.LaneID == laneID {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func generatedReferenceBinding(bindings []pipeline.ReferenceBinding, slotName string) (pipeline.ReferenceBinding, bool) {
|
||||
for _, binding := range bindings {
|
||||
if binding.SlotName == slotName && binding.Artifact != nil {
|
||||
return binding, true
|
||||
}
|
||||
}
|
||||
return pipeline.ReferenceBinding{}, false
|
||||
}
|
||||
@@ -15,6 +15,10 @@ import (
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
locationoccurrenceextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/locationoccurrences"
|
||||
locationextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/locations"
|
||||
locationoccurrencenormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/locationoccurrences"
|
||||
locationnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/locations"
|
||||
spellnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/spells"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/seriatim/input/transcript"
|
||||
)
|
||||
@@ -24,6 +28,9 @@ func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
|
||||
for _, example := range maintainedExampleFiles(t) {
|
||||
t.Run(example.name, func(t *testing.T) {
|
||||
cfg := loadMaintainedExample(t, example.path)
|
||||
if example.name == "complete" && (cfg.Concurrency.TotalLLM != 2 || cfg.Concurrency.StageWorkers["extract"] != 2) {
|
||||
t.Fatalf("complete example concurrency = %#v, want explicit limits of 2", cfg.Concurrency)
|
||||
}
|
||||
raw, err := os.ReadFile(example.transcriptPath)
|
||||
if err != nil {
|
||||
t.Fatalf("read maintained transcript %q: %v", example.transcriptPath, err)
|
||||
@@ -51,8 +58,30 @@ func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
|
||||
t.Fatalf("materialize maintained example references for %q: %v", pipelineID, err)
|
||||
}
|
||||
if example.name == "complete" {
|
||||
if got := exampleStepLaneIDs(materialized); strings.Join(got, "|") != "describe-session:item-events,npcs,scene-descriptions|extract-events:combat-turns,npc-interactions,spells" {
|
||||
t.Fatalf("complete example steps and lanes = %v, want every D&D extractor in the documented two-step composition", got)
|
||||
if got := exampleStepLaneIDs(materialized); strings.Join(got, "|") != "describe-session:item-events,locations,npcs,scene-descriptions|extract-events:combat-turns,location-occurrences,npc-interactions,spells|track-enemies:enemy-events" {
|
||||
t.Fatalf("complete example steps and lanes = %v, want the documented D&D extractor composition", got)
|
||||
}
|
||||
locationLane := referenceContractLane(t, materialized, "locations")
|
||||
if locationLane.ArtifactKind != dnd.LocationListKind || locationLane.Extract.Module != locationextract.Key || locationLane.Extract.Retries != 2 || locationLane.Merge.Module != pipeline.DefaultMergeModule || locationLane.Normalize.Module != locationnormalize.Key || locationLane.Normalize.Retries != 2 {
|
||||
t.Fatalf("location lane = %#v, want typed registry composition", locationLane)
|
||||
}
|
||||
occurrenceLane := referenceContractLane(t, materialized, "location-occurrences")
|
||||
if occurrenceLane.ArtifactKind != dnd.LocationOccurrenceListKind || occurrenceLane.Extract.Module != locationoccurrenceextract.Key || occurrenceLane.Extract.Retries != 2 || occurrenceLane.Merge.Module != pipeline.DefaultMergeModule || occurrenceLane.Normalize.Module != locationoccurrencenormalize.Key {
|
||||
t.Fatalf("location occurrence lane = %#v, want typed occurrence composition", occurrenceLane)
|
||||
}
|
||||
for _, target := range []pipeline.ResolvedReferenceTarget{occurrenceLane.ExtractReferences, occurrenceLane.NormalizeReferences} {
|
||||
binding, found := generatedReferenceBinding(target.Bindings, "locations")
|
||||
if !found || binding.Artifact.Step != "describe-session" || binding.Artifact.Lane != "locations" {
|
||||
t.Fatalf("location occurrence %s reference = %#v, want generated location registry", target.Stage, binding)
|
||||
}
|
||||
}
|
||||
for _, slot := range []string{"party", "glossary"} {
|
||||
if len(occurrenceLane.ExtractReferences.ReferenceSet.Slots[slot].Items) != 1 {
|
||||
t.Fatalf("location occurrence extractor %s reference was not materialized: %#v", slot, occurrenceLane.ExtractReferences)
|
||||
}
|
||||
if _, found := occurrenceLane.NormalizeReferences.ReferenceSet.Slots[slot]; found {
|
||||
t.Fatalf("location occurrence normalizer unexpectedly consumes %s: %#v", slot, occurrenceLane.NormalizeReferences)
|
||||
}
|
||||
}
|
||||
spellLane := referenceContractLane(t, materialized, "spells")
|
||||
if len(spellLane.ExtractReferences.ReferenceSet.Slots["spell_catalog"].Items) != 1 ||
|
||||
@@ -68,6 +97,18 @@ func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
|
||||
t.Fatalf("item event lane unexpectedly depends on generated scene descriptions: %#v", itemEventLane)
|
||||
}
|
||||
}
|
||||
enemyEventLane := referenceContractLane(t, materialized, "enemy-events")
|
||||
for slot, want := range map[string]struct{ step, lane string }{
|
||||
"npcs": {step: "describe-session", lane: "npcs"},
|
||||
"scene_descriptions": {step: "describe-session", lane: "scene-descriptions"},
|
||||
"combat_turns": {step: "extract-events", lane: "combat-turns"},
|
||||
"npc_interactions": {step: "extract-events", lane: "npc-interactions"},
|
||||
} {
|
||||
binding, found := generatedReferenceBinding(enemyEventLane.ExtractReferences.Bindings, slot)
|
||||
if !found || binding.Artifact.Step != want.step || binding.Artifact.Lane != want.lane {
|
||||
t.Fatalf("enemy event %s reference = %#v, want generated %s/%s artifact", slot, binding, want.step, want.lane)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
var stdout, stderr strings.Builder
|
||||
@@ -242,6 +283,12 @@ func TestMaintainedMalformedInputOnlyRecordsDebugFailureWhenRequested(t *testing
|
||||
if report.Succeeded || report.PipelineID != "dnd-session" {
|
||||
t.Fatalf("failure report = %#v, want failed dnd-session report", report)
|
||||
}
|
||||
invocation := readProductionJSON[debugbundle.Invocation](t, filepath.Join(bundle, "summary", "invocation.json"))
|
||||
manifest := readProductionJSON[artifacts.RunManifest](t, filepath.Join(bundle, "summary", "run-manifest.json"))
|
||||
manifestSession, found := manifest.Metadata["session_id"]
|
||||
if invocation.SessionID == "" || !found || manifestSession != invocation.SessionID {
|
||||
t.Fatalf("failed run sessions: invocation=%q manifest=%#v metadata=%#v", invocation.SessionID, manifestSession, manifest.Metadata)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -29,12 +29,15 @@ import (
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/chunk/scenes"
|
||||
combatcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/combatturns"
|
||||
enemyeventcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/enemyevents"
|
||||
itemeventcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/itemevents"
|
||||
spellcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/spells"
|
||||
combatextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/combatturns"
|
||||
enemyeventextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/enemyevents"
|
||||
itemeventextract "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/itemevents"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
|
||||
combatnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/combatturns"
|
||||
enemyeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/enemyevents"
|
||||
itemeventnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/itemevents"
|
||||
spellnormalize "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/normalize/spells"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/generic/normalize/noop"
|
||||
@@ -47,9 +50,9 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
||||
|
||||
assertProductionContains(t, "inputs", registries.Inputs.RegisteredKeys(), []string{"seriatim"})
|
||||
assertProductionContains(t, "chunkers", registries.Chunkers.RegisteredKeys(), []string{"dnd/scenes", "generic"})
|
||||
assertProductionContains(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells", "dnd/npcs", combatextract.Key, itemeventextract.Key})
|
||||
assertProductionContains(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells", "dnd/npcs", combatextract.Key, itemeventextract.Key, enemyeventextract.Key})
|
||||
assertProductionContains(t, "mergers", registries.Mergers.RegisteredKeys(), []string{"appendorder"})
|
||||
assertProductionContains(t, "normalizers", registries.Normalizers.RegisteredKeys(), []string{"noop", spellnormalize.Key, "dnd/npcs", combatnormalize.Key, itemeventnormalize.Key})
|
||||
assertProductionContains(t, "normalizers", registries.Normalizers.RegisteredKeys(), []string{"noop", spellnormalize.Key, "dnd/npcs", combatnormalize.Key, itemeventnormalize.Key, enemyeventnormalize.Key})
|
||||
assertProductionContains(t, "outputs", registries.Outputs.RegisteredKeys(), []string{"json"})
|
||||
assertProductionContains(t, "validators", registries.Validators.RegisteredKeys(), []string{
|
||||
"extract/dnd/spells/catalog",
|
||||
@@ -64,17 +67,23 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
||||
"extract/dnd/item-events/source_refs",
|
||||
"extract/dnd/item-events/source_relatedness",
|
||||
"normalize/dnd/item-events/invariants",
|
||||
"extract/dnd/enemy-events/shape",
|
||||
"extract/dnd/enemy-events/engagements",
|
||||
"extract/dnd/enemy-events/source_refs",
|
||||
"extract/dnd/enemy-events/source_relatedness",
|
||||
"normalize/dnd/enemy-events/invariants",
|
||||
"generic/always_accept",
|
||||
"generic/always_reject",
|
||||
"generic/valid_json",
|
||||
"generic/valid_json_schema",
|
||||
})
|
||||
assertProductionContains(t, "artifact codec kinds", registries.ArtifactCodecs.RegisteredKinds(), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind})
|
||||
assertProductionContains(t, "merger variants", registries.Mergers.RegisteredArtifactKinds(pipeline.DefaultMergeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind})
|
||||
assertProductionContains(t, "normalizer variants", registries.Normalizers.RegisteredArtifactKinds(pipeline.DefaultNormalizeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind})
|
||||
assertProductionContains(t, "artifact codec kinds", registries.ArtifactCodecs.RegisteredKinds(), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind, dnd.EnemyEventListKind})
|
||||
assertProductionContains(t, "merger variants", registries.Mergers.RegisteredArtifactKinds(pipeline.DefaultMergeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind, dnd.EnemyEventListKind})
|
||||
assertProductionContains(t, "normalizer variants", registries.Normalizers.RegisteredArtifactKinds(pipeline.DefaultNormalizeModule), []contracts.ArtifactKind{dnd.SpellListKind, dnd.NPCListKind, dnd.CombatTurnListKind, dnd.ItemEventListKind, dnd.EnemyEventListKind})
|
||||
assertProductionContains(t, "spell normalizer variants", registries.Normalizers.RegisteredArtifactKinds(spellnormalize.Key), []contracts.ArtifactKind{dnd.SpellListKind})
|
||||
assertProductionContains(t, "combat normalizer variants", registries.Normalizers.RegisteredArtifactKinds(combatnormalize.Key), []contracts.ArtifactKind{dnd.CombatTurnListKind})
|
||||
assertProductionContains(t, "item event normalizer variants", registries.Normalizers.RegisteredArtifactKinds(itemeventnormalize.Key), []contracts.ArtifactKind{dnd.ItemEventListKind})
|
||||
assertProductionContains(t, "enemy event normalizer variants", registries.Normalizers.RegisteredArtifactKinds(enemyeventnormalize.Key), []contracts.ArtifactKind{dnd.EnemyEventListKind})
|
||||
|
||||
wantChain := []pipeline.ModuleBinding{
|
||||
pipeline.Binding("generic/valid_json"),
|
||||
@@ -132,6 +141,17 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
||||
if got := registries.ValidatorChains.Validators(pipeline.StageNormalize, itemeventnormalize.Key); !reflect.DeepEqual(got, itemEventNormalizeChain) {
|
||||
t.Fatalf("item event normalize validator chain = %#v, want %#v", got, itemEventNormalizeChain)
|
||||
}
|
||||
enemyEventExtractChain := []pipeline.ModuleBinding{
|
||||
pipeline.Binding("generic/valid_json"),
|
||||
pipeline.Binding("extract/dnd/enemy-events/shape"),
|
||||
pipeline.Binding("extract/dnd/enemy-events/engagements"),
|
||||
pipeline.Binding("extract/dnd/enemy-events/source_refs"),
|
||||
pipeline.Binding("generic/valid_json_schema"),
|
||||
pipeline.Binding("extract/dnd/enemy-events/source_relatedness"),
|
||||
}
|
||||
if got := registries.ValidatorChains.Validators(pipeline.StageExtract, enemyeventextract.Key); !reflect.DeepEqual(got, enemyEventExtractChain) {
|
||||
t.Fatalf("enemy event extract validator chain = %#v, want %#v", got, enemyEventExtractChain)
|
||||
}
|
||||
|
||||
assetNames := productionAssetNames(t, components.assets.PromptFS)
|
||||
requiredAssets := []string{
|
||||
@@ -162,6 +182,10 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
||||
"dnd.item_events/sharedassets/common-dnd-system.md",
|
||||
"dnd.item_events/sharedassets/common-dnd-transcript.md",
|
||||
"dnd.item_events/task.md",
|
||||
"dnd.enemy_events/dnd.enemy_events.yaml",
|
||||
"dnd.enemy_events/grounding.md",
|
||||
"dnd.enemy_events/instructions.md",
|
||||
"dnd.enemy_events/task.md",
|
||||
}
|
||||
assertProductionContains(t, "production prompt assets", assetNames, requiredAssets)
|
||||
|
||||
@@ -180,6 +204,7 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
||||
{stage: pipeline.StageExtract, key: "dnd/item-events", want: contracts.ExecutionClassLLMBacked},
|
||||
{stage: pipeline.StageExtract, key: "dnd/npc-interactions", want: contracts.ExecutionClassLLMBacked},
|
||||
{stage: pipeline.StageExtract, key: "dnd/scene-descriptions", want: contracts.ExecutionClassLLMBacked},
|
||||
{stage: pipeline.StageExtract, key: enemyeventextract.Key, want: contracts.ExecutionClassLLMBacked},
|
||||
{stage: pipeline.StageMerge, key: "appendorder", want: contracts.ExecutionClassDeterministic},
|
||||
{stage: pipeline.StageNormalize, key: "noop", want: contracts.ExecutionClassDeterministic},
|
||||
{stage: pipeline.StageNormalize, key: "dnd/spells", want: contracts.ExecutionClassDeterministic},
|
||||
@@ -188,6 +213,7 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
||||
{stage: pipeline.StageNormalize, key: "dnd/item-events", want: contracts.ExecutionClassDeterministic},
|
||||
{stage: pipeline.StageNormalize, key: "dnd/npc-interactions", want: contracts.ExecutionClassDeterministic},
|
||||
{stage: pipeline.StageNormalize, key: "dnd/scene-descriptions", want: contracts.ExecutionClassDeterministic},
|
||||
{stage: pipeline.StageNormalize, key: enemyeventnormalize.Key, want: contracts.ExecutionClassDeterministic},
|
||||
{stage: pipeline.StageOutput, key: "json", want: contracts.ExecutionClassDeterministic},
|
||||
} {
|
||||
got, ok := catalog.ExecutionClass(test.stage, test.key)
|
||||
@@ -211,6 +237,10 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) {
|
||||
if !ok || itemEventCodecSpec.Kind != dnd.ItemEventListKind || itemEventCodecSpec.Schema.ID != itemeventcodec.SchemaID {
|
||||
t.Fatalf("item event codec spec = %#v, ok=%t, want typed D&D item-event codec", itemEventCodecSpec, ok)
|
||||
}
|
||||
enemyEventCodecSpec, ok := catalog.ArtifactCodecs.Spec(dnd.EnemyEventListKind)
|
||||
if !ok || enemyEventCodecSpec.Kind != dnd.EnemyEventListKind || enemyEventCodecSpec.Schema.ID != enemyeventcodec.SchemaID {
|
||||
t.Fatalf("enemy event codec spec = %#v, ok=%t, want typed D&D enemy-event codec", enemyEventCodecSpec, ok)
|
||||
}
|
||||
if got := catalog.ValidatorChains.Validators(pipeline.StageExtract, spells.Key); !reflect.DeepEqual(got, wantChain) {
|
||||
t.Fatalf("catalog validator chain = %#v, want %#v", got, wantChain)
|
||||
}
|
||||
|
||||
@@ -153,10 +153,10 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
|
||||
resume := fs.Bool("resume", false, "reuse compatible recorded checkpoints")
|
||||
recomputeStep := singleValueFlag{name: "--recompute-step"}
|
||||
chunkCache := chunkCacheFlag{}
|
||||
sessionID := sessionIDFlag{}
|
||||
requestedSessionID := sessionIDFlag{}
|
||||
referenceFlags := stringListFlag{}
|
||||
withoutReferenceFlags := stringListFlag{}
|
||||
fs.Var(&sessionID, "session-id", "prompt session identifier")
|
||||
fs.Var(&requestedSessionID, "session-id", "prompt session identifier")
|
||||
fs.Var(&reasoningEffort, "reasoning-effort", "reasoning effort override")
|
||||
fs.Var(&chunkCache, "chunk_cache", "chunk plan cache mode: auto, bypass, or refresh")
|
||||
fs.Var(&referenceFlags, "reference", "reference binding, as slot=path, chunk.slot=path, merge.slot=path, lane.slot=path, lane.extract.slot=path, lane.merge.slot=path, or lane.normalize.slot=path")
|
||||
@@ -199,7 +199,7 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
|
||||
fmt.Fprintln(stderr, "notarius: --debug-dir must not be empty")
|
||||
return 2
|
||||
}
|
||||
if sessionID.set && strings.TrimSpace(sessionID.value) == "" {
|
||||
if requestedSessionID.set && strings.TrimSpace(requestedSessionID.value) == "" {
|
||||
fmt.Fprintln(stderr, "notarius: --session-id must not be empty")
|
||||
return 2
|
||||
}
|
||||
@@ -414,11 +414,19 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("read input %q: %w", strings.TrimSpace(*inputPath), err))
|
||||
}
|
||||
effectiveSessionID, err := resolvePromptSessionID(requestedSessionID.value, effective.ResolvedPipeline.Input.Module, rawInput)
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
||||
}
|
||||
invocation.SessionID = effectiveSessionID
|
||||
if err := writeSummary(summary, func() error { return summary.WriteInvocation(invocation) }); err != nil {
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, fmt.Errorf("write debug invocation metadata: %w", err))
|
||||
}
|
||||
chunkPlans, err := chunkPlanStoreForRun(effective.Config.Cache.ChunkPlans, opts)
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
||||
}
|
||||
checkpointRecorder, checkpointLoader, err := checkpointHandlersForRun(effective.Config.Cache.Checkpoints, opts, effective.ResolvedPipeline, prepared.CheckpointFingerprints(), llmFingerprints, rawInput, only, llmProfiles, strings.TrimSpace(*llmProfile), strings.TrimSpace(sessionID.value), runtimeOverrides, *resume)
|
||||
checkpointRecorder, checkpointLoader, err := checkpointHandlersForRun(effective.Config.Cache.Checkpoints, opts, effective.ResolvedPipeline, prepared.CheckpointFingerprints(), llmFingerprints, rawInput, only, llmProfiles, strings.TrimSpace(*llmProfile), effectiveSessionID, runtimeOverrides, *resume)
|
||||
if err != nil {
|
||||
return failPipelineCommand(stderr, commandState, terminalWriter, err)
|
||||
}
|
||||
@@ -427,7 +435,7 @@ func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) i
|
||||
Prepared: prepared,
|
||||
Path: strings.TrimSpace(*inputPath),
|
||||
RawInput: rawInput,
|
||||
SessionID: strings.TrimSpace(sessionID.value),
|
||||
SessionID: effectiveSessionID,
|
||||
RunID: runID,
|
||||
StartedAt: startedAt,
|
||||
LLMProfiles: llmProfiles,
|
||||
|
||||
@@ -492,19 +492,30 @@ func TestEffectiveLLMProfileIDsAreSortedDeduplicatedAndLLMOnly(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunSessionIDUsesExplicitValueOrSourceDocumentID(t *testing.T) {
|
||||
func TestRunSessionIDUsesEffectiveValueForPromptRequests(t *testing.T) {
|
||||
for _, tt := range []struct {
|
||||
name string
|
||||
args []string
|
||||
want string
|
||||
}{
|
||||
{name: "source document", want: "source"},
|
||||
{name: "derived default"},
|
||||
{name: "explicit trimmed value", args: []string{"--session-id", " explicit-session "}, want: "explicit-session"},
|
||||
} {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
roots := newStateTestRoots(t)
|
||||
want := tt.want
|
||||
if want == "" {
|
||||
rawInput, err := os.ReadFile(roots.input)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want, err = resolvePromptSessionID("", "test/input", rawInput)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
harness := newStateTestHarness()
|
||||
args := append([]string{"run", "sample", "--config", roots.config, "--input", roots.input, "--chunk_cache", "bypass"}, tt.args...)
|
||||
args := append([]string{"run", "sample", "--config", roots.config, "--input", roots.input, "--chunk_cache", "bypass", "--debug"}, tt.args...)
|
||||
var stdout, stderr bytes.Buffer
|
||||
code := RunWithOptions(args, &stdout, &stderr, harness.options())
|
||||
if code != 0 || stderr.Len() != 0 {
|
||||
@@ -517,10 +528,15 @@ func TestRunSessionIDUsesExplicitValueOrSourceDocumentID(t *testing.T) {
|
||||
t.Fatalf("session IDs = %#v, want all prompt-facing module requests", sessions)
|
||||
}
|
||||
for _, session := range sessions {
|
||||
if session != tt.want {
|
||||
t.Fatalf("session IDs = %#v, want %q", sessions, tt.want)
|
||||
if session != want {
|
||||
t.Fatalf("session IDs = %#v, want %q", sessions, want)
|
||||
}
|
||||
}
|
||||
var manifest artifacts.RunManifest
|
||||
readStateTestSummaryJSON(t, onlyChildDir(t, roots.debug), "run-manifest.json", &manifest)
|
||||
if session, ok := manifest.Metadata["session_id"]; !ok || session != want {
|
||||
t.Fatalf("manifest session = %#v, want %q; metadata = %#v", session, want, manifest.Metadata)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
26
internal/cli/session.go
Normal file
26
internal/cli/session.go
Normal file
@@ -0,0 +1,26 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const generatedSessionIDPrefix = "notarius:v1:"
|
||||
|
||||
func resolvePromptSessionID(explicitSessionID, inputModule string, rawInput []byte) (string, error) {
|
||||
inputModule = strings.TrimSpace(inputModule)
|
||||
if inputModule == "" {
|
||||
return "", fmt.Errorf("resolve prompt session: input module key must not be empty")
|
||||
}
|
||||
if sessionID := strings.TrimSpace(explicitSessionID); sessionID != "" {
|
||||
return sessionID, nil
|
||||
}
|
||||
|
||||
hasher := sha256.New()
|
||||
_, _ = hasher.Write([]byte(inputModule))
|
||||
_, _ = hasher.Write([]byte{0})
|
||||
_, _ = hasher.Write(rawInput)
|
||||
return generatedSessionIDPrefix + hex.EncodeToString(hasher.Sum(nil)), nil
|
||||
}
|
||||
61
internal/cli/session_test.go
Normal file
61
internal/cli/session_test.go
Normal file
@@ -0,0 +1,61 @@
|
||||
package cli
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestResolvePromptSessionIDUsesVersionedInputIdentity(t *testing.T) {
|
||||
got, err := resolvePromptSessionID("", " seriatim/input/transcript ", []byte("{\"entries\":[\"one\"]}\n"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
const want = "notarius:v1:e15fefdca48653e73248b4157900547be1fd34250c0138f8d3d1f2e89b43bb25"
|
||||
if got != want {
|
||||
t.Fatalf("resolved session = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolvePromptSessionIDStabilityAndOverride(t *testing.T) {
|
||||
rawInput := []byte("same input")
|
||||
baseline, err := resolvePromptSessionID("", "input/transcript", rawInput)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
repeated, err := resolvePromptSessionID("", "input/transcript", rawInput)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if baseline != repeated {
|
||||
t.Fatalf("resolved sessions = %q and %q, want stable value", baseline, repeated)
|
||||
}
|
||||
|
||||
differentModule, err := resolvePromptSessionID("", "input/other", rawInput)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if baseline == differentModule {
|
||||
t.Fatalf("resolved sessions = %q for distinct input modules", baseline)
|
||||
}
|
||||
|
||||
differentInput, err := resolvePromptSessionID("", "input/transcript", []byte("same inpuu"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if baseline == differentInput {
|
||||
t.Fatalf("resolved sessions = %q for distinct input bytes", baseline)
|
||||
}
|
||||
|
||||
override, err := resolvePromptSessionID(" explicit-session ", "input/other", []byte("different input"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if override != "explicit-session" {
|
||||
t.Fatalf("resolved override = %q, want %q", override, "explicit-session")
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolvePromptSessionIDRejectsEmptyInputModule(t *testing.T) {
|
||||
for _, explicitSessionID := range []string{"", "explicit-session"} {
|
||||
if _, err := resolvePromptSessionID(explicitSessionID, " \t", []byte("input")); err == nil {
|
||||
t.Fatalf("resolvePromptSessionID(%q) error = nil, want empty module failure", explicitSessionID)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -329,11 +329,25 @@ func TestRunUsesOneInjectedIdentityForDebugOutputAndManifest(t *testing.T) {
|
||||
if manifest.StartedAt == nil || !manifest.StartedAt.Equal(wantStartedAt) {
|
||||
t.Fatalf("manifest started at = %v, want %v", manifest.StartedAt, wantStartedAt)
|
||||
}
|
||||
rawInput, err := os.ReadFile(roots.input)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wantSessionID, err := resolvePromptSessionID("", "test/input", rawInput)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if sessionID, ok := manifest.Metadata["session_id"]; !ok || sessionID != wantSessionID {
|
||||
t.Fatalf("manifest session = %#v, want %q; metadata = %#v", sessionID, wantSessionID, manifest.Metadata)
|
||||
}
|
||||
var invocation debugbundle.Invocation
|
||||
readStateTestSummaryJSON(t, debugPath, "invocation.json", &invocation)
|
||||
if invocation.RunID != runID || !invocation.StartedAt.Equal(wantStartedAt) {
|
||||
t.Fatalf("debug invocation identity = %#v, want run %q at %v", invocation, runID, wantStartedAt)
|
||||
}
|
||||
if invocation.SessionID != wantSessionID {
|
||||
t.Fatalf("debug invocation session = %q, want %q", invocation.SessionID, wantSessionID)
|
||||
}
|
||||
report := readStateTestRunReport(t, debugPath)
|
||||
if !report.Succeeded || report.RunID != runID || report.PipelineID != "sample" || report.OutputPath != outputPath || report.DebugPath != debugPath || report.OutputCount != 1 || report.RejectedCount != 0 || report.WarningCount != 0 || report.ValidationStatus != "approved" {
|
||||
t.Fatalf("success report = %#v", report)
|
||||
@@ -343,6 +357,45 @@ func TestRunUsesOneInjectedIdentityForDebugOutputAndManifest(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunCheckpointReuseRequiresSameEffectiveSession(t *testing.T) {
|
||||
roots := newStateTestRoots(t)
|
||||
harness := newStateTestHarness()
|
||||
run := func(extra ...string) stateTestResult {
|
||||
args := []string{"run", "sample", "--config", roots.config, "--input", roots.input, "--chunk_cache", "bypass"}
|
||||
args = append(args, extra...)
|
||||
var stdout, stderr bytes.Buffer
|
||||
return stateTestResult{code: RunWithOptions(args, &stdout, &stderr, harness.options()), stdout: stdout.String(), stderr: stderr.String()}
|
||||
}
|
||||
|
||||
if result := run(); result.code != 0 {
|
||||
t.Fatalf("initial run code=%d stdout=%q stderr=%q", result.code, result.stdout, result.stderr)
|
||||
}
|
||||
harness.mu.Lock()
|
||||
initialExtractCalls := harness.extractCalls
|
||||
harness.mu.Unlock()
|
||||
if initialExtractCalls != 1 {
|
||||
t.Fatalf("initial extract calls = %d, want 1", initialExtractCalls)
|
||||
}
|
||||
if result := run("--resume"); result.code != 0 {
|
||||
t.Fatalf("same-session resume code=%d stdout=%q stderr=%q", result.code, result.stdout, result.stderr)
|
||||
}
|
||||
harness.mu.Lock()
|
||||
reusedExtractCalls := harness.extractCalls
|
||||
harness.mu.Unlock()
|
||||
if reusedExtractCalls != initialExtractCalls {
|
||||
t.Fatalf("extract calls after same-session resume = %d, want %d", reusedExtractCalls, initialExtractCalls)
|
||||
}
|
||||
if result := run("--resume", "--session-id", "different-session"); result.code != 0 {
|
||||
t.Fatalf("different-session resume code=%d stdout=%q stderr=%q", result.code, result.stdout, result.stderr)
|
||||
}
|
||||
harness.mu.Lock()
|
||||
differentSessionExtractCalls := harness.extractCalls
|
||||
harness.mu.Unlock()
|
||||
if differentSessionExtractCalls != initialExtractCalls+1 {
|
||||
t.Fatalf("extract calls after different-session resume = %d, want %d", differentSessionExtractCalls, initialExtractCalls+1)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunWritesTerminalArtifactsForResolutionPipelineAndOutputFailures(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
|
||||
@@ -34,6 +34,8 @@ type ConcurrencyConfig struct {
|
||||
defaultedExtractWorkers int
|
||||
}
|
||||
|
||||
const defaultLLMConcurrency = 16
|
||||
|
||||
type OutputConfig struct {
|
||||
Directory string `json:"directory"`
|
||||
}
|
||||
@@ -60,9 +62,9 @@ func Default() Config {
|
||||
return Config{
|
||||
Pipelines: map[string]pipeline.PipelineProfile{},
|
||||
Concurrency: ConcurrencyConfig{
|
||||
TotalLLM: 1,
|
||||
StageWorkers: map[string]int{"extract": 1},
|
||||
defaultedExtractWorkers: 1,
|
||||
TotalLLM: defaultLLMConcurrency,
|
||||
StageWorkers: map[string]int{"extract": defaultLLMConcurrency},
|
||||
defaultedExtractWorkers: defaultLLMConcurrency,
|
||||
},
|
||||
Output: OutputConfig{Directory: "./notarius-output"},
|
||||
Cache: CacheConfig{ChunkPlans: ChunkPlanCacheConfig{Mode: pipeline.ChunkCacheAuto}},
|
||||
|
||||
@@ -14,7 +14,7 @@ import (
|
||||
|
||||
func TestDefaultReturnsDocumentedValuesAndIndependentMaps(t *testing.T) {
|
||||
first := Default()
|
||||
if first.Concurrency.TotalLLM != 1 || first.Concurrency.StageWorkers["extract"] != 1 {
|
||||
if first.Concurrency.TotalLLM != 16 || first.Concurrency.StageWorkers["extract"] != 16 {
|
||||
t.Fatalf("concurrency defaults = %#v", first.Concurrency)
|
||||
}
|
||||
if first.Output.Directory != "./notarius-output" || first.Debug.Directory != "./notarius-debug" {
|
||||
@@ -31,7 +31,7 @@ func TestDefaultReturnsDocumentedValuesAndIndependentMaps(t *testing.T) {
|
||||
first.Concurrency.StageWorkers["other"] = 100
|
||||
first.Pipelines["changed"] = pipeline.PipelineProfile{}
|
||||
second := Default()
|
||||
if second.Concurrency.StageWorkers["extract"] != 1 || len(second.Concurrency.StageWorkers) != 1 || len(second.Pipelines) != 0 {
|
||||
if second.Concurrency.StageWorkers["extract"] != 16 || len(second.Concurrency.StageWorkers) != 1 || len(second.Pipelines) != 0 {
|
||||
t.Fatalf("Default() returned state shared with an earlier result: %#v", second)
|
||||
}
|
||||
}
|
||||
@@ -45,7 +45,7 @@ func TestFileConfigMinimalVersion4AppliesOverDefaults(t *testing.T) {
|
||||
if cfg.Output.Directory != "./notarius-output" || cfg.Debug.Directory != "./notarius-debug" || cfg.Cache.ChunkPlans.Mode != pipeline.ChunkCacheAuto {
|
||||
t.Fatalf("minimal file changed unrelated defaults: %#v", cfg)
|
||||
}
|
||||
if cfg.Concurrency.TotalLLM != 1 || cfg.Concurrency.StageWorkers["extract"] != 1 || len(cfg.Pipelines) != 0 {
|
||||
if cfg.Concurrency.TotalLLM != 16 || cfg.Concurrency.StageWorkers["extract"] != 16 || len(cfg.Pipelines) != 0 {
|
||||
t.Fatalf("minimal file did not retain defaults: %#v", cfg)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -179,6 +179,30 @@ func TestWriteInvocationPreservesReasoningEffortOverrideStates(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteInvocationOmitsEmptySessionID(t *testing.T) {
|
||||
for _, sessionID := range []string{"", "session-123"} {
|
||||
bundle, err := Allocate(t.TempDir(), testBundleRunID, time.Unix(0, 42))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := bundle.Summary().WriteInvocation(Invocation{Operation: "run", SessionID: sessionID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
data, err := os.ReadFile(filepath.Join(bundle.SummaryRoot(), ArtifactInvocationMetadata))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var payload map[string]any
|
||||
if err := json.Unmarshal(data, &payload); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
value, found := payload["session_id"]
|
||||
if found != (sessionID != "") || (found && value != sessionID) {
|
||||
t.Fatalf("session found=%t value=%#v, want found=%t value=%q; JSON=%s", found, value, sessionID != "", sessionID, data)
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestSummaryWriterInternalWritesConfineArtifacts(t *testing.T) {
|
||||
bundle, err := Allocate(t.TempDir(), testBundleRunID, time.Unix(0, 42))
|
||||
if err != nil {
|
||||
|
||||
@@ -40,6 +40,7 @@ type Invocation struct {
|
||||
OnlyLanes []string `json:"only_lanes,omitempty"`
|
||||
ChunkCacheOverride string `json:"chunk_cache_override,omitempty"`
|
||||
ReasoningEffortOverride *string `json:"reasoning_effort_override,omitempty"`
|
||||
SessionID string `json:"session_id,omitempty"`
|
||||
RunID string `json:"run_id"`
|
||||
StartedAt time.Time `json:"started_at"`
|
||||
}
|
||||
|
||||
@@ -91,6 +91,11 @@ func (r *Runner) Run(ctx context.Context, input RunInput) (output RunOutput, err
|
||||
input.llmClient = input.Prepared.dependencies.LLM
|
||||
|
||||
output.Manifest = manifestFromPipeline(input)
|
||||
sessionID := strings.TrimSpace(input.SessionID)
|
||||
output.Manifest.Metadata, err = manifestMetadataWithSessionID(output.Manifest.Metadata, sessionID)
|
||||
if err != nil {
|
||||
return failOutput(output), err
|
||||
}
|
||||
output.ChunkPlan = &artifacts.ChunkPlanSummary{
|
||||
Mode: string(effectiveChunkCacheMode(input.ChunkCacheMode)), RequestedModule: input.pipeline.Chunk.Module,
|
||||
LookupStatus: "skipped", LookupReason: "chunk plan lookup skipped",
|
||||
@@ -191,11 +196,6 @@ func (r *Runner) Run(ctx context.Context, input RunInput) (output RunOutput, err
|
||||
return failOutput(output), fmt.Errorf("write source debug artifact: %w", err)
|
||||
}
|
||||
sourceInput := sourceInputMaterial(input.Path, input.RawInput)
|
||||
sessionID := resolvedSessionID(input.SessionID, doc.ID)
|
||||
output.Manifest.Metadata, err = manifestMetadataWithSessionID(output.Manifest.Metadata, sessionID)
|
||||
if err != nil {
|
||||
return failOutput(output), err
|
||||
}
|
||||
output.Manifest.SourceDigests = []string{doc.Digest}
|
||||
|
||||
chunker := input.Prepared.chunker
|
||||
@@ -896,13 +896,6 @@ func sourceInputOriginURI(inputPath string) string {
|
||||
return fileURI(inputPath)
|
||||
}
|
||||
|
||||
func resolvedSessionID(explicit string, sourceDocumentID string) string {
|
||||
if trimmed := strings.TrimSpace(explicit); trimmed != "" {
|
||||
return trimmed
|
||||
}
|
||||
return strings.TrimSpace(sourceDocumentID)
|
||||
}
|
||||
|
||||
func manifestMetadataWithSessionID(metadata map[string]any, sessionID string) (map[string]any, error) {
|
||||
out, err := cloneMetadata(metadata)
|
||||
if err != nil {
|
||||
|
||||
37
internal/framework/pipeline/runner_session_test.go
Normal file
37
internal/framework/pipeline/runner_session_test.go
Normal file
@@ -0,0 +1,37 @@
|
||||
package pipeline
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRunnerUsesProvidedSessionWithoutSourceFallback(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
session string
|
||||
wantSet bool
|
||||
want string
|
||||
}{
|
||||
{name: "provided", session: " supplied-session ", wantSet: true, want: "supplied-session"},
|
||||
{name: "empty", wantSet: false},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
prepared, _ := preparedTerminalDebugPipeline(t)
|
||||
output, err := New().Run(context.Background(), RunInput{
|
||||
Prepared: prepared,
|
||||
RawInput: []byte("input"),
|
||||
SessionID: test.session,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
value, found := output.Manifest.Metadata["session_id"]
|
||||
if found != test.wantSet {
|
||||
t.Fatalf("manifest session presence = %t, want %t; metadata = %#v", found, test.wantSet, output.Manifest.Metadata)
|
||||
}
|
||||
if found && value != test.want {
|
||||
t.Fatalf("manifest session = %#v, want %q", value, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "notarius.dnd.enemy_events",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["events"],
|
||||
"properties": {
|
||||
"events": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["name", "kind", "source_refs"],
|
||||
"properties": {
|
||||
"name": {"type": "string", "minLength": 1},
|
||||
"kind": {"type": "string", "enum": ["engaged", "killed", "fled", "captured", "incapacitated"]},
|
||||
"source_refs": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["source_id", "start_unit_id", "end_unit_id"],
|
||||
"properties": {
|
||||
"source_id": {"type": "string", "minLength": 1},
|
||||
"start_unit_id": {"type": "integer", "minimum": 1},
|
||||
"end_unit_id": {"type": "integer", "minimum": 1}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
182
internal/modules/dnd/codec/enemyevents/codec.go
Normal file
182
internal/modules/dnd/codec/enemyevents/codec.go
Normal file
@@ -0,0 +1,182 @@
|
||||
// Package enemyevents encodes durable D&D enemy-event artifacts.
|
||||
package enemyevents
|
||||
|
||||
import (
|
||||
"embed"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/candidatejson"
|
||||
)
|
||||
|
||||
const (
|
||||
SchemaID = "notarius.dnd.enemy_events"
|
||||
SchemaName = "notarius_dnd_enemy_events_v1"
|
||||
SchemaVersion = "v1"
|
||||
MediaType = "application/json"
|
||||
)
|
||||
|
||||
//go:embed assets/schemas/dnd_enemy_events.v1.json
|
||||
var schemaAssets embed.FS
|
||||
|
||||
var _ contracts.ArtifactCodec[dnd.EnemyEventList] = (*Codec)(nil)
|
||||
|
||||
type Codec struct{}
|
||||
|
||||
func New() *Codec { return &Codec{} }
|
||||
|
||||
func (c *Codec) Kind() contracts.ArtifactKind { return dnd.EnemyEventListKind }
|
||||
|
||||
func (c *Codec) Schema() contracts.ArtifactSchema {
|
||||
raw, err := schemaAssets.ReadFile("assets/schemas/dnd_enemy_events.v1.json")
|
||||
if err != nil {
|
||||
return contracts.ArtifactSchema{}
|
||||
}
|
||||
return contracts.ArtifactSchema{
|
||||
ID: SchemaID,
|
||||
Name: SchemaName,
|
||||
Version: SchemaVersion,
|
||||
JSONSchema: append([]byte(nil), raw...),
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Codec) MediaType() string { return MediaType }
|
||||
|
||||
func (c *Codec) Metadata(value dnd.EnemyEventList) map[string]any {
|
||||
return map[string]any{"event_count": len(value.Events)}
|
||||
}
|
||||
|
||||
func (c *Codec) Encode(value dnd.EnemyEventList) ([]byte, error) {
|
||||
if err := validateRequiredValueFields(value); err != nil {
|
||||
return nil, fmt.Errorf("encode dnd enemy event list: %w", err)
|
||||
}
|
||||
return c.EncodeCandidate(value)
|
||||
}
|
||||
|
||||
// EncodeCandidate provides the durable representation before semantic
|
||||
// validators have approved a value.
|
||||
func (c *Codec) EncodeCandidate(value dnd.EnemyEventList) ([]byte, error) {
|
||||
return candidatejson.EncodeCandidate("dnd enemy event list", cloneList(value))
|
||||
}
|
||||
|
||||
func (c *Codec) Decode(content []byte) (dnd.EnemyEventList, error) {
|
||||
value, err := c.DecodeCandidate(content)
|
||||
if err != nil {
|
||||
return dnd.EnemyEventList{}, err
|
||||
}
|
||||
if err := validateRequiredJSONFields(content); err != nil {
|
||||
return dnd.EnemyEventList{}, fmt.Errorf("decode dnd enemy event list: %w", err)
|
||||
}
|
||||
return value, nil
|
||||
}
|
||||
|
||||
// DecodeCandidate reads one strict durable JSON value before semantic
|
||||
// validators have approved it.
|
||||
func (c *Codec) DecodeCandidate(content []byte) (dnd.EnemyEventList, error) {
|
||||
value, err := candidatejson.DecodeCandidate[dnd.EnemyEventList]("dnd enemy event list", content)
|
||||
if err != nil {
|
||||
return dnd.EnemyEventList{}, err
|
||||
}
|
||||
return cloneList(value), nil
|
||||
}
|
||||
|
||||
func validateRequiredValueFields(value dnd.EnemyEventList) error {
|
||||
if value.Events == nil {
|
||||
return fmt.Errorf("events must be present")
|
||||
}
|
||||
for index, event := range value.Events {
|
||||
if event.SourceRefs == nil {
|
||||
return fmt.Errorf("events[%d].source_refs must be present", index)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func validateRequiredJSONFields(content []byte) error {
|
||||
var root map[string]json.RawMessage
|
||||
if err := json.Unmarshal(content, &root); err != nil || root == nil {
|
||||
return fmt.Errorf("must be a JSON object")
|
||||
}
|
||||
events, err := requiredArray(root, "events", "")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for eventIndex, rawEvent := range events {
|
||||
path := fmt.Sprintf("events[%d]", eventIndex)
|
||||
var event map[string]json.RawMessage
|
||||
if err := json.Unmarshal(rawEvent, &event); err != nil || event == nil {
|
||||
return fmt.Errorf("%s must be an object", path)
|
||||
}
|
||||
for _, field := range []string{"name", "kind"} {
|
||||
if err := requireField(event, field, path); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
refs, err := requiredArray(event, "source_refs", path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for refIndex, rawRef := range refs {
|
||||
refPath := fmt.Sprintf("%s.source_refs[%d]", path, refIndex)
|
||||
var ref map[string]json.RawMessage
|
||||
if err := json.Unmarshal(rawRef, &ref); err != nil || ref == nil {
|
||||
return fmt.Errorf("%s must be an object", refPath)
|
||||
}
|
||||
for _, field := range []string{"source_id", "start_unit_id", "end_unit_id"} {
|
||||
if err := requireField(ref, field, refPath); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func requiredArray(object map[string]json.RawMessage, field, path string) ([]json.RawMessage, error) {
|
||||
raw, err := requiredField(object, field, path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var values []json.RawMessage
|
||||
if err := json.Unmarshal(raw, &values); err != nil {
|
||||
return nil, fmt.Errorf("%s must be an array", fieldPath(path, field))
|
||||
}
|
||||
return values, nil
|
||||
}
|
||||
|
||||
func requireField(object map[string]json.RawMessage, field, path string) error {
|
||||
_, err := requiredField(object, field, path)
|
||||
return err
|
||||
}
|
||||
|
||||
func requiredField(object map[string]json.RawMessage, field, path string) (json.RawMessage, error) {
|
||||
raw, ok := object[field]
|
||||
if !ok || string(raw) == "null" {
|
||||
return nil, fmt.Errorf("%s must be present", fieldPath(path, field))
|
||||
}
|
||||
return raw, nil
|
||||
}
|
||||
|
||||
func fieldPath(path, field string) string {
|
||||
if path == "" {
|
||||
return field
|
||||
}
|
||||
return path + "." + field
|
||||
}
|
||||
|
||||
func cloneList(value dnd.EnemyEventList) dnd.EnemyEventList {
|
||||
if value.Events == nil {
|
||||
return dnd.EnemyEventList{}
|
||||
}
|
||||
cloned := dnd.EnemyEventList{Events: make([]dnd.EnemyEvent, len(value.Events))}
|
||||
for index, event := range value.Events {
|
||||
cloned.Events[index] = event
|
||||
if event.SourceRefs != nil {
|
||||
cloned.Events[index].SourceRefs = append([]source.SourceRef(nil), event.SourceRefs...)
|
||||
}
|
||||
}
|
||||
return cloned
|
||||
}
|
||||
99
internal/modules/dnd/codec/enemyevents/codec_test.go
Normal file
99
internal/modules/dnd/codec/enemyevents/codec_test.go
Normal file
@@ -0,0 +1,99 @@
|
||||
package enemyevents
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
)
|
||||
|
||||
func validList() dnd.EnemyEventList {
|
||||
return dnd.EnemyEventList{Events: []dnd.EnemyEvent{
|
||||
{Name: "Ashfang", Kind: dnd.EnemyEventKindEngaged, SourceRefs: refs(1, 2)},
|
||||
{Name: "Ashfang", Kind: dnd.EnemyEventKindFled, SourceRefs: refs(5, 6)},
|
||||
}}
|
||||
}
|
||||
|
||||
func refs(start, end int) []source.SourceRef {
|
||||
return []source.SourceRef{{SourceID: "session", StartUnitID: start, EndUnitID: end}}
|
||||
}
|
||||
|
||||
func TestCodecRoundTripAndIdentity(t *testing.T) {
|
||||
codec := New()
|
||||
value := validList()
|
||||
content, err := codec.Encode(value)
|
||||
if err != nil {
|
||||
t.Fatalf("Encode() error = %v", err)
|
||||
}
|
||||
decoded, err := codec.Decode(content)
|
||||
if err != nil || !reflect.DeepEqual(decoded, value) {
|
||||
t.Fatalf("Decode() = %#v, %v; want %#v", decoded, err, value)
|
||||
}
|
||||
|
||||
schema := codec.Schema()
|
||||
if codec.Kind() != dnd.EnemyEventListKind || codec.MediaType() != MediaType || schema.ID != SchemaID || schema.Name != SchemaName || schema.Version != SchemaVersion || !json.Valid(schema.JSONSchema) {
|
||||
t.Fatalf("codec identity/schema = %q/%q %#v", codec.Kind(), codec.MediaType(), schema)
|
||||
}
|
||||
registry := pipeline.NewArtifactCodecRegistry()
|
||||
if err := pipeline.RegisterArtifactCodec(registry, codec); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
spec, ok := registry.Spec(dnd.EnemyEventListKind)
|
||||
if !ok || spec.SchemaDigest != contracts.DigestArtifactSchema(schema) {
|
||||
t.Fatalf("registered spec = %#v, %t", spec, ok)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecRejectsStrictJSONAndMissingRequiredFields(t *testing.T) {
|
||||
validJSON := `{"events":[{"name":"Ashfang","kind":"engaged","source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]}]}`
|
||||
tests := []struct {
|
||||
name, raw, want string
|
||||
}{
|
||||
{"invalid JSON", `{`, "decode dnd enemy event list"},
|
||||
{"unknown root field", `{"events":[],"unexpected":true}`, "unknown field"},
|
||||
{"unknown event field", strings.Replace(validJSON, `"kind":"engaged"`, `"kind":"engaged","unexpected":true`, 1), "unknown field"},
|
||||
{"missing events", `{}`, "events must be present"},
|
||||
{"missing name", strings.Replace(validJSON, `"name":"Ashfang",`, "", 1), "events[0].name must be present"},
|
||||
{"missing kind", strings.Replace(validJSON, `"kind":"engaged",`, "", 1), "events[0].kind must be present"},
|
||||
{"missing refs", strings.Replace(validJSON, `,"source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]`, "", 1), "events[0].source_refs must be present"},
|
||||
{"missing source ID", strings.Replace(validJSON, `"source_id":"session",`, "", 1), "source_refs[0].source_id must be present"},
|
||||
{"trailing JSON", `{"events":[]} {}`, "multiple JSON values"},
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if _, err := New().Decode([]byte(test.raw)); err == nil || !strings.Contains(err.Error(), test.want) {
|
||||
t.Fatalf("Decode() error = %v, want %q", err, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecDefensivelyOwnsValuesAndDefersSemanticValidation(t *testing.T) {
|
||||
codec := New()
|
||||
candidate := dnd.EnemyEventList{Events: []dnd.EnemyEvent{{
|
||||
Name: " ", Kind: "unsupported", SourceRefs: []source.SourceRef{{SourceID: "", StartUnitID: 0, EndUnitID: -1}},
|
||||
}}}
|
||||
content, err := codec.EncodeCandidate(candidate)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
decoded, err := codec.Decode(content)
|
||||
if err != nil || !reflect.DeepEqual(decoded, candidate) {
|
||||
t.Fatalf("Decode() = %#v, %v; want semantic candidate preservation", decoded, err)
|
||||
}
|
||||
decoded.Events[0].SourceRefs[0].SourceID = "changed"
|
||||
if candidate.Events[0].SourceRefs[0].SourceID != "" {
|
||||
t.Fatal("Decode() retained caller-owned source references")
|
||||
}
|
||||
|
||||
first := codec.Schema()
|
||||
first.JSONSchema[0] = '['
|
||||
if second := codec.Schema(); !json.Valid(second.JSONSchema) || second.JSONSchema[0] == '[' {
|
||||
t.Fatal("Schema() returned shared bytes")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "notarius.dnd.location_occurrences",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["occurrences"],
|
||||
"properties": {
|
||||
"occurrences": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["location_id", "name", "kind", "source_refs"],
|
||||
"properties": {
|
||||
"location_id": {
|
||||
"type": "string",
|
||||
"pattern": "^location:sha256:[0-9a-f]{64}$"
|
||||
},
|
||||
"name": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"kind": {
|
||||
"type": "string",
|
||||
"enum": ["visited", "planned", "recalled", "mentioned"]
|
||||
},
|
||||
"source_refs": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["source_id", "start_unit_id", "end_unit_id"],
|
||||
"properties": {
|
||||
"source_id": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"start_unit_id": {
|
||||
"type": "integer",
|
||||
"minimum": 1
|
||||
},
|
||||
"end_unit_id": {
|
||||
"type": "integer",
|
||||
"minimum": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
126
internal/modules/dnd/codec/locationoccurrences/codec.go
Normal file
126
internal/modules/dnd/codec/locationoccurrences/codec.go
Normal file
@@ -0,0 +1,126 @@
|
||||
// Package locationoccurrences encodes durable D&D location occurrence artifacts.
|
||||
package locationoccurrences
|
||||
|
||||
import (
|
||||
"embed"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/candidatejson"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||
)
|
||||
|
||||
const (
|
||||
SchemaID = "notarius.dnd.location_occurrences"
|
||||
SchemaName = "notarius_dnd_location_occurrences_v1"
|
||||
SchemaVersion = "v1"
|
||||
MediaType = "application/json"
|
||||
)
|
||||
|
||||
//go:embed assets/schemas/dnd_location_occurrences.v1.json
|
||||
var schemaAssets embed.FS
|
||||
|
||||
var _ contracts.ArtifactCodec[dnd.LocationOccurrenceList] = (*Codec)(nil)
|
||||
|
||||
type Codec struct{}
|
||||
|
||||
func New() *Codec { return &Codec{} }
|
||||
|
||||
func (c *Codec) Kind() contracts.ArtifactKind { return dnd.LocationOccurrenceListKind }
|
||||
|
||||
func (c *Codec) Schema() contracts.ArtifactSchema {
|
||||
raw, err := schemaAssets.ReadFile("assets/schemas/dnd_location_occurrences.v1.json")
|
||||
if err != nil {
|
||||
return contracts.ArtifactSchema{}
|
||||
}
|
||||
return contracts.ArtifactSchema{
|
||||
ID: SchemaID,
|
||||
Name: SchemaName,
|
||||
Version: SchemaVersion,
|
||||
JSONSchema: append([]byte(nil), raw...),
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Codec) MediaType() string { return MediaType }
|
||||
|
||||
func (c *Codec) Metadata(value dnd.LocationOccurrenceList) map[string]any {
|
||||
return map[string]any{"occurrence_count": len(value.Occurrences)}
|
||||
}
|
||||
|
||||
func (c *Codec) Encode(value dnd.LocationOccurrenceList) ([]byte, error) {
|
||||
if err := validate(value); err != nil {
|
||||
return nil, fmt.Errorf("encode dnd location occurrence list: %w", err)
|
||||
}
|
||||
return c.EncodeCandidate(value)
|
||||
}
|
||||
|
||||
// EncodeCandidate provides the durable representation before semantic
|
||||
// validators have approved a value.
|
||||
func (c *Codec) EncodeCandidate(value dnd.LocationOccurrenceList) ([]byte, error) {
|
||||
return candidatejson.EncodeCandidate("dnd location occurrence list", value)
|
||||
}
|
||||
|
||||
func (c *Codec) Decode(content []byte) (dnd.LocationOccurrenceList, error) {
|
||||
value, err := c.DecodeCandidate(content)
|
||||
if err != nil {
|
||||
return dnd.LocationOccurrenceList{}, err
|
||||
}
|
||||
if err := validate(value); err != nil {
|
||||
return dnd.LocationOccurrenceList{}, fmt.Errorf("decode dnd location occurrence list: %w", err)
|
||||
}
|
||||
return value, nil
|
||||
}
|
||||
|
||||
// DecodeCandidate reads one strict durable JSON value before semantic
|
||||
// validators have approved it.
|
||||
func (c *Codec) DecodeCandidate(content []byte) (dnd.LocationOccurrenceList, error) {
|
||||
return candidatejson.DecodeCandidate[dnd.LocationOccurrenceList]("dnd location occurrence list", content)
|
||||
}
|
||||
|
||||
func validate(value dnd.LocationOccurrenceList) error {
|
||||
if value.Occurrences == nil {
|
||||
return fmt.Errorf("occurrences must be present")
|
||||
}
|
||||
for index, occurrence := range value.Occurrences {
|
||||
prefix := fmt.Sprintf("occurrences[%d]", index)
|
||||
if !identity.IsValidID(occurrence.LocationID) {
|
||||
return fmt.Errorf("%s.location_id must match location ID pattern", prefix)
|
||||
}
|
||||
if strings.TrimSpace(occurrence.Name) == "" {
|
||||
return fmt.Errorf("%s.name must not be empty", prefix)
|
||||
}
|
||||
if !validOccurrenceKind(occurrence.Kind) {
|
||||
return fmt.Errorf("%s.kind must be supported", prefix)
|
||||
}
|
||||
if len(occurrence.SourceRefs) == 0 {
|
||||
return fmt.Errorf("%s.source_refs must contain at least one reference", prefix)
|
||||
}
|
||||
for refIndex, ref := range occurrence.SourceRefs {
|
||||
refPrefix := fmt.Sprintf("%s.source_refs[%d]", prefix, refIndex)
|
||||
if strings.TrimSpace(ref.SourceID) == "" {
|
||||
return fmt.Errorf("%s.source_id must not be empty", refPrefix)
|
||||
}
|
||||
if ref.StartUnitID <= 0 {
|
||||
return fmt.Errorf("%s.start_unit_id must be positive", refPrefix)
|
||||
}
|
||||
if ref.EndUnitID <= 0 {
|
||||
return fmt.Errorf("%s.end_unit_id must be positive", refPrefix)
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func validOccurrenceKind(value dnd.LocationOccurrenceKind) bool {
|
||||
switch value {
|
||||
case dnd.LocationOccurrenceKindVisited,
|
||||
dnd.LocationOccurrenceKindPlanned,
|
||||
dnd.LocationOccurrenceKindRecalled,
|
||||
dnd.LocationOccurrenceKindMentioned:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
163
internal/modules/dnd/codec/locationoccurrences/codec_test.go
Normal file
163
internal/modules/dnd/codec/locationoccurrences/codec_test.go
Normal file
@@ -0,0 +1,163 @@
|
||||
package locationoccurrences
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||
)
|
||||
|
||||
func validList() dnd.LocationOccurrenceList {
|
||||
refs := []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 2}}
|
||||
return dnd.LocationOccurrenceList{Occurrences: []dnd.LocationOccurrence{{
|
||||
LocationID: identity.DeriveID("The Old Tavern", refs),
|
||||
Name: "The Old Tavern",
|
||||
Kind: dnd.LocationOccurrenceKindVisited,
|
||||
SourceRefs: refs,
|
||||
}}}
|
||||
}
|
||||
|
||||
func TestCodecMatchesMaintainedDurableFixture(t *testing.T) {
|
||||
raw, err := os.ReadFile("testdata/dnd_location_occurrences.v1.json")
|
||||
if err != nil {
|
||||
t.Fatalf("read durable fixture: %v", err)
|
||||
}
|
||||
codec := New()
|
||||
value, err := codec.Decode(raw)
|
||||
if err != nil {
|
||||
t.Fatalf("Decode() error = %v, want nil", err)
|
||||
}
|
||||
if want := validList(); !reflect.DeepEqual(value, want) {
|
||||
t.Fatalf("Decode() = %#v, want %#v", value, want)
|
||||
}
|
||||
encoded, err := codec.Encode(value)
|
||||
if err != nil {
|
||||
t.Fatalf("Encode() error = %v, want nil", err)
|
||||
}
|
||||
var compact bytes.Buffer
|
||||
if err := json.Compact(&compact, raw); err != nil {
|
||||
t.Fatalf("compact durable fixture: %v", err)
|
||||
}
|
||||
if !bytes.Equal(encoded, compact.Bytes()) {
|
||||
t.Fatalf("Encode() = %s, want %s", encoded, compact.Bytes())
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecOwnsDurableSchemaAndMetadata(t *testing.T) {
|
||||
codec := New()
|
||||
schema := codec.Schema()
|
||||
if codec.Kind() != dnd.LocationOccurrenceListKind || codec.MediaType() != MediaType {
|
||||
t.Fatalf("codec identity = %q/%q", codec.Kind(), codec.MediaType())
|
||||
}
|
||||
if schema.ID != SchemaID || schema.Name != SchemaName || schema.Version != SchemaVersion || !json.Valid(schema.JSONSchema) {
|
||||
t.Fatalf("schema = %#v, want durable location occurrence schema", schema)
|
||||
}
|
||||
var document map[string]any
|
||||
if err := json.Unmarshal(schema.JSONSchema, &document); err != nil || document["$id"] != SchemaID {
|
||||
t.Fatalf("durable schema document = %#v, %v", document, err)
|
||||
}
|
||||
registry := pipeline.NewArtifactCodecRegistry()
|
||||
if err := pipeline.RegisterArtifactCodec(registry, codec); err != nil {
|
||||
t.Fatalf("RegisterArtifactCodec() error = %v", err)
|
||||
}
|
||||
if spec, ok := registry.Spec(dnd.LocationOccurrenceListKind); !ok || spec.SchemaDigest != contracts.DigestArtifactSchema(schema) {
|
||||
t.Fatalf("registered spec = %#v, %t", spec, ok)
|
||||
}
|
||||
first := schema.JSONSchema
|
||||
first[0] = '['
|
||||
if next := codec.Schema().JSONSchema; !json.Valid(next) || next[0] == '[' {
|
||||
t.Fatal("Schema() returned shared bytes")
|
||||
}
|
||||
metadata := codec.Metadata(validList())
|
||||
metadata["other"] = true
|
||||
if next := codec.Metadata(validList()); len(next) != 1 || next["occurrence_count"] != 1 {
|
||||
t.Fatalf("Metadata() = %#v", next)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecSupportsEmptyListsAndCandidateSemanticFailures(t *testing.T) {
|
||||
codec := New()
|
||||
empty := dnd.LocationOccurrenceList{Occurrences: []dnd.LocationOccurrence{}}
|
||||
if content, err := codec.Encode(empty); err != nil || string(content) != `{"occurrences":[]}` {
|
||||
t.Fatalf("Encode() = %s, %v", content, err)
|
||||
}
|
||||
for _, candidate := range []dnd.LocationOccurrenceList{
|
||||
{},
|
||||
empty,
|
||||
{Occurrences: []dnd.LocationOccurrence{{LocationID: "bad", Name: " ", Kind: "unsupported", SourceRefs: nil}}},
|
||||
{Occurrences: []dnd.LocationOccurrence{{LocationID: "bad", Name: " ", Kind: "unsupported", SourceRefs: []source.SourceRef{}}}},
|
||||
{Occurrences: []dnd.LocationOccurrence{{LocationID: "bad", Name: " ", Kind: "unsupported", SourceRefs: []source.SourceRef{{StartUnitID: 0, EndUnitID: -1}}}}},
|
||||
} {
|
||||
content, err := codec.EncodeCandidate(candidate)
|
||||
if err != nil || !json.Valid(content) {
|
||||
t.Fatalf("EncodeCandidate() = %s, %v", content, err)
|
||||
}
|
||||
decoded, err := codec.DecodeCandidate(content)
|
||||
if err != nil || !reflect.DeepEqual(decoded, candidate) {
|
||||
t.Fatalf("DecodeCandidate() = %#v, %v, want %#v", decoded, err, candidate)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecRejectsStructuralJSONBeforeSemanticApproval(t *testing.T) {
|
||||
valid := `{"occurrences":[{"location_id":"location:sha256:0000000000000000000000000000000000000000000000000000000000000000","name":"The Tavern","kind":"visited","source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]}]}`
|
||||
for _, test := range []struct{ name, raw, want string }{
|
||||
{"malformed", `{`, "decode dnd location occurrence list"},
|
||||
{"unknown top-level", `{"occurrences":[],"unexpected":true}`, "unknown field"},
|
||||
{"unknown occurrence field", strings.Replace(valid, `"kind":"visited"`, `"kind":"visited","unexpected":true`, 1), "unknown field"},
|
||||
{"unknown source reference field", strings.Replace(valid, `"end_unit_id":1`, `"end_unit_id":1,"unexpected":true`, 1), "unknown field"},
|
||||
{"invalid type", `{"occurrences":"not-an-array"}`, "cannot unmarshal string"},
|
||||
{"trailing", `{"occurrences":[]} {}`, "multiple JSON values"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if _, err := New().DecodeCandidate([]byte(test.raw)); err == nil || !strings.Contains(err.Error(), test.want) {
|
||||
t.Fatalf("DecodeCandidate() error = %v, want %q", err, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecRejectsApprovedShapeEnumAndReferenceBoundaries(t *testing.T) {
|
||||
base := validList().Occurrences[0]
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
value dnd.LocationOccurrenceList
|
||||
want string
|
||||
}{
|
||||
{"nil occurrences", dnd.LocationOccurrenceList{}, "occurrences must be present"},
|
||||
{"invalid ID", dnd.LocationOccurrenceList{Occurrences: []dnd.LocationOccurrence{{LocationID: "bad", Name: base.Name, Kind: base.Kind, SourceRefs: base.SourceRefs}}}, "location_id must match location ID pattern"},
|
||||
{"blank name", dnd.LocationOccurrenceList{Occurrences: []dnd.LocationOccurrence{{LocationID: base.LocationID, Name: " ", Kind: base.Kind, SourceRefs: base.SourceRefs}}}, "name must not be empty"},
|
||||
{"invalid kind", dnd.LocationOccurrenceList{Occurrences: []dnd.LocationOccurrence{{LocationID: base.LocationID, Name: base.Name, Kind: "unsupported", SourceRefs: base.SourceRefs}}}, "kind must be supported"},
|
||||
{"empty source refs", dnd.LocationOccurrenceList{Occurrences: []dnd.LocationOccurrence{{LocationID: base.LocationID, Name: base.Name, Kind: base.Kind}}}, "source_refs must contain"},
|
||||
{"malformed source reference", dnd.LocationOccurrenceList{Occurrences: []dnd.LocationOccurrence{{LocationID: base.LocationID, Name: base.Name, Kind: base.Kind, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 1, EndUnitID: 0}}}}}, "end_unit_id must be positive"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if _, err := New().Encode(test.value); err == nil || !strings.Contains(err.Error(), test.want) {
|
||||
t.Fatalf("Encode() error = %v, want %q", err, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecAcceptsEveryOccurrenceKind(t *testing.T) {
|
||||
for _, kind := range []dnd.LocationOccurrenceKind{
|
||||
dnd.LocationOccurrenceKindVisited,
|
||||
dnd.LocationOccurrenceKindPlanned,
|
||||
dnd.LocationOccurrenceKindRecalled,
|
||||
dnd.LocationOccurrenceKindMentioned,
|
||||
} {
|
||||
value := validList()
|
||||
value.Occurrences[0].Kind = kind
|
||||
if _, err := New().Encode(value); err != nil {
|
||||
t.Fatalf("Encode(%q) error = %v", kind, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
12
internal/modules/dnd/codec/locationoccurrences/testdata/dnd_location_occurrences.v1.json
vendored
Normal file
12
internal/modules/dnd/codec/locationoccurrences/testdata/dnd_location_occurrences.v1.json
vendored
Normal file
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"occurrences": [
|
||||
{
|
||||
"location_id": "location:sha256:cdd2b57615b56a4e92d506cf053cd3985e11d4df4895c5383b30ec2ce9f39ed0",
|
||||
"name": "The Old Tavern",
|
||||
"kind": "visited",
|
||||
"source_refs": [
|
||||
{"source_id": "session-alpha", "start_unit_id": 1, "end_unit_id": 2}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "notarius.dnd.locations",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["locations"],
|
||||
"properties": {
|
||||
"locations": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["id", "name", "source_refs"],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string",
|
||||
"pattern": "^location:sha256:[0-9a-f]{64}$"
|
||||
},
|
||||
"name": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"source_refs": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["source_id", "start_unit_id", "end_unit_id"],
|
||||
"properties": {
|
||||
"source_id": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"start_unit_id": {
|
||||
"type": "integer",
|
||||
"minimum": 1
|
||||
},
|
||||
"end_unit_id": {
|
||||
"type": "integer",
|
||||
"minimum": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
111
internal/modules/dnd/codec/locations/codec.go
Normal file
111
internal/modules/dnd/codec/locations/codec.go
Normal file
@@ -0,0 +1,111 @@
|
||||
// Package locations encodes durable D&D location artifacts.
|
||||
package locations
|
||||
|
||||
import (
|
||||
"embed"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/candidatejson"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||
)
|
||||
|
||||
const (
|
||||
SchemaID = "notarius.dnd.locations"
|
||||
SchemaName = "notarius_dnd_locations_v1"
|
||||
SchemaVersion = "v1"
|
||||
MediaType = "application/json"
|
||||
)
|
||||
|
||||
//go:embed assets/schemas/dnd_locations.v1.json
|
||||
var schemaAssets embed.FS
|
||||
|
||||
var _ contracts.ArtifactCodec[dnd.LocationList] = (*Codec)(nil)
|
||||
|
||||
type Codec struct{}
|
||||
|
||||
func New() *Codec { return &Codec{} }
|
||||
|
||||
func (c *Codec) Kind() contracts.ArtifactKind { return dnd.LocationListKind }
|
||||
|
||||
func (c *Codec) Schema() contracts.ArtifactSchema {
|
||||
raw, err := schemaAssets.ReadFile("assets/schemas/dnd_locations.v1.json")
|
||||
if err != nil {
|
||||
return contracts.ArtifactSchema{}
|
||||
}
|
||||
return contracts.ArtifactSchema{
|
||||
ID: SchemaID,
|
||||
Name: SchemaName,
|
||||
Version: SchemaVersion,
|
||||
JSONSchema: append([]byte(nil), raw...),
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Codec) MediaType() string { return MediaType }
|
||||
|
||||
func (c *Codec) Metadata(value dnd.LocationList) map[string]any {
|
||||
return map[string]any{"location_count": len(value.Locations)}
|
||||
}
|
||||
|
||||
func (c *Codec) Encode(value dnd.LocationList) ([]byte, error) {
|
||||
if err := validate(value); err != nil {
|
||||
return nil, fmt.Errorf("encode dnd location list: %w", err)
|
||||
}
|
||||
return c.EncodeCandidate(value)
|
||||
}
|
||||
|
||||
// EncodeCandidate provides the durable representation before semantic
|
||||
// validators have approved a value.
|
||||
func (c *Codec) EncodeCandidate(value dnd.LocationList) ([]byte, error) {
|
||||
return candidatejson.EncodeCandidate("dnd location list", value)
|
||||
}
|
||||
|
||||
func (c *Codec) Decode(content []byte) (dnd.LocationList, error) {
|
||||
value, err := c.DecodeCandidate(content)
|
||||
if err != nil {
|
||||
return dnd.LocationList{}, err
|
||||
}
|
||||
if err := validate(value); err != nil {
|
||||
return dnd.LocationList{}, fmt.Errorf("decode dnd location list: %w", err)
|
||||
}
|
||||
return value, nil
|
||||
}
|
||||
|
||||
// DecodeCandidate reads one strict durable JSON value before semantic
|
||||
// validators have approved it.
|
||||
func (c *Codec) DecodeCandidate(content []byte) (dnd.LocationList, error) {
|
||||
return candidatejson.DecodeCandidate[dnd.LocationList]("dnd location list", content)
|
||||
}
|
||||
|
||||
func validate(value dnd.LocationList) error {
|
||||
if value.Locations == nil {
|
||||
return fmt.Errorf("locations must be present")
|
||||
}
|
||||
for index, location := range value.Locations {
|
||||
prefix := fmt.Sprintf("locations[%d]", index)
|
||||
if !identity.IsValidID(location.ID) {
|
||||
return fmt.Errorf("%s.id must match location ID pattern", prefix)
|
||||
}
|
||||
if strings.TrimSpace(location.Name) == "" {
|
||||
return fmt.Errorf("%s.name must not be empty", prefix)
|
||||
}
|
||||
if len(location.SourceRefs) == 0 {
|
||||
return fmt.Errorf("%s.source_refs must contain at least one reference", prefix)
|
||||
}
|
||||
for refIndex, ref := range location.SourceRefs {
|
||||
refPrefix := fmt.Sprintf("%s.source_refs[%d]", prefix, refIndex)
|
||||
if strings.TrimSpace(ref.SourceID) == "" {
|
||||
return fmt.Errorf("%s.source_id must not be empty", refPrefix)
|
||||
}
|
||||
if ref.StartUnitID <= 0 {
|
||||
return fmt.Errorf("%s.start_unit_id must be positive", refPrefix)
|
||||
}
|
||||
if ref.EndUnitID <= 0 {
|
||||
return fmt.Errorf("%s.end_unit_id must be positive", refPrefix)
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
146
internal/modules/dnd/codec/locations/codec_test.go
Normal file
146
internal/modules/dnd/codec/locations/codec_test.go
Normal file
@@ -0,0 +1,146 @@
|
||||
package locations
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||
)
|
||||
|
||||
func validList() dnd.LocationList {
|
||||
refs := []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 2}}
|
||||
return dnd.LocationList{Locations: []dnd.Location{{
|
||||
ID: identity.DeriveID("The Old Tavern", refs),
|
||||
Name: "The Old Tavern",
|
||||
SourceRefs: refs,
|
||||
}}}
|
||||
}
|
||||
|
||||
func TestCodecMatchesMaintainedDurableFixture(t *testing.T) {
|
||||
raw, err := os.ReadFile("testdata/dnd_locations.v1.json")
|
||||
if err != nil {
|
||||
t.Fatalf("read durable fixture: %v", err)
|
||||
}
|
||||
codec := New()
|
||||
value, err := codec.Decode(raw)
|
||||
if err != nil {
|
||||
t.Fatalf("Decode() error = %v, want nil", err)
|
||||
}
|
||||
if want := validList(); !reflect.DeepEqual(value, want) {
|
||||
t.Fatalf("Decode() = %#v, want %#v", value, want)
|
||||
}
|
||||
encoded, err := codec.Encode(value)
|
||||
if err != nil {
|
||||
t.Fatalf("Encode() error = %v, want nil", err)
|
||||
}
|
||||
var compact bytes.Buffer
|
||||
if err := json.Compact(&compact, raw); err != nil {
|
||||
t.Fatalf("compact durable fixture: %v", err)
|
||||
}
|
||||
if !bytes.Equal(encoded, compact.Bytes()) {
|
||||
t.Fatalf("Encode() = %s, want %s", encoded, compact.Bytes())
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecOwnsDurableSchemaAndMetadata(t *testing.T) {
|
||||
codec := New()
|
||||
schema := codec.Schema()
|
||||
if codec.Kind() != dnd.LocationListKind || codec.MediaType() != MediaType {
|
||||
t.Fatalf("codec identity = %q/%q", codec.Kind(), codec.MediaType())
|
||||
}
|
||||
if schema.ID != SchemaID || schema.Name != SchemaName || schema.Version != SchemaVersion || !json.Valid(schema.JSONSchema) {
|
||||
t.Fatalf("schema = %#v, want durable location schema", schema)
|
||||
}
|
||||
var document map[string]any
|
||||
if err := json.Unmarshal(schema.JSONSchema, &document); err != nil || document["$id"] != SchemaID {
|
||||
t.Fatalf("durable schema document = %#v, %v", document, err)
|
||||
}
|
||||
registry := pipeline.NewArtifactCodecRegistry()
|
||||
if err := pipeline.RegisterArtifactCodec(registry, codec); err != nil {
|
||||
t.Fatalf("RegisterArtifactCodec() error = %v", err)
|
||||
}
|
||||
if spec, ok := registry.Spec(dnd.LocationListKind); !ok || spec.SchemaDigest != contracts.DigestArtifactSchema(schema) {
|
||||
t.Fatalf("registered spec = %#v, %t", spec, ok)
|
||||
}
|
||||
first := schema.JSONSchema
|
||||
first[0] = '['
|
||||
if next := codec.Schema().JSONSchema; !json.Valid(next) || next[0] == '[' {
|
||||
t.Fatal("Schema() returned shared bytes")
|
||||
}
|
||||
metadata := codec.Metadata(validList())
|
||||
metadata["other"] = true
|
||||
if next := codec.Metadata(validList()); len(next) != 1 || next["location_count"] != 1 {
|
||||
t.Fatalf("Metadata() = %#v", next)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecSupportsEmptyListsAndCandidateSemanticFailures(t *testing.T) {
|
||||
codec := New()
|
||||
empty := dnd.LocationList{Locations: []dnd.Location{}}
|
||||
if content, err := codec.Encode(empty); err != nil || string(content) != `{"locations":[]}` {
|
||||
t.Fatalf("Encode() = %s, %v", content, err)
|
||||
}
|
||||
for _, candidate := range []dnd.LocationList{
|
||||
{},
|
||||
empty,
|
||||
{Locations: []dnd.Location{{ID: "bad", Name: " ", SourceRefs: nil}}},
|
||||
{Locations: []dnd.Location{{ID: "bad", Name: " ", SourceRefs: []source.SourceRef{}}}},
|
||||
{Locations: []dnd.Location{{ID: "bad", Name: " ", SourceRefs: []source.SourceRef{{StartUnitID: 0, EndUnitID: -1}}}}},
|
||||
} {
|
||||
content, err := codec.EncodeCandidate(candidate)
|
||||
if err != nil || !json.Valid(content) {
|
||||
t.Fatalf("EncodeCandidate() = %s, %v", content, err)
|
||||
}
|
||||
decoded, err := codec.DecodeCandidate(content)
|
||||
if err != nil || !reflect.DeepEqual(decoded, candidate) {
|
||||
t.Fatalf("DecodeCandidate() = %#v, %v, want %#v", decoded, err, candidate)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecRejectsStructuralJSONBeforeSemanticApproval(t *testing.T) {
|
||||
valid := `{"locations":[{"id":"location:sha256:0000000000000000000000000000000000000000000000000000000000000000","name":"The Tavern","source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]}]}`
|
||||
for _, test := range []struct{ name, raw, want string }{
|
||||
{"malformed", `{`, "decode dnd location list"},
|
||||
{"unknown top-level", `{"locations":[],"unexpected":true}`, "unknown field"},
|
||||
{"unknown location field", strings.Replace(valid, `"name":"The Tavern"`, `"name":"The Tavern","unexpected":true`, 1), "unknown field"},
|
||||
{"unknown source reference field", strings.Replace(valid, `"end_unit_id":1`, `"end_unit_id":1,"unexpected":true`, 1), "unknown field"},
|
||||
{"invalid type", `{"locations":"not-an-array"}`, "cannot unmarshal string"},
|
||||
{"trailing", `{"locations":[]} {}`, "multiple JSON values"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if _, err := New().DecodeCandidate([]byte(test.raw)); err == nil || !strings.Contains(err.Error(), test.want) {
|
||||
t.Fatalf("DecodeCandidate() error = %v, want %q", err, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodecRejectsApprovedShapeBoundaries(t *testing.T) {
|
||||
base := validList().Locations[0]
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
value dnd.LocationList
|
||||
want string
|
||||
}{
|
||||
{"nil locations", dnd.LocationList{}, "locations must be present"},
|
||||
{"invalid ID", dnd.LocationList{Locations: []dnd.Location{{ID: "bad", Name: base.Name, SourceRefs: base.SourceRefs}}}, "id must match location ID pattern"},
|
||||
{"blank name", dnd.LocationList{Locations: []dnd.Location{{ID: base.ID, Name: " ", SourceRefs: base.SourceRefs}}}, "name must not be empty"},
|
||||
{"empty source refs", dnd.LocationList{Locations: []dnd.Location{{ID: base.ID, Name: base.Name, SourceRefs: nil}}}, "source_refs must contain"},
|
||||
{"malformed source reference", dnd.LocationList{Locations: []dnd.Location{{ID: base.ID, Name: base.Name, SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 0, EndUnitID: 1}}}}}, "start_unit_id must be positive"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if _, err := New().Encode(test.value); err == nil || !strings.Contains(err.Error(), test.want) {
|
||||
t.Fatalf("Encode() error = %v, want %q", err, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
11
internal/modules/dnd/codec/locations/testdata/dnd_locations.v1.json
vendored
Normal file
11
internal/modules/dnd/codec/locations/testdata/dnd_locations.v1.json
vendored
Normal file
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"locations": [
|
||||
{
|
||||
"id": "location:sha256:cdd2b57615b56a4e92d506cf053cd3985e11d4df4895c5383b30ec2ce9f39ed0",
|
||||
"name": "The Old Tavern",
|
||||
"source_refs": [
|
||||
{"source_id": "session-alpha", "start_unit_id": 1, "end_unit_id": 2}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
107
internal/modules/dnd/enemyevents/enemyevents.go
Normal file
107
internal/modules/dnd/enemyevents/enemyevents.go
Normal file
@@ -0,0 +1,107 @@
|
||||
// Package enemyevents owns canonical ordering and identity policy for D&D
|
||||
// enemy-event artifacts.
|
||||
package enemyevents
|
||||
|
||||
import (
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
// NormalizeDisplay returns the durable display form for an enemy subject.
|
||||
func NormalizeDisplay(value string) string { return identity.NormalizeDisplay(value) }
|
||||
|
||||
// ComparisonKey returns the shared D&D identity key for an enemy subject.
|
||||
func ComparisonKey(value string) string { return identity.ComparisonKey(NormalizeDisplay(value)) }
|
||||
|
||||
// SupportedKind reports whether kind belongs to the durable enemy-event vocabulary.
|
||||
func SupportedKind(kind dnd.EnemyEventKind) bool {
|
||||
_, ok := KindRank(kind)
|
||||
return ok
|
||||
}
|
||||
|
||||
// KindRank returns the explicit chronological tie-break order for event kinds.
|
||||
func KindRank(kind dnd.EnemyEventKind) (int, bool) {
|
||||
switch kind {
|
||||
case dnd.EnemyEventKindEngaged:
|
||||
return 0, true
|
||||
case dnd.EnemyEventKindIncapacitated:
|
||||
return 1, true
|
||||
case dnd.EnemyEventKindCaptured:
|
||||
return 2, true
|
||||
case dnd.EnemyEventKindFled:
|
||||
return 3, true
|
||||
case dnd.EnemyEventKindKilled:
|
||||
return 4, true
|
||||
default:
|
||||
return 0, false
|
||||
}
|
||||
}
|
||||
|
||||
// SourceRefsEqual reports whether two reference sequences are equal after
|
||||
// canonical ordering and exact duplicate removal.
|
||||
func SourceRefsEqual(order shared.SourceRefOrder, left, right []source.SourceRef) bool {
|
||||
left = order.Canonicalize(left)
|
||||
right = order.Canonicalize(right)
|
||||
if (left == nil) != (right == nil) || len(left) != len(right) {
|
||||
return false
|
||||
}
|
||||
for index := range left {
|
||||
if left[index] != right[index] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// ExactEqual reports whether events have the same subject identity, kind, and
|
||||
// complete canonical evidence sequence.
|
||||
func ExactEqual(order shared.SourceRefOrder, left, right dnd.EnemyEvent) bool {
|
||||
return ComparisonKey(left.Name) == ComparisonKey(right.Name) &&
|
||||
left.Kind == right.Kind &&
|
||||
SourceRefsEqual(order, left.SourceRefs, right.SourceRefs)
|
||||
}
|
||||
|
||||
// Less defines the canonical stable event order. Invalid source references
|
||||
// remain comparable through SourceRefOrder's literal fallback so malformed
|
||||
// candidates can still be sorted for later validation.
|
||||
func Less(order shared.SourceRefOrder, left, right dnd.EnemyEvent) bool {
|
||||
leftPosition, leftHasEvidence := order.EarliestValid(left.SourceRefs)
|
||||
rightPosition, rightHasEvidence := order.EarliestValid(right.SourceRefs)
|
||||
if leftHasEvidence != rightHasEvidence {
|
||||
return leftHasEvidence
|
||||
}
|
||||
if leftHasEvidence && leftPosition != rightPosition {
|
||||
return leftPosition < rightPosition
|
||||
}
|
||||
if leftKey, rightKey := ComparisonKey(left.Name), ComparisonKey(right.Name); leftKey != rightKey {
|
||||
return leftKey < rightKey
|
||||
}
|
||||
if leftName, rightName := NormalizeDisplay(left.Name), NormalizeDisplay(right.Name); leftName != rightName {
|
||||
return leftName < rightName
|
||||
}
|
||||
if leftRank, leftKnown := KindRank(left.Kind); leftKnown {
|
||||
if rightRank, rightKnown := KindRank(right.Kind); rightKnown && leftRank != rightRank {
|
||||
return leftRank < rightRank
|
||||
} else if !rightKnown {
|
||||
return true
|
||||
}
|
||||
} else if _, rightKnown := KindRank(right.Kind); rightKnown {
|
||||
return false
|
||||
}
|
||||
if left.Kind != right.Kind {
|
||||
return left.Kind < right.Kind
|
||||
}
|
||||
return sourceRefsLess(order, order.Canonicalize(left.SourceRefs), order.Canonicalize(right.SourceRefs))
|
||||
}
|
||||
|
||||
func sourceRefsLess(order shared.SourceRefOrder, left, right []source.SourceRef) bool {
|
||||
for index := 0; index < len(left) && index < len(right); index++ {
|
||||
if left[index] == right[index] {
|
||||
continue
|
||||
}
|
||||
return order.Less(left[index], right[index])
|
||||
}
|
||||
return len(left) < len(right)
|
||||
}
|
||||
90
internal/modules/dnd/enemyevents/enemyevents_test.go
Normal file
90
internal/modules/dnd/enemyevents/enemyevents_test.go
Normal file
@@ -0,0 +1,90 @@
|
||||
package enemyevents
|
||||
|
||||
import (
|
||||
"sort"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
func TestSupportedKindAndRank(t *testing.T) {
|
||||
tests := []struct {
|
||||
kind dnd.EnemyEventKind
|
||||
rank int
|
||||
}{
|
||||
{dnd.EnemyEventKindEngaged, 0},
|
||||
{dnd.EnemyEventKindIncapacitated, 1},
|
||||
{dnd.EnemyEventKindCaptured, 2},
|
||||
{dnd.EnemyEventKindFled, 3},
|
||||
{dnd.EnemyEventKindKilled, 4},
|
||||
}
|
||||
for _, test := range tests {
|
||||
if rank, ok := KindRank(test.kind); !ok || rank != test.rank || !SupportedKind(test.kind) {
|
||||
t.Fatalf("kind %q = (%d, %t), supported %t", test.kind, rank, ok, SupportedKind(test.kind))
|
||||
}
|
||||
}
|
||||
if _, ok := KindRank("unknown"); ok || SupportedKind("unknown") {
|
||||
t.Fatal("unsupported kind was accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestLessOrdersChronologyThenKind(t *testing.T) {
|
||||
order := testOrder()
|
||||
events := []dnd.EnemyEvent{
|
||||
{Name: "Ashfang", Kind: dnd.EnemyEventKindKilled, SourceRefs: refs(20)},
|
||||
{Name: "Ashfang", Kind: dnd.EnemyEventKindFled, SourceRefs: refs(20)},
|
||||
{Name: "Ashfang", Kind: dnd.EnemyEventKindCaptured, SourceRefs: refs(20)},
|
||||
{Name: "Ashfang", Kind: dnd.EnemyEventKindIncapacitated, SourceRefs: refs(20)},
|
||||
{Name: "Ashfang", Kind: dnd.EnemyEventKindEngaged, SourceRefs: refs(20)},
|
||||
{Name: "Later", Kind: dnd.EnemyEventKindEngaged, SourceRefs: refs(30)},
|
||||
{Name: "Earlier", Kind: dnd.EnemyEventKindKilled, SourceRefs: refs(10)},
|
||||
}
|
||||
sort.SliceStable(events, func(left, right int) bool { return Less(order, events[left], events[right]) })
|
||||
want := []dnd.EnemyEventKind{
|
||||
dnd.EnemyEventKindKilled,
|
||||
dnd.EnemyEventKindEngaged,
|
||||
dnd.EnemyEventKindIncapacitated,
|
||||
dnd.EnemyEventKindCaptured,
|
||||
dnd.EnemyEventKindFled,
|
||||
dnd.EnemyEventKindKilled,
|
||||
dnd.EnemyEventKindEngaged,
|
||||
}
|
||||
for index, kind := range want {
|
||||
if events[index].Kind != kind {
|
||||
t.Fatalf("event %d kind = %q, want %q", index, events[index].Kind, kind)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestExactEqualUsesSubjectIdentityAndCanonicalEvidence(t *testing.T) {
|
||||
order := testOrder()
|
||||
first := dnd.EnemyEvent{Name: " ASHFANG ", Kind: dnd.EnemyEventKindFled, SourceRefs: []source.SourceRef{
|
||||
{SourceID: "session", StartUnitID: 30, EndUnitID: 30},
|
||||
{SourceID: "session", StartUnitID: 10, EndUnitID: 10},
|
||||
{SourceID: "session", StartUnitID: 10, EndUnitID: 10},
|
||||
}}
|
||||
second := dnd.EnemyEvent{Name: "Ashfang", Kind: dnd.EnemyEventKindFled, SourceRefs: []source.SourceRef{
|
||||
{SourceID: "session", StartUnitID: 10, EndUnitID: 10},
|
||||
{SourceID: "session", StartUnitID: 30, EndUnitID: 30},
|
||||
}}
|
||||
if !ExactEqual(order, first, second) || !SourceRefsEqual(order, first.SourceRefs, second.SourceRefs) {
|
||||
t.Fatal("canonical duplicate identity was not recognized")
|
||||
}
|
||||
differentKind := second
|
||||
differentKind.Kind = dnd.EnemyEventKindKilled
|
||||
differentEvidence := second
|
||||
differentEvidence.SourceRefs = []source.SourceRef{{SourceID: "session", StartUnitID: 20, EndUnitID: 20}}
|
||||
if ExactEqual(order, first, differentKind) || ExactEqual(order, first, differentEvidence) {
|
||||
t.Fatal("distinct event was treated as an exact duplicate")
|
||||
}
|
||||
}
|
||||
|
||||
func refs(unitID int) []source.SourceRef {
|
||||
return []source.SourceRef{{SourceID: "session", StartUnitID: unitID, EndUnitID: unitID}}
|
||||
}
|
||||
|
||||
func testOrder() shared.SourceRefOrder {
|
||||
return shared.NewSourceRefOrder(&source.SourceDocument{ID: "session", Units: []source.SourceUnit{{ID: 10}, {ID: 20}, {ID: 30}}})
|
||||
}
|
||||
6
internal/modules/dnd/extract/enemyevents/assets.go
Normal file
6
internal/modules/dnd/extract/enemyevents/assets.go
Normal file
@@ -0,0 +1,6 @@
|
||||
package enemyevents
|
||||
|
||||
import "embed"
|
||||
|
||||
//go:embed assets/schemas/dnd_enemy_events_llm.v1.json assets/prompts/*.yaml assets/prompts/*.md
|
||||
var embeddedAssets embed.FS
|
||||
@@ -0,0 +1,55 @@
|
||||
id: dnd.enemy_events
|
||||
version: "v1"
|
||||
default_profile: dnd-extraction
|
||||
inputs:
|
||||
- name: transcript
|
||||
required: true
|
||||
content_type: application/json
|
||||
- name: players
|
||||
required: false
|
||||
content_type: text/plain
|
||||
- name: party
|
||||
required: false
|
||||
content_type: text/plain
|
||||
- name: glossary
|
||||
required: false
|
||||
content_type: text/plain
|
||||
- name: npcs
|
||||
required: true
|
||||
content_type: application/json
|
||||
- name: combat_turns
|
||||
required: true
|
||||
content_type: application/json
|
||||
- name: npc_interactions
|
||||
required: true
|
||||
content_type: application/json
|
||||
messages:
|
||||
- role: system
|
||||
content_file: ./sharedassets/common-dnd-system.md
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-identity.md
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-references.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-transcript.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-extraction-evidence.md
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-npcs.md
|
||||
- role: user
|
||||
content_file: ./grounding.md
|
||||
- role: user
|
||||
content_file: ./task.md
|
||||
- role: user
|
||||
content_file: ./instructions.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
output:
|
||||
format: json
|
||||
validation_mode: json_schema
|
||||
schema_path: dnd_enemy_events_llm.v1.json
|
||||
repair_attempts: 0
|
||||
@@ -0,0 +1,12 @@
|
||||
Compact combat grounding is supplied below. It can guide attention and
|
||||
disambiguation, but it is not evidence. Do not derive an event, subject,
|
||||
outcome, or source range from either list. The current transcript alone must
|
||||
directly establish every returned event.
|
||||
|
||||
Combat-turn grounding:
|
||||
|
||||
{{ input "combat_turns" }}
|
||||
|
||||
Named combat-opponent grounding:
|
||||
|
||||
{{ input "npc_interactions" }}
|
||||
@@ -0,0 +1,14 @@
|
||||
Exclude party members, allies, neutral observers, mentioned-but-absent enemies,
|
||||
hazards, traps, environmental effects, uncertain allegiance, table talk,
|
||||
planning, hypotheses, recaps outside this passage, and downstream inference.
|
||||
|
||||
Do not infer an engagement or outcome from initiative, turn absence, damage,
|
||||
defeat, movement, a scene ending, combat-opponent grounding, or any auxiliary
|
||||
artifact. Auxiliary inputs can guide attention but cannot prove or supply an
|
||||
event. Cite only narrow current-transcript ranges that establish each event.
|
||||
|
||||
Return the `events` array even when no enemy event is established. Every event
|
||||
must contain only `name`, `kind`, and `source_refs`. Use exactly one kind:
|
||||
`engaged`, `killed`, `fled`, `captured`, or `incapacitated`. Each source range
|
||||
uses integer `start_unit_id` and `end_unit_id`; omit `source_id` because
|
||||
Notarius assigns the current source identity.
|
||||
@@ -0,0 +1,20 @@
|
||||
Extract Dungeons & Dragons enemy events from the supplied combat transcript.
|
||||
|
||||
Return an `engaged` event only when the transcript directly establishes that a
|
||||
subject is actively opposing the party in combat. Return `killed`, `fled`,
|
||||
`captured`, or `incapacitated` only when the transcript explicitly establishes
|
||||
that outcome. An outcome may share evidence with an engagement, and a later
|
||||
engagement or outcome for the same subject remains a separate observation.
|
||||
Emit at most one engagement for the same subject in this combat scene.
|
||||
|
||||
For `killed`, direct death or killing is required. For `fled`, the subject must
|
||||
explicitly escape, retreat, or leave combat to avoid continued engagement. For
|
||||
`captured`, the subject must be explicitly taken prisoner or secured under the
|
||||
party's control. For `incapacitated`, the subject must be explicitly unable to
|
||||
continue acting without being established as killed or captured.
|
||||
|
||||
Use a normalized NPC registry spelling when the transcript identifies that
|
||||
named NPC. A hostile creature without a registry entry is allowed. For unnamed
|
||||
individuals or groups, use only the narrowest transcript-grounded label, such
|
||||
as `Orcs`, `One orc`, or `Remaining orcs`; never invent member names, IDs, or
|
||||
quantities.
|
||||
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "notarius.dnd.enemy_events.llm",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["events"],
|
||||
"properties": {
|
||||
"events": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["name", "kind", "source_refs"],
|
||||
"properties": {
|
||||
"name": {"type": "string"},
|
||||
"kind": {"type": "string"},
|
||||
"source_refs": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["start_unit_id", "end_unit_id"],
|
||||
"properties": {
|
||||
"start_unit_id": {"type": "integer"},
|
||||
"end_unit_id": {"type": "integer"}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
210
internal/modules/dnd/extract/enemyevents/extractor.go
Normal file
210
internal/modules/dnd/extract/enemyevents/extractor.go
Normal file
@@ -0,0 +1,210 @@
|
||||
package enemyevents
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
sceneregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/scenedescriptions/registry"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
const (
|
||||
Key = "dnd/enemy-events"
|
||||
mappingPolicy = "dnd.enemy_events.extract_mapping.v1"
|
||||
sceneGatePolicy = "dnd.enemy_events.scene_gate.v1"
|
||||
)
|
||||
|
||||
var requiredCapabilities = []string{
|
||||
"chunks",
|
||||
"source.transcript",
|
||||
}
|
||||
|
||||
var providedCapabilities = []string{
|
||||
"dnd.enemy_events",
|
||||
}
|
||||
|
||||
var _ contracts.Extractor[dnd.EnemyEventList] = (*Extractor)(nil)
|
||||
var _ contracts.ManifestMetadataProvider = (*Extractor)(nil)
|
||||
var _ pipeline.CheckpointFingerprintProvider = (*Extractor)(nil)
|
||||
|
||||
type Options struct{}
|
||||
|
||||
type Extractor struct {
|
||||
llm contracts.StructuredLLMClient
|
||||
grounding *groundingResolver
|
||||
|
||||
promptSHA string
|
||||
responseSchemaSHA string
|
||||
}
|
||||
|
||||
func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contracts.ReferenceSet) (*Extractor, error) {
|
||||
if llmClient == nil {
|
||||
return nil, extractorErrorf("LLM client must not be nil")
|
||||
}
|
||||
if len(references) > 1 {
|
||||
return nil, extractorErrorf("at most one reference set may be supplied")
|
||||
}
|
||||
var referenceSet contracts.ReferenceSet
|
||||
if len(references) == 1 {
|
||||
referenceSet = references[0]
|
||||
}
|
||||
grounding, err := newGroundingResolver(referenceSet)
|
||||
if err != nil {
|
||||
return nil, extractorErrorf("prepare grounding: %w", err)
|
||||
}
|
||||
promptSHA, err := promptAssetMetadata()
|
||||
if err != nil {
|
||||
return nil, extractorErrorf("load prompt metadata: %w", err)
|
||||
}
|
||||
responseSchema, err := loadResponseSchema()
|
||||
if err != nil {
|
||||
return nil, extractorErrorf("load response schema: %w", err)
|
||||
}
|
||||
return &Extractor{
|
||||
llm: llmClient,
|
||||
grounding: grounding,
|
||||
promptSHA: promptSHA,
|
||||
responseSchemaSHA: responseSchema.SHA256,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (e *Extractor) Key() string { return Key }
|
||||
|
||||
func (e *Extractor) ReferenceSlots() []contracts.ReferenceSlot { return referenceSlots() }
|
||||
|
||||
func (e *Extractor) ManifestMetadata() map[string]any {
|
||||
if e == nil {
|
||||
return nil
|
||||
}
|
||||
return map[string]any{
|
||||
"prompt_id": PromptID,
|
||||
"prompt_version": SchemaVersion,
|
||||
"prompt_sha256": e.promptSHA,
|
||||
"mapping_policy": mappingPolicy,
|
||||
"scene_gate_policy": sceneGatePolicy,
|
||||
"response_schema_key": string(ResponseSchemaKey),
|
||||
"response_schema_id": ResponseSchemaID,
|
||||
"response_schema_name": ResponseSchemaName,
|
||||
"response_schema_version": SchemaVersion,
|
||||
"response_schema_sha256": e.responseSchemaSHA,
|
||||
}
|
||||
}
|
||||
|
||||
func (e *Extractor) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
|
||||
if e == nil {
|
||||
return nil
|
||||
}
|
||||
return []pipeline.CheckpointFingerprint{
|
||||
{Name: "prompt", Value: e.promptSHA},
|
||||
{Name: "response_schema", Value: e.responseSchemaSHA},
|
||||
{Name: "mapping_policy", Value: mappingPolicy},
|
||||
{Name: "scene_gate_policy", Value: sceneGatePolicy},
|
||||
}
|
||||
}
|
||||
|
||||
func (e *Extractor) Extract(ctx context.Context, req contracts.TypedExtractionRequest) (contracts.TypedExtractionResult[dnd.EnemyEventList], error) {
|
||||
if e == nil {
|
||||
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("extractor must not be nil")
|
||||
}
|
||||
if e.llm == nil {
|
||||
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("LLM client must not be nil")
|
||||
}
|
||||
if e.grounding == nil {
|
||||
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("grounding resolver must not be nil")
|
||||
}
|
||||
sourceInput, err := shared.PrepareChunkExtraction(ctx, req)
|
||||
if err != nil {
|
||||
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("%w", err)
|
||||
}
|
||||
match, err := e.grounding.SceneMatch(req.References, req.Chunk)
|
||||
if err != nil {
|
||||
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("resolve scene eligibility: %w", err)
|
||||
}
|
||||
switch match.State {
|
||||
case sceneregistry.MatchExact:
|
||||
if match.Kind != dnd.SceneKindCombat {
|
||||
return emptyResult(), nil
|
||||
}
|
||||
case sceneregistry.MatchMissing, sceneregistry.MatchMismatched:
|
||||
return unavailableSceneResult(), nil
|
||||
default:
|
||||
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("unsupported scene eligibility match state %q", match.State)
|
||||
}
|
||||
grounding, err := e.grounding.Resolve(req.References)
|
||||
if err != nil {
|
||||
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("resolve enemy-event grounding: %w", err)
|
||||
}
|
||||
inputs := shared.PromptInputs(sourceInput, req.References)
|
||||
for name, input := range grounding.PromptInputs() {
|
||||
inputs[name] = input
|
||||
}
|
||||
var response extractionResponse
|
||||
if _, err := e.llm.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
|
||||
StageName: Key,
|
||||
PromptID: PromptID,
|
||||
PromptVersion: SchemaVersion,
|
||||
ProfileID: req.LLMProfile,
|
||||
SessionID: req.SessionID,
|
||||
Inputs: inputs,
|
||||
}, &response); err != nil {
|
||||
return contracts.TypedExtractionResult[dnd.EnemyEventList]{}, extractorErrorf("complete structured output: %w", err)
|
||||
}
|
||||
return contracts.TypedExtractionResult[dnd.EnemyEventList]{
|
||||
Value: canonicalEnemyEventList(response, shared.NewSourceRefOrder(req.Source), req.Source.ID),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func emptyResult() contracts.TypedExtractionResult[dnd.EnemyEventList] {
|
||||
return contracts.TypedExtractionResult[dnd.EnemyEventList]{Value: dnd.EnemyEventList{Events: []dnd.EnemyEvent{}}}
|
||||
}
|
||||
|
||||
func unavailableSceneResult() contracts.TypedExtractionResult[dnd.EnemyEventList] {
|
||||
result := emptyResult()
|
||||
result.Warnings = []contracts.Warning{{
|
||||
Scope: SceneDescriptionReferenceSlot,
|
||||
ReasonCode: "scene_classification_unavailable",
|
||||
Message: "No exact scene classification was available; enemy-event extraction was skipped.",
|
||||
}}
|
||||
return result
|
||||
}
|
||||
|
||||
func ModuleSpec() pipeline.ModuleSpec {
|
||||
return pipeline.ModuleSpec{
|
||||
Key: Key,
|
||||
Stage: pipeline.StageExtract,
|
||||
ExecutionClass: contracts.ExecutionClassLLMBacked,
|
||||
Requires: append([]string(nil), requiredCapabilities...),
|
||||
Provides: append([]string(nil), providedCapabilities...),
|
||||
ArtifactKind: dnd.EnemyEventListKind,
|
||||
ReferenceSlots: referenceSlots(),
|
||||
}
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ExtractorRegistry) error {
|
||||
return pipeline.RegisterExtractorBuilder(registry, ModuleSpec(), validateOptions, func(request pipeline.BuildRequest) (contracts.Extractor[dnd.EnemyEventList], error) {
|
||||
options, err := DecodeOptions(request.Options)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return New(request.Dependencies.LLM, options, request.References)
|
||||
})
|
||||
}
|
||||
|
||||
func validateOptions(options map[string]any) error {
|
||||
_, err := DecodeOptions(options)
|
||||
return err
|
||||
}
|
||||
|
||||
func DecodeOptions(options map[string]any) (Options, error) {
|
||||
if err := pipeline.RejectUnknownOptions(options); err != nil {
|
||||
return Options{}, extractorErrorf("%w", err)
|
||||
}
|
||||
return Options{}, nil
|
||||
}
|
||||
|
||||
func extractorErrorf(format string, args ...any) error {
|
||||
return fmt.Errorf("dnd enemy events extractor: "+format, args...)
|
||||
}
|
||||
229
internal/modules/dnd/extract/enemyevents/extractor_test.go
Normal file
229
internal/modules/dnd/extract/enemyevents/extractor_test.go
Normal file
@@ -0,0 +1,229 @@
|
||||
package enemyevents
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
)
|
||||
|
||||
func TestExtractMapsEnemyEventsInSourceOrder(t *testing.T) {
|
||||
client := &fakeEnemyEventsLLMClient{response: extractionResponse{Events: []enemyEventResponse{
|
||||
{Name: "Ashfang", Kind: "killed", SourceRefs: []enemySourceRefResponse{{StartUnitID: 4, EndUnitID: 4}}},
|
||||
{Name: "Ashfang", Kind: "engaged", SourceRefs: []enemySourceRefResponse{{StartUnitID: 1, EndUnitID: 1}, {StartUnitID: 1, EndUnitID: 1}}},
|
||||
{Name: "Orcs", Kind: "fled", SourceRefs: []enemySourceRefResponse{{StartUnitID: 3, EndUnitID: 3}}},
|
||||
{Name: "One orc", Kind: "captured", SourceRefs: []enemySourceRefResponse{{StartUnitID: 2, EndUnitID: 2}}},
|
||||
{Name: "Ashfang", Kind: "incapacitated", SourceRefs: []enemySourceRefResponse{{StartUnitID: 2, EndUnitID: 2}}},
|
||||
}}}
|
||||
|
||||
result, err := newEnemyExtractor(t, client).Extract(context.Background(), enemyExtractionRequest(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := []dnd.EnemyEventKind{result.Value.Events[0].Kind, result.Value.Events[1].Kind, result.Value.Events[2].Kind, result.Value.Events[3].Kind, result.Value.Events[4].Kind}; !reflect.DeepEqual(got, []dnd.EnemyEventKind{
|
||||
dnd.EnemyEventKindEngaged,
|
||||
dnd.EnemyEventKindCaptured,
|
||||
dnd.EnemyEventKindIncapacitated,
|
||||
dnd.EnemyEventKindFled,
|
||||
dnd.EnemyEventKindKilled,
|
||||
}) {
|
||||
t.Fatalf("event order = %#v", got)
|
||||
}
|
||||
if refs := result.Value.Events[0].SourceRefs; !reflect.DeepEqual(refs, []source.SourceRef{{SourceID: "combat-session", StartUnitID: 1, EndUnitID: 1}}) {
|
||||
t.Fatalf("canonical source refs = %#v", refs)
|
||||
}
|
||||
if len(client.requests) != 1 {
|
||||
t.Fatalf("LLM calls = %d, want 1", len(client.requests))
|
||||
}
|
||||
request := client.requests[0]
|
||||
if request.StageName != Key || request.PromptID != PromptID || request.PromptVersion != SchemaVersion || request.ProfileID != "enemy-profile" || request.SessionID != "session-123" {
|
||||
t.Fatalf("LLM request identity = %#v", request)
|
||||
}
|
||||
if got := string(request.Inputs[CombatTurnReferenceSlot].Content); !strings.Contains(got, `"actor":"Ashfang"`) || strings.Contains(got, "source_ref") {
|
||||
t.Fatalf("combat grounding = %s", got)
|
||||
}
|
||||
if got := string(request.Inputs[NPCInteractionReferenceSlot].Content); !strings.Contains(got, `"kind":"combat_opponent"`) || strings.Contains(got, "Aria") {
|
||||
t.Fatalf("interaction grounding = %s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractPreservesSemanticCandidatesAndResponseOwnership(t *testing.T) {
|
||||
client := &fakeEnemyEventsLLMClient{response: extractionResponse{Events: []enemyEventResponse{{
|
||||
Name: " ", Kind: "unsupported", SourceRefs: []enemySourceRefResponse{{StartUnitID: 99, EndUnitID: 0}},
|
||||
}}}}
|
||||
result, err := newEnemyExtractor(t, client).Extract(context.Background(), enemyExtractionRequest(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
event := result.Value.Events[0]
|
||||
if event.Name != " " || event.Kind != "unsupported" || event.SourceRefs[0] != (source.SourceRef{SourceID: "combat-session", StartUnitID: 99}) {
|
||||
t.Fatalf("semantic candidate = %#v", event)
|
||||
}
|
||||
result.Value.Events[0].SourceRefs[0].StartUnitID = 7
|
||||
if client.response.Events[0].SourceRefs[0].StartUnitID != 99 {
|
||||
t.Fatal("mapped event aliases model-owned source references")
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractSkipsModelForIneligibleScenes(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
kind dnd.SceneKind
|
||||
change func(*source.Chunk)
|
||||
wantWarning bool
|
||||
}{
|
||||
{name: "non-combat", kind: dnd.SceneKindNarrative},
|
||||
{name: "missing", kind: dnd.SceneKindCombat, change: func(chunk *source.Chunk) { chunk.ID = "other" }, wantWarning: true},
|
||||
{name: "mismatched", kind: dnd.SceneKindCombat, change: func(chunk *source.Chunk) { chunk.Ref.EndUnitID++ }, wantWarning: true},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
client := &fakeEnemyEventsLLMClient{}
|
||||
request := enemyExtractionRequest(t)
|
||||
request.References = groundingReferences(t, "Ashfang", test.kind)
|
||||
if test.change != nil {
|
||||
test.change(request.Chunk)
|
||||
}
|
||||
result, err := newEnemyExtractor(t, client).Extract(context.Background(), request)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(client.requests) != 0 || result.Value.Events == nil || len(result.Value.Events) != 0 {
|
||||
t.Fatalf("result = %#v, calls = %d", result, len(client.requests))
|
||||
}
|
||||
if test.wantWarning {
|
||||
if len(result.Warnings) != 1 || result.Warnings[0].ReasonCode != "scene_classification_unavailable" || result.Warnings[0].Scope != SceneDescriptionReferenceSlot {
|
||||
t.Fatalf("warnings = %#v", result.Warnings)
|
||||
}
|
||||
} else if len(result.Warnings) != 0 {
|
||||
t.Fatalf("warnings = %#v", result.Warnings)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractReportsRequiredGroundingAndProviderFailures(t *testing.T) {
|
||||
client := &fakeEnemyEventsLLMClient{}
|
||||
request := enemyExtractionRequest(t)
|
||||
request.References = withoutSlot(request.References, NPCRegistryReferenceSlot)
|
||||
if _, err := (&Extractor{llm: client, grounding: mustGroundingResolver(t, request.References)}).Extract(context.Background(), request); err == nil || !strings.Contains(err.Error(), "NPC registry") {
|
||||
t.Fatalf("Extract() error = %v, want required grounding context", err)
|
||||
}
|
||||
|
||||
provider := &fakeEnemyEventsLLMClient{err: errors.New("provider unavailable")}
|
||||
if _, err := newEnemyExtractor(t, provider).Extract(context.Background(), enemyExtractionRequest(t)); err == nil || !strings.Contains(err.Error(), "complete structured output") || !strings.Contains(err.Error(), "provider unavailable") {
|
||||
t.Fatalf("Extract() error = %v, want provider context", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestConstructorSpecOptionsAndSafeMetadata(t *testing.T) {
|
||||
if _, err := New(nil, Options{}); err == nil || !strings.Contains(err.Error(), "LLM client") {
|
||||
t.Fatalf("New(nil) error = %v", err)
|
||||
}
|
||||
if _, err := New(&fakeEnemyEventsLLMClient{}, Options{}, contracts.ReferenceSet{}, contracts.ReferenceSet{}); err == nil || !strings.Contains(err.Error(), "at most one") {
|
||||
t.Fatalf("New() error = %v", err)
|
||||
}
|
||||
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil || !strings.Contains(err.Error(), "unknown option") {
|
||||
t.Fatalf("DecodeOptions() error = %v", err)
|
||||
}
|
||||
|
||||
first := ModuleSpec()
|
||||
first.Requires[0] = "changed"
|
||||
first.ReferenceSlots[0].AcceptedMediaTypes[0] = "changed"
|
||||
second := ModuleSpec()
|
||||
if second.Requires[0] != "chunks" || second.ArtifactKind != dnd.EnemyEventListKind || second.ExecutionClass != contracts.ExecutionClassLLMBacked || second.ReferenceSlots[0].AcceptedMediaTypes[0] == "changed" {
|
||||
t.Fatalf("ModuleSpec() reused mutable state: %#v", second)
|
||||
}
|
||||
registry := pipeline.NewExtractorRegistry()
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if spec, ok := registry.Spec(Key); !ok || spec.Key != Key || spec.ArtifactKind != dnd.EnemyEventListKind {
|
||||
t.Fatalf("registered spec = %#v, %t", spec, ok)
|
||||
}
|
||||
|
||||
extractor := newEnemyExtractor(t, &fakeEnemyEventsLLMClient{}, groundingReferences(t, "Ashfang", dnd.SceneKindCombat))
|
||||
metadata := extractor.ManifestMetadata()
|
||||
encoded, err := json.Marshal(metadata)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(string(encoded), "Ashfang") || metadata["mapping_policy"] != mappingPolicy || metadata["scene_gate_policy"] != sceneGatePolicy {
|
||||
t.Fatalf("unsafe or incomplete metadata = %s", encoded)
|
||||
}
|
||||
fingerprints := extractor.CheckpointFingerprints()
|
||||
if len(fingerprints) != 4 || fingerprints[0].Value != metadata["prompt_sha256"] || fingerprints[1].Value != metadata["response_schema_sha256"] || fingerprints[2].Value != mappingPolicy || fingerprints[3].Value != sceneGatePolicy {
|
||||
t.Fatalf("fingerprints = %#v", fingerprints)
|
||||
}
|
||||
}
|
||||
|
||||
func newEnemyExtractor(t *testing.T, client contracts.StructuredLLMClient, references ...contracts.ReferenceSet) *Extractor {
|
||||
t.Helper()
|
||||
if len(references) == 0 {
|
||||
references = []contracts.ReferenceSet{groundingReferences(t, "Ashfang", dnd.SceneKindCombat)}
|
||||
}
|
||||
extractor, err := New(client, Options{}, references...)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return extractor
|
||||
}
|
||||
|
||||
func mustGroundingResolver(t *testing.T, references contracts.ReferenceSet) *groundingResolver {
|
||||
t.Helper()
|
||||
resolver, err := newGroundingResolver(references)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return resolver
|
||||
}
|
||||
|
||||
func enemyExtractionRequest(t *testing.T) contracts.TypedExtractionRequest {
|
||||
t.Helper()
|
||||
chunk := combatChunk()
|
||||
chunk.Units = []source.SourceUnit{{ID: 1}, {ID: 2}, {ID: 3}, {ID: 4}}
|
||||
chunk.MediaType = "application/json"
|
||||
chunk.Content = []byte(`{"id":"combat-scene","units":[1,2,3,4]}`)
|
||||
return contracts.TypedExtractionRequest{
|
||||
Source: &source.SourceDocument{ID: "combat-session", Units: append([]source.SourceUnit(nil), chunk.Units...)},
|
||||
Chunk: chunk,
|
||||
SourceInput: contracts.NewLLMInputMaterial("source", "application/json", chunk.Content, digest(chunk.Content), "file:///combat-session.json"),
|
||||
References: groundingReferences(t, "Ashfang", dnd.SceneKindCombat),
|
||||
LLMProfile: "enemy-profile",
|
||||
SessionID: "session-123",
|
||||
}
|
||||
}
|
||||
|
||||
type fakeEnemyEventsLLMClient struct {
|
||||
response extractionResponse
|
||||
err error
|
||||
requests []contracts.StructuredCompletionRequest
|
||||
}
|
||||
|
||||
func (client *fakeEnemyEventsLLMClient) CompleteStructured(_ context.Context, request contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
|
||||
client.requests = append(client.requests, cloneEnemyRequest(request))
|
||||
if client.err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, client.err
|
||||
}
|
||||
target, ok := out.(*extractionResponse)
|
||||
if !ok {
|
||||
return contracts.StructuredCompletionResponse{}, errors.New("unexpected output target")
|
||||
}
|
||||
*target = client.response
|
||||
content, err := json.Marshal(client.response)
|
||||
if err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, err
|
||||
}
|
||||
return contracts.StructuredCompletionResponse{Content: content}, nil
|
||||
}
|
||||
|
||||
func cloneEnemyRequest(request contracts.StructuredCompletionRequest) contracts.StructuredCompletionRequest {
|
||||
request.Inputs = request.Inputs.Clone()
|
||||
return request
|
||||
}
|
||||
293
internal/modules/dnd/extract/enemyevents/grounding.go
Normal file
293
internal/modules/dnd/extract/enemyevents/grounding.go
Normal file
@@ -0,0 +1,293 @@
|
||||
// Package enemyevents prepares validated D&D combat grounding for enemy-event
|
||||
// extraction.
|
||||
package enemyevents
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"mime"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
combatturncodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/combatturns"
|
||||
interactioncodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcinteractions"
|
||||
npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs"
|
||||
scenecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
|
||||
npcregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/registry"
|
||||
sceneregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/scenedescriptions/registry"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
const (
|
||||
NPCRegistryReferenceSlot = npcregistry.ReferenceSlot
|
||||
SceneDescriptionReferenceSlot = sceneregistry.ReferenceSlot
|
||||
CombatTurnReferenceSlot = "combat_turns"
|
||||
NPCInteractionReferenceSlot = "npc_interactions"
|
||||
ReferenceMaxBytes = 1048576
|
||||
promptProjectionMediaType = "application/json"
|
||||
)
|
||||
|
||||
var referenceSlotDescriptions = shared.ReferenceSlotDescriptions{
|
||||
Glossary: "Optional campaign glossary reference material used only for disambiguation.",
|
||||
Party: "Optional party roster reference material used only for disambiguation.",
|
||||
Players: "Optional player list reference material used only for disambiguation.",
|
||||
Roster: "Deprecated alias for party roster reference material used only for disambiguation.",
|
||||
}
|
||||
|
||||
func referenceSlots() []contracts.ReferenceSlot {
|
||||
slots := shared.ReferenceSlots(referenceSlotDescriptions)
|
||||
slots = append(slots,
|
||||
contracts.ReferenceSlot{
|
||||
Name: NPCRegistryReferenceSlot,
|
||||
Description: "Required normalized NPC registry used only for enemy-subject grounding, never as event evidence.",
|
||||
Required: true,
|
||||
AcceptedMediaTypes: []string{npccodec.MediaType},
|
||||
AcceptedArtifactKinds: []contracts.ArtifactKind{dnd.NPCListKind},
|
||||
MaxBytes: ReferenceMaxBytes,
|
||||
},
|
||||
contracts.ReferenceSlot{
|
||||
Name: SceneDescriptionReferenceSlot,
|
||||
Description: "Required scene descriptions used only to determine exact combat eligibility, never as event evidence.",
|
||||
Required: true,
|
||||
AcceptedMediaTypes: []string{scenecodec.MediaType},
|
||||
AcceptedArtifactKinds: []contracts.ArtifactKind{dnd.SceneDescriptionListKind},
|
||||
MaxBytes: ReferenceMaxBytes,
|
||||
},
|
||||
contracts.ReferenceSlot{
|
||||
Name: CombatTurnReferenceSlot,
|
||||
Description: "Required combat-turn artifact used only as source-free enemy-event grounding.",
|
||||
Required: true,
|
||||
AcceptedMediaTypes: []string{combatturncodec.MediaType},
|
||||
AcceptedArtifactKinds: []contracts.ArtifactKind{dnd.CombatTurnListKind},
|
||||
MaxBytes: ReferenceMaxBytes,
|
||||
},
|
||||
contracts.ReferenceSlot{
|
||||
Name: NPCInteractionReferenceSlot,
|
||||
Description: "Required NPC-interaction artifact used only as source-free enemy-event grounding.",
|
||||
Required: true,
|
||||
AcceptedMediaTypes: []string{interactioncodec.MediaType},
|
||||
AcceptedArtifactKinds: []contracts.ArtifactKind{dnd.NPCInteractionListKind},
|
||||
MaxBytes: ReferenceMaxBytes,
|
||||
},
|
||||
)
|
||||
sort.Slice(slots, func(left, right int) bool { return slots[left].Name < slots[right].Name })
|
||||
return contracts.CloneReferenceSlots(slots)
|
||||
}
|
||||
|
||||
// groundingResolver retains only validated, compact construction-time views.
|
||||
// Per-operation generated references are decoded when supplied and never
|
||||
// become static metadata.
|
||||
type groundingResolver struct {
|
||||
npcs *npcregistry.Resolver
|
||||
scenes *sceneregistry.Resolver
|
||||
|
||||
combatTurns *contracts.LLMInputMaterial
|
||||
npcInteractions *contracts.LLMInputMaterial
|
||||
}
|
||||
|
||||
type grounding struct {
|
||||
npcInput contracts.LLMInputMaterial
|
||||
combatTurnInput contracts.LLMInputMaterial
|
||||
npcInteractionInput contracts.LLMInputMaterial
|
||||
}
|
||||
|
||||
func newGroundingResolver(references contracts.ReferenceSet) (*groundingResolver, error) {
|
||||
npcs, err := npcregistry.NewResolver(references)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("prepare NPC registry grounding: %w", err)
|
||||
}
|
||||
scenes, err := sceneregistry.NewResolver(references)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("prepare scene eligibility: %w", err)
|
||||
}
|
||||
combatTurns, err := prepareCombatTurnInput(references)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
npcInteractions, err := prepareNPCInteractionInput(references)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &groundingResolver{
|
||||
npcs: npcs,
|
||||
scenes: scenes,
|
||||
combatTurns: combatTurns,
|
||||
npcInteractions: npcInteractions,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (r *groundingResolver) Resolve(references contracts.ReferenceSet) (grounding, error) {
|
||||
if r == nil {
|
||||
return grounding{}, fmt.Errorf("grounding resolver must not be nil")
|
||||
}
|
||||
npcs, err := r.npcs.Resolve(references)
|
||||
if err != nil {
|
||||
return grounding{}, fmt.Errorf("resolve NPC registry grounding: %w", err)
|
||||
}
|
||||
if !npcs.Bound() {
|
||||
return grounding{}, fmt.Errorf("NPC registry reference is required")
|
||||
}
|
||||
combatTurns, err := resolveInput(references, CombatTurnReferenceSlot, r.combatTurns, prepareCombatTurnInput)
|
||||
if err != nil {
|
||||
return grounding{}, err
|
||||
}
|
||||
npcInteractions, err := resolveInput(references, NPCInteractionReferenceSlot, r.npcInteractions, prepareNPCInteractionInput)
|
||||
if err != nil {
|
||||
return grounding{}, err
|
||||
}
|
||||
return grounding{
|
||||
npcInput: npcs.PromptInput(),
|
||||
combatTurnInput: combatTurns,
|
||||
npcInteractionInput: npcInteractions,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (r *groundingResolver) SceneMatch(references contracts.ReferenceSet, chunk *source.Chunk) (sceneregistry.ChunkMatch, error) {
|
||||
if r == nil {
|
||||
return sceneregistry.ChunkMatch{}, fmt.Errorf("grounding resolver must not be nil")
|
||||
}
|
||||
scenes, err := r.resolveScenes(references)
|
||||
if err != nil {
|
||||
return sceneregistry.ChunkMatch{}, err
|
||||
}
|
||||
return scenes.Match(chunk), nil
|
||||
}
|
||||
|
||||
func (r *groundingResolver) resolveScenes(references contracts.ReferenceSet) (*sceneregistry.Registry, error) {
|
||||
scenes, err := r.scenes.Resolve(references)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("resolve scene eligibility: %w", err)
|
||||
}
|
||||
if !scenes.Bound() {
|
||||
return nil, fmt.Errorf("scene descriptions reference is required")
|
||||
}
|
||||
return scenes, nil
|
||||
}
|
||||
|
||||
func resolveInput(references contracts.ReferenceSet, slot string, seeded *contracts.LLMInputMaterial, prepare func(contracts.ReferenceSet) (*contracts.LLMInputMaterial, error)) (contracts.LLMInputMaterial, error) {
|
||||
if _, ok := references.Slots[slot]; ok {
|
||||
value, err := prepare(references)
|
||||
if err != nil {
|
||||
return contracts.LLMInputMaterial{}, err
|
||||
}
|
||||
if value == nil {
|
||||
return contracts.LLMInputMaterial{}, fmt.Errorf("%s reference is required", slot)
|
||||
}
|
||||
return value.Clone(), nil
|
||||
}
|
||||
if seeded == nil {
|
||||
return contracts.LLMInputMaterial{}, fmt.Errorf("%s reference is required", slot)
|
||||
}
|
||||
return seeded.Clone(), nil
|
||||
}
|
||||
|
||||
func (g grounding) PromptInputs() contracts.LLMInputSet {
|
||||
return contracts.LLMInputSet{
|
||||
NPCRegistryReferenceSlot: g.npcInput.Clone(),
|
||||
CombatTurnReferenceSlot: g.combatTurnInput.Clone(),
|
||||
NPCInteractionReferenceSlot: g.npcInteractionInput.Clone(),
|
||||
}
|
||||
}
|
||||
|
||||
func prepareCombatTurnInput(references contracts.ReferenceSet) (*contracts.LLMInputMaterial, error) {
|
||||
item, ok, err := referenceItem(references, CombatTurnReferenceSlot, combatturncodec.MediaType)
|
||||
if err != nil || !ok {
|
||||
return nil, err
|
||||
}
|
||||
value, err := combatturncodec.New().Decode(item.Content)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("decode combat-turn grounding: invalid approved combat-turn JSON")
|
||||
}
|
||||
content, err := json.Marshal(struct {
|
||||
CombatTurns []combatTurnProjection `json:"combat_turns"`
|
||||
}{CombatTurns: projectCombatTurns(value.CombatTurns)})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("encode combat-turn grounding: %w", err)
|
||||
}
|
||||
return newPromptInput(CombatTurnReferenceSlot, content), nil
|
||||
}
|
||||
|
||||
func prepareNPCInteractionInput(references contracts.ReferenceSet) (*contracts.LLMInputMaterial, error) {
|
||||
item, ok, err := referenceItem(references, NPCInteractionReferenceSlot, interactioncodec.MediaType)
|
||||
if err != nil || !ok {
|
||||
return nil, err
|
||||
}
|
||||
value, err := interactioncodec.New().Decode(item.Content)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("decode NPC-interaction grounding: invalid approved NPC-interaction JSON")
|
||||
}
|
||||
content, err := json.Marshal(struct {
|
||||
Interactions []npcInteractionProjection `json:"npc_interactions"`
|
||||
}{Interactions: projectNPCInteractions(value.Interactions)})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("encode NPC-interaction grounding: %w", err)
|
||||
}
|
||||
return newPromptInput(NPCInteractionReferenceSlot, content), nil
|
||||
}
|
||||
|
||||
func referenceItem(references contracts.ReferenceSet, slotName, expectedMediaType string) (contracts.ReferenceItem, bool, error) {
|
||||
slot, ok := references.Slots[slotName]
|
||||
if !ok {
|
||||
return contracts.ReferenceItem{}, false, nil
|
||||
}
|
||||
if len(slot.Items) == 0 {
|
||||
return contracts.ReferenceItem{}, false, nil
|
||||
}
|
||||
if len(slot.Items) != 1 {
|
||||
return contracts.ReferenceItem{}, false, fmt.Errorf("reference slot %q must contain exactly one item", slotName)
|
||||
}
|
||||
item := slot.Items[0]
|
||||
mediaType, _, err := mime.ParseMediaType(item.MediaType)
|
||||
if err != nil {
|
||||
return contracts.ReferenceItem{}, false, fmt.Errorf("reference slot %q item media type is invalid", slotName)
|
||||
}
|
||||
if !strings.EqualFold(mediaType, expectedMediaType) {
|
||||
return contracts.ReferenceItem{}, false, fmt.Errorf("reference slot %q item media type must be %s", slotName, expectedMediaType)
|
||||
}
|
||||
if len(item.Content) > ReferenceMaxBytes {
|
||||
return contracts.ReferenceItem{}, false, fmt.Errorf("reference slot %q item is %d bytes, limit %d", slotName, len(item.Content), ReferenceMaxBytes)
|
||||
}
|
||||
return item, true, nil
|
||||
}
|
||||
|
||||
type combatTurnProjection struct {
|
||||
Actor string `json:"actor"`
|
||||
TurnKind dnd.CombatTurnKind `json:"turn_kind"`
|
||||
}
|
||||
|
||||
func projectCombatTurns(turns []dnd.CombatTurn) []combatTurnProjection {
|
||||
if turns == nil {
|
||||
return nil
|
||||
}
|
||||
projection := make([]combatTurnProjection, len(turns))
|
||||
for index, turn := range turns {
|
||||
projection[index] = combatTurnProjection{Actor: turn.Actor, TurnKind: turn.TurnKind}
|
||||
}
|
||||
return projection
|
||||
}
|
||||
|
||||
type npcInteractionProjection struct {
|
||||
Name string `json:"name"`
|
||||
Kind dnd.NPCInteractionKind `json:"kind"`
|
||||
}
|
||||
|
||||
func projectNPCInteractions(interactions []dnd.NPCInteraction) []npcInteractionProjection {
|
||||
projection := make([]npcInteractionProjection, 0, len(interactions))
|
||||
for _, interaction := range interactions {
|
||||
if interaction.Kind == dnd.NPCInteractionKindCombatOpponent {
|
||||
projection = append(projection, npcInteractionProjection{Name: interaction.Name, Kind: interaction.Kind})
|
||||
}
|
||||
}
|
||||
return projection
|
||||
}
|
||||
|
||||
func newPromptInput(name string, content []byte) *contracts.LLMInputMaterial {
|
||||
sum := sha256.Sum256(content)
|
||||
material := contracts.NewLLMInputMaterial(name, promptProjectionMediaType, content, "sha256:"+hex.EncodeToString(sum[:]), "")
|
||||
return &material
|
||||
}
|
||||
279
internal/modules/dnd/extract/enemyevents/grounding_test.go
Normal file
279
internal/modules/dnd/extract/enemyevents/grounding_test.go
Normal file
@@ -0,0 +1,279 @@
|
||||
package enemyevents
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
combatturncodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/combatturns"
|
||||
interactioncodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcinteractions"
|
||||
npccodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/npcs"
|
||||
scenecodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/scenedescriptions"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/npcs/identity"
|
||||
sceneregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/scenedescriptions/registry"
|
||||
)
|
||||
|
||||
func TestReferenceSlotsDescribeRequiredTypedArtifacts(t *testing.T) {
|
||||
slots := referenceSlots()
|
||||
if len(slots) != 8 {
|
||||
t.Fatalf("ReferenceSlots() count = %d, want 8", len(slots))
|
||||
}
|
||||
byName := make(map[string]contracts.ReferenceSlot, len(slots))
|
||||
for index, slot := range slots {
|
||||
if index > 0 && slots[index-1].Name > slot.Name {
|
||||
t.Fatalf("ReferenceSlots() is not sorted: %#v", slots)
|
||||
}
|
||||
byName[slot.Name] = slot
|
||||
}
|
||||
for _, want := range []struct {
|
||||
name string
|
||||
kind contracts.ArtifactKind
|
||||
}{
|
||||
{NPCRegistryReferenceSlot, dnd.NPCListKind},
|
||||
{SceneDescriptionReferenceSlot, dnd.SceneDescriptionListKind},
|
||||
{CombatTurnReferenceSlot, dnd.CombatTurnListKind},
|
||||
{NPCInteractionReferenceSlot, dnd.NPCInteractionListKind},
|
||||
} {
|
||||
slot, ok := byName[want.name]
|
||||
if !ok || !slot.Required || slot.MaxBytes != ReferenceMaxBytes || len(slot.AcceptedMediaTypes) != 1 || slot.AcceptedMediaTypes[0] != "application/json" || len(slot.AcceptedArtifactKinds) != 1 || slot.AcceptedArtifactKinds[0] != want.kind {
|
||||
t.Fatalf("slot %q = %#v", want.name, slot)
|
||||
}
|
||||
}
|
||||
|
||||
slots[0].AcceptedMediaTypes[0] = "changed"
|
||||
if referenceSlots()[0].AcceptedMediaTypes[0] == "changed" {
|
||||
t.Fatal("ReferenceSlots() returned caller-owned storage")
|
||||
}
|
||||
}
|
||||
|
||||
func TestGroundingProducesExactSourceFreePromptInputs(t *testing.T) {
|
||||
resolver, err := newGroundingResolver(groundingReferences(t, "Ashfang", dnd.SceneKindCombat))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
resolved, err := resolver.Resolve(contracts.ReferenceSet{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
inputs := resolved.PromptInputs()
|
||||
want := map[string]string{
|
||||
NPCRegistryReferenceSlot: `{"npcs":[{"name":"Ashfang"}]}`,
|
||||
CombatTurnReferenceSlot: `{"combat_turns":[{"actor":"Ashfang","turn_kind":"turn"},{"actor":"Aria","turn_kind":"reaction"}]}`,
|
||||
NPCInteractionReferenceSlot: `{"npc_interactions":[{"name":"Ashfang","kind":"combat_opponent"}]}`,
|
||||
}
|
||||
if len(inputs) != len(want) {
|
||||
t.Fatalf("PromptInputs() = %#v", inputs)
|
||||
}
|
||||
for name, content := range want {
|
||||
input, ok := inputs[name]
|
||||
if !ok || string(input.Content) != content || input.Digest != digest([]byte(content)) || input.OriginURI != "" || input.MediaType != "application/json" {
|
||||
t.Fatalf("PromptInputs()[%q] = %#v, want %q", name, input, content)
|
||||
}
|
||||
for _, forbidden := range []string{"source_ref", "npc-ashfang", "origin", "summary", "combat-session"} {
|
||||
if strings.Contains(string(input.Content), forbidden) {
|
||||
t.Fatalf("PromptInputs()[%q] leaked %q: %s", name, forbidden, input.Content)
|
||||
}
|
||||
}
|
||||
}
|
||||
if _, ok := inputs[SceneDescriptionReferenceSlot]; ok {
|
||||
t.Fatal("scene descriptions were rendered as prompt grounding")
|
||||
}
|
||||
|
||||
inputs[CombatTurnReferenceSlot] = contracts.NewLLMInputMaterial("changed", "text/plain", []byte("changed"), "", "")
|
||||
if next := resolved.PromptInputs()[CombatTurnReferenceSlot]; string(next.Content) != want[CombatTurnReferenceSlot] {
|
||||
t.Fatal("PromptInputs() did not return a defensive copy")
|
||||
}
|
||||
}
|
||||
|
||||
func TestGroundingResolvesGeneratedReferencesAndSceneEligibility(t *testing.T) {
|
||||
prepared := groundingReferences(t, "Ashfang", dnd.SceneKindCombat)
|
||||
resolver, err := newGroundingResolver(prepared)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
generated := groundingReferences(t, "Grimjaw", dnd.SceneKindNarrative)
|
||||
resolved, err := resolver.Resolve(generated)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := string(resolved.PromptInputs()[NPCRegistryReferenceSlot].Content); got != `{"npcs":[{"name":"Grimjaw"}]}` {
|
||||
t.Fatalf("generated NPC projection = %s", got)
|
||||
}
|
||||
match, err := resolver.SceneMatch(generated, combatChunk())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if match != (sceneregistry.ChunkMatch{State: sceneregistry.MatchExact, Kind: dnd.SceneKindNarrative}) {
|
||||
t.Fatalf("generated scene match = %#v", match)
|
||||
}
|
||||
match, err = resolver.SceneMatch(generated, &source.Chunk{ID: "other", Ref: combatChunk().Ref})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if match.State != sceneregistry.MatchMissing {
|
||||
t.Fatal("missing scene was not reported")
|
||||
}
|
||||
mismatched := combatChunk()
|
||||
mismatched.Ref.EndUnitID++
|
||||
match, err = resolver.SceneMatch(generated, mismatched)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if match.State != sceneregistry.MatchMismatched {
|
||||
t.Fatal("mismatched scene was not reported")
|
||||
}
|
||||
|
||||
static, err := resolver.Resolve(contracts.ReferenceSet{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := string(static.PromptInputs()[NPCRegistryReferenceSlot].Content); got != `{"npcs":[{"name":"Ashfang"}]}` {
|
||||
t.Fatalf("static NPC projection changed after generated resolution: %s", got)
|
||||
}
|
||||
match, err = resolver.SceneMatch(contracts.ReferenceSet{}, combatChunk())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if match != (sceneregistry.ChunkMatch{State: sceneregistry.MatchExact, Kind: dnd.SceneKindCombat}) {
|
||||
t.Fatalf("static scene match = %#v", match)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGroundingRejectsMissingAndInvalidReferences(t *testing.T) {
|
||||
valid := groundingReferences(t, "Ashfang", dnd.SceneKindCombat)
|
||||
for _, test := range []struct {
|
||||
slot string
|
||||
want string
|
||||
}{
|
||||
{NPCRegistryReferenceSlot, "NPC registry"},
|
||||
{SceneDescriptionReferenceSlot, "scene descriptions"},
|
||||
{CombatTurnReferenceSlot, CombatTurnReferenceSlot},
|
||||
{NPCInteractionReferenceSlot, NPCInteractionReferenceSlot},
|
||||
} {
|
||||
t.Run("missing "+test.slot, func(t *testing.T) {
|
||||
resolver, err := newGroundingResolver(withoutSlot(valid, test.slot))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if test.slot == SceneDescriptionReferenceSlot {
|
||||
_, err = resolver.SceneMatch(contracts.ReferenceSet{}, combatChunk())
|
||||
} else {
|
||||
_, err = resolver.Resolve(contracts.ReferenceSet{})
|
||||
}
|
||||
if err == nil || !strings.Contains(err.Error(), test.want) {
|
||||
t.Fatalf("grounding error = %v, want missing %q reference", err, test.slot)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
set contracts.ReferenceSet
|
||||
}{
|
||||
{"multiple items", replaceSlot(valid, CombatTurnReferenceSlot, contracts.ResolvedReferenceSlot{Items: []contracts.ReferenceItem{{MediaType: "application/json"}, {MediaType: "application/json"}}})},
|
||||
{"malformed durable content", replaceItem(valid, NPCInteractionReferenceSlot, contracts.ReferenceItem{MediaType: "application/json", Content: []byte(`{}`)})},
|
||||
{"wrong media type", replaceItem(valid, CombatTurnReferenceSlot, contracts.ReferenceItem{MediaType: "text/plain", Content: valid.Slots[CombatTurnReferenceSlot].Items[0].Content})},
|
||||
{"oversize", replaceItem(valid, CombatTurnReferenceSlot, contracts.ReferenceItem{MediaType: "application/json", Content: make([]byte, ReferenceMaxBytes+1)})},
|
||||
}
|
||||
resolver, err := newGroundingResolver(valid)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if _, err := resolver.Resolve(test.set); err == nil {
|
||||
t.Fatal("Resolve() error = nil")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func groundingReferences(t *testing.T, enemy string, sceneKind dnd.SceneKind) contracts.ReferenceSet {
|
||||
t.Helper()
|
||||
npcContent, err := npccodec.New().Encode(dnd.NPCList{NPCs: []dnd.NPC{{
|
||||
ID: identity.DeriveID(enemy),
|
||||
Name: enemy,
|
||||
SourceRefs: []source.SourceRef{{SourceID: "combat-session", StartUnitID: 1, EndUnitID: 1}},
|
||||
}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sceneContent, err := scenecodec.New().Encode(dnd.SceneDescriptionList{Scenes: []dnd.SceneDescription{{
|
||||
ID: "combat-scene",
|
||||
SourceRef: source.SourceRef{SourceID: "combat-session", StartUnitID: 1, EndUnitID: 4},
|
||||
Kind: sceneKind,
|
||||
Title: "Combat session",
|
||||
Summary: "A detailed scene summary that must not reach the prompt.",
|
||||
}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
turnContent, err := combatturncodec.New().Encode(dnd.CombatTurnList{CombatTurns: []dnd.CombatTurn{
|
||||
{Actor: enemy, TurnKind: dnd.CombatTurnKindTurn, SourceRefs: []source.SourceRef{{SourceID: "combat-session", StartUnitID: 1, EndUnitID: 1}}},
|
||||
{Actor: "Aria", TurnKind: dnd.CombatTurnKindReaction, SourceRefs: []source.SourceRef{{SourceID: "combat-session", StartUnitID: 2, EndUnitID: 2}}},
|
||||
}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
interactionContent, err := interactioncodec.New().Encode(dnd.NPCInteractionList{Interactions: []dnd.NPCInteraction{
|
||||
{Name: enemy, Kind: dnd.NPCInteractionKindCombatOpponent, SourceRefs: []source.SourceRef{{SourceID: "combat-session", StartUnitID: 1, EndUnitID: 1}}},
|
||||
{Name: "Aria", Kind: dnd.NPCInteractionKindCombatAlly, SourceRefs: []source.SourceRef{{SourceID: "combat-session", StartUnitID: 2, EndUnitID: 2}}},
|
||||
}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
|
||||
NPCRegistryReferenceSlot: {Items: []contracts.ReferenceItem{newReferenceItem(NPCRegistryReferenceSlot, npcContent)}},
|
||||
SceneDescriptionReferenceSlot: {Items: []contracts.ReferenceItem{newReferenceItem(SceneDescriptionReferenceSlot, sceneContent)}},
|
||||
CombatTurnReferenceSlot: {Items: []contracts.ReferenceItem{newReferenceItem(CombatTurnReferenceSlot, turnContent)}},
|
||||
NPCInteractionReferenceSlot: {Items: []contracts.ReferenceItem{newReferenceItem(NPCInteractionReferenceSlot, interactionContent)}},
|
||||
}}
|
||||
}
|
||||
|
||||
func newReferenceItem(slot string, content []byte) contracts.ReferenceItem {
|
||||
return contracts.ReferenceItem{
|
||||
SlotName: slot,
|
||||
MediaType: "application/json",
|
||||
Content: append([]byte(nil), content...),
|
||||
Digest: digest(content),
|
||||
Origin: contracts.ReferenceOrigin{Type: "file", URI: "file:///private/reference.json"},
|
||||
Producer: contracts.ReferenceProducer{PipelineID: "prior", StepID: "extract", LaneID: "lane", ModuleKey: "dnd/example"},
|
||||
}
|
||||
}
|
||||
|
||||
func withoutSlot(set contracts.ReferenceSet, name string) contracts.ReferenceSet {
|
||||
cloned := contracts.ReferenceSet{Slots: make(map[string]contracts.ResolvedReferenceSlot, len(set.Slots)-1)}
|
||||
for slotName, slot := range set.Slots {
|
||||
if slotName != name {
|
||||
cloned.Slots[slotName] = slot
|
||||
}
|
||||
}
|
||||
return cloned
|
||||
}
|
||||
|
||||
func replaceSlot(set contracts.ReferenceSet, name string, value contracts.ResolvedReferenceSlot) contracts.ReferenceSet {
|
||||
cloned := contracts.ReferenceSet{Slots: make(map[string]contracts.ResolvedReferenceSlot, len(set.Slots))}
|
||||
for slotName, slot := range set.Slots {
|
||||
cloned.Slots[slotName] = slot
|
||||
}
|
||||
cloned.Slots[name] = value
|
||||
return cloned
|
||||
}
|
||||
|
||||
func replaceItem(set contracts.ReferenceSet, name string, item contracts.ReferenceItem) contracts.ReferenceSet {
|
||||
return replaceSlot(set, name, contracts.ResolvedReferenceSlot{Items: []contracts.ReferenceItem{item}})
|
||||
}
|
||||
|
||||
func combatChunk() *source.Chunk {
|
||||
return &source.Chunk{ID: "combat-scene", Ref: source.SourceRef{SourceID: "combat-session", StartUnitID: 1, EndUnitID: 4}}
|
||||
}
|
||||
|
||||
func digest(content []byte) string {
|
||||
sum := sha256.Sum256(content)
|
||||
return "sha256:" + hex.EncodeToString(sum[:])
|
||||
}
|
||||
60
internal/modules/dnd/extract/enemyevents/mapping.go
Normal file
60
internal/modules/dnd/extract/enemyevents/mapping.go
Normal file
@@ -0,0 +1,60 @@
|
||||
package enemyevents
|
||||
|
||||
import (
|
||||
"sort"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
type orderedEnemyEvent struct {
|
||||
value dnd.EnemyEvent
|
||||
earliest int
|
||||
hasEvidence bool
|
||||
}
|
||||
|
||||
func canonicalEnemyEventList(response extractionResponse, order shared.SourceRefOrder, sourceID string) dnd.EnemyEventList {
|
||||
if response.Events == nil {
|
||||
return dnd.EnemyEventList{}
|
||||
}
|
||||
ordered := make([]orderedEnemyEvent, len(response.Events))
|
||||
for index, event := range response.Events {
|
||||
refs := order.Canonicalize(sourceRefs(event.SourceRefs, sourceID))
|
||||
earliest, hasEvidence := order.EarliestValid(refs)
|
||||
ordered[index] = orderedEnemyEvent{
|
||||
value: dnd.EnemyEvent{
|
||||
Name: event.Name,
|
||||
Kind: dnd.EnemyEventKind(event.Kind),
|
||||
SourceRefs: refs,
|
||||
},
|
||||
earliest: earliest,
|
||||
hasEvidence: hasEvidence,
|
||||
}
|
||||
}
|
||||
sort.SliceStable(ordered, func(left, right int) bool {
|
||||
if ordered[left].hasEvidence != ordered[right].hasEvidence {
|
||||
return ordered[left].hasEvidence
|
||||
}
|
||||
if !ordered[left].hasEvidence {
|
||||
return false
|
||||
}
|
||||
return ordered[left].earliest < ordered[right].earliest
|
||||
})
|
||||
events := make([]dnd.EnemyEvent, len(ordered))
|
||||
for index := range ordered {
|
||||
events[index] = ordered[index].value
|
||||
}
|
||||
return dnd.EnemyEventList{Events: events}
|
||||
}
|
||||
|
||||
func sourceRefs(refs []enemySourceRefResponse, sourceID string) []source.SourceRef {
|
||||
if refs == nil {
|
||||
return nil
|
||||
}
|
||||
values := make([]source.SourceRef, len(refs))
|
||||
for index, ref := range refs {
|
||||
values[index] = source.SourceRef{SourceID: sourceID, StartUnitID: ref.StartUnitID, EndUnitID: ref.EndUnitID}
|
||||
}
|
||||
return values
|
||||
}
|
||||
16
internal/modules/dnd/extract/enemyevents/model.go
Normal file
16
internal/modules/dnd/extract/enemyevents/model.go
Normal file
@@ -0,0 +1,16 @@
|
||||
package enemyevents
|
||||
|
||||
type extractionResponse struct {
|
||||
Events []enemyEventResponse `json:"events"`
|
||||
}
|
||||
|
||||
type enemyEventResponse struct {
|
||||
Name string `json:"name"`
|
||||
Kind string `json:"kind"`
|
||||
SourceRefs []enemySourceRefResponse `json:"source_refs"`
|
||||
}
|
||||
|
||||
type enemySourceRefResponse struct {
|
||||
StartUnitID int `json:"start_unit_id"`
|
||||
EndUnitID int `json:"end_unit_id"`
|
||||
}
|
||||
54
internal/modules/dnd/extract/enemyevents/prompt_assets.go
Normal file
54
internal/modules/dnd/extract/enemyevents/prompt_assets.go
Normal file
@@ -0,0 +1,54 @@
|
||||
package enemyevents
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sync"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/promptfs"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
const promptAssetRoot = "assets/prompts"
|
||||
|
||||
var promptAssetManifest = shared.PromptAssetManifest{
|
||||
ModuleDir: "dnd.enemy_events",
|
||||
ModuleFiles: []promptfs.ModulePromptFile{
|
||||
{Name: "dnd.enemy_events.yaml", Path: "assets/prompts/dnd.enemy_events.yaml"},
|
||||
{Name: "grounding.md", Path: "assets/prompts/grounding.md"},
|
||||
{Name: "task.md", Path: "assets/prompts/task.md"},
|
||||
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||
},
|
||||
SharedFiles: []string{
|
||||
"common-dnd-system.md",
|
||||
"common-dnd-extraction-evidence.md",
|
||||
"common-dnd-identity.md",
|
||||
"common-dnd-transcript.md",
|
||||
"common-dnd-references.md",
|
||||
"common-dnd-npcs.md",
|
||||
},
|
||||
}
|
||||
|
||||
func RegisterPromptAssets(registry *llm.AssetRegistry) error {
|
||||
promptFS, err := promptAssetManifest.PromptFS(embeddedAssets)
|
||||
if err != nil {
|
||||
return fmt.Errorf("prepare enemy-event prompt assets: %w", err)
|
||||
}
|
||||
if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
|
||||
return err
|
||||
}
|
||||
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
|
||||
}
|
||||
|
||||
func promptAssetMetadata() (string, error) {
|
||||
promptAssetHashOnce.Do(func() {
|
||||
promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
|
||||
})
|
||||
return promptAssetHash, promptAssetHashErr
|
||||
}
|
||||
|
||||
var (
|
||||
promptAssetHashOnce sync.Once
|
||||
promptAssetHash string
|
||||
promptAssetHashErr error
|
||||
)
|
||||
123
internal/modules/dnd/extract/enemyevents/prompt_assets_test.go
Normal file
123
internal/modules/dnd/extract/enemyevents/prompt_assets_test.go
Normal file
@@ -0,0 +1,123 @@
|
||||
package enemyevents
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io/fs"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||
"gitea.maximumdirect.net/eric/promptkit"
|
||||
)
|
||||
|
||||
func TestRegisterPromptAssetsAndPrepareEnemyEventPrompt(t *testing.T) {
|
||||
registry := llm.NewAssetRegistry()
|
||||
if err := RegisterPromptAssets(registry); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
schemaFS, err := registry.SchemaFS()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := fs.ReadFile(schemaFS, "dnd_enemy_events_llm.v1.json"); err != nil {
|
||||
t.Fatalf("response schema asset: %v", err)
|
||||
}
|
||||
|
||||
prepared := prepareEnemyEventPrompt(t)
|
||||
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_enemy_events_llm.v1.json" || prepared.SelectedProfileID != "dnd-extraction" {
|
||||
t.Fatalf("prepared prompt = %#v", prepared)
|
||||
}
|
||||
transcriptIndex := renderedMessageIndex(t, prepared.Messages, "enemy-transcript")
|
||||
for _, sentinel := range []string{"enemy-npc", "enemy-turn", "enemy-opponent", "Extract Dungeons & Dragons enemy events"} {
|
||||
if index := renderedMessageIndex(t, prepared.Messages, sentinel); index <= transcriptIndex {
|
||||
t.Fatalf("message containing %q has index %d, want after transcript index %d", sentinel, index, transcriptIndex)
|
||||
}
|
||||
}
|
||||
instructionIndex := renderedMessageIndex(t, prepared.Messages, "Return the `events` array")
|
||||
if instructionIndex <= transcriptIndex {
|
||||
t.Fatalf("instruction index = %d, want after transcript index %d", instructionIndex, transcriptIndex)
|
||||
}
|
||||
if instructionIndex != len(prepared.Messages)-1 {
|
||||
t.Fatalf("instruction message index = %d, want final message", instructionIndex)
|
||||
}
|
||||
if cache := prepared.Messages[instructionIndex].CacheControl; cache == nil || cache.Type != promptkit.CacheControlEphemeral {
|
||||
t.Fatalf("final instruction cache control = %#v", cache)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEnemyEventPromptRequiresGroundingInputs(t *testing.T) {
|
||||
engine := newEnemyEventPromptEngine(t)
|
||||
for _, inputName := range []string{"npcs", "combat_turns", "npc_interactions"} {
|
||||
t.Run(inputName, func(t *testing.T) {
|
||||
inputs := enemyEventPromptInputs()
|
||||
delete(inputs, inputName)
|
||||
_, err := engine.Prepare(context.Background(), promptkit.RunRequest{
|
||||
PromptID: PromptID, PromptVersion: SchemaVersion, Inputs: inputs,
|
||||
})
|
||||
if err == nil || !strings.Contains(err.Error(), inputName) {
|
||||
t.Fatalf("Prepare() error = %v, want required %q input", err, inputName)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func prepareEnemyEventPrompt(t *testing.T) *promptkit.PreparedRun {
|
||||
t.Helper()
|
||||
prepared, err := newEnemyEventPromptEngine(t).Prepare(context.Background(), promptkit.RunRequest{
|
||||
PromptID: PromptID, PromptVersion: SchemaVersion, Inputs: enemyEventPromptInputs(),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return prepared
|
||||
}
|
||||
|
||||
func newEnemyEventPromptEngine(t *testing.T) *promptkit.Engine {
|
||||
t.Helper()
|
||||
registry := llm.NewAssetRegistry()
|
||||
if err := RegisterPromptAssets(registry); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
options, err := registry.PromptKitOptions()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
|
||||
ID: "dnd-extraction", Endpoint: "http://127.0.0.1:1/v1", Model: "enemy-test-model",
|
||||
})))
|
||||
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return engine
|
||||
}
|
||||
|
||||
func enemyEventPromptInputs() map[string]promptkit.ArtifactRef {
|
||||
return map[string]promptkit.ArtifactRef{
|
||||
"transcript": promptkit.Inline(`{"value":"enemy-transcript"}`),
|
||||
"players": promptkit.Inline("enemy-player"),
|
||||
"party": promptkit.Inline("enemy-party"),
|
||||
"glossary": promptkit.Inline("enemy-glossary"),
|
||||
"npcs": promptkit.Inline(`{"npcs":[{"name":"enemy-npc"}]}`),
|
||||
"combat_turns": promptkit.Inline(`{"combat_turns":[{"actor":"enemy-turn","turn_kind":"turn"}]}`),
|
||||
"npc_interactions": promptkit.Inline(`{"npc_interactions":[{"name":"enemy-opponent","kind":"combat_opponent"}]}`),
|
||||
}
|
||||
}
|
||||
|
||||
func renderedMessageIndex(t *testing.T, messages []promptkit.RenderedMessage, sentinel string) int {
|
||||
t.Helper()
|
||||
index := -1
|
||||
occurrences := 0
|
||||
for messageIndex, message := range messages {
|
||||
count := strings.Count(message.Content, sentinel)
|
||||
if count > 0 {
|
||||
index = messageIndex
|
||||
occurrences += count
|
||||
}
|
||||
}
|
||||
if occurrences != 1 {
|
||||
t.Fatalf("message sentinel %q rendered %d times, want exactly once", sentinel, occurrences)
|
||||
}
|
||||
return index
|
||||
}
|
||||
@@ -1,12 +1,12 @@
|
||||
package npcs
|
||||
package enemyevents
|
||||
|
||||
import "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||
|
||||
const (
|
||||
PromptID = "dnd.npcs.normalize"
|
||||
ResponseSchemaKey = llm.ResponseSchemaKey("dnd_npcs_normalize_llm")
|
||||
ResponseSchemaID = "notarius.dnd.npcs.normalize.llm"
|
||||
ResponseSchemaName = "notarius_dnd_npcs_normalize_llm_v1"
|
||||
PromptID = "dnd.enemy_events"
|
||||
ResponseSchemaKey = llm.ResponseSchemaKey("dnd_enemy_events_llm")
|
||||
ResponseSchemaID = "notarius.dnd.enemy_events.llm"
|
||||
ResponseSchemaName = "notarius_dnd_enemy_events_llm_v1"
|
||||
SchemaVersion = "v1"
|
||||
)
|
||||
|
||||
@@ -16,6 +16,6 @@ func loadResponseSchema() (llm.ResponseSchema, error) {
|
||||
ID: ResponseSchemaID,
|
||||
Version: SchemaVersion,
|
||||
Name: ResponseSchemaName,
|
||||
AssetPath: "assets/schemas/dnd_npcs_normalize_llm.v1.json",
|
||||
AssetPath: "assets/schemas/dnd_enemy_events_llm.v1.json",
|
||||
})
|
||||
}
|
||||
97
internal/modules/dnd/extract/enemyevents/schema_test.go
Normal file
97
internal/modules/dnd/extract/enemyevents/schema_test.go
Normal file
@@ -0,0 +1,97 @@
|
||||
package enemyevents
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/santhosh-tekuri/jsonschema/v6"
|
||||
)
|
||||
|
||||
func TestResponseSchemaDefinesPrivateStructuralBoundary(t *testing.T) {
|
||||
schema, err := loadResponseSchema()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if schema.Key != ResponseSchemaKey || schema.ID != ResponseSchemaID || schema.Name != ResponseSchemaName || schema.Version != SchemaVersion || !strings.HasPrefix(schema.SHA256, "sha256:") || !json.Valid(schema.JSONSchema) {
|
||||
t.Fatalf("schema = %#v", schema)
|
||||
}
|
||||
valid := validEnemyResponse()
|
||||
content, err := json.Marshal(valid)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := validateEnemySchema(content, schema.JSONSchema); err != nil {
|
||||
t.Fatalf("valid response rejected: %v", err)
|
||||
}
|
||||
semantic := validEnemyResponse()
|
||||
event := semantic["events"].([]any)[0].(map[string]any)
|
||||
event["name"] = ""
|
||||
event["kind"] = "unsupported"
|
||||
ref := event["source_refs"].([]any)[0].(map[string]any)
|
||||
ref["start_unit_id"] = 0
|
||||
ref["end_unit_id"] = -1
|
||||
content, err = json.Marshal(semantic)
|
||||
if err != nil || validateEnemySchema(content, schema.JSONSchema) != nil {
|
||||
t.Fatalf("validator-owned semantics were rejected: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResponseSchemaRejectsInvalidStructure(t *testing.T) {
|
||||
schema, err := loadResponseSchema()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, mutate := range []func(map[string]any){
|
||||
func(event map[string]any) { delete(event, "name") },
|
||||
func(event map[string]any) { event["kind"] = 1 },
|
||||
func(event map[string]any) { event["unexpected"] = true },
|
||||
func(event map[string]any) { event["source_refs"].([]any)[0].(map[string]any)["source_id"] = "session" },
|
||||
} {
|
||||
candidate := validEnemyResponse()
|
||||
mutate(candidate["events"].([]any)[0].(map[string]any))
|
||||
content, err := json.Marshal(candidate)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := validateEnemySchema(content, schema.JSONSchema); err == nil {
|
||||
t.Fatal("private schema accepted structurally invalid response")
|
||||
}
|
||||
}
|
||||
first, err := loadResponseSchema()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
first.JSONSchema[0] = '['
|
||||
second, err := loadResponseSchema()
|
||||
if err != nil || !json.Valid(second.JSONSchema) || bytes.Equal(first.JSONSchema, second.JSONSchema) {
|
||||
t.Fatalf("schema defensive copy = %s, %v", second.JSONSchema, err)
|
||||
}
|
||||
}
|
||||
|
||||
func validEnemyResponse() map[string]any {
|
||||
return map[string]any{"events": []any{map[string]any{
|
||||
"name": "Ashfang", "kind": "engaged", "source_refs": []any{map[string]any{"start_unit_id": 1, "end_unit_id": 1}},
|
||||
}}}
|
||||
}
|
||||
|
||||
func validateEnemySchema(instanceContent, schemaContent []byte) error {
|
||||
instance, err := jsonschema.UnmarshalJSON(bytes.NewReader(instanceContent))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
document, err := jsonschema.UnmarshalJSON(bytes.NewReader(schemaContent))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
compiler := jsonschema.NewCompiler()
|
||||
if err := compiler.AddResource("schema.json", document); err != nil {
|
||||
return err
|
||||
}
|
||||
schema, err := compiler.Compile("schema.json")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return schema.Validate(instance)
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package locationoccurrences
|
||||
|
||||
import "embed"
|
||||
|
||||
//go:embed assets/prompts/*.yaml assets/prompts/*.md assets/schemas/*.json
|
||||
var embeddedAssets embed.FS
|
||||
@@ -0,0 +1,47 @@
|
||||
id: dnd.location_occurrences
|
||||
version: "v1"
|
||||
default_profile: dnd-extraction
|
||||
inputs:
|
||||
- name: transcript
|
||||
required: true
|
||||
content_type: application/json
|
||||
- name: players
|
||||
required: false
|
||||
content_type: text/plain
|
||||
- name: party
|
||||
required: false
|
||||
content_type: text/plain
|
||||
- name: glossary
|
||||
required: false
|
||||
content_type: text/plain
|
||||
- name: locations
|
||||
required: true
|
||||
content_type: application/json
|
||||
messages:
|
||||
- role: system
|
||||
content_file: ./sharedassets/common-dnd-system.md
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-identity.md
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-references.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-transcript.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-extraction-evidence.md
|
||||
- role: user
|
||||
content_file: ./locations.md
|
||||
- role: user
|
||||
content_file: ./task.md
|
||||
- role: user
|
||||
content_file: ./instructions.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
output:
|
||||
format: json
|
||||
validation_mode: json_schema
|
||||
schema_path: dnd_location_occurrences_llm.v1.json
|
||||
repair_attempts: 0
|
||||
@@ -0,0 +1,8 @@
|
||||
Return the occurrences array even when no occurrence is established. Every
|
||||
record must contain location_id, name, kind, and source_refs. Copy location_id
|
||||
and name from one supplied registry record, and cite only narrow transcript
|
||||
ranges that support both that location and its classified occurrence.
|
||||
|
||||
Do not summarize location descriptions, infer a missing registry record, or
|
||||
use registry context as evidence. Omit source_id; Notarius assigns the current
|
||||
transcript source identity.
|
||||
@@ -0,0 +1,9 @@
|
||||
A normalized location registry is provided below for identity grounding. It may
|
||||
be empty. Each record contains the exact location ID and canonical display name
|
||||
to copy when the transcript establishes an occurrence of that place.
|
||||
|
||||
Registry content is context, not occurrence evidence. Do not derive an
|
||||
occurrence or a source range from the registry, and do not infer a location
|
||||
that is absent from it.
|
||||
|
||||
{{ input "locations" }}
|
||||
@@ -0,0 +1,29 @@
|
||||
Extract Dungeons & Dragons location occurrences from the supplied transcript.
|
||||
|
||||
Include an occurrence only when the transcript establishes one supplied
|
||||
location, one occurrence kind, and a coherent passage supporting both. Use
|
||||
only the exact ID and name pair from the supplied location registry. Return an
|
||||
empty occurrences array when no supplied location has an evidenced occurrence
|
||||
in this transcript passage.
|
||||
|
||||
Use exactly one kind per occurrence:
|
||||
|
||||
- visited: party members are physically present, arrive, remain, or depart;
|
||||
- planned: the party explicitly proposes, intends, or agrees to future travel;
|
||||
- recalled: the transcript explicitly recounts an earlier party visit; or
|
||||
- mentioned: the location is explicitly referenced without stronger support,
|
||||
including non-actionable speculation or a mere hypothetical reference.
|
||||
|
||||
A mere hypothetical or speculative reference is not planned unless the
|
||||
transcript also establishes an actual proposal, intention, or agreement to
|
||||
travel. When the hypothetical itself explicitly names a supplied registry
|
||||
location, it may be mentioned using the narrow passage that supports that
|
||||
reference.
|
||||
|
||||
For overlapping support, visited outranks planned, recalled, and mentioned;
|
||||
planned outranks recalled and mentioned; recalled outranks mentioned. A passage
|
||||
may produce multiple records when it independently establishes separate facts,
|
||||
such as recalling an earlier visit while planning a return. Omit inferred,
|
||||
unstated, uncertain, or unsupported places and occurrences. Do not infer a
|
||||
location or occurrence from surrounding events when the transcript does not
|
||||
state it.
|
||||
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "notarius.dnd.location_occurrences.llm",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["occurrences"],
|
||||
"properties": {
|
||||
"occurrences": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["location_id", "name", "kind", "source_refs"],
|
||||
"properties": {
|
||||
"location_id": {"type": "string"},
|
||||
"name": {"type": "string"},
|
||||
"kind": {"enum": ["visited", "planned", "recalled", "mentioned"]},
|
||||
"source_refs": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["start_unit_id", "end_unit_id"],
|
||||
"properties": {
|
||||
"start_unit_id": {"type": "integer"},
|
||||
"end_unit_id": {"type": "integer"}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
106
internal/modules/dnd/extract/locationoccurrences/canonicalize.go
Normal file
106
internal/modules/dnd/extract/locationoccurrences/canonicalize.go
Normal file
@@ -0,0 +1,106 @@
|
||||
package locationoccurrences
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"sort"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
type orderedOccurrence struct {
|
||||
value dnd.LocationOccurrence
|
||||
earliest int
|
||||
hasEvidence bool
|
||||
}
|
||||
|
||||
func canonicalOccurrenceList(response extractionResponse, order shared.SourceRefOrder, sourceID string) dnd.LocationOccurrenceList {
|
||||
if response.Occurrences == nil {
|
||||
return dnd.LocationOccurrenceList{}
|
||||
}
|
||||
ordered := make([]orderedOccurrence, len(response.Occurrences))
|
||||
for index, occurrence := range response.Occurrences {
|
||||
refs := order.Canonicalize(canonicalSourceRefs(occurrence.SourceRefs, sourceID))
|
||||
earliest, hasEvidence := order.EarliestValid(refs)
|
||||
ordered[index] = orderedOccurrence{value: dnd.LocationOccurrence{
|
||||
LocationID: occurrence.LocationID,
|
||||
Name: occurrence.Name,
|
||||
Kind: dnd.LocationOccurrenceKind(occurrence.Kind),
|
||||
SourceRefs: refs,
|
||||
}, earliest: earliest, hasEvidence: hasEvidence}
|
||||
}
|
||||
sort.SliceStable(ordered, func(left, right int) bool {
|
||||
return lessOccurrence(ordered[left], ordered[right], order)
|
||||
})
|
||||
occurrences := make([]dnd.LocationOccurrence, 0, len(ordered))
|
||||
for _, occurrence := range ordered {
|
||||
if len(occurrences) == 0 || !sameOccurrence(occurrences[len(occurrences)-1], occurrence.value) {
|
||||
occurrences = append(occurrences, occurrence.value)
|
||||
}
|
||||
}
|
||||
return dnd.LocationOccurrenceList{Occurrences: occurrences}
|
||||
}
|
||||
|
||||
func lessOccurrence(left, right orderedOccurrence, order shared.SourceRefOrder) bool {
|
||||
if left.hasEvidence != right.hasEvidence {
|
||||
return left.hasEvidence
|
||||
}
|
||||
if left.hasEvidence && left.earliest != right.earliest {
|
||||
return left.earliest < right.earliest
|
||||
}
|
||||
if left.value.LocationID != right.value.LocationID {
|
||||
return left.value.LocationID < right.value.LocationID
|
||||
}
|
||||
if left.value.Name != right.value.Name {
|
||||
return left.value.Name < right.value.Name
|
||||
}
|
||||
if kindOrder(left.value.Kind) != kindOrder(right.value.Kind) {
|
||||
return kindOrder(left.value.Kind) < kindOrder(right.value.Kind)
|
||||
}
|
||||
return lessReferences(left.value.SourceRefs, right.value.SourceRefs, order)
|
||||
}
|
||||
|
||||
func kindOrder(kind dnd.LocationOccurrenceKind) int {
|
||||
switch kind {
|
||||
case dnd.LocationOccurrenceKindVisited:
|
||||
return 0
|
||||
case dnd.LocationOccurrenceKindPlanned:
|
||||
return 1
|
||||
case dnd.LocationOccurrenceKindRecalled:
|
||||
return 2
|
||||
case dnd.LocationOccurrenceKindMentioned:
|
||||
return 3
|
||||
default:
|
||||
return 4
|
||||
}
|
||||
}
|
||||
|
||||
func lessReferences(left, right []source.SourceRef, order shared.SourceRefOrder) bool {
|
||||
limit := len(left)
|
||||
if len(right) < limit {
|
||||
limit = len(right)
|
||||
}
|
||||
for index := 0; index < limit; index++ {
|
||||
if left[index] == right[index] {
|
||||
continue
|
||||
}
|
||||
return order.Less(left[index], right[index])
|
||||
}
|
||||
return len(left) < len(right)
|
||||
}
|
||||
|
||||
func sameOccurrence(left, right dnd.LocationOccurrence) bool {
|
||||
return left.LocationID == right.LocationID && left.Name == right.Name && left.Kind == right.Kind && reflect.DeepEqual(left.SourceRefs, right.SourceRefs)
|
||||
}
|
||||
|
||||
func canonicalSourceRefs(values []occurrenceSourceRefResponse, sourceID string) []source.SourceRef {
|
||||
if values == nil {
|
||||
return nil
|
||||
}
|
||||
refs := make([]source.SourceRef, len(values))
|
||||
for index, value := range values {
|
||||
refs[index] = source.SourceRef{SourceID: sourceID, StartUnitID: value.StartUnitID, EndUnitID: value.EndUnitID}
|
||||
}
|
||||
return refs
|
||||
}
|
||||
188
internal/modules/dnd/extract/locationoccurrences/extractor.go
Normal file
188
internal/modules/dnd/extract/locationoccurrences/extractor.go
Normal file
@@ -0,0 +1,188 @@
|
||||
// Package locationoccurrences extracts source-grounded D&D location occurrences.
|
||||
package locationoccurrences
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"sort"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
locationregistry "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/registry"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
const (
|
||||
Key = "dnd/location-occurrences"
|
||||
mappingPolicy = "dnd.location_occurrences.extract_mapping.v1"
|
||||
)
|
||||
|
||||
const (
|
||||
LocationRegistryReferenceSlot = locationregistry.ReferenceSlot
|
||||
LocationRegistryMaxBytes = locationregistry.MaxBytes
|
||||
)
|
||||
|
||||
var requiredCapabilities = []string{"chunks", "source.transcript"}
|
||||
var providedCapabilities = []string{"dnd.location_occurrences"}
|
||||
|
||||
var referenceSlotDescriptions = shared.ReferenceSlotDescriptions{
|
||||
Glossary: "Optional campaign glossary reference material used only for location-occurrence disambiguation.",
|
||||
Party: "Optional party roster reference material used only for location-occurrence disambiguation.",
|
||||
Players: "Optional player list reference material used only for location-occurrence disambiguation.",
|
||||
Roster: "Deprecated alias for party roster reference material used only for location-occurrence disambiguation.",
|
||||
}
|
||||
|
||||
func referenceSlots() []contracts.ReferenceSlot {
|
||||
slots := shared.ReferenceSlots(referenceSlotDescriptions)
|
||||
slots = append(slots, contracts.ReferenceSlot{
|
||||
Name: LocationRegistryReferenceSlot,
|
||||
Description: "Required normalized location registry used only for location identity grounding, never as occurrence evidence.",
|
||||
Required: true,
|
||||
AcceptedMediaTypes: []string{"application/json"},
|
||||
AcceptedArtifactKinds: []contracts.ArtifactKind{dnd.LocationListKind},
|
||||
MaxBytes: LocationRegistryMaxBytes,
|
||||
})
|
||||
sort.Slice(slots, func(left, right int) bool { return slots[left].Name < slots[right].Name })
|
||||
return slots
|
||||
}
|
||||
|
||||
var _ contracts.Extractor[dnd.LocationOccurrenceList] = (*Extractor)(nil)
|
||||
var _ contracts.ManifestMetadataProvider = (*Extractor)(nil)
|
||||
var _ pipeline.CheckpointFingerprintProvider = (*Extractor)(nil)
|
||||
|
||||
type Options struct{}
|
||||
|
||||
type Extractor struct {
|
||||
llm contracts.StructuredLLMClient
|
||||
locationResolver *locationregistry.Resolver
|
||||
promptSHA string
|
||||
responseSchemaSHA string
|
||||
}
|
||||
|
||||
func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contracts.ReferenceSet) (*Extractor, error) {
|
||||
if llmClient == nil {
|
||||
return nil, extractorErrorf("LLM client must not be nil")
|
||||
}
|
||||
if len(references) > 1 {
|
||||
return nil, extractorErrorf("at most one reference set may be supplied")
|
||||
}
|
||||
var referenceSet contracts.ReferenceSet
|
||||
if len(references) == 1 {
|
||||
referenceSet = references[0]
|
||||
}
|
||||
locationResolver, err := locationregistry.NewResolver(referenceSet)
|
||||
if err != nil {
|
||||
return nil, extractorErrorf("prepare location registry prompt input: %w", err)
|
||||
}
|
||||
promptSHA, err := promptAssetMetadata()
|
||||
if err != nil {
|
||||
return nil, extractorErrorf("load prompt metadata: %w", err)
|
||||
}
|
||||
responseSchema, err := loadResponseSchema()
|
||||
if err != nil {
|
||||
return nil, extractorErrorf("load response schema: %w", err)
|
||||
}
|
||||
return &Extractor{llm: llmClient, locationResolver: locationResolver, promptSHA: promptSHA, responseSchemaSHA: responseSchema.SHA256}, nil
|
||||
}
|
||||
|
||||
func (e *Extractor) Key() string { return Key }
|
||||
|
||||
func (e *Extractor) ReferenceSlots() []contracts.ReferenceSlot { return referenceSlots() }
|
||||
|
||||
func (e *Extractor) ManifestMetadata() map[string]any {
|
||||
if e == nil {
|
||||
return nil
|
||||
}
|
||||
metadata := map[string]any{
|
||||
"prompt_id": PromptID,
|
||||
"prompt_version": SchemaVersion,
|
||||
"prompt_sha256": e.promptSHA,
|
||||
"mapping_policy": mappingPolicy,
|
||||
"response_schema_key": string(ResponseSchemaKey),
|
||||
"response_schema_id": ResponseSchemaID,
|
||||
"response_schema_name": ResponseSchemaName,
|
||||
"response_schema_version": SchemaVersion,
|
||||
"response_schema_sha256": e.responseSchemaSHA,
|
||||
}
|
||||
seeded := e.locationResolver.Seeded()
|
||||
if seeded.Bound() {
|
||||
metadata["location_registry_digest"] = seeded.Digest()
|
||||
metadata["location_count"] = seeded.Count()
|
||||
}
|
||||
return metadata
|
||||
}
|
||||
|
||||
func (e *Extractor) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
|
||||
if e == nil {
|
||||
return nil
|
||||
}
|
||||
return []pipeline.CheckpointFingerprint{
|
||||
{Name: "prompt", Value: e.promptSHA},
|
||||
{Name: "response_schema", Value: e.responseSchemaSHA},
|
||||
{Name: "mapping_policy", Value: mappingPolicy},
|
||||
{Name: "location_registry", Value: e.locationResolver.Seeded().ProjectionDigest()},
|
||||
}
|
||||
}
|
||||
|
||||
func (e *Extractor) Extract(ctx context.Context, req contracts.TypedExtractionRequest) (contracts.TypedExtractionResult[dnd.LocationOccurrenceList], error) {
|
||||
if e == nil {
|
||||
return contracts.TypedExtractionResult[dnd.LocationOccurrenceList]{}, extractorErrorf("extractor must not be nil")
|
||||
}
|
||||
if e.llm == nil {
|
||||
return contracts.TypedExtractionResult[dnd.LocationOccurrenceList]{}, extractorErrorf("LLM client must not be nil")
|
||||
}
|
||||
sourceInput, err := shared.PrepareChunkExtraction(ctx, req)
|
||||
if err != nil {
|
||||
return contracts.TypedExtractionResult[dnd.LocationOccurrenceList]{}, extractorErrorf("%w", err)
|
||||
}
|
||||
registry, err := e.locationResolver.Resolve(req.References)
|
||||
if err != nil {
|
||||
return contracts.TypedExtractionResult[dnd.LocationOccurrenceList]{}, extractorErrorf("resolve location registry: %w", err)
|
||||
}
|
||||
if !registry.Bound() {
|
||||
return contracts.TypedExtractionResult[dnd.LocationOccurrenceList]{}, extractorErrorf("location registry reference is required")
|
||||
}
|
||||
|
||||
var response extractionResponse
|
||||
inputs := shared.PromptInputs(sourceInput, req.References)
|
||||
inputs[LocationRegistryReferenceSlot] = registry.PromptInput()
|
||||
if _, err := e.llm.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
|
||||
StageName: Key, PromptID: PromptID, PromptVersion: SchemaVersion,
|
||||
ProfileID: req.LLMProfile, SessionID: req.SessionID, Inputs: inputs,
|
||||
}, &response); err != nil {
|
||||
return contracts.TypedExtractionResult[dnd.LocationOccurrenceList]{}, extractorErrorf("complete structured output: %w", err)
|
||||
}
|
||||
return contracts.TypedExtractionResult[dnd.LocationOccurrenceList]{Value: canonicalOccurrenceList(response, shared.NewSourceRefOrder(req.Source), req.Source.ID)}, nil
|
||||
}
|
||||
|
||||
func ModuleSpec() pipeline.ModuleSpec {
|
||||
return pipeline.ModuleSpec{
|
||||
Key: Key, Stage: pipeline.StageExtract, ExecutionClass: contracts.ExecutionClassLLMBacked,
|
||||
Requires: append([]string(nil), requiredCapabilities...), Provides: append([]string(nil), providedCapabilities...),
|
||||
ArtifactKind: dnd.LocationOccurrenceListKind, ReferenceSlots: referenceSlots(),
|
||||
}
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ExtractorRegistry) error {
|
||||
return pipeline.RegisterExtractorBuilder(registry, ModuleSpec(), validateOptions, func(request pipeline.BuildRequest) (contracts.Extractor[dnd.LocationOccurrenceList], error) {
|
||||
options, err := DecodeOptions(request.Options)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return New(request.Dependencies.LLM, options, request.References)
|
||||
})
|
||||
}
|
||||
|
||||
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }
|
||||
|
||||
func DecodeOptions(options map[string]any) (Options, error) {
|
||||
if err := pipeline.RejectUnknownOptions(options); err != nil {
|
||||
return Options{}, extractorErrorf("%w", err)
|
||||
}
|
||||
return Options{}, nil
|
||||
}
|
||||
|
||||
func extractorErrorf(format string, args ...any) error {
|
||||
return fmt.Errorf("dnd location occurrences extractor: "+format, args...)
|
||||
}
|
||||
@@ -0,0 +1,243 @@
|
||||
package locationoccurrences
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
locationcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/locations"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||
)
|
||||
|
||||
func TestExtractMapsKindsOrdersOccurrencesAndPreservesIndependentFacts(t *testing.T) {
|
||||
locations := locationRegistry(t, "The Tavern", "The Tavern")
|
||||
first, second := locations.Locations[0], locations.Locations[1]
|
||||
client := &fakeOccurrencesLLMClient{response: extractionResponse{Occurrences: []occurrenceResponse{
|
||||
{LocationID: second.ID, Name: second.Name, Kind: "mentioned", SourceRefs: occurrenceRefs(30, 30)},
|
||||
{LocationID: first.ID, Name: first.Name, Kind: "mentioned", SourceRefs: occurrenceRefs(10, 10)},
|
||||
{LocationID: first.ID, Name: first.Name, Kind: "recalled", SourceRefs: occurrenceRefs(10, 10)},
|
||||
{LocationID: first.ID, Name: first.Name, Kind: "planned", SourceRefs: occurrenceRefs(10, 10)},
|
||||
{LocationID: first.ID, Name: first.Name, Kind: "visited", SourceRefs: append(occurrenceRefs(10, 10), occurrenceRefs(10, 10)...)},
|
||||
{LocationID: first.ID, Name: first.Name, Kind: "visited", SourceRefs: occurrenceRefs(20, 20)},
|
||||
{LocationID: first.ID, Name: first.Name, Kind: "visited", SourceRefs: occurrenceRefs(10, 10)},
|
||||
}}}
|
||||
references := registryReferences(t, locations)
|
||||
req := extractionRequest()
|
||||
req.References = references
|
||||
result, err := newExtractor(t, client, references).Extract(context.Background(), req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(result.Value.Occurrences) != 6 {
|
||||
t.Fatalf("occurrences = %#v, want exact duplicate removed", result.Value.Occurrences)
|
||||
}
|
||||
got := result.Value.Occurrences
|
||||
if kinds := []dnd.LocationOccurrenceKind{got[0].Kind, got[1].Kind, got[2].Kind, got[3].Kind}; !reflect.DeepEqual(kinds, []dnd.LocationOccurrenceKind{dnd.LocationOccurrenceKindVisited, dnd.LocationOccurrenceKindPlanned, dnd.LocationOccurrenceKindRecalled, dnd.LocationOccurrenceKindMentioned}) {
|
||||
t.Fatalf("same-evidence kind order = %#v", kinds)
|
||||
}
|
||||
if got[4].Kind != dnd.LocationOccurrenceKindVisited || got[4].SourceRefs[0].StartUnitID != 20 || got[5].LocationID != second.ID || got[5].SourceRefs[0].StartUnitID != 30 {
|
||||
t.Fatalf("occurrence order = %#v", got)
|
||||
}
|
||||
if !reflect.DeepEqual(got[0].SourceRefs, []source.SourceRef{{SourceID: req.Source.ID, StartUnitID: 10, EndUnitID: 10}}) {
|
||||
t.Fatalf("canonical evidence = %#v", got[0].SourceRefs)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractUsesIDsNamesAndCurrentTranscriptEvidenceOnly(t *testing.T) {
|
||||
locations := locationRegistry(t, "The Tavern", "The Tavern")
|
||||
first, second := locations.Locations[0], locations.Locations[1]
|
||||
client := &fakeOccurrencesLLMClient{response: extractionResponse{Occurrences: []occurrenceResponse{{
|
||||
LocationID: second.ID, Name: second.Name, Kind: "visited", SourceRefs: occurrenceRefs(10, 10),
|
||||
}}}}
|
||||
references := registryReferences(t, locations)
|
||||
req := extractionRequest()
|
||||
req.References = references
|
||||
result, err := newExtractor(t, client, references).Extract(context.Background(), req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if occurrence := result.Value.Occurrences[0]; occurrence.LocationID != second.ID || occurrence.Name != second.Name || occurrence.SourceRefs[0].SourceID != req.Source.ID {
|
||||
t.Fatalf("occurrence = %#v", occurrence)
|
||||
}
|
||||
input := client.requests[0].Inputs[LocationRegistryReferenceSlot]
|
||||
if input.Name != LocationRegistryReferenceSlot || !strings.Contains(string(input.Content), first.ID) || !strings.Contains(string(input.Content), second.ID) {
|
||||
t.Fatalf("location prompt input = %#v", input)
|
||||
}
|
||||
for _, forbidden := range []string{"source_refs", "source_id", "other-session"} {
|
||||
if strings.Contains(string(input.Content), forbidden) {
|
||||
t.Fatalf("location prompt leaked %q: %s", forbidden, input.Content)
|
||||
}
|
||||
}
|
||||
if strings.Contains(string(client.requests[0].Inputs["transcript"].Content), "other-session") {
|
||||
t.Fatal("transcript input contains registry evidence")
|
||||
}
|
||||
metadata, err := json.Marshal(newExtractor(t, &fakeOccurrencesLLMClient{}, references).ManifestMetadata())
|
||||
if err != nil || strings.Contains(string(metadata), "other-session") || strings.Contains(string(metadata), first.ID) {
|
||||
t.Fatalf("manifest metadata = %s, %v", metadata, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractPreservesUnknownOrMismatchedGroundingForValidators(t *testing.T) {
|
||||
locations := locationRegistry(t, "The Mill")
|
||||
known := locations.Locations[0]
|
||||
client := &fakeOccurrencesLLMClient{response: extractionResponse{Occurrences: []occurrenceResponse{
|
||||
{LocationID: "location:sha256:unknown", Name: "The Mill", Kind: "mentioned", SourceRefs: occurrenceRefs(10, 10)},
|
||||
{LocationID: known.ID, Name: "A Different Mill", Kind: "mentioned", SourceRefs: occurrenceRefs(20, 20)},
|
||||
}}}
|
||||
references := registryReferences(t, locations)
|
||||
req := extractionRequest()
|
||||
req.References = references
|
||||
result, err := newExtractor(t, client, references).Extract(context.Background(), req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if result.Value.Occurrences[0].LocationID != "location:sha256:unknown" || result.Value.Occurrences[1].Name != "A Different Mill" {
|
||||
t.Fatalf("extractor repaired validator-owned grounding errors: %#v", result.Value.Occurrences)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractRequiresRegistryAndAcceptsEmptyRegistryWithNoOccurrences(t *testing.T) {
|
||||
client := &fakeOccurrencesLLMClient{response: extractionResponse{Occurrences: []occurrenceResponse{}}}
|
||||
if _, err := newExtractor(t, client).Extract(context.Background(), extractionRequest()); err == nil || !strings.Contains(err.Error(), "location registry reference is required") {
|
||||
t.Fatalf("Extract() error = %v", err)
|
||||
}
|
||||
if len(client.requests) != 0 {
|
||||
t.Fatalf("LLM calls = %d", len(client.requests))
|
||||
}
|
||||
empty := dnd.LocationList{Locations: []dnd.Location{}}
|
||||
references := registryReferences(t, empty)
|
||||
req := extractionRequest()
|
||||
req.References = references
|
||||
result, err := newExtractor(t, client, references).Extract(context.Background(), req)
|
||||
if err != nil || result.Value.Occurrences == nil || len(result.Value.Occurrences) != 0 {
|
||||
t.Fatalf("empty registry result = %#v, %v", result, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractResolvesGeneratedRegistryAtOperationTimeAndDoesNotMutateResponse(t *testing.T) {
|
||||
locations := locationRegistry(t, "The Mill")
|
||||
location := locations.Locations[0]
|
||||
client := &fakeOccurrencesLLMClient{response: extractionResponse{Occurrences: []occurrenceResponse{{
|
||||
LocationID: location.ID, Name: location.Name, Kind: "mentioned", SourceRefs: occurrenceRefs(30, 30),
|
||||
}}}}
|
||||
references := registryReferences(t, locations)
|
||||
req := extractionRequest()
|
||||
req.References = references
|
||||
extractor := newExtractor(t, client)
|
||||
result, err := extractor.Extract(context.Background(), req)
|
||||
if err != nil || result.Value.Occurrences[0].Name != "The Mill" {
|
||||
t.Fatalf("Extract() = %#v, %v", result, err)
|
||||
}
|
||||
if input := client.requests[0].Inputs[LocationRegistryReferenceSlot]; !strings.Contains(string(input.Content), location.ID) || input.OriginURI != "" {
|
||||
t.Fatalf("generated registry prompt input = %#v", input)
|
||||
}
|
||||
if _, ok := extractor.ManifestMetadata()["location_registry_digest"]; ok {
|
||||
t.Fatalf("operation registry leaked into static metadata: %#v", extractor.ManifestMetadata())
|
||||
}
|
||||
if client.response.Occurrences[0].SourceRefs[0].StartUnitID != 30 {
|
||||
t.Fatalf("model response mutated: %#v", client.response)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractorContractsMetadataAndFailures(t *testing.T) {
|
||||
if _, err := New(nil, Options{}); err == nil || !strings.Contains(err.Error(), "LLM client") {
|
||||
t.Fatalf("New(nil) error = %v", err)
|
||||
}
|
||||
if _, err := New(&fakeOccurrencesLLMClient{}, Options{}, contracts.ReferenceSet{}, contracts.ReferenceSet{}); err == nil || !strings.Contains(err.Error(), "at most one reference set") {
|
||||
t.Fatalf("New() error = %v", err)
|
||||
}
|
||||
malformed := contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{LocationRegistryReferenceSlot: {
|
||||
Items: []contracts.ReferenceItem{{SlotName: LocationRegistryReferenceSlot, MediaType: "application/json", Content: []byte(`{"secret":"registry evidence"}`)}},
|
||||
}}}
|
||||
if _, err := New(&fakeOccurrencesLLMClient{}, Options{}, malformed); err == nil || !strings.Contains(err.Error(), "prepare location registry") || strings.Contains(err.Error(), "registry evidence") {
|
||||
t.Fatalf("New() error = %v", err)
|
||||
}
|
||||
|
||||
locations := locationRegistry(t, "The Mill")
|
||||
references := registryReferences(t, locations)
|
||||
req := extractionRequest()
|
||||
req.References = references
|
||||
extractor := newExtractor(t, &fakeOccurrencesLLMClient{}, references)
|
||||
var nilExtractor *Extractor
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
extractor *Extractor
|
||||
req contracts.TypedExtractionRequest
|
||||
want string
|
||||
}{
|
||||
{"nil extractor", nilExtractor, req, "extractor"},
|
||||
{"nil client", &Extractor{}, req, "LLM client"},
|
||||
{"invalid request", extractor, mismatchedSourceInputRequest(req), "must match chunk"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if _, err := test.extractor.Extract(context.Background(), test.req); err == nil || !strings.Contains(err.Error(), test.want) {
|
||||
t.Fatalf("Extract() error = %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
if _, err := newExtractor(t, &fakeOccurrencesLLMClient{err: errors.New("provider unavailable")}, references).Extract(context.Background(), req); err == nil || !strings.Contains(err.Error(), "provider unavailable") {
|
||||
t.Fatalf("provider error = %v", err)
|
||||
}
|
||||
|
||||
spec := ModuleSpec()
|
||||
if spec.Key != Key || spec.Stage != pipeline.StageExtract || spec.ExecutionClass != contracts.ExecutionClassLLMBacked || spec.ArtifactKind != dnd.LocationOccurrenceListKind {
|
||||
t.Fatalf("ModuleSpec() = %#v", spec)
|
||||
}
|
||||
var slot contracts.ReferenceSlot
|
||||
for _, candidate := range spec.ReferenceSlots {
|
||||
if candidate.Name == LocationRegistryReferenceSlot {
|
||||
slot = candidate
|
||||
}
|
||||
}
|
||||
if !slot.Required || !reflect.DeepEqual(slot.AcceptedArtifactKinds, []contracts.ArtifactKind{dnd.LocationListKind}) || slot.MaxBytes != LocationRegistryMaxBytes {
|
||||
t.Fatalf("location registry slot = %#v", slot)
|
||||
}
|
||||
registry := pipeline.NewExtractorRegistry()
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, ok := registry.Spec(Key); !ok {
|
||||
t.Fatalf("registration missing %q", Key)
|
||||
}
|
||||
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
|
||||
t.Fatal("DecodeOptions() accepted unknown options")
|
||||
}
|
||||
metadata := newExtractor(t, &fakeOccurrencesLLMClient{}, references).ManifestMetadata()
|
||||
for _, key := range []string{"prompt_sha256", "response_schema_sha256", "location_registry_digest"} {
|
||||
if value, ok := metadata[key].(string); !ok || !strings.HasPrefix(value, "sha256:") {
|
||||
t.Fatalf("metadata[%q] = %#v", key, metadata[key])
|
||||
}
|
||||
}
|
||||
if got := newExtractor(t, &fakeOccurrencesLLMClient{}, references).CheckpointFingerprints(); len(got) != 4 || got[3].Name != "location_registry" {
|
||||
t.Fatalf("fingerprints = %#v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func locationRegistry(t *testing.T, names ...string) dnd.LocationList {
|
||||
t.Helper()
|
||||
locations := make([]dnd.Location, len(names))
|
||||
for index, name := range names {
|
||||
refs := []source.SourceRef{{SourceID: "other-session", StartUnitID: index + 1, EndUnitID: index + 1}}
|
||||
locations[index] = dnd.Location{ID: identity.DeriveID(name, refs), Name: name, SourceRefs: refs}
|
||||
}
|
||||
return dnd.LocationList{Locations: locations}
|
||||
}
|
||||
|
||||
func registryReferences(t *testing.T, locations dnd.LocationList) contracts.ReferenceSet {
|
||||
t.Helper()
|
||||
content, err := locationcodec.New().Encode(locations)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{LocationRegistryReferenceSlot: {
|
||||
Slot: contracts.ReferenceSlot{Name: LocationRegistryReferenceSlot},
|
||||
Items: []contracts.ReferenceItem{{SlotName: LocationRegistryReferenceSlot, MediaType: locationcodec.MediaType, Content: content, Origin: contracts.ReferenceOrigin{Type: "generated"}}},
|
||||
}}}
|
||||
}
|
||||
17
internal/modules/dnd/extract/locationoccurrences/model.go
Normal file
17
internal/modules/dnd/extract/locationoccurrences/model.go
Normal file
@@ -0,0 +1,17 @@
|
||||
package locationoccurrences
|
||||
|
||||
type extractionResponse struct {
|
||||
Occurrences []occurrenceResponse `json:"occurrences"`
|
||||
}
|
||||
|
||||
type occurrenceResponse struct {
|
||||
LocationID string `json:"location_id"`
|
||||
Name string `json:"name"`
|
||||
Kind string `json:"kind"`
|
||||
SourceRefs []occurrenceSourceRefResponse `json:"source_refs"`
|
||||
}
|
||||
|
||||
type occurrenceSourceRefResponse struct {
|
||||
StartUnitID int `json:"start_unit_id"`
|
||||
EndUnitID int `json:"end_unit_id"`
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
package locationoccurrences
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sync"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/promptfs"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
const promptAssetRoot = "assets/prompts"
|
||||
|
||||
var promptAssetManifest = shared.PromptAssetManifest{
|
||||
ModuleDir: PromptID,
|
||||
ModuleFiles: []promptfs.ModulePromptFile{
|
||||
{Name: "dnd.location_occurrences.yaml", Path: "assets/prompts/dnd.location_occurrences.yaml"},
|
||||
{Name: "locations.md", Path: "assets/prompts/locations.md"},
|
||||
{Name: "task.md", Path: "assets/prompts/task.md"},
|
||||
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||
},
|
||||
SharedFiles: []string{
|
||||
"common-dnd-system.md",
|
||||
"common-dnd-extraction-evidence.md",
|
||||
"common-dnd-identity.md",
|
||||
"common-dnd-transcript.md",
|
||||
"common-dnd-references.md",
|
||||
},
|
||||
}
|
||||
|
||||
func RegisterPromptAssets(registry *llm.AssetRegistry) error {
|
||||
promptFS, err := promptAssetManifest.PromptFS(embeddedAssets)
|
||||
if err != nil {
|
||||
return fmt.Errorf("prepare location-occurrence prompt assets: %w", err)
|
||||
}
|
||||
if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
|
||||
return err
|
||||
}
|
||||
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
|
||||
}
|
||||
|
||||
func promptAssetMetadata() (string, error) {
|
||||
promptAssetHashOnce.Do(func() {
|
||||
promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
|
||||
})
|
||||
return promptAssetHash, promptAssetHashErr
|
||||
}
|
||||
|
||||
var (
|
||||
promptAssetHashOnce sync.Once
|
||||
promptAssetHash string
|
||||
promptAssetHashErr error
|
||||
)
|
||||
@@ -0,0 +1,64 @@
|
||||
package locationoccurrences
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||
"gitea.maximumdirect.net/eric/promptkit"
|
||||
)
|
||||
|
||||
func TestRegisterPromptAssetsPreparesLocationOccurrencePrompt(t *testing.T) {
|
||||
registry := llm.NewAssetRegistry()
|
||||
if err := RegisterPromptAssets(registry); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
options, err := registry.PromptKitOptions()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{
|
||||
ID: "location-occurrences-test", Endpoint: "http://127.0.0.1:1/v1", Model: "test",
|
||||
})))
|
||||
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
|
||||
PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "location-occurrences-test",
|
||||
Inputs: map[string]promptkit.ArtifactRef{
|
||||
"transcript": promptkit.Inline(`{"units":[1]}`), "players": promptkit.Inline(" "), "party": promptkit.Inline(" "), "glossary": promptkit.Inline(" "),
|
||||
"locations": promptkit.Inline(`{"locations":[{"id":"location:sha256:test","name":"The Mill"}]}`),
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if prepared.PromptID != PromptID || prepared.OutputContract.SchemaPath != "dnd_location_occurrences_llm.v1.json" {
|
||||
t.Fatalf("prepared prompt = %#v", prepared)
|
||||
}
|
||||
var registryMessage string
|
||||
registryIndex := -1
|
||||
evidenceIndex := -1
|
||||
taskIndex := -1
|
||||
for index, message := range prepared.Messages {
|
||||
if strings.Contains(message.Content, "normalized location registry") {
|
||||
registryMessage = message.Content
|
||||
registryIndex = index
|
||||
}
|
||||
if strings.Contains(message.Content, "Transcript units are the only evidence") {
|
||||
evidenceIndex = index
|
||||
}
|
||||
if strings.Contains(message.Content, "Extract Dungeons & Dragons location occurrences") {
|
||||
taskIndex = index
|
||||
}
|
||||
}
|
||||
if !strings.Contains(registryMessage, "location:sha256:test") || !strings.Contains(registryMessage, "The Mill") || strings.Contains(registryMessage, "source_refs") {
|
||||
t.Fatalf("rendered prompt did not preserve source-free registry grounding: %s", registryMessage)
|
||||
}
|
||||
if evidenceIndex < 0 || taskIndex < 0 || registryIndex <= evidenceIndex || registryIndex >= taskIndex {
|
||||
t.Fatalf("registry prompt placement = evidence %d, registry %d, task %d", evidenceIndex, registryIndex, taskIndex)
|
||||
}
|
||||
}
|
||||
21
internal/modules/dnd/extract/locationoccurrences/schema.go
Normal file
21
internal/modules/dnd/extract/locationoccurrences/schema.go
Normal file
@@ -0,0 +1,21 @@
|
||||
package locationoccurrences
|
||||
|
||||
import "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||
|
||||
const (
|
||||
PromptID = "dnd.location_occurrences"
|
||||
ResponseSchemaKey = llm.ResponseSchemaKey("dnd_location_occurrences_llm")
|
||||
ResponseSchemaID = "notarius.dnd.location_occurrences.llm"
|
||||
ResponseSchemaName = "notarius_dnd_location_occurrences_llm_v1"
|
||||
SchemaVersion = "v1"
|
||||
)
|
||||
|
||||
func loadResponseSchema() (llm.ResponseSchema, error) {
|
||||
return llm.LoadResponseSchema(embeddedAssets, llm.ResponseSchemaDefinition{
|
||||
Key: ResponseSchemaKey,
|
||||
ID: ResponseSchemaID,
|
||||
Version: SchemaVersion,
|
||||
Name: ResponseSchemaName,
|
||||
AssetPath: "assets/schemas/dnd_location_occurrences_llm.v1.json",
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
package locationoccurrences
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/santhosh-tekuri/jsonschema/v6"
|
||||
)
|
||||
|
||||
func TestResponseSchemaRestrictsPrivateOccurrenceStructureAndKinds(t *testing.T) {
|
||||
schema, err := loadResponseSchema()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if schema.Key != ResponseSchemaKey || schema.ID != ResponseSchemaID || schema.Name != ResponseSchemaName || schema.Version != SchemaVersion || !strings.HasPrefix(schema.SHA256, "sha256:") || !json.Valid(schema.JSONSchema) {
|
||||
t.Fatalf("schema = %#v", schema)
|
||||
}
|
||||
valid := map[string]any{"occurrences": []any{map[string]any{
|
||||
"location_id": "location:sha256:test", "name": "The Mill", "kind": "visited",
|
||||
"source_refs": []any{map[string]any{"start_unit_id": 1, "end_unit_id": 2}},
|
||||
}}}
|
||||
if err := validateSchema(t, valid, schema.JSONSchema); err != nil {
|
||||
t.Fatalf("valid response rejected: %v", err)
|
||||
}
|
||||
for _, mutate := range []func(map[string]any){
|
||||
func(value map[string]any) { delete(value, "location_id") },
|
||||
func(value map[string]any) { value["kind"] = "other" },
|
||||
func(value map[string]any) { value["unexpected"] = true },
|
||||
func(value map[string]any) {
|
||||
value["source_refs"].([]any)[0].(map[string]any)["source_id"] = "assigned later"
|
||||
},
|
||||
} {
|
||||
candidate := cloneCandidate(t, valid)
|
||||
mutate(candidate["occurrences"].([]any)[0].(map[string]any))
|
||||
if err := validateSchema(t, candidate, schema.JSONSchema); err == nil {
|
||||
t.Fatal("schema accepted structurally invalid response")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResponseSchemaIsDefensiveAndContentSafe(t *testing.T) {
|
||||
first, err := loadResponseSchema()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
first.JSONSchema[0] = '['
|
||||
second, err := loadResponseSchema()
|
||||
if err != nil || !json.Valid(second.JSONSchema) || bytes.Equal(first.JSONSchema, second.JSONSchema) {
|
||||
t.Fatalf("schema = %s, %v", second.JSONSchema, err)
|
||||
}
|
||||
if diagnostics := second.DiagnosticsMap(); diagnostics["key"] != ResponseSchemaKey || diagnostics["id"] != ResponseSchemaID {
|
||||
t.Fatalf("diagnostics = %#v", diagnostics)
|
||||
} else if _, ok := diagnostics["json_schema"]; ok {
|
||||
t.Fatalf("diagnostics leaked schema content: %#v", diagnostics)
|
||||
}
|
||||
}
|
||||
|
||||
func cloneCandidate(t *testing.T, value map[string]any) map[string]any {
|
||||
t.Helper()
|
||||
content, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var clone map[string]any
|
||||
if err := json.Unmarshal(content, &clone); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return clone
|
||||
}
|
||||
|
||||
func validateSchema(t *testing.T, value map[string]any, schemaContent []byte) error {
|
||||
t.Helper()
|
||||
content, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
instance, err := jsonschema.UnmarshalJSON(bytes.NewReader(content))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
document, err := jsonschema.UnmarshalJSON(bytes.NewReader(schemaContent))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
compiler := jsonschema.NewCompiler()
|
||||
if err := compiler.AddResource("schema.json", document); err != nil {
|
||||
return err
|
||||
}
|
||||
schema, err := compiler.Compile("schema.json")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return schema.Validate(instance)
|
||||
}
|
||||
@@ -0,0 +1,82 @@
|
||||
package locationoccurrences
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
)
|
||||
|
||||
func extractionRequest() contracts.TypedExtractionRequest {
|
||||
doc := sourceDocument()
|
||||
chunk := &source.Chunk{
|
||||
ID: "session-occurrences:chunk:0", SourceID: doc.ID, Index: 0,
|
||||
Ref: source.SourceRef{SourceID: doc.ID, StartUnitID: 10, EndUnitID: 30},
|
||||
Content: []byte(`{"units":[10,20,30]}`), MediaType: "application/json",
|
||||
Units: append([]source.SourceUnit(nil), doc.Units...),
|
||||
}
|
||||
return contracts.TypedExtractionRequest{
|
||||
Source: doc, Chunk: chunk,
|
||||
SourceInput: contracts.NewLLMInputMaterial("source", chunk.MediaType, chunk.Content, "sha256:chunk", "file:///session-occurrences.json"),
|
||||
SessionID: "occurrence-session", LLMProfile: "occurrence-profile",
|
||||
}
|
||||
}
|
||||
|
||||
func sourceDocument() *source.SourceDocument {
|
||||
return &source.SourceDocument{ID: "session-occurrences", Kind: "transcript", Format: "application/json", Digest: "sha256:test", Units: []source.SourceUnit{
|
||||
{ID: 10, Kind: "transcript_segment", Text: "The party returns to the tavern."},
|
||||
{ID: 20, Kind: "transcript_segment", Text: "They plan to travel to the tavern tomorrow."},
|
||||
{ID: 30, Kind: "transcript_segment", Text: "They recall their first visit to the tavern."},
|
||||
}}
|
||||
}
|
||||
|
||||
func occurrenceRefs(start, end int) []occurrenceSourceRefResponse {
|
||||
return []occurrenceSourceRefResponse{{StartUnitID: start, EndUnitID: end}}
|
||||
}
|
||||
|
||||
func newExtractor(t *testing.T, client contracts.StructuredLLMClient, references ...contracts.ReferenceSet) *Extractor {
|
||||
t.Helper()
|
||||
extractor, err := New(client, Options{}, references...)
|
||||
if err != nil {
|
||||
t.Fatalf("New() error = %v", err)
|
||||
}
|
||||
return extractor
|
||||
}
|
||||
|
||||
func mismatchedSourceInputRequest(req contracts.TypedExtractionRequest) contracts.TypedExtractionRequest {
|
||||
req.SourceInput = contracts.NewLLMInputMaterial("source", "application/json", []byte(`{"different":true}`), "sha256:other", "")
|
||||
return req
|
||||
}
|
||||
|
||||
type fakeOccurrencesLLMClient struct {
|
||||
response extractionResponse
|
||||
err error
|
||||
requests []contracts.StructuredCompletionRequest
|
||||
}
|
||||
|
||||
func (client *fakeOccurrencesLLMClient) CompleteStructured(_ context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
|
||||
client.requests = append(client.requests, cloneStructuredCompletionRequest(req))
|
||||
if client.err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, client.err
|
||||
}
|
||||
target, ok := out.(*extractionResponse)
|
||||
if !ok {
|
||||
return contracts.StructuredCompletionResponse{}, errors.New("unexpected output target")
|
||||
}
|
||||
content, err := json.Marshal(client.response)
|
||||
if err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, err
|
||||
}
|
||||
if err := json.Unmarshal(content, target); err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, err
|
||||
}
|
||||
return contracts.StructuredCompletionResponse{Content: content}, nil
|
||||
}
|
||||
|
||||
func cloneStructuredCompletionRequest(req contracts.StructuredCompletionRequest) contracts.StructuredCompletionRequest {
|
||||
req.Inputs = req.Inputs.Clone()
|
||||
return req
|
||||
}
|
||||
6
internal/modules/dnd/extract/locations/assets.go
Normal file
6
internal/modules/dnd/extract/locations/assets.go
Normal file
@@ -0,0 +1,6 @@
|
||||
package locations
|
||||
|
||||
import "embed"
|
||||
|
||||
//go:embed assets/prompts/*.yaml assets/prompts/*.md assets/schemas/*.json
|
||||
var embeddedAssets embed.FS
|
||||
@@ -0,0 +1,42 @@
|
||||
id: dnd.locations
|
||||
version: "v1"
|
||||
default_profile: dnd-extraction
|
||||
inputs:
|
||||
- name: transcript
|
||||
required: true
|
||||
content_type: application/json
|
||||
- name: players
|
||||
required: false
|
||||
content_type: text/plain
|
||||
- name: party
|
||||
required: false
|
||||
content_type: text/plain
|
||||
- name: glossary
|
||||
required: false
|
||||
content_type: text/plain
|
||||
messages:
|
||||
- role: system
|
||||
content_file: ./sharedassets/common-dnd-system.md
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-identity.md
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-references.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-transcript.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
- role: user
|
||||
content_file: ./sharedassets/common-dnd-extraction-evidence.md
|
||||
- role: user
|
||||
content_file: ./task.md
|
||||
- role: user
|
||||
content_file: ./instructions.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
output:
|
||||
format: json
|
||||
validation_mode: json_schema
|
||||
schema_path: dnd_locations_llm.v1.json
|
||||
repair_attempts: 0
|
||||
@@ -0,0 +1,6 @@
|
||||
Return only observed location display names and narrow transcript source ranges.
|
||||
|
||||
Exclude people, creatures, objects, organizations, abstract concepts, and
|
||||
places merely inferred from an event. Omit uncertain or unsupported places.
|
||||
Campaign references may clarify terms already present in the transcript, but
|
||||
they are not evidence and must never supply a source range.
|
||||
@@ -0,0 +1,8 @@
|
||||
Extract physical places established by the provided Dungeons & Dragons
|
||||
transcript and cite where each place is identified.
|
||||
|
||||
Include planes, regions, settlements, districts, buildings, rooms, landmarks,
|
||||
routes, and geographic features. A generic label such as "the tavern" is
|
||||
allowed only when the transcript uses it for a specific place. Keep aliases and
|
||||
nested places when the transcript identifies them; do not merge or invent
|
||||
qualifiers for similarly named places.
|
||||
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "notarius.dnd.locations.llm",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["locations"],
|
||||
"properties": {
|
||||
"locations": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["name", "source_refs"],
|
||||
"properties": {
|
||||
"name": {"type": "string"},
|
||||
"source_refs": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["start_unit_id", "end_unit_id"],
|
||||
"properties": {
|
||||
"start_unit_id": {"type": "integer"},
|
||||
"end_unit_id": {"type": "integer"}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
86
internal/modules/dnd/extract/locations/canonicalize.go
Normal file
86
internal/modules/dnd/extract/locations/canonicalize.go
Normal file
@@ -0,0 +1,86 @@
|
||||
package locations
|
||||
|
||||
import (
|
||||
"sort"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
type orderedLocationResponse struct {
|
||||
value locationResponse
|
||||
earliest int
|
||||
hasEvidence bool
|
||||
}
|
||||
|
||||
func canonicalizeResponse(response *extractionResponse, order shared.SourceRefOrder, sourceID string) {
|
||||
if response == nil {
|
||||
return
|
||||
}
|
||||
ordered := make([]orderedLocationResponse, len(response.Locations))
|
||||
for index := range response.Locations {
|
||||
earliest, hasEvidence := canonicalizeLocation(&response.Locations[index], order, sourceID)
|
||||
ordered[index] = orderedLocationResponse{value: response.Locations[index], earliest: earliest, hasEvidence: hasEvidence}
|
||||
}
|
||||
sort.SliceStable(ordered, func(left, right int) bool {
|
||||
if ordered[left].hasEvidence != ordered[right].hasEvidence {
|
||||
return ordered[left].hasEvidence
|
||||
}
|
||||
if !ordered[left].hasEvidence {
|
||||
return false
|
||||
}
|
||||
return ordered[left].earliest < ordered[right].earliest
|
||||
})
|
||||
for index := range ordered {
|
||||
response.Locations[index] = ordered[index].value
|
||||
}
|
||||
}
|
||||
|
||||
func canonicalizeLocation(location *locationResponse, order shared.SourceRefOrder, sourceID string) (int, bool) {
|
||||
if location == nil {
|
||||
return 0, false
|
||||
}
|
||||
refs := order.Canonicalize(canonicalSourceRefs(location.SourceRefs, sourceID))
|
||||
location.SourceRefs = locationResponseRefs(refs)
|
||||
return order.EarliestValid(refs)
|
||||
}
|
||||
|
||||
func canonicalLocationList(response extractionResponse, sourceID string) dnd.LocationList {
|
||||
if response.Locations == nil {
|
||||
return dnd.LocationList{Locations: nil}
|
||||
}
|
||||
locations := make([]dnd.Location, len(response.Locations))
|
||||
for index, location := range response.Locations {
|
||||
refs := canonicalSourceRefs(location.SourceRefs, sourceID)
|
||||
locations[index] = dnd.Location{
|
||||
ID: identity.DeriveID(location.Name, refs),
|
||||
Name: location.Name,
|
||||
SourceRefs: refs,
|
||||
}
|
||||
}
|
||||
return dnd.LocationList{Locations: locations}
|
||||
}
|
||||
|
||||
func canonicalSourceRefs(values []locationSourceRefResponse, sourceID string) []source.SourceRef {
|
||||
if values == nil {
|
||||
return nil
|
||||
}
|
||||
refs := make([]source.SourceRef, len(values))
|
||||
for index, value := range values {
|
||||
refs[index] = source.SourceRef{SourceID: sourceID, StartUnitID: value.StartUnitID, EndUnitID: value.EndUnitID}
|
||||
}
|
||||
return refs
|
||||
}
|
||||
|
||||
func locationResponseRefs(values []source.SourceRef) []locationSourceRefResponse {
|
||||
if values == nil {
|
||||
return nil
|
||||
}
|
||||
refs := make([]locationSourceRefResponse, len(values))
|
||||
for index, value := range values {
|
||||
refs[index] = locationSourceRefResponse{StartUnitID: value.StartUnitID, EndUnitID: value.EndUnitID}
|
||||
}
|
||||
return refs
|
||||
}
|
||||
152
internal/modules/dnd/extract/locations/extractor.go
Normal file
152
internal/modules/dnd/extract/locations/extractor.go
Normal file
@@ -0,0 +1,152 @@
|
||||
// Package locations extracts source-grounded D&D physical location candidates.
|
||||
package locations
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
const (
|
||||
Key = "dnd/locations"
|
||||
mappingPolicy = "dnd.locations.extract_mapping.v1"
|
||||
)
|
||||
|
||||
var requiredCapabilities = []string{"chunks", "source.transcript"}
|
||||
var providedCapabilities = []string{"dnd.locations"}
|
||||
|
||||
var referenceSlotDescriptions = shared.ReferenceSlotDescriptions{
|
||||
Glossary: "Optional campaign glossary reference material used only for location disambiguation.",
|
||||
Party: "Optional party roster reference material used only for location disambiguation.",
|
||||
Players: "Optional player list reference material used only for location disambiguation.",
|
||||
Roster: "Deprecated alias for party roster reference material used only for location disambiguation.",
|
||||
}
|
||||
|
||||
func referenceSlots() []contracts.ReferenceSlot {
|
||||
return shared.ReferenceSlots(referenceSlotDescriptions)
|
||||
}
|
||||
|
||||
var _ contracts.Extractor[dnd.LocationList] = (*Extractor)(nil)
|
||||
var _ contracts.ManifestMetadataProvider = (*Extractor)(nil)
|
||||
var _ pipeline.CheckpointFingerprintProvider = (*Extractor)(nil)
|
||||
|
||||
type Options struct{}
|
||||
|
||||
type Extractor struct {
|
||||
llm contracts.StructuredLLMClient
|
||||
promptSHA string
|
||||
responseSchemaSHA string
|
||||
}
|
||||
|
||||
func New(llmClient contracts.StructuredLLMClient, _ Options, references ...contracts.ReferenceSet) (*Extractor, error) {
|
||||
if llmClient == nil {
|
||||
return nil, extractorErrorf("LLM client must not be nil")
|
||||
}
|
||||
if len(references) > 1 {
|
||||
return nil, extractorErrorf("at most one reference set may be supplied")
|
||||
}
|
||||
promptSHA, err := promptAssetMetadata()
|
||||
if err != nil {
|
||||
return nil, extractorErrorf("load prompt metadata: %w", err)
|
||||
}
|
||||
responseSchema, err := loadResponseSchema()
|
||||
if err != nil {
|
||||
return nil, extractorErrorf("load response schema: %w", err)
|
||||
}
|
||||
return &Extractor{llm: llmClient, promptSHA: promptSHA, responseSchemaSHA: responseSchema.SHA256}, nil
|
||||
}
|
||||
|
||||
func (e *Extractor) Key() string { return Key }
|
||||
|
||||
func (e *Extractor) ReferenceSlots() []contracts.ReferenceSlot { return referenceSlots() }
|
||||
|
||||
func (e *Extractor) ManifestMetadata() map[string]any {
|
||||
if e == nil {
|
||||
return nil
|
||||
}
|
||||
return map[string]any{
|
||||
"prompt_id": PromptID,
|
||||
"prompt_version": SchemaVersion,
|
||||
"prompt_sha256": e.promptSHA,
|
||||
"response_schema_key": string(ResponseSchemaKey),
|
||||
"response_schema_id": ResponseSchemaID,
|
||||
"response_schema_name": ResponseSchemaName,
|
||||
"response_schema_version": SchemaVersion,
|
||||
"response_schema_sha256": e.responseSchemaSHA,
|
||||
"identity_policy": identity.Policy,
|
||||
"mapping_policy": mappingPolicy,
|
||||
}
|
||||
}
|
||||
|
||||
func (e *Extractor) CheckpointFingerprints() []pipeline.CheckpointFingerprint {
|
||||
if e == nil {
|
||||
return nil
|
||||
}
|
||||
return []pipeline.CheckpointFingerprint{
|
||||
{Name: "prompt", Value: e.promptSHA},
|
||||
{Name: "response_schema", Value: e.responseSchemaSHA},
|
||||
{Name: "identity_policy", Value: identity.Policy},
|
||||
{Name: "mapping_policy", Value: mappingPolicy},
|
||||
}
|
||||
}
|
||||
|
||||
func (e *Extractor) Extract(ctx context.Context, req contracts.TypedExtractionRequest) (contracts.TypedExtractionResult[dnd.LocationList], error) {
|
||||
if e == nil {
|
||||
return contracts.TypedExtractionResult[dnd.LocationList]{}, extractorErrorf("extractor must not be nil")
|
||||
}
|
||||
if e.llm == nil {
|
||||
return contracts.TypedExtractionResult[dnd.LocationList]{}, extractorErrorf("LLM client must not be nil")
|
||||
}
|
||||
sourceInput, err := shared.PrepareChunkExtraction(ctx, req)
|
||||
if err != nil {
|
||||
return contracts.TypedExtractionResult[dnd.LocationList]{}, extractorErrorf("%w", err)
|
||||
}
|
||||
order := shared.NewSourceRefOrder(req.Source)
|
||||
|
||||
var response extractionResponse
|
||||
if _, err := e.llm.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
|
||||
StageName: Key, PromptID: PromptID, PromptVersion: SchemaVersion,
|
||||
ProfileID: req.LLMProfile, SessionID: req.SessionID,
|
||||
Inputs: shared.PromptInputs(sourceInput, req.References),
|
||||
}, &response); err != nil {
|
||||
return contracts.TypedExtractionResult[dnd.LocationList]{}, extractorErrorf("complete structured output: %w", err)
|
||||
}
|
||||
canonicalizeResponse(&response, order, req.Source.ID)
|
||||
return contracts.TypedExtractionResult[dnd.LocationList]{Value: canonicalLocationList(response, req.Source.ID)}, nil
|
||||
}
|
||||
|
||||
func ModuleSpec() pipeline.ModuleSpec {
|
||||
return pipeline.ModuleSpec{
|
||||
Key: Key, Stage: pipeline.StageExtract, ExecutionClass: contracts.ExecutionClassLLMBacked,
|
||||
Requires: append([]string(nil), requiredCapabilities...), Provides: append([]string(nil), providedCapabilities...),
|
||||
ArtifactKind: dnd.LocationListKind, ReferenceSlots: referenceSlots(),
|
||||
}
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ExtractorRegistry) error {
|
||||
return pipeline.RegisterExtractorBuilder(registry, ModuleSpec(), validateOptions, func(request pipeline.BuildRequest) (contracts.Extractor[dnd.LocationList], error) {
|
||||
options, err := DecodeOptions(request.Options)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return New(request.Dependencies.LLM, options, request.References)
|
||||
})
|
||||
}
|
||||
|
||||
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }
|
||||
|
||||
func DecodeOptions(options map[string]any) (Options, error) {
|
||||
if err := pipeline.RejectUnknownOptions(options); err != nil {
|
||||
return Options{}, extractorErrorf("%w", err)
|
||||
}
|
||||
return Options{}, nil
|
||||
}
|
||||
|
||||
func extractorErrorf(format string, args ...any) error {
|
||||
return fmt.Errorf("dnd locations extractor: "+format, args...)
|
||||
}
|
||||
125
internal/modules/dnd/extract/locations/extractor_test.go
Normal file
125
internal/modules/dnd/extract/locations/extractor_test.go
Normal file
@@ -0,0 +1,125 @@
|
||||
package locations
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||
)
|
||||
|
||||
func TestExtractMapsLocationsWithOwnedEvidenceAndDeterministicOrder(t *testing.T) {
|
||||
client := &fakeLocationsLLMClient{response: extractionResponse{Locations: []locationResponse{
|
||||
{Name: "The Tavern", SourceRefs: responseSourceRefs(3, 3)},
|
||||
{Name: "Old Mill", SourceRefs: []locationSourceRefResponse{{StartUnitID: 2, EndUnitID: 2}, {StartUnitID: 1, EndUnitID: 1}, {StartUnitID: 1, EndUnitID: 1}}},
|
||||
}}}
|
||||
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
|
||||
if err != nil {
|
||||
t.Fatalf("Extract() error = %v, want nil", err)
|
||||
}
|
||||
refs := []source.SourceRef{{SourceID: "session-locations", StartUnitID: 1, EndUnitID: 1}, {SourceID: "session-locations", StartUnitID: 2, EndUnitID: 2}}
|
||||
want := dnd.LocationList{Locations: []dnd.Location{
|
||||
{ID: identity.DeriveID("Old Mill", refs), Name: "Old Mill", SourceRefs: refs},
|
||||
{ID: identity.DeriveID("The Tavern", []source.SourceRef{{SourceID: "session-locations", StartUnitID: 3, EndUnitID: 3}}), Name: "The Tavern", SourceRefs: []source.SourceRef{{SourceID: "session-locations", StartUnitID: 3, EndUnitID: 3}}},
|
||||
}}
|
||||
if !reflect.DeepEqual(result.Value, want) {
|
||||
t.Fatalf("Value = %#v, want %#v", result.Value, want)
|
||||
}
|
||||
result.Value.Locations[0].SourceRefs[0].StartUnitID = 99
|
||||
for _, location := range client.response.Locations {
|
||||
for _, ref := range location.SourceRefs {
|
||||
if ref.StartUnitID == 99 {
|
||||
t.Fatal("result source references alias the model response")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractRetainsSameNameLocationsAtDifferentAnchors(t *testing.T) {
|
||||
client := &fakeLocationsLLMClient{response: extractionResponse{Locations: []locationResponse{
|
||||
{Name: "the tavern", SourceRefs: responseSourceRefs(1, 1)},
|
||||
{Name: "the tavern", SourceRefs: responseSourceRefs(3, 3)},
|
||||
}}}
|
||||
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
|
||||
if err != nil || len(result.Value.Locations) != 2 {
|
||||
t.Fatalf("Extract() = %#v, %v; want both same-name candidates", result, err)
|
||||
}
|
||||
if result.Value.Locations[0].ID == result.Value.Locations[1].ID || result.Value.Locations[0].Name != result.Value.Locations[1].Name {
|
||||
t.Fatalf("locations = %#v, want distinct evidence-anchored IDs", result.Value.Locations)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractPreservesInvalidCandidatesForValidators(t *testing.T) {
|
||||
client := &fakeLocationsLLMClient{content: []byte(`{"locations":[{"name":"","source_refs":[{"start_unit_id":0,"end_unit_id":-1}]}]}`)}
|
||||
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
|
||||
if err != nil {
|
||||
t.Fatalf("Extract() error = %v, want nil", err)
|
||||
}
|
||||
location := result.Value.Locations[0]
|
||||
if location.ID != "" || location.Name != "" || !reflect.DeepEqual(location.SourceRefs, []source.SourceRef{{SourceID: "session-locations", StartUnitID: 0, EndUnitID: -1}}) {
|
||||
t.Fatalf("location = %#v, want invalid candidate preserved", location)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractPassesReferencesWithoutTreatingThemAsEvidence(t *testing.T) {
|
||||
client := &fakeLocationsLLMClient{response: extractionResponse{Locations: []locationResponse{}}}
|
||||
req := extractionRequest()
|
||||
req.References = contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
|
||||
"glossary": {Slot: contracts.ReferenceSlot{Name: "glossary"}, Items: []contracts.ReferenceItem{{SlotName: "glossary", Content: []byte("Old Mill: abandoned granary")}}},
|
||||
}}
|
||||
if _, err := newExtractor(t, client).Extract(context.Background(), req); err != nil {
|
||||
t.Fatalf("Extract() error = %v", err)
|
||||
}
|
||||
inputs := client.requests[0].Inputs
|
||||
if string(inputs["glossary"].Content) != "Old Mill: abandoned granary" || strings.Contains(string(inputs["transcript"].Content), "abandoned granary") {
|
||||
t.Fatalf("prompt inputs = %#v, want separated reference material", inputs)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractDoesNotMutateRequestMaterials(t *testing.T) {
|
||||
client := &fakeLocationsLLMClient{response: extractionResponse{Locations: []locationResponse{{Name: "Old Mill", SourceRefs: responseSourceRefs(1, 1)}}}}
|
||||
req := extractionRequest()
|
||||
beforeUnits := append([]source.SourceUnit(nil), req.Source.Units...)
|
||||
beforeChunkUnits := append([]source.SourceUnit(nil), req.Chunk.Units...)
|
||||
beforeContent := append([]byte(nil), req.Chunk.Content...)
|
||||
if _, err := newExtractor(t, client).Extract(context.Background(), req); err != nil {
|
||||
t.Fatalf("Extract() error = %v", err)
|
||||
}
|
||||
if !reflect.DeepEqual(req.Source.Units, beforeUnits) || !reflect.DeepEqual(req.Chunk.Units, beforeChunkUnits) || !reflect.DeepEqual(req.Chunk.Content, beforeContent) {
|
||||
t.Fatalf("Extract() mutated request: %#v", req)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractHandlesEmptyOutputAndLocalFailures(t *testing.T) {
|
||||
empty, err := newExtractor(t, &fakeLocationsLLMClient{response: extractionResponse{Locations: []locationResponse{}}}).Extract(context.Background(), extractionRequest())
|
||||
if err != nil || empty.Value.Locations == nil || len(empty.Value.Locations) != 0 {
|
||||
t.Fatalf("empty Extract() = %#v, %v; want empty list", empty, err)
|
||||
}
|
||||
request := extractionRequest()
|
||||
var nilExtractor *Extractor
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
extractor *Extractor
|
||||
req contracts.TypedExtractionRequest
|
||||
want string
|
||||
}{
|
||||
{name: "nil extractor", extractor: nilExtractor, req: request, want: "extractor"},
|
||||
{name: "nil client", extractor: &Extractor{}, req: request, want: "LLM client"},
|
||||
{name: "preflight", extractor: newExtractor(t, &fakeLocationsLLMClient{}), req: mismatchedSourceInputRequest(request), want: "must match chunk"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if _, err := test.extractor.Extract(context.Background(), test.req); err == nil || !strings.Contains(err.Error(), "dnd locations") || !strings.Contains(err.Error(), test.want) {
|
||||
t.Fatalf("Extract() error = %v, want local context", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
_, err = newExtractor(t, &fakeLocationsLLMClient{err: errors.New("provider unavailable")}).Extract(context.Background(), request)
|
||||
if err == nil || !strings.Contains(err.Error(), "dnd locations") || !strings.Contains(err.Error(), "provider unavailable") {
|
||||
t.Fatalf("provider error = %v, want contextual provider error", err)
|
||||
}
|
||||
}
|
||||
15
internal/modules/dnd/extract/locations/model.go
Normal file
15
internal/modules/dnd/extract/locations/model.go
Normal file
@@ -0,0 +1,15 @@
|
||||
package locations
|
||||
|
||||
type extractionResponse struct {
|
||||
Locations []locationResponse `json:"locations"`
|
||||
}
|
||||
|
||||
type locationResponse struct {
|
||||
Name string `json:"name"`
|
||||
SourceRefs []locationSourceRefResponse `json:"source_refs"`
|
||||
}
|
||||
|
||||
type locationSourceRefResponse struct {
|
||||
StartUnitID int `json:"start_unit_id"`
|
||||
EndUnitID int `json:"end_unit_id"`
|
||||
}
|
||||
52
internal/modules/dnd/extract/locations/prompt_assets.go
Normal file
52
internal/modules/dnd/extract/locations/prompt_assets.go
Normal file
@@ -0,0 +1,52 @@
|
||||
package locations
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sync"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/promptfs"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
|
||||
)
|
||||
|
||||
const promptAssetRoot = "assets/prompts"
|
||||
|
||||
var promptAssetManifest = shared.PromptAssetManifest{
|
||||
ModuleDir: PromptID,
|
||||
ModuleFiles: []promptfs.ModulePromptFile{
|
||||
{Name: "dnd.locations.yaml", Path: "assets/prompts/dnd.locations.yaml"},
|
||||
{Name: "task.md", Path: "assets/prompts/task.md"},
|
||||
{Name: "instructions.md", Path: "assets/prompts/instructions.md"},
|
||||
},
|
||||
SharedFiles: []string{
|
||||
"common-dnd-system.md",
|
||||
"common-dnd-extraction-evidence.md",
|
||||
"common-dnd-identity.md",
|
||||
"common-dnd-transcript.md",
|
||||
"common-dnd-references.md",
|
||||
},
|
||||
}
|
||||
|
||||
func RegisterPromptAssets(registry *llm.AssetRegistry) error {
|
||||
promptFS, err := promptAssetManifest.PromptFS(embeddedAssets)
|
||||
if err != nil {
|
||||
return fmt.Errorf("prepare location prompt assets: %w", err)
|
||||
}
|
||||
if err := registry.RegisterPromptFS(promptFS, promptAssetRoot); err != nil {
|
||||
return err
|
||||
}
|
||||
return registry.RegisterSchemaFS(embeddedAssets, "assets/schemas")
|
||||
}
|
||||
|
||||
func promptAssetMetadata() (string, error) {
|
||||
promptAssetHashOnce.Do(func() {
|
||||
promptAssetHash, promptAssetHashErr = promptAssetManifest.Hash(embeddedAssets)
|
||||
})
|
||||
return promptAssetHash, promptAssetHashErr
|
||||
}
|
||||
|
||||
var (
|
||||
promptAssetHashOnce sync.Once
|
||||
promptAssetHash string
|
||||
promptAssetHashErr error
|
||||
)
|
||||
50
internal/modules/dnd/extract/locations/prompt_assets_test.go
Normal file
50
internal/modules/dnd/extract/locations/prompt_assets_test.go
Normal file
@@ -0,0 +1,50 @@
|
||||
package locations
|
||||
|
||||
import (
|
||||
"context"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||
"gitea.maximumdirect.net/eric/promptkit"
|
||||
)
|
||||
|
||||
func TestRegisterPromptAssetsPreparesLocationPrompt(t *testing.T) {
|
||||
for _, name := range []string{"common-dnd-system.md", "common-dnd-identity.md", "common-dnd-references.md", "common-dnd-transcript.md", "common-dnd-extraction-evidence.md"} {
|
||||
if !slices.Contains(promptAssetManifest.SharedFiles, name) {
|
||||
t.Fatalf("shared prompt assets = %#v, missing %q", promptAssetManifest.SharedFiles, name)
|
||||
}
|
||||
}
|
||||
registry := llm.NewAssetRegistry()
|
||||
if err := RegisterPromptAssets(registry); err != nil {
|
||||
t.Fatalf("RegisterPromptAssets() error = %v", err)
|
||||
}
|
||||
options, err := registry.PromptKitOptions()
|
||||
if err != nil {
|
||||
t.Fatalf("PromptKitOptions() error = %v", err)
|
||||
}
|
||||
options = append(options, promptkit.WithProfiles(promptkit.OpenAICompatibleProfile(promptkit.OpenAICompatibleProfileConfig{ID: "location-test-profile", Endpoint: "http://127.0.0.1:1/v1", Model: "location-test-model"})))
|
||||
engine, err := promptkit.NewEngine(promptkit.Config{Timeout: time.Second}, options...)
|
||||
if err != nil {
|
||||
t.Fatalf("NewEngine() error = %v", err)
|
||||
}
|
||||
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{PromptID: PromptID, PromptVersion: SchemaVersion, ProfileID: "location-test-profile", Inputs: map[string]promptkit.ArtifactRef{
|
||||
"transcript": promptkit.Inline(`{"units":[1]}`), "players": promptkit.Inline(" "), "party": promptkit.Inline(" "), "glossary": promptkit.Inline(" "),
|
||||
}})
|
||||
if err != nil {
|
||||
t.Fatalf("Prepare() error = %v", err)
|
||||
}
|
||||
if prepared.OutputContract.SchemaPath != "dnd_locations_llm.v1.json" {
|
||||
t.Fatalf("output contract = %#v", prepared.OutputContract)
|
||||
}
|
||||
if len(prepared.Messages) != 7 || !strings.Contains(prepared.Messages[3].Content, `"units"`) || strings.Contains(prepared.Messages[3].Content, "location-test") {
|
||||
t.Fatalf("prepared messages = %#v, want rendered transcript only in transcript message", prepared.Messages)
|
||||
}
|
||||
for _, index := range []int{2, 3, 6} {
|
||||
if prepared.Messages[index].CacheControl == nil || prepared.Messages[index].CacheControl.Type != promptkit.CacheControlEphemeral {
|
||||
t.Fatalf("message %d cache control = %#v, want ephemeral", index, prepared.Messages[index].CacheControl)
|
||||
}
|
||||
}
|
||||
}
|
||||
53
internal/modules/dnd/extract/locations/registry_test.go
Normal file
53
internal/modules/dnd/extract/locations/registry_test.go
Normal file
@@ -0,0 +1,53 @@
|
||||
package locations
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||
)
|
||||
|
||||
func TestModuleRegistrationAndMetadata(t *testing.T) {
|
||||
if _, err := New(nil, Options{}); err == nil || !strings.Contains(err.Error(), "LLM client") {
|
||||
t.Fatalf("New(nil) error = %v, want client rejection", err)
|
||||
}
|
||||
if _, err := New(&fakeLocationsLLMClient{}, Options{}, contracts.ReferenceSet{}, contracts.ReferenceSet{}); err == nil || !strings.Contains(err.Error(), "at most one") {
|
||||
t.Fatalf("New() error = %v, want reference-set rejection", err)
|
||||
}
|
||||
want := pipeline.ModuleSpec{Key: Key, Stage: pipeline.StageExtract, ExecutionClass: contracts.ExecutionClassLLMBacked, Requires: []string{"chunks", "source.transcript"}, Provides: []string{"dnd.locations"}, ArtifactKind: dnd.LocationListKind, ReferenceSlots: referenceSlots()}
|
||||
if got := ModuleSpec(); !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("ModuleSpec() = %#v, want %#v", got, want)
|
||||
}
|
||||
registry := pipeline.NewExtractorRegistry()
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatalf("Register() error = %v", err)
|
||||
}
|
||||
if got, ok := registry.Spec(Key); !ok || !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("registry spec = %#v, present = %t", got, ok)
|
||||
}
|
||||
if err := Register(nil); err == nil || !strings.Contains(err.Error(), "extractor registry") {
|
||||
t.Fatalf("Register(nil) error = %v, want registry rejection", err)
|
||||
}
|
||||
if _, err := DecodeOptions(map[string]any{"unknown": true}); err == nil {
|
||||
t.Fatal("DecodeOptions() accepted unknown option")
|
||||
}
|
||||
extractor := newExtractor(t, &fakeLocationsLLMClient{})
|
||||
metadata := extractor.ManifestMetadata()
|
||||
for key, value := range map[string]string{
|
||||
"prompt_id": PromptID, "prompt_version": SchemaVersion,
|
||||
"response_schema_key": string(ResponseSchemaKey), "response_schema_id": ResponseSchemaID,
|
||||
"response_schema_name": ResponseSchemaName, "response_schema_version": SchemaVersion,
|
||||
"identity_policy": identity.Policy, "mapping_policy": mappingPolicy,
|
||||
} {
|
||||
if metadata[key] != value {
|
||||
t.Fatalf("metadata[%q] = %#v, want %q", key, metadata[key], value)
|
||||
}
|
||||
}
|
||||
if got := extractor.CheckpointFingerprints(); len(got) != 4 || got[0].Name != "prompt" || got[1].Name != "response_schema" || got[2].Value != identity.Policy || got[3].Value != mappingPolicy {
|
||||
t.Fatalf("CheckpointFingerprints() = %#v", got)
|
||||
}
|
||||
}
|
||||
21
internal/modules/dnd/extract/locations/schema.go
Normal file
21
internal/modules/dnd/extract/locations/schema.go
Normal file
@@ -0,0 +1,21 @@
|
||||
package locations
|
||||
|
||||
import "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
|
||||
|
||||
const (
|
||||
PromptID = "dnd.locations"
|
||||
ResponseSchemaKey = llm.ResponseSchemaKey("dnd_locations_llm")
|
||||
ResponseSchemaID = "notarius.dnd.locations.llm"
|
||||
ResponseSchemaName = "notarius_dnd_locations_llm_v1"
|
||||
SchemaVersion = "v1"
|
||||
)
|
||||
|
||||
func loadResponseSchema() (llm.ResponseSchema, error) {
|
||||
return llm.LoadResponseSchema(embeddedAssets, llm.ResponseSchemaDefinition{
|
||||
Key: ResponseSchemaKey,
|
||||
ID: ResponseSchemaID,
|
||||
Version: SchemaVersion,
|
||||
Name: ResponseSchemaName,
|
||||
AssetPath: "assets/schemas/dnd_locations_llm.v1.json",
|
||||
})
|
||||
}
|
||||
25
internal/modules/dnd/extract/locations/schema_test.go
Normal file
25
internal/modules/dnd/extract/locations/schema_test.go
Normal file
@@ -0,0 +1,25 @@
|
||||
package locations
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestLocationResponseSchemaIsPrivateAndStructural(t *testing.T) {
|
||||
schema, err := loadResponseSchema()
|
||||
if err != nil {
|
||||
t.Fatalf("loadResponseSchema() error = %v", err)
|
||||
}
|
||||
if schema.Key != ResponseSchemaKey || schema.ID != ResponseSchemaID || schema.Name != ResponseSchemaName || schema.Version != SchemaVersion || !strings.HasPrefix(schema.SHA256, "sha256:") || !json.Valid(schema.JSONSchema) {
|
||||
t.Fatalf("schema = %#v, want location response schema", schema)
|
||||
}
|
||||
var document map[string]any
|
||||
if err := json.Unmarshal(schema.JSONSchema, &document); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
encoded, err := json.Marshal(document)
|
||||
if err != nil || strings.Contains(string(encoded), `"id"`) {
|
||||
t.Fatalf("schema = %s, want no durable ID field", encoded)
|
||||
}
|
||||
}
|
||||
89
internal/modules/dnd/extract/locations/test_helpers_test.go
Normal file
89
internal/modules/dnd/extract/locations/test_helpers_test.go
Normal file
@@ -0,0 +1,89 @@
|
||||
package locations
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
)
|
||||
|
||||
func extractionRequest() contracts.TypedExtractionRequest {
|
||||
doc := sourceDocument()
|
||||
chunk := &source.Chunk{
|
||||
ID: "session-locations:chunk:0", SourceID: doc.ID, Index: 0,
|
||||
Ref: source.SourceRef{SourceID: doc.ID, StartUnitID: 1, EndUnitID: 3},
|
||||
Content: []byte(`{"units":[1,2,3]}`), MediaType: "application/json",
|
||||
Units: append([]source.SourceUnit(nil), doc.Units...),
|
||||
}
|
||||
return contracts.TypedExtractionRequest{
|
||||
Source: doc, Chunk: chunk,
|
||||
SourceInput: contracts.NewLLMInputMaterial("source", chunk.MediaType, chunk.Content, "sha256:chunk", "file:///session-locations.json"),
|
||||
SessionID: "location-session", LLMProfile: "location-profile",
|
||||
}
|
||||
}
|
||||
|
||||
func sourceDocument() *source.SourceDocument {
|
||||
return &source.SourceDocument{ID: "session-locations", Kind: "transcript", Format: "application/json", Digest: "sha256:test", Units: []source.SourceUnit{
|
||||
{ID: 1, Kind: "transcript_segment", Text: "The party enters the Old Mill."},
|
||||
{ID: 2, Kind: "transcript_segment", Text: "They leave the old road behind."},
|
||||
{ID: 3, Kind: "transcript_segment", Text: "The tavern is quiet."},
|
||||
}}
|
||||
}
|
||||
|
||||
func responseSourceRefs(startUnitID, endUnitID int) []locationSourceRefResponse {
|
||||
return []locationSourceRefResponse{{StartUnitID: startUnitID, EndUnitID: endUnitID}}
|
||||
}
|
||||
|
||||
func newExtractor(t *testing.T, client contracts.StructuredLLMClient, references ...contracts.ReferenceSet) *Extractor {
|
||||
t.Helper()
|
||||
extractor, err := New(client, Options{}, references...)
|
||||
if err != nil {
|
||||
t.Fatalf("New() error = %v, want nil", err)
|
||||
}
|
||||
return extractor
|
||||
}
|
||||
|
||||
func mismatchedSourceInputRequest(req contracts.TypedExtractionRequest) contracts.TypedExtractionRequest {
|
||||
req.SourceInput = contracts.NewLLMInputMaterial("source", "application/json", []byte(`{"different":true}`), "sha256:other", "file:///other.json")
|
||||
return req
|
||||
}
|
||||
|
||||
type fakeLocationsLLMClient struct {
|
||||
response extractionResponse
|
||||
content []byte
|
||||
err error
|
||||
requests []contracts.StructuredCompletionRequest
|
||||
}
|
||||
|
||||
func (client *fakeLocationsLLMClient) CompleteStructured(_ context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
|
||||
client.requests = append(client.requests, cloneStructuredCompletionRequest(req))
|
||||
if client.err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, client.err
|
||||
}
|
||||
target, ok := out.(*extractionResponse)
|
||||
if !ok {
|
||||
return contracts.StructuredCompletionResponse{}, errors.New("unexpected output target")
|
||||
}
|
||||
content := append([]byte(nil), client.content...)
|
||||
if len(content) != 0 {
|
||||
if err := json.Unmarshal(content, target); err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, err
|
||||
}
|
||||
} else {
|
||||
*target = client.response
|
||||
var err error
|
||||
content, err = json.Marshal(client.response)
|
||||
if err != nil {
|
||||
return contracts.StructuredCompletionResponse{}, err
|
||||
}
|
||||
}
|
||||
return contracts.StructuredCompletionResponse{Content: content}, nil
|
||||
}
|
||||
|
||||
func cloneStructuredCompletionRequest(req contracts.StructuredCompletionRequest) contracts.StructuredCompletionRequest {
|
||||
req.Inputs = req.Inputs.Clone()
|
||||
return req
|
||||
}
|
||||
183
internal/modules/dnd/locations/identity/identity.go
Normal file
183
internal/modules/dnd/locations/identity/identity.go
Normal file
@@ -0,0 +1,183 @@
|
||||
// Package identity implements deterministic session-scoped location identity.
|
||||
package identity
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
"golang.org/x/text/cases"
|
||||
"golang.org/x/text/unicode/norm"
|
||||
)
|
||||
|
||||
const (
|
||||
// IdentityPolicy identifies the durable location ID derivation policy.
|
||||
IdentityPolicy = "dnd.locations.identity.v1"
|
||||
// Policy is an alias for IdentityPolicy.
|
||||
Policy = IdentityPolicy
|
||||
|
||||
idPrefix = "location:sha256:"
|
||||
)
|
||||
|
||||
// IssueCode identifies one deterministic registry identity problem.
|
||||
type IssueCode string
|
||||
|
||||
const (
|
||||
IssueEmptyCanonicalName IssueCode = "empty_canonical_name"
|
||||
IssueMissingEvidence IssueCode = "missing_evidence"
|
||||
IssueInvalidID IssueCode = "invalid_id"
|
||||
IssueIDMismatch IssueCode = "id_mismatch"
|
||||
IssueDuplicateID IssueCode = "duplicate_id"
|
||||
)
|
||||
|
||||
// Issue is an inspectable identity validation problem.
|
||||
type Issue struct {
|
||||
Code IssueCode
|
||||
RecordIndex int
|
||||
Value string
|
||||
}
|
||||
|
||||
func (i Issue) Error() string { return string(i.Code) }
|
||||
|
||||
// NormalizeDisplay returns the durable display form without changing spelling
|
||||
// or punctuation.
|
||||
func NormalizeDisplay(value string) string {
|
||||
return strings.Join(strings.Fields(value), " ")
|
||||
}
|
||||
|
||||
// ComparisonKey returns the stable key used for location identity comparisons.
|
||||
func ComparisonKey(value string) string {
|
||||
value = norm.NFKC.String(value)
|
||||
value = strings.Map(func(r rune) rune {
|
||||
switch r {
|
||||
case '\u2018', '\u2019', '\u02bc':
|
||||
return '\''
|
||||
default:
|
||||
return r
|
||||
}
|
||||
}, value)
|
||||
value = strings.Join(strings.Fields(value), " ")
|
||||
return cases.Fold().String(value)
|
||||
}
|
||||
|
||||
// DeriveID derives a location ID from name and the earliest canonical evidence
|
||||
// reference. It returns an empty string when either identity component is not
|
||||
// available, leaving validation to report the problem instead of manufacturing
|
||||
// an ID.
|
||||
func DeriveID(name string, refs []source.SourceRef) string {
|
||||
comparisonName := ComparisonKey(name)
|
||||
anchor, ok := earliestReference(refs)
|
||||
if comparisonName == "" || !ok {
|
||||
return ""
|
||||
}
|
||||
|
||||
input, err := json.Marshal([]any{
|
||||
IdentityPolicy,
|
||||
comparisonName,
|
||||
anchor.SourceID,
|
||||
anchor.StartUnitID,
|
||||
anchor.EndUnitID,
|
||||
})
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
digest := sha256.Sum256(input)
|
||||
return idPrefix + hex.EncodeToString(digest[:])
|
||||
}
|
||||
|
||||
// IDFor is an alias for DeriveID.
|
||||
func IDFor(name string, refs []source.SourceRef) string { return DeriveID(name, refs) }
|
||||
|
||||
// IsValidID reports whether value has the exact durable location ID syntax.
|
||||
func IsValidID(value string) bool {
|
||||
if len(value) != len(idPrefix)+sha256.Size*2 || !strings.HasPrefix(value, idPrefix) {
|
||||
return false
|
||||
}
|
||||
for _, r := range value[len(idPrefix):] {
|
||||
if !((r >= '0' && r <= '9') || (r >= 'a' && r <= 'f')) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// ValidID is an alias for IsValidID.
|
||||
func ValidID(value string) bool { return IsValidID(value) }
|
||||
|
||||
// ValidateList validates the identity members of list.
|
||||
func ValidateList(list dnd.LocationList) []Issue { return ValidateRegistry(list.Locations) }
|
||||
|
||||
// ValidateRegistry validates location IDs without modifying records or their
|
||||
// source references. Equal comparison names are allowed because their evidence
|
||||
// anchors are part of the identity policy.
|
||||
func ValidateRegistry(locations []dnd.Location) []Issue {
|
||||
issues := make([]Issue, 0)
|
||||
idOwners := make(map[string][]int)
|
||||
|
||||
for recordIndex, location := range locations {
|
||||
if ComparisonKey(location.Name) == "" {
|
||||
issues = append(issues, Issue{Code: IssueEmptyCanonicalName, RecordIndex: recordIndex, Value: location.Name})
|
||||
}
|
||||
if _, ok := earliestReference(location.SourceRefs); !ok {
|
||||
issues = append(issues, Issue{Code: IssueMissingEvidence, RecordIndex: recordIndex})
|
||||
}
|
||||
|
||||
if !IsValidID(location.ID) {
|
||||
issues = append(issues, Issue{Code: IssueInvalidID, RecordIndex: recordIndex, Value: location.ID})
|
||||
} else if expected := DeriveID(location.Name, location.SourceRefs); location.ID != expected {
|
||||
issues = append(issues, Issue{Code: IssueIDMismatch, RecordIndex: recordIndex, Value: location.ID})
|
||||
}
|
||||
if location.ID != "" {
|
||||
idOwners[location.ID] = append(idOwners[location.ID], recordIndex)
|
||||
}
|
||||
}
|
||||
|
||||
for recordIndex, location := range locations {
|
||||
if location.ID != "" && len(idOwners[location.ID]) > 1 && idOwners[location.ID][0] != recordIndex {
|
||||
issues = append(issues, Issue{Code: IssueDuplicateID, RecordIndex: recordIndex, Value: location.ID})
|
||||
}
|
||||
}
|
||||
|
||||
return issues
|
||||
}
|
||||
|
||||
func earliestReference(refs []source.SourceRef) (source.SourceRef, bool) {
|
||||
canonical := canonicalReferences(refs)
|
||||
if len(canonical) == 0 {
|
||||
return source.SourceRef{}, false
|
||||
}
|
||||
return canonical[0], true
|
||||
}
|
||||
|
||||
func canonicalReferences(refs []source.SourceRef) []source.SourceRef {
|
||||
canonical := make([]source.SourceRef, 0, len(refs))
|
||||
for _, ref := range refs {
|
||||
if validIdentityReference(ref) {
|
||||
canonical = append(canonical, ref)
|
||||
}
|
||||
}
|
||||
sort.Slice(canonical, func(left, right int) bool {
|
||||
if canonical[left].SourceID != canonical[right].SourceID {
|
||||
return canonical[left].SourceID < canonical[right].SourceID
|
||||
}
|
||||
if canonical[left].StartUnitID != canonical[right].StartUnitID {
|
||||
return canonical[left].StartUnitID < canonical[right].StartUnitID
|
||||
}
|
||||
return canonical[left].EndUnitID < canonical[right].EndUnitID
|
||||
})
|
||||
unique := canonical[:0]
|
||||
for _, ref := range canonical {
|
||||
if len(unique) == 0 || unique[len(unique)-1] != ref {
|
||||
unique = append(unique, ref)
|
||||
}
|
||||
}
|
||||
return unique
|
||||
}
|
||||
|
||||
func validIdentityReference(ref source.SourceRef) bool {
|
||||
return strings.TrimSpace(ref.SourceID) == ref.SourceID && ref.SourceID != "" && ref.StartUnitID > 0 && ref.EndUnitID > 0
|
||||
}
|
||||
146
internal/modules/dnd/locations/identity/identity_test.go
Normal file
146
internal/modules/dnd/locations/identity/identity_test.go
Normal file
@@ -0,0 +1,146 @@
|
||||
package identity
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
)
|
||||
|
||||
func TestNormalizeDisplayOnlyChangesWhitespace(t *testing.T) {
|
||||
if got, want := NormalizeDisplay(" The\u2003Old\nTavern "), "The Old Tavern"; got != want {
|
||||
t.Fatalf("NormalizeDisplay() = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestComparisonKeyNormalizesUnicodeWhitespaceAndApostrophes(t *testing.T) {
|
||||
for _, pair := range [][2]string{
|
||||
{" Caf\u00e9\u2003d\u2019Or ", "cafe\u0301 d'Or"},
|
||||
{"\uff34\uff48\uff45\u00a0\uff34\uff41\uff56\uff45\uff52\uff4e", "the tavern"},
|
||||
} {
|
||||
if left, right := ComparisonKey(pair[0]), ComparisonKey(pair[1]); left != right {
|
||||
t.Fatalf("ComparisonKey(%q) = %q, want value equal to %q", pair[0], left, pair[1])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeriveIDUsesDocumentedCompactJSONInput(t *testing.T) {
|
||||
refs := []source.SourceRef{
|
||||
{SourceID: "session-alpha", StartUnitID: 9, EndUnitID: 9},
|
||||
{SourceID: "session-alpha", StartUnitID: 3, EndUnitID: 9},
|
||||
{SourceID: "session-alpha", StartUnitID: 3, EndUnitID: 9},
|
||||
}
|
||||
const want = "location:sha256:379bd996e754e143b47e6b613a95166e58c111c7451750df7066ca816bb1866c"
|
||||
|
||||
if got := DeriveID(" The\u2003Tavern ", refs); got != want {
|
||||
t.Fatalf("DeriveID() = %q, want %q", got, want)
|
||||
}
|
||||
if got := IDFor("The Tavern", refs); got != want {
|
||||
t.Fatalf("IDFor() = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeriveIDUsesEarliestCanonicalEvidenceWithoutMutatingInput(t *testing.T) {
|
||||
refs := []source.SourceRef{
|
||||
{SourceID: "zeta", StartUnitID: 1, EndUnitID: 1},
|
||||
{SourceID: "alpha", StartUnitID: 8, EndUnitID: 8},
|
||||
{SourceID: "alpha", StartUnitID: 2, EndUnitID: 3},
|
||||
}
|
||||
wantRefs := append([]source.SourceRef(nil), refs...)
|
||||
|
||||
first := DeriveID("The Tavern", refs)
|
||||
second := DeriveID("The Tavern", []source.SourceRef{refs[2], refs[0], refs[1]})
|
||||
if first != second {
|
||||
t.Fatalf("DeriveID() changed when evidence order changed: %q != %q", first, second)
|
||||
}
|
||||
if !reflect.DeepEqual(refs, wantRefs) {
|
||||
t.Fatalf("DeriveID() mutated refs: got %#v, want %#v", refs, wantRefs)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeriveIDDistinguishesSameNameAtDifferentEvidenceAnchors(t *testing.T) {
|
||||
first := DeriveID("The Tavern", []source.SourceRef{{SourceID: "session", StartUnitID: 3, EndUnitID: 3}})
|
||||
second := DeriveID("the\u00a0tavern", []source.SourceRef{{SourceID: "session", StartUnitID: 18, EndUnitID: 18}})
|
||||
if first == "" || second == "" || first == second {
|
||||
t.Fatalf("same-name locations have IDs %q and %q, want distinct nonempty IDs", first, second)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeriveIDRejectsMissingIdentityComponents(t *testing.T) {
|
||||
for _, tt := range []struct {
|
||||
name string
|
||||
nameValue string
|
||||
refs []source.SourceRef
|
||||
}{
|
||||
{name: "blank name", nameValue: " \u2003 ", refs: []source.SourceRef{{SourceID: "session", StartUnitID: 1, EndUnitID: 1}}},
|
||||
{name: "missing refs", nameValue: "The Tavern"},
|
||||
{name: "malformed ref", nameValue: "The Tavern", refs: []source.SourceRef{{SourceID: " ", StartUnitID: 1, EndUnitID: 1}}},
|
||||
} {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
if got := DeriveID(tt.nameValue, tt.refs); got != "" {
|
||||
t.Fatalf("DeriveID() = %q, want empty ID", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsValidID(t *testing.T) {
|
||||
valid := DeriveID("The Tavern", []source.SourceRef{{SourceID: "session", StartUnitID: 1, EndUnitID: 1}})
|
||||
if !IsValidID(valid) || !ValidID(valid) {
|
||||
t.Fatalf("derived ID %q is not valid", valid)
|
||||
}
|
||||
for _, value := range []string{"", "location:sha256:", "location:sha256:" + strings.Repeat("A", 64), "location:sha256:" + strings.Repeat("0", 63)} {
|
||||
if IsValidID(value) {
|
||||
t.Fatalf("IsValidID(%q) = true, want false", value)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateRegistryAllowsSameComparisonNameWithDifferentAnchors(t *testing.T) {
|
||||
firstRefs := []source.SourceRef{{SourceID: "session", StartUnitID: 2, EndUnitID: 2}}
|
||||
secondRefs := []source.SourceRef{{SourceID: "session", StartUnitID: 8, EndUnitID: 8}}
|
||||
locations := []dnd.Location{
|
||||
{ID: DeriveID("The Tavern", firstRefs), Name: "The Tavern", SourceRefs: firstRefs},
|
||||
{ID: DeriveID("the\u00a0tavern", secondRefs), Name: "the\u00a0tavern", SourceRefs: secondRefs},
|
||||
}
|
||||
if issues := ValidateRegistry(locations); len(issues) != 0 {
|
||||
t.Fatalf("ValidateRegistry() = %#v, want no issues", issues)
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateRegistryReportsIdentityProblemsDeterministicallyWithoutMutation(t *testing.T) {
|
||||
refs := []source.SourceRef{{SourceID: "session", StartUnitID: 2, EndUnitID: 2}}
|
||||
validID := DeriveID("The Tavern", refs)
|
||||
locations := []dnd.Location{
|
||||
{ID: validID, Name: "The Tavern", SourceRefs: refs},
|
||||
{ID: validID, Name: "Elsewhere", SourceRefs: refs},
|
||||
{ID: "bad", Name: "", SourceRefs: nil},
|
||||
}
|
||||
wantLocations := cloneLocations(locations)
|
||||
|
||||
issues := ValidateList(dnd.LocationList{Locations: locations})
|
||||
want := []Issue{
|
||||
{Code: IssueIDMismatch, RecordIndex: 1, Value: validID},
|
||||
{Code: IssueEmptyCanonicalName, RecordIndex: 2, Value: ""},
|
||||
{Code: IssueMissingEvidence, RecordIndex: 2},
|
||||
{Code: IssueInvalidID, RecordIndex: 2, Value: "bad"},
|
||||
{Code: IssueDuplicateID, RecordIndex: 1, Value: validID},
|
||||
}
|
||||
if !reflect.DeepEqual(issues, want) {
|
||||
t.Fatalf("ValidateList() = %#v, want %#v", issues, want)
|
||||
}
|
||||
if !reflect.DeepEqual(locations, wantLocations) {
|
||||
t.Fatalf("ValidateList() mutated locations: got %#v, want %#v", locations, wantLocations)
|
||||
}
|
||||
}
|
||||
|
||||
func cloneLocations(input []dnd.Location) []dnd.Location {
|
||||
output := make([]dnd.Location, len(input))
|
||||
copy(output, input)
|
||||
for index := range output {
|
||||
output[index].SourceRefs = append([]source.SourceRef(nil), input[index].SourceRefs...)
|
||||
}
|
||||
return output
|
||||
}
|
||||
277
internal/modules/dnd/locations/registry/registry.go
Normal file
277
internal/modules/dnd/locations/registry/registry.go
Normal file
@@ -0,0 +1,277 @@
|
||||
// Package registry resolves normalized location artifacts into immutable
|
||||
// ID-grounding data for D&D extraction modules.
|
||||
package registry
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
locationcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/locations"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/diagnostics"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared/registryresolver"
|
||||
)
|
||||
|
||||
const (
|
||||
ReferenceSlot = "locations"
|
||||
MaxBytes = 1048576
|
||||
emptyPrompt = `{"locations":[]}`
|
||||
)
|
||||
|
||||
// Registry is an immutable, validated location registry prepared for prompt
|
||||
// grounding. All accessors return defensive copies.
|
||||
type Registry struct {
|
||||
bound bool
|
||||
list dnd.LocationList
|
||||
canonical []byte
|
||||
digest string
|
||||
projectionDigest string
|
||||
promptInput contracts.LLMInputMaterial
|
||||
lookupByID map[string]int
|
||||
}
|
||||
|
||||
// Resolver selects and memoizes immutable location registry views.
|
||||
type Resolver struct {
|
||||
resolver *registryresolver.Resolver[*Registry]
|
||||
}
|
||||
|
||||
// NewResolver validates a materialized construction-time location reference.
|
||||
// An empty slot is permitted because a generated reference is supplied only at
|
||||
// operation time.
|
||||
func NewResolver(references contracts.ReferenceSet) (*Resolver, error) {
|
||||
resolver, err := registryresolver.New(registryResolverConfig(), references)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Resolver{resolver: resolver}, nil
|
||||
}
|
||||
|
||||
// Seeded returns the validated construction-time registry.
|
||||
func (r *Resolver) Seeded() *Registry {
|
||||
if r == nil {
|
||||
return nil
|
||||
}
|
||||
return r.resolver.Seeded()
|
||||
}
|
||||
|
||||
// Resolve returns the generated operation-time registry when the locations
|
||||
// slot is present, otherwise it returns the construction-time registry.
|
||||
func (r *Resolver) Resolve(references contracts.ReferenceSet) (*Registry, error) {
|
||||
if r == nil || r.resolver == nil {
|
||||
return Resolve(references)
|
||||
}
|
||||
return r.resolver.Resolve(references)
|
||||
}
|
||||
|
||||
// Resolve validates an optional location-list reference. An absent reference
|
||||
// uses the canonical empty projection and has no durable registry identity.
|
||||
func Resolve(references contracts.ReferenceSet) (*Registry, error) {
|
||||
item, present, err := registryresolver.ResolveOptionalSingleItem(references, locationReferenceSpec())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !present {
|
||||
return emptyRegistry(), nil
|
||||
}
|
||||
return loadRegistry(item.Content)
|
||||
}
|
||||
|
||||
func registryResolverConfig() registryresolver.Config[*Registry] {
|
||||
return registryresolver.Config[*Registry]{
|
||||
Reference: locationReferenceSpec(),
|
||||
Absent: func() (*Registry, error) {
|
||||
return emptyRegistry(), nil
|
||||
},
|
||||
Load: loadRegistry,
|
||||
SemanticIdentity: func(registry *Registry) string {
|
||||
return registry.Digest()
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func locationReferenceSpec() registryresolver.ReferenceSpec {
|
||||
return registryresolver.ReferenceSpec{SlotName: ReferenceSlot, AcceptedMediaType: locationcodec.MediaType, MaxBytes: MaxBytes}
|
||||
}
|
||||
|
||||
func emptyRegistry() *Registry {
|
||||
content := []byte(emptyPrompt)
|
||||
projectionDigest := semanticDigest(content)
|
||||
return &Registry{
|
||||
list: dnd.LocationList{Locations: []dnd.Location{}},
|
||||
canonical: append([]byte(nil), content...),
|
||||
projectionDigest: projectionDigest,
|
||||
promptInput: contracts.NewLLMInputMaterial(ReferenceSlot, locationcodec.MediaType, content, projectionDigest, ""),
|
||||
lookupByID: map[string]int{},
|
||||
}
|
||||
}
|
||||
|
||||
func loadRegistry(referenceContent []byte) (*Registry, error) {
|
||||
codec := locationcodec.New()
|
||||
value, err := codec.Decode(referenceContent)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("decode location registry: invalid approved location JSON")
|
||||
}
|
||||
if issues := identity.ValidateList(value); len(issues) > 0 {
|
||||
return nil, fmt.Errorf("%s", formatIdentityIssues(issues))
|
||||
}
|
||||
content, err := codec.Encode(value)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("encode canonical location registry: approved location value could not be encoded")
|
||||
}
|
||||
|
||||
list := cloneLocationList(value)
|
||||
lookupByID := make(map[string]int, len(list.Locations))
|
||||
for index, location := range list.Locations {
|
||||
lookupByID[location.ID] = index
|
||||
}
|
||||
projection, err := promptProjection(list)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("encode location prompt projection: %w", err)
|
||||
}
|
||||
digest := semanticDigest(content)
|
||||
projectionDigest := semanticDigest(projection)
|
||||
return &Registry{
|
||||
bound: true,
|
||||
list: list,
|
||||
canonical: append([]byte(nil), content...),
|
||||
digest: digest,
|
||||
projectionDigest: projectionDigest,
|
||||
promptInput: contracts.NewLLMInputMaterial(ReferenceSlot, locationcodec.MediaType, projection, projectionDigest, ""),
|
||||
lookupByID: lookupByID,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Bound reports whether a location reference was supplied and validated.
|
||||
func (r *Registry) Bound() bool { return r != nil && r.bound }
|
||||
|
||||
// Locations returns a defensive copy of the validated location records.
|
||||
func (r *Registry) Locations() []dnd.Location {
|
||||
if r == nil {
|
||||
return nil
|
||||
}
|
||||
return cloneLocations(r.list.Locations)
|
||||
}
|
||||
|
||||
// List returns a defensive copy of the validated location list.
|
||||
func (r *Registry) List() dnd.LocationList {
|
||||
if r == nil {
|
||||
return dnd.LocationList{}
|
||||
}
|
||||
return cloneLocationList(r.list)
|
||||
}
|
||||
|
||||
// CanonicalBytes returns a defensive copy of the canonical durable JSON.
|
||||
func (r *Registry) CanonicalBytes() []byte {
|
||||
if r == nil {
|
||||
return nil
|
||||
}
|
||||
return append([]byte(nil), r.canonical...)
|
||||
}
|
||||
|
||||
// Digest returns the semantic digest of the canonical JSON, or an empty string
|
||||
// when the registry is unbound.
|
||||
func (r *Registry) Digest() string {
|
||||
if r == nil {
|
||||
return ""
|
||||
}
|
||||
return r.digest
|
||||
}
|
||||
|
||||
// ProjectionDigest returns the digest of the exact source-free prompt
|
||||
// projection, including for an unbound or empty registry.
|
||||
func (r *Registry) ProjectionDigest() string {
|
||||
if r == nil {
|
||||
return ""
|
||||
}
|
||||
return r.projectionDigest
|
||||
}
|
||||
|
||||
// Count returns the number of validated location records.
|
||||
func (r *Registry) Count() int {
|
||||
if r == nil {
|
||||
return 0
|
||||
}
|
||||
return len(r.list.Locations)
|
||||
}
|
||||
|
||||
// PromptInput returns the ordered ID-and-name projection without evidence or
|
||||
// reference provenance.
|
||||
func (r *Registry) PromptInput() contracts.LLMInputMaterial {
|
||||
if r == nil {
|
||||
return contracts.LLMInputMaterial{}
|
||||
}
|
||||
return r.promptInput.Clone()
|
||||
}
|
||||
|
||||
// Lookup returns the canonical location for an exact durable location ID.
|
||||
func (r *Registry) Lookup(id string) (dnd.Location, bool) {
|
||||
if r == nil {
|
||||
return dnd.Location{}, false
|
||||
}
|
||||
index, ok := r.lookupByID[id]
|
||||
if !ok {
|
||||
return dnd.Location{}, false
|
||||
}
|
||||
return cloneLocation(r.list.Locations[index]), true
|
||||
}
|
||||
|
||||
// Matches reports whether id resolves to exactly the supplied canonical name.
|
||||
func (r *Registry) Matches(id, name string) bool {
|
||||
location, ok := r.Lookup(id)
|
||||
return ok && location.Name == name
|
||||
}
|
||||
|
||||
func semanticDigest(content []byte) string {
|
||||
sum := sha256.Sum256(content)
|
||||
return "sha256:" + hex.EncodeToString(sum[:])
|
||||
}
|
||||
|
||||
type projectedLocation struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
|
||||
type projectedLocationList struct {
|
||||
Locations []projectedLocation `json:"locations"`
|
||||
}
|
||||
|
||||
func promptProjection(list dnd.LocationList) ([]byte, error) {
|
||||
projection := projectedLocationList{Locations: make([]projectedLocation, len(list.Locations))}
|
||||
for index, location := range list.Locations {
|
||||
projection.Locations[index] = projectedLocation{ID: location.ID, Name: location.Name}
|
||||
}
|
||||
return json.Marshal(projection)
|
||||
}
|
||||
|
||||
func formatIdentityIssues(issues []identity.Issue) string {
|
||||
parts := make([]string, len(issues))
|
||||
for index, issue := range issues {
|
||||
parts[index] = fmt.Sprintf("%s at record %d", issue.Code, issue.RecordIndex)
|
||||
}
|
||||
return diagnostics.Aggregate("validate location registry identity", parts)
|
||||
}
|
||||
|
||||
func cloneLocationList(value dnd.LocationList) dnd.LocationList {
|
||||
return dnd.LocationList{Locations: cloneLocations(value.Locations)}
|
||||
}
|
||||
|
||||
func cloneLocations(values []dnd.Location) []dnd.Location {
|
||||
if values == nil {
|
||||
return nil
|
||||
}
|
||||
cloned := make([]dnd.Location, len(values))
|
||||
for index, value := range values {
|
||||
cloned[index] = cloneLocation(value)
|
||||
}
|
||||
return cloned
|
||||
}
|
||||
|
||||
func cloneLocation(value dnd.Location) dnd.Location {
|
||||
value.SourceRefs = append([]source.SourceRef(nil), value.SourceRefs...)
|
||||
return value
|
||||
}
|
||||
191
internal/modules/dnd/locations/registry/registry_test.go
Normal file
191
internal/modules/dnd/locations/registry/registry_test.go
Normal file
@@ -0,0 +1,191 @@
|
||||
package registry
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
|
||||
locationcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/locations"
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/locations/identity"
|
||||
)
|
||||
|
||||
func TestResolveUnboundRegistryHasEmptyProjection(t *testing.T) {
|
||||
registry, err := Resolve(contracts.ReferenceSet{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
input := registry.PromptInput()
|
||||
if registry.Bound() || registry.Digest() != "" || registry.Count() != 0 || string(input.Content) != emptyPrompt {
|
||||
t.Fatalf("unbound registry = %#v, input = %#v", registry, input)
|
||||
}
|
||||
if registry.ProjectionDigest() == "" || input.Digest != registry.ProjectionDigest() || input.OriginURI != "" {
|
||||
t.Fatalf("projection digest/input = %q/%#v", registry.ProjectionDigest(), input)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveProjectsOrderedLocationsWithoutEvidence(t *testing.T) {
|
||||
registry := resolveList(t, registryFixture())
|
||||
if !registry.Bound() || registry.Count() != 2 || registry.Digest() == "" {
|
||||
t.Fatalf("registry identity = bound %t count %d digest %q", registry.Bound(), registry.Count(), registry.Digest())
|
||||
}
|
||||
projection := string(registry.PromptInput().Content)
|
||||
if !strings.Contains(projection, `"locations":[{"id":`) || !strings.Contains(projection, `"name":"The Tavern"`) || !strings.Contains(projection, `"name":"The Tavern"},{"id":`) {
|
||||
t.Fatalf("projection ordering = %s", projection)
|
||||
}
|
||||
for _, forbidden := range []string{"source_refs", "source_id", "session-alpha"} {
|
||||
if strings.Contains(projection, forbidden) {
|
||||
t.Fatalf("projection leaked %q: %s", forbidden, projection)
|
||||
}
|
||||
}
|
||||
if registry.PromptInput().Digest != registry.ProjectionDigest() || registry.Digest() == registry.ProjectionDigest() {
|
||||
t.Fatalf("full/projection digests = %q/%q", registry.Digest(), registry.ProjectionDigest())
|
||||
}
|
||||
}
|
||||
|
||||
func TestRegistryLookupUsesIDAndReturnsDefensiveCopies(t *testing.T) {
|
||||
registry := resolveList(t, registryFixture())
|
||||
first := registry.Locations()[0]
|
||||
if got, ok := registry.Lookup(first.ID); !ok || got.Name != first.Name || !registry.Matches(first.ID, first.Name) || registry.Matches(first.ID, "Other Tavern") {
|
||||
t.Fatalf("ID lookup/match = %#v, %t", got, ok)
|
||||
}
|
||||
if _, ok := registry.Lookup("The Tavern"); ok {
|
||||
t.Fatal("Lookup accepted a name as an ID")
|
||||
}
|
||||
|
||||
locations := registry.Locations()
|
||||
locations[0].Name = "changed"
|
||||
locations[0].SourceRefs[0].SourceID = "changed"
|
||||
canonical := registry.CanonicalBytes()
|
||||
canonical[0] = '['
|
||||
input := registry.PromptInput()
|
||||
input.Content[0] = '['
|
||||
if next, ok := registry.Lookup(first.ID); !ok || next.Name != first.Name || next.SourceRefs[0].SourceID != "session-alpha" {
|
||||
t.Fatalf("registry mutated through accessor: %#v, %t", next, ok)
|
||||
}
|
||||
if registry.CanonicalBytes()[0] != '{' || registry.PromptInput().Content[0] != '{' {
|
||||
t.Fatal("registry bytes mutated through accessor")
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveRejectsInvalidReferenceInputs(t *testing.T) {
|
||||
valid := registryFixture()
|
||||
content, err := locationcodec.New().Encode(valid)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
invalidIdentity := append([]byte(nil), content...)
|
||||
invalidIdentity = bytes.Replace(invalidIdentity, []byte(valid.Locations[0].ID), []byte("location:sha256:0000000000000000000000000000000000000000000000000000000000000000"), 1)
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
set contracts.ReferenceSet
|
||||
want string
|
||||
}{
|
||||
{"empty bound slot", referenceSet(), "exactly one item"},
|
||||
{"multiple items", referenceSet(item(content), item(content)), "exactly one item"},
|
||||
{"malformed JSON", referenceSet(item([]byte(`{"locations":[`))), "invalid approved"},
|
||||
{"wrong media type", referenceSet(contracts.ReferenceItem{MediaType: "text/plain", Content: content}), "media type must be"},
|
||||
{"oversized", referenceSet(item(make([]byte, MaxBytes+1))), "limit"},
|
||||
{"invalid identity", referenceSet(item(invalidIdentity)), "id_mismatch"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if _, err := Resolve(test.set); err == nil || !strings.Contains(err.Error(), test.want) {
|
||||
t.Fatalf("Resolve() error = %v, want %q", err, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolverHandlesConstructionAndOperationReferences(t *testing.T) {
|
||||
placeholder, err := NewResolver(referenceSet())
|
||||
if err != nil || placeholder.Seeded().Bound() {
|
||||
t.Fatalf("generated placeholder = %#v, %v; want unbound seed", placeholder, err)
|
||||
}
|
||||
|
||||
validContent := encodeList(t, registryFixture())
|
||||
malformed := referenceSet(item([]byte(`{"locations":[`)))
|
||||
if _, err := NewResolver(malformed); err == nil {
|
||||
t.Fatal("NewResolver(malformed) error = nil")
|
||||
}
|
||||
if _, err := placeholder.Resolve(malformed); err == nil {
|
||||
t.Fatal("Resolve(malformed) error = nil")
|
||||
}
|
||||
|
||||
staticContent := append([]byte(nil), validContent...)
|
||||
staticReferences := referenceSet(item(staticContent))
|
||||
seeded, err := NewResolver(staticReferences)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
resolved, err := seeded.Resolve(staticReferences)
|
||||
if err != nil || resolved != seeded.Seeded() {
|
||||
t.Fatalf("seed reuse = %p / %p, %v", resolved, seeded.Seeded(), err)
|
||||
}
|
||||
staticContent[0] = '['
|
||||
delete(staticReferences.Slots, ReferenceSlot)
|
||||
if seeded.Seeded().Count() != 2 || seeded.Seeded().CanonicalBytes()[0] != '{' {
|
||||
t.Fatalf("seeded registry retained construction references: %#v", seeded.Seeded())
|
||||
}
|
||||
if fallback, err := seeded.Resolve(contracts.ReferenceSet{}); err != nil || fallback != seeded.Seeded() {
|
||||
t.Fatalf("fallback = %#v, %v; want seeded registry", fallback, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolverReusesEquivalentCanonicalRegistries(t *testing.T) {
|
||||
resolver, err := NewResolver(referenceSet())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
content, err := locationcodec.New().Encode(registryFixture())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
firstSet := referenceSet(item(content))
|
||||
first, err := resolver.Resolve(firstSet)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
spaced := append([]byte("\n "), content...)
|
||||
spaced = append(spaced, '\n')
|
||||
second, err := resolver.Resolve(referenceSet(contracts.ReferenceItem{MediaType: "APPLICATION/JSON; charset=utf-8", Content: spaced}))
|
||||
if err != nil || second != first {
|
||||
t.Fatalf("equivalent canonical registry = %p / %p, %v", first, second, err)
|
||||
}
|
||||
}
|
||||
|
||||
func registryFixture() dnd.LocationList {
|
||||
firstRefs := []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 1}}
|
||||
secondRefs := []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 2, EndUnitID: 2}}
|
||||
return dnd.LocationList{Locations: []dnd.Location{
|
||||
{ID: identity.DeriveID("The Tavern", firstRefs), Name: "The Tavern", SourceRefs: firstRefs},
|
||||
{ID: identity.DeriveID("The Tavern", secondRefs), Name: "The Tavern", SourceRefs: secondRefs},
|
||||
}}
|
||||
}
|
||||
|
||||
func resolveList(t *testing.T, list dnd.LocationList) *Registry {
|
||||
t.Helper()
|
||||
registry, err := Resolve(referenceSet(item(encodeList(t, list))))
|
||||
if err != nil {
|
||||
t.Fatalf("Resolve() error = %v", err)
|
||||
}
|
||||
return registry
|
||||
}
|
||||
|
||||
func encodeList(t *testing.T, list dnd.LocationList) []byte {
|
||||
t.Helper()
|
||||
content, err := locationcodec.New().Encode(list)
|
||||
if err != nil {
|
||||
t.Fatalf("Encode() error = %v", err)
|
||||
}
|
||||
return content
|
||||
}
|
||||
|
||||
func item(content []byte) contracts.ReferenceItem {
|
||||
return contracts.ReferenceItem{SlotName: ReferenceSlot, MediaType: locationcodec.MediaType, Content: content}
|
||||
}
|
||||
|
||||
func referenceSet(items ...contracts.ReferenceItem) contracts.ReferenceSet {
|
||||
return contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{ReferenceSlot: {Items: items}}}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user