Repair reversed D&D evidence ranges
This commit is contained in:
@@ -1,6 +1,8 @@
|
|||||||
Transcript units are the only evidence for extracted events and factual claims.
|
Transcript units are the only evidence for extracted events and factual claims.
|
||||||
Every reported factual claim must be supported by cited transcript units. Use
|
Every reported factual claim must be supported by cited transcript units. Use
|
||||||
integer `start_unit_id` and `end_unit_id` values from the transcript.
|
integer `start_unit_id` and `end_unit_id` values from the transcript.
|
||||||
|
Within each range, `start_unit_id` must identify the earlier transcript unit and
|
||||||
|
`end_unit_id` the same or a later unit according to transcript order.
|
||||||
|
|
||||||
When supporting evidence is non-contiguous, use multiple narrow ranges rather
|
When supporting evidence is non-contiguous, use multiple narrow ranges rather
|
||||||
than a broad range that bridges unrelated conversation.
|
than a broad range that bridges unrelated conversation.
|
||||||
|
|||||||
@@ -120,11 +120,13 @@ manifest and prompt declaration.
|
|||||||
## Evidence, Candidates, And Normalization
|
## Evidence, Candidates, And Normalization
|
||||||
|
|
||||||
The current transcript is the only durable evidence source. Extractors assign
|
The current transcript is the only durable evidence source. Extractors assign
|
||||||
the current source identity, preserve candidate evidence ranges for validators,
|
the current source identity and losslessly order any reversed range whose two
|
||||||
and canonically order or remove exact duplicate ranges without asking the
|
endpoints resolve in that source, using transcript position rather than numeric
|
||||||
model to repair semantic errors. Campaign context and generated artifacts may
|
unit-ID order. They then canonically order ranges and remove exact duplicates.
|
||||||
ground names or control routing, but they never establish evidence for a D&D
|
This routine canonicalization does not request a retry or emit a warning.
|
||||||
result.
|
Unresolvable or wrong-source ranges remain unchanged for validators to reject.
|
||||||
|
Campaign context and generated artifacts may ground names or control routing,
|
||||||
|
but they never establish evidence for a D&D result.
|
||||||
|
|
||||||
Default chains keep responsibilities separate: structural validators assess the
|
Default chains keep responsibilities separate: structural validators assess the
|
||||||
candidate, source-reference validators resolve cited ranges against the current
|
candidate, source-reference validators resolve cited ranges against the current
|
||||||
|
|||||||
@@ -14,8 +14,8 @@ eligibility, and the separation of actionable process warnings from quality
|
|||||||
diagnostics. The remaining near-term work applies those completed foundations
|
diagnostics. The remaining near-term work applies those completed foundations
|
||||||
to domain review and empirical evaluation.
|
to domain review and empirical evaluation.
|
||||||
|
|
||||||
The active first production application is defined by
|
The active D&D reliability work is defined by
|
||||||
[D&D Combat Scene Semantic Validation](combat-scene-validation.md).
|
[D&D Source-Reference Endpoint Canonicalization](source-reference-canonicalization.md).
|
||||||
|
|
||||||
## Near-Term D&D Pipeline
|
## Near-Term D&D Pipeline
|
||||||
|
|
||||||
|
|||||||
96
docs/roadmap/source-reference-canonicalization.md
Normal file
96
docs/roadmap/source-reference-canonicalization.md
Normal file
@@ -0,0 +1,96 @@
|
|||||||
|
# D&D Source-Reference Endpoint Canonicalization
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Make model-supplied D&D evidence ranges resilient to reversed endpoints without
|
||||||
|
spending producer retries on a losslessly repairable representation error.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
The shared extraction-evidence prompt requires `start_unit_id` and
|
||||||
|
`end_unit_id`, but does not state that start means the earlier unit in
|
||||||
|
transcript order. The private LLM schemas cannot express that cross-field
|
||||||
|
relationship. Current extraction adapters sort and de-duplicate ranges but
|
||||||
|
preserve reversed endpoints, so deterministic source-reference validators
|
||||||
|
reject otherwise useful candidates and may exhaust the producer retry budget.
|
||||||
|
|
||||||
|
## Target State
|
||||||
|
|
||||||
|
- The shared D&D extraction-evidence prompt states that `start_unit_id` is the
|
||||||
|
earlier endpoint and `end_unit_id` is the same or a later endpoint according
|
||||||
|
to transcript order.
|
||||||
|
- The existing LLM-facing and durable `start_unit_id`/`end_unit_id` shapes stay
|
||||||
|
unchanged. No unordered-pair schema or compatibility migration is introduced.
|
||||||
|
- One helper owned by `internal/modules/dnd/shared` orders the endpoints of a
|
||||||
|
source reference using `source.DocumentIndex` positions. It never assumes
|
||||||
|
that numerically smaller unit IDs occur earlier.
|
||||||
|
- The helper swaps endpoints only when the reference belongs to the current
|
||||||
|
source and both endpoint IDs resolve in that source. Forward and single-unit
|
||||||
|
ranges remain unchanged. Wrong-source, missing, non-positive, or otherwise
|
||||||
|
unresolved endpoints remain unchanged for deterministic validators to reject.
|
||||||
|
- Every D&D extraction adapter that consumes model-supplied evidence ranges
|
||||||
|
applies this endpoint operation before its existing range sorting,
|
||||||
|
de-duplication, earliest-evidence calculation, and validation. The covered
|
||||||
|
artifact families are spells, NPC registry, NPC occurrences, item registry,
|
||||||
|
item occurrences, location registry, location occurrences, combat turns, and
|
||||||
|
enemy events.
|
||||||
|
- Successful endpoint swapping is ordinary deterministic canonicalization. It
|
||||||
|
does not emit a warning, quality diagnostic, or retry request.
|
||||||
|
- The exact raw model response remains available through existing model-candidate
|
||||||
|
and debug provenance. Only the typed candidate presented to validators and
|
||||||
|
later stages receives the canonical endpoint order.
|
||||||
|
- Source-reference validators remain strict and continue rejecting reversed
|
||||||
|
ranges that reach them from another producer or indicate an application bug.
|
||||||
|
Normalizers do not become a fallback repair boundary for invalid extraction
|
||||||
|
candidates.
|
||||||
|
|
||||||
|
## Required Work
|
||||||
|
|
||||||
|
1. Clarify endpoint ordering in
|
||||||
|
`assets/dnd/shared/prompts/common-dnd-extraction-evidence.md`, preserving one
|
||||||
|
byte-identical shared instruction for every consuming prompt.
|
||||||
|
2. Extend the shared source-reference ordering utility with a mutation-safe
|
||||||
|
endpoint-ordering operation that uses document position and preserves
|
||||||
|
unresolvable references.
|
||||||
|
3. Integrate the shared operation into all nine model-response-to-candidate
|
||||||
|
mappings listed above. Do not duplicate endpoint comparison logic in the
|
||||||
|
individual artifact packages.
|
||||||
|
4. Update `docs/internal/dnd.md` to distinguish lossless endpoint ordering from
|
||||||
|
malformed-reference validation. External artifact documentation and schemas
|
||||||
|
require no change because their contract already requires ordered ranges.
|
||||||
|
5. Add lean regression coverage for forward, reversed, single-unit,
|
||||||
|
non-monotonic-ID, wrong-source, and unresolved endpoint cases. Add enough
|
||||||
|
adapter-level coverage to prove the shared operation is wired into every
|
||||||
|
affected extraction family without duplicating the helper's full case
|
||||||
|
matrix in each package.
|
||||||
|
6. Update prompt-asset and fingerprint expectations affected by the shared
|
||||||
|
prompt revision, while avoiding exact token-, byte-length-, or prose-snapshot
|
||||||
|
change-detector tests.
|
||||||
|
|
||||||
|
## Non-Goals
|
||||||
|
|
||||||
|
- Changing durable source-reference schemas or replacing named endpoints with
|
||||||
|
an unordered LLM response shape.
|
||||||
|
- Weakening source-reference validation or silently repairing missing,
|
||||||
|
out-of-source, or out-of-chunk evidence.
|
||||||
|
- Applying this policy to scene chunk-plan ranges, which are not extraction
|
||||||
|
evidence references and have a separate ownership and validation boundary.
|
||||||
|
- Adding warnings or durable normalization observations for routine endpoint
|
||||||
|
ordering.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- A D&D extraction candidate whose two resolvable evidence endpoints are
|
||||||
|
reversed reaches validation as the equivalent forward range without a
|
||||||
|
producer retry.
|
||||||
|
- Ordering follows document position even when unit IDs are not numerically
|
||||||
|
increasing.
|
||||||
|
- Every affected D&D extractor uses the shared implementation; no module-local
|
||||||
|
endpoint-swap implementation remains.
|
||||||
|
- References that cannot be safely ordered remain unchanged and are rejected by
|
||||||
|
the existing deterministic validators where applicable.
|
||||||
|
- Raw model-candidate provenance remains byte-faithful to the provider response.
|
||||||
|
- LLM-facing and durable schemas, validator strictness, module keys, default
|
||||||
|
chains, and output contracts remain unchanged.
|
||||||
|
- Focused package tests, repository-wide tests, `go vet ./...`, and
|
||||||
|
`go build ./cmd/notarius` pass.
|
||||||
@@ -264,7 +264,7 @@ func TestChangedSemanticSpellCatalogFingerprintCannotResumeRecordedCheckpoint(t
|
|||||||
if _, decision := changedLoader.Normalize("spells", spellnormalize.Key, normalizeDependencies); decision.Reused {
|
if _, decision := changedLoader.Normalize("spells", spellnormalize.Key, normalizeDependencies); decision.Reused {
|
||||||
t.Fatalf("changed normalize fingerprint decision = %#v, want normalize checkpoint cold miss", decision)
|
t.Fatalf("changed normalize fingerprint decision = %#v, want normalize checkpoint cold miss", decision)
|
||||||
}
|
}
|
||||||
changedMapping := replaceCheckpointFingerprintValue(t, fingerprints, extractSpellMappingFingerprintName(), "dnd.spells.extract_mapping.v3")
|
changedMapping := replaceCheckpointFingerprintValue(t, fingerprints, extractSpellMappingFingerprintName(), "dnd.spells.extract_mapping.changed")
|
||||||
assertOnlyCheckpointFingerprintChanged(t, fingerprints, changedMapping, extractSpellMappingFingerprintName())
|
assertOnlyCheckpointFingerprintChanged(t, fingerprints, changedMapping, extractSpellMappingFingerprintName())
|
||||||
_, mappingLoader, err := checkpointHandlersForRun(settings, Options{}, materialized, changedMapping, llmFingerprints, []byte("same input"), nil, nil, "", "", LLMRuntimeOverrides{}, true)
|
_, mappingLoader, err := checkpointHandlersForRun(settings, Options{}, materialized, changedMapping, llmFingerprints, []byte("same input"), nil, nil, "", "", LLMRuntimeOverrides{}, true)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -45,7 +45,7 @@ func canonicalizeCombatTurn(turn *combatTurnResponse, order shared.SourceRefOrde
|
|||||||
if turn == nil {
|
if turn == nil {
|
||||||
return 0, false
|
return 0, false
|
||||||
}
|
}
|
||||||
refs := order.Canonicalize(canonicalSourceRefs(turn.SourceRefs, sourceID))
|
refs := order.Canonicalize(order.OrderEndpoints(canonicalSourceRefs(turn.SourceRefs, sourceID)))
|
||||||
turn.SourceRefs = combatResponseRefs(refs)
|
turn.SourceRefs = combatResponseRefs(refs)
|
||||||
return order.EarliestValid(refs)
|
return order.EarliestValid(refs)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ import (
|
|||||||
|
|
||||||
const (
|
const (
|
||||||
Key = "dnd/combat-turns"
|
Key = "dnd/combat-turns"
|
||||||
mappingPolicy = "dnd.combat_turns.extract_mapping.v2"
|
mappingPolicy = "dnd.combat_turns.extract_mapping.v3"
|
||||||
sceneGatePolicy = "dnd.combat_turns.scene_gate.v1"
|
sceneGatePolicy = "dnd.combat_turns.scene_gate.v1"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -25,8 +25,8 @@ func TestExtractMapsAndOrdersCombatTurnsBySourcePosition(t *testing.T) {
|
|||||||
},
|
},
|
||||||
{
|
{
|
||||||
Actor: "Aria", TurnKind: "reaction", SourceRefs: []combatSourceRefResponse{
|
Actor: "Aria", TurnKind: "reaction", SourceRefs: []combatSourceRefResponse{
|
||||||
{StartUnitID: 10, EndUnitID: 10},
|
{StartUnitID: 2, EndUnitID: 10},
|
||||||
{StartUnitID: 10, EndUnitID: 10},
|
{StartUnitID: 2, EndUnitID: 10},
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -42,7 +42,7 @@ func TestExtractMapsAndOrdersCombatTurnsBySourcePosition(t *testing.T) {
|
|||||||
if got := []string{result.Value.CombatTurns[0].Actor, result.Value.CombatTurns[1].Actor, result.Value.CombatTurns[2].Actor}; !reflect.DeepEqual(got, []string{"Aria", "Borin", "Unknown"}) {
|
if got := []string{result.Value.CombatTurns[0].Actor, result.Value.CombatTurns[1].Actor, result.Value.CombatTurns[2].Actor}; !reflect.DeepEqual(got, []string{"Aria", "Borin", "Unknown"}) {
|
||||||
t.Fatalf("actor order = %#v, want source-position order with invalid evidence last", got)
|
t.Fatalf("actor order = %#v, want source-position order with invalid evidence last", got)
|
||||||
}
|
}
|
||||||
wantRefs := []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 10, EndUnitID: 10}}
|
wantRefs := []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 10, EndUnitID: 2}}
|
||||||
if !reflect.DeepEqual(result.Value.CombatTurns[0].SourceRefs, wantRefs) {
|
if !reflect.DeepEqual(result.Value.CombatTurns[0].SourceRefs, wantRefs) {
|
||||||
t.Fatalf("canonical refs = %#v, want %#v", result.Value.CombatTurns[0].SourceRefs, wantRefs)
|
t.Fatalf("canonical refs = %#v, want %#v", result.Value.CombatTurns[0].SourceRefs, wantRefs)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ import (
|
|||||||
|
|
||||||
const (
|
const (
|
||||||
Key = "dnd/enemy-events"
|
Key = "dnd/enemy-events"
|
||||||
mappingPolicy = "dnd.enemy_events.extract_mapping.v1"
|
mappingPolicy = "dnd.enemy_events.extract_mapping.v2"
|
||||||
sceneGatePolicy = "dnd.enemy_events.scene_gate.v1"
|
sceneGatePolicy = "dnd.enemy_events.scene_gate.v1"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ import (
|
|||||||
|
|
||||||
func TestExtractMapsEnemyEventsInSourceOrder(t *testing.T) {
|
func TestExtractMapsEnemyEventsInSourceOrder(t *testing.T) {
|
||||||
client := &fakeEnemyEventsLLMClient{response: extractionResponse{Events: []enemyEventResponse{
|
client := &fakeEnemyEventsLLMClient{response: extractionResponse{Events: []enemyEventResponse{
|
||||||
{Name: "Ashfang", Kind: "killed", SourceRefs: []enemySourceRefResponse{{StartUnitID: 4, EndUnitID: 4}}},
|
{Name: "Ashfang", Kind: "killed", SourceRefs: []enemySourceRefResponse{{StartUnitID: 4, EndUnitID: 3}}},
|
||||||
{Name: "Ashfang", Kind: "engaged", SourceRefs: []enemySourceRefResponse{{StartUnitID: 1, EndUnitID: 1}, {StartUnitID: 1, EndUnitID: 1}}},
|
{Name: "Ashfang", Kind: "engaged", SourceRefs: []enemySourceRefResponse{{StartUnitID: 1, EndUnitID: 1}, {StartUnitID: 1, EndUnitID: 1}}},
|
||||||
{Name: "Orcs", Kind: "fled", SourceRefs: []enemySourceRefResponse{{StartUnitID: 3, EndUnitID: 3}}},
|
{Name: "Orcs", Kind: "fled", SourceRefs: []enemySourceRefResponse{{StartUnitID: 3, EndUnitID: 3}}},
|
||||||
{Name: "One orc", Kind: "captured", SourceRefs: []enemySourceRefResponse{{StartUnitID: 2, EndUnitID: 2}}},
|
{Name: "One orc", Kind: "captured", SourceRefs: []enemySourceRefResponse{{StartUnitID: 2, EndUnitID: 2}}},
|
||||||
@@ -31,14 +31,17 @@ func TestExtractMapsEnemyEventsInSourceOrder(t *testing.T) {
|
|||||||
dnd.EnemyEventKindEngaged,
|
dnd.EnemyEventKindEngaged,
|
||||||
dnd.EnemyEventKindCaptured,
|
dnd.EnemyEventKindCaptured,
|
||||||
dnd.EnemyEventKindIncapacitated,
|
dnd.EnemyEventKindIncapacitated,
|
||||||
dnd.EnemyEventKindFled,
|
|
||||||
dnd.EnemyEventKindKilled,
|
dnd.EnemyEventKindKilled,
|
||||||
|
dnd.EnemyEventKindFled,
|
||||||
}) {
|
}) {
|
||||||
t.Fatalf("event order = %#v", got)
|
t.Fatalf("event order = %#v", got)
|
||||||
}
|
}
|
||||||
if refs := result.Value.Events[0].SourceRefs; !reflect.DeepEqual(refs, []source.SourceRef{{SourceID: "combat-session", StartUnitID: 1, EndUnitID: 1}}) {
|
if refs := result.Value.Events[0].SourceRefs; !reflect.DeepEqual(refs, []source.SourceRef{{SourceID: "combat-session", StartUnitID: 1, EndUnitID: 1}}) {
|
||||||
t.Fatalf("canonical source refs = %#v", refs)
|
t.Fatalf("canonical source refs = %#v", refs)
|
||||||
}
|
}
|
||||||
|
if refs := result.Value.Events[3].SourceRefs; !reflect.DeepEqual(refs, []source.SourceRef{{SourceID: "combat-session", StartUnitID: 3, EndUnitID: 4}}) {
|
||||||
|
t.Fatalf("reversed source refs = %#v", refs)
|
||||||
|
}
|
||||||
if len(client.requests) != 1 {
|
if len(client.requests) != 1 {
|
||||||
t.Fatalf("LLM calls = %d, want 1", len(client.requests))
|
t.Fatalf("LLM calls = %d, want 1", len(client.requests))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ func canonicalEnemyEventList(response extractionResponse, order shared.SourceRef
|
|||||||
}
|
}
|
||||||
ordered := make([]orderedEnemyEvent, len(response.Events))
|
ordered := make([]orderedEnemyEvent, len(response.Events))
|
||||||
for index, event := range response.Events {
|
for index, event := range response.Events {
|
||||||
refs := order.Canonicalize(sourceRefs(event.SourceRefs, sourceID))
|
refs := order.Canonicalize(order.OrderEndpoints(sourceRefs(event.SourceRefs, sourceID)))
|
||||||
earliest, hasEvidence := order.EarliestValid(refs)
|
earliest, hasEvidence := order.EarliestValid(refs)
|
||||||
ordered[index] = orderedEnemyEvent{
|
ordered[index] = orderedEnemyEvent{
|
||||||
value: dnd.EnemyEvent{
|
value: dnd.EnemyEvent{
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ func canonicalItemOccurrenceList(response extractionResponse, order shared.Sourc
|
|||||||
if !found {
|
if !found {
|
||||||
return dnd.ItemOccurrenceList{}, fmt.Errorf("occurrences[%d].name is not in the item registry", index)
|
return dnd.ItemOccurrenceList{}, fmt.Errorf("occurrences[%d].name is not in the item registry", index)
|
||||||
}
|
}
|
||||||
refs := order.Canonicalize(itemOccurrenceSourceRefs(occurrence.SourceRefs, sourceID))
|
refs := order.Canonicalize(order.OrderEndpoints(itemOccurrenceSourceRefs(occurrence.SourceRefs, sourceID)))
|
||||||
earliest, hasEvidence := order.EarliestValid(refs)
|
earliest, hasEvidence := order.EarliestValid(refs)
|
||||||
ordered[index] = orderedItemOccurrence{
|
ordered[index] = orderedItemOccurrence{
|
||||||
value: dnd.ItemOccurrence{
|
value: dnd.ItemOccurrence{
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ const (
|
|||||||
ItemRegistryMaxBytes = itemregistry.MaxBytes
|
ItemRegistryMaxBytes = itemregistry.MaxBytes
|
||||||
)
|
)
|
||||||
|
|
||||||
const mappingPolicy = "dnd.item_occurrences.extract_mapping.v2"
|
const mappingPolicy = "dnd.item_occurrences.extract_mapping.v3"
|
||||||
|
|
||||||
var requiredCapabilities = []string{
|
var requiredCapabilities = []string{
|
||||||
"chunks",
|
"chunks",
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ import (
|
|||||||
|
|
||||||
func TestExtractGroundsOccurrencesInRequiredRegistry(t *testing.T) {
|
func TestExtractGroundsOccurrencesInRequiredRegistry(t *testing.T) {
|
||||||
id := itemidentity.DeriveID("Torch")
|
id := itemidentity.DeriveID("Torch")
|
||||||
rawResponse := []byte(`{"occurrences":[{"name":"Torch","kind":"lost","from":"party","source_refs":[{"start_unit_id":1,"end_unit_id":1}]}]}`)
|
rawResponse := []byte(`{"occurrences":[{"name":"Torch","kind":"lost","from":"party","source_refs":[{"start_unit_id":2,"end_unit_id":1}]}]}`)
|
||||||
client := &fakeItemOccurrencesLLMClient{content: append([]byte(nil), rawResponse...)}
|
client := &fakeItemOccurrencesLLMClient{content: append([]byte(nil), rawResponse...)}
|
||||||
req := extractionRequest()
|
req := extractionRequest()
|
||||||
req.References = itemRegistryReferences(t)
|
req.References = itemRegistryReferences(t)
|
||||||
@@ -27,7 +27,7 @@ func TestExtractGroundsOccurrencesInRequiredRegistry(t *testing.T) {
|
|||||||
if len(result.Value.Occurrences) != 1 || result.Value.Occurrences[0].ItemID != id || result.Value.Occurrences[0].Name != "Torch" {
|
if len(result.Value.Occurrences) != 1 || result.Value.Occurrences[0].ItemID != id || result.Value.Occurrences[0].Name != "Torch" {
|
||||||
t.Fatalf("occurrences = %#v", result.Value.Occurrences)
|
t.Fatalf("occurrences = %#v", result.Value.Occurrences)
|
||||||
}
|
}
|
||||||
if refs := result.Value.Occurrences[0].SourceRefs; len(refs) != 1 || refs[0].SourceID != req.Source.ID || refs[0].StartUnitID != 1 || refs[0].EndUnitID != 1 {
|
if refs := result.Value.Occurrences[0].SourceRefs; len(refs) != 1 || refs[0].SourceID != req.Source.ID || refs[0].StartUnitID != 1 || refs[0].EndUnitID != 2 {
|
||||||
t.Fatalf("occurrence evidence = %#v, want current-source unit range", refs)
|
t.Fatalf("occurrence evidence = %#v, want current-source unit range", refs)
|
||||||
}
|
}
|
||||||
request := client.requests[0]
|
request := client.requests[0]
|
||||||
|
|||||||
@@ -43,7 +43,7 @@ func canonicalizeItem(item *itemResponse, order shared.SourceRefOrder, sourceID
|
|||||||
return 0, false
|
return 0, false
|
||||||
}
|
}
|
||||||
item.Name = identity.NormalizeDisplay(item.Name)
|
item.Name = identity.NormalizeDisplay(item.Name)
|
||||||
refs := order.Canonicalize(canonicalSourceRefs(item.SourceRefs, sourceID))
|
refs := order.Canonicalize(order.OrderEndpoints(canonicalSourceRefs(item.SourceRefs, sourceID)))
|
||||||
item.SourceRefs = itemResponseRefs(refs)
|
item.SourceRefs = itemResponseRefs(refs)
|
||||||
return order.EarliestValid(refs)
|
return order.EarliestValid(refs)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ import (
|
|||||||
|
|
||||||
const (
|
const (
|
||||||
Key = "dnd/item-registry"
|
Key = "dnd/item-registry"
|
||||||
mappingPolicy = "dnd.item_registry.extract_mapping.v1"
|
mappingPolicy = "dnd.item_registry.extract_mapping.v2"
|
||||||
)
|
)
|
||||||
|
|
||||||
var requiredCapabilities = []string{"chunks", "source.transcript"}
|
var requiredCapabilities = []string{"chunks", "source.transcript"}
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ import (
|
|||||||
|
|
||||||
func TestExtractMapsItemsWithOwnedEvidenceAndDeterministicOrder(t *testing.T) {
|
func TestExtractMapsItemsWithOwnedEvidenceAndDeterministicOrder(t *testing.T) {
|
||||||
client := &fakeItemsLLMClient{response: extractionResponse{Items: []itemResponse{
|
client := &fakeItemsLLMClient{response: extractionResponse{Items: []itemResponse{
|
||||||
{Name: " Gold Pieces ", SourceRefs: responseSourceRefs(3, 3)},
|
{Name: " Gold Pieces ", SourceRefs: responseSourceRefs(3, 2)},
|
||||||
{Name: "Star Compass", SourceRefs: []itemSourceRefResponse{{StartUnitID: 2, EndUnitID: 2}, {StartUnitID: 1, EndUnitID: 1}, {StartUnitID: 1, EndUnitID: 1}}},
|
{Name: "Star Compass", SourceRefs: []itemSourceRefResponse{{StartUnitID: 2, EndUnitID: 2}, {StartUnitID: 1, EndUnitID: 1}, {StartUnitID: 1, EndUnitID: 1}}},
|
||||||
}}}
|
}}}
|
||||||
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
|
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
|
||||||
@@ -25,7 +25,7 @@ func TestExtractMapsItemsWithOwnedEvidenceAndDeterministicOrder(t *testing.T) {
|
|||||||
refs := []source.SourceRef{{SourceID: "session-items", StartUnitID: 1, EndUnitID: 1}, {SourceID: "session-items", StartUnitID: 2, EndUnitID: 2}}
|
refs := []source.SourceRef{{SourceID: "session-items", StartUnitID: 1, EndUnitID: 1}, {SourceID: "session-items", StartUnitID: 2, EndUnitID: 2}}
|
||||||
want := dnd.ItemRegistry{Items: []dnd.Item{
|
want := dnd.ItemRegistry{Items: []dnd.Item{
|
||||||
{ID: identity.DeriveID("Star Compass"), Name: "Star Compass", SourceRefs: refs},
|
{ID: identity.DeriveID("Star Compass"), Name: "Star Compass", SourceRefs: refs},
|
||||||
{ID: identity.DeriveID("Gold Pieces"), Name: "Gold Pieces", SourceRefs: []source.SourceRef{{SourceID: "session-items", StartUnitID: 3, EndUnitID: 3}}},
|
{ID: identity.DeriveID("Gold Pieces"), Name: "Gold Pieces", SourceRefs: []source.SourceRef{{SourceID: "session-items", StartUnitID: 2, EndUnitID: 3}}},
|
||||||
}}
|
}}
|
||||||
if !reflect.DeepEqual(result.Value, want) {
|
if !reflect.DeepEqual(result.Value, want) {
|
||||||
t.Fatalf("Value = %#v, want %#v", result.Value, want)
|
t.Fatalf("Value = %#v, want %#v", result.Value, want)
|
||||||
|
|||||||
@@ -27,7 +27,7 @@ func canonicalOccurrenceList(response extractionResponse, order shared.SourceRef
|
|||||||
if !ok {
|
if !ok {
|
||||||
return dnd.LocationOccurrenceList{}, fmt.Errorf("occurrence %d does not match a supplied location selector", index)
|
return dnd.LocationOccurrenceList{}, fmt.Errorf("occurrence %d does not match a supplied location selector", index)
|
||||||
}
|
}
|
||||||
refs := order.Canonicalize(canonicalSourceRefs(occurrence.SourceRefs, sourceID))
|
refs := order.Canonicalize(order.OrderEndpoints(canonicalSourceRefs(occurrence.SourceRefs, sourceID)))
|
||||||
earliest, hasEvidence := order.EarliestValid(refs)
|
earliest, hasEvidence := order.EarliestValid(refs)
|
||||||
ordered[index] = orderedOccurrence{value: dnd.LocationOccurrence{
|
ordered[index] = orderedOccurrence{value: dnd.LocationOccurrence{
|
||||||
LocationID: location.ID,
|
LocationID: location.ID,
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ import (
|
|||||||
|
|
||||||
const (
|
const (
|
||||||
Key = "dnd/location-occurrences"
|
Key = "dnd/location-occurrences"
|
||||||
mappingPolicy = "dnd.location_occurrences.extract_mapping.v2"
|
mappingPolicy = "dnd.location_occurrences.extract_mapping.v3"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ func TestExtractMapsKindsOrdersOccurrencesAndPreservesIndependentFacts(t *testin
|
|||||||
client := &fakeOccurrencesLLMClient{response: extractionResponse{Occurrences: []occurrenceResponse{
|
client := &fakeOccurrencesLLMClient{response: extractionResponse{Occurrences: []occurrenceResponse{
|
||||||
{Name: second.Name, RegistryRefs: registryRefs(second), Kind: "mentioned", SourceRefs: occurrenceRefs(30, 30)},
|
{Name: second.Name, RegistryRefs: registryRefs(second), Kind: "mentioned", SourceRefs: occurrenceRefs(30, 30)},
|
||||||
{Name: first.Name, RegistryRefs: registryRefs(first), Kind: "mentioned", SourceRefs: occurrenceRefs(10, 10)},
|
{Name: first.Name, RegistryRefs: registryRefs(first), Kind: "mentioned", SourceRefs: occurrenceRefs(10, 10)},
|
||||||
{Name: first.Name, RegistryRefs: registryRefs(first), Kind: "recalled", SourceRefs: occurrenceRefs(10, 10)},
|
{Name: first.Name, RegistryRefs: registryRefs(first), Kind: "recalled", SourceRefs: occurrenceRefs(20, 10)},
|
||||||
{Name: first.Name, RegistryRefs: registryRefs(first), Kind: "planned", SourceRefs: occurrenceRefs(10, 10)},
|
{Name: first.Name, RegistryRefs: registryRefs(first), Kind: "planned", SourceRefs: occurrenceRefs(10, 10)},
|
||||||
{Name: first.Name, RegistryRefs: registryRefs(first), Kind: "visited", SourceRefs: append(occurrenceRefs(10, 10), occurrenceRefs(10, 10)...)},
|
{Name: first.Name, RegistryRefs: registryRefs(first), Kind: "visited", SourceRefs: append(occurrenceRefs(10, 10), occurrenceRefs(10, 10)...)},
|
||||||
{Name: first.Name, RegistryRefs: registryRefs(first), Kind: "visited", SourceRefs: occurrenceRefs(20, 20)},
|
{Name: first.Name, RegistryRefs: registryRefs(first), Kind: "visited", SourceRefs: occurrenceRefs(20, 20)},
|
||||||
@@ -49,6 +49,9 @@ func TestExtractMapsKindsOrdersOccurrencesAndPreservesIndependentFacts(t *testin
|
|||||||
if !reflect.DeepEqual(got[0].SourceRefs, []source.SourceRef{{SourceID: req.Source.ID, StartUnitID: 10, EndUnitID: 10}}) {
|
if !reflect.DeepEqual(got[0].SourceRefs, []source.SourceRef{{SourceID: req.Source.ID, StartUnitID: 10, EndUnitID: 10}}) {
|
||||||
t.Fatalf("canonical evidence = %#v", got[0].SourceRefs)
|
t.Fatalf("canonical evidence = %#v", got[0].SourceRefs)
|
||||||
}
|
}
|
||||||
|
if !reflect.DeepEqual(got[2].SourceRefs, []source.SourceRef{{SourceID: req.Source.ID, StartUnitID: 10, EndUnitID: 20}}) {
|
||||||
|
t.Fatalf("reversed evidence = %#v", got[2].SourceRefs)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestExtractResolvesContextualSelectorsAndUsesCurrentTranscriptEvidenceOnly(t *testing.T) {
|
func TestExtractResolvesContextualSelectorsAndUsesCurrentTranscriptEvidenceOnly(t *testing.T) {
|
||||||
|
|||||||
@@ -42,7 +42,7 @@ func canonicalizeLocation(location *locationResponse, order shared.SourceRefOrde
|
|||||||
if location == nil {
|
if location == nil {
|
||||||
return 0, false
|
return 0, false
|
||||||
}
|
}
|
||||||
refs := order.Canonicalize(canonicalSourceRefs(location.SourceRefs, sourceID))
|
refs := order.Canonicalize(order.OrderEndpoints(canonicalSourceRefs(location.SourceRefs, sourceID)))
|
||||||
location.SourceRefs = locationResponseRefs(refs)
|
location.SourceRefs = locationResponseRefs(refs)
|
||||||
return order.EarliestValid(refs)
|
return order.EarliestValid(refs)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ import (
|
|||||||
|
|
||||||
const (
|
const (
|
||||||
Key = "dnd/location-registry"
|
Key = "dnd/location-registry"
|
||||||
mappingPolicy = "dnd.location_registry.extract_mapping.v1"
|
mappingPolicy = "dnd.location_registry.extract_mapping.v2"
|
||||||
)
|
)
|
||||||
|
|
||||||
var requiredCapabilities = []string{"chunks", "source.transcript"}
|
var requiredCapabilities = []string{"chunks", "source.transcript"}
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ import (
|
|||||||
|
|
||||||
func TestExtractMapsLocationsWithOwnedEvidenceAndDeterministicOrder(t *testing.T) {
|
func TestExtractMapsLocationsWithOwnedEvidenceAndDeterministicOrder(t *testing.T) {
|
||||||
client := &fakeLocationsLLMClient{response: extractionResponse{Locations: []locationResponse{
|
client := &fakeLocationsLLMClient{response: extractionResponse{Locations: []locationResponse{
|
||||||
{Name: "The Tavern", SourceRefs: responseSourceRefs(3, 3)},
|
{Name: "The Tavern", SourceRefs: responseSourceRefs(3, 2)},
|
||||||
{Name: "Old Mill", SourceRefs: []locationSourceRefResponse{{StartUnitID: 2, EndUnitID: 2}, {StartUnitID: 1, EndUnitID: 1}, {StartUnitID: 1, EndUnitID: 1}}},
|
{Name: "Old Mill", SourceRefs: []locationSourceRefResponse{{StartUnitID: 2, EndUnitID: 2}, {StartUnitID: 1, EndUnitID: 1}, {StartUnitID: 1, EndUnitID: 1}}},
|
||||||
}}}
|
}}}
|
||||||
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
|
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
|
||||||
@@ -25,7 +25,7 @@ func TestExtractMapsLocationsWithOwnedEvidenceAndDeterministicOrder(t *testing.T
|
|||||||
refs := []source.SourceRef{{SourceID: "session-locations", StartUnitID: 1, EndUnitID: 1}, {SourceID: "session-locations", StartUnitID: 2, EndUnitID: 2}}
|
refs := []source.SourceRef{{SourceID: "session-locations", StartUnitID: 1, EndUnitID: 1}, {SourceID: "session-locations", StartUnitID: 2, EndUnitID: 2}}
|
||||||
want := dnd.LocationRegistry{Locations: []dnd.Location{
|
want := dnd.LocationRegistry{Locations: []dnd.Location{
|
||||||
{ID: identity.DeriveID("Old Mill", refs), Name: "Old Mill", SourceRefs: refs},
|
{ID: identity.DeriveID("Old Mill", refs), Name: "Old Mill", SourceRefs: refs},
|
||||||
{ID: identity.DeriveID("The Tavern", []source.SourceRef{{SourceID: "session-locations", StartUnitID: 3, EndUnitID: 3}}), Name: "The Tavern", SourceRefs: []source.SourceRef{{SourceID: "session-locations", StartUnitID: 3, EndUnitID: 3}}},
|
{ID: identity.DeriveID("The Tavern", []source.SourceRef{{SourceID: "session-locations", StartUnitID: 2, EndUnitID: 3}}), Name: "The Tavern", SourceRefs: []source.SourceRef{{SourceID: "session-locations", StartUnitID: 2, EndUnitID: 3}}},
|
||||||
}}
|
}}
|
||||||
if !reflect.DeepEqual(result.Value, want) {
|
if !reflect.DeepEqual(result.Value, want) {
|
||||||
t.Fatalf("Value = %#v, want %#v", result.Value, want)
|
t.Fatalf("Value = %#v, want %#v", result.Value, want)
|
||||||
|
|||||||
@@ -47,7 +47,7 @@ func canonicalizeOccurrence(occurrence *occurrenceResponse, order shared.SourceR
|
|||||||
if occurrence == nil {
|
if occurrence == nil {
|
||||||
return 0, false
|
return 0, false
|
||||||
}
|
}
|
||||||
refs := order.Canonicalize(canonicalSourceRefs(occurrence.SourceRefs, sourceID))
|
refs := order.Canonicalize(order.OrderEndpoints(canonicalSourceRefs(occurrence.SourceRefs, sourceID)))
|
||||||
occurrence.SourceRefs = occurrenceResponseRefs(refs)
|
occurrence.SourceRefs = occurrenceResponseRefs(refs)
|
||||||
return order.EarliestValid(refs)
|
return order.EarliestValid(refs)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ import (
|
|||||||
|
|
||||||
const (
|
const (
|
||||||
Key = "dnd/npc-occurrences"
|
Key = "dnd/npc-occurrences"
|
||||||
mappingPolicy = "dnd.npc_occurrences.extract_mapping.v3"
|
mappingPolicy = "dnd.npc_occurrences.extract_mapping.v4"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ func TestExtractMapsEveryKindAndOrdersBySourcePosition(t *testing.T) {
|
|||||||
{Name: "Opponent", Kind: "combat_opponent", SourceRefs: occurrenceRefs(20, 20)},
|
{Name: "Opponent", Kind: "combat_opponent", SourceRefs: occurrenceRefs(20, 20)},
|
||||||
{Name: "Ally", Kind: "combat_ally", SourceRefs: occurrenceRefs(5, 5)},
|
{Name: "Ally", Kind: "combat_ally", SourceRefs: occurrenceRefs(5, 5)},
|
||||||
{Name: "Speaker", Kind: "dialogue", SourceRefs: append(occurrenceRefs(2, 2), occurrenceRefs(2, 2)...)},
|
{Name: "Speaker", Kind: "dialogue", SourceRefs: append(occurrenceRefs(2, 2), occurrenceRefs(2, 2)...)},
|
||||||
{Name: "Present", Kind: "noncombat_presence", SourceRefs: occurrenceRefs(7, 7)},
|
{Name: "Present", Kind: "noncombat_presence", SourceRefs: occurrenceRefs(7, 2)},
|
||||||
{Name: "Mentioned", Kind: "mentioned", SourceRefs: occurrenceRefs(10, 10)},
|
{Name: "Mentioned", Kind: "mentioned", SourceRefs: occurrenceRefs(10, 10)},
|
||||||
{Name: "Invalid", Kind: "unsupported", SourceRefs: occurrenceRefs(0, 0)},
|
{Name: "Invalid", Kind: "unsupported", SourceRefs: occurrenceRefs(0, 0)},
|
||||||
}}}
|
}}}
|
||||||
@@ -51,6 +51,9 @@ func TestExtractMapsEveryKindAndOrdersBySourcePosition(t *testing.T) {
|
|||||||
if refs := result.Value.Occurrences[1].SourceRefs; !reflect.DeepEqual(refs, []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 2, EndUnitID: 2}}) {
|
if refs := result.Value.Occurrences[1].SourceRefs; !reflect.DeepEqual(refs, []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 2, EndUnitID: 2}}) {
|
||||||
t.Fatalf("canonical source refs = %#v", refs)
|
t.Fatalf("canonical source refs = %#v", refs)
|
||||||
}
|
}
|
||||||
|
if refs := result.Value.Occurrences[2].SourceRefs; !reflect.DeepEqual(refs, []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 2, EndUnitID: 7}}) {
|
||||||
|
t.Fatalf("reversed source refs = %#v", refs)
|
||||||
|
}
|
||||||
if id := result.Value.Occurrences[0].NPCID; id != identity.DeriveID("Mentioned") {
|
if id := result.Value.Occurrences[0].NPCID; id != identity.DeriveID("Mentioned") {
|
||||||
t.Fatalf("durable NPC ID = %q, want registry identity", id)
|
t.Fatalf("durable NPC ID = %q, want registry identity", id)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -46,7 +46,7 @@ func canonicalizeNPC(npc *npcResponse, order shared.SourceRefOrder, sourceID str
|
|||||||
if npc == nil {
|
if npc == nil {
|
||||||
return 0, false
|
return 0, false
|
||||||
}
|
}
|
||||||
refs := order.Canonicalize(canonicalSourceRefs(npc.SourceRefs, sourceID))
|
refs := order.Canonicalize(order.OrderEndpoints(canonicalSourceRefs(npc.SourceRefs, sourceID)))
|
||||||
npc.SourceRefs = npcResponseRefs(refs)
|
npc.SourceRefs = npcResponseRefs(refs)
|
||||||
return order.EarliestValid(refs)
|
return order.EarliestValid(refs)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ import (
|
|||||||
|
|
||||||
const Key = "dnd/npc-registry"
|
const Key = "dnd/npc-registry"
|
||||||
|
|
||||||
const mappingPolicy = "dnd.npc_registry.extract_mapping.v2"
|
const mappingPolicy = "dnd.npc_registry.extract_mapping.v3"
|
||||||
|
|
||||||
var requiredCapabilities = []string{
|
var requiredCapabilities = []string{
|
||||||
"chunks",
|
"chunks",
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ import (
|
|||||||
func TestExtractReturnsCanonicalNPCRegistryFromPrivateResponse(t *testing.T) {
|
func TestExtractReturnsCanonicalNPCRegistryFromPrivateResponse(t *testing.T) {
|
||||||
client := &fakeNPCsLLMClient{response: extractionResponse{NPCs: []npcResponse{
|
client := &fakeNPCsLLMClient{response: extractionResponse{NPCs: []npcResponse{
|
||||||
{
|
{
|
||||||
Name: "Captain Vale", SourceRefs: responseSourceRefs(3, 3),
|
Name: "Captain Vale", SourceRefs: responseSourceRefs(3, 2),
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
Name: "Mira Thorn",
|
Name: "Mira Thorn",
|
||||||
@@ -34,7 +34,7 @@ func TestExtractReturnsCanonicalNPCRegistryFromPrivateResponse(t *testing.T) {
|
|||||||
}
|
}
|
||||||
want := dnd.NPCRegistry{NPCs: []dnd.NPC{
|
want := dnd.NPCRegistry{NPCs: []dnd.NPC{
|
||||||
{ID: identity.DeriveID("Mira Thorn"), Name: "Mira Thorn", SourceRefs: []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 2}, {SourceID: "session-alpha", StartUnitID: 2, EndUnitID: 2}}},
|
{ID: identity.DeriveID("Mira Thorn"), Name: "Mira Thorn", SourceRefs: []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 2}, {SourceID: "session-alpha", StartUnitID: 2, EndUnitID: 2}}},
|
||||||
{ID: identity.DeriveID("Captain Vale"), Name: "Captain Vale", SourceRefs: []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 3, EndUnitID: 3}}},
|
{ID: identity.DeriveID("Captain Vale"), Name: "Captain Vale", SourceRefs: []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 2, EndUnitID: 3}}},
|
||||||
}}
|
}}
|
||||||
if !reflect.DeepEqual(result.Value, want) {
|
if !reflect.DeepEqual(result.Value, want) {
|
||||||
t.Fatalf("Value = %#v, want %#v", result.Value, want)
|
t.Fatalf("Value = %#v, want %#v", result.Value, want)
|
||||||
|
|||||||
@@ -45,7 +45,7 @@ func canonicalizeSpellCast(spell *spellCastResponse, order shared.SourceRefOrder
|
|||||||
if spell == nil {
|
if spell == nil {
|
||||||
return 0, false
|
return 0, false
|
||||||
}
|
}
|
||||||
refs := order.Canonicalize(spellSourceRefs(spell.SourceRefs, sourceID))
|
refs := order.Canonicalize(order.OrderEndpoints(spellSourceRefs(spell.SourceRefs, sourceID)))
|
||||||
spell.SourceRefs = spellResponseRefs(refs)
|
spell.SourceRefs = spellResponseRefs(refs)
|
||||||
return order.EarliestValid(refs)
|
return order.EarliestValid(refs)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ import (
|
|||||||
const Key = "dnd/spells"
|
const Key = "dnd/spells"
|
||||||
const SchemaVersion = "v1"
|
const SchemaVersion = "v1"
|
||||||
|
|
||||||
const mappingPolicy = "dnd.spells.extract_mapping.v2"
|
const mappingPolicy = "dnd.spells.extract_mapping.v3"
|
||||||
|
|
||||||
const (
|
const (
|
||||||
NPCRegistryReferenceSlot = npcregistry.ReferenceSlot
|
NPCRegistryReferenceSlot = npcregistry.ReferenceSlot
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ func TestExtractReturnsCanonicalSpellListFromPrivateResponse(t *testing.T) {
|
|||||||
{
|
{
|
||||||
Caster: " Aria ",
|
Caster: " Aria ",
|
||||||
Spell: " Cure Wounds ",
|
Spell: " Cure Wounds ",
|
||||||
SourceRefs: responseSourceRefs(1, 2),
|
SourceRefs: responseSourceRefs(2, 1),
|
||||||
},
|
},
|
||||||
}}}
|
}}}
|
||||||
req := extractionRequest()
|
req := extractionRequest()
|
||||||
|
|||||||
@@ -30,6 +30,7 @@ func TestExtractionPromptComposition(t *testing.T) {
|
|||||||
npcSentinel = "npc-registry-sentinel"
|
npcSentinel = "npc-registry-sentinel"
|
||||||
catalogSentinel = "spell-catalog-sentinel"
|
catalogSentinel = "spell-catalog-sentinel"
|
||||||
evidenceSentinel = "Transcript units are the only evidence"
|
evidenceSentinel = "Transcript units are the only evidence"
|
||||||
|
endpointOrderSentinel = "according to transcript order"
|
||||||
combatPolicySentinel = "substantive active combat materially organizes the"
|
combatPolicySentinel = "substantive active combat materially organizes the"
|
||||||
)
|
)
|
||||||
registry := llm.NewAssetRegistry()
|
registry := llm.NewAssetRegistry()
|
||||||
@@ -51,12 +52,12 @@ func TestExtractionPromptComposition(t *testing.T) {
|
|||||||
suffixGroups [][]string
|
suffixGroups [][]string
|
||||||
middleSentinel string
|
middleSentinel string
|
||||||
}{
|
}{
|
||||||
{name: "npcs", promptID: npcextract.PromptID, promptVersion: npcextract.SchemaVersion, inputs: commonInputs, suffixGroups: [][]string{{evidenceSentinel}}},
|
{name: "npcs", promptID: npcextract.PromptID, promptVersion: npcextract.SchemaVersion, inputs: commonInputs, suffixGroups: [][]string{{evidenceSentinel, endpointOrderSentinel}}},
|
||||||
{name: "locations", promptID: locationextract.PromptID, promptVersion: locationextract.SchemaVersion, inputs: commonInputs, suffixGroups: [][]string{{evidenceSentinel}}},
|
{name: "locations", promptID: locationextract.PromptID, promptVersion: locationextract.SchemaVersion, inputs: commonInputs, suffixGroups: [][]string{{evidenceSentinel, endpointOrderSentinel}}},
|
||||||
{name: "item occurrences", promptID: itemoccurrenceextract.PromptID, promptVersion: itemoccurrenceextract.SchemaVersion, inputs: withPromptInputs(commonInputs, map[string]promptkit.ArtifactRef{
|
{name: "item occurrences", promptID: itemoccurrenceextract.PromptID, promptVersion: itemoccurrenceextract.SchemaVersion, inputs: withPromptInputs(commonInputs, map[string]promptkit.ArtifactRef{
|
||||||
"item_registry": promptkit.Inline(`{"items":[{"id":"item-registry-sentinel","name":"Torch"}]}`),
|
"item_registry": promptkit.Inline(`{"items":[{"id":"item-registry-sentinel","name":"Torch"}]}`),
|
||||||
}), suffixGroups: [][]string{{evidenceSentinel}, {"item-registry-sentinel"}}},
|
}), suffixGroups: [][]string{{evidenceSentinel, endpointOrderSentinel}, {"item-registry-sentinel"}}},
|
||||||
{name: "item registry", promptID: itemregistryextract.PromptID, promptVersion: itemregistryextract.SchemaVersion, inputs: commonInputs, suffixGroups: [][]string{{evidenceSentinel}}},
|
{name: "item registry", promptID: itemregistryextract.PromptID, promptVersion: itemregistryextract.SchemaVersion, inputs: commonInputs, suffixGroups: [][]string{{evidenceSentinel, endpointOrderSentinel}}},
|
||||||
{name: "scene descriptions", promptID: scenedescriptionextract.PromptID, promptVersion: scenedescriptionextract.SchemaVersion, inputs: commonInputs, middleSentinel: combatPolicySentinel},
|
{name: "scene descriptions", promptID: scenedescriptionextract.PromptID, promptVersion: scenedescriptionextract.SchemaVersion, inputs: commonInputs, middleSentinel: combatPolicySentinel},
|
||||||
{
|
{
|
||||||
name: "combat turns",
|
name: "combat turns",
|
||||||
@@ -65,7 +66,7 @@ func TestExtractionPromptComposition(t *testing.T) {
|
|||||||
inputs: withPromptInputs(commonInputs, map[string]promptkit.ArtifactRef{
|
inputs: withPromptInputs(commonInputs, map[string]promptkit.ArtifactRef{
|
||||||
"npc_registry": promptkit.Inline(`{"sentinel":"` + npcSentinel + `"}`),
|
"npc_registry": promptkit.Inline(`{"sentinel":"` + npcSentinel + `"}`),
|
||||||
}),
|
}),
|
||||||
suffixGroups: [][]string{{evidenceSentinel}, {npcSentinel}},
|
suffixGroups: [][]string{{evidenceSentinel, endpointOrderSentinel}, {npcSentinel}},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "enemy events",
|
name: "enemy events",
|
||||||
@@ -77,7 +78,7 @@ func TestExtractionPromptComposition(t *testing.T) {
|
|||||||
"npc_occurrences": promptkit.Inline(`{"sentinel":"npc-occurrences-sentinel"}`),
|
"npc_occurrences": promptkit.Inline(`{"sentinel":"npc-occurrences-sentinel"}`),
|
||||||
}),
|
}),
|
||||||
suffixGroups: [][]string{
|
suffixGroups: [][]string{
|
||||||
{evidenceSentinel},
|
{evidenceSentinel, endpointOrderSentinel},
|
||||||
{npcSentinel},
|
{npcSentinel},
|
||||||
{"combat-turns-sentinel", "npc-occurrences-sentinel"},
|
{"combat-turns-sentinel", "npc-occurrences-sentinel"},
|
||||||
},
|
},
|
||||||
@@ -89,7 +90,7 @@ func TestExtractionPromptComposition(t *testing.T) {
|
|||||||
inputs: withPromptInputs(commonInputs, map[string]promptkit.ArtifactRef{
|
inputs: withPromptInputs(commonInputs, map[string]promptkit.ArtifactRef{
|
||||||
"npc_registry": promptkit.Inline(`{"sentinel":"` + npcSentinel + `"}`),
|
"npc_registry": promptkit.Inline(`{"sentinel":"` + npcSentinel + `"}`),
|
||||||
}),
|
}),
|
||||||
suffixGroups: [][]string{{evidenceSentinel}, {npcSentinel}},
|
suffixGroups: [][]string{{evidenceSentinel, endpointOrderSentinel}, {npcSentinel}},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "location occurrences",
|
name: "location occurrences",
|
||||||
@@ -98,7 +99,7 @@ func TestExtractionPromptComposition(t *testing.T) {
|
|||||||
inputs: withPromptInputs(commonInputs, map[string]promptkit.ArtifactRef{
|
inputs: withPromptInputs(commonInputs, map[string]promptkit.ArtifactRef{
|
||||||
"location_registry": promptkit.Inline(`{"sentinel":"location-registry-sentinel"}`),
|
"location_registry": promptkit.Inline(`{"sentinel":"location-registry-sentinel"}`),
|
||||||
}),
|
}),
|
||||||
suffixGroups: [][]string{{evidenceSentinel}, {"location-registry-sentinel"}},
|
suffixGroups: [][]string{{evidenceSentinel, endpointOrderSentinel}, {"location-registry-sentinel"}},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "spells",
|
name: "spells",
|
||||||
@@ -108,7 +109,7 @@ func TestExtractionPromptComposition(t *testing.T) {
|
|||||||
"npc_registry": promptkit.Inline(`{"sentinel":"` + npcSentinel + `"}`),
|
"npc_registry": promptkit.Inline(`{"sentinel":"` + npcSentinel + `"}`),
|
||||||
"spell_catalog": promptkit.Inline(`{"sentinel":"` + catalogSentinel + `"}`),
|
"spell_catalog": promptkit.Inline(`{"sentinel":"` + catalogSentinel + `"}`),
|
||||||
}),
|
}),
|
||||||
suffixGroups: [][]string{{evidenceSentinel}, {npcSentinel}, {catalogSentinel}},
|
suffixGroups: [][]string{{evidenceSentinel, endpointOrderSentinel}, {npcSentinel}, {catalogSentinel}},
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -63,6 +63,29 @@ func (o SourceRefOrder) EarliestValid(refs []source.SourceRef) (int, bool) {
|
|||||||
return earliest, found
|
return earliest, found
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// OrderEndpoints returns an owned copy of refs with each resolvable range
|
||||||
|
// ordered by document position. References outside the indexed source, or
|
||||||
|
// whose endpoints cannot both be resolved, remain unchanged so validators can
|
||||||
|
// diagnose them.
|
||||||
|
func (o SourceRefOrder) OrderEndpoints(refs []source.SourceRef) []source.SourceRef {
|
||||||
|
if refs == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
ordered := append([]source.SourceRef{}, refs...)
|
||||||
|
for index := range ordered {
|
||||||
|
ref := &ordered[index]
|
||||||
|
if ref.SourceID != o.sourceID || ref.StartUnitID <= 0 || ref.EndUnitID <= 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
start, startOK := o.index.Position(ref.StartUnitID)
|
||||||
|
end, endOK := o.index.Position(ref.EndUnitID)
|
||||||
|
if startOK && endOK && start > end {
|
||||||
|
ref.StartUnitID, ref.EndUnitID = ref.EndUnitID, ref.StartUnitID
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ordered
|
||||||
|
}
|
||||||
|
|
||||||
// Canonicalize returns an owned, stable-sorted, exactly de-duplicated copy of
|
// Canonicalize returns an owned, stable-sorted, exactly de-duplicated copy of
|
||||||
// refs. It deliberately preserves invalid references for diagnostics.
|
// refs. It deliberately preserves invalid references for diagnostics.
|
||||||
func (o SourceRefOrder) Canonicalize(refs []source.SourceRef) []source.SourceRef {
|
func (o SourceRefOrder) Canonicalize(refs []source.SourceRef) []source.SourceRef {
|
||||||
|
|||||||
@@ -58,6 +58,38 @@ func TestSourceRefOrderCanonicalizePreservesRepresentationAndOwnership(t *testin
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestSourceRefOrderOrdersOnlyResolvableCurrentSourceEndpoints(t *testing.T) {
|
||||||
|
doc := unitRefSourceDocument(30, 10, 20)
|
||||||
|
order := NewSourceRefOrder(doc)
|
||||||
|
input := []source.SourceRef{
|
||||||
|
{SourceID: doc.ID, StartUnitID: 30, EndUnitID: 10},
|
||||||
|
{SourceID: doc.ID, StartUnitID: 20, EndUnitID: 30},
|
||||||
|
{SourceID: doc.ID, StartUnitID: 10, EndUnitID: 10},
|
||||||
|
{SourceID: "other", StartUnitID: 20, EndUnitID: 30},
|
||||||
|
{SourceID: doc.ID, StartUnitID: 999, EndUnitID: 30},
|
||||||
|
{SourceID: doc.ID, StartUnitID: 20, EndUnitID: 0},
|
||||||
|
}
|
||||||
|
want := []source.SourceRef{
|
||||||
|
{SourceID: doc.ID, StartUnitID: 30, EndUnitID: 10},
|
||||||
|
{SourceID: doc.ID, StartUnitID: 30, EndUnitID: 20},
|
||||||
|
{SourceID: doc.ID, StartUnitID: 10, EndUnitID: 10},
|
||||||
|
{SourceID: "other", StartUnitID: 20, EndUnitID: 30},
|
||||||
|
{SourceID: doc.ID, StartUnitID: 999, EndUnitID: 30},
|
||||||
|
{SourceID: doc.ID, StartUnitID: 20, EndUnitID: 0},
|
||||||
|
}
|
||||||
|
got := order.OrderEndpoints(input)
|
||||||
|
if !reflect.DeepEqual(got, want) {
|
||||||
|
t.Fatalf("OrderEndpoints() = %#v, want %#v", got, want)
|
||||||
|
}
|
||||||
|
got[0].StartUnitID = 999
|
||||||
|
if input[0].StartUnitID != 30 {
|
||||||
|
t.Fatal("OrderEndpoints() output aliases input")
|
||||||
|
}
|
||||||
|
if got := order.OrderEndpoints(nil); got != nil {
|
||||||
|
t.Fatalf("OrderEndpoints(nil) = %#v, want nil", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestSourceRefOrderSnapshotAndEarliestValid(t *testing.T) {
|
func TestSourceRefOrderSnapshotAndEarliestValid(t *testing.T) {
|
||||||
doc := unitRefSourceDocument(30, 10, 20)
|
doc := unitRefSourceDocument(30, 10, 20)
|
||||||
sourceID := doc.ID
|
sourceID := doc.ID
|
||||||
|
|||||||
@@ -51,11 +51,11 @@ func TestNPCOutputGroundsSpellAndCombatConsumersThroughOneOperation(t *testing.T
|
|||||||
t.Fatalf("Prepare() error = %v", err)
|
t.Fatalf("Prepare() error = %v", err)
|
||||||
}
|
}
|
||||||
for name, value := range map[string]string{
|
for name, value := range map[string]string{
|
||||||
"extract:npc_registry:dnd/npc-registry:mapping_policy": "dnd.npc_registry.extract_mapping.v2",
|
"extract:npc_registry:dnd/npc-registry:mapping_policy": "dnd.npc_registry.extract_mapping.v3",
|
||||||
"normalize:npc_registry:dnd/npc-registry:identity_policy": "dnd.npc_registry.identity.v1",
|
"normalize:npc_registry:dnd/npc-registry:identity_policy": "dnd.npc_registry.identity.v1",
|
||||||
"normalize:npc_registry:dnd/npc-registry:normalization_policy": npcnormalize.NormalizationPolicy,
|
"normalize:npc_registry:dnd/npc-registry:normalization_policy": npcnormalize.NormalizationPolicy,
|
||||||
"normalize:npc_registry:dnd/npc-registry:semantic_reconciliation_policy": semanticreconcile.Policy,
|
"normalize:npc_registry:dnd/npc-registry:semantic_reconciliation_policy": semanticreconcile.Policy,
|
||||||
"extract:spells:dnd/spells:mapping_policy": "dnd.spells.extract_mapping.v2",
|
"extract:spells:dnd/spells:mapping_policy": "dnd.spells.extract_mapping.v3",
|
||||||
"extract:combat:dnd/combat-turns:scene_gate_policy": "dnd.combat_turns.scene_gate.v1",
|
"extract:combat:dnd/combat-turns:scene_gate_policy": "dnd.combat_turns.scene_gate.v1",
|
||||||
} {
|
} {
|
||||||
assertFingerprintValue(t, prepared.CheckpointFingerprints(), name, value)
|
assertFingerprintValue(t, prepared.CheckpointFingerprints(), name, value)
|
||||||
|
|||||||
Reference in New Issue
Block a user