Introduce typed D&D spell artifacts

This commit is contained in:
2026-07-17 06:50:08 +00:00
parent b949e9bbc0
commit 142ba36695
27 changed files with 836 additions and 608 deletions

View File

@@ -6,6 +6,7 @@ extractor. Selectable extractor keys are cataloged in
## Identity
- Artifact kind: `dnd/spell-list`
- Prompt ID: `dnd.spells`
- Response schema key: `dnd_spells`
- Response schema ID: `notarius.dnd.spells`
@@ -13,6 +14,12 @@ extractor. Selectable extractor keys are cataloged in
- Response schema version: `v1`
- Media type: `application/json`
The durable JSON Schema is owned by the D&D spell artifact codec. The
extractor's private LLM response schema is a separate transport contract: its
source-reference objects omit `source_id`, which the extractor assigns while
mapping the response to the canonical artifact. The LLM DTO and transport
schema are not part of this durable contract.
The output contains canonical spell casts derived from transcript evidence.
Source IDs are assigned from the input identity; source-unit ranges identify
the evidence location.

View File

@@ -17,15 +17,14 @@ validator registry. Package-family registrars compose those leaf registrations
into the production catalog and own family-level policy such as default
validator chains and prompt asset collection.
Production input, chunk, and output packages register strict option decoders and
run-local builders. Preparation decodes their options into implementation-owned
values and injects dependencies; their operation requests contain run context,
not raw option maps or LLM clients. Production extract, merge, normalize, and
validator packages still use the explicit legacy raw registration APIs and
temporary adapters around zero-argument constructors. Their raw option maps and
LLM clients remain operation inputs while that part of the catalog migrates.
Typed registration is framework-ready, but no production artifact kind or
codec is registered yet.
Production input, chunk, output, and D&D spell-extract packages register strict
option decoders and run-local builders. Preparation decodes their options into
implementation-owned values and injects dependencies. The spell extractor is
typed over the canonical D&D model; a temporary raw adapter preserves the
current downstream production path. Production merge, normalize, and validator
packages still use the explicit legacy raw registration APIs and temporary
adapters around zero-argument constructors. Their raw option maps and LLM
clients remain operation inputs while that part of the catalog migrates.
Specs expose capability and execution metadata without constructing an
implementation. Registry entries separately expose option validation and
@@ -106,12 +105,16 @@ The spell extractor prepares a structured request from one chunk, the
chunk-scoped source input, the session, and optional D&D reference inputs. It
decodes the model response, assigns the generic source identity to every source
reference, canonicalizes duplicate references, orders spell casts by their
earliest cited unit, and returns raw JSON plus response-schema provenance.
earliest cited unit, and returns `dnd.SpellList`.
The package owns its embedded prompt, response schemas, and prompt/schema
manifest metadata. Shared D&D helpers keep prompt input names and source-unit
reference conversion consistent with the scene chunker. The extractor produces
raw output; production validators own approval policy.
The extractor owns its private model-response DTO, embedded prompt, LLM response
schema, strict option decoder, injected shared LLM client, and prompt/schema
manifest metadata. The separate `internal/modules/dnd/codec/spells` package
owns the durable schema and stable JSON representation for artifact kind
`dnd/spell-list`. Production composition currently wraps the typed extractor
with a raw adapter that encodes through this codec, so existing validators and
later stages remain unchanged. Shared D&D helpers keep prompt input names and
source-unit reference conversion consistent with the scene chunker.
The durable payload and manifest metadata shapes are defined in the
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).

View File

@@ -62,10 +62,12 @@ run-local construction closures. Preparation injects shared dependencies and
constructs input, chunk, validators, ordered lanes, and output before source
parsing. Production input, chunk, and output modules use strict construction-time
option decoding, and the LLM-backed scene chunker retains the injected shared
client. Production artifact-lane modules and validators do not register typed
variants yet and continue through explicitly named legacy raw registrations and
temporary zero-argument constructor adapters. The current runner rejects a
typed prepared lane instead of routing it through raw execution.
client. The D&D family registers the canonical `dnd/spell-list` codec and a
typed spell extractor. A temporary raw adapter serializes that typed result for
the still-raw production validators, merger, normalizer, and runner. Other
artifact-lane modules and validators continue through explicitly named legacy
raw registrations and temporary zero-argument constructor adapters. The current
runner rejects a typed prepared lane instead of routing it through raw execution.
## Production Extensions
@@ -79,7 +81,9 @@ Configuration. The implemented module packages are:
| `internal/modules/seriatim/input/transcript` | Parses the supported Seriatim transcript format into the generic source model. |
| `internal/modules/generic/chunk/units` | Splits ordered source units by unit count and overlap. |
| `internal/modules/dnd/chunk/scenes` | Produces contiguous D&D scene chunks from structured model output. |
| `internal/modules/dnd/extract/spells` | Produces source-grounded D&D spell-cast raw output. |
| `internal/modules/dnd` | Owns the canonical D&D spell-list and spell-cast artifact types. |
| `internal/modules/dnd/codec/spells` | Strictly decodes and stably encodes the durable D&D spell-list representation. |
| `internal/modules/dnd/extract/spells` | Maps private structured model output to canonical source-grounded D&D spell lists. |
| `internal/modules/generic/merge/appendorder` | Combines accepted extraction results in chunk order. |
| `internal/modules/generic/normalize/noop` | Preserves accepted merged output. |
| `internal/modules/generic/output/json` | Encodes manifests, lane payloads, warnings, and rejections as logical JSON files. |

View File

@@ -73,9 +73,11 @@ separate target namespaces; serialized registrations declare whether they
support chunks, artifacts, or both. Duplicate variants and exact Go-type
mismatches are rejected deterministically.
Production composition initializes the artifact codec registry without adding
codec entries, and production artifact-lane modules use the explicitly named
legacy raw registration APIs. A raw registration cannot satisfy a typed lane.
Production composition registers the D&D spell-list codec. The typed spell
extractor also registers a temporary raw adapter, which resolution selects
until its downstream production lane is typed. Other production artifact-lane
modules use the explicitly named legacy raw registration APIs. A standalone raw
registration cannot satisfy a typed lane.
A `ModuleSpec` declares its stage plus required and provided capabilities.
Chunk, extract, merge, and normalize specs may also declare reference slots.
@@ -84,8 +86,10 @@ and verify that a constructed implementation reports the registered key.
Builder registrations accept `ModuleDependencies` and cloned raw options through
one `BuildRequest`. Production input, chunk, and output builders decode those
options and retain typed values or injected dependencies in the constructed
implementation. Remaining production raw-stage registrations are adapted from
their zero-argument constructors during migration.
implementation. A typed extractor registration may explicitly supply a raw
adapter builder for a still-raw downstream lane; the adapter is selected as one
unit and does not expose the typed value to raw consumers. Remaining production
raw-stage registrations are adapted from their zero-argument constructors.
A `ValidatorSpec` declares a validator key and execution class. Resolution uses
the execution class to reject incompatible profile bindings before execution.

View File

@@ -106,13 +106,12 @@ func TestProductionCompatibilitySnapshot(t *testing.T) {
})
assertAssetNames(t, assets.SchemaFS, []string{
"dnd_scenes.v1.json",
"dnd_spells.v1.json",
"dnd_spells_llm.v1.json",
})
identitySnapshot := map[string]map[string]any{
"scenes": sceneManifestMetadata(t),
"spells": spells.New().ManifestMetadata(),
"spells": spellManifestMetadata(t),
}
for name, metadata := range identitySnapshot {
for _, key := range []string{"prompt_id", "prompt_version", "prompt_sha256", "response_schema_key", "response_schema_id", "response_schema_name", "response_schema_version", "response_schema_sha256"} {
@@ -348,7 +347,10 @@ func TestProductionLLMCallersShareScheduledClient(t *testing.T) {
}()
go func() {
started.Done()
_, err := spells.New().Extract(context.Background(), contracts.ExtractionRequest{Source: doc, Chunk: &chunk, LLMClient: client})
extractor, err := spells.New(client, spells.Options{})
if err == nil {
_, err = extractor.Extract(context.Background(), contracts.TypedExtractionRequest{Source: doc, Chunk: &chunk})
}
errs <- err
}()
started.Wait()
@@ -520,6 +522,15 @@ func sceneManifestMetadata(t *testing.T) map[string]any {
return chunker.ManifestMetadata()
}
func spellManifestMetadata(t *testing.T) map[string]any {
t.Helper()
extractor, err := spells.New(orderingProductionLLMClient{}, spells.Options{})
if err != nil {
t.Fatalf("construct spell extractor: %v", err)
}
return extractor.ManifestMetadata()
}
func assertAssetNames(t *testing.T, getFS func() (fs.FS, error), want []string) {
t.Helper()
fSys, err := getFS()

View File

@@ -138,8 +138,8 @@ func TestProductionCatalogIncludesProductionModulesValidatorsAndDefaults(t *test
if catalog.ArtifactCodecs == nil {
t.Fatal("production artifact codec registry = nil, want initialized empty registry")
}
if got := catalog.ArtifactCodecs.RegisteredKinds(); len(got) != 0 {
t.Fatalf("production artifact codec kinds = %#v, want legacy raw production path", got)
if got := catalog.ArtifactCodecs.RegisteredKinds(); !reflect.DeepEqual(got, []contracts.ArtifactKind{"dnd/spell-list"}) {
t.Fatalf("production artifact codec kinds = %#v, want dnd/spell-list", got)
}
moduleTests := []struct {

View File

@@ -23,6 +23,7 @@ type typedExtractorEntry struct {
valueType reflect.Type
validateOptions OptionValidator
builder func(BuildRequest) (any, error)
rawBuilder LegacyRawExtractorBuilder
}
func NewExtractorRegistry() *ExtractorRegistry {
@@ -92,6 +93,20 @@ func RegisterExtractor[T any](registry *ExtractorRegistry, spec ModuleSpec, cons
}
func RegisterExtractorBuilder[T any](registry *ExtractorRegistry, spec ModuleSpec, validateOptions OptionValidator, builder func(BuildRequest) (contracts.Extractor[T], error)) error {
return registerExtractorBuilder(registry, spec, validateOptions, builder, nil)
}
// RegisterExtractorBuilderWithRawAdapter registers a typed extractor while a
// raw downstream remains in use. Resolution selects the adapter until the
// registration is replaced with the typed-only builder.
func RegisterExtractorBuilderWithRawAdapter[T any](registry *ExtractorRegistry, spec ModuleSpec, validateOptions OptionValidator, builder func(BuildRequest) (contracts.Extractor[T], error), rawBuilder LegacyRawExtractorBuilder) error {
if rawBuilder == nil {
return fmt.Errorf("extractor raw adapter builder for %q must not be nil", strings.TrimSpace(spec.Key))
}
return registerExtractorBuilder(registry, spec, validateOptions, builder, rawBuilder)
}
func registerExtractorBuilder[T any](registry *ExtractorRegistry, spec ModuleSpec, validateOptions OptionValidator, builder func(BuildRequest) (contracts.Extractor[T], error), rawBuilder LegacyRawExtractorBuilder) error {
if registry == nil {
return fmt.Errorf("extractor registry must not be nil")
}
@@ -118,6 +133,7 @@ func RegisterExtractorBuilder[T any](registry *ExtractorRegistry, spec ModuleSpe
builder: func(request BuildRequest) (any, error) {
return builder(cloneBuildRequest(request))
},
rawBuilder: rawBuilder,
}
if registry.typedEntries == nil {
registry.typedEntries = make(map[string]typedExtractorEntry)
@@ -143,6 +159,12 @@ func (r *ExtractorRegistry) BuildLegacyRawWithRequest(key string, request BuildR
return nil, fmt.Errorf("extractor key must not be empty")
}
builder, ok := r.legacyBuilders[normalizedKey]
if !ok {
if entry, typedOK := r.typedEntries[normalizedKey]; typedOK && entry.rawBuilder != nil {
builder = entry.rawBuilder
ok = true
}
}
if !ok {
return nil, fmt.Errorf("legacy raw extractor %q is not registered", normalizedKey)
}
@@ -193,6 +215,11 @@ func (r *ExtractorRegistry) typedEntry(key string) (typedExtractorEntry, bool) {
return entry, ok
}
func (r *ExtractorRegistry) usesRawAdapter(key string) bool {
entry, ok := r.typedEntry(key)
return ok && entry.rawBuilder != nil
}
func (r *ExtractorRegistry) RegisteredKeys() []string {
if r == nil {
return nil

View File

@@ -353,10 +353,48 @@ func TestExtractorRegistryBuildRejectsEmptyKey(t *testing.T) {
}
}
func TestExtractorRegistryTypedRegistrationCanProvideRawAdapter(t *testing.T) {
registry := NewExtractorRegistry()
spec := ModuleSpec{Key: "typed-extractor", Stage: StageExtract, ArtifactKind: "test/value"}
if err := RegisterExtractorBuilderWithRawAdapter(registry, spec, func(map[string]any) error { return nil },
func(BuildRequest) (contracts.Extractor[registryTypedValue], error) {
return registryTypedExtractor{key: spec.Key}, nil
},
func(BuildRequest) (contracts.LegacyRawExtractor, error) {
return registryFakeExtractor{key: spec.Key}, nil
},
); err != nil {
t.Fatalf("RegisterExtractorBuilderWithRawAdapter() error = %v", err)
}
if !registry.usesRawAdapter(spec.Key) {
t.Fatal("usesRawAdapter() = false, want true")
}
if _, ok := registry.typedEntry(spec.Key); !ok {
t.Fatal("typedEntry() ok = false, want true")
}
adapter, err := registry.BuildLegacyRaw(spec.Key)
if err != nil || adapter.Key() != spec.Key {
t.Fatalf("BuildLegacyRaw() = %#v, %v", adapter, err)
}
if err := RegisterExtractorBuilderWithRawAdapter[registryTypedValue](NewExtractorRegistry(), spec, func(map[string]any) error { return nil }, nil, nil); err == nil || !strings.Contains(err.Error(), "raw adapter") {
t.Fatalf("nil raw builder error = %v, want raw adapter context", err)
}
}
type registryFakeExtractor struct {
key string
}
type registryTypedValue struct{ Value string }
type registryTypedExtractor struct{ key string }
func (extractor registryTypedExtractor) Key() string { return extractor.key }
func (registryTypedExtractor) ReferenceSlots() []contracts.ReferenceSlot { return nil }
func (registryTypedExtractor) Extract(context.Context, contracts.TypedExtractionRequest) (contracts.TypedExtractionResult[registryTypedValue], error) {
return contracts.TypedExtractionResult[registryTypedValue]{}, nil
}
func fakeExtractorConstructor(key string) LegacyRawExtractorConstructor {
return func() (contracts.LegacyRawExtractor, error) {
return registryFakeExtractor{key: key}, nil

View File

@@ -393,6 +393,9 @@ func resolveArtifactIdentity(pipelineID, laneID string, lane *ResolvedArtifactLa
if !ok {
return nil, fmt.Errorf("pipeline %q lane %q extract module %q declares artifact kind %q without a typed registration", pipelineID, laneID, lane.Extract.Module, extractSpec.ArtifactKind)
}
if catalog.Extractors.usesRawAdapter(lane.Extract.Module) {
return nil, nil
}
if catalog.ArtifactCodecs == nil {
return nil, fmt.Errorf("pipeline %q lane %q artifact codec registry must not be nil for kind %q", pipelineID, laneID, extractSpec.ArtifactKind)
}
@@ -639,12 +642,16 @@ func validatePipelineReferenceDefaults(
merge := resolveBinding(laneProfile.Merge, DefaultMergeModule)
var artifactType reflect.Type
if extractSpec.ArtifactKind != "" && catalog.Extractors != nil {
artifactKind := extractSpec.ArtifactKind
if catalog.Extractors != nil && catalog.Extractors.usesRawAdapter(extract.Module) {
artifactKind = ""
}
if artifactKind != "" && catalog.Extractors != nil {
if entry, ok := catalog.Extractors.typedEntry(extract.Module); ok {
artifactType = entry.valueType
}
}
mergeSpec, err := mergerSpecForArtifact(catalog, merge.Module, extractSpec.ArtifactKind, artifactType)
mergeSpec, err := mergerSpecForArtifact(catalog, merge.Module, artifactKind, artifactType)
if err != nil {
return moduleLookupError(pipelineID, laneID, StageMerge, merge.Module, err)
}
@@ -653,7 +660,7 @@ func validatePipelineReferenceDefaults(
}
normalize := resolveBinding(laneProfile.Normalize, DefaultNormalizeModule)
normalizeSpec, err := normalizerSpecForArtifact(catalog, normalize.Module, extractSpec.ArtifactKind, artifactType)
normalizeSpec, err := normalizerSpecForArtifact(catalog, normalize.Module, artifactKind, artifactType)
if err != nil {
return moduleLookupError(pipelineID, laneID, StageNormalize, normalize.Module, err)
}

View File

@@ -0,0 +1,110 @@
package spells
import (
"bytes"
"embed"
"encoding/json"
"fmt"
"io"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
)
const (
SchemaID = "notarius.dnd.spells"
SchemaName = "notarius_dnd_spells_v1"
SchemaVersion = "v1"
MediaType = "application/json"
)
//go:embed assets/schemas/dnd_spells.v1.json
var schemaAssets embed.FS
var _ contracts.ArtifactCodec[dnd.SpellList] = (*Codec)(nil)
type Codec struct{}
func New() *Codec { return &Codec{} }
func (c *Codec) Kind() contracts.ArtifactKind { return dnd.SpellListKind }
func (c *Codec) Schema() contracts.ArtifactSchema {
raw, err := schemaAssets.ReadFile("assets/schemas/dnd_spells.v1.json")
if err != nil {
return contracts.ArtifactSchema{}
}
return contracts.ArtifactSchema{ID: SchemaID, Name: SchemaName, Version: SchemaVersion, JSONSchema: raw}
}
func (c *Codec) MediaType() string { return MediaType }
func (c *Codec) Encode(value dnd.SpellList) ([]byte, error) {
if err := validate(value); err != nil {
return nil, fmt.Errorf("encode dnd spell list: %w", err)
}
return c.EncodeCandidate(value)
}
// EncodeCandidate provides the same stable representation before typed
// validators have approved a value on the temporary raw downstream path.
func (c *Codec) EncodeCandidate(value dnd.SpellList) ([]byte, error) {
content, err := json.Marshal(value)
if err != nil {
return nil, fmt.Errorf("encode dnd spell list: %w", err)
}
return content, nil
}
func (c *Codec) Decode(content []byte) (dnd.SpellList, error) {
decoder := json.NewDecoder(bytes.NewReader(content))
decoder.DisallowUnknownFields()
var value dnd.SpellList
if err := decoder.Decode(&value); err != nil {
return dnd.SpellList{}, fmt.Errorf("decode dnd spell list: %w", err)
}
var trailing any
if err := decoder.Decode(&trailing); err != io.EOF {
return dnd.SpellList{}, fmt.Errorf("decode dnd spell list: multiple JSON values")
}
if err := validate(value); err != nil {
return dnd.SpellList{}, fmt.Errorf("decode dnd spell list: %w", err)
}
return value, nil
}
func validate(value dnd.SpellList) error {
if value.SpellCasts == nil {
return fmt.Errorf("spell_casts must be present")
}
for index, spell := range value.SpellCasts {
if strings.TrimSpace(spell.Caster) == "" {
return fmt.Errorf("spell_casts[%d].caster must not be empty", index)
}
if strings.TrimSpace(spell.Spell) == "" {
return fmt.Errorf("spell_casts[%d].spell must not be empty", index)
}
if strings.TrimSpace(spell.Effect) == "" {
return fmt.Errorf("spell_casts[%d].effect must not be empty", index)
}
if strings.TrimSpace(spell.NarrativeDescription) == "" {
return fmt.Errorf("spell_casts[%d].narrative_description must not be empty", index)
}
if len(spell.SourceRefs) == 0 {
return fmt.Errorf("spell_casts[%d].source_refs must not be empty", index)
}
for refIndex, ref := range spell.SourceRefs {
if strings.TrimSpace(ref.SourceID) == "" {
return fmt.Errorf("spell_casts[%d].source_refs[%d].source_id must not be empty", index, refIndex)
}
if ref.StartUnitID <= 0 {
return fmt.Errorf("spell_casts[%d].source_refs[%d].start_unit_id must be positive", index, refIndex)
}
if ref.EndUnitID <= 0 {
return fmt.Errorf("spell_casts[%d].source_refs[%d].end_unit_id must be positive", index, refIndex)
}
}
}
return nil
}

View File

@@ -0,0 +1,111 @@
package spells
import (
"bytes"
"encoding/json"
"os"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
)
func TestCodecMatchesMaintainedDurableFixture(t *testing.T) {
raw, err := os.ReadFile("testdata/dnd_spells.v1.json")
if err != nil {
t.Fatalf("read durable fixture: %v", err)
}
codec := New()
value, err := codec.Decode(raw)
if err != nil {
t.Fatalf("Decode() error = %v, want nil", err)
}
want := dnd.SpellList{SpellCasts: []dnd.SpellCast{
{Caster: "Aria", Spell: "Cure Wounds", Effect: "Heals an injured ally.", NarrativeDescription: "Aria restores the fighter after the fight.", SourceRefs: []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 2}}},
{Caster: "Borin", Spell: "Fire Bolt", Effect: "Scorches the wight.", NarrativeDescription: "Borin hurls fire at the wight.", SourceRefs: []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 3, EndUnitID: 3}}},
}}
if !reflect.DeepEqual(value, want) {
t.Fatalf("Decode() = %#v, want %#v", value, want)
}
encoded, err := codec.Encode(value)
if err != nil {
t.Fatalf("Encode() error = %v, want nil", err)
}
var compact bytes.Buffer
if err := json.Compact(&compact, raw); err != nil {
t.Fatalf("compact durable fixture: %v", err)
}
if !bytes.Equal(encoded, compact.Bytes()) {
t.Fatalf("Encode() = %s, want stable durable JSON %s", encoded, compact.Bytes())
}
second, err := codec.Encode(value)
if err != nil || !bytes.Equal(second, encoded) {
t.Fatalf("second Encode() = %s, %v; want deterministic bytes", second, err)
}
}
func TestCodecOwnsDurableSchemaAndRegistersExactType(t *testing.T) {
codec := New()
schema := codec.Schema()
if codec.Kind() != dnd.SpellListKind || codec.MediaType() != MediaType {
t.Fatalf("codec identity = %q/%q", codec.Kind(), codec.MediaType())
}
if schema.ID != SchemaID || schema.Name != SchemaName || schema.Version != SchemaVersion || !json.Valid(schema.JSONSchema) {
t.Fatalf("schema = %#v, want durable spell schema", schema)
}
var document map[string]any
if err := json.Unmarshal(schema.JSONSchema, &document); err != nil || document["$id"] != SchemaID {
t.Fatalf("durable schema document = %#v, %v", document, err)
}
registry := pipeline.NewArtifactCodecRegistry()
if err := pipeline.RegisterArtifactCodec(registry, codec); err != nil {
t.Fatalf("RegisterArtifactCodec() error = %v", err)
}
spec, ok := registry.Spec(dnd.SpellListKind)
if !ok || spec.SchemaDigest != contracts.DigestArtifactSchema(schema) {
t.Fatalf("registered spec = %#v, %t", spec, ok)
}
}
func TestCodecStrictlyRejectsInvalidRepresentations(t *testing.T) {
codec := New()
tests := []struct {
name string
raw string
want string
}{
{name: "unknown", raw: `{"spell_casts":[],"unexpected":true}`, want: "unknown field"},
{name: "trailing", raw: `{"spell_casts":[]} {}`, want: "multiple JSON values"},
{name: "missing", raw: `{}`, want: "spell_casts must be present"},
{name: "invalid evidence", raw: `{"spell_casts":[{"caster":"Aria","spell":"Cure Wounds","effect":"Heals","narrative_description":"Aria heals","source_refs":[{"source_id":"session","start_unit_id":0,"end_unit_id":1}]}]}`, want: "start_unit_id"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
_, err := codec.Decode([]byte(test.raw))
if err == nil || !strings.Contains(err.Error(), test.want) {
t.Fatalf("Decode() error = %v, want %q", err, test.want)
}
})
}
}
func TestCodecRejectsInvalidCanonicalValues(t *testing.T) {
_, err := New().Encode(dnd.SpellList{})
if err == nil || !strings.Contains(err.Error(), "spell_casts must be present") {
t.Fatalf("Encode() error = %v, want strict shape error", err)
}
}
func TestCodecSchemaIsMutationSafe(t *testing.T) {
first := New().Schema()
first.JSONSchema[0] = '['
second := New().Schema()
if !json.Valid(second.JSONSchema) || second.JSONSchema[0] == '[' {
t.Fatalf("Schema() returned shared bytes: %s", second.JSONSchema)
}
}

View File

@@ -0,0 +1,30 @@
{
"spell_casts": [
{
"caster": "Aria",
"spell": "Cure Wounds",
"effect": "Heals an injured ally.",
"narrative_description": "Aria restores the fighter after the fight.",
"source_refs": [
{
"source_id": "session-alpha",
"start_unit_id": 1,
"end_unit_id": 2
}
]
},
{
"caster": "Borin",
"spell": "Fire Bolt",
"effect": "Scorches the wight.",
"narrative_description": "Borin hurls fire at the wight.",
"source_refs": [
{
"source_id": "session-alpha",
"start_unit_id": 3,
"end_unit_id": 3
}
]
}
]
}

View File

@@ -2,5 +2,5 @@ package spells
import "embed"
//go:embed assets/schemas/*.json assets/prompts/*.yaml assets/prompts/*.md
//go:embed assets/schemas/dnd_spells_llm.v1.json assets/prompts/*.yaml assets/prompts/*.md
var embeddedAssets embed.FS

View File

@@ -3,15 +3,17 @@ package spells
import (
"sort"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
func canonicalizeResponse(response *extractionResponse, sourceID string) {
func canonicalizeResponse(response *extractionResponse) {
if response == nil {
return
}
for index := range response.SpellCasts {
canonicalizeSpellCast(&response.SpellCasts[index], sourceID)
canonicalizeSpellCast(&response.SpellCasts[index])
}
sort.SliceStable(response.SpellCasts, func(i, j int) bool {
left, leftOK := earliestSourceUnit(response.SpellCasts[i])
@@ -26,9 +28,8 @@ func canonicalizeResponse(response *extractionResponse, sourceID string) {
})
}
func canonicalizeSpellCast(spell *spellCastResponse, sourceID string) {
func canonicalizeSpellCast(spell *spellCastResponse) {
for index := range spell.SourceRefs {
spell.SourceRefs[index].SourceID = sourceID
spell.SourceRefs[index].StartUnitID = canonicalUnitRef(spell.SourceRefs[index].StartUnitID)
spell.SourceRefs[index].EndUnitID = canonicalUnitRef(spell.SourceRefs[index].EndUnitID)
}
@@ -51,12 +52,12 @@ func canonicalUnitRef(ref shared.UnitRef) shared.UnitRef {
return shared.UnitRefFromInt(value)
}
func dedupeSourceRefs(refs []shared.SourceRefResponse) []shared.SourceRefResponse {
func dedupeSourceRefs(refs []spellSourceRefResponse) []spellSourceRefResponse {
if len(refs) < 2 {
return refs
}
out := refs[:0]
var previous shared.SourceRefResponse
var previous spellSourceRefResponse
for index, ref := range refs {
if index > 0 && sameSourceRef(previous, ref) {
continue
@@ -67,9 +68,8 @@ func dedupeSourceRefs(refs []shared.SourceRefResponse) []shared.SourceRefRespons
return out
}
func sameSourceRef(left shared.SourceRefResponse, right shared.SourceRefResponse) bool {
return left.SourceID == right.SourceID &&
left.StartUnitID.Int() == right.StartUnitID.Int() &&
func sameSourceRef(left spellSourceRefResponse, right spellSourceRefResponse) bool {
return left.StartUnitID.Int() == right.StartUnitID.Int() &&
left.EndUnitID.Int() == right.EndUnitID.Int()
}
@@ -90,3 +90,28 @@ func unitSortValue(ref shared.UnitRef) int {
}
return value
}
func canonicalSpellList(response extractionResponse, sourceID string) dnd.SpellList {
spellCasts := make([]dnd.SpellCast, len(response.SpellCasts))
for index, spell := range response.SpellCasts {
refs := make([]source.SourceRef, len(spell.SourceRefs))
for refIndex, ref := range spell.SourceRefs {
refs[refIndex] = source.SourceRef{
SourceID: sourceID,
StartUnitID: ref.StartUnitID.Int(),
EndUnitID: ref.EndUnitID.Int(),
}
}
spellCasts[index] = dnd.SpellCast{
Caster: spell.Caster,
Spell: spell.Spell,
Effect: spell.Effect,
NarrativeDescription: spell.NarrativeDescription,
SourceRefs: refs,
}
}
if response.SpellCasts == nil {
spellCasts = nil
}
return dnd.SpellList{SpellCasts: spellCasts}
}

View File

@@ -3,11 +3,11 @@ package spells
import (
"bytes"
"context"
"encoding/json"
"fmt"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
@@ -31,12 +31,31 @@ var referenceSlotDescriptions = shared.ReferenceSlotDescriptions{
Roster: "Deprecated alias for party roster reference material used only for disambiguation.",
}
var _ contracts.LegacyRawExtractor = (*Extractor)(nil)
var _ contracts.Extractor[dnd.SpellList] = (*Extractor)(nil)
type Extractor struct{}
type Options struct{}
func New() *Extractor {
return &Extractor{}
type Extractor struct {
llm contracts.StructuredLLMClient
}
type rawAdapter struct {
extractor *Extractor
codec RawAdapterCodec
}
type RawAdapterCodec interface {
contracts.ArtifactCodec[dnd.SpellList]
EncodeCandidate(dnd.SpellList) ([]byte, error)
}
var _ contracts.LegacyRawExtractor = (*rawAdapter)(nil)
func New(llmClient contracts.StructuredLLMClient, _ Options) (*Extractor, error) {
if llmClient == nil {
return nil, extractorErrorf("LLM client must not be nil")
}
return &Extractor{llm: llmClient}, nil
}
func (e *Extractor) Key() string {
@@ -67,35 +86,35 @@ func (e *Extractor) ManifestMetadata() map[string]any {
return metadata
}
func (e *Extractor) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
func (e *Extractor) Extract(ctx context.Context, req contracts.TypedExtractionRequest) (contracts.TypedExtractionResult[dnd.SpellList], error) {
if e == nil {
return contracts.ExtractionResult{}, extractorErrorf("extractor must not be nil")
return contracts.TypedExtractionResult[dnd.SpellList]{}, extractorErrorf("extractor must not be nil")
}
if e.llm == nil {
return contracts.TypedExtractionResult[dnd.SpellList]{}, extractorErrorf("LLM client must not be nil")
}
if ctx == nil {
return contracts.ExtractionResult{}, extractorErrorf("context must not be nil")
return contracts.TypedExtractionResult[dnd.SpellList]{}, extractorErrorf("context must not be nil")
}
if err := ctx.Err(); err != nil {
return contracts.ExtractionResult{}, extractorErrorf("context error before extraction: %w", err)
return contracts.TypedExtractionResult[dnd.SpellList]{}, extractorErrorf("context error before extraction: %w", err)
}
if req.Source == nil {
return contracts.ExtractionResult{}, extractorErrorf("source must not be nil")
return contracts.TypedExtractionResult[dnd.SpellList]{}, extractorErrorf("source must not be nil")
}
if req.Chunk == nil {
return contracts.ExtractionResult{}, extractorErrorf("chunk must not be nil")
return contracts.TypedExtractionResult[dnd.SpellList]{}, extractorErrorf("chunk must not be nil")
}
if len(req.Chunk.Units) == 0 {
return contracts.ExtractionResult{}, extractorErrorf("chunk %q units must not be empty", req.Chunk.ID)
}
if req.LLMClient == nil {
return contracts.ExtractionResult{}, extractorErrorf("LLM client must not be nil")
return contracts.TypedExtractionResult[dnd.SpellList]{}, extractorErrorf("chunk %q units must not be empty", req.Chunk.ID)
}
sourceInput, err := chunkSourceInput(req)
if err != nil {
return contracts.ExtractionResult{}, err
return contracts.TypedExtractionResult[dnd.SpellList]{}, err
}
var response extractionResponse
if _, err := req.LLMClient.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
if _, err := e.llm.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
StageName: Key,
PromptID: PromptID,
PromptVersion: SchemaVersion,
@@ -103,37 +122,13 @@ func (e *Extractor) Extract(ctx context.Context, req contracts.ExtractionRequest
SessionID: req.SessionID,
Inputs: shared.PromptInputs(sourceInput, req.References),
}, &response); err != nil {
return contracts.ExtractionResult{}, extractorErrorf("complete structured output: %w", err)
return contracts.TypedExtractionResult[dnd.SpellList]{}, extractorErrorf("complete structured output: %w", err)
}
canonicalizeResponse(&response, req.Source.ID)
content, err := json.Marshal(response)
if err != nil {
return contracts.ExtractionResult{}, extractorErrorf("marshal canonical output: %w", err)
}
schema, err := loadResponseSchema()
if err != nil {
return contracts.ExtractionResult{}, extractorErrorf("load response schema: %w", err)
}
return contracts.ExtractionResult{
Output: contracts.ExtractOutput{
Schema: contracts.ResponseSchema{
ID: ResponseSchemaID,
Name: ResponseSchemaName,
Version: SchemaVersion,
JSONSchema: append([]byte(nil), schema.JSONSchema...),
},
Payload: contracts.RawPayload{
Content: content,
MediaType: "application/json",
Metadata: map[string]any{
"spell_cast_count": len(response.SpellCasts),
},
},
},
}, nil
canonicalizeResponse(&response)
return contracts.TypedExtractionResult[dnd.SpellList]{Value: canonicalSpellList(response, req.Source.ID)}, nil
}
func chunkSourceInput(req contracts.ExtractionRequest) (contracts.LLMInputMaterial, error) {
func chunkSourceInput(req contracts.TypedExtractionRequest) (contracts.LLMInputMaterial, error) {
material := req.SourceInput.Clone()
if len(material.Content) == 0 {
material = contracts.NewLLMInputMaterial("source", req.Chunk.MediaType, req.Chunk.Content, "", "")
@@ -159,16 +154,93 @@ func ModuleSpec() pipeline.ModuleSpec {
Stage: pipeline.StageExtract,
Requires: append([]string(nil), requiredCapabilities...),
Provides: append([]string(nil), providedCapabilities...),
ArtifactKind: dnd.SpellListKind,
ReferenceSlots: shared.ReferenceSlots(referenceSlotDescriptions),
}
}
func Register(registry *pipeline.ExtractorRegistry) error {
return registry.RegisterLegacyRawWithSpec(ModuleSpec(), func() (contracts.LegacyRawExtractor, error) {
return New(), nil
return pipeline.RegisterExtractorBuilder(registry, ModuleSpec(), validateOptions, func(request pipeline.BuildRequest) (contracts.Extractor[dnd.SpellList], error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return New(request.Dependencies.LLM, options)
})
}
// RegisterWithRawAdapter keeps existing raw downstream implementations usable
// while the extractor itself produces the canonical typed artifact.
func RegisterWithRawAdapter(registry *pipeline.ExtractorRegistry, codec RawAdapterCodec) error {
if codec == nil {
return extractorErrorf("artifact codec must not be nil")
}
build := func(request pipeline.BuildRequest) (*Extractor, error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return New(request.Dependencies.LLM, options)
}
return pipeline.RegisterExtractorBuilderWithRawAdapter(registry, ModuleSpec(), validateOptions,
func(request pipeline.BuildRequest) (contracts.Extractor[dnd.SpellList], error) {
return build(request)
},
func(request pipeline.BuildRequest) (contracts.LegacyRawExtractor, error) {
extractor, err := build(request)
if err != nil {
return nil, err
}
return &rawAdapter{extractor: extractor, codec: codec}, nil
},
)
}
func (adapter *rawAdapter) Key() string { return Key }
func (adapter *rawAdapter) ReferenceSlots() []contracts.ReferenceSlot {
return adapter.extractor.ReferenceSlots()
}
func (adapter *rawAdapter) ManifestMetadata() map[string]any {
return adapter.extractor.ManifestMetadata()
}
func (adapter *rawAdapter) Extract(ctx context.Context, req contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
result, err := adapter.extractor.Extract(ctx, contracts.TypedExtractionRequest{
Source: req.Source, Chunk: req.Chunk, AmbientContext: req.AmbientContext,
SourceInput: req.SourceInput, SessionID: req.SessionID, References: req.References,
LLMProfile: req.LLMProfile, Metadata: req.Metadata,
})
if err != nil {
return contracts.ExtractionResult{}, err
}
content, err := adapter.codec.EncodeCandidate(result.Value)
if err != nil {
return contracts.ExtractionResult{}, extractorErrorf("encode canonical output: %w", err)
}
schema := adapter.codec.Schema()
return contracts.ExtractionResult{
Output: contracts.ExtractOutput{
Schema: contracts.ResponseSchema{ID: schema.ID, Name: schema.Name, Version: schema.Version, JSONSchema: append([]byte(nil), schema.JSONSchema...)},
Payload: contracts.RawPayload{Content: content, MediaType: adapter.codec.MediaType(), Metadata: map[string]any{"spell_cast_count": len(result.Value.SpellCasts)}},
},
Warnings: result.Warnings,
}, nil
}
func validateOptions(options map[string]any) error {
_, err := DecodeOptions(options)
return err
}
func DecodeOptions(options map[string]any) (Options, error) {
if err := pipeline.RejectUnknownOptions(options); err != nil {
return Options{}, extractorErrorf("%w", err)
}
return Options{}, nil
}
func extractorErrorf(format string, args ...any) error {
return fmt.Errorf("dnd spells extractor: "+format, args...)
}

View File

@@ -2,47 +2,55 @@ package spells
import (
"context"
"encoding/json"
"errors"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
func TestExtractReturnsCanonicalOutputFromStructuredResponse(t *testing.T) {
client := &fakeSpellsLLMClient{
response: extractionResponse{
SpellCasts: []spellCastResponse{
{
Caster: " Aria ",
Spell: " Cure Wounds ",
Effect: " Heals an injured ally. ",
NarrativeDescription: " Aria restores the fighter after the fight. ",
SourceRefs: responseSourceRefsInt("transcript", 1, 2),
},
},
func TestExtractReturnsCanonicalSpellListFromPrivateResponse(t *testing.T) {
client := &fakeSpellsLLMClient{response: extractionResponse{SpellCasts: []spellCastResponse{
{
Caster: " Aria ",
Spell: " Cure Wounds ",
Effect: " Heals an injured ally. ",
NarrativeDescription: " Aria restores the fighter after the fight. ",
SourceRefs: responseSourceRefs(1, 2),
},
content: []byte(`{"spell_casts":[{"caster":" Aria ","spell":" Cure Wounds ","effect":" Heals an injured ally. ","narrative_description":" Aria restores the fighter after the fight. ","source_refs":[{"source_id":"session-alpha","start_unit_id":1,"end_unit_id":2}]}],"raw_marker":true}`),
}
}}}
req := extractionRequest()
extractReq := extractionRequestWithClient(client)
result, err := New().Extract(context.Background(), extractReq)
result, err := newExtractor(t, client).Extract(context.Background(), req)
if err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
want := dnd.SpellList{SpellCasts: []dnd.SpellCast{
{
Caster: " Aria ",
Spell: " Cure Wounds ",
Effect: " Heals an injured ally. ",
NarrativeDescription: " Aria restores the fighter after the fight. ",
SourceRefs: []source.SourceRef{{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 2}},
},
}}
if !reflect.DeepEqual(result.Value, want) {
t.Fatalf("Value = %#v, want %#v", result.Value, want)
}
if len(result.Warnings) != 0 {
t.Fatalf("Warnings = %#v, want none", result.Warnings)
}
if len(client.requests) != 1 {
t.Fatalf("LLM calls = %d, want 1", len(client.requests))
}
llmReq := client.requests[0]
if llmReq.StageName != Key {
t.Fatalf("StageName = %q, want %q", llmReq.StageName, Key)
}
if llmReq.PromptID != PromptID || llmReq.PromptVersion != SchemaVersion {
t.Fatalf("prompt = %q/%q, want %q/%q", llmReq.PromptID, llmReq.PromptVersion, PromptID, SchemaVersion)
if llmReq.StageName != Key || llmReq.PromptID != PromptID || llmReq.PromptVersion != SchemaVersion {
t.Fatalf("LLM request identity = %#v, want spell prompt", llmReq)
}
if llmReq.SessionID != "session-123" || llmReq.ProfileID != "profile-spells" {
t.Fatalf("session/profile = %q/%q, want session-123/profile-spells", llmReq.SessionID, llmReq.ProfileID)
@@ -51,45 +59,17 @@ func TestExtractReturnsCanonicalOutputFromStructuredResponse(t *testing.T) {
if transcript.Name != "transcript" || transcript.MediaType != "application/json" || transcript.Digest != "sha256:chunk" || transcript.OriginURI != "file:///session-alpha.json" {
t.Fatalf("transcript metadata = %#v", transcript)
}
if got := string(transcript.Content); got != string(extractReq.Chunk.Content) {
t.Fatalf("transcript content = %q, want chunk content %q", got, extractReq.Chunk.Content)
}
if result.Output.Payload.MediaType != "application/json" {
t.Fatalf("MediaType = %q, want application/json", result.Output.Payload.MediaType)
}
if result.Output.Schema.ID != ResponseSchemaID || result.Output.Schema.Name != ResponseSchemaName || result.Output.Schema.Version != SchemaVersion {
t.Fatalf("schema = %#v, want response schema provenance", result.Output.Schema)
}
if !json.Valid(result.Output.Schema.JSONSchema) {
t.Fatalf("schema JSON is invalid or missing: %s", result.Output.Schema.JSONSchema)
}
if strings.Contains(string(result.Output.Payload.Content), "raw_marker") {
t.Fatalf("content = %q, want canonical payload without raw completion marker", result.Output.Payload.Content)
}
var payload extractionResponse
if err := json.Unmarshal(result.Output.Payload.Content, &payload); err != nil {
t.Fatalf("Unmarshal(Content) error = %v, want nil", err)
}
if len(payload.SpellCasts) != 1 || payload.SpellCasts[0].Spell != " Cure Wounds " {
t.Fatalf("payload = %#v, want structured response fields", payload)
}
if got := payload.SpellCasts[0].SourceRefs[0].SourceID; got != "session-alpha" {
t.Fatalf("source_id = %q, want canonical source document ID", got)
if got := string(transcript.Content); got != string(req.Chunk.Content) {
t.Fatalf("transcript content = %q, want chunk content %q", got, req.Chunk.Content)
}
}
func TestExtractorManifestMetadataIncludesPromptAndSchemaProvenance(t *testing.T) {
metadata := New().ManifestMetadata()
func TestExtractorManifestMetadataIncludesLLMSchemaProvenance(t *testing.T) {
metadata := newExtractor(t, &fakeSpellsLLMClient{}).ManifestMetadata()
tests := map[string]string{
"prompt_id": PromptID,
"prompt_version": SchemaVersion,
"response_schema_key": string(ResponseSchemaKey),
"response_schema_id": ResponseSchemaID,
"response_schema_name": ResponseSchemaName,
"response_schema_version": SchemaVersion,
"prompt_id": PromptID, "prompt_version": SchemaVersion,
"response_schema_key": string(ResponseSchemaKey), "response_schema_id": ResponseSchemaID,
"response_schema_name": ResponseSchemaName, "response_schema_version": SchemaVersion,
}
for key, want := range tests {
if metadata[key] != want {
@@ -106,373 +86,125 @@ func TestExtractorManifestMetadataIncludesPromptAndSchemaProvenance(t *testing.T
func TestExtractPassesReferencesAsPromptInputs(t *testing.T) {
client := &fakeSpellsLLMClient{response: extractionResponse{SpellCasts: []spellCastResponse{}}}
req := extractionRequestWithClient(client)
req.References = contracts.ReferenceSet{
Slots: map[string]contracts.ResolvedReferenceSlot{
"players": {
Slot: contracts.ReferenceSlot{Name: "players"},
Items: []contracts.ReferenceItem{
{SlotName: "players", Content: []byte("Alice: Aria Brightmantle")},
},
},
"party": {
Slot: contracts.ReferenceSlot{Name: "party"},
Items: []contracts.ReferenceItem{
{SlotName: "party", Content: []byte("Aria Brightmantle: party cleric")},
},
},
"glossary": {
Slot: contracts.ReferenceSlot{Name: "glossary"},
Items: []contracts.ReferenceItem{
{SlotName: "glossary", Content: []byte("Brightmantle: local temple name")},
},
},
},
}
req := extractionRequest()
req.References = contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
"players": {Slot: contracts.ReferenceSlot{Name: "players"}, Items: []contracts.ReferenceItem{{SlotName: "players", Content: []byte("Alice: Aria Brightmantle")}}},
"party": {Slot: contracts.ReferenceSlot{Name: "party"}, Items: []contracts.ReferenceItem{{SlotName: "party", Content: []byte("Aria Brightmantle: party cleric")}}},
"glossary": {Slot: contracts.ReferenceSlot{Name: "glossary"}, Items: []contracts.ReferenceItem{{SlotName: "glossary", Content: []byte("Brightmantle: local temple name")}}},
}}
if _, err := New().Extract(context.Background(), req); err != nil {
if _, err := newExtractor(t, client).Extract(context.Background(), req); err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
if len(client.requests) != 1 {
t.Fatalf("LLM calls = %d, want 1", len(client.requests))
inputs := client.requests[0].Inputs
if string(inputs["players"].Content) != "Alice: Aria Brightmantle" || string(inputs["party"].Content) != "Aria Brightmantle: party cleric" || string(inputs["glossary"].Content) != "Brightmantle: local temple name" {
t.Fatalf("reference inputs = %#v, want configured content", inputs)
}
request := client.requests[0]
if request.PromptID != PromptID || request.PromptVersion != SchemaVersion {
t.Fatalf("prompt = %q/%q, want %q/%q", request.PromptID, request.PromptVersion, PromptID, SchemaVersion)
}
if got := string(request.Inputs["players"].Content); got != "Alice: Aria Brightmantle" {
t.Fatalf("players input = %q, want reference content", got)
}
if got := string(request.Inputs["party"].Content); got != "Aria Brightmantle: party cleric" {
t.Fatalf("party input = %q, want reference content", got)
}
if got := string(request.Inputs["glossary"].Content); got != "Brightmantle: local temple name" {
t.Fatalf("glossary input = %q, want reference content", got)
}
if strings.Contains(string(request.Inputs["transcript"].Content), "Aria Brightmantle: party cleric") {
t.Fatalf("transcript input contains reference content")
if strings.Contains(string(inputs["transcript"].Content), "party cleric") {
t.Fatal("transcript input contains reference content")
}
}
func TestPromptInputsMapLegacyRosterReferenceToParty(t *testing.T) {
inputs := shared.PromptInputs(spellSourceInput(), contracts.ReferenceSet{
Slots: map[string]contracts.ResolvedReferenceSlot{
"roster": {
Slot: contracts.ReferenceSlot{Name: "roster"},
Items: []contracts.ReferenceItem{
{SlotName: "roster", Content: []byte("Legacy roster text")},
},
},
},
})
inputs := shared.PromptInputs(spellSourceInput(), contracts.ReferenceSet{Slots: map[string]contracts.ResolvedReferenceSlot{
"roster": {Slot: contracts.ReferenceSlot{Name: "roster"}, Items: []contracts.ReferenceItem{{SlotName: "roster", Content: []byte("Legacy roster text")}}},
}})
if got := string(inputs["party"].Content); got != "Legacy roster text" {
t.Fatalf("party input = %q, want legacy roster content", got)
}
if _, ok := inputs["roster"]; ok {
t.Fatalf("roster prompt input was present; want only party input")
t.Fatal("roster prompt input was present; want only party input")
}
}
func TestExtractReturnsRawOutputForEmptyResponse(t *testing.T) {
client := &fakeSpellsLLMClient{response: extractionResponse{SpellCasts: []spellCastResponse{}}}
result, err := New().Extract(context.Background(), extractionRequestWithClient(client))
if err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
var payload extractionResponse
if err := json.Unmarshal(result.Output.Payload.Content, &payload); err != nil {
t.Fatalf("Unmarshal(Content) error = %v, want nil", err)
}
if len(payload.SpellCasts) != 0 {
t.Fatalf("SpellCasts = %#v, want none", payload.SpellCasts)
}
}
func TestExtractReturnsCanonicalOutputForMalformedStructuredResponse(t *testing.T) {
client := &fakeSpellsLLMClient{response: extractionResponse{}}
result, err := New().Extract(context.Background(), extractionRequestWithClient(client))
if err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
if string(result.Output.Payload.Content) != `{"spell_casts":null}` {
t.Fatalf("content = %s, want canonical structured output", result.Output.Payload.Content)
}
}
func TestExtractWrapsLLMClientError(t *testing.T) {
client := &fakeSpellsLLMClient{err: errors.New("provider unavailable")}
_, err := New().Extract(context.Background(), extractionRequestWithClient(client))
if err == nil {
t.Fatal("Extract() error = nil, want LLM error")
}
if !strings.Contains(err.Error(), "dnd spells") || !strings.Contains(err.Error(), "provider unavailable") {
t.Fatalf("Extract() error = %q, want wrapped LLM context", err.Error())
}
}
func TestExtractRejectsInvalidRequests(t *testing.T) {
validClient := &fakeSpellsLLMClient{response: extractionResponse{SpellCasts: []spellCastResponse{}}}
validReq := extractionRequestWithClient(validClient)
canceledCtx, cancel := context.WithCancel(context.Background())
cancel()
func TestExtractPreservesEmptyAndMalformedValuesForTypedValidators(t *testing.T) {
tests := []struct {
name string
extractor *Extractor
ctx context.Context
req contracts.ExtractionRequest
want string
name string
response extractionResponse
wantNil bool
}{
{name: "nil extractor", extractor: nil, ctx: context.Background(), req: validReq, want: "extractor"},
{name: "nil context", extractor: New(), ctx: nil, req: validReq, want: "context"},
{name: "canceled context", extractor: New(), ctx: canceledCtx, req: validReq, want: "context"},
{name: "nil source", extractor: New(), ctx: context.Background(), req: contracts.ExtractionRequest{Chunk: validReq.Chunk, LLMClient: validReq.LLMClient}, want: "source"},
{name: "nil chunk", extractor: New(), ctx: context.Background(), req: contracts.ExtractionRequest{Source: validReq.Source, LLMClient: validReq.LLMClient}, want: "chunk"},
{name: "empty chunk units", extractor: New(), ctx: context.Background(), req: emptyChunkRequest(validReq), want: "units"},
{name: "nil LLM client", extractor: New(), ctx: context.Background(), req: contracts.ExtractionRequest{Source: validReq.Source, Chunk: validReq.Chunk}, want: "LLM client"},
{name: "source input mismatches chunk", extractor: New(), ctx: context.Background(), req: mismatchedSourceInputRequest(validReq), want: "must match chunk"},
{name: "empty", response: extractionResponse{SpellCasts: []spellCastResponse{}}},
{name: "missing", response: extractionResponse{}, wantNil: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := tt.extractor.Extract(tt.ctx, tt.req)
if err == nil {
t.Fatal("Extract() error = nil, want error")
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
result, err := newExtractor(t, &fakeSpellsLLMClient{response: test.response}).Extract(context.Background(), extractionRequest())
if err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
if !strings.Contains(err.Error(), "dnd spells") || !strings.Contains(err.Error(), tt.want) {
t.Fatalf("Extract() error = %q, want %q context", err.Error(), tt.want)
if (result.Value.SpellCasts == nil) != test.wantNil || len(result.Value.SpellCasts) != 0 {
t.Fatalf("SpellCasts = %#v, want empty with nil=%t", result.Value.SpellCasts, test.wantNil)
}
})
}
}
func TestExtractOrdersSpellCastsByEarliestSourceUnit(t *testing.T) {
client := &fakeSpellsLLMClient{
response: extractionResponse{
SpellCasts: []spellCastResponse{
{
Caster: "Bandit Shaman",
Spell: "Fire Bolt",
Effect: "Burns.",
NarrativeDescription: "Second spell.",
SourceRefs: responseSourceRefs("session-alpha", 2, 2),
},
{
Caster: "Aria",
Spell: "Cure Wounds",
Effect: "Heals.",
NarrativeDescription: "First spell.",
SourceRefs: responseSourceRefs("session-alpha", 1, 1),
},
{
Caster: "Narrator",
Spell: "Unknown Spell",
Effect: "No cited range.",
NarrativeDescription: "This should sort after cited spell casts.",
},
},
},
func TestExtractWrapsLLMClientError(t *testing.T) {
_, err := newExtractor(t, &fakeSpellsLLMClient{err: errors.New("provider unavailable")}).Extract(context.Background(), extractionRequest())
if err == nil || !strings.Contains(err.Error(), "dnd spells") || !strings.Contains(err.Error(), "provider unavailable") {
t.Fatalf("Extract() error = %v, want wrapped provider error", err)
}
}
result, err := New().Extract(context.Background(), extractionRequestWithClient(client))
func TestExtractRejectsInvalidRequests(t *testing.T) {
validReq := extractionRequest()
canceledCtx, cancel := context.WithCancel(context.Background())
cancel()
validExtractor := newExtractor(t, &fakeSpellsLLMClient{response: extractionResponse{SpellCasts: []spellCastResponse{}}})
tests := []struct {
name string
extractor *Extractor
ctx context.Context
req contracts.TypedExtractionRequest
want string
}{
{name: "nil extractor", ctx: context.Background(), req: validReq, want: "extractor"},
{name: "nil context", extractor: validExtractor, req: validReq, want: "context"},
{name: "canceled context", extractor: validExtractor, ctx: canceledCtx, req: validReq, want: "context"},
{name: "nil source", extractor: validExtractor, ctx: context.Background(), req: contracts.TypedExtractionRequest{Chunk: validReq.Chunk}, want: "source"},
{name: "nil chunk", extractor: validExtractor, ctx: context.Background(), req: contracts.TypedExtractionRequest{Source: validReq.Source}, want: "chunk"},
{name: "empty chunk units", extractor: validExtractor, ctx: context.Background(), req: emptyChunkRequest(validReq), want: "units"},
{name: "source input mismatches chunk", extractor: validExtractor, ctx: context.Background(), req: mismatchedSourceInputRequest(validReq), want: "must match chunk"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
_, err := test.extractor.Extract(test.ctx, test.req)
if err == nil || !strings.Contains(err.Error(), "dnd spells") || !strings.Contains(err.Error(), test.want) {
t.Fatalf("Extract() error = %v, want %q context", err, test.want)
}
})
}
}
func TestExtractOrdersAndDeduplicatesEvidence(t *testing.T) {
client := &fakeSpellsLLMClient{response: extractionResponse{SpellCasts: []spellCastResponse{
{Caster: "Borin", Spell: "Fire Bolt", Effect: "Burns.", NarrativeDescription: "Second.", SourceRefs: responseSourceRefs(2, 2)},
{Caster: "Aria", Spell: "Cure Wounds", Effect: "Heals.", NarrativeDescription: "First.", SourceRefs: []spellSourceRefResponse{{StartUnitID: shared.UnitRefFromInt(1), EndUnitID: shared.UnitRefFromInt(2)}, {StartUnitID: shared.UnitRefFromInt(1), EndUnitID: shared.UnitRefFromInt(2)}}},
{Caster: "Narrator", Spell: "Unknown", Effect: "Unknown.", NarrativeDescription: "Uncited."},
}}}
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
if err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
var payload extractionResponse
if err := json.Unmarshal(result.Output.Payload.Content, &payload); err != nil {
t.Fatalf("Unmarshal(Content) error = %v, want nil", err)
if got := []string{result.Value.SpellCasts[0].Spell, result.Value.SpellCasts[1].Spell, result.Value.SpellCasts[2].Spell}; !reflect.DeepEqual(got, []string{"Cure Wounds", "Fire Bolt", "Unknown"}) {
t.Fatalf("spell order = %#v, want evidence order", got)
}
if len(payload.SpellCasts) != 3 ||
payload.SpellCasts[0].Spell != "Cure Wounds" ||
payload.SpellCasts[1].Spell != "Fire Bolt" ||
payload.SpellCasts[2].Spell != "Unknown Spell" {
t.Fatalf("spell order = %#v, want earliest source-unit order with uncited spell last", payload.SpellCasts)
if refs := result.Value.SpellCasts[0].SourceRefs; len(refs) != 1 || refs[0] != (source.SourceRef{SourceID: "session-alpha", StartUnitID: 1, EndUnitID: 2}) {
t.Fatalf("source refs = %#v, want one canonical ref", refs)
}
}
func TestExtractCanonicalizesSourceRefs(t *testing.T) {
client := &fakeSpellsLLMClient{
response: extractionResponse{
SpellCasts: []spellCastResponse{
{
Caster: "Aria",
Spell: "Cure Wounds",
Effect: "Heals.",
NarrativeDescription: "Aria heals.",
SourceRefs: []shared.SourceRefResponse{
{SourceID: "gameplay_transcript", StartUnitID: shared.UnitRefFromInt(2), EndUnitID: shared.UnitRefFromInt(2)},
{SourceID: "", StartUnitID: shared.UnitRefFromInt(1), EndUnitID: shared.UnitRefFromInt(2)},
{SourceID: "transcript", StartUnitID: shared.UnitRefFromInt(1), EndUnitID: shared.UnitRefFromInt(2)},
},
},
},
},
}
result, err := New().Extract(context.Background(), extractionRequestWithClient(client))
func TestExtractPreservesInvalidEvidenceForValidators(t *testing.T) {
client := &fakeSpellsLLMClient{response: extractionResponse{SpellCasts: []spellCastResponse{{
Caster: "Aria", Spell: "Cure Wounds", Effect: "Heals.", NarrativeDescription: "Aria heals.",
SourceRefs: []spellSourceRefResponse{{StartUnitID: shared.UnitRefFromInt(99), EndUnitID: shared.UnitRefFromString("missing")}},
}}}}
result, err := newExtractor(t, client).Extract(context.Background(), extractionRequest())
if err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
var payload extractionResponse
if err := json.Unmarshal(result.Output.Payload.Content, &payload); err != nil {
t.Fatalf("Unmarshal(Content) error = %v, want nil", err)
}
refs := payload.SpellCasts[0].SourceRefs
if len(refs) != 2 {
t.Fatalf("source refs = %#v, want duplicate collapsed", refs)
}
for _, ref := range refs {
if ref.SourceID != "session-alpha" {
t.Fatalf("source ref = %#v, want canonical source_id", ref)
}
}
if refs[0].StartUnitID.Int() != 1 || refs[0].EndUnitID.Int() != 2 ||
refs[1].StartUnitID.Int() != 2 || refs[1].EndUnitID.Int() != 2 {
t.Fatalf("source refs = %#v, want sorted unit ranges", refs)
ref := result.Value.SpellCasts[0].SourceRefs[0]
if ref != (source.SourceRef{SourceID: "session-alpha", StartUnitID: 99}) {
t.Fatalf("source ref = %#v, want canonical source with invalid range preserved", ref)
}
}
func TestExtractPreservesInvalidSourceRefsForValidators(t *testing.T) {
client := &fakeSpellsLLMClient{
response: extractionResponse{
SpellCasts: []spellCastResponse{
{
Caster: "Aria",
Spell: "Cure Wounds",
Effect: "Heals.",
NarrativeDescription: "Aria heals.",
SourceRefs: []shared.SourceRefResponse{
{SourceID: "transcript", StartUnitID: shared.UnitRefFromInt(99), EndUnitID: shared.UnitRefFromString("missing")},
},
},
},
},
}
result, err := New().Extract(context.Background(), extractionRequestWithClient(client))
if err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
var payload map[string][]map[string]any
if err := json.Unmarshal(result.Output.Payload.Content, &payload); err != nil {
t.Fatalf("Unmarshal(Content) error = %v, want nil", err)
}
ref := payload["spell_casts"][0]["source_refs"].([]any)[0].(map[string]any)
if ref["source_id"] != "session-alpha" || ref["start_unit_id"] != float64(99) || ref["end_unit_id"] != "" {
t.Fatalf("source ref = %#v, want source_id canonicalized without unit repair", ref)
}
}
func TestExtractDefensivelyCopiesRawContent(t *testing.T) {
client := &fakeSpellsLLMClient{
response: extractionResponse{
SpellCasts: []spellCastResponse{
{
Caster: "Aria",
Spell: "Cure Wounds",
Effect: "Heals.",
NarrativeDescription: "Aria heals.",
SourceRefs: responseSourceRefs("session-alpha", 1, 2),
},
},
},
}
result, err := New().Extract(context.Background(), extractionRequestWithClient(client))
if err != nil {
t.Fatalf("Extract() error = %v, want nil", err)
}
client.response.SpellCasts[0].SourceRefs[0].StartUnitID = shared.UnitRefFromInt(99)
var payload extractionResponse
if err := json.Unmarshal(result.Output.Payload.Content, &payload); err != nil {
t.Fatalf("Unmarshal(Content) error = %v, want nil", err)
}
if got := payload.SpellCasts[0].SourceRefs[0].StartUnitID.String(); got != "1" {
t.Fatalf("source ref start = %q, want copied 1", got)
}
}
func extractionRequestWithClient(client contracts.StructuredLLMClient) contracts.ExtractionRequest {
req := promptExtractionRequest()
req.LLMClient = client
req.SourceInput = spellChunkInput(req.Chunk)
req.SessionID = "session-123"
req.LLMProfile = "profile-spells"
return req
}
const spellTranscriptJSON = `{"id":"session-alpha","segments":[{"id":1,"text":"Aria raises her hand and casts Cure Wounds."}]}`
func spellSourceInput() contracts.LLMInputMaterial {
return contracts.NewLLMInputMaterial("source", "application/json", []byte(spellTranscriptJSON), "sha256:transcript", "file:///session-alpha.json")
}
func spellChunkInput(chunk *source.Chunk) contracts.LLMInputMaterial {
return contracts.NewLLMInputMaterial("source", chunk.MediaType, chunk.Content, "sha256:chunk", "file:///session-alpha.json")
}
func emptyChunkRequest(req contracts.ExtractionRequest) contracts.ExtractionRequest {
req.Chunk = &source.Chunk{
ID: req.Chunk.ID,
SourceID: req.Chunk.SourceID,
Index: req.Chunk.Index,
}
return req
}
func mismatchedSourceInputRequest(req contracts.ExtractionRequest) contracts.ExtractionRequest {
req.SourceInput = spellSourceInput()
return req
}
type fakeSpellsLLMClient struct {
response extractionResponse
content []byte
err error
requests []contracts.StructuredCompletionRequest
}
func (client *fakeSpellsLLMClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
client.requests = append(client.requests, cloneStructuredCompletionRequest(req))
if client.err != nil {
return contracts.StructuredCompletionResponse{}, client.err
}
target, ok := out.(*extractionResponse)
if !ok {
return contracts.StructuredCompletionResponse{}, errors.New("unexpected output target")
}
*target = client.response
content := append([]byte(nil), client.content...)
if len(content) == 0 {
var err error
content, err = json.Marshal(client.response)
if err != nil {
return contracts.StructuredCompletionResponse{}, err
}
}
return contracts.StructuredCompletionResponse{Content: content}, nil
}
func cloneStructuredCompletionRequest(req contracts.StructuredCompletionRequest) contracts.StructuredCompletionRequest {
req.Inputs = req.Inputs.Clone()
req.Vars = cloneVars(req.Vars)
return req
}
func cloneVars(in map[string]any) map[string]any {
if len(in) == 0 {
return nil
}
out := make(map[string]any, len(in))
for key, value := range in {
out[key] = value
}
return out
}

View File

@@ -2,21 +2,19 @@ package spells
import "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
type SpellCast struct {
Caster string `json:"caster"`
Spell string `json:"spell"`
Effect string `json:"effect"`
NarrativeDescription string `json:"narrative_description"`
}
type extractionResponse struct {
SpellCasts []spellCastResponse `json:"spell_casts"`
}
type spellCastResponse struct {
Caster string `json:"caster"`
Spell string `json:"spell"`
Effect string `json:"effect"`
NarrativeDescription string `json:"narrative_description"`
SourceRefs []shared.SourceRefResponse `json:"source_refs"`
Caster string `json:"caster"`
Spell string `json:"spell"`
Effect string `json:"effect"`
NarrativeDescription string `json:"narrative_description"`
SourceRefs []spellSourceRefResponse `json:"source_refs"`
}
type spellSourceRefResponse struct {
StartUnitID shared.UnitRef `json:"start_unit_id"`
EndUnitID shared.UnitRef `json:"end_unit_id"`
}

View File

@@ -7,13 +7,14 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
)
func TestNewReturnsExtractorWithMetadata(t *testing.T) {
extractor := New()
if extractor == nil {
t.Fatal("New() = nil, want extractor")
func TestNewRequiresLLMClientAndReturnsExtractor(t *testing.T) {
if _, err := New(nil, Options{}); err == nil || !strings.Contains(err.Error(), "LLM client") {
t.Fatalf("New(nil) error = %v, want LLM client error", err)
}
extractor := newExtractor(t, &fakeSpellsLLMClient{})
if extractor.Key() != Key {
t.Fatalf("extractor.Key() = %q, want %q", extractor.Key(), Key)
}
@@ -31,6 +32,7 @@ func TestModuleSpec(t *testing.T) {
Provides: []string{
"dnd.spell_casts",
},
ArtifactKind: dnd.SpellListKind,
ReferenceSlots: []contracts.ReferenceSlot{
{
Name: "glossary",
@@ -74,12 +76,11 @@ func TestRegisterMakesExtractorBuildable(t *testing.T) {
t.Fatalf("Register() error = %v, want nil", err)
}
extractor, err := registry.BuildLegacyRaw(Key)
if err != nil {
t.Fatalf("Build() error = %v, want nil", err)
if _, err := registry.BuildLegacyRaw(Key); err == nil || !strings.Contains(err.Error(), "legacy raw") {
t.Fatalf("BuildLegacyRaw() error = %v, want typed registration error", err)
}
if extractor.Key() != Key {
t.Fatalf("extractor.Key() = %q, want %q", extractor.Key(), Key)
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil || !strings.Contains(err.Error(), "unknown option") {
t.Fatalf("DecodeOptions() error = %v, want unknown option error", err)
}
}
@@ -101,7 +102,7 @@ func TestRegisterStoresModuleSpec(t *testing.T) {
}
func TestRuntimeReferenceSlotsMatchModuleSpec(t *testing.T) {
extractor := New()
extractor := newExtractor(t, &fakeSpellsLLMClient{})
spec := ModuleSpec()
if !reflect.DeepEqual(extractor.ReferenceSlots(), spec.ReferenceSlots) {

View File

@@ -15,6 +15,6 @@ func loadResponseSchema() (llm.ResponseSchema, error) {
ID: ResponseSchemaID,
Version: SchemaVersion,
Name: ResponseSchemaName,
AssetPath: "assets/schemas/dnd_spells.v1.json",
AssetPath: "assets/schemas/dnd_spells_llm.v1.json",
})
}

View File

@@ -1,130 +1,62 @@
package spells
import (
"bytes"
"encoding/json"
"strings"
"testing"
)
func TestLoadResponseSchemaForSpells(t *testing.T) {
func TestLoadResponseSchemaUsesExtractorOwnedLLMSchema(t *testing.T) {
schema, err := loadResponseSchema()
if err != nil {
t.Fatalf("loadResponseSchema() error = %v, want nil", err)
}
if schema.Key != ResponseSchemaKey {
t.Fatalf("schema.Key = %q, want %q", schema.Key, ResponseSchemaKey)
if schema.Key != ResponseSchemaKey || schema.ID != ResponseSchemaID || schema.Version != SchemaVersion || schema.Name != ResponseSchemaName {
t.Fatalf("schema identity = %#v, want maintained response identity", schema)
}
if schema.ID != ResponseSchemaID {
t.Fatalf("schema.ID = %q, want %q", schema.ID, ResponseSchemaID)
}
if schema.Version != SchemaVersion {
t.Fatalf("schema.Version = %q, want %q", schema.Version, SchemaVersion)
}
if schema.Name != ResponseSchemaName {
t.Fatalf("schema.Name = %q, want %q", schema.Name, ResponseSchemaName)
}
if !strings.HasPrefix(schema.SHA256, "sha256:") {
t.Fatalf("schema.SHA256 = %q, want sha256 prefix", schema.SHA256)
}
if !json.Valid(schema.JSONSchema) {
t.Fatalf("schema.JSONSchema is invalid JSON: %s", schema.JSONSchema)
if !strings.HasPrefix(schema.SHA256, "sha256:") || !json.Valid(schema.JSONSchema) {
t.Fatalf("schema metadata = %#v, want valid hashed JSON", schema)
}
var decoded map[string]any
if err := json.Unmarshal(schema.JSONSchema, &decoded); err != nil {
t.Fatalf("Unmarshal(schema.JSONSchema) error = %v, want nil", err)
t.Fatalf("Unmarshal(schema.JSONSchema) error = %v", err)
}
properties := decoded["properties"].(map[string]any)
spellCastProperties := properties["spell_casts"].(map[string]any)["items"].(map[string]any)["properties"].(map[string]any)
sourceRefProperties := spellCastProperties["source_refs"].(map[string]any)["items"].(map[string]any)["properties"].(map[string]any)
sourceRefRequired := spellCastProperties["source_refs"].(map[string]any)["items"].(map[string]any)["required"].([]any)
if !containsJSONField(sourceRefRequired, "source_id") {
t.Fatalf("canonical source refs required = %#v, want source_id", sourceRefRequired)
}
for _, field := range []string{"start_unit_id", "end_unit_id"} {
property := sourceRefProperties[field].(map[string]any)
if property["type"] != "integer" {
t.Fatalf("%s type = %#v, want integer", field, property["type"])
}
if property["minimum"] != float64(1) {
t.Fatalf("%s minimum = %#v, want 1", field, property["minimum"])
}
}
}
func TestLLMResponseSchemaOmitsSourceID(t *testing.T) {
raw, err := embeddedAssets.ReadFile("assets/schemas/dnd_spells_llm.v1.json")
if err != nil {
t.Fatalf("ReadFile(LLM schema) error = %v, want nil", err)
}
if !json.Valid(raw) {
t.Fatalf("LLM schema is invalid JSON: %s", raw)
}
var decoded map[string]any
if err := json.Unmarshal(raw, &decoded); err != nil {
t.Fatalf("Unmarshal(LLM schema) error = %v, want nil", err)
if decoded["$id"] != "notarius.dnd.spells.llm" {
t.Fatalf("LLM schema $id = %#v, want extractor transport schema", decoded["$id"])
}
properties := decoded["properties"].(map[string]any)
spellCastProperties := properties["spell_casts"].(map[string]any)["items"].(map[string]any)["properties"].(map[string]any)
sourceRefItems := spellCastProperties["source_refs"].(map[string]any)["items"].(map[string]any)
sourceRefProperties := sourceRefItems["properties"].(map[string]any)
sourceRefRequired := sourceRefItems["required"].([]any)
if _, ok := sourceRefProperties["source_id"]; ok {
t.Fatalf("LLM source ref schema contains source_id property: %#v", sourceRefProperties)
}
if containsJSONField(sourceRefRequired, "source_id") {
t.Fatalf("LLM source refs required = %#v, want no source_id", sourceRefRequired)
t.Fatalf("LLM source ref schema contains canonical source_id: %#v", sourceRefProperties)
}
}
func TestResponseSchemaJSONIsMutationSafe(t *testing.T) {
first, err := loadResponseSchema()
if err != nil {
t.Fatalf("loadResponseSchema() error = %v, want nil", err)
t.Fatalf("loadResponseSchema() error = %v", err)
}
first.JSONSchema[0] = '['
second, err := loadResponseSchema()
if err != nil {
t.Fatalf("loadResponseSchema() error = %v, want nil", err)
}
if !json.Valid(second.JSONSchema) {
t.Fatalf("schema JSON was mutated: %s", second.JSONSchema)
}
if len(second.JSONSchema) > 0 && second.JSONSchema[0] == '[' {
t.Fatalf("schema JSON did not use defensive copy")
if err != nil || !json.Valid(second.JSONSchema) || bytes.Equal(first.JSONSchema, second.JSONSchema) {
t.Fatalf("second schema = %s, %v; want defensive valid copy", second.JSONSchema, err)
}
}
func TestResponseSchemaDiagnosticsOmitRawSchema(t *testing.T) {
schema, err := loadResponseSchema()
if err != nil {
t.Fatalf("loadResponseSchema() error = %v, want nil", err)
t.Fatalf("loadResponseSchema() error = %v", err)
}
diagnostics := schema.DiagnosticsMap()
if diagnostics["key"] != ResponseSchemaKey {
t.Fatalf("diagnostics[key] = %#v, want %q", diagnostics["key"], ResponseSchemaKey)
}
for _, key := range []string{"id", "version", "name", "sha256"} {
if diagnostics[key] == "" {
t.Fatalf("diagnostics[%q] = %#v, want value", key, diagnostics[key])
}
t.Fatalf("diagnostics = %#v, want response key", diagnostics)
}
if _, ok := diagnostics["json_schema"]; ok {
t.Fatalf("diagnostics should omit raw schema content: %#v", diagnostics)
}
if _, ok := diagnostics["JSONSchema"]; ok {
t.Fatalf("diagnostics should omit raw schema content: %#v", diagnostics)
t.Fatalf("diagnostics include raw schema: %#v", diagnostics)
}
}
func containsJSONField(fields []any, want string) bool {
for _, field := range fields {
if field == want {
return true
}
}
return false
}

View File

@@ -57,7 +57,7 @@ func TestScriptoriumPromptDiagnosticsOmitRawMaterials(t *testing.T) {
transcript := []byte(`{"secret":"source text"}`)
reference := "private party note"
prepared := prepareSpellsPrompt(t, transcript, "private player note", reference, " ")
metadata := New().ManifestMetadata()
metadata := newExtractor(t, &fakeSpellsLLMClient{}).ManifestMetadata()
payload, err := json.Marshal(map[string]any{
"prepared": map[string]any{

View File

@@ -1,7 +1,9 @@
package spells
import (
"context"
"encoding/json"
"errors"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
@@ -9,7 +11,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
func promptExtractionRequest() contracts.ExtractionRequest {
func promptExtractionRequest() contracts.TypedExtractionRequest {
doc := promptSourceDocument()
chunk := &source.Chunk{
ID: "session-alpha:chunk:0",
@@ -25,7 +27,7 @@ func promptExtractionRequest() contracts.ExtractionRequest {
Units: append([]source.SourceUnit(nil), doc.Units...),
Metadata: map[string]any{"ignored": "chunk metadata"},
}
return contracts.ExtractionRequest{
return contracts.TypedExtractionRequest{
Source: doc,
Chunk: chunk,
}
@@ -70,22 +72,90 @@ func mustJSON(t *testing.T, value any) string {
return string(encoded)
}
func responseSourceRefs(sourceID string, startUnitID int, endUnitID int) []shared.SourceRefResponse {
return []shared.SourceRefResponse{
func responseSourceRefs(startUnitID int, endUnitID int) []spellSourceRefResponse {
return []spellSourceRefResponse{
{
SourceID: sourceID,
StartUnitID: shared.UnitRefFromInt(startUnitID),
EndUnitID: shared.UnitRefFromInt(endUnitID),
},
}
}
func responseSourceRefsInt(sourceID string, startUnitID int, endUnitID int) []shared.SourceRefResponse {
return []shared.SourceRefResponse{
{
SourceID: sourceID,
StartUnitID: shared.UnitRefFromInt(startUnitID),
EndUnitID: shared.UnitRefFromInt(endUnitID),
},
}
const spellTranscriptJSON = `{"id":"session-alpha","segments":[{"id":1,"text":"Aria raises her hand and casts Cure Wounds."}]}`
func spellSourceInput() contracts.LLMInputMaterial {
return contracts.NewLLMInputMaterial("source", "application/json", []byte(spellTranscriptJSON), "sha256:transcript", "file:///session-alpha.json")
}
func spellChunkInput(chunk *source.Chunk) contracts.LLMInputMaterial {
return contracts.NewLLMInputMaterial("source", chunk.MediaType, chunk.Content, "sha256:chunk", "file:///session-alpha.json")
}
func extractionRequest() contracts.TypedExtractionRequest {
req := promptExtractionRequest()
req.SourceInput = spellChunkInput(req.Chunk)
req.SessionID = "session-123"
req.LLMProfile = "profile-spells"
return req
}
func emptyChunkRequest(req contracts.TypedExtractionRequest) contracts.TypedExtractionRequest {
req.Chunk = &source.Chunk{ID: req.Chunk.ID, SourceID: req.Chunk.SourceID, Index: req.Chunk.Index}
return req
}
func mismatchedSourceInputRequest(req contracts.TypedExtractionRequest) contracts.TypedExtractionRequest {
req.SourceInput = spellSourceInput()
return req
}
func newExtractor(t *testing.T, client contracts.StructuredLLMClient) *Extractor {
t.Helper()
extractor, err := New(client, Options{})
if err != nil {
t.Fatalf("New() error = %v, want nil", err)
}
return extractor
}
type fakeSpellsLLMClient struct {
response extractionResponse
content []byte
err error
requests []contracts.StructuredCompletionRequest
}
func (client *fakeSpellsLLMClient) CompleteStructured(_ context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
client.requests = append(client.requests, cloneStructuredCompletionRequest(req))
if client.err != nil {
return contracts.StructuredCompletionResponse{}, client.err
}
target, ok := out.(*extractionResponse)
if !ok {
return contracts.StructuredCompletionResponse{}, errors.New("unexpected output target")
}
*target = client.response
content := append([]byte(nil), client.content...)
if len(content) == 0 {
var err error
content, err = json.Marshal(client.response)
if err != nil {
return contracts.StructuredCompletionResponse{}, err
}
}
return contracts.StructuredCompletionResponse{Content: content}, nil
}
func cloneStructuredCompletionRequest(req contracts.StructuredCompletionRequest) contracts.StructuredCompletionRequest {
req.Inputs = req.Inputs.Clone()
if len(req.Vars) == 0 {
req.Vars = nil
return req
}
vars := make(map[string]any, len(req.Vars))
for key, value := range req.Vars {
vars[key] = value
}
req.Vars = vars
return req
}

View File

@@ -7,6 +7,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/chunk/scenes"
spellcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/spells"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
spellshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/shape"
spellsourcerefs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/source_refs"
@@ -24,8 +25,9 @@ func Register(registries pipeline.Registries, assets *llm.AssetRegistry) error {
name string
register func() error
}{
{name: "spells codec", register: func() error { return pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, spellcodec.New()) }},
{name: "scenes chunker", register: func() error { return scenes.Register(registries.Chunkers) }},
{name: "spells extractor", register: func() error { return spells.Register(registries.Extractors) }},
{name: "spells extractor", register: func() error { return spells.RegisterWithRawAdapter(registries.Extractors, spellcodec.New()) }},
{name: "spell shape validator", register: func() error { return spellshape.Register(registries.Validators) }},
{name: "spell source references validator", register: func() error { return spellsourcerefs.Register(registries.Validators) }},
{name: "spell source relatedness validator", register: func() error { return spellrelatedness.Register(registries.Validators) }},
@@ -57,6 +59,8 @@ func validateRegistries(registries pipeline.Registries, assets *llm.AssetRegistr
switch {
case registries.Chunkers == nil:
return fmt.Errorf("dnd registrar: chunker registry must not be nil")
case registries.ArtifactCodecs == nil:
return fmt.Errorf("dnd registrar: artifact codec registry must not be nil")
case registries.Extractors == nil:
return fmt.Errorf("dnd registrar: extractor registry must not be nil")
case registries.Validators == nil:

View File

@@ -7,6 +7,7 @@ import (
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
@@ -20,6 +21,9 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
}
assertKeys(t, "chunkers", registries.Chunkers.RegisteredKeys(), []string{"dnd/scenes"})
assertKeys(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells"})
if got := registries.ArtifactCodecs.RegisteredKinds(); !reflect.DeepEqual(got, []contracts.ArtifactKind{"dnd/spell-list"}) {
t.Fatalf("artifact codec kinds = %#v, want dnd/spell-list", got)
}
assertKeys(t, "validators", registries.Validators.RegisteredKeys(), []string{
"extract/dnd/spells/shape",
"extract/dnd/spells/source_refs",
@@ -51,7 +55,6 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
})
assertAssetNames(t, assets.SchemaFS, []string{
"dnd_scenes.v1.json",
"dnd_spells.v1.json",
"dnd_spells_llm.v1.json",
})
}
@@ -63,6 +66,7 @@ func TestRegisterRejectsMissingDNDDependenciesBeforeMutation(t *testing.T) {
wantErr string
}{
{name: "chunkers", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.Chunkers = nil }, wantErr: "chunker registry"},
{name: "artifact codecs", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.ArtifactCodecs = nil }, wantErr: "artifact codec registry"},
{name: "extractors", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.Extractors = nil }, wantErr: "extractor registry"},
{name: "validators", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.Validators = nil }, wantErr: "validator registry"},
{name: "validator chains", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.ValidatorChains = nil }, wantErr: "validator chain registry"},
@@ -91,7 +95,7 @@ func TestRegisterReportsDuplicateDNDRegistration(t *testing.T) {
t.Fatalf("first Register() error = %v, want nil", err)
}
err := Register(registries, assets)
if err == nil || !strings.Contains(err.Error(), "register dnd scenes chunker") || !strings.Contains(err.Error(), "already registered") {
if err == nil || !strings.Contains(err.Error(), "register dnd spells codec") || !strings.Contains(err.Error(), "already registered") {
t.Fatalf("second Register() error = %v, want contextual duplicate error", err)
}
}

View File

@@ -0,0 +1,21 @@
// Package dnd owns the canonical in-process artifact types for the D&D domain.
package dnd
import (
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
)
const SpellListKind contracts.ArtifactKind = "dnd/spell-list"
type SpellList struct {
SpellCasts []SpellCast `json:"spell_casts"`
}
type SpellCast struct {
Caster string `json:"caster"`
Spell string `json:"spell"`
Effect string `json:"effect"`
NarrativeDescription string `json:"narrative_description"`
SourceRefs []source.SourceRef `json:"source_refs"`
}

View File

@@ -10,6 +10,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
spellcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/spells"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
"gitea.maximumdirect.net/eric/notarius/internal/modules/generic/merge/appendorder"
"gitea.maximumdirect.net/eric/notarius/internal/modules/generic/normalize/noop"
@@ -152,6 +153,7 @@ func dndSpellsTestCatalog(t *testing.T, specs dndSpellsCatalogSpecs) pipeline.Mo
inputs := pipeline.NewInputAdapterRegistry()
chunkers := pipeline.NewChunkerRegistry()
extractors := pipeline.NewExtractorRegistry()
codecs := pipeline.NewArtifactCodecRegistry()
mergers := pipeline.NewMergerRegistry()
normalizers := pipeline.NewNormalizerRegistry()
outputs := pipeline.NewOutputEncoderRegistry()
@@ -177,13 +179,20 @@ func dndSpellsTestCatalog(t *testing.T, specs dndSpellsCatalogSpecs) pipeline.Mo
}
if specs.extractor.Key == "" {
if err := spells.Register(extractors); err != nil {
codec := spellcodec.New()
if err := pipeline.RegisterArtifactCodec(codecs, codec); err != nil {
t.Fatalf("register dnd spells codec: %v", err)
}
if err := spells.RegisterWithRawAdapter(extractors, codec); err != nil {
t.Fatalf("register dnd spells extractor: %v", err)
}
} else if err := extractors.RegisterLegacyRawWithSpec(specs.extractor, func() (contracts.LegacyRawExtractor, error) {
return spells.New(), nil
}); err != nil {
t.Fatalf("register dnd spells extractor override: %v", err)
} else {
specs.extractor.ArtifactKind = ""
if err := extractors.RegisterLegacyRawWithSpec(specs.extractor, func() (contracts.LegacyRawExtractor, error) {
return configLegacyExtractor{key: specs.extractor.Key}, nil
}); err != nil {
t.Fatalf("register dnd spells extractor override: %v", err)
}
}
if err := mergers.RegisterLegacyRawWithSpec(pipeline.ModuleSpec{
@@ -215,7 +224,7 @@ func dndSpellsTestCatalog(t *testing.T, specs dndSpellsCatalogSpecs) pipeline.Mo
return pipeline.ModuleCatalog{
Inputs: inputs,
Chunkers: chunkers,
ArtifactCodecs: pipeline.NewArtifactCodecRegistry(),
ArtifactCodecs: codecs,
Extractors: extractors,
Mergers: mergers,
Normalizers: normalizers,
@@ -224,6 +233,14 @@ func dndSpellsTestCatalog(t *testing.T, specs dndSpellsCatalogSpecs) pipeline.Mo
}
}
type configLegacyExtractor struct{ key string }
func (extractor configLegacyExtractor) Key() string { return extractor.key }
func (configLegacyExtractor) ReferenceSlots() []contracts.ReferenceSlot { return nil }
func (configLegacyExtractor) Extract(context.Context, contracts.ExtractionRequest) (contracts.ExtractionResult, error) {
return contracts.ExtractionResult{}, nil
}
func dndSpellsChunkerSpec() pipeline.ModuleSpec {
return pipeline.ModuleSpec{
Key: "fake/chunk",