eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add typed spell validation strategies

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-17 07:02:30 +00:00
parent 142ba36695
commit 52e6b31408
25 changed files with 708 additions and 410 deletions
Download Patch File Download Diff File Expand all files Collapse all files

51
docs/internal/modules.md
View File

@@ -21,10 +21,10 @@ Production input, chunk, output, and D&D spell-extract packages register strict
option decoders and run-local builders. Preparation decodes their options into
implementation-owned values and injects dependencies. The spell extractor is
typed over the canonical D&D model; a temporary raw adapter preserves the
current downstream production path. Production merge, normalize, and validator
packages still use the explicit legacy raw registration APIs and temporary
adapters around zero-argument constructors. Their raw option maps and LLM
clients remain operation inputs while that part of the catalog migrates.
current runner path. D&D validators, merge, and normalize use typed variants;
JSON representation validators use serialized requests; and unconditional
validators expose separate chunk and typed variants. Narrow legacy
registrations preserve current raw execution until the runner migrates.
Specs expose capability and execution metadata without constructing an
implementation. Registry entries separately expose option validation and
@@ -123,16 +123,16 @@ The durable payload and manifest metadata shapes are defined in the
### `internal/modules/generic/merge/appendorder`
The merger preserves extract-result order. It passes through one JSON result,
concatenates a common top-level array field across multiple JSON objects, and
otherwise emits an array of the decoded values. It rejects invalid JSON and
non-JSON media types, and it preserves compatible schema provenance.
The typed merger passes values to an injected combine function in framework
source-chunk order. The D&D registrar specializes it with a spell-list append
function. Its temporary raw implementation retains the prior JSON merge
behavior for the current runner.
### `internal/modules/generic/normalize/noop`
The normalizer defensively clones the accepted merge result, including payload
bytes, metadata, warnings, and schema provenance, without changing its logical
content.
The typed normalizer returns the merged domain value unchanged and is reusable
for any registered artifact type. Its temporary raw implementation defensively
clones the accepted payload for the current runner.
## Output Encoder
@@ -152,26 +152,23 @@ paths and schemas.
The generic validator implementations live under
`internal/modules/generic/validate`.
The unconditional accept and reject validators provide deterministic production
registrations used primarily for controlled composition and tests.
The unconditional accept and reject validators provide explicit chunk and
typed-artifact variants used primarily for controlled composition and tests.
The JSON syntax validator uses `encoding/json` to reject malformed payloads. The
JSON Schema validator requires schema bytes on the validation request, parses
the instance and schema with `jsonschema`, and distinguishes payload rejection
from schema loading or compilation errors. Neither validator calls the LLM.
The serialized JSON syntax validator uses `encoding/json` to reject malformed
representation bytes. The serialized JSON Schema validator requires schema
bytes, parses the instance and schema with `jsonschema`, and distinguishes
payload rejection from schema loading or compilation errors. The framework
serialized-validation request carries either canonical chunk bytes or artifact
codec bytes according to its target context. Neither validator calls the LLM.
## D&D Spell Validators
`internal/modules/dnd/validate/spells/spellpayload` provides strict decoding,
shape checks, source-reference candidates, and cited-text lookup shared by the
three validators.
The shape validator rejects malformed JSON, unknown fields, missing or empty
spell fields, and empty reference lists. The source-reference validator applies
generic source-reference validation to every cited range. The relatedness
validator approves structurally valid payloads but warns when a case-insensitive
spell name is absent from all cited source text. It leaves malformed payloads to
the earlier validators in the configured chain.
All three validators receive `dnd.SpellList` directly. The shape validator
rejects missing or empty spell fields and empty reference lists. The
source-reference validator applies generic source-reference validation to every
cited range. The relatedness validator warns when a case-insensitive spell name
is absent from all cited source text.
These validators are deterministic. Their selectable keys and production order
are defined in

13
docs/internal/overview.md
View File

@@ -64,10 +64,11 @@ parsing. Production input, chunk, and output modules use strict construction-tim
option decoding, and the LLM-backed scene chunker retains the injected shared
client. The D&D family registers the canonical `dnd/spell-list` codec and a
typed spell extractor. A temporary raw adapter serializes that typed result for
the still-raw production validators, merger, normalizer, and runner. Other
artifact-lane modules and validators continue through explicitly named legacy
raw registrations and temporary zero-argument constructor adapters. The current
runner rejects a typed prepared lane instead of routing it through raw execution.
the current runner. The D&D family also registers typed spell validators and
kind-specific generic merge and normalize strategies; generic JSON validators
use the serialized-validation contract. Narrow legacy registrations preserve
the existing raw runner path until typed execution lands. The current runner
rejects a typed prepared lane instead of routing it through raw execution.
## Production Extensions
@@ -96,8 +97,8 @@ helpers. Domain-neutral prompt filesystem composition lives in
Generic validators under `internal/modules/generic/validate` provide
unconditional test decisions, JSON syntax validation, and JSON Schema
validation. D&D spell validators under `internal/modules/dnd/validate/spells`
provide shape, source-reference, and source-relatedness decisions, with
`spellpayload` holding their shared parser and lookup helpers.
consume the canonical spell-list type directly to provide shape,
source-reference, and source-relatedness decisions.
Production composition is grouped behind package-family registrars, and every
implemented production extension uses its domain-first tree:

8
docs/internal/pipeline.md
View File

@@ -75,9 +75,11 @@ mismatches are rejected deterministically.
Production composition registers the D&D spell-list codec. The typed spell
extractor also registers a temporary raw adapter, which resolution selects
until its downstream production lane is typed. Other production artifact-lane
modules use the explicitly named legacy raw registration APIs. A standalone raw
registration cannot satisfy a typed lane.
until runner execution is typed. The D&D family registers matching typed merge,
normalize, and semantic-validator variants, while JSON validators register for
serialized chunk and artifact targets. Narrow legacy registrations preserve the
current raw runner path. A standalone raw registration cannot satisfy a typed
lane.
A `ModuleSpec` declares its stage plus required and provided capabilities.
Chunk, extract, merge, and normalize specs may also declare reference slots.

16
internal/modules/dnd/codec/spells/codec.go
View File

@@ -58,6 +58,19 @@ func (c *Codec) EncodeCandidate(value dnd.SpellList) ([]byte, error) {
}
func (c *Codec) Decode(content []byte) (dnd.SpellList, error) {
value, err := c.DecodeCandidate(content)
if err != nil {
return dnd.SpellList{}, err
}
if err := validate(value); err != nil {
return dnd.SpellList{}, fmt.Errorf("decode dnd spell list: %w", err)
}
return value, nil
}
// DecodeCandidate reads the durable representation before semantic validators
// have approved it on the temporary raw runner path.
func (c *Codec) DecodeCandidate(content []byte) (dnd.SpellList, error) {
decoder := json.NewDecoder(bytes.NewReader(content))
decoder.DisallowUnknownFields()
var value dnd.SpellList
@@ -68,9 +81,6 @@ func (c *Codec) Decode(content []byte) (dnd.SpellList, error) {
if err := decoder.Decode(&trailing); err != io.EOF {
return dnd.SpellList{}, fmt.Errorf("decode dnd spell list: multiple JSON values")
}
if err := validate(value); err != nil {
return dnd.SpellList{}, fmt.Errorf("decode dnd spell list: %w", err)
}
return value, nil
}

39
internal/modules/dnd/register/register.go
View File

@@ -6,12 +6,17 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/chunk/scenes"
spellcodec "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/codec/spells"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/extract/spells"
spellshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/shape"
spellsourcerefs "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/source_refs"
spellrelatedness "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/source_relatedness"
"gitea.maximumdirect.net/eric/notarius/internal/modules/generic/merge/appendorder"
"gitea.maximumdirect.net/eric/notarius/internal/modules/generic/normalize/noop"
alwaysaccept "gitea.maximumdirect.net/eric/notarius/internal/modules/generic/validate/always_accept"
alwaysreject "gitea.maximumdirect.net/eric/notarius/internal/modules/generic/validate/always_reject"
validjson "gitea.maximumdirect.net/eric/notarius/internal/modules/generic/validate/valid_json"
validjsonschema "gitea.maximumdirect.net/eric/notarius/internal/modules/generic/validate/valid_json_schema"
)
@@ -21,16 +26,30 @@ func Register(registries pipeline.Registries, assets *llm.AssetRegistry) error {
if err := validateRegistries(registries, assets); err != nil {
return err
}
codec := spellcodec.New()
registrations := []struct {
name string
register func() error
}{
{name: "spells codec", register: func() error { return pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, spellcodec.New()) }},
{name: "spells codec", register: func() error { return pipeline.RegisterArtifactCodec(registries.ArtifactCodecs, codec) }},
{name: "scenes chunker", register: func() error { return scenes.Register(registries.Chunkers) }},
{name: "spells extractor", register: func() error { return spells.RegisterWithRawAdapter(registries.Extractors, spellcodec.New()) }},
{name: "spells extractor", register: func() error { return spells.RegisterWithRawAdapter(registries.Extractors, codec) }},
{name: "spell-list appendorder merger", register: func() error {
return appendorder.RegisterTyped(registries.Mergers, dnd.SpellListKind, appendSpellLists)
}},
{name: "spell-list noop normalizer", register: func() error { return noop.RegisterTyped[dnd.SpellList](registries.Normalizers, dnd.SpellListKind) }},
{name: "spell shape validator", register: func() error { return spellshape.Register(registries.Validators) }},
{name: "legacy spell shape validator", register: func() error { return spellshape.RegisterLegacy(registries.Validators, codec) }},
{name: "spell source references validator", register: func() error { return spellsourcerefs.Register(registries.Validators) }},
{name: "legacy spell source references validator", register: func() error { return spellsourcerefs.RegisterLegacy(registries.Validators, codec) }},
{name: "spell source relatedness validator", register: func() error { return spellrelatedness.Register(registries.Validators) }},
{name: "legacy spell source relatedness validator", register: func() error { return spellrelatedness.RegisterLegacy(registries.Validators, codec) }},
{name: "spell-list always accept validator", register: func() error {
return alwaysaccept.RegisterTyped[dnd.SpellList](registries.Validators, dnd.SpellListKind)
}},
{name: "spell-list always reject validator", register: func() error {
return alwaysreject.RegisterTyped[dnd.SpellList](registries.Validators, dnd.SpellListKind)
}},
{name: "scenes prompt assets", register: func() error { return scenes.RegisterPromptAssets(assets) }},
{name: "spells prompt assets", register: func() error { return spells.RegisterPromptAssets(assets) }},
}
@@ -55,6 +74,18 @@ func Register(registries pipeline.Registries, assets *llm.AssetRegistry) error {
return nil
}
func appendSpellLists(values []dnd.SpellList) (dnd.SpellList, error) {
count := 0
for _, value := range values {
count += len(value.SpellCasts)
}
combined := dnd.SpellList{SpellCasts: make([]dnd.SpellCast, 0, count)}
for _, value := range values {
combined.SpellCasts = append(combined.SpellCasts, value.SpellCasts...)
}
return combined, nil
}
func validateRegistries(registries pipeline.Registries, assets *llm.AssetRegistry) error {
switch {
case registries.Chunkers == nil:
@@ -63,6 +94,10 @@ func validateRegistries(registries pipeline.Registries, assets *llm.AssetRegistr
return fmt.Errorf("dnd registrar: artifact codec registry must not be nil")
case registries.Extractors == nil:
return fmt.Errorf("dnd registrar: extractor registry must not be nil")
case registries.Mergers == nil:
return fmt.Errorf("dnd registrar: merger registry must not be nil")
case registries.Normalizers == nil:
return fmt.Errorf("dnd registrar: normalizer registry must not be nil")
case registries.Validators == nil:
return fmt.Errorf("dnd registrar: validator registry must not be nil")
case registries.ValidatorChains == nil:

4
internal/modules/dnd/register/register_test.go
View File

@@ -28,6 +28,8 @@ func TestRegisterAddsDNDFamily(t *testing.T) {
"extract/dnd/spells/shape",
"extract/dnd/spells/source_refs",
"extract/dnd/spells/source_relatedness",
"generic/always_accept",
"generic/always_reject",
})
wantChain := []pipeline.ModuleBinding{
pipeline.Binding("generic/valid_json"),
@@ -68,6 +70,8 @@ func TestRegisterRejectsMissingDNDDependenciesBeforeMutation(t *testing.T) {
{name: "chunkers", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.Chunkers = nil }, wantErr: "chunker registry"},
{name: "artifact codecs", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.ArtifactCodecs = nil }, wantErr: "artifact codec registry"},
{name: "extractors", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.Extractors = nil }, wantErr: "extractor registry"},
{name: "mergers", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.Mergers = nil }, wantErr: "merger registry"},
{name: "normalizers", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.Normalizers = nil }, wantErr: "normalizer registry"},
{name: "validators", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.Validators = nil }, wantErr: "validator registry"},
{name: "validator chains", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.ValidatorChains = nil }, wantErr: "validator chain registry"},
{name: "assets", remove: func(_ *pipeline.Registries, assets **llm.AssetRegistry) { *assets = nil }, wantErr: "asset registry"},

101
internal/modules/dnd/validate/spells/shape/validator.go
View File

@@ -2,59 +2,104 @@ package shape
import (
"context"
"fmt"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/spellpayload"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
)
const Key = "extract/dnd/spells/shape"
const ReasonCode = "invalid_spell_shape"
var _ contracts.LegacyRawValidator = (*Validator)(nil)
type Options struct{}
type Validator struct{}
func New() *Validator {
return &Validator{}
type legacyValidator struct{ codec decoder }
type decoder interface {
DecodeCandidate([]byte) (dnd.SpellList, error)
}
func (v *Validator) Name() string {
return Key
}
var _ contracts.TypedValidator[dnd.SpellList] = (*Validator)(nil)
func New(Options) *Validator { return &Validator{} }
func (v *Validator) Name() string { return Key }
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *Validator) Validate(ctx context.Context, req contracts.ValidationRequest) (contracts.ValidationResult, error) {
payload, err := spellpayload.ValidationRequestPayload(req)
if err != nil {
return rejection(err.Error()), nil
}
if err := spellpayload.ValidateShape(payload); err != nil {
func (v *Validator) Validate(_ context.Context, req contracts.TypedValidationRequest[dnd.SpellList]) (contracts.ValidationResult, error) {
if err := Validate(req.Value); err != nil {
return rejection(err.Error()), nil
}
return contracts.ValidationResult{Approved: true}, nil
}
func Spec() pipeline.ValidatorSpec {
return pipeline.ValidatorSpec{
Key: Key,
ExecutionClass: contracts.ExecutionClassDeterministic,
func Validate(value dnd.SpellList) error {
if value.SpellCasts == nil {
return fmt.Errorf("spell_casts must be present")
}
for index, spell := range value.SpellCasts {
if strings.TrimSpace(spell.Caster) == "" {
return fmt.Errorf("spell_casts[%d].caster must not be empty", index)
}
if strings.TrimSpace(spell.Spell) == "" {
return fmt.Errorf("spell_casts[%d].spell must not be empty", index)
}
if strings.TrimSpace(spell.Effect) == "" {
return fmt.Errorf("spell_casts[%d].effect must not be empty", index)
}
if strings.TrimSpace(spell.NarrativeDescription) == "" {
return fmt.Errorf("spell_casts[%d].narrative_description must not be empty", index)
}
if len(spell.SourceRefs) == 0 {
return fmt.Errorf("spell_casts[%d].source_refs must not be empty", index)
}
}
return nil
}
func (v *legacyValidator) Name() string { return Key }
func (v *legacyValidator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *legacyValidator) Validate(ctx context.Context, req contracts.ValidationRequest) (contracts.ValidationResult, error) {
value, err := v.codec.DecodeCandidate(req.Payload.Content)
if err != nil {
return rejection(err.Error()), nil
}
return New(Options{}).Validate(ctx, contracts.TypedValidationRequest[dnd.SpellList]{Value: value})
}
func Spec() pipeline.ValidatorSpec {
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
}
func Register(registry *pipeline.ValidatorRegistry) error {
return registry.RegisterLegacyRawWithSpec(Spec(), func() (contracts.LegacyRawValidator, error) {
return New(), nil
return pipeline.RegisterTypedValidatorBuilder(registry, dnd.SpellListKind, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.TypedValidator[dnd.SpellList], error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return New(options), nil
})
}
func rejection(message string) contracts.ValidationResult {
return contracts.ValidationResult{
Approved: false,
ReasonCode: ReasonCode,
Message: message,
func RegisterLegacy(registry *pipeline.ValidatorRegistry, codec decoder) error {
if codec == nil {
return fmt.Errorf("spell shape validator codec must not be nil")
}
return registry.RegisterLegacyRawBuilderWithSpec(Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.LegacyRawValidator, error) {
if _, err := DecodeOptions(request.Options); err != nil {
return nil, err
}
return &legacyValidator{codec: codec}, nil
})
}
func DecodeOptions(options map[string]any) (Options, error) {
if err := pipeline.RejectUnknownOptions(options); err != nil {
return Options{}, err
}
return Options{}, nil
}
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }
func rejection(message string) contracts.ValidationResult {
return contracts.ValidationResult{Approved: false, ReasonCode: ReasonCode, Message: message}
}

33
internal/modules/dnd/validate/spells/shape/validator_test.go
View File

@@ -4,12 +4,14 @@ import (
"context"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
)
func TestValidatorApprovesWellFormedSpellPayload(t *testing.T) {
result, err := New().Validate(context.Background(), requestWithPayload(`{"spell_casts":[{"caster":"Aria","spell":"Cure Wounds","effect":"heals","narrative_description":"Aria heals Borin.","source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]}]}`))
result, err := New(Options{}).Validate(context.Background(), requestWithValue(validSpellList()))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
@@ -18,8 +20,8 @@ func TestValidatorApprovesWellFormedSpellPayload(t *testing.T) {
}
}
func TestValidatorRejectsMalformedPayload(t *testing.T) {
result, err := New().Validate(context.Background(), requestWithPayload(`{"spell_casts":`))
func TestValidatorRejectsMissingSpellList(t *testing.T) {
result, err := New(Options{}).Validate(context.Background(), requestWithValue(dnd.SpellList{}))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
@@ -32,7 +34,9 @@ func TestValidatorRejectsMalformedPayload(t *testing.T) {
}
func TestValidatorRejectsMissingRequiredSpellFields(t *testing.T) {
result, err := New().Validate(context.Background(), requestWithPayload(`{"spell_casts":[{"caster":"Aria","effect":"heals","narrative_description":"Aria heals Borin.","source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]}]}`))
value := validSpellList()
value.SpellCasts[0].Spell = ""
result, err := New(Options{}).Validate(context.Background(), requestWithValue(value))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
@@ -49,20 +53,15 @@ func TestSpecAndRegister(t *testing.T) {
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
validator, err := registry.BuildLegacyRaw(Key)
if err != nil {
t.Fatalf("Build(%q) error = %v, want nil", Key, err)
}
if validator.Name() != Key || validator.ExecutionClass() != contracts.ExecutionClassDeterministic {
t.Fatalf("validator = %q/%q, want key and deterministic execution", validator.Name(), validator.ExecutionClass())
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
t.Fatal("DecodeOptions() error = nil, want unknown option error")
}
}
func requestWithPayload(payload string) contracts.ValidationRequest {
return contracts.ValidationRequest{
Payload: contracts.RawPayload{
Content: []byte(payload),
MediaType: "application/json",
},
}
func requestWithValue(value dnd.SpellList) contracts.TypedValidationRequest[dnd.SpellList] {
return contracts.TypedValidationRequest[dnd.SpellList]{Value: value}
}
func validSpellList() dnd.SpellList {
return dnd.SpellList{SpellCasts: []dnd.SpellCast{{Caster: "Aria", Spell: "Cure Wounds", Effect: "heals", NarrativeDescription: "Aria heals Borin.", SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: 1, EndUnitID: 1}}}}}
}

87
internal/modules/dnd/validate/spells/source_refs/validator.go
View File

@@ -7,38 +7,33 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/spellpayload"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
spellshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/shape"
)
const Key = "extract/dnd/spells/source_refs"
const ReasonCode = "invalid_source_refs"
var _ contracts.LegacyRawValidator = (*Validator)(nil)
type Options struct{}
type Validator struct{}
func New() *Validator {
return &Validator{}
type legacyValidator struct{ codec decoder }
type decoder interface {
DecodeCandidate([]byte) (dnd.SpellList, error)
}
func (v *Validator) Name() string {
return Key
}
var _ contracts.TypedValidator[dnd.SpellList] = (*Validator)(nil)
func New(Options) *Validator { return &Validator{} }
func (v *Validator) Name() string { return Key }
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *Validator) Validate(ctx context.Context, req contracts.ValidationRequest) (contracts.ValidationResult, error) {
payload, err := spellpayload.ValidationRequestPayload(req)
if err != nil {
func (v *Validator) Validate(_ context.Context, req contracts.TypedValidationRequest[dnd.SpellList]) (contracts.ValidationResult, error) {
if err := spellshape.Validate(req.Value); err != nil {
return rejection(err.Error()), nil
}
if err := spellpayload.ValidateShape(payload); err != nil {
return rejection(err.Error()), nil
}
for spellIndex, spell := range payload.SpellCasts {
for refIndex, ref := range spellpayload.SourceRefCandidates(req.Source, spell) {
for spellIndex, spell := range req.Value.SpellCasts {
for refIndex, ref := range spell.SourceRefs {
if err := source.ValidateRef(req.Source, ref); err != nil {
return rejection(fmt.Sprintf("spell_casts[%d].source_refs[%d]: %v", spellIndex, refIndex, err)), nil
}
@@ -46,24 +41,50 @@ func (v *Validator) Validate(ctx context.Context, req contracts.ValidationReques
}
return contracts.ValidationResult{Approved: true}, nil
}
func Spec() pipeline.ValidatorSpec {
return pipeline.ValidatorSpec{
Key: Key,
ExecutionClass: contracts.ExecutionClassDeterministic,
}
func (v *legacyValidator) Name() string { return Key }
func (v *legacyValidator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *legacyValidator) Validate(ctx context.Context, req contracts.ValidationRequest) (contracts.ValidationResult, error) {
value, err := v.codec.DecodeCandidate(req.Payload.Content)
if err != nil {
return rejection(err.Error()), nil
}
if err := spellshape.Validate(value); err != nil {
return rejection(err.Error()), nil
}
return New(Options{}).Validate(ctx, contracts.TypedValidationRequest[dnd.SpellList]{Source: req.Source, Value: value})
}
func Spec() pipeline.ValidatorSpec {
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
}
func Register(registry *pipeline.ValidatorRegistry) error {
return registry.RegisterLegacyRawWithSpec(Spec(), func() (contracts.LegacyRawValidator, error) {
return New(), nil
return pipeline.RegisterTypedValidatorBuilder(registry, dnd.SpellListKind, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.TypedValidator[dnd.SpellList], error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return New(options), nil
})
}
func rejection(message string) contracts.ValidationResult {
return contracts.ValidationResult{
Approved: false,
ReasonCode: ReasonCode,
Message: message,
func RegisterLegacy(registry *pipeline.ValidatorRegistry, codec decoder) error {
if codec == nil {
return fmt.Errorf("spell source references validator codec must not be nil")
}
return registry.RegisterLegacyRawBuilderWithSpec(Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.LegacyRawValidator, error) {
if _, err := DecodeOptions(request.Options); err != nil {
return nil, err
}
return &legacyValidator{codec: codec}, nil
})
}
func DecodeOptions(options map[string]any) (Options, error) {
if err := pipeline.RejectUnknownOptions(options); err != nil {
return Options{}, err
}
return Options{}, nil
}
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }
func rejection(message string) contracts.ValidationResult {
return contracts.ValidationResult{Approved: false, ReasonCode: ReasonCode, Message: message}
}

27
internal/modules/dnd/validate/spells/source_refs/validator_test.go
View File

@@ -7,10 +7,11 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
)
func TestValidatorApprovesValidSourceRefs(t *testing.T) {
result, err := New().Validate(context.Background(), requestWithPayload(validDocument(), `{"spell_casts":[{"caster":"Aria","spell":"Cure Wounds","effect":"heals","narrative_description":"Aria casts Cure Wounds.","source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":2}]}]}`))
result, err := New(Options{}).Validate(context.Background(), requestWithValue(validDocument(), source.SourceRef{SourceID: "session", StartUnitID: 1, EndUnitID: 2}))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
@@ -20,7 +21,7 @@ func TestValidatorApprovesValidSourceRefs(t *testing.T) {
}
func TestValidatorRejectsInvalidSourceRefs(t *testing.T) {
result, err := New().Validate(context.Background(), requestWithPayload(validDocument(), `{"spell_casts":[{"caster":"Aria","spell":"Cure Wounds","effect":"heals","narrative_description":"Aria casts Cure Wounds.","source_refs":[{"source_id":"session","start_unit_id":99,"end_unit_id":99}]}]}`))
result, err := New(Options{}).Validate(context.Background(), requestWithValue(validDocument(), source.SourceRef{SourceID: "session", StartUnitID: 99, EndUnitID: 99}))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
@@ -33,7 +34,7 @@ func TestValidatorRejectsInvalidSourceRefs(t *testing.T) {
}
func TestValidatorRejectsMissingSourceDocument(t *testing.T) {
result, err := New().Validate(context.Background(), requestWithPayload(nil, `{"spell_casts":[{"caster":"Aria","spell":"Cure Wounds","effect":"heals","narrative_description":"Aria casts Cure Wounds.","source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]}]}`))
result, err := New(Options{}).Validate(context.Background(), requestWithValue(nil, source.SourceRef{SourceID: "session", StartUnitID: 1, EndUnitID: 1}))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
@@ -50,23 +51,15 @@ func TestSpecAndRegister(t *testing.T) {
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
validator, err := registry.BuildLegacyRaw(Key)
if err != nil {
t.Fatalf("Build(%q) error = %v, want nil", Key, err)
}
if validator.Name() != Key || validator.ExecutionClass() != contracts.ExecutionClassDeterministic {
t.Fatalf("validator = %q/%q, want key and deterministic execution", validator.Name(), validator.ExecutionClass())
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
t.Fatal("DecodeOptions() error = nil, want unknown option error")
}
}
func requestWithPayload(doc *source.SourceDocument, payload string) contracts.ValidationRequest {
return contracts.ValidationRequest{
Source: doc,
Payload: contracts.RawPayload{
Content: []byte(payload),
MediaType: "application/json",
},
}
func requestWithValue(doc *source.SourceDocument, ref source.SourceRef) contracts.TypedValidationRequest[dnd.SpellList] {
return contracts.TypedValidationRequest[dnd.SpellList]{Source: doc, Value: dnd.SpellList{SpellCasts: []dnd.SpellCast{{
Caster: "Aria", Spell: "Cure Wounds", Effect: "heals", NarrativeDescription: "Aria casts Cure Wounds.", SourceRefs: []source.SourceRef{ref},
}}}}
}
func validDocument() *source.SourceDocument {

121
internal/modules/dnd/validate/spells/source_relatedness/validator.go
View File

@@ -8,76 +8,113 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/spellpayload"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
spellshape "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/validate/spells/shape"
)
const Key = "extract/dnd/spells/source_relatedness"
const WarningReasonCode = "spell_not_near_source"
var _ contracts.LegacyRawValidator = (*Validator)(nil)
type Options struct{}
type Validator struct{}
func New() *Validator {
return &Validator{}
type legacyValidator struct{ codec decoder }
type decoder interface {
DecodeCandidate([]byte) (dnd.SpellList, error)
}
func (v *Validator) Name() string {
return Key
}
var _ contracts.TypedValidator[dnd.SpellList] = (*Validator)(nil)
func New(Options) *Validator { return &Validator{} }
func (v *Validator) Name() string { return Key }
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *Validator) Validate(ctx context.Context, req contracts.ValidationRequest) (contracts.ValidationResult, error) {
payload, err := spellpayload.ValidationRequestPayload(req)
if err != nil {
func (v *Validator) Validate(_ context.Context, req contracts.TypedValidationRequest[dnd.SpellList]) (contracts.ValidationResult, error) {
if err := spellshape.Validate(req.Value); err != nil {
return contracts.ValidationResult{Approved: true}, nil
}
if err := spellpayload.ValidateShape(payload); err != nil {
return contracts.ValidationResult{Approved: true}, nil
}
var warnings []contracts.Warning
for spellIndex, spell := range payload.SpellCasts {
for spellIndex, spell := range req.Value.SpellCasts {
if !spellAppearsInCitedText(req.Source, spell) {
warnings = append(warnings, contracts.Warning{
Scope: fmt.Sprintf("spell_casts[%d]", spellIndex),
ReasonCode: WarningReasonCode,
Message: fmt.Sprintf("spell %q was not found in cited source text", strings.TrimSpace(spell.Spell)),
})
warnings = append(warnings, contracts.Warning{Scope: fmt.Sprintf("spell_casts[%d]", spellIndex), ReasonCode: WarningReasonCode, Message: fmt.Sprintf("spell %q was not found in cited source text", strings.TrimSpace(spell.Spell))})
}
}
return contracts.ValidationResult{Approved: true, Warnings: warnings}, nil
}
func Spec() pipeline.ValidatorSpec {
return pipeline.ValidatorSpec{
Key: Key,
ExecutionClass: contracts.ExecutionClassDeterministic,
func (v *legacyValidator) Name() string { return Key }
func (v *legacyValidator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *legacyValidator) Validate(ctx context.Context, req contracts.ValidationRequest) (contracts.ValidationResult, error) {
value, err := v.codec.DecodeCandidate(req.Payload.Content)
if err != nil {
return contracts.ValidationResult{Approved: true}, nil
}
if err := spellshape.Validate(value); err != nil {
return contracts.ValidationResult{Approved: true}, nil
}
return New(Options{}).Validate(ctx, contracts.TypedValidationRequest[dnd.SpellList]{Source: req.Source, Value: value})
}
func Register(registry *pipeline.ValidatorRegistry) error {
return registry.RegisterLegacyRawWithSpec(Spec(), func() (contracts.LegacyRawValidator, error) {
return New(), nil
})
}
func spellAppearsInCitedText(doc *source.SourceDocument, spell spellpayload.SpellCast) bool {
func spellAppearsInCitedText(doc *source.SourceDocument, spell dnd.SpellCast) bool {
name := strings.ToLower(strings.TrimSpace(spell.Spell))
if name == "" {
return true
}
for _, ref := range spellpayload.SourceRefCandidates(doc, spell) {
text, ok := spellpayload.CitedText(doc, ref)
if !ok {
continue
}
if strings.Contains(strings.ToLower(text), name) {
for _, ref := range spell.SourceRefs {
if text, ok := citedText(doc, ref); ok && strings.Contains(strings.ToLower(text), name) {
return true
}
}
return false
}
func citedText(doc *source.SourceDocument, ref source.SourceRef) (string, bool) {
if doc == nil {
return "", false
}
start, ok := source.UnitIndex(doc, ref.StartUnitID)
if !ok {
return "", false
}
end, ok := source.UnitIndex(doc, ref.EndUnitID)
if !ok || start > end {
return "", false
}
var b strings.Builder
for i := start; i <= end; i++ {
if b.Len() > 0 {
b.WriteByte('\n')
}
b.WriteString(doc.Units[i].Text)
}
return b.String(), true
}
func Spec() pipeline.ValidatorSpec {
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
}
func Register(registry *pipeline.ValidatorRegistry) error {
return pipeline.RegisterTypedValidatorBuilder(registry, dnd.SpellListKind, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.TypedValidator[dnd.SpellList], error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return New(options), nil
})
}
func RegisterLegacy(registry *pipeline.ValidatorRegistry, codec decoder) error {
if codec == nil {
return fmt.Errorf("spell source relatedness validator codec must not be nil")
}
return registry.RegisterLegacyRawBuilderWithSpec(Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.LegacyRawValidator, error) {
if _, err := DecodeOptions(request.Options); err != nil {
return nil, err
}
return &legacyValidator{codec: codec}, nil
})
}
func DecodeOptions(options map[string]any) (Options, error) {
if err := pipeline.RejectUnknownOptions(options); err != nil {
return Options{}, err
}
return Options{}, nil
}
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }

30
internal/modules/dnd/validate/spells/source_relatedness/validator_test.go
View File

@@ -7,10 +7,11 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd"
)
func TestValidatorApprovesWithoutWarningWhenSpellAppearsInCitedText(t *testing.T) {
result, err := New().Validate(context.Background(), requestWithPayload(validDocument(), `{"spell_casts":[{"caster":"Aria","spell":"Cure Wounds","effect":"heals","narrative_description":"Aria casts Cure Wounds.","source_refs":[{"source_id":"session","start_unit_id":2,"end_unit_id":2}]}]}`))
result, err := New(Options{}).Validate(context.Background(), requestWithSpell(validDocument(), "Cure Wounds", 2))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
@@ -23,7 +24,7 @@ func TestValidatorApprovesWithoutWarningWhenSpellAppearsInCitedText(t *testing.T
}
func TestValidatorWarnsWhenSpellDoesNotAppearInCitedText(t *testing.T) {
result, err := New().Validate(context.Background(), requestWithPayload(validDocument(), `{"spell_casts":[{"caster":"Borin","spell":"Fire Bolt","effect":"scorches","narrative_description":"Borin casts Fire Bolt.","source_refs":[{"source_id":"session","start_unit_id":1,"end_unit_id":1}]}]}`))
result, err := New(Options{}).Validate(context.Background(), requestWithSpell(validDocument(), "Fire Bolt", 1))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
@@ -38,8 +39,8 @@ func TestValidatorWarnsWhenSpellDoesNotAppearInCitedText(t *testing.T) {
}
}
func TestValidatorApprovesMalformedPayloadWithoutWarning(t *testing.T) {
result, err := New().Validate(context.Background(), requestWithPayload(validDocument(), `{"spell_casts":`))
func TestValidatorApprovesEmptySpellListWithoutWarning(t *testing.T) {
result, err := New(Options{}).Validate(context.Background(), contracts.TypedValidationRequest[dnd.SpellList]{Source: validDocument(), Value: dnd.SpellList{SpellCasts: []dnd.SpellCast{}}})
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
@@ -53,23 +54,16 @@ func TestSpecAndRegister(t *testing.T) {
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
validator, err := registry.BuildLegacyRaw(Key)
if err != nil {
t.Fatalf("Build(%q) error = %v, want nil", Key, err)
}
if validator.Name() != Key || validator.ExecutionClass() != contracts.ExecutionClassDeterministic {
t.Fatalf("validator = %q/%q, want key and deterministic execution", validator.Name(), validator.ExecutionClass())
if _, err := DecodeOptions(map[string]any{"unexpected": true}); err == nil {
t.Fatal("DecodeOptions() error = nil, want unknown option error")
}
}
func requestWithPayload(doc *source.SourceDocument, payload string) contracts.ValidationRequest {
return contracts.ValidationRequest{
Source: doc,
Payload: contracts.RawPayload{
Content: []byte(payload),
MediaType: "application/json",
},
}
func requestWithSpell(doc *source.SourceDocument, name string, unitID int) contracts.TypedValidationRequest[dnd.SpellList] {
return contracts.TypedValidationRequest[dnd.SpellList]{Source: doc, Value: dnd.SpellList{SpellCasts: []dnd.SpellCast{{
Caster: "Aria", Spell: name, Effect: "effect", NarrativeDescription: "description",
SourceRefs: []source.SourceRef{{SourceID: "session", StartUnitID: unitID, EndUnitID: unitID}},
}}}}
}
func validDocument() *source.SourceDocument {

98
internal/modules/dnd/validate/spells/spellpayload/payload.go
View File

@@ -1,98 +0,0 @@
package spellpayload
import (
"bytes"
"encoding/json"
"fmt"
"io"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/shared"
)
type Payload struct {
SpellCasts []SpellCast `json:"spell_casts"`
}
type SpellCast struct {
Caster string `json:"caster"`
Spell string `json:"spell"`
Effect string `json:"effect"`
NarrativeDescription string `json:"narrative_description"`
SourceRefs []shared.SourceRefResponse `json:"source_refs"`
}
func Parse(raw []byte) (Payload, error) {
decoder := json.NewDecoder(bytes.NewReader(raw))
decoder.DisallowUnknownFields()
var payload Payload
if err := decoder.Decode(&payload); err != nil {
return Payload{}, fmt.Errorf("parse spell payload: %w", err)
}
var extra any
if err := decoder.Decode(&extra); err != io.EOF {
return Payload{}, fmt.Errorf("parse spell payload: multiple JSON values")
}
return payload, nil
}
func ValidateShape(payload Payload) error {
if payload.SpellCasts == nil {
return fmt.Errorf("spell_casts must be present")
}
for index, spell := range payload.SpellCasts {
if strings.TrimSpace(spell.Caster) == "" {
return fmt.Errorf("spell_casts[%d].caster must not be empty", index)
}
if strings.TrimSpace(spell.Spell) == "" {
return fmt.Errorf("spell_casts[%d].spell must not be empty", index)
}
if strings.TrimSpace(spell.Effect) == "" {
return fmt.Errorf("spell_casts[%d].effect must not be empty", index)
}
if strings.TrimSpace(spell.NarrativeDescription) == "" {
return fmt.Errorf("spell_casts[%d].narrative_description must not be empty", index)
}
if len(spell.SourceRefs) == 0 {
return fmt.Errorf("spell_casts[%d].source_refs must not be empty", index)
}
}
return nil
}
func SourceRefCandidates(doc *source.SourceDocument, spell SpellCast) []source.SourceRef {
refs := make([]source.SourceRef, 0, len(spell.SourceRefs))
for _, ref := range spell.SourceRefs {
refs = append(refs, shared.SourceRefCandidate(doc, ref))
}
return refs
}
func CitedText(doc *source.SourceDocument, ref source.SourceRef) (string, bool) {
if doc == nil {
return "", false
}
startIndex, ok := source.UnitIndex(doc, ref.StartUnitID)
if !ok {
return "", false
}
endIndex, ok := source.UnitIndex(doc, ref.EndUnitID)
if !ok || startIndex > endIndex {
return "", false
}
var b strings.Builder
for i := startIndex; i <= endIndex; i++ {
if b.Len() > 0 {
b.WriteByte('\n')
}
b.WriteString(doc.Units[i].Text)
}
return b.String(), true
}
func ValidationRequestPayload(req contracts.ValidationRequest) (Payload, error) {
return Parse(req.Payload.Content)
}

24
internal/modules/generic/merge/appendorder/merger_test.go
View File

@@ -174,6 +174,30 @@ func TestMergeRejectsInvalidJSONAndNonJSONMediaTypes(t *testing.T) {
}
}
func TestTypedMergeUsesRequestOrderForReusableValueType(t *testing.T) {
type notes struct{ Values []string }
merger, err := NewTyped(func(values []notes) (notes, error) {
var combined notes
for _, value := range values {
combined.Values = append(combined.Values, value.Values...)
}
return combined, nil
})
if err != nil {
t.Fatalf("NewTyped() error = %v", err)
}
result, err := merger.Merge(context.Background(), contracts.TypedMergeRequest[notes]{ExtractOutputs: []contracts.ExtractArtifact[notes]{
{ChunkIndex: 4, Value: notes{Values: []string{"first"}}},
{ChunkIndex: 1, Value: notes{Values: []string{"second"}}},
}})
if err != nil {
t.Fatalf("Merge() error = %v", err)
}
if got := result.Value.Values; !reflect.DeepEqual(got, []string{"first", "second"}) {
t.Fatalf("Values = %#v", got)
}
}
func extractOutput(chunkID string, chunkIndex int, content string) contracts.ExtractOutput {
return contracts.ExtractOutput{
LaneID: "events",

62
internal/modules/generic/merge/appendorder/typed.go Normal file
View File

@@ -0,0 +1,62 @@
package appendorder
import (
"context"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
// CombineFunc combines values in the source-chunk order supplied by the
// framework. Implementations must not reorder the slice.
type CombineFunc[T any] func([]T) (T, error)
type TypedMerger[T any] struct {
combine CombineFunc[T]
}
func NewTyped[T any](combine CombineFunc[T]) (*TypedMerger[T], error) {
if combine == nil {
return nil, mergerErrorf("combine function must not be nil")
}
return &TypedMerger[T]{combine: combine}, nil
}
func (m *TypedMerger[T]) Key() string { return Key }
func (m *TypedMerger[T]) Merge(ctx context.Context, req contracts.TypedMergeRequest[T]) (contracts.TypedMergeResult[T], error) {
if m == nil || m.combine == nil {
return contracts.TypedMergeResult[T]{}, mergerErrorf("merger must not be nil")
}
if ctx == nil {
return contracts.TypedMergeResult[T]{}, mergerErrorf("context must not be nil")
}
if err := ctx.Err(); err != nil {
return contracts.TypedMergeResult[T]{}, mergerErrorf("context error before merge: %w", err)
}
values := make([]T, len(req.ExtractOutputs))
for i, output := range req.ExtractOutputs {
values[i] = output.Value
}
value, err := m.combine(values)
if err != nil {
return contracts.TypedMergeResult[T]{}, mergerErrorf("combine values: %w", err)
}
return contracts.TypedMergeResult[T]{Value: value}, nil
}
func TypedModuleSpec(kind contracts.ArtifactKind) pipeline.ModuleSpec {
spec := ModuleSpec()
spec.ArtifactKind = kind
return spec
}
func RegisterTyped[T any](registry *pipeline.MergerRegistry, kind contracts.ArtifactKind, combine CombineFunc[T]) error {
validateOptions := func(options map[string]any) error { return pipeline.RejectUnknownOptions(options) }
return pipeline.RegisterMergerBuilder(registry, TypedModuleSpec(kind), validateOptions, func(request pipeline.BuildRequest) (contracts.Merger[T], error) {
if err := pipeline.RejectUnknownOptions(request.Options); err != nil {
return nil, err
}
return NewTyped(combine)
})
}

13
internal/modules/generic/normalize/noop/normalizer_test.go
View File

@@ -84,6 +84,19 @@ func TestNormalizeDefensivelyCopiesRawPayload(t *testing.T) {
}
}
func TestTypedNormalizePassesThroughReusableValueType(t *testing.T) {
type score struct{ Value int }
result, err := NewTyped[score]().Normalize(context.Background(), contracts.TypedNormalizeRequest[score]{
MergeOutput: contracts.MergeArtifact[score]{Value: score{Value: 7}},
})
if err != nil {
t.Fatalf("Normalize() error = %v", err)
}
if result.Value.Value != 7 {
t.Fatalf("Value = %d, want 7", result.Value.Value)
}
}
func mergeOutput(content string) contracts.MergeOutput {
return contracts.MergeOutput{
LaneID: "events",

44
internal/modules/generic/normalize/noop/typed.go Normal file
View File

@@ -0,0 +1,44 @@
package noop
import (
"context"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
type TypedNormalizer[T any] struct{}
func NewTyped[T any]() *TypedNormalizer[T] { return &TypedNormalizer[T]{} }
func (n *TypedNormalizer[T]) Key() string { return Key }
func (n *TypedNormalizer[T]) ReferenceSlots() []contracts.ReferenceSlot { return nil }
func (n *TypedNormalizer[T]) Normalize(ctx context.Context, req contracts.TypedNormalizeRequest[T]) (contracts.TypedNormalizeResult[T], error) {
if n == nil {
return contracts.TypedNormalizeResult[T]{}, normalizerErrorf("normalizer must not be nil")
}
if ctx == nil {
return contracts.TypedNormalizeResult[T]{}, normalizerErrorf("context must not be nil")
}
if err := ctx.Err(); err != nil {
return contracts.TypedNormalizeResult[T]{}, normalizerErrorf("context error before normalize: %w", err)
}
return contracts.TypedNormalizeResult[T]{Value: req.MergeOutput.Value}, nil
}
func TypedModuleSpec(kind contracts.ArtifactKind) pipeline.ModuleSpec {
spec := ModuleSpec()
spec.ArtifactKind = kind
return spec
}
func RegisterTyped[T any](registry *pipeline.NormalizerRegistry, kind contracts.ArtifactKind) error {
validateOptions := func(options map[string]any) error { return pipeline.RejectUnknownOptions(options) }
return pipeline.RegisterNormalizerBuilder(registry, TypedModuleSpec(kind), validateOptions, func(request pipeline.BuildRequest) (contracts.Normalizer[T], error) {
if err := pipeline.RejectUnknownOptions(request.Options); err != nil {
return nil, err
}
return NewTyped[T](), nil
})
}

75
internal/modules/generic/validate/always_accept/validator.go
View File

@@ -9,35 +9,76 @@ import (
const Key = "generic/always_accept"
var _ contracts.LegacyRawValidator = (*Validator)(nil)
type Options struct{}
type ChunkValidator struct{}
type TypedValidator[T any] struct{}
type legacyValidator struct{}
type Validator struct{}
var _ contracts.ChunkValidator = (*ChunkValidator)(nil)
func New() *Validator {
return &Validator{}
}
func NewChunk(Options) *ChunkValidator { return &ChunkValidator{} }
func NewTyped[T any](Options) *TypedValidator[T] { return &TypedValidator[T]{} }
func (v *Validator) Name() string {
return Key
}
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
func (v *ChunkValidator) Name() string { return Key }
func (v *ChunkValidator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *ChunkValidator) Validate(context.Context, contracts.ChunkValidationRequest) (contracts.ValidationResult, error) {
return contracts.ValidationResult{Approved: true}, nil
}
func (v *Validator) Validate(ctx context.Context, req contracts.ValidationRequest) (contracts.ValidationResult, error) {
func (v *TypedValidator[T]) Name() string { return Key }
func (v *TypedValidator[T]) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *TypedValidator[T]) Validate(context.Context, contracts.TypedValidationRequest[T]) (contracts.ValidationResult, error) {
return contracts.ValidationResult{Approved: true}, nil
}
func (v *legacyValidator) Name() string { return Key }
func (v *legacyValidator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *legacyValidator) Validate(context.Context, contracts.ValidationRequest) (contracts.ValidationResult, error) {
return contracts.ValidationResult{Approved: true}, nil
}
func Spec() pipeline.ValidatorSpec {
return pipeline.ValidatorSpec{
Key: Key,
ExecutionClass: contracts.ExecutionClassDeterministic,
}
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
}
func Register(registry *pipeline.ValidatorRegistry) error {
return registry.RegisterLegacyRawWithSpec(Spec(), func() (contracts.LegacyRawValidator, error) {
return New(), nil
if err := pipeline.RegisterChunkValidatorBuilder(registry, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.ChunkValidator, error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return NewChunk(options), nil
}); err != nil {
return err
}
return registry.RegisterLegacyRawBuilderWithSpec(Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.LegacyRawValidator, error) {
if _, err := DecodeOptions(request.Options); err != nil {
return nil, err
}
return &legacyValidator{}, nil
})
}
func RegisterTyped[T any](registry *pipeline.ValidatorRegistry, kind contracts.ArtifactKind) error {
return pipeline.RegisterTypedValidatorBuilder(registry, kind, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.TypedValidator[T], error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return NewTyped[T](options), nil
})
}
func DecodeOptions(options map[string]any) (Options, error) {
if err := pipeline.RejectUnknownOptions(options); err != nil {
return Options{}, err
}
return Options{}, nil
}
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }

2
internal/modules/generic/validate/always_accept/validator_test.go
View File

@@ -9,7 +9,7 @@ import (
)
func TestValidatorApproves(t *testing.T) {
result, err := New().Validate(context.Background(), contracts.ValidationRequest{})
result, err := NewChunk(Options{}).Validate(context.Background(), contracts.ChunkValidationRequest{})
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}

81
internal/modules/generic/validate/always_reject/validator.go
View File

@@ -10,39 +10,72 @@ import (
const Key = "generic/always_reject"
const ReasonCode = "always_reject"
var _ contracts.LegacyRawValidator = (*Validator)(nil)
type Options struct{}
type ChunkValidator struct{}
type TypedValidator[T any] struct{}
type legacyValidator struct{}
type Validator struct{}
func NewChunk(Options) *ChunkValidator { return &ChunkValidator{} }
func NewTyped[T any](Options) *TypedValidator[T] { return &TypedValidator[T]{} }
func New() *Validator {
return &Validator{}
func rejection() contracts.ValidationResult {
return contracts.ValidationResult{Approved: false, ReasonCode: ReasonCode, Message: "output rejected by always-reject validator"}
}
func (v *Validator) Name() string {
return Key
}
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
func (v *ChunkValidator) Name() string { return Key }
func (v *ChunkValidator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *Validator) Validate(ctx context.Context, req contracts.ValidationRequest) (contracts.ValidationResult, error) {
return contracts.ValidationResult{
Approved: false,
ReasonCode: ReasonCode,
Message: "output rejected by always-reject validator",
}, nil
func (v *ChunkValidator) Validate(context.Context, contracts.ChunkValidationRequest) (contracts.ValidationResult, error) {
return rejection(), nil
}
func (v *TypedValidator[T]) Name() string { return Key }
func (v *TypedValidator[T]) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *TypedValidator[T]) Validate(context.Context, contracts.TypedValidationRequest[T]) (contracts.ValidationResult, error) {
return rejection(), nil
}
func (v *legacyValidator) Name() string { return Key }
func (v *legacyValidator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *legacyValidator) Validate(context.Context, contracts.ValidationRequest) (contracts.ValidationResult, error) {
return rejection(), nil
}
func Spec() pipeline.ValidatorSpec {
return pipeline.ValidatorSpec{
Key: Key,
ExecutionClass: contracts.ExecutionClassDeterministic,
}
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
}
func Register(registry *pipeline.ValidatorRegistry) error {
return registry.RegisterLegacyRawWithSpec(Spec(), func() (contracts.LegacyRawValidator, error) {
return New(), nil
if err := pipeline.RegisterChunkValidatorBuilder(registry, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.ChunkValidator, error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return NewChunk(options), nil
}); err != nil {
return err
}
return registry.RegisterLegacyRawBuilderWithSpec(Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.LegacyRawValidator, error) {
if _, err := DecodeOptions(request.Options); err != nil {
return nil, err
}
return &legacyValidator{}, nil
})
}
func RegisterTyped[T any](registry *pipeline.ValidatorRegistry, kind contracts.ArtifactKind) error {
return pipeline.RegisterTypedValidatorBuilder(registry, kind, Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.TypedValidator[T], error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return NewTyped[T](options), nil
})
}
func DecodeOptions(options map[string]any) (Options, error) {
if err := pipeline.RejectUnknownOptions(options); err != nil {
return Options{}, err
}
return Options{}, nil
}
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }

2
internal/modules/generic/validate/always_reject/validator_test.go
View File

@@ -9,7 +9,7 @@ import (
)
func TestValidatorRejects(t *testing.T) {
result, err := New().Validate(context.Background(), contracts.ValidationRequest{})
result, err := NewTyped[string](Options{}).Validate(context.Background(), contracts.TypedValidationRequest[string]{Value: "value"})
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}

73
internal/modules/generic/validate/valid_json/validator.go
View File

@@ -11,42 +11,73 @@ import (
const Key = "generic/valid_json"
const ReasonCodeInvalidJSON = "invalid_json"
var _ contracts.LegacyRawValidator = (*Validator)(nil)
type Options struct{}
type Validator struct{}
func New() *Validator {
return &Validator{}
}
type legacyValidator struct{}
func (v *Validator) Name() string {
return Key
}
var _ contracts.SerializedValidator = (*Validator)(nil)
var _ contracts.LegacyRawValidator = (*legacyValidator)(nil)
func New(Options) *Validator { return &Validator{} }
func (v *Validator) Name() string { return Key }
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *Validator) Validate(ctx context.Context, req contracts.ValidationRequest) (contracts.ValidationResult, error) {
if !json.Valid(req.Payload.Content) {
return contracts.ValidationResult{
Approved: false,
ReasonCode: ReasonCodeInvalidJSON,
Message: "payload is not valid JSON",
}, nil
func (v *Validator) Validate(_ context.Context, req contracts.SerializedValidationRequest) (contracts.ValidationResult, error) {
return validate(req.Content), nil
}
func (v *legacyValidator) Name() string { return Key }
func (v *legacyValidator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *legacyValidator) Validate(_ context.Context, req contracts.ValidationRequest) (contracts.ValidationResult, error) {
return validate(req.Payload.Content), nil
}
func validate(content []byte) contracts.ValidationResult {
if !json.Valid(content) {
return contracts.ValidationResult{Approved: false, ReasonCode: ReasonCodeInvalidJSON, Message: "payload is not valid JSON"}
}
return contracts.ValidationResult{Approved: true}, nil
return contracts.ValidationResult{Approved: true}
}
func Spec() pipeline.ValidatorSpec {
return pipeline.ValidatorSpec{
Key: Key,
ExecutionClass: contracts.ExecutionClassDeterministic,
}
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
}
func Register(registry *pipeline.ValidatorRegistry) error {
return registry.RegisterLegacyRawWithSpec(Spec(), func() (contracts.LegacyRawValidator, error) {
return New(), nil
if err := pipeline.RegisterSerializedValidatorBuilder(registry, pipeline.SerializedValidatorSpec{
ValidatorSpec: Spec(), SupportsChunks: true, SupportsArtifacts: true,
}, validateOptions, func(request pipeline.BuildRequest) (contracts.SerializedValidator, error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return New(options), nil
}); err != nil {
return err
}
return registry.RegisterLegacyRawBuilderWithSpec(Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.LegacyRawValidator, error) {
if _, err := DecodeOptions(request.Options); err != nil {
return nil, err
}
return &legacyValidator{}, nil
})
}
func DecodeOptions(options map[string]any) (Options, error) {
if err := pipeline.RejectUnknownOptions(options); err != nil {
return Options{}, err
}
return Options{}, nil
}
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }

13
internal/modules/generic/validate/valid_json/validator_test.go
View File

@@ -15,7 +15,7 @@ func TestValidatorAcceptsValidJSON(t *testing.T) {
`"value"`,
}
for _, payload := range tests {
result, err := New().Validate(context.Background(), requestWithPayload(payload))
result, err := New(Options{}).Validate(context.Background(), requestWithPayload(payload))
if err != nil {
t.Fatalf("Validate(%s) error = %v, want nil", payload, err)
}
@@ -26,7 +26,7 @@ func TestValidatorAcceptsValidJSON(t *testing.T) {
}
func TestValidatorRejectsInvalidJSON(t *testing.T) {
result, err := New().Validate(context.Background(), requestWithPayload(`{"value":`))
result, err := New(Options{}).Validate(context.Background(), requestWithPayload(`{"value":`))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
@@ -56,11 +56,6 @@ func TestSpecAndRegister(t *testing.T) {
}
}
func requestWithPayload(payload string) contracts.ValidationRequest {
return contracts.ValidationRequest{
Payload: contracts.RawPayload{
Content: []byte(payload),
MediaType: "application/json",
},
}
func requestWithPayload(payload string) contracts.SerializedValidationRequest {
return contracts.SerializedValidationRequest{Content: []byte(payload), MediaType: "application/json"}
}

80
internal/modules/generic/validate/valid_json_schema/validator.go
View File

@@ -15,37 +15,40 @@ const Key = "generic/valid_json_schema"
const ReasonCodeInvalidJSON = "invalid_json"
const ReasonCodeSchemaInvalid = "json_schema_invalid"
var _ contracts.LegacyRawValidator = (*Validator)(nil)
type Options struct{}
type Validator struct{}
type legacyValidator struct{}
func New() *Validator {
return &Validator{}
}
func (v *Validator) Name() string {
return Key
}
var _ contracts.SerializedValidator = (*Validator)(nil)
var _ contracts.LegacyRawValidator = (*legacyValidator)(nil)
func New(Options) *Validator { return &Validator{} }
func (v *Validator) Name() string { return Key }
func (v *Validator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *Validator) Validate(ctx context.Context, req contracts.ValidationRequest) (contracts.ValidationResult, error) {
if len(req.Schema.JSONSchema) == 0 {
func (v *Validator) Validate(_ context.Context, req contracts.SerializedValidationRequest) (contracts.ValidationResult, error) {
return validate(req.Content, req.Schema.JSONSchema)
}
func (v *legacyValidator) Name() string { return Key }
func (v *legacyValidator) ExecutionClass() contracts.ExecutionClass {
return contracts.ExecutionClassDeterministic
}
func (v *legacyValidator) Validate(_ context.Context, req contracts.ValidationRequest) (contracts.ValidationResult, error) {
return validate(req.Payload.Content, req.Schema.JSONSchema)
}
func validate(content, schemaContent []byte) (contracts.ValidationResult, error) {
if len(schemaContent) == 0 {
return contracts.ValidationResult{}, fmt.Errorf("response schema content is not available")
}
instance, err := jsonschema.UnmarshalJSON(bytes.NewReader(req.Payload.Content))
instance, err := jsonschema.UnmarshalJSON(bytes.NewReader(content))
if err != nil {
return contracts.ValidationResult{
Approved: false,
ReasonCode: ReasonCodeInvalidJSON,
Message: "payload is not valid JSON",
}, nil
return contracts.ValidationResult{Approved: false, ReasonCode: ReasonCodeInvalidJSON, Message: "payload is not valid JSON"}, nil
}
schemaDocument, err := jsonschema.UnmarshalJSON(bytes.NewReader(req.Schema.JSONSchema))
schemaDocument, err := jsonschema.UnmarshalJSON(bytes.NewReader(schemaContent))
if err != nil {
return contracts.ValidationResult{}, fmt.Errorf("parse response schema: %w", err)
}
@@ -58,24 +61,39 @@ func (v *Validator) Validate(ctx context.Context, req contracts.ValidationReques
return contracts.ValidationResult{}, fmt.Errorf("compile response schema: %w", err)
}
if err := schema.Validate(instance); err != nil {
return contracts.ValidationResult{
Approved: false,
ReasonCode: ReasonCodeSchemaInvalid,
Message: "payload does not conform to response schema",
}, nil
return contracts.ValidationResult{Approved: false, ReasonCode: ReasonCodeSchemaInvalid, Message: "payload does not conform to response schema"}, nil
}
return contracts.ValidationResult{Approved: true}, nil
}
func Spec() pipeline.ValidatorSpec {
return pipeline.ValidatorSpec{
Key: Key,
ExecutionClass: contracts.ExecutionClassDeterministic,
}
return pipeline.ValidatorSpec{Key: Key, ExecutionClass: contracts.ExecutionClassDeterministic}
}
func Register(registry *pipeline.ValidatorRegistry) error {
return registry.RegisterLegacyRawWithSpec(Spec(), func() (contracts.LegacyRawValidator, error) {
return New(), nil
if err := pipeline.RegisterSerializedValidatorBuilder(registry, pipeline.SerializedValidatorSpec{
ValidatorSpec: Spec(), SupportsChunks: true, SupportsArtifacts: true,
}, validateOptions, func(request pipeline.BuildRequest) (contracts.SerializedValidator, error) {
options, err := DecodeOptions(request.Options)
if err != nil {
return nil, err
}
return New(options), nil
}); err != nil {
return err
}
return registry.RegisterLegacyRawBuilderWithSpec(Spec(), validateOptions, func(request pipeline.BuildRequest) (contracts.LegacyRawValidator, error) {
if _, err := DecodeOptions(request.Options); err != nil {
return nil, err
}
return &legacyValidator{}, nil
})
}
func DecodeOptions(options map[string]any) (Options, error) {
if err := pipeline.RejectUnknownOptions(options); err != nil {
return Options{}, err
}
return Options{}, nil
}
func validateOptions(options map[string]any) error { _, err := DecodeOptions(options); return err }

21
internal/modules/generic/validate/valid_json_schema/validator_test.go
View File

@@ -10,7 +10,7 @@ import (
)
func TestValidatorAcceptsSchemaConformantJSON(t *testing.T) {
result, err := New().Validate(context.Background(), requestWithSchema(`{"name":"Aria"}`, objectSchema()))
result, err := New(Options{}).Validate(context.Background(), requestWithSchema(`{"name":"Aria"}`, objectSchema()))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
@@ -20,7 +20,7 @@ func TestValidatorAcceptsSchemaConformantJSON(t *testing.T) {
}
func TestValidatorRejectsInvalidPayloadJSON(t *testing.T) {
result, err := New().Validate(context.Background(), requestWithSchema(`{"name":`, objectSchema()))
result, err := New(Options{}).Validate(context.Background(), requestWithSchema(`{"name":`, objectSchema()))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
@@ -33,7 +33,7 @@ func TestValidatorRejectsInvalidPayloadJSON(t *testing.T) {
}
func TestValidatorRejectsSchemaNonConformance(t *testing.T) {
result, err := New().Validate(context.Background(), requestWithSchema(`{"name":3}`, objectSchema()))
result, err := New(Options{}).Validate(context.Background(), requestWithSchema(`{"name":3}`, objectSchema()))
if err != nil {
t.Fatalf("Validate() error = %v, want nil", err)
}
@@ -46,7 +46,7 @@ func TestValidatorRejectsSchemaNonConformance(t *testing.T) {
}
func TestValidatorErrorsWhenSchemaContentMissing(t *testing.T) {
_, err := New().Validate(context.Background(), requestWithSchema(`{"name":"Aria"}`, nil))
_, err := New(Options{}).Validate(context.Background(), requestWithSchema(`{"name":"Aria"}`, nil))
if err == nil {
t.Fatal("Validate() error = nil, want missing schema content error")
}
@@ -56,7 +56,7 @@ func TestValidatorErrorsWhenSchemaContentMissing(t *testing.T) {
}
func TestValidatorErrorsWhenSchemaContentIsMalformed(t *testing.T) {
_, err := New().Validate(context.Background(), requestWithSchema(`{"name":"Aria"}`, []byte(`{"type":`)))
_, err := New(Options{}).Validate(context.Background(), requestWithSchema(`{"name":"Aria"}`, []byte(`{"type":`)))
if err == nil {
t.Fatal("Validate() error = nil, want malformed schema error")
}
@@ -83,18 +83,15 @@ func TestSpecAndRegister(t *testing.T) {
}
}
func requestWithSchema(payload string, schema []byte) contracts.ValidationRequest {
return contracts.ValidationRequest{
Schema: contracts.ResponseSchema{
func requestWithSchema(payload string, schema []byte) contracts.SerializedValidationRequest {
return contracts.SerializedValidationRequest{
Schema: contracts.ArtifactSchema{
ID: "test.schema",
Name: "test_schema",
Version: "v1",
JSONSchema: append([]byte(nil), schema...),
},
Payload: contracts.RawPayload{
Content: []byte(payload),
MediaType: "application/json",
},
Content: []byte(payload), MediaType: "application/json",
}
}