Add type-safe artifact lane resolution
This commit is contained in:
@@ -17,6 +17,10 @@ validator registry. Package-family registrars compose those leaf registrations
|
||||
into the production catalog and own family-level policy such as default
|
||||
validator chains and prompt asset collection.
|
||||
|
||||
Production extract, merge, normalize, and validator packages currently use the
|
||||
explicit legacy raw registration APIs. Typed registration is framework-ready,
|
||||
but no production artifact kind or codec is registered yet.
|
||||
|
||||
Specs expose capability and execution metadata without constructing an
|
||||
implementation. Chunk, extract, merge, and normalize modules that accept
|
||||
auxiliary material declare identical reference slots from both
|
||||
|
||||
@@ -46,15 +46,19 @@ a sorted set of artifact lanes before the runner constructs any stage module.
|
||||
| `internal/framework/checkpoint` | Workspace-backed checkpoint loading, recording, and payload serialization. |
|
||||
| `internal/framework/debug` | Workspace-backed framework and LLM debug recording. |
|
||||
|
||||
Framework contracts carry raw stage results between implementations. The
|
||||
runner owns handoff provenance, validation sequencing, rejection handling,
|
||||
checkpoint and debug boundaries, and final manifest assembly.
|
||||
Framework contracts provide both the production raw stage interfaces and typed
|
||||
artifact, provenance-wrapper, chunk-validator, serialized-validator, and
|
||||
typed-validator interfaces. The runner owns handoff provenance, validation
|
||||
sequencing, rejection handling, checkpoint and debug boundaries, and final
|
||||
manifest assembly.
|
||||
|
||||
The artifact codec registry is an implemented foundation for heterogeneous
|
||||
typed artifacts. It validates codec metadata and schema identity and keeps
|
||||
exact Go-type checks behind framework-owned erased operations. Production
|
||||
module families do not register codecs yet, so pipeline resolution and
|
||||
execution continue to use the existing raw artifact contracts.
|
||||
Artifact registries support heterogeneous typed extraction entries and
|
||||
kind-specific merger, normalizer, and validator variants. Resolution derives a
|
||||
lane's kind from its extractor, requires the matching codec, verifies exact Go
|
||||
type equality across the lane, and records schema identity in the resolved lane
|
||||
and pipeline digest. Production module families do not register typed variants
|
||||
yet and continue through explicitly named legacy raw registrations. The current
|
||||
runner rejects a typed resolved lane instead of routing it through raw execution.
|
||||
|
||||
## Production Extensions
|
||||
|
||||
|
||||
@@ -20,9 +20,12 @@ calls `pipeline.ResolvePipeline`.
|
||||
1. selects and sorts artifact lanes;
|
||||
2. completes omitted bindings using the documented configuration defaults;
|
||||
3. looks up each module and validator spec without constructing it;
|
||||
4. checks required and provided capabilities in workflow order;
|
||||
5. resolves target-aware reference bindings and validator chains;
|
||||
6. calculates a digest over the resolved structure.
|
||||
4. for a typed extractor, derives its artifact kind, requires the codec, and
|
||||
selects exact-type merger, normalizer, and validator variants;
|
||||
5. checks required and provided capabilities in workflow order;
|
||||
6. resolves target-aware reference bindings and validator chains;
|
||||
7. calculates a digest over the resolved structure, including typed artifact
|
||||
kind and schema identity.
|
||||
|
||||
Resolution returns a `ResolvedPipeline` containing ordered lanes, concrete
|
||||
bindings, validator chains, reference targets, and the digest. It does not read
|
||||
@@ -59,9 +62,16 @@ one codec per stable artifact kind, validates its schema metadata and JSON
|
||||
Schema, retains the exact schema digest and Go type, and safely encodes or
|
||||
decodes framework-erased values with typed errors on incompatibility.
|
||||
|
||||
Typed extractor entries are keyed by module key and declare one artifact kind.
|
||||
Merger, normalizer, and typed-validator variants are keyed by module or
|
||||
validator key plus artifact kind. Chunk and serialized validators occupy
|
||||
separate target namespaces; serialized registrations declare whether they
|
||||
support chunks, artifacts, or both. Duplicate variants and exact Go-type
|
||||
mismatches are rejected deterministically.
|
||||
|
||||
Production composition initializes the artifact codec registry without adding
|
||||
codec entries. The resolver and runner do not consult it yet, and production
|
||||
lanes remain on the raw artifact path.
|
||||
codec entries, and production modules use the explicitly named legacy raw
|
||||
registration APIs. A raw registration cannot satisfy a typed lane.
|
||||
|
||||
A `ModuleSpec` declares its stage plus required and provided capabilities.
|
||||
Chunk, extract, merge, and normalize specs may also declare reference slots.
|
||||
@@ -82,6 +92,10 @@ selected input adapter. Later stage requests receive the generic source model;
|
||||
extract requests receive chunk-scoped input material, while chunk, merge, and
|
||||
normalize requests retain access to the original source material.
|
||||
|
||||
Typed lanes can be composed and resolved but are not passed to the current raw
|
||||
runner. The runner rejects such input before source work; production lanes are
|
||||
still resolved and executed exclusively through the legacy raw path.
|
||||
|
||||
Source validation requires every unit to carry a canonical self-reference to
|
||||
its containing document and its own unit ID. Explicit clone, checkpoint, and
|
||||
debug boundaries retain that reference, and the canonical source digest covers
|
||||
@@ -193,6 +207,9 @@ durable manifest and logical file schemas are defined in the
|
||||
capabilities, validator chains, and digest behavior.
|
||||
- `internal/framework/pipeline/artifact_codec_registry_test.go`: typed codec
|
||||
metadata, registration, erasure safety, strict decoding, and cloning.
|
||||
- `internal/framework/pipeline/typed_resolution_test.go`: heterogeneous typed
|
||||
lane resolution, target-specific validators, incompatibilities, ordering, and
|
||||
schema-sensitive pipeline identity.
|
||||
- `internal/framework/pipeline/references_test.go`: target resolution and
|
||||
materialization.
|
||||
- `internal/framework/pipeline/runner_test.go`: stage transitions, retries,
|
||||
|
||||
@@ -73,6 +73,12 @@ Extract modules own artifact semantics, prompt use, response schemas, and
|
||||
domain interpretation. Domain-specific concepts remain in the relevant module,
|
||||
validator, shared domain helper, and artifact contract.
|
||||
|
||||
Typed artifact registrations declare one stable artifact kind and exact Go
|
||||
type from extraction through merge, normalization, and semantic validation.
|
||||
Pipeline resolution requires a compatible codec and matching kind-specific
|
||||
variants before a typed lane can be accepted. Framework-owned erasure remains
|
||||
private and must report type incompatibility as an error rather than a panic.
|
||||
|
||||
Auxiliary references provide context or disambiguation. They are not source
|
||||
evidence and must not be converted into source references.
|
||||
|
||||
@@ -104,6 +110,11 @@ extract, merge, and normalize stages. Validators receive immutable stage output
|
||||
and make an explicit whole-output decision: approve, approve with warnings, or
|
||||
reject.
|
||||
|
||||
Typed artifact validators receive the domain value directly. Chunk validators
|
||||
receive source-zone chunks, while serialized validators receive immutable
|
||||
representation bytes and declared schema metadata. A validator registered for
|
||||
one target or artifact kind cannot satisfy an incompatible selection.
|
||||
|
||||
Rejection is a recorded pipeline outcome, not a framework execution error.
|
||||
Validator execution failures are framework errors. Rejected output does not
|
||||
advance to the next stage.
|
||||
|
||||
@@ -3567,7 +3567,7 @@ func fakeExecutionRegistries(t *testing.T) pipeline.Registries {
|
||||
}); err != nil {
|
||||
t.Fatalf("register fake chunker: %v", err)
|
||||
}
|
||||
if err := extractors.RegisterWithSpec(pipeline.ModuleSpec{
|
||||
if err := extractors.RegisterLegacyRawWithSpec(pipeline.ModuleSpec{
|
||||
Key: "fake/extract",
|
||||
Stage: pipeline.StageExtract,
|
||||
Requires: []string{"chunks"},
|
||||
@@ -3575,17 +3575,17 @@ func fakeExecutionRegistries(t *testing.T) pipeline.Registries {
|
||||
ReferenceSlots: []contracts.ReferenceSlot{
|
||||
{Name: "roster"},
|
||||
},
|
||||
}, func() (contracts.Extractor, error) {
|
||||
}, func() (contracts.LegacyRawExtractor, error) {
|
||||
return fakeRunExtractor{}, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register fake extractor: %v", err)
|
||||
}
|
||||
if err := mergers.RegisterWithSpec(pipeline.ModuleSpec{Key: "appendorder", Stage: pipeline.StageMerge, Requires: []string{"artifact"}, Provides: []string{"merged"}}, func() (contracts.Merger, error) {
|
||||
if err := mergers.RegisterLegacyRawWithSpec(pipeline.ModuleSpec{Key: "appendorder", Stage: pipeline.StageMerge, Requires: []string{"artifact"}, Provides: []string{"merged"}}, func() (contracts.LegacyRawMerger, error) {
|
||||
return fakeRunMerger{}, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register fake merger: %v", err)
|
||||
}
|
||||
if err := normalizers.RegisterWithSpec(pipeline.ModuleSpec{Key: "noop", Stage: pipeline.StageNormalize, Requires: []string{"merged"}, Provides: []string{"normalized"}}, func() (contracts.Normalizer, error) {
|
||||
if err := normalizers.RegisterLegacyRawWithSpec(pipeline.ModuleSpec{Key: "noop", Stage: pipeline.StageNormalize, Requires: []string{"merged"}, Provides: []string{"normalized"}}, func() (contracts.LegacyRawNormalizer, error) {
|
||||
return fakeRunNormalizer{}, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register fake normalizer: %v", err)
|
||||
@@ -4003,21 +4003,21 @@ func mustRegisterChunker(t *testing.T, registry *pipeline.ChunkerRegistry, spec
|
||||
|
||||
func mustRegisterExtractor(t *testing.T, registry *pipeline.ExtractorRegistry, spec pipeline.ModuleSpec) {
|
||||
t.Helper()
|
||||
if err := registry.RegisterWithSpec(spec, func() (contracts.Extractor, error) { return nil, nil }); err != nil {
|
||||
if err := registry.RegisterLegacyRawWithSpec(spec, func() (contracts.LegacyRawExtractor, error) { return nil, nil }); err != nil {
|
||||
t.Fatalf("register extractor: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func mustRegisterMerger(t *testing.T, registry *pipeline.MergerRegistry, spec pipeline.ModuleSpec) {
|
||||
t.Helper()
|
||||
if err := registry.RegisterWithSpec(spec, func() (contracts.Merger, error) { return nil, nil }); err != nil {
|
||||
if err := registry.RegisterLegacyRawWithSpec(spec, func() (contracts.LegacyRawMerger, error) { return nil, nil }); err != nil {
|
||||
t.Fatalf("register merger: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func mustRegisterNormalizer(t *testing.T, registry *pipeline.NormalizerRegistry, spec pipeline.ModuleSpec) {
|
||||
t.Helper()
|
||||
if err := registry.RegisterWithSpec(spec, func() (contracts.Normalizer, error) { return nil, nil }); err != nil {
|
||||
if err := registry.RegisterLegacyRawWithSpec(spec, func() (contracts.LegacyRawNormalizer, error) { return nil, nil }); err != nil {
|
||||
t.Fatalf("register normalizer: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -4031,7 +4031,7 @@ func mustRegisterOutput(t *testing.T, registry *pipeline.OutputEncoderRegistry,
|
||||
|
||||
func mustRegisterValidator(t *testing.T, registry *pipeline.ValidatorRegistry, spec pipeline.ValidatorSpec) {
|
||||
t.Helper()
|
||||
if err := registry.RegisterWithSpec(spec, func() (contracts.Validator, error) {
|
||||
if err := registry.RegisterLegacyRawWithSpec(spec, func() (contracts.LegacyRawValidator, error) {
|
||||
return fakeConfigValidator{name: spec.Key, executionClass: spec.ExecutionClass}, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register validator: %v", err)
|
||||
|
||||
@@ -50,6 +50,8 @@ func cloneResolvedValidatorChain(in pipeline.ResolvedValidatorChain) pipeline.Re
|
||||
out.Validators[i] = pipeline.ResolvedValidator{
|
||||
Binding: cloneModuleBinding(validator.Binding),
|
||||
ExecutionClass: validator.ExecutionClass,
|
||||
Target: validator.Target,
|
||||
ArtifactKind: validator.ArtifactKind,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -544,21 +544,21 @@ func mustRegisterChunker(t *testing.T, registry *pipeline.ChunkerRegistry, spec
|
||||
|
||||
func mustRegisterExtractor(t *testing.T, registry *pipeline.ExtractorRegistry, spec pipeline.ModuleSpec) {
|
||||
t.Helper()
|
||||
if err := registry.RegisterWithSpec(spec, func() (contracts.Extractor, error) { return nil, nil }); err != nil {
|
||||
if err := registry.RegisterLegacyRawWithSpec(spec, func() (contracts.LegacyRawExtractor, error) { return nil, nil }); err != nil {
|
||||
t.Fatalf("register extractor: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func mustRegisterMerger(t *testing.T, registry *pipeline.MergerRegistry, spec pipeline.ModuleSpec) {
|
||||
t.Helper()
|
||||
if err := registry.RegisterWithSpec(spec, func() (contracts.Merger, error) { return nil, nil }); err != nil {
|
||||
if err := registry.RegisterLegacyRawWithSpec(spec, func() (contracts.LegacyRawMerger, error) { return nil, nil }); err != nil {
|
||||
t.Fatalf("register merger: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func mustRegisterNormalizer(t *testing.T, registry *pipeline.NormalizerRegistry, spec pipeline.ModuleSpec) {
|
||||
t.Helper()
|
||||
if err := registry.RegisterWithSpec(spec, func() (contracts.Normalizer, error) { return nil, nil }); err != nil {
|
||||
if err := registry.RegisterLegacyRawWithSpec(spec, func() (contracts.LegacyRawNormalizer, error) { return nil, nil }); err != nil {
|
||||
t.Fatalf("register normalizer: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -570,7 +570,7 @@ func mustRegisterValidator(t *testing.T, registry *pipeline.ValidatorRegistry, s
|
||||
executionClass = contracts.ExecutionClassLLMBacked
|
||||
}
|
||||
validatorSpec := pipeline.ValidatorSpec{Key: spec.Key, ExecutionClass: executionClass}
|
||||
if err := registry.RegisterWithSpec(validatorSpec, func() (contracts.Validator, error) { return nil, nil }); err != nil {
|
||||
if err := registry.RegisterLegacyRawWithSpec(validatorSpec, func() (contracts.LegacyRawValidator, error) { return nil, nil }); err != nil {
|
||||
t.Fatalf("register validator: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -13,10 +13,10 @@ import (
|
||||
|
||||
var _ contracts.InputAdapter = compositionAdapter{}
|
||||
var _ contracts.Chunker = compositionChunker{}
|
||||
var _ contracts.Extractor = compositionExtractor{}
|
||||
var _ contracts.Merger = compositionMerger{}
|
||||
var _ contracts.Normalizer = compositionNormalizer{}
|
||||
var _ contracts.Validator = compositionValidator{}
|
||||
var _ contracts.LegacyRawExtractor = compositionExtractor{}
|
||||
var _ contracts.LegacyRawMerger = compositionMerger{}
|
||||
var _ contracts.LegacyRawNormalizer = compositionNormalizer{}
|
||||
var _ contracts.LegacyRawValidator = compositionValidator{}
|
||||
var _ contracts.StructuredLLMClient = compositionLLMClient{}
|
||||
var _ contracts.OutputEncoder = compositionOutputEncoder{}
|
||||
|
||||
|
||||
@@ -228,7 +228,7 @@ type ExtractionResult struct {
|
||||
Warnings []Warning `json:"warnings,omitempty"`
|
||||
}
|
||||
|
||||
type Extractor interface {
|
||||
type LegacyRawExtractor interface {
|
||||
Key() string
|
||||
ReferenceSlots() []ReferenceSlot
|
||||
Extract(ctx context.Context, req ExtractionRequest) (ExtractionResult, error)
|
||||
@@ -279,7 +279,7 @@ type ValidationResult struct {
|
||||
Warnings []Warning `json:"warnings,omitempty"`
|
||||
}
|
||||
|
||||
type Validator interface {
|
||||
type LegacyRawValidator interface {
|
||||
Name() string
|
||||
ExecutionClass() ExecutionClass
|
||||
Validate(ctx context.Context, req ValidationRequest) (ValidationResult, error)
|
||||
@@ -328,7 +328,7 @@ type MergeOutput struct {
|
||||
Payload RawPayload `json:"payload"`
|
||||
}
|
||||
|
||||
type Merger interface {
|
||||
type LegacyRawMerger interface {
|
||||
Key() string
|
||||
Merge(ctx context.Context, req MergeRequest) (MergeResult, error)
|
||||
}
|
||||
@@ -359,7 +359,7 @@ type NormalizeOutput struct {
|
||||
Payload RawPayload `json:"payload"`
|
||||
}
|
||||
|
||||
type Normalizer interface {
|
||||
type LegacyRawNormalizer interface {
|
||||
Key() string
|
||||
ReferenceSlots() []ReferenceSlot
|
||||
Normalize(ctx context.Context, req NormalizeRequest) (NormalizeResult, error)
|
||||
|
||||
@@ -12,10 +12,10 @@ import (
|
||||
|
||||
var _ InputAdapter = fakeAdapter{}
|
||||
var _ Chunker = fakeChunker{}
|
||||
var _ Extractor = fakeExtractor{}
|
||||
var _ Merger = fakeMerger{}
|
||||
var _ Normalizer = fakeNormalizer{}
|
||||
var _ Validator = fakeValidator{}
|
||||
var _ LegacyRawExtractor = fakeExtractor{}
|
||||
var _ LegacyRawMerger = fakeMerger{}
|
||||
var _ LegacyRawNormalizer = fakeNormalizer{}
|
||||
var _ LegacyRawValidator = fakeValidator{}
|
||||
var _ StructuredLLMClient = fakeLLMClient{}
|
||||
var _ OutputEncoder = fakeOutputEncoder{}
|
||||
|
||||
|
||||
164
internal/framework/contracts/typed_pipeline.go
Normal file
164
internal/framework/contracts/typed_pipeline.go
Normal file
@@ -0,0 +1,164 @@
|
||||
package contracts
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
|
||||
)
|
||||
|
||||
// ExtractArtifact carries a typed per-chunk value with framework provenance.
|
||||
type ExtractArtifact[T any] struct {
|
||||
LaneID string
|
||||
ExtractorKey string
|
||||
SourceID string
|
||||
ChunkID string
|
||||
ChunkIndex int
|
||||
ChunkRef source.SourceRef
|
||||
Value T
|
||||
}
|
||||
|
||||
// MergeArtifact carries a typed merged value with framework provenance.
|
||||
type MergeArtifact[T any] struct {
|
||||
LaneID string
|
||||
MergerKey string
|
||||
SourceID string
|
||||
Value T
|
||||
}
|
||||
|
||||
// NormalizeArtifact carries a typed normalized value with framework provenance.
|
||||
type NormalizeArtifact[T any] struct {
|
||||
LaneID string
|
||||
NormalizerKey string
|
||||
SourceID string
|
||||
Value T
|
||||
}
|
||||
|
||||
type TypedExtractionRequest struct {
|
||||
Source *source.SourceDocument
|
||||
Chunk *source.Chunk
|
||||
AmbientContext map[string]any
|
||||
SourceInput LLMInputMaterial
|
||||
SessionID string
|
||||
References ReferenceSet
|
||||
LLMProfile string
|
||||
Metadata map[string]any
|
||||
}
|
||||
|
||||
type TypedExtractionResult[T any] struct {
|
||||
Value T
|
||||
Warnings []Warning
|
||||
}
|
||||
|
||||
type Extractor[T any] interface {
|
||||
Key() string
|
||||
ReferenceSlots() []ReferenceSlot
|
||||
Extract(context.Context, TypedExtractionRequest) (TypedExtractionResult[T], error)
|
||||
}
|
||||
|
||||
type TypedMergeRequest[T any] struct {
|
||||
Source *source.SourceDocument
|
||||
LaneID string
|
||||
ExtractOutputs []ExtractArtifact[T]
|
||||
SourceInput LLMInputMaterial
|
||||
SessionID string
|
||||
References ReferenceSet
|
||||
LLMProfile string
|
||||
Metadata map[string]any
|
||||
}
|
||||
|
||||
type TypedMergeResult[T any] struct {
|
||||
Value T
|
||||
Warnings []Warning
|
||||
}
|
||||
|
||||
type Merger[T any] interface {
|
||||
Key() string
|
||||
Merge(context.Context, TypedMergeRequest[T]) (TypedMergeResult[T], error)
|
||||
}
|
||||
|
||||
type TypedNormalizeRequest[T any] struct {
|
||||
Source *source.SourceDocument
|
||||
LaneID string
|
||||
MergeOutput MergeArtifact[T]
|
||||
SourceInput LLMInputMaterial
|
||||
SessionID string
|
||||
References ReferenceSet
|
||||
LLMProfile string
|
||||
Metadata map[string]any
|
||||
}
|
||||
|
||||
type TypedNormalizeResult[T any] struct {
|
||||
Value T
|
||||
Warnings []Warning
|
||||
}
|
||||
|
||||
type Normalizer[T any] interface {
|
||||
Key() string
|
||||
ReferenceSlots() []ReferenceSlot
|
||||
Normalize(context.Context, TypedNormalizeRequest[T]) (TypedNormalizeResult[T], error)
|
||||
}
|
||||
|
||||
type TypedValidationRequest[T any] struct {
|
||||
Stage string
|
||||
LaneID string
|
||||
ModuleKey string
|
||||
Source *source.SourceDocument
|
||||
SourceID string
|
||||
SourceInput LLMInputMaterial
|
||||
SessionID string
|
||||
References ReferenceSet
|
||||
LLMProfile string
|
||||
Metadata map[string]any
|
||||
Chunk *source.Chunk
|
||||
Chunks []source.Chunk
|
||||
Ref source.SourceRef
|
||||
Value T
|
||||
}
|
||||
|
||||
type TypedValidator[T any] interface {
|
||||
Name() string
|
||||
ExecutionClass() ExecutionClass
|
||||
Validate(context.Context, TypedValidationRequest[T]) (ValidationResult, error)
|
||||
}
|
||||
|
||||
type ChunkValidationRequest struct {
|
||||
ModuleKey string
|
||||
Source *source.SourceDocument
|
||||
SourceID string
|
||||
SourceInput LLMInputMaterial
|
||||
SessionID string
|
||||
References ReferenceSet
|
||||
LLMProfile string
|
||||
Metadata map[string]any
|
||||
Chunks []source.Chunk
|
||||
}
|
||||
|
||||
type ChunkValidator interface {
|
||||
Name() string
|
||||
ExecutionClass() ExecutionClass
|
||||
Validate(context.Context, ChunkValidationRequest) (ValidationResult, error)
|
||||
}
|
||||
|
||||
type SerializedValidationRequest struct {
|
||||
Stage string
|
||||
LaneID string
|
||||
ModuleKey string
|
||||
Source *source.SourceDocument
|
||||
SourceID string
|
||||
SourceInput LLMInputMaterial
|
||||
SessionID string
|
||||
References ReferenceSet
|
||||
LLMProfile string
|
||||
Metadata map[string]any
|
||||
Chunk *source.Chunk
|
||||
Chunks []source.Chunk
|
||||
Schema ArtifactSchema
|
||||
MediaType string
|
||||
Content []byte
|
||||
}
|
||||
|
||||
type SerializedValidator interface {
|
||||
Name() string
|
||||
ExecutionClass() ExecutionClass
|
||||
Validate(context.Context, SerializedValidationRequest) (ValidationResult, error)
|
||||
}
|
||||
@@ -136,6 +136,17 @@ func (r *ArtifactCodecRegistry) Spec(kind contracts.ArtifactKind) (ArtifactCodec
|
||||
return cloneArtifactCodecSpec(entry.spec), true
|
||||
}
|
||||
|
||||
func (r *ArtifactCodecRegistry) valueType(kind contracts.ArtifactKind) (reflect.Type, bool) {
|
||||
if r == nil {
|
||||
return nil, false
|
||||
}
|
||||
entry, ok := r.entries[normalizeArtifactKind(kind)]
|
||||
if !ok {
|
||||
return nil, false
|
||||
}
|
||||
return entry.valueType, true
|
||||
}
|
||||
|
||||
func (r *ArtifactCodecRegistry) RegisteredKinds() []contracts.ArtifactKind {
|
||||
if r == nil || len(r.entries) == 0 {
|
||||
return nil
|
||||
|
||||
@@ -71,12 +71,12 @@ func defaultModuleCatalog(t *testing.T) pipeline.ModuleCatalog {
|
||||
if err := units.Register(chunkers); err != nil {
|
||||
t.Fatalf("register generic chunker: %v", err)
|
||||
}
|
||||
if err := extractors.RegisterWithSpec(pipeline.ModuleSpec{
|
||||
if err := extractors.RegisterLegacyRawWithSpec(pipeline.ModuleSpec{
|
||||
Key: "extract",
|
||||
Stage: pipeline.StageExtract,
|
||||
Requires: []string{"chunks"},
|
||||
Provides: []string{"records"},
|
||||
}, func() (contracts.Extractor, error) {
|
||||
}, func() (contracts.LegacyRawExtractor, error) {
|
||||
return defaultExtractor{}, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register extractor: %v", err)
|
||||
|
||||
@@ -2,71 +2,113 @@ package pipeline
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"reflect"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
)
|
||||
|
||||
type ExtractorConstructor func() (contracts.Extractor, error)
|
||||
type LegacyRawExtractorConstructor func() (contracts.LegacyRawExtractor, error)
|
||||
|
||||
type ExtractorRegistry struct {
|
||||
constructors map[string]ExtractorConstructor
|
||||
specs map[string]ModuleSpec
|
||||
legacyConstructors map[string]LegacyRawExtractorConstructor
|
||||
typedEntries map[string]typedExtractorEntry
|
||||
specs map[string]ModuleSpec
|
||||
}
|
||||
|
||||
type typedExtractorEntry struct {
|
||||
spec ModuleSpec
|
||||
valueType reflect.Type
|
||||
constructor func() (any, error)
|
||||
}
|
||||
|
||||
func NewExtractorRegistry() *ExtractorRegistry {
|
||||
return &ExtractorRegistry{
|
||||
constructors: make(map[string]ExtractorConstructor),
|
||||
specs: make(map[string]ModuleSpec),
|
||||
legacyConstructors: make(map[string]LegacyRawExtractorConstructor),
|
||||
typedEntries: make(map[string]typedExtractorEntry),
|
||||
specs: make(map[string]ModuleSpec),
|
||||
}
|
||||
}
|
||||
|
||||
func (r *ExtractorRegistry) Register(key string, constructor ExtractorConstructor) error {
|
||||
return r.RegisterWithSpec(defaultModuleSpec(key, StageExtract), constructor)
|
||||
func (r *ExtractorRegistry) RegisterLegacyRaw(key string, constructor LegacyRawExtractorConstructor) error {
|
||||
return r.RegisterLegacyRawWithSpec(defaultModuleSpec(key, StageExtract), constructor)
|
||||
}
|
||||
|
||||
func (r *ExtractorRegistry) RegisterWithSpec(spec ModuleSpec, constructor ExtractorConstructor) error {
|
||||
func (r *ExtractorRegistry) RegisterLegacyRawWithSpec(spec ModuleSpec, constructor LegacyRawExtractorConstructor) error {
|
||||
if r == nil {
|
||||
return fmt.Errorf("extractor registry must not be nil")
|
||||
}
|
||||
|
||||
normalizedSpec := normalizeModuleSpec(spec)
|
||||
if err := validateModuleSpec("extractor", StageExtract, normalizedSpec); err != nil {
|
||||
return err
|
||||
}
|
||||
if normalizedSpec.ArtifactKind != "" {
|
||||
return fmt.Errorf("legacy raw extractor %q must not declare an artifact kind", normalizedSpec.Key)
|
||||
}
|
||||
if constructor == nil {
|
||||
return fmt.Errorf("extractor constructor for %q must not be nil", normalizedSpec.Key)
|
||||
}
|
||||
if _, ok := r.constructors[normalizedSpec.Key]; ok {
|
||||
if _, ok := r.specs[normalizedSpec.Key]; ok {
|
||||
return fmt.Errorf("extractor %q is already registered", normalizedSpec.Key)
|
||||
}
|
||||
|
||||
if r.constructors == nil {
|
||||
r.constructors = make(map[string]ExtractorConstructor)
|
||||
if r.legacyConstructors == nil {
|
||||
r.legacyConstructors = make(map[string]LegacyRawExtractorConstructor)
|
||||
}
|
||||
if r.specs == nil {
|
||||
r.specs = make(map[string]ModuleSpec)
|
||||
}
|
||||
r.constructors[normalizedSpec.Key] = constructor
|
||||
r.legacyConstructors[normalizedSpec.Key] = constructor
|
||||
r.specs[normalizedSpec.Key] = cloneModuleSpec(normalizedSpec)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *ExtractorRegistry) Build(key string) (contracts.Extractor, error) {
|
||||
func RegisterExtractor[T any](registry *ExtractorRegistry, spec ModuleSpec, constructor func() (contracts.Extractor[T], error)) error {
|
||||
if registry == nil {
|
||||
return fmt.Errorf("extractor registry must not be nil")
|
||||
}
|
||||
normalizedSpec := normalizeModuleSpec(spec)
|
||||
if err := validateModuleSpec("extractor", StageExtract, normalizedSpec); err != nil {
|
||||
return err
|
||||
}
|
||||
if normalizedSpec.ArtifactKind == "" {
|
||||
return fmt.Errorf("typed extractor %q artifact kind must not be empty", normalizedSpec.Key)
|
||||
}
|
||||
if constructor == nil {
|
||||
return fmt.Errorf("extractor constructor for %q must not be nil", normalizedSpec.Key)
|
||||
}
|
||||
if _, ok := registry.specs[normalizedSpec.Key]; ok {
|
||||
return fmt.Errorf("extractor %q is already registered", normalizedSpec.Key)
|
||||
}
|
||||
entry := typedExtractorEntry{
|
||||
spec: cloneModuleSpec(normalizedSpec),
|
||||
valueType: reflect.TypeFor[T](),
|
||||
constructor: func() (any, error) {
|
||||
return constructor()
|
||||
},
|
||||
}
|
||||
if registry.typedEntries == nil {
|
||||
registry.typedEntries = make(map[string]typedExtractorEntry)
|
||||
}
|
||||
if registry.specs == nil {
|
||||
registry.specs = make(map[string]ModuleSpec)
|
||||
}
|
||||
registry.typedEntries[normalizedSpec.Key] = entry
|
||||
registry.specs[normalizedSpec.Key] = cloneModuleSpec(normalizedSpec)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *ExtractorRegistry) BuildLegacyRaw(key string) (contracts.LegacyRawExtractor, error) {
|
||||
if r == nil {
|
||||
return nil, fmt.Errorf("extractor registry must not be nil")
|
||||
}
|
||||
|
||||
normalizedKey := strings.TrimSpace(key)
|
||||
if normalizedKey == "" {
|
||||
return nil, fmt.Errorf("extractor key must not be empty")
|
||||
}
|
||||
|
||||
constructor, ok := r.constructors[normalizedKey]
|
||||
constructor, ok := r.legacyConstructors[normalizedKey]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("extractor %q is not registered", normalizedKey)
|
||||
return nil, fmt.Errorf("legacy raw extractor %q is not registered", normalizedKey)
|
||||
}
|
||||
|
||||
extractor, err := constructor()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("build extractor %q: %w", normalizedKey, err)
|
||||
@@ -77,7 +119,6 @@ func (r *ExtractorRegistry) Build(key string) (contracts.Extractor, error) {
|
||||
if extractor.Key() != normalizedKey {
|
||||
return nil, fmt.Errorf("extractor %q returned key %q", normalizedKey, extractor.Key())
|
||||
}
|
||||
|
||||
return extractor, nil
|
||||
}
|
||||
|
||||
@@ -85,7 +126,6 @@ func (r *ExtractorRegistry) Spec(key string) (ModuleSpec, bool) {
|
||||
if r == nil {
|
||||
return ModuleSpec{}, false
|
||||
}
|
||||
|
||||
spec, ok := r.specs[strings.TrimSpace(key)]
|
||||
if !ok {
|
||||
return ModuleSpec{}, false
|
||||
@@ -93,10 +133,17 @@ func (r *ExtractorRegistry) Spec(key string) (ModuleSpec, bool) {
|
||||
return cloneModuleSpec(spec), true
|
||||
}
|
||||
|
||||
func (r *ExtractorRegistry) typedEntry(key string) (typedExtractorEntry, bool) {
|
||||
if r == nil {
|
||||
return typedExtractorEntry{}, false
|
||||
}
|
||||
entry, ok := r.typedEntries[strings.TrimSpace(key)]
|
||||
return entry, ok
|
||||
}
|
||||
|
||||
func (r *ExtractorRegistry) RegisteredKeys() []string {
|
||||
if r == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
return sortedRegistryKeys(r.constructors)
|
||||
return sortedRegistryKeys(r.specs)
|
||||
}
|
||||
|
||||
@@ -13,11 +13,11 @@ import (
|
||||
func TestExtractorRegistryRegisterAndBuild(t *testing.T) {
|
||||
registry := NewExtractorRegistry()
|
||||
|
||||
if err := registry.Register("generic-extractor", fakeExtractorConstructor("generic-extractor")); err != nil {
|
||||
if err := registry.RegisterLegacyRaw("generic-extractor", fakeExtractorConstructor("generic-extractor")); err != nil {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
|
||||
extractor, err := registry.Build("generic-extractor")
|
||||
extractor, err := registry.BuildLegacyRaw("generic-extractor")
|
||||
if err != nil {
|
||||
t.Fatalf("Build() error = %v, want nil", err)
|
||||
}
|
||||
@@ -29,11 +29,11 @@ func TestExtractorRegistryRegisterAndBuild(t *testing.T) {
|
||||
func TestExtractorRegistryRegisterAndBuildTrimKeys(t *testing.T) {
|
||||
registry := NewExtractorRegistry()
|
||||
|
||||
if err := registry.Register(" generic-extractor ", fakeExtractorConstructor("generic-extractor")); err != nil {
|
||||
if err := registry.RegisterLegacyRaw(" generic-extractor ", fakeExtractorConstructor("generic-extractor")); err != nil {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
|
||||
extractor, err := registry.Build("\tgeneric-extractor\n")
|
||||
extractor, err := registry.BuildLegacyRaw("\tgeneric-extractor\n")
|
||||
if err != nil {
|
||||
t.Fatalf("Build() error = %v, want nil", err)
|
||||
}
|
||||
@@ -65,7 +65,7 @@ func TestExtractorRegistryRegisterWithSpecStoresMetadata(t *testing.T) {
|
||||
},
|
||||
}
|
||||
|
||||
if err := registry.RegisterWithSpec(spec, fakeExtractorConstructor("generic-extractor")); err != nil {
|
||||
if err := registry.RegisterLegacyRawWithSpec(spec, fakeExtractorConstructor("generic-extractor")); err != nil {
|
||||
t.Fatalf("RegisterWithSpec() error = %v, want nil", err)
|
||||
}
|
||||
|
||||
@@ -112,7 +112,7 @@ func TestExtractorRegistryRegisterWithSpecStoresMetadata(t *testing.T) {
|
||||
func TestExtractorRegistryRegisterStoresDefaultSpec(t *testing.T) {
|
||||
registry := NewExtractorRegistry()
|
||||
|
||||
if err := registry.Register(" generic-extractor ", fakeExtractorConstructor("generic-extractor")); err != nil {
|
||||
if err := registry.RegisterLegacyRaw(" generic-extractor ", fakeExtractorConstructor("generic-extractor")); err != nil {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
|
||||
@@ -129,7 +129,7 @@ func TestExtractorRegistryRegisterStoresDefaultSpec(t *testing.T) {
|
||||
func TestExtractorRegistryRegisterWithSpecRejectsWrongStage(t *testing.T) {
|
||||
registry := NewExtractorRegistry()
|
||||
|
||||
err := registry.RegisterWithSpec(ModuleSpec{Key: "generic-extractor", Stage: StageInput}, fakeExtractorConstructor("generic-extractor"))
|
||||
err := registry.RegisterLegacyRawWithSpec(ModuleSpec{Key: "generic-extractor", Stage: StageInput}, fakeExtractorConstructor("generic-extractor"))
|
||||
|
||||
if err == nil {
|
||||
t.Fatal("RegisterWithSpec() error = nil, want error")
|
||||
@@ -168,7 +168,7 @@ func TestExtractorRegistryRejectsInvalidReferenceSlots(t *testing.T) {
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
registry := NewExtractorRegistry()
|
||||
err := registry.RegisterWithSpec(ModuleSpec{
|
||||
err := registry.RegisterLegacyRawWithSpec(ModuleSpec{
|
||||
Key: "generic-extractor",
|
||||
Stage: StageExtract,
|
||||
ReferenceSlots: test.slots,
|
||||
@@ -194,7 +194,7 @@ func TestExtractorRegistrySpecRejectsUnknownKey(t *testing.T) {
|
||||
func TestExtractorRegistryRegisterRejectsEmptyKey(t *testing.T) {
|
||||
registry := NewExtractorRegistry()
|
||||
|
||||
err := registry.Register(" \t", fakeExtractorConstructor("generic-extractor"))
|
||||
err := registry.RegisterLegacyRaw(" \t", fakeExtractorConstructor("generic-extractor"))
|
||||
|
||||
if err == nil {
|
||||
t.Fatal("Register() error = nil, want error")
|
||||
@@ -206,11 +206,11 @@ func TestExtractorRegistryRegisterRejectsEmptyKey(t *testing.T) {
|
||||
|
||||
func TestExtractorRegistryRegisterRejectsDuplicateKey(t *testing.T) {
|
||||
registry := NewExtractorRegistry()
|
||||
if err := registry.Register("generic-extractor", fakeExtractorConstructor("generic-extractor")); err != nil {
|
||||
if err := registry.RegisterLegacyRaw("generic-extractor", fakeExtractorConstructor("generic-extractor")); err != nil {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
|
||||
err := registry.Register(" generic-extractor ", fakeExtractorConstructor("generic-extractor"))
|
||||
err := registry.RegisterLegacyRaw(" generic-extractor ", fakeExtractorConstructor("generic-extractor"))
|
||||
|
||||
if err == nil {
|
||||
t.Fatal("Register() error = nil, want error")
|
||||
@@ -223,7 +223,7 @@ func TestExtractorRegistryRegisterRejectsDuplicateKey(t *testing.T) {
|
||||
func TestExtractorRegistryRegisterRejectsNilConstructor(t *testing.T) {
|
||||
registry := NewExtractorRegistry()
|
||||
|
||||
err := registry.Register("generic-extractor", nil)
|
||||
err := registry.RegisterLegacyRaw("generic-extractor", nil)
|
||||
|
||||
if err == nil {
|
||||
t.Fatal("Register() error = nil, want error")
|
||||
@@ -236,7 +236,7 @@ func TestExtractorRegistryRegisterRejectsNilConstructor(t *testing.T) {
|
||||
func TestExtractorRegistryBuildRejectsUnknownKey(t *testing.T) {
|
||||
registry := NewExtractorRegistry()
|
||||
|
||||
_, err := registry.Build("missing-extractor")
|
||||
_, err := registry.BuildLegacyRaw("missing-extractor")
|
||||
|
||||
if err == nil {
|
||||
t.Fatal("Build() error = nil, want error")
|
||||
@@ -249,13 +249,13 @@ func TestExtractorRegistryBuildRejectsUnknownKey(t *testing.T) {
|
||||
func TestExtractorRegistryBuildWrapsConstructorError(t *testing.T) {
|
||||
registry := NewExtractorRegistry()
|
||||
constructorErr := errors.New("constructor failed")
|
||||
if err := registry.Register("generic-extractor", func() (contracts.Extractor, error) {
|
||||
if err := registry.RegisterLegacyRaw("generic-extractor", func() (contracts.LegacyRawExtractor, error) {
|
||||
return nil, constructorErr
|
||||
}); err != nil {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
|
||||
_, err := registry.Build("generic-extractor")
|
||||
_, err := registry.BuildLegacyRaw("generic-extractor")
|
||||
|
||||
if err == nil {
|
||||
t.Fatal("Build() error = nil, want error")
|
||||
@@ -270,13 +270,13 @@ func TestExtractorRegistryBuildWrapsConstructorError(t *testing.T) {
|
||||
|
||||
func TestExtractorRegistryBuildRejectsNilExtractor(t *testing.T) {
|
||||
registry := NewExtractorRegistry()
|
||||
if err := registry.Register("generic-extractor", func() (contracts.Extractor, error) {
|
||||
if err := registry.RegisterLegacyRaw("generic-extractor", func() (contracts.LegacyRawExtractor, error) {
|
||||
return nil, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
|
||||
_, err := registry.Build("generic-extractor")
|
||||
_, err := registry.BuildLegacyRaw("generic-extractor")
|
||||
|
||||
if err == nil {
|
||||
t.Fatal("Build() error = nil, want error")
|
||||
@@ -288,11 +288,11 @@ func TestExtractorRegistryBuildRejectsNilExtractor(t *testing.T) {
|
||||
|
||||
func TestExtractorRegistryBuildRejectsExtractorKeyMismatch(t *testing.T) {
|
||||
registry := NewExtractorRegistry()
|
||||
if err := registry.Register("generic-extractor", fakeExtractorConstructor("other-extractor")); err != nil {
|
||||
if err := registry.RegisterLegacyRaw("generic-extractor", fakeExtractorConstructor("other-extractor")); err != nil {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
|
||||
_, err := registry.Build("generic-extractor")
|
||||
_, err := registry.BuildLegacyRaw("generic-extractor")
|
||||
|
||||
if err == nil {
|
||||
t.Fatal("Build() error = nil, want error")
|
||||
@@ -305,7 +305,7 @@ func TestExtractorRegistryBuildRejectsExtractorKeyMismatch(t *testing.T) {
|
||||
func TestExtractorRegistryRegisteredKeysReturnsSortedCopy(t *testing.T) {
|
||||
registry := NewExtractorRegistry()
|
||||
for _, key := range []string{"zeta", "alpha", "middle"} {
|
||||
if err := registry.Register(key, fakeExtractorConstructor(key)); err != nil {
|
||||
if err := registry.RegisterLegacyRaw(key, fakeExtractorConstructor(key)); err != nil {
|
||||
t.Fatalf("Register(%q) error = %v, want nil", key, err)
|
||||
}
|
||||
}
|
||||
@@ -326,10 +326,10 @@ func TestExtractorRegistryRegisteredKeysReturnsSortedCopy(t *testing.T) {
|
||||
func TestExtractorRegistryNilRegistryBehavior(t *testing.T) {
|
||||
var registry *ExtractorRegistry
|
||||
|
||||
if err := registry.Register("generic-extractor", fakeExtractorConstructor("generic-extractor")); err == nil {
|
||||
if err := registry.RegisterLegacyRaw("generic-extractor", fakeExtractorConstructor("generic-extractor")); err == nil {
|
||||
t.Fatal("Register() error = nil, want error")
|
||||
}
|
||||
if _, err := registry.Build("generic-extractor"); err == nil {
|
||||
if _, err := registry.BuildLegacyRaw("generic-extractor"); err == nil {
|
||||
t.Fatal("Build() error = nil, want error")
|
||||
}
|
||||
if _, ok := registry.Spec("generic-extractor"); ok {
|
||||
@@ -343,7 +343,7 @@ func TestExtractorRegistryNilRegistryBehavior(t *testing.T) {
|
||||
func TestExtractorRegistryBuildRejectsEmptyKey(t *testing.T) {
|
||||
registry := NewExtractorRegistry()
|
||||
|
||||
_, err := registry.Build(" \n")
|
||||
_, err := registry.BuildLegacyRaw(" \n")
|
||||
|
||||
if err == nil {
|
||||
t.Fatal("Build() error = nil, want error")
|
||||
@@ -357,8 +357,8 @@ type registryFakeExtractor struct {
|
||||
key string
|
||||
}
|
||||
|
||||
func fakeExtractorConstructor(key string) ExtractorConstructor {
|
||||
return func() (contracts.Extractor, error) {
|
||||
func fakeExtractorConstructor(key string) LegacyRawExtractorConstructor {
|
||||
return func() (contracts.LegacyRawExtractor, error) {
|
||||
return registryFakeExtractor{key: key}, nil
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,71 +2,114 @@ package pipeline
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"reflect"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
)
|
||||
|
||||
type MergerConstructor func() (contracts.Merger, error)
|
||||
type LegacyRawMergerConstructor func() (contracts.LegacyRawMerger, error)
|
||||
|
||||
type artifactVariantKey struct {
|
||||
module string
|
||||
kind contracts.ArtifactKind
|
||||
}
|
||||
|
||||
type MergerRegistry struct {
|
||||
constructors map[string]MergerConstructor
|
||||
specs map[string]ModuleSpec
|
||||
legacyConstructors map[string]LegacyRawMergerConstructor
|
||||
legacySpecs map[string]ModuleSpec
|
||||
typedEntries map[artifactVariantKey]typedMergerEntry
|
||||
}
|
||||
|
||||
type typedMergerEntry struct {
|
||||
spec ModuleSpec
|
||||
valueType reflect.Type
|
||||
constructor func() (any, error)
|
||||
}
|
||||
|
||||
func NewMergerRegistry() *MergerRegistry {
|
||||
return &MergerRegistry{
|
||||
constructors: make(map[string]MergerConstructor),
|
||||
specs: make(map[string]ModuleSpec),
|
||||
legacyConstructors: make(map[string]LegacyRawMergerConstructor),
|
||||
legacySpecs: make(map[string]ModuleSpec),
|
||||
typedEntries: make(map[artifactVariantKey]typedMergerEntry),
|
||||
}
|
||||
}
|
||||
|
||||
func (r *MergerRegistry) Register(key string, constructor MergerConstructor) error {
|
||||
return r.RegisterWithSpec(defaultModuleSpec(key, StageMerge), constructor)
|
||||
func (r *MergerRegistry) RegisterLegacyRaw(key string, constructor LegacyRawMergerConstructor) error {
|
||||
return r.RegisterLegacyRawWithSpec(defaultModuleSpec(key, StageMerge), constructor)
|
||||
}
|
||||
|
||||
func (r *MergerRegistry) RegisterWithSpec(spec ModuleSpec, constructor MergerConstructor) error {
|
||||
func (r *MergerRegistry) RegisterLegacyRawWithSpec(spec ModuleSpec, constructor LegacyRawMergerConstructor) error {
|
||||
if r == nil {
|
||||
return fmt.Errorf("merger registry must not be nil")
|
||||
}
|
||||
|
||||
normalizedSpec := normalizeModuleSpec(spec)
|
||||
if err := validateModuleSpec("merger", StageMerge, normalizedSpec); err != nil {
|
||||
return err
|
||||
}
|
||||
if normalizedSpec.ArtifactKind != "" {
|
||||
return fmt.Errorf("legacy raw merger %q must not declare an artifact kind", normalizedSpec.Key)
|
||||
}
|
||||
if constructor == nil {
|
||||
return fmt.Errorf("merger constructor for %q must not be nil", normalizedSpec.Key)
|
||||
}
|
||||
if _, ok := r.constructors[normalizedSpec.Key]; ok {
|
||||
return fmt.Errorf("merger %q is already registered", normalizedSpec.Key)
|
||||
if _, ok := r.legacyConstructors[normalizedSpec.Key]; ok {
|
||||
return fmt.Errorf("legacy raw merger %q is already registered", normalizedSpec.Key)
|
||||
}
|
||||
|
||||
if r.constructors == nil {
|
||||
r.constructors = make(map[string]MergerConstructor)
|
||||
if r.legacyConstructors == nil {
|
||||
r.legacyConstructors = make(map[string]LegacyRawMergerConstructor)
|
||||
}
|
||||
if r.specs == nil {
|
||||
r.specs = make(map[string]ModuleSpec)
|
||||
if r.legacySpecs == nil {
|
||||
r.legacySpecs = make(map[string]ModuleSpec)
|
||||
}
|
||||
r.constructors[normalizedSpec.Key] = constructor
|
||||
r.specs[normalizedSpec.Key] = cloneModuleSpec(normalizedSpec)
|
||||
r.legacyConstructors[normalizedSpec.Key] = constructor
|
||||
r.legacySpecs[normalizedSpec.Key] = cloneModuleSpec(normalizedSpec)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *MergerRegistry) Build(key string) (contracts.Merger, error) {
|
||||
func RegisterMerger[T any](registry *MergerRegistry, spec ModuleSpec, constructor func() (contracts.Merger[T], error)) error {
|
||||
if registry == nil {
|
||||
return fmt.Errorf("merger registry must not be nil")
|
||||
}
|
||||
normalizedSpec := normalizeModuleSpec(spec)
|
||||
if err := validateModuleSpec("merger", StageMerge, normalizedSpec); err != nil {
|
||||
return err
|
||||
}
|
||||
if normalizedSpec.ArtifactKind == "" {
|
||||
return fmt.Errorf("typed merger %q artifact kind must not be empty", normalizedSpec.Key)
|
||||
}
|
||||
if constructor == nil {
|
||||
return fmt.Errorf("merger constructor for %q must not be nil", normalizedSpec.Key)
|
||||
}
|
||||
key := artifactVariantKey{module: normalizedSpec.Key, kind: normalizedSpec.ArtifactKind}
|
||||
if _, ok := registry.typedEntries[key]; ok {
|
||||
return fmt.Errorf("merger %q variant for artifact kind %q is already registered", key.module, key.kind)
|
||||
}
|
||||
if registry.typedEntries == nil {
|
||||
registry.typedEntries = make(map[artifactVariantKey]typedMergerEntry)
|
||||
}
|
||||
registry.typedEntries[key] = typedMergerEntry{
|
||||
spec: cloneModuleSpec(normalizedSpec),
|
||||
valueType: reflect.TypeFor[T](),
|
||||
constructor: func() (any, error) {
|
||||
return constructor()
|
||||
},
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *MergerRegistry) BuildLegacyRaw(key string) (contracts.LegacyRawMerger, error) {
|
||||
if r == nil {
|
||||
return nil, fmt.Errorf("merger registry must not be nil")
|
||||
}
|
||||
|
||||
normalizedKey := strings.TrimSpace(key)
|
||||
if normalizedKey == "" {
|
||||
return nil, fmt.Errorf("merger key must not be empty")
|
||||
}
|
||||
|
||||
constructor, ok := r.constructors[normalizedKey]
|
||||
constructor, ok := r.legacyConstructors[normalizedKey]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("merger %q is not registered", normalizedKey)
|
||||
return nil, fmt.Errorf("legacy raw merger %q is not registered", normalizedKey)
|
||||
}
|
||||
|
||||
merger, err := constructor()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("build merger %q: %w", normalizedKey, err)
|
||||
@@ -77,7 +120,6 @@ func (r *MergerRegistry) Build(key string) (contracts.Merger, error) {
|
||||
if merger.Key() != normalizedKey {
|
||||
return nil, fmt.Errorf("merger %q returned key %q", normalizedKey, merger.Key())
|
||||
}
|
||||
|
||||
return merger, nil
|
||||
}
|
||||
|
||||
@@ -85,18 +127,46 @@ func (r *MergerRegistry) Spec(key string) (ModuleSpec, bool) {
|
||||
if r == nil {
|
||||
return ModuleSpec{}, false
|
||||
}
|
||||
|
||||
spec, ok := r.specs[strings.TrimSpace(key)]
|
||||
spec, ok := r.legacySpecs[strings.TrimSpace(key)]
|
||||
if !ok {
|
||||
return ModuleSpec{}, false
|
||||
}
|
||||
return cloneModuleSpec(spec), true
|
||||
}
|
||||
|
||||
func (r *MergerRegistry) typedEntry(key string, kind contracts.ArtifactKind) (typedMergerEntry, bool) {
|
||||
if r == nil {
|
||||
return typedMergerEntry{}, false
|
||||
}
|
||||
entry, ok := r.typedEntries[artifactVariantKey{module: strings.TrimSpace(key), kind: normalizeArtifactKind(kind)}]
|
||||
return entry, ok
|
||||
}
|
||||
|
||||
func (r *MergerRegistry) registeredKinds(key string) []contracts.ArtifactKind {
|
||||
if r == nil {
|
||||
return nil
|
||||
}
|
||||
module := strings.TrimSpace(key)
|
||||
kinds := make([]contracts.ArtifactKind, 0)
|
||||
for variant := range r.typedEntries {
|
||||
if variant.module == module {
|
||||
kinds = append(kinds, variant.kind)
|
||||
}
|
||||
}
|
||||
sortArtifactKinds(kinds)
|
||||
return kinds
|
||||
}
|
||||
|
||||
func (r *MergerRegistry) RegisteredKeys() []string {
|
||||
if r == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
return sortedRegistryKeys(r.constructors)
|
||||
keys := make(map[string]struct{}, len(r.legacySpecs)+len(r.typedEntries))
|
||||
for key := range r.legacySpecs {
|
||||
keys[key] = struct{}{}
|
||||
}
|
||||
for key := range r.typedEntries {
|
||||
keys[key.module] = struct{}{}
|
||||
}
|
||||
return sortedRegistryKeys(keys)
|
||||
}
|
||||
|
||||
@@ -7,7 +7,7 @@ import (
|
||||
)
|
||||
|
||||
func TestMergerRegistryBehavior(t *testing.T) {
|
||||
runRegistryBehaviorTests(t, registryBehaviorCase[contracts.Merger]{
|
||||
runRegistryBehaviorTests(t, registryBehaviorCase[contracts.LegacyRawMerger]{
|
||||
name: "MergerRegistry",
|
||||
key: "generic-merger",
|
||||
stage: StageMerge,
|
||||
@@ -15,14 +15,14 @@ func TestMergerRegistryBehavior(t *testing.T) {
|
||||
newRegistry: func() any {
|
||||
return NewMergerRegistry()
|
||||
},
|
||||
register: func(registry any, key string, constructor func() (contracts.Merger, error)) error {
|
||||
return registry.(*MergerRegistry).Register(key, constructor)
|
||||
register: func(registry any, key string, constructor func() (contracts.LegacyRawMerger, error)) error {
|
||||
return registry.(*MergerRegistry).RegisterLegacyRaw(key, constructor)
|
||||
},
|
||||
registerWithSpec: func(registry any, spec ModuleSpec, constructor func() (contracts.Merger, error)) error {
|
||||
return registry.(*MergerRegistry).RegisterWithSpec(spec, constructor)
|
||||
registerWithSpec: func(registry any, spec ModuleSpec, constructor func() (contracts.LegacyRawMerger, error)) error {
|
||||
return registry.(*MergerRegistry).RegisterLegacyRawWithSpec(spec, constructor)
|
||||
},
|
||||
build: func(registry any, key string) (contracts.Merger, error) {
|
||||
return registry.(*MergerRegistry).Build(key)
|
||||
build: func(registry any, key string) (contracts.LegacyRawMerger, error) {
|
||||
return registry.(*MergerRegistry).BuildLegacyRaw(key)
|
||||
},
|
||||
spec: func(registry any, key string) (ModuleSpec, bool) {
|
||||
return registry.(*MergerRegistry).Spec(key)
|
||||
@@ -30,13 +30,13 @@ func TestMergerRegistryBehavior(t *testing.T) {
|
||||
registeredKeys: func(registry any) []string {
|
||||
return registry.(*MergerRegistry).RegisteredKeys()
|
||||
},
|
||||
nilRegister: func(key string, constructor func() (contracts.Merger, error)) error {
|
||||
nilRegister: func(key string, constructor func() (contracts.LegacyRawMerger, error)) error {
|
||||
var registry *MergerRegistry
|
||||
return registry.Register(key, constructor)
|
||||
return registry.RegisterLegacyRaw(key, constructor)
|
||||
},
|
||||
nilBuild: func(key string) (contracts.Merger, error) {
|
||||
nilBuild: func(key string) (contracts.LegacyRawMerger, error) {
|
||||
var registry *MergerRegistry
|
||||
return registry.Build(key)
|
||||
return registry.BuildLegacyRaw(key)
|
||||
},
|
||||
nilSpec: func(key string) (ModuleSpec, bool) {
|
||||
var registry *MergerRegistry
|
||||
@@ -46,12 +46,12 @@ func TestMergerRegistryBehavior(t *testing.T) {
|
||||
var registry *MergerRegistry
|
||||
return registry.RegisteredKeys()
|
||||
},
|
||||
constructor: func(key string) func() (contracts.Merger, error) {
|
||||
return func() (contracts.Merger, error) {
|
||||
constructor: func(key string) func() (contracts.LegacyRawMerger, error) {
|
||||
return func() (contracts.LegacyRawMerger, error) {
|
||||
return registryMerger{key: key}, nil
|
||||
}
|
||||
},
|
||||
moduleKey: func(module contracts.Merger) string {
|
||||
moduleKey: func(module contracts.LegacyRawMerger) string {
|
||||
return module.Key()
|
||||
},
|
||||
})
|
||||
|
||||
@@ -23,6 +23,7 @@ const (
|
||||
type ModuleSpec struct {
|
||||
Key string
|
||||
Stage ModuleStage
|
||||
ArtifactKind contracts.ArtifactKind
|
||||
Provides []string
|
||||
Requires []string
|
||||
ReferenceSlots []contracts.ReferenceSlot
|
||||
@@ -39,6 +40,7 @@ func normalizeModuleSpec(spec ModuleSpec) ModuleSpec {
|
||||
return ModuleSpec{
|
||||
Key: strings.TrimSpace(spec.Key),
|
||||
Stage: spec.Stage,
|
||||
ArtifactKind: normalizeArtifactKind(spec.ArtifactKind),
|
||||
Provides: normalizeCapabilities(spec.Provides),
|
||||
Requires: normalizeCapabilities(spec.Requires),
|
||||
ReferenceSlots: normalizeReferenceSlots(spec.ReferenceSlots),
|
||||
@@ -74,6 +76,7 @@ func cloneModuleSpec(spec ModuleSpec) ModuleSpec {
|
||||
return ModuleSpec{
|
||||
Key: spec.Key,
|
||||
Stage: spec.Stage,
|
||||
ArtifactKind: spec.ArtifactKind,
|
||||
Provides: append([]string(nil), spec.Provides...),
|
||||
Requires: append([]string(nil), spec.Requires...),
|
||||
ReferenceSlots: contracts.CloneReferenceSlots(spec.ReferenceSlots),
|
||||
@@ -87,6 +90,9 @@ func validateModuleSpec(kind string, expectedStage ModuleStage, spec ModuleSpec)
|
||||
if spec.Stage != expectedStage {
|
||||
return fmt.Errorf("%s %q must use %q stage, got %q", kind, spec.Key, expectedStage, spec.Stage)
|
||||
}
|
||||
if spec.ArtifactKind != "" && spec.Stage != StageExtract && spec.Stage != StageMerge && spec.Stage != StageNormalize {
|
||||
return fmt.Errorf("%s %q must not declare an artifact kind", kind, spec.Key)
|
||||
}
|
||||
if !referenceSlotStage(spec.Stage) && len(spec.ReferenceSlots) > 0 {
|
||||
return fmt.Errorf("%s %q must not declare reference slots", kind, spec.Key)
|
||||
}
|
||||
@@ -113,6 +119,10 @@ func sortedRegistryKeys[C any](constructors map[string]C) []string {
|
||||
return keys
|
||||
}
|
||||
|
||||
func sortArtifactKinds(kinds []contracts.ArtifactKind) {
|
||||
sort.Slice(kinds, func(i, j int) bool { return kinds[i] < kinds[j] })
|
||||
}
|
||||
|
||||
func normalizeReferenceSlots(slots []contracts.ReferenceSlot) []contracts.ReferenceSlot {
|
||||
if len(slots) == 0 {
|
||||
return nil
|
||||
|
||||
@@ -2,71 +2,109 @@ package pipeline
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"reflect"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
)
|
||||
|
||||
type NormalizerConstructor func() (contracts.Normalizer, error)
|
||||
type LegacyRawNormalizerConstructor func() (contracts.LegacyRawNormalizer, error)
|
||||
|
||||
type NormalizerRegistry struct {
|
||||
constructors map[string]NormalizerConstructor
|
||||
specs map[string]ModuleSpec
|
||||
legacyConstructors map[string]LegacyRawNormalizerConstructor
|
||||
legacySpecs map[string]ModuleSpec
|
||||
typedEntries map[artifactVariantKey]typedNormalizerEntry
|
||||
}
|
||||
|
||||
type typedNormalizerEntry struct {
|
||||
spec ModuleSpec
|
||||
valueType reflect.Type
|
||||
constructor func() (any, error)
|
||||
}
|
||||
|
||||
func NewNormalizerRegistry() *NormalizerRegistry {
|
||||
return &NormalizerRegistry{
|
||||
constructors: make(map[string]NormalizerConstructor),
|
||||
specs: make(map[string]ModuleSpec),
|
||||
legacyConstructors: make(map[string]LegacyRawNormalizerConstructor),
|
||||
legacySpecs: make(map[string]ModuleSpec),
|
||||
typedEntries: make(map[artifactVariantKey]typedNormalizerEntry),
|
||||
}
|
||||
}
|
||||
|
||||
func (r *NormalizerRegistry) Register(key string, constructor NormalizerConstructor) error {
|
||||
return r.RegisterWithSpec(defaultModuleSpec(key, StageNormalize), constructor)
|
||||
func (r *NormalizerRegistry) RegisterLegacyRaw(key string, constructor LegacyRawNormalizerConstructor) error {
|
||||
return r.RegisterLegacyRawWithSpec(defaultModuleSpec(key, StageNormalize), constructor)
|
||||
}
|
||||
|
||||
func (r *NormalizerRegistry) RegisterWithSpec(spec ModuleSpec, constructor NormalizerConstructor) error {
|
||||
func (r *NormalizerRegistry) RegisterLegacyRawWithSpec(spec ModuleSpec, constructor LegacyRawNormalizerConstructor) error {
|
||||
if r == nil {
|
||||
return fmt.Errorf("normalizer registry must not be nil")
|
||||
}
|
||||
|
||||
normalizedSpec := normalizeModuleSpec(spec)
|
||||
if err := validateModuleSpec("normalizer", StageNormalize, normalizedSpec); err != nil {
|
||||
return err
|
||||
}
|
||||
if normalizedSpec.ArtifactKind != "" {
|
||||
return fmt.Errorf("legacy raw normalizer %q must not declare an artifact kind", normalizedSpec.Key)
|
||||
}
|
||||
if constructor == nil {
|
||||
return fmt.Errorf("normalizer constructor for %q must not be nil", normalizedSpec.Key)
|
||||
}
|
||||
if _, ok := r.constructors[normalizedSpec.Key]; ok {
|
||||
return fmt.Errorf("normalizer %q is already registered", normalizedSpec.Key)
|
||||
if _, ok := r.legacyConstructors[normalizedSpec.Key]; ok {
|
||||
return fmt.Errorf("legacy raw normalizer %q is already registered", normalizedSpec.Key)
|
||||
}
|
||||
|
||||
if r.constructors == nil {
|
||||
r.constructors = make(map[string]NormalizerConstructor)
|
||||
if r.legacyConstructors == nil {
|
||||
r.legacyConstructors = make(map[string]LegacyRawNormalizerConstructor)
|
||||
}
|
||||
if r.specs == nil {
|
||||
r.specs = make(map[string]ModuleSpec)
|
||||
if r.legacySpecs == nil {
|
||||
r.legacySpecs = make(map[string]ModuleSpec)
|
||||
}
|
||||
r.constructors[normalizedSpec.Key] = constructor
|
||||
r.specs[normalizedSpec.Key] = cloneModuleSpec(normalizedSpec)
|
||||
r.legacyConstructors[normalizedSpec.Key] = constructor
|
||||
r.legacySpecs[normalizedSpec.Key] = cloneModuleSpec(normalizedSpec)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *NormalizerRegistry) Build(key string) (contracts.Normalizer, error) {
|
||||
func RegisterNormalizer[T any](registry *NormalizerRegistry, spec ModuleSpec, constructor func() (contracts.Normalizer[T], error)) error {
|
||||
if registry == nil {
|
||||
return fmt.Errorf("normalizer registry must not be nil")
|
||||
}
|
||||
normalizedSpec := normalizeModuleSpec(spec)
|
||||
if err := validateModuleSpec("normalizer", StageNormalize, normalizedSpec); err != nil {
|
||||
return err
|
||||
}
|
||||
if normalizedSpec.ArtifactKind == "" {
|
||||
return fmt.Errorf("typed normalizer %q artifact kind must not be empty", normalizedSpec.Key)
|
||||
}
|
||||
if constructor == nil {
|
||||
return fmt.Errorf("normalizer constructor for %q must not be nil", normalizedSpec.Key)
|
||||
}
|
||||
key := artifactVariantKey{module: normalizedSpec.Key, kind: normalizedSpec.ArtifactKind}
|
||||
if _, ok := registry.typedEntries[key]; ok {
|
||||
return fmt.Errorf("normalizer %q variant for artifact kind %q is already registered", key.module, key.kind)
|
||||
}
|
||||
if registry.typedEntries == nil {
|
||||
registry.typedEntries = make(map[artifactVariantKey]typedNormalizerEntry)
|
||||
}
|
||||
registry.typedEntries[key] = typedNormalizerEntry{
|
||||
spec: cloneModuleSpec(normalizedSpec),
|
||||
valueType: reflect.TypeFor[T](),
|
||||
constructor: func() (any, error) {
|
||||
return constructor()
|
||||
},
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *NormalizerRegistry) BuildLegacyRaw(key string) (contracts.LegacyRawNormalizer, error) {
|
||||
if r == nil {
|
||||
return nil, fmt.Errorf("normalizer registry must not be nil")
|
||||
}
|
||||
|
||||
normalizedKey := strings.TrimSpace(key)
|
||||
if normalizedKey == "" {
|
||||
return nil, fmt.Errorf("normalizer key must not be empty")
|
||||
}
|
||||
|
||||
constructor, ok := r.constructors[normalizedKey]
|
||||
constructor, ok := r.legacyConstructors[normalizedKey]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("normalizer %q is not registered", normalizedKey)
|
||||
return nil, fmt.Errorf("legacy raw normalizer %q is not registered", normalizedKey)
|
||||
}
|
||||
|
||||
normalizer, err := constructor()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("build normalizer %q: %w", normalizedKey, err)
|
||||
@@ -77,7 +115,6 @@ func (r *NormalizerRegistry) Build(key string) (contracts.Normalizer, error) {
|
||||
if normalizer.Key() != normalizedKey {
|
||||
return nil, fmt.Errorf("normalizer %q returned key %q", normalizedKey, normalizer.Key())
|
||||
}
|
||||
|
||||
return normalizer, nil
|
||||
}
|
||||
|
||||
@@ -85,18 +122,46 @@ func (r *NormalizerRegistry) Spec(key string) (ModuleSpec, bool) {
|
||||
if r == nil {
|
||||
return ModuleSpec{}, false
|
||||
}
|
||||
|
||||
spec, ok := r.specs[strings.TrimSpace(key)]
|
||||
spec, ok := r.legacySpecs[strings.TrimSpace(key)]
|
||||
if !ok {
|
||||
return ModuleSpec{}, false
|
||||
}
|
||||
return cloneModuleSpec(spec), true
|
||||
}
|
||||
|
||||
func (r *NormalizerRegistry) typedEntry(key string, kind contracts.ArtifactKind) (typedNormalizerEntry, bool) {
|
||||
if r == nil {
|
||||
return typedNormalizerEntry{}, false
|
||||
}
|
||||
entry, ok := r.typedEntries[artifactVariantKey{module: strings.TrimSpace(key), kind: normalizeArtifactKind(kind)}]
|
||||
return entry, ok
|
||||
}
|
||||
|
||||
func (r *NormalizerRegistry) registeredKinds(key string) []contracts.ArtifactKind {
|
||||
if r == nil {
|
||||
return nil
|
||||
}
|
||||
module := strings.TrimSpace(key)
|
||||
kinds := make([]contracts.ArtifactKind, 0)
|
||||
for variant := range r.typedEntries {
|
||||
if variant.module == module {
|
||||
kinds = append(kinds, variant.kind)
|
||||
}
|
||||
}
|
||||
sortArtifactKinds(kinds)
|
||||
return kinds
|
||||
}
|
||||
|
||||
func (r *NormalizerRegistry) RegisteredKeys() []string {
|
||||
if r == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
return sortedRegistryKeys(r.constructors)
|
||||
keys := make(map[string]struct{}, len(r.legacySpecs)+len(r.typedEntries))
|
||||
for key := range r.legacySpecs {
|
||||
keys[key] = struct{}{}
|
||||
}
|
||||
for key := range r.typedEntries {
|
||||
keys[key.module] = struct{}{}
|
||||
}
|
||||
return sortedRegistryKeys(keys)
|
||||
}
|
||||
|
||||
@@ -7,7 +7,7 @@ import (
|
||||
)
|
||||
|
||||
func TestNormalizerRegistryBehavior(t *testing.T) {
|
||||
runRegistryBehaviorTests(t, registryBehaviorCase[contracts.Normalizer]{
|
||||
runRegistryBehaviorTests(t, registryBehaviorCase[contracts.LegacyRawNormalizer]{
|
||||
name: "NormalizerRegistry",
|
||||
key: "generic-normalizer",
|
||||
stage: StageNormalize,
|
||||
@@ -15,14 +15,14 @@ func TestNormalizerRegistryBehavior(t *testing.T) {
|
||||
newRegistry: func() any {
|
||||
return NewNormalizerRegistry()
|
||||
},
|
||||
register: func(registry any, key string, constructor func() (contracts.Normalizer, error)) error {
|
||||
return registry.(*NormalizerRegistry).Register(key, constructor)
|
||||
register: func(registry any, key string, constructor func() (contracts.LegacyRawNormalizer, error)) error {
|
||||
return registry.(*NormalizerRegistry).RegisterLegacyRaw(key, constructor)
|
||||
},
|
||||
registerWithSpec: func(registry any, spec ModuleSpec, constructor func() (contracts.Normalizer, error)) error {
|
||||
return registry.(*NormalizerRegistry).RegisterWithSpec(spec, constructor)
|
||||
registerWithSpec: func(registry any, spec ModuleSpec, constructor func() (contracts.LegacyRawNormalizer, error)) error {
|
||||
return registry.(*NormalizerRegistry).RegisterLegacyRawWithSpec(spec, constructor)
|
||||
},
|
||||
build: func(registry any, key string) (contracts.Normalizer, error) {
|
||||
return registry.(*NormalizerRegistry).Build(key)
|
||||
build: func(registry any, key string) (contracts.LegacyRawNormalizer, error) {
|
||||
return registry.(*NormalizerRegistry).BuildLegacyRaw(key)
|
||||
},
|
||||
spec: func(registry any, key string) (ModuleSpec, bool) {
|
||||
return registry.(*NormalizerRegistry).Spec(key)
|
||||
@@ -30,13 +30,13 @@ func TestNormalizerRegistryBehavior(t *testing.T) {
|
||||
registeredKeys: func(registry any) []string {
|
||||
return registry.(*NormalizerRegistry).RegisteredKeys()
|
||||
},
|
||||
nilRegister: func(key string, constructor func() (contracts.Normalizer, error)) error {
|
||||
nilRegister: func(key string, constructor func() (contracts.LegacyRawNormalizer, error)) error {
|
||||
var registry *NormalizerRegistry
|
||||
return registry.Register(key, constructor)
|
||||
return registry.RegisterLegacyRaw(key, constructor)
|
||||
},
|
||||
nilBuild: func(key string) (contracts.Normalizer, error) {
|
||||
nilBuild: func(key string) (contracts.LegacyRawNormalizer, error) {
|
||||
var registry *NormalizerRegistry
|
||||
return registry.Build(key)
|
||||
return registry.BuildLegacyRaw(key)
|
||||
},
|
||||
nilSpec: func(key string) (ModuleSpec, bool) {
|
||||
var registry *NormalizerRegistry
|
||||
@@ -46,12 +46,12 @@ func TestNormalizerRegistryBehavior(t *testing.T) {
|
||||
var registry *NormalizerRegistry
|
||||
return registry.RegisteredKeys()
|
||||
},
|
||||
constructor: func(key string) func() (contracts.Normalizer, error) {
|
||||
return func() (contracts.Normalizer, error) {
|
||||
constructor: func(key string) func() (contracts.LegacyRawNormalizer, error) {
|
||||
return func() (contracts.LegacyRawNormalizer, error) {
|
||||
return registryNormalizer{key: key}, nil
|
||||
}
|
||||
},
|
||||
moduleKey: func(module contracts.Normalizer) string {
|
||||
moduleKey: func(module contracts.LegacyRawNormalizer) string {
|
||||
return module.Key()
|
||||
},
|
||||
})
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
@@ -102,14 +103,19 @@ type ResolvedReferenceTarget struct {
|
||||
}
|
||||
|
||||
type ResolvedArtifactLane struct {
|
||||
ID string
|
||||
Extract ModuleBinding
|
||||
Merge ModuleBinding
|
||||
Normalize ModuleBinding
|
||||
Validators []ModuleBinding
|
||||
ExtractReferences ResolvedReferenceTarget `json:"extract_references"`
|
||||
MergeReferences ResolvedReferenceTarget `json:"merge_references"`
|
||||
NormalizeReferences ResolvedReferenceTarget `json:"normalize_references"`
|
||||
ID string
|
||||
ArtifactKind contracts.ArtifactKind `json:"artifact_kind,omitempty"`
|
||||
ArtifactSchemaID string `json:"artifact_schema_id,omitempty"`
|
||||
ArtifactSchemaName string `json:"artifact_schema_name,omitempty"`
|
||||
ArtifactSchemaVersion string `json:"artifact_schema_version,omitempty"`
|
||||
ArtifactSchemaDigest string `json:"artifact_schema_digest,omitempty"`
|
||||
Extract ModuleBinding
|
||||
Merge ModuleBinding
|
||||
Normalize ModuleBinding
|
||||
Validators []ModuleBinding
|
||||
ExtractReferences ResolvedReferenceTarget `json:"extract_references"`
|
||||
MergeReferences ResolvedReferenceTarget `json:"merge_references"`
|
||||
NormalizeReferences ResolvedReferenceTarget `json:"normalize_references"`
|
||||
}
|
||||
|
||||
type ResolvedValidatorChain struct {
|
||||
@@ -122,6 +128,8 @@ type ResolvedValidatorChain struct {
|
||||
type ResolvedValidator struct {
|
||||
Binding ModuleBinding `json:"binding"`
|
||||
ExecutionClass contracts.ExecutionClass `json:"execution_class"`
|
||||
Target ValidatorTarget `json:"target,omitempty"`
|
||||
ArtifactKind contracts.ArtifactKind `json:"artifact_kind,omitempty"`
|
||||
}
|
||||
|
||||
type ResolvedPipeline struct {
|
||||
@@ -216,7 +224,7 @@ func ResolvePipeline(profile PipelineProfile, options ResolveOptions, catalog Mo
|
||||
ChunkReferences: referenceTarget(StageChunk, "", chunk.Module, chunkReferences),
|
||||
Output: resolveBinding(profile.Output, DefaultOutputModule),
|
||||
}
|
||||
chunkValidatorChain, err := resolveValidatorChain(pipelineID, "", StageChunk, chunk.Module, chunk.Validators, catalog)
|
||||
chunkValidatorChain, err := resolveValidatorChain(pipelineID, "", StageChunk, chunk.Module, chunk.Validators, "", nil, catalog)
|
||||
if err != nil {
|
||||
return ResolvedPipeline{}, err
|
||||
}
|
||||
@@ -279,6 +287,10 @@ func resolveArtifactLane(
|
||||
if missing, ok := capabilities.missing(extractSpec.Requires); ok {
|
||||
return ResolvedArtifactLane{}, nil, nil, capabilityError(pipelineID, laneID, StageExtract, lane.Extract.Module, missing)
|
||||
}
|
||||
artifactType, err := resolveArtifactIdentity(pipelineID, laneID, &lane, extractSpec, catalog)
|
||||
if err != nil {
|
||||
return ResolvedArtifactLane{}, nil, nil, err
|
||||
}
|
||||
extractReferences := mergeReferenceMaps(profile.References, lane.Extract.References)
|
||||
references, err := resolveReferenceTargetBindings(referenceResolutionTarget{
|
||||
PipelineID: pipelineID,
|
||||
@@ -296,7 +308,7 @@ func resolveArtifactLane(
|
||||
lane.ExtractReferences = referenceTarget(StageExtract, laneID, lane.Extract.Module, references)
|
||||
capabilities.add(extractSpec.Provides...)
|
||||
|
||||
mergeSpec, err := mergerSpec(catalog, lane.Merge.Module)
|
||||
mergeSpec, err := mergerSpecForArtifact(catalog, lane.Merge.Module, lane.ArtifactKind, artifactType)
|
||||
if err != nil {
|
||||
return ResolvedArtifactLane{}, nil, nil, moduleLookupError(pipelineID, laneID, StageMerge, lane.Merge.Module, err)
|
||||
}
|
||||
@@ -319,7 +331,7 @@ func resolveArtifactLane(
|
||||
lane.MergeReferences = referenceTarget(StageMerge, laneID, lane.Merge.Module, mergeReferences)
|
||||
capabilities.add(mergeSpec.Provides...)
|
||||
|
||||
normalizeSpec, err := normalizerSpec(catalog, lane.Normalize.Module)
|
||||
normalizeSpec, err := normalizerSpecForArtifact(catalog, lane.Normalize.Module, lane.ArtifactKind, artifactType)
|
||||
if err != nil {
|
||||
return ResolvedArtifactLane{}, nil, nil, moduleLookupError(pipelineID, laneID, StageNormalize, lane.Normalize.Module, err)
|
||||
}
|
||||
@@ -346,15 +358,15 @@ func resolveArtifactLane(
|
||||
return ResolvedArtifactLane{}, nil, nil, configuredValidatorsError(pipelineID, laneID)
|
||||
}
|
||||
|
||||
extractValidatorChain, err := resolveValidatorChain(pipelineID, laneID, StageExtract, lane.Extract.Module, lane.Extract.Validators, catalog)
|
||||
extractValidatorChain, err := resolveValidatorChain(pipelineID, laneID, StageExtract, lane.Extract.Module, lane.Extract.Validators, lane.ArtifactKind, artifactType, catalog)
|
||||
if err != nil {
|
||||
return ResolvedArtifactLane{}, nil, nil, err
|
||||
}
|
||||
mergeValidatorChain, err := resolveValidatorChain(pipelineID, laneID, StageMerge, lane.Merge.Module, lane.Merge.Validators, catalog)
|
||||
mergeValidatorChain, err := resolveValidatorChain(pipelineID, laneID, StageMerge, lane.Merge.Module, lane.Merge.Validators, lane.ArtifactKind, artifactType, catalog)
|
||||
if err != nil {
|
||||
return ResolvedArtifactLane{}, nil, nil, err
|
||||
}
|
||||
normalizeValidatorChain, err := resolveValidatorChain(pipelineID, laneID, StageNormalize, lane.Normalize.Module, lane.Normalize.Validators, catalog)
|
||||
normalizeValidatorChain, err := resolveValidatorChain(pipelineID, laneID, StageNormalize, lane.Normalize.Module, lane.Normalize.Validators, lane.ArtifactKind, artifactType, catalog)
|
||||
if err != nil {
|
||||
return ResolvedArtifactLane{}, nil, nil, err
|
||||
}
|
||||
@@ -367,7 +379,128 @@ func configuredValidatorsError(pipelineID string, laneID string) error {
|
||||
return fmt.Errorf("pipeline %q lane %q validators are not supported at artifact lane level; use extract.validators, merge.validators, or normalize.validators", pipelineID, laneID)
|
||||
}
|
||||
|
||||
func resolveValidatorChain(pipelineID string, laneID string, stage ModuleStage, module string, override ValidatorOverride, catalog ModuleCatalog) (ResolvedValidatorChain, error) {
|
||||
func resolveArtifactIdentity(pipelineID, laneID string, lane *ResolvedArtifactLane, extractSpec ModuleSpec, catalog ModuleCatalog) (reflect.Type, error) {
|
||||
if extractSpec.ArtifactKind == "" {
|
||||
return nil, nil
|
||||
}
|
||||
if catalog.Extractors == nil {
|
||||
return nil, fmt.Errorf("pipeline %q lane %q extractor registry must not be nil", pipelineID, laneID)
|
||||
}
|
||||
extractor, ok := catalog.Extractors.typedEntry(lane.Extract.Module)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("pipeline %q lane %q extract module %q declares artifact kind %q without a typed registration", pipelineID, laneID, lane.Extract.Module, extractSpec.ArtifactKind)
|
||||
}
|
||||
if catalog.ArtifactCodecs == nil {
|
||||
return nil, fmt.Errorf("pipeline %q lane %q artifact codec registry must not be nil for kind %q", pipelineID, laneID, extractSpec.ArtifactKind)
|
||||
}
|
||||
codecSpec, ok := catalog.ArtifactCodecs.Spec(extractSpec.ArtifactKind)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("pipeline %q lane %q artifact codec %q is not registered", pipelineID, laneID, extractSpec.ArtifactKind)
|
||||
}
|
||||
codecType, ok := catalog.ArtifactCodecs.valueType(extractSpec.ArtifactKind)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("pipeline %q lane %q artifact codec %q has no Go type", pipelineID, laneID, extractSpec.ArtifactKind)
|
||||
}
|
||||
if extractor.valueType != codecType {
|
||||
return nil, artifactTypeMismatchError(pipelineID, laneID, StageExtract, lane.Extract.Module, extractSpec.ArtifactKind, codecType, extractor.valueType)
|
||||
}
|
||||
lane.ArtifactKind = codecSpec.Kind
|
||||
lane.ArtifactSchemaID = codecSpec.Schema.ID
|
||||
lane.ArtifactSchemaName = codecSpec.Schema.Name
|
||||
lane.ArtifactSchemaVersion = codecSpec.Schema.Version
|
||||
lane.ArtifactSchemaDigest = codecSpec.SchemaDigest
|
||||
return codecType, nil
|
||||
}
|
||||
|
||||
func mergerSpecForArtifact(catalog ModuleCatalog, key string, kind contracts.ArtifactKind, expectedType reflect.Type) (ModuleSpec, error) {
|
||||
if kind == "" {
|
||||
return mergerSpec(catalog, key)
|
||||
}
|
||||
if catalog.Mergers == nil {
|
||||
return ModuleSpec{}, fmt.Errorf("module %q is not registered", key)
|
||||
}
|
||||
entry, ok := catalog.Mergers.typedEntry(key, kind)
|
||||
if !ok {
|
||||
return ModuleSpec{}, missingArtifactVariantError("merger", key, kind, catalog.Mergers.registeredKinds(key))
|
||||
}
|
||||
if entry.valueType != expectedType {
|
||||
return ModuleSpec{}, fmt.Errorf("artifact kind %q requires Go type %s, but merger %q variant uses %s", kind, typeName(expectedType), key, typeName(entry.valueType))
|
||||
}
|
||||
return cloneModuleSpec(entry.spec), nil
|
||||
}
|
||||
|
||||
func normalizerSpecForArtifact(catalog ModuleCatalog, key string, kind contracts.ArtifactKind, expectedType reflect.Type) (ModuleSpec, error) {
|
||||
if kind == "" {
|
||||
return normalizerSpec(catalog, key)
|
||||
}
|
||||
if catalog.Normalizers == nil {
|
||||
return ModuleSpec{}, fmt.Errorf("module %q is not registered", key)
|
||||
}
|
||||
entry, ok := catalog.Normalizers.typedEntry(key, kind)
|
||||
if !ok {
|
||||
return ModuleSpec{}, missingArtifactVariantError("normalizer", key, kind, catalog.Normalizers.registeredKinds(key))
|
||||
}
|
||||
if entry.valueType != expectedType {
|
||||
return ModuleSpec{}, fmt.Errorf("artifact kind %q requires Go type %s, but normalizer %q variant uses %s", kind, typeName(expectedType), key, typeName(entry.valueType))
|
||||
}
|
||||
return cloneModuleSpec(entry.spec), nil
|
||||
}
|
||||
|
||||
func validatorSpecForTarget(registry *ValidatorRegistry, stage ModuleStage, key string, kind contracts.ArtifactKind, expectedType reflect.Type) (ValidatorSpec, ValidatorTarget, error) {
|
||||
key = strings.TrimSpace(key)
|
||||
if stage == StageChunk {
|
||||
if entry, ok := registry.chunkEntry(key); ok {
|
||||
return entry.spec, ValidatorTargetChunk, nil
|
||||
}
|
||||
if entry, ok := registry.serializedEntry(key); ok && entry.spec.SupportsChunks {
|
||||
return entry.spec.ValidatorSpec, ValidatorTargetSerialized, nil
|
||||
}
|
||||
if spec, ok := registry.Spec(key); ok {
|
||||
return spec, "", nil
|
||||
}
|
||||
return ValidatorSpec{}, "", fmt.Errorf("references unknown validator %q for chunk target", key)
|
||||
}
|
||||
if kind == "" {
|
||||
if spec, ok := registry.Spec(key); ok {
|
||||
return spec, "", nil
|
||||
}
|
||||
return ValidatorSpec{}, "", fmt.Errorf("references unknown validator %q on legacy raw path", key)
|
||||
}
|
||||
if entry, ok := registry.typedEntry(key, kind); ok {
|
||||
if entry.valueType != expectedType {
|
||||
return ValidatorSpec{}, "", fmt.Errorf("artifact kind %q requires Go type %s, but validator %q variant uses %s", kind, typeName(expectedType), key, typeName(entry.valueType))
|
||||
}
|
||||
return entry.spec, ValidatorTargetTyped, nil
|
||||
}
|
||||
if entry, ok := registry.serializedEntry(key); ok && entry.spec.SupportsArtifacts {
|
||||
return entry.spec.ValidatorSpec, ValidatorTargetSerialized, nil
|
||||
}
|
||||
return ValidatorSpec{}, "", missingArtifactVariantError("validator", key, kind, registry.registeredTypedKinds(key))
|
||||
}
|
||||
|
||||
func missingArtifactVariantError(moduleType, key string, kind contracts.ArtifactKind, registered []contracts.ArtifactKind) error {
|
||||
if len(registered) == 0 {
|
||||
return fmt.Errorf("%s %q has no typed variant for artifact kind %q", moduleType, key, kind)
|
||||
}
|
||||
values := make([]string, len(registered))
|
||||
for i, value := range registered {
|
||||
values[i] = string(value)
|
||||
}
|
||||
return fmt.Errorf("%s %q has no typed variant for artifact kind %q; registered kinds: %s", moduleType, key, kind, strings.Join(values, ", "))
|
||||
}
|
||||
|
||||
func artifactTypeMismatchError(pipelineID, laneID string, stage ModuleStage, module string, kind contracts.ArtifactKind, expected, actual reflect.Type) error {
|
||||
return fmt.Errorf("pipeline %q lane %q %s module %q artifact kind %q requires Go type %s, got %s", pipelineID, laneID, stage, module, kind, typeName(expected), typeName(actual))
|
||||
}
|
||||
|
||||
func typeName(value reflect.Type) string {
|
||||
if value == nil {
|
||||
return "<nil>"
|
||||
}
|
||||
return value.String()
|
||||
}
|
||||
|
||||
func resolveValidatorChain(pipelineID string, laneID string, stage ModuleStage, module string, override ValidatorOverride, artifactKind contracts.ArtifactKind, artifactType reflect.Type, catalog ModuleCatalog) (ResolvedValidatorChain, error) {
|
||||
chain := ResolvedValidatorChain{
|
||||
Stage: stage,
|
||||
LaneID: strings.TrimSpace(laneID),
|
||||
@@ -396,9 +529,9 @@ func resolveValidatorChain(pipelineID string, laneID string, stage ModuleStage,
|
||||
}
|
||||
chain.Validators = make([]ResolvedValidator, 0, len(bindings))
|
||||
for _, validator := range bindings {
|
||||
spec, ok := catalog.Validators.Spec(validator.Module)
|
||||
if !ok {
|
||||
return ResolvedValidatorChain{}, fmt.Errorf("pipeline %q %s validator chain for module %q references unknown validator %q", pipelineID, stage, chain.ModuleKey, validator.Module)
|
||||
spec, target, err := validatorSpecForTarget(catalog.Validators, stage, validator.Module, artifactKind, artifactType)
|
||||
if err != nil {
|
||||
return ResolvedValidatorChain{}, fmt.Errorf("pipeline %q %s validator chain for module %q: %w", pipelineID, stage, chain.ModuleKey, err)
|
||||
}
|
||||
if strings.TrimSpace(validator.LLMProfile) != "" && spec.ExecutionClass != contracts.ExecutionClassLLMBacked {
|
||||
return ResolvedValidatorChain{}, fmt.Errorf("pipeline %q %s validator chain for module %q assigns llm_profile to deterministic validator %q", pipelineID, stage, chain.ModuleKey, validator.Module)
|
||||
@@ -406,6 +539,8 @@ func resolveValidatorChain(pipelineID string, laneID string, stage ModuleStage,
|
||||
chain.Validators = append(chain.Validators, ResolvedValidator{
|
||||
Binding: cloneModuleBinding(validator),
|
||||
ExecutionClass: spec.ExecutionClass,
|
||||
Target: target,
|
||||
ArtifactKind: artifactKind,
|
||||
})
|
||||
}
|
||||
return chain, nil
|
||||
@@ -436,6 +571,8 @@ func cloneResolvedValidators(validators []ResolvedValidator) []ResolvedValidator
|
||||
out[i] = ResolvedValidator{
|
||||
Binding: cloneModuleBinding(validator.Binding),
|
||||
ExecutionClass: validator.ExecutionClass,
|
||||
Target: validator.Target,
|
||||
ArtifactKind: validator.ArtifactKind,
|
||||
}
|
||||
}
|
||||
return out
|
||||
@@ -498,7 +635,13 @@ func validatePipelineReferenceDefaults(
|
||||
}
|
||||
|
||||
merge := resolveBinding(laneProfile.Merge, DefaultMergeModule)
|
||||
mergeSpec, err := mergerSpec(catalog, merge.Module)
|
||||
var artifactType reflect.Type
|
||||
if extractSpec.ArtifactKind != "" && catalog.Extractors != nil {
|
||||
if entry, ok := catalog.Extractors.typedEntry(extract.Module); ok {
|
||||
artifactType = entry.valueType
|
||||
}
|
||||
}
|
||||
mergeSpec, err := mergerSpecForArtifact(catalog, merge.Module, extractSpec.ArtifactKind, artifactType)
|
||||
if err != nil {
|
||||
return moduleLookupError(pipelineID, laneID, StageMerge, merge.Module, err)
|
||||
}
|
||||
@@ -507,7 +650,7 @@ func validatePipelineReferenceDefaults(
|
||||
}
|
||||
|
||||
normalize := resolveBinding(laneProfile.Normalize, DefaultNormalizeModule)
|
||||
normalizeSpec, err := normalizerSpec(catalog, normalize.Module)
|
||||
normalizeSpec, err := normalizerSpecForArtifact(catalog, normalize.Module, extractSpec.ArtifactKind, artifactType)
|
||||
if err != nil {
|
||||
return moduleLookupError(pipelineID, laneID, StageNormalize, normalize.Module, err)
|
||||
}
|
||||
|
||||
@@ -771,7 +771,7 @@ func TestResolvePipelineUsesReferenceSlotsFromSpecWithoutConstructingExtractor(t
|
||||
registerProfileSpecs(t, catalog, spec)
|
||||
}
|
||||
}
|
||||
if err := catalog.Extractors.RegisterWithSpec(ModuleSpec{
|
||||
if err := catalog.Extractors.RegisterLegacyRawWithSpec(ModuleSpec{
|
||||
Key: "event-extractor",
|
||||
Stage: StageExtract,
|
||||
Requires: []string{"chunk"},
|
||||
@@ -779,7 +779,7 @@ func TestResolvePipelineUsesReferenceSlotsFromSpecWithoutConstructingExtractor(t
|
||||
ReferenceSlots: []contracts.ReferenceSlot{
|
||||
{Name: "roster", Required: true},
|
||||
},
|
||||
}, func() (contracts.Extractor, error) {
|
||||
}, func() (contracts.LegacyRawExtractor, error) {
|
||||
return nil, errors.New("constructor should not run")
|
||||
}); err != nil {
|
||||
t.Fatalf("RegisterWithSpec() error = %v, want nil", err)
|
||||
@@ -1251,20 +1251,20 @@ func registerProfileSpecs(t *testing.T, catalog ModuleCatalog, specs ...ModuleSp
|
||||
t.Fatalf("register chunk spec %#v: %v", spec, err)
|
||||
}
|
||||
case StageExtract:
|
||||
if err := catalog.Extractors.RegisterWithSpec(spec, profileExtractorConstructor(spec.Key)); err != nil {
|
||||
if err := catalog.Extractors.RegisterLegacyRawWithSpec(spec, profileExtractorConstructor(spec.Key)); err != nil {
|
||||
t.Fatalf("register extractor spec %#v: %v", spec, err)
|
||||
}
|
||||
case StageMerge:
|
||||
if err := catalog.Mergers.RegisterWithSpec(spec, profileMergerConstructor(spec.Key)); err != nil {
|
||||
if err := catalog.Mergers.RegisterLegacyRawWithSpec(spec, profileMergerConstructor(spec.Key)); err != nil {
|
||||
t.Fatalf("register merger spec %#v: %v", spec, err)
|
||||
}
|
||||
case StageNormalize:
|
||||
if err := catalog.Normalizers.RegisterWithSpec(spec, profileNormalizerConstructor(spec.Key)); err != nil {
|
||||
if err := catalog.Normalizers.RegisterLegacyRawWithSpec(spec, profileNormalizerConstructor(spec.Key)); err != nil {
|
||||
t.Fatalf("register normalizer spec %#v: %v", spec, err)
|
||||
}
|
||||
case StageValidate:
|
||||
validatorSpec := ValidatorSpec{Key: spec.Key, ExecutionClass: contracts.ExecutionClassDeterministic}
|
||||
if err := catalog.Validators.RegisterWithSpec(validatorSpec, profileValidatorConstructor(spec.Key)); err != nil {
|
||||
if err := catalog.Validators.RegisterLegacyRawWithSpec(validatorSpec, profileValidatorConstructor(spec.Key)); err != nil {
|
||||
t.Fatalf("register validator spec %#v: %v", spec, err)
|
||||
}
|
||||
case StageOutput:
|
||||
@@ -1279,7 +1279,7 @@ func registerProfileSpecs(t *testing.T, catalog ModuleCatalog, specs ...ModuleSp
|
||||
|
||||
func registerProfileValidatorSpec(t *testing.T, catalog ModuleCatalog, spec ValidatorSpec) {
|
||||
t.Helper()
|
||||
if err := catalog.Validators.RegisterWithSpec(spec, profileValidatorConstructor(spec.Key)); err != nil {
|
||||
if err := catalog.Validators.RegisterLegacyRawWithSpec(spec, profileValidatorConstructor(spec.Key)); err != nil {
|
||||
t.Fatalf("register validator spec %#v: %v", spec, err)
|
||||
}
|
||||
}
|
||||
@@ -1308,26 +1308,26 @@ func profileChunkerConstructor(key string) ChunkerConstructor {
|
||||
}
|
||||
}
|
||||
|
||||
func profileExtractorConstructor(key string) ExtractorConstructor {
|
||||
return func() (contracts.Extractor, error) {
|
||||
func profileExtractorConstructor(key string) LegacyRawExtractorConstructor {
|
||||
return func() (contracts.LegacyRawExtractor, error) {
|
||||
return registryFakeExtractor{key: key}, nil
|
||||
}
|
||||
}
|
||||
|
||||
func profileMergerConstructor(key string) MergerConstructor {
|
||||
return func() (contracts.Merger, error) {
|
||||
func profileMergerConstructor(key string) LegacyRawMergerConstructor {
|
||||
return func() (contracts.LegacyRawMerger, error) {
|
||||
return registryMerger{key: key}, nil
|
||||
}
|
||||
}
|
||||
|
||||
func profileNormalizerConstructor(key string) NormalizerConstructor {
|
||||
return func() (contracts.Normalizer, error) {
|
||||
func profileNormalizerConstructor(key string) LegacyRawNormalizerConstructor {
|
||||
return func() (contracts.LegacyRawNormalizer, error) {
|
||||
return registryNormalizer{key: key}, nil
|
||||
}
|
||||
}
|
||||
|
||||
func profileValidatorConstructor(key string) ValidatorConstructor {
|
||||
return func() (contracts.Validator, error) {
|
||||
func profileValidatorConstructor(key string) LegacyRawValidatorConstructor {
|
||||
return func() (contracts.LegacyRawValidator, error) {
|
||||
return registryValidator{name: key}, nil
|
||||
}
|
||||
}
|
||||
|
||||
@@ -64,13 +64,13 @@ func integrationRegistries(t *testing.T, built, executed *[]string) Registries {
|
||||
}
|
||||
registerIntegrationExtractor(t, registries.Extractors, "extract-first", built, executed)
|
||||
registerIntegrationExtractor(t, registries.Extractors, "extract-second", built, executed)
|
||||
if err := registries.Mergers.Register("merge", func() (contracts.Merger, error) {
|
||||
if err := registries.Mergers.RegisterLegacyRaw("merge", func() (contracts.LegacyRawMerger, error) {
|
||||
*built = append(*built, "merge")
|
||||
return integrationMerger{}, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register merger: %v", err)
|
||||
}
|
||||
if err := registries.Normalizers.Register("normalize", func() (contracts.Normalizer, error) {
|
||||
if err := registries.Normalizers.RegisterLegacyRaw("normalize", func() (contracts.LegacyRawNormalizer, error) {
|
||||
*built = append(*built, "normalize")
|
||||
return integrationNormalizer{}, nil
|
||||
}); err != nil {
|
||||
@@ -88,7 +88,7 @@ func integrationRegistries(t *testing.T, built, executed *[]string) Registries {
|
||||
func registerIntegrationExtractor(t *testing.T, registry *ExtractorRegistry, key string, built, executed *[]string) {
|
||||
t.Helper()
|
||||
|
||||
if err := registry.Register(key, func() (contracts.Extractor, error) {
|
||||
if err := registry.RegisterLegacyRaw(key, func() (contracts.LegacyRawExtractor, error) {
|
||||
*built = append(*built, key)
|
||||
return integrationExtractor{key: key, executed: executed}, nil
|
||||
}); err != nil {
|
||||
|
||||
@@ -390,15 +390,15 @@ func (r *Runner) Run(ctx context.Context, input RunInput) (output RunOutput, err
|
||||
}
|
||||
|
||||
func (r *Runner) runLane(ctx context.Context, input RunInput, checkpoints CheckpointRecorder, checkpointLoader CheckpointLoader, doc *source.SourceDocument, sourceInput contracts.LLMInputMaterial, sessionID string, chunks []source.Chunk, lane ResolvedArtifactLane, output *RunOutput) error {
|
||||
extractor, err := r.registries.Extractors.Build(lane.Extract.Module)
|
||||
extractor, err := r.registries.Extractors.BuildLegacyRaw(lane.Extract.Module)
|
||||
if err != nil {
|
||||
return fmt.Errorf("build extractor %q for lane %q: %w", lane.Extract.Module, lane.ID, err)
|
||||
}
|
||||
merger, err := r.registries.Mergers.Build(lane.Merge.Module)
|
||||
merger, err := r.registries.Mergers.BuildLegacyRaw(lane.Merge.Module)
|
||||
if err != nil {
|
||||
return fmt.Errorf("build merger %q for lane %q: %w", lane.Merge.Module, lane.ID, err)
|
||||
}
|
||||
normalizer, err := r.registries.Normalizers.Build(lane.Normalize.Module)
|
||||
normalizer, err := r.registries.Normalizers.BuildLegacyRaw(lane.Normalize.Module)
|
||||
if err != nil {
|
||||
return fmt.Errorf("build normalizer %q for lane %q: %w", lane.Normalize.Module, lane.ID, err)
|
||||
}
|
||||
@@ -980,7 +980,7 @@ func (r *Runner) validateRaw(ctx context.Context, target rawValidationTarget) ([
|
||||
|
||||
var warnings []contracts.Warning
|
||||
for index, validatorBinding := range chain.Validators {
|
||||
validator, err := r.registries.Validators.Build(validatorBinding.Binding.Module)
|
||||
validator, err := r.registries.Validators.BuildLegacyRaw(validatorBinding.Binding.Module)
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("build validator %q: %w", validatorBinding.Binding.Module, err)
|
||||
}
|
||||
@@ -1133,6 +1133,9 @@ func validateRunInput(input RunInput) error {
|
||||
if lane.ID == "" {
|
||||
return fmt.Errorf("resolved pipeline artifact lane id must not be empty")
|
||||
}
|
||||
if lane.ArtifactKind != "" {
|
||||
return fmt.Errorf("resolved pipeline lane %q uses typed artifact kind %q, which the legacy raw runner cannot execute", lane.ID, lane.ArtifactKind)
|
||||
}
|
||||
if lane.Extract.Module == "" {
|
||||
return fmt.Errorf("resolved pipeline lane %q extract module must not be empty", lane.ID)
|
||||
}
|
||||
@@ -1340,11 +1343,11 @@ func attachModuleManifestMetadata(output *RunOutput, moduleKey string, module an
|
||||
|
||||
func manifestMetadataKey(module any) string {
|
||||
switch module.(type) {
|
||||
case contracts.Extractor:
|
||||
case contracts.LegacyRawExtractor:
|
||||
return "extractor"
|
||||
case contracts.Merger:
|
||||
case contracts.LegacyRawMerger:
|
||||
return "merger"
|
||||
case contracts.Normalizer:
|
||||
case contracts.LegacyRawNormalizer:
|
||||
return "normalizer"
|
||||
default:
|
||||
return ""
|
||||
|
||||
@@ -1948,7 +1948,7 @@ type runnerModules struct {
|
||||
extractors map[string]*runnerExtractor
|
||||
mergers map[string]*runnerMerger
|
||||
normalizers map[string]*runnerNormalizer
|
||||
validators map[string]contracts.Validator
|
||||
validators map[string]contracts.LegacyRawValidator
|
||||
output *runnerOutputEncoder
|
||||
inputBuildErr error
|
||||
chunkerBuildErr error
|
||||
@@ -1967,7 +1967,7 @@ func defaultRunnerModules() *runnerModules {
|
||||
normalizers: map[string]*runnerNormalizer{
|
||||
"normalize": {key: "normalize"},
|
||||
},
|
||||
validators: map[string]contracts.Validator{
|
||||
validators: map[string]contracts.LegacyRawValidator{
|
||||
"configured": &runnerValidator{name: "configured"},
|
||||
"second-validator": &runnerValidator{name: "second-validator"},
|
||||
},
|
||||
@@ -2015,26 +2015,26 @@ func newRunnerRegistries(t *testing.T, modules *runnerModules) Registries {
|
||||
}
|
||||
for key, extractor := range modules.extractors {
|
||||
extractor := extractor
|
||||
if err := registries.Extractors.Register(key, func() (contracts.Extractor, error) { return extractor, nil }); err != nil {
|
||||
if err := registries.Extractors.RegisterLegacyRaw(key, func() (contracts.LegacyRawExtractor, error) { return extractor, nil }); err != nil {
|
||||
t.Fatalf("register extractor %q: %v", key, err)
|
||||
}
|
||||
}
|
||||
for key, merger := range modules.mergers {
|
||||
merger := merger
|
||||
if err := registries.Mergers.Register(key, func() (contracts.Merger, error) { return merger, nil }); err != nil {
|
||||
if err := registries.Mergers.RegisterLegacyRaw(key, func() (contracts.LegacyRawMerger, error) { return merger, nil }); err != nil {
|
||||
t.Fatalf("register merger %q: %v", key, err)
|
||||
}
|
||||
}
|
||||
for key, normalizer := range modules.normalizers {
|
||||
normalizer := normalizer
|
||||
if err := registries.Normalizers.Register(key, func() (contracts.Normalizer, error) { return normalizer, nil }); err != nil {
|
||||
if err := registries.Normalizers.RegisterLegacyRaw(key, func() (contracts.LegacyRawNormalizer, error) { return normalizer, nil }); err != nil {
|
||||
t.Fatalf("register normalizer %q: %v", key, err)
|
||||
}
|
||||
}
|
||||
for key, validator := range modules.validators {
|
||||
validator := validator
|
||||
spec := ValidatorSpec{Key: key, ExecutionClass: validator.ExecutionClass()}
|
||||
if err := registries.Validators.RegisterWithSpec(spec, func() (contracts.Validator, error) { return validator, nil }); err != nil {
|
||||
if err := registries.Validators.RegisterLegacyRawWithSpec(spec, func() (contracts.LegacyRawValidator, error) { return validator, nil }); err != nil {
|
||||
t.Fatalf("register validator %q: %v", key, err)
|
||||
}
|
||||
}
|
||||
@@ -2653,7 +2653,7 @@ func assertRunError(t *testing.T, err error, want string) {
|
||||
}
|
||||
}
|
||||
|
||||
func resolvedValidatorForTest(validator contracts.Validator) ResolvedValidator {
|
||||
func resolvedValidatorForTest(validator contracts.LegacyRawValidator) ResolvedValidator {
|
||||
return ResolvedValidator{
|
||||
Binding: Binding(validator.Name()),
|
||||
ExecutionClass: validator.ExecutionClass(),
|
||||
|
||||
362
internal/framework/pipeline/typed_resolution_test.go
Normal file
362
internal/framework/pipeline/typed_resolution_test.go
Normal file
@@ -0,0 +1,362 @@
|
||||
package pipeline
|
||||
|
||||
import (
|
||||
"context"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
)
|
||||
|
||||
type typedTestExtractor[T any] struct{ key string }
|
||||
|
||||
func (e typedTestExtractor[T]) Key() string { return e.key }
|
||||
func (typedTestExtractor[T]) ReferenceSlots() []contracts.ReferenceSlot { return nil }
|
||||
func (typedTestExtractor[T]) Extract(context.Context, contracts.TypedExtractionRequest) (contracts.TypedExtractionResult[T], error) {
|
||||
return contracts.TypedExtractionResult[T]{}, nil
|
||||
}
|
||||
|
||||
type typedTestMerger[T any] struct{ key string }
|
||||
|
||||
func (m typedTestMerger[T]) Key() string { return m.key }
|
||||
func (typedTestMerger[T]) Merge(context.Context, contracts.TypedMergeRequest[T]) (contracts.TypedMergeResult[T], error) {
|
||||
return contracts.TypedMergeResult[T]{}, nil
|
||||
}
|
||||
|
||||
type typedTestNormalizer[T any] struct{ key string }
|
||||
|
||||
func (n typedTestNormalizer[T]) Key() string { return n.key }
|
||||
func (typedTestNormalizer[T]) ReferenceSlots() []contracts.ReferenceSlot { return nil }
|
||||
func (typedTestNormalizer[T]) Normalize(context.Context, contracts.TypedNormalizeRequest[T]) (contracts.TypedNormalizeResult[T], error) {
|
||||
return contracts.TypedNormalizeResult[T]{}, nil
|
||||
}
|
||||
|
||||
type typedTestValidator[T any] struct{ key string }
|
||||
|
||||
func (v typedTestValidator[T]) Name() string { return v.key }
|
||||
func (typedTestValidator[T]) ExecutionClass() contracts.ExecutionClass {
|
||||
return contracts.ExecutionClassDeterministic
|
||||
}
|
||||
func (typedTestValidator[T]) Validate(context.Context, contracts.TypedValidationRequest[T]) (contracts.ValidationResult, error) {
|
||||
return contracts.ValidationResult{Approved: true}, nil
|
||||
}
|
||||
|
||||
type typedTestChunkValidator struct{ key string }
|
||||
|
||||
func (v typedTestChunkValidator) Name() string { return v.key }
|
||||
func (typedTestChunkValidator) ExecutionClass() contracts.ExecutionClass {
|
||||
return contracts.ExecutionClassDeterministic
|
||||
}
|
||||
func (typedTestChunkValidator) Validate(context.Context, contracts.ChunkValidationRequest) (contracts.ValidationResult, error) {
|
||||
return contracts.ValidationResult{Approved: true}, nil
|
||||
}
|
||||
|
||||
type typedTestSerializedValidator struct{ key string }
|
||||
|
||||
func (v typedTestSerializedValidator) Name() string { return v.key }
|
||||
func (typedTestSerializedValidator) ExecutionClass() contracts.ExecutionClass {
|
||||
return contracts.ExecutionClassDeterministic
|
||||
}
|
||||
func (typedTestSerializedValidator) Validate(context.Context, contracts.SerializedValidationRequest) (contracts.ValidationResult, error) {
|
||||
return contracts.ValidationResult{Approved: true}, nil
|
||||
}
|
||||
|
||||
type typedCatalogOptions struct {
|
||||
registerNotesCodec bool
|
||||
registerScoreCodec bool
|
||||
registerNotesMerger bool
|
||||
registerScoreMerger bool
|
||||
registerNotesNormalizer bool
|
||||
registerScoreNormalizer bool
|
||||
registerScoreValidator bool
|
||||
scoreExtractorUsesNotes bool
|
||||
notesCodec testArtifactCodec[codecNotes]
|
||||
scoreCodec testArtifactCodec[codecScore]
|
||||
}
|
||||
|
||||
func completeTypedCatalogOptions() typedCatalogOptions {
|
||||
return typedCatalogOptions{
|
||||
registerNotesCodec: true,
|
||||
registerScoreCodec: true,
|
||||
registerNotesMerger: true,
|
||||
registerScoreMerger: true,
|
||||
registerNotesNormalizer: true,
|
||||
registerScoreNormalizer: true,
|
||||
registerScoreValidator: true,
|
||||
notesCodec: notesCodec(),
|
||||
scoreCodec: scoreCodec(),
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveTypedHeterogeneousLanes(t *testing.T) {
|
||||
catalog := typedResolutionCatalog(t, completeTypedCatalogOptions())
|
||||
resolved, err := ResolvePipeline(typedResolutionProfile(), ResolveOptions{}, catalog)
|
||||
if err != nil {
|
||||
t.Fatalf("ResolvePipeline() error = %v, want nil", err)
|
||||
}
|
||||
|
||||
if got, want := resolvedLaneIDs(resolved.ArtifactLanes), []string{"notes", "score"}; !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("lane order = %#v, want %#v", got, want)
|
||||
}
|
||||
assertResolvedArtifactIdentity(t, resolved.ArtifactLanes[0], "test/notes", "notes.v1")
|
||||
assertResolvedArtifactIdentity(t, resolved.ArtifactLanes[1], "test/score", "score.v1")
|
||||
|
||||
chunkChain := resolved.ValidatorChains[0]
|
||||
if got := resolvedValidatorTargets(chunkChain.Validators); !reflect.DeepEqual(got, []ValidatorTarget{ValidatorTargetChunk, ValidatorTargetSerialized}) {
|
||||
t.Fatalf("chunk validator targets = %#v, want chunk then serialized", got)
|
||||
}
|
||||
notesExtract := resolved.ValidatorChains[1]
|
||||
if got := resolvedValidatorTargets(notesExtract.Validators); !reflect.DeepEqual(got, []ValidatorTarget{ValidatorTargetTyped, ValidatorTargetSerialized}) {
|
||||
t.Fatalf("notes extract validator targets = %#v, want typed then serialized", got)
|
||||
}
|
||||
if notesExtract.Validators[0].ArtifactKind != "test/notes" || resolved.ValidatorChains[4].Validators[0].ArtifactKind != "test/score" {
|
||||
t.Fatalf("resolved validator kinds = %#v, want lane kinds", resolved.ValidatorChains)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveTypedLaneRejectsIncompatibleComposition(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
mutate func(*typedCatalogOptions)
|
||||
want string
|
||||
}{
|
||||
{name: "missing codec", mutate: func(options *typedCatalogOptions) { options.registerScoreCodec = false }, want: `artifact codec "test/score" is not registered`},
|
||||
{name: "missing merger variant", mutate: func(options *typedCatalogOptions) { options.registerScoreMerger = false }, want: `merger "typed/merge" has no typed variant for artifact kind "test/score"`},
|
||||
{name: "missing normalizer variant", mutate: func(options *typedCatalogOptions) { options.registerScoreNormalizer = false }, want: `normalizer "typed/normalize" has no typed variant for artifact kind "test/score"`},
|
||||
{name: "extractor Go type mismatch", mutate: func(options *typedCatalogOptions) { options.scoreExtractorUsesNotes = true }, want: `artifact kind "test/score" requires Go type pipeline.codecScore, got pipeline.codecNotes`},
|
||||
{name: "wrong validator kind", mutate: func(options *typedCatalogOptions) { options.registerScoreValidator = false }, want: `validator "typed/check" has no typed variant for artifact kind "test/score"; registered kinds: test/notes`},
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
options := completeTypedCatalogOptions()
|
||||
test.mutate(&options)
|
||||
_, err := ResolvePipeline(typedResolutionProfile(), ResolveOptions{}, typedResolutionCatalog(t, options))
|
||||
if err == nil || !strings.Contains(err.Error(), test.want) {
|
||||
t.Fatalf("ResolvePipeline() error = %v, want %q", err, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLegacyRawRegistrationCannotSatisfyTypedLane(t *testing.T) {
|
||||
options := completeTypedCatalogOptions()
|
||||
options.registerScoreMerger = false
|
||||
catalog := typedResolutionCatalog(t, options)
|
||||
if err := catalog.Mergers.RegisterLegacyRaw("typed/merge", func() (contracts.LegacyRawMerger, error) { return nil, nil }); err != nil {
|
||||
t.Fatalf("RegisterLegacyRaw() error = %v, want nil", err)
|
||||
}
|
||||
|
||||
_, err := ResolvePipeline(typedResolutionProfile(), ResolveOptions{}, catalog)
|
||||
if err == nil || !strings.Contains(err.Error(), `no typed variant for artifact kind "test/score"`) {
|
||||
t.Fatalf("ResolvePipeline() error = %v, want typed variant error", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTypedVariantRegistrationRejectsDuplicates(t *testing.T) {
|
||||
registry := NewMergerRegistry()
|
||||
spec := ModuleSpec{Key: "typed/merge", Stage: StageMerge, ArtifactKind: "test/notes"}
|
||||
constructor := func() (contracts.Merger[codecNotes], error) {
|
||||
return typedTestMerger[codecNotes]{key: "typed/merge"}, nil
|
||||
}
|
||||
if err := RegisterMerger(registry, spec, constructor); err != nil {
|
||||
t.Fatalf("RegisterMerger() error = %v, want nil", err)
|
||||
}
|
||||
if err := RegisterMerger(registry, spec, constructor); err == nil || !strings.Contains(err.Error(), "already registered") {
|
||||
t.Fatalf("duplicate RegisterMerger() error = %v, want duplicate variant error", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolvedPipelineDigestIncludesArtifactSchemaIdentity(t *testing.T) {
|
||||
baseOptions := completeTypedCatalogOptions()
|
||||
base, err := ResolvePipeline(typedResolutionProfile(), ResolveOptions{}, typedResolutionCatalog(t, baseOptions))
|
||||
if err != nil {
|
||||
t.Fatalf("ResolvePipeline(base) error = %v, want nil", err)
|
||||
}
|
||||
|
||||
identityOptions := completeTypedCatalogOptions()
|
||||
identityOptions.notesCodec.schema.ID = "notes-renamed.v1"
|
||||
identity, err := ResolvePipeline(typedResolutionProfile(), ResolveOptions{}, typedResolutionCatalog(t, identityOptions))
|
||||
if err != nil {
|
||||
t.Fatalf("ResolvePipeline(identity) error = %v, want nil", err)
|
||||
}
|
||||
if base.Digest == identity.Digest {
|
||||
t.Fatalf("pipeline digest = %q after schema identity change, want different digest", identity.Digest)
|
||||
}
|
||||
|
||||
digestOptions := completeTypedCatalogOptions()
|
||||
digestOptions.notesCodec.schema.JSONSchema = []byte(`{"additionalProperties":false,"description":"changed","properties":{"items":{"items":{"type":"string"},"type":"array"}},"required":["items"],"type":"object"}`)
|
||||
changedSchema, err := ResolvePipeline(typedResolutionProfile(), ResolveOptions{}, typedResolutionCatalog(t, digestOptions))
|
||||
if err != nil {
|
||||
t.Fatalf("ResolvePipeline(schema bytes) error = %v, want nil", err)
|
||||
}
|
||||
if base.Digest == changedSchema.Digest {
|
||||
t.Fatalf("pipeline digest = %q after schema digest change, want different digest", changedSchema.Digest)
|
||||
}
|
||||
}
|
||||
|
||||
func typedResolutionCatalog(t *testing.T, options typedCatalogOptions) ModuleCatalog {
|
||||
t.Helper()
|
||||
catalog := ModuleCatalog{
|
||||
Inputs: NewInputAdapterRegistry(),
|
||||
Chunkers: NewChunkerRegistry(),
|
||||
ArtifactCodecs: NewArtifactCodecRegistry(),
|
||||
Extractors: NewExtractorRegistry(),
|
||||
Mergers: NewMergerRegistry(),
|
||||
Normalizers: NewNormalizerRegistry(),
|
||||
Validators: NewValidatorRegistry(),
|
||||
ValidatorChains: NewValidatorChainRegistry(),
|
||||
Outputs: NewOutputEncoderRegistry(),
|
||||
}
|
||||
mustRegisterTypedTestBase(t, catalog)
|
||||
if options.registerNotesCodec {
|
||||
mustRegisterArtifactCodec(t, catalog.ArtifactCodecs, options.notesCodec)
|
||||
}
|
||||
if options.registerScoreCodec {
|
||||
mustRegisterArtifactCodec(t, catalog.ArtifactCodecs, options.scoreCodec)
|
||||
}
|
||||
mustRegisterTypedExtractor(t, catalog.Extractors, "typed/extract-notes", "test/notes", typedTestExtractor[codecNotes]{key: "typed/extract-notes"})
|
||||
if options.scoreExtractorUsesNotes {
|
||||
mustRegisterTypedExtractor(t, catalog.Extractors, "typed/extract-score", "test/score", typedTestExtractor[codecNotes]{key: "typed/extract-score"})
|
||||
} else {
|
||||
mustRegisterTypedExtractor(t, catalog.Extractors, "typed/extract-score", "test/score", typedTestExtractor[codecScore]{key: "typed/extract-score"})
|
||||
}
|
||||
if options.registerNotesMerger {
|
||||
mustRegisterTypedMerger(t, catalog.Mergers, "test/notes", typedTestMerger[codecNotes]{key: "typed/merge"})
|
||||
}
|
||||
if options.registerScoreMerger {
|
||||
mustRegisterTypedMerger(t, catalog.Mergers, "test/score", typedTestMerger[codecScore]{key: "typed/merge"})
|
||||
}
|
||||
if options.registerNotesNormalizer {
|
||||
mustRegisterTypedNormalizer(t, catalog.Normalizers, "test/notes", typedTestNormalizer[codecNotes]{key: "typed/normalize"})
|
||||
}
|
||||
if options.registerScoreNormalizer {
|
||||
mustRegisterTypedNormalizer(t, catalog.Normalizers, "test/score", typedTestNormalizer[codecScore]{key: "typed/normalize"})
|
||||
}
|
||||
mustRegisterTypedValidator(t, catalog.Validators, "test/notes", typedTestValidator[codecNotes]{key: "typed/check"})
|
||||
if options.registerScoreValidator {
|
||||
mustRegisterTypedValidator(t, catalog.Validators, "test/score", typedTestValidator[codecScore]{key: "typed/check"})
|
||||
}
|
||||
if err := RegisterChunkValidator(catalog.Validators, ValidatorSpec{Key: "chunk/check", ExecutionClass: contracts.ExecutionClassDeterministic}, func() (contracts.ChunkValidator, error) {
|
||||
return typedTestChunkValidator{key: "chunk/check"}, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("RegisterChunkValidator() error = %v", err)
|
||||
}
|
||||
if err := RegisterSerializedValidator(catalog.Validators, SerializedValidatorSpec{
|
||||
ValidatorSpec: ValidatorSpec{Key: "serialized/check", ExecutionClass: contracts.ExecutionClassDeterministic},
|
||||
SupportsArtifacts: true,
|
||||
}, func() (contracts.SerializedValidator, error) {
|
||||
return typedTestSerializedValidator{key: "serialized/check"}, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("RegisterSerializedValidator() error = %v", err)
|
||||
}
|
||||
if err := RegisterSerializedValidator(catalog.Validators, SerializedValidatorSpec{
|
||||
ValidatorSpec: ValidatorSpec{Key: "serialized/chunks", ExecutionClass: contracts.ExecutionClassDeterministic},
|
||||
SupportsChunks: true,
|
||||
}, func() (contracts.SerializedValidator, error) {
|
||||
return typedTestSerializedValidator{key: "serialized/chunks"}, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("RegisterSerializedValidator(chunks) error = %v", err)
|
||||
}
|
||||
return catalog
|
||||
}
|
||||
|
||||
func mustRegisterTypedTestBase(t *testing.T, catalog ModuleCatalog) {
|
||||
t.Helper()
|
||||
if err := catalog.Inputs.RegisterWithSpec(ModuleSpec{Key: "typed/input", Stage: StageInput}, func() (contracts.InputAdapter, error) { return nil, nil }); err != nil {
|
||||
t.Fatalf("register input: %v", err)
|
||||
}
|
||||
if err := catalog.Chunkers.RegisterWithSpec(ModuleSpec{Key: "typed/chunk", Stage: StageChunk}, func() (contracts.Chunker, error) { return nil, nil }); err != nil {
|
||||
t.Fatalf("register chunker: %v", err)
|
||||
}
|
||||
if err := catalog.Outputs.RegisterWithSpec(ModuleSpec{Key: "typed/output", Stage: StageOutput}, func() (contracts.OutputEncoder, error) { return nil, nil }); err != nil {
|
||||
t.Fatalf("register output: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func mustRegisterArtifactCodec[T any](t *testing.T, registry *ArtifactCodecRegistry, codec contracts.ArtifactCodec[T]) {
|
||||
t.Helper()
|
||||
if err := RegisterArtifactCodec(registry, codec); err != nil {
|
||||
t.Fatalf("RegisterArtifactCodec() error = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func mustRegisterTypedExtractor[T any](t *testing.T, registry *ExtractorRegistry, key string, kind contracts.ArtifactKind, extractor contracts.Extractor[T]) {
|
||||
t.Helper()
|
||||
if err := RegisterExtractor(registry, ModuleSpec{Key: key, Stage: StageExtract, ArtifactKind: kind}, func() (contracts.Extractor[T], error) { return extractor, nil }); err != nil {
|
||||
t.Fatalf("RegisterExtractor() error = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func mustRegisterTypedMerger[T any](t *testing.T, registry *MergerRegistry, kind contracts.ArtifactKind, merger contracts.Merger[T]) {
|
||||
t.Helper()
|
||||
if err := RegisterMerger(registry, ModuleSpec{Key: "typed/merge", Stage: StageMerge, ArtifactKind: kind}, func() (contracts.Merger[T], error) { return merger, nil }); err != nil {
|
||||
t.Fatalf("RegisterMerger() error = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func mustRegisterTypedNormalizer[T any](t *testing.T, registry *NormalizerRegistry, kind contracts.ArtifactKind, normalizer contracts.Normalizer[T]) {
|
||||
t.Helper()
|
||||
if err := RegisterNormalizer(registry, ModuleSpec{Key: "typed/normalize", Stage: StageNormalize, ArtifactKind: kind}, func() (contracts.Normalizer[T], error) { return normalizer, nil }); err != nil {
|
||||
t.Fatalf("RegisterNormalizer() error = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func mustRegisterTypedValidator[T any](t *testing.T, registry *ValidatorRegistry, kind contracts.ArtifactKind, validator contracts.TypedValidator[T]) {
|
||||
t.Helper()
|
||||
if err := RegisterTypedValidator(registry, kind, ValidatorSpec{Key: "typed/check", ExecutionClass: contracts.ExecutionClassDeterministic}, func() (contracts.TypedValidator[T], error) { return validator, nil }); err != nil {
|
||||
t.Fatalf("RegisterTypedValidator() error = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func typedResolutionProfile() PipelineProfile {
|
||||
validatorOverride := ValidatorOverride{Set: true, Validators: []ModuleBinding{Binding("typed/check"), Binding("serialized/check")}}
|
||||
return PipelineProfile{
|
||||
ID: "typed-pipeline",
|
||||
Input: Binding("typed/input"),
|
||||
Chunk: ModuleBinding{Module: "typed/chunk", Validators: ValidatorOverride{Set: true, Validators: []ModuleBinding{Binding("chunk/check"), Binding("serialized/chunks")}}},
|
||||
Artifacts: map[string]ArtifactLaneProfile{
|
||||
"score": typedLaneProfile("typed/extract-score", validatorOverride),
|
||||
"notes": typedLaneProfile("typed/extract-notes", validatorOverride),
|
||||
},
|
||||
Output: Binding("typed/output"),
|
||||
}
|
||||
}
|
||||
|
||||
func typedLaneProfile(extractor string, validators ValidatorOverride) ArtifactLaneProfile {
|
||||
return ArtifactLaneProfile{
|
||||
Extract: ModuleBinding{Module: extractor, Validators: validators},
|
||||
Merge: Binding("typed/merge"),
|
||||
Normalize: Binding("typed/normalize"),
|
||||
}
|
||||
}
|
||||
|
||||
func assertResolvedArtifactIdentity(t *testing.T, lane ResolvedArtifactLane, kind contracts.ArtifactKind, schemaID string) {
|
||||
t.Helper()
|
||||
if lane.ArtifactKind != kind || lane.ArtifactSchemaID != schemaID || lane.ArtifactSchemaName == "" || lane.ArtifactSchemaVersion == "" || lane.ArtifactSchemaDigest == "" {
|
||||
t.Fatalf("resolved lane identity = %#v, want kind %q schema %q with complete metadata", lane, kind, schemaID)
|
||||
}
|
||||
}
|
||||
|
||||
func resolvedLaneIDs(lanes []ResolvedArtifactLane) []string {
|
||||
ids := make([]string, len(lanes))
|
||||
for i, lane := range lanes {
|
||||
ids[i] = lane.ID
|
||||
}
|
||||
return ids
|
||||
}
|
||||
|
||||
func resolvedValidatorTargets(validators []ResolvedValidator) []ValidatorTarget {
|
||||
targets := make([]ValidatorTarget, len(validators))
|
||||
for i, validator := range validators {
|
||||
targets[i] = validator.Target
|
||||
}
|
||||
return targets
|
||||
}
|
||||
|
||||
var _ contracts.Extractor[codecNotes] = typedTestExtractor[codecNotes]{}
|
||||
var _ contracts.Merger[codecNotes] = typedTestMerger[codecNotes]{}
|
||||
var _ contracts.Normalizer[codecNotes] = typedTestNormalizer[codecNotes]{}
|
||||
var _ contracts.TypedValidator[codecNotes] = typedTestValidator[codecNotes]{}
|
||||
var _ contracts.ChunkValidator = typedTestChunkValidator{}
|
||||
var _ contracts.SerializedValidator = typedTestSerializedValidator{}
|
||||
@@ -2,40 +2,78 @@ package pipeline
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"reflect"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
|
||||
)
|
||||
|
||||
type ValidatorConstructor func() (contracts.Validator, error)
|
||||
type LegacyRawValidatorConstructor func() (contracts.LegacyRawValidator, error)
|
||||
|
||||
type ValidatorSpec struct {
|
||||
Key string `json:"key"`
|
||||
ExecutionClass contracts.ExecutionClass `json:"execution_class"`
|
||||
}
|
||||
|
||||
type SerializedValidatorSpec struct {
|
||||
ValidatorSpec
|
||||
SupportsChunks bool `json:"supports_chunks,omitempty"`
|
||||
SupportsArtifacts bool `json:"supports_artifacts,omitempty"`
|
||||
}
|
||||
|
||||
type ValidatorTarget string
|
||||
|
||||
const (
|
||||
ValidatorTargetLegacyRaw ValidatorTarget = "legacy_raw"
|
||||
ValidatorTargetChunk ValidatorTarget = "chunk"
|
||||
ValidatorTargetSerialized ValidatorTarget = "serialized"
|
||||
ValidatorTargetTyped ValidatorTarget = "typed"
|
||||
)
|
||||
|
||||
type ValidatorRegistry struct {
|
||||
constructors map[string]ValidatorConstructor
|
||||
specs map[string]ValidatorSpec
|
||||
legacyConstructors map[string]LegacyRawValidatorConstructor
|
||||
legacySpecs map[string]ValidatorSpec
|
||||
typedEntries map[artifactVariantKey]typedValidatorEntry
|
||||
chunkEntries map[string]chunkValidatorEntry
|
||||
serializedEntries map[string]serializedValidatorEntry
|
||||
}
|
||||
|
||||
type typedValidatorEntry struct {
|
||||
spec ValidatorSpec
|
||||
kind contracts.ArtifactKind
|
||||
valueType reflect.Type
|
||||
constructor func() (any, error)
|
||||
}
|
||||
|
||||
type chunkValidatorEntry struct {
|
||||
spec ValidatorSpec
|
||||
constructor func() (contracts.ChunkValidator, error)
|
||||
}
|
||||
|
||||
type serializedValidatorEntry struct {
|
||||
spec SerializedValidatorSpec
|
||||
constructor func() (contracts.SerializedValidator, error)
|
||||
}
|
||||
|
||||
func NewValidatorRegistry() *ValidatorRegistry {
|
||||
return &ValidatorRegistry{
|
||||
constructors: make(map[string]ValidatorConstructor),
|
||||
specs: make(map[string]ValidatorSpec),
|
||||
legacyConstructors: make(map[string]LegacyRawValidatorConstructor),
|
||||
legacySpecs: make(map[string]ValidatorSpec),
|
||||
typedEntries: make(map[artifactVariantKey]typedValidatorEntry),
|
||||
chunkEntries: make(map[string]chunkValidatorEntry),
|
||||
serializedEntries: make(map[string]serializedValidatorEntry),
|
||||
}
|
||||
}
|
||||
|
||||
func (r *ValidatorRegistry) Register(key string, constructor ValidatorConstructor) error {
|
||||
return r.RegisterWithSpec(ValidatorSpec{Key: key, ExecutionClass: contracts.ExecutionClassDeterministic}, constructor)
|
||||
func (r *ValidatorRegistry) RegisterLegacyRaw(key string, constructor LegacyRawValidatorConstructor) error {
|
||||
return r.RegisterLegacyRawWithSpec(ValidatorSpec{Key: key, ExecutionClass: contracts.ExecutionClassDeterministic}, constructor)
|
||||
}
|
||||
|
||||
func (r *ValidatorRegistry) RegisterWithSpec(spec ValidatorSpec, constructor ValidatorConstructor) error {
|
||||
func (r *ValidatorRegistry) RegisterLegacyRawWithSpec(spec ValidatorSpec, constructor LegacyRawValidatorConstructor) error {
|
||||
if r == nil {
|
||||
return fmt.Errorf("validator registry must not be nil")
|
||||
}
|
||||
|
||||
normalizedSpec, err := normalizeValidatorSpec(spec)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -43,36 +81,111 @@ func (r *ValidatorRegistry) RegisterWithSpec(spec ValidatorSpec, constructor Val
|
||||
if constructor == nil {
|
||||
return fmt.Errorf("validator constructor for %q must not be nil", normalizedSpec.Key)
|
||||
}
|
||||
if _, ok := r.constructors[normalizedSpec.Key]; ok {
|
||||
return fmt.Errorf("validator %q is already registered", normalizedSpec.Key)
|
||||
if _, ok := r.legacyConstructors[normalizedSpec.Key]; ok {
|
||||
return fmt.Errorf("legacy raw validator %q is already registered", normalizedSpec.Key)
|
||||
}
|
||||
|
||||
if r.constructors == nil {
|
||||
r.constructors = make(map[string]ValidatorConstructor)
|
||||
if r.legacyConstructors == nil {
|
||||
r.legacyConstructors = make(map[string]LegacyRawValidatorConstructor)
|
||||
}
|
||||
if r.specs == nil {
|
||||
r.specs = make(map[string]ValidatorSpec)
|
||||
if r.legacySpecs == nil {
|
||||
r.legacySpecs = make(map[string]ValidatorSpec)
|
||||
}
|
||||
r.constructors[normalizedSpec.Key] = constructor
|
||||
r.specs[normalizedSpec.Key] = normalizedSpec
|
||||
r.legacyConstructors[normalizedSpec.Key] = constructor
|
||||
r.legacySpecs[normalizedSpec.Key] = normalizedSpec
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *ValidatorRegistry) Build(key string) (contracts.Validator, error) {
|
||||
func RegisterTypedValidator[T any](registry *ValidatorRegistry, kind contracts.ArtifactKind, spec ValidatorSpec, constructor func() (contracts.TypedValidator[T], error)) error {
|
||||
if registry == nil {
|
||||
return fmt.Errorf("validator registry must not be nil")
|
||||
}
|
||||
normalizedSpec, err := normalizeValidatorSpec(spec)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
kind = normalizeArtifactKind(kind)
|
||||
if kind == "" {
|
||||
return fmt.Errorf("typed validator %q artifact kind must not be empty", normalizedSpec.Key)
|
||||
}
|
||||
if constructor == nil {
|
||||
return fmt.Errorf("validator constructor for %q must not be nil", normalizedSpec.Key)
|
||||
}
|
||||
key := artifactVariantKey{module: normalizedSpec.Key, kind: kind}
|
||||
if _, ok := registry.typedEntries[key]; ok {
|
||||
return fmt.Errorf("validator %q variant for artifact kind %q is already registered", key.module, key.kind)
|
||||
}
|
||||
if registry.typedEntries == nil {
|
||||
registry.typedEntries = make(map[artifactVariantKey]typedValidatorEntry)
|
||||
}
|
||||
registry.typedEntries[key] = typedValidatorEntry{
|
||||
spec: normalizedSpec,
|
||||
kind: kind,
|
||||
valueType: reflect.TypeFor[T](),
|
||||
constructor: func() (any, error) {
|
||||
return constructor()
|
||||
},
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func RegisterChunkValidator(registry *ValidatorRegistry, spec ValidatorSpec, constructor func() (contracts.ChunkValidator, error)) error {
|
||||
if registry == nil {
|
||||
return fmt.Errorf("validator registry must not be nil")
|
||||
}
|
||||
normalizedSpec, err := normalizeValidatorSpec(spec)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if constructor == nil {
|
||||
return fmt.Errorf("validator constructor for %q must not be nil", normalizedSpec.Key)
|
||||
}
|
||||
if _, ok := registry.chunkEntries[normalizedSpec.Key]; ok {
|
||||
return fmt.Errorf("chunk validator %q is already registered", normalizedSpec.Key)
|
||||
}
|
||||
if registry.chunkEntries == nil {
|
||||
registry.chunkEntries = make(map[string]chunkValidatorEntry)
|
||||
}
|
||||
registry.chunkEntries[normalizedSpec.Key] = chunkValidatorEntry{spec: normalizedSpec, constructor: constructor}
|
||||
return nil
|
||||
}
|
||||
|
||||
func RegisterSerializedValidator(registry *ValidatorRegistry, spec SerializedValidatorSpec, constructor func() (contracts.SerializedValidator, error)) error {
|
||||
if registry == nil {
|
||||
return fmt.Errorf("validator registry must not be nil")
|
||||
}
|
||||
normalizedValidatorSpec, err := normalizeValidatorSpec(spec.ValidatorSpec)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
spec.ValidatorSpec = normalizedValidatorSpec
|
||||
if !spec.SupportsChunks && !spec.SupportsArtifacts {
|
||||
return fmt.Errorf("serialized validator %q must support chunks, artifacts, or both", spec.Key)
|
||||
}
|
||||
if constructor == nil {
|
||||
return fmt.Errorf("validator constructor for %q must not be nil", spec.Key)
|
||||
}
|
||||
if _, ok := registry.serializedEntries[spec.Key]; ok {
|
||||
return fmt.Errorf("serialized validator %q is already registered", spec.Key)
|
||||
}
|
||||
if registry.serializedEntries == nil {
|
||||
registry.serializedEntries = make(map[string]serializedValidatorEntry)
|
||||
}
|
||||
registry.serializedEntries[spec.Key] = serializedValidatorEntry{spec: spec, constructor: constructor}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *ValidatorRegistry) BuildLegacyRaw(key string) (contracts.LegacyRawValidator, error) {
|
||||
if r == nil {
|
||||
return nil, fmt.Errorf("validator registry must not be nil")
|
||||
}
|
||||
|
||||
normalizedKey := strings.TrimSpace(key)
|
||||
if normalizedKey == "" {
|
||||
return nil, fmt.Errorf("validator key must not be empty")
|
||||
}
|
||||
|
||||
constructor, ok := r.constructors[normalizedKey]
|
||||
constructor, ok := r.legacyConstructors[normalizedKey]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("validator %q is not registered", normalizedKey)
|
||||
return nil, fmt.Errorf("legacy raw validator %q is not registered", normalizedKey)
|
||||
}
|
||||
|
||||
validator, err := constructor()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("build validator %q: %w", normalizedKey, err)
|
||||
@@ -83,14 +196,10 @@ func (r *ValidatorRegistry) Build(key string) (contracts.Validator, error) {
|
||||
if validator.Name() != normalizedKey {
|
||||
return nil, fmt.Errorf("validator %q returned name %q", normalizedKey, validator.Name())
|
||||
}
|
||||
spec, ok := r.specs[normalizedKey]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("validator %q spec is not registered", normalizedKey)
|
||||
}
|
||||
spec := r.legacySpecs[normalizedKey]
|
||||
if validator.ExecutionClass() != spec.ExecutionClass {
|
||||
return nil, fmt.Errorf("validator %q returned execution class %q, want %q", normalizedKey, validator.ExecutionClass(), spec.ExecutionClass)
|
||||
}
|
||||
|
||||
return validator, nil
|
||||
}
|
||||
|
||||
@@ -98,28 +207,57 @@ func (r *ValidatorRegistry) Spec(key string) (ValidatorSpec, bool) {
|
||||
if r == nil {
|
||||
return ValidatorSpec{}, false
|
||||
}
|
||||
spec, ok := r.legacySpecs[strings.TrimSpace(key)]
|
||||
return spec, ok
|
||||
}
|
||||
|
||||
spec, ok := r.specs[strings.TrimSpace(key)]
|
||||
if !ok {
|
||||
return ValidatorSpec{}, false
|
||||
func (r *ValidatorRegistry) typedEntry(key string, kind contracts.ArtifactKind) (typedValidatorEntry, bool) {
|
||||
if r == nil {
|
||||
return typedValidatorEntry{}, false
|
||||
}
|
||||
return spec, true
|
||||
entry, ok := r.typedEntries[artifactVariantKey{module: strings.TrimSpace(key), kind: normalizeArtifactKind(kind)}]
|
||||
return entry, ok
|
||||
}
|
||||
|
||||
func (r *ValidatorRegistry) chunkEntry(key string) (chunkValidatorEntry, bool) {
|
||||
if r == nil {
|
||||
return chunkValidatorEntry{}, false
|
||||
}
|
||||
entry, ok := r.chunkEntries[strings.TrimSpace(key)]
|
||||
return entry, ok
|
||||
}
|
||||
|
||||
func (r *ValidatorRegistry) serializedEntry(key string) (serializedValidatorEntry, bool) {
|
||||
if r == nil {
|
||||
return serializedValidatorEntry{}, false
|
||||
}
|
||||
entry, ok := r.serializedEntries[strings.TrimSpace(key)]
|
||||
return entry, ok
|
||||
}
|
||||
|
||||
func (r *ValidatorRegistry) registeredTypedKinds(key string) []contracts.ArtifactKind {
|
||||
if r == nil {
|
||||
return nil
|
||||
}
|
||||
module := strings.TrimSpace(key)
|
||||
kinds := make([]contracts.ArtifactKind, 0)
|
||||
for variant := range r.typedEntries {
|
||||
if variant.module == module {
|
||||
kinds = append(kinds, variant.kind)
|
||||
}
|
||||
}
|
||||
sortArtifactKinds(kinds)
|
||||
return kinds
|
||||
}
|
||||
|
||||
func (r *ValidatorRegistry) RegisteredSpecs() []ValidatorSpec {
|
||||
if r == nil || len(r.specs) == 0 {
|
||||
if r == nil || len(r.legacySpecs) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
keys := make([]string, 0, len(r.specs))
|
||||
for key := range r.specs {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
|
||||
keys := sortedRegistryKeys(r.legacySpecs)
|
||||
specs := make([]ValidatorSpec, 0, len(keys))
|
||||
for _, key := range keys {
|
||||
specs = append(specs, r.specs[key])
|
||||
specs = append(specs, r.legacySpecs[key])
|
||||
}
|
||||
return specs
|
||||
}
|
||||
@@ -128,15 +266,24 @@ func (r *ValidatorRegistry) RegisteredKeys() []string {
|
||||
if r == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
return sortedRegistryKeys(r.constructors)
|
||||
keys := make(map[string]struct{})
|
||||
for key := range r.legacySpecs {
|
||||
keys[key] = struct{}{}
|
||||
}
|
||||
for key := range r.typedEntries {
|
||||
keys[key.module] = struct{}{}
|
||||
}
|
||||
for key := range r.chunkEntries {
|
||||
keys[key] = struct{}{}
|
||||
}
|
||||
for key := range r.serializedEntries {
|
||||
keys[key] = struct{}{}
|
||||
}
|
||||
return sortedRegistryKeys(keys)
|
||||
}
|
||||
|
||||
func normalizeValidatorSpec(spec ValidatorSpec) (ValidatorSpec, error) {
|
||||
normalized := ValidatorSpec{
|
||||
Key: strings.TrimSpace(spec.Key),
|
||||
ExecutionClass: spec.ExecutionClass,
|
||||
}
|
||||
normalized := ValidatorSpec{Key: strings.TrimSpace(spec.Key), ExecutionClass: spec.ExecutionClass}
|
||||
if normalized.Key == "" {
|
||||
return ValidatorSpec{}, fmt.Errorf("validator key must not be empty")
|
||||
}
|
||||
@@ -147,3 +294,7 @@ func normalizeValidatorSpec(spec ValidatorSpec) (ValidatorSpec, error) {
|
||||
}
|
||||
return normalized, nil
|
||||
}
|
||||
|
||||
func sortValidatorSpecs(specs []ValidatorSpec) {
|
||||
sort.Slice(specs, func(i, j int) bool { return specs[i].Key < specs[j].Key })
|
||||
}
|
||||
|
||||
@@ -11,11 +11,11 @@ import (
|
||||
|
||||
func TestValidatorRegistryBehavior(t *testing.T) {
|
||||
registry := NewValidatorRegistry()
|
||||
if err := registry.Register(" generic-validator ", validatorConstructor("generic-validator", contracts.ExecutionClassDeterministic)); err != nil {
|
||||
if err := registry.RegisterLegacyRaw(" generic-validator ", validatorConstructor("generic-validator", contracts.ExecutionClassDeterministic)); err != nil {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
|
||||
validator, err := registry.Build("generic-validator")
|
||||
validator, err := registry.BuildLegacyRaw("generic-validator")
|
||||
if err != nil {
|
||||
t.Fatalf("Build() error = %v, want nil", err)
|
||||
}
|
||||
@@ -36,7 +36,7 @@ func TestValidatorRegistryBehavior(t *testing.T) {
|
||||
func TestValidatorRegistryRegistersSpecs(t *testing.T) {
|
||||
registry := NewValidatorRegistry()
|
||||
spec := ValidatorSpec{Key: " llm-validator ", ExecutionClass: contracts.ExecutionClassLLMBacked}
|
||||
if err := registry.RegisterWithSpec(spec, validatorConstructor("llm-validator", contracts.ExecutionClassLLMBacked)); err != nil {
|
||||
if err := registry.RegisterLegacyRawWithSpec(spec, validatorConstructor("llm-validator", contracts.ExecutionClassLLMBacked)); err != nil {
|
||||
t.Fatalf("RegisterWithSpec() error = %v, want nil", err)
|
||||
}
|
||||
|
||||
@@ -53,7 +53,7 @@ func TestValidatorRegistryRegistersSpecs(t *testing.T) {
|
||||
func TestValidatorRegistryRegisteredSpecsAreSorted(t *testing.T) {
|
||||
registry := NewValidatorRegistry()
|
||||
for _, key := range []string{"zeta", "alpha"} {
|
||||
if err := registry.Register(key, validatorConstructor(key, contracts.ExecutionClassDeterministic)); err != nil {
|
||||
if err := registry.RegisterLegacyRaw(key, validatorConstructor(key, contracts.ExecutionClassDeterministic)); err != nil {
|
||||
t.Fatalf("Register(%q) error = %v", key, err)
|
||||
}
|
||||
}
|
||||
@@ -66,7 +66,7 @@ func TestValidatorRegistryRegisteredSpecsAreSorted(t *testing.T) {
|
||||
|
||||
func TestValidatorRegistryRejectsUnsupportedExecutionClass(t *testing.T) {
|
||||
registry := NewValidatorRegistry()
|
||||
err := registry.RegisterWithSpec(
|
||||
err := registry.RegisterLegacyRawWithSpec(
|
||||
ValidatorSpec{Key: "invalid-validator", ExecutionClass: contracts.ExecutionClass("unsupported")},
|
||||
validatorConstructor("invalid-validator", contracts.ExecutionClass("unsupported")),
|
||||
)
|
||||
@@ -77,14 +77,14 @@ func TestValidatorRegistryRejectsUnsupportedExecutionClass(t *testing.T) {
|
||||
|
||||
func TestValidatorRegistryRejectsConstructorExecutionClassMismatch(t *testing.T) {
|
||||
registry := NewValidatorRegistry()
|
||||
if err := registry.RegisterWithSpec(
|
||||
if err := registry.RegisterLegacyRawWithSpec(
|
||||
ValidatorSpec{Key: "validator", ExecutionClass: contracts.ExecutionClassDeterministic},
|
||||
validatorConstructor("validator", contracts.ExecutionClassLLMBacked),
|
||||
); err != nil {
|
||||
t.Fatalf("RegisterWithSpec() error = %v, want nil", err)
|
||||
}
|
||||
|
||||
_, err := registry.Build("validator")
|
||||
_, err := registry.BuildLegacyRaw("validator")
|
||||
if err == nil {
|
||||
t.Fatal("Build() error = nil, want execution class mismatch")
|
||||
}
|
||||
@@ -98,8 +98,8 @@ type testValidator struct {
|
||||
executionClass contracts.ExecutionClass
|
||||
}
|
||||
|
||||
func validatorConstructor(name string, executionClass contracts.ExecutionClass) ValidatorConstructor {
|
||||
return func() (contracts.Validator, error) {
|
||||
func validatorConstructor(name string, executionClass contracts.ExecutionClass) LegacyRawValidatorConstructor {
|
||||
return func() (contracts.LegacyRawValidator, error) {
|
||||
return testValidator{name: name, executionClass: executionClass}, nil
|
||||
}
|
||||
}
|
||||
|
||||
@@ -51,12 +51,12 @@ func TestWalkingSkeletonFixture(t *testing.T) {
|
||||
func TestWalkingSkeletonResolutionRejectsMissingCapability(t *testing.T) {
|
||||
catalog := walkingSkeletonCatalog(t)
|
||||
catalog.Extractors = NewExtractorRegistry()
|
||||
if err := catalog.Extractors.RegisterWithSpec(ModuleSpec{
|
||||
if err := catalog.Extractors.RegisterLegacyRawWithSpec(ModuleSpec{
|
||||
Key: "fake/extract",
|
||||
Stage: StageExtract,
|
||||
Requires: []string{"missing"},
|
||||
Provides: []string{"fake_artifacts"},
|
||||
}, func() (contracts.Extractor, error) {
|
||||
}, func() (contracts.LegacyRawExtractor, error) {
|
||||
return walkingSkeletonExtractor{}, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("RegisterWithSpec() error = %v, want nil", err)
|
||||
@@ -125,29 +125,29 @@ func walkingSkeletonCatalog(t *testing.T) ModuleCatalog {
|
||||
}); err != nil {
|
||||
t.Fatalf("register fake chunker: %v", err)
|
||||
}
|
||||
if err := catalog.Extractors.RegisterWithSpec(ModuleSpec{
|
||||
if err := catalog.Extractors.RegisterLegacyRawWithSpec(ModuleSpec{
|
||||
Key: "fake/extract",
|
||||
Stage: StageExtract,
|
||||
Requires: []string{"chunks"},
|
||||
Provides: []string{"fake_artifacts"},
|
||||
}, func() (contracts.Extractor, error) {
|
||||
}, func() (contracts.LegacyRawExtractor, error) {
|
||||
return walkingSkeletonExtractor{}, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register fake extractor: %v", err)
|
||||
}
|
||||
if err := catalog.Mergers.RegisterWithSpec(ModuleSpec{
|
||||
if err := catalog.Mergers.RegisterLegacyRawWithSpec(ModuleSpec{
|
||||
Key: DefaultMergeModule,
|
||||
Stage: StageMerge,
|
||||
Requires: []string{"fake_artifacts"},
|
||||
}, func() (contracts.Merger, error) {
|
||||
}, func() (contracts.LegacyRawMerger, error) {
|
||||
return walkingSkeletonMerger{}, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register append-order merger: %v", err)
|
||||
}
|
||||
if err := catalog.Normalizers.RegisterWithSpec(ModuleSpec{
|
||||
if err := catalog.Normalizers.RegisterLegacyRawWithSpec(ModuleSpec{
|
||||
Key: DefaultNormalizeModule,
|
||||
Stage: StageNormalize,
|
||||
}, func() (contracts.Normalizer, error) {
|
||||
}, func() (contracts.LegacyRawNormalizer, error) {
|
||||
return walkingSkeletonNormalizer{}, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register no-op normalizer: %v", err)
|
||||
|
||||
@@ -31,7 +31,7 @@ var referenceSlotDescriptions = shared.ReferenceSlotDescriptions{
|
||||
Roster: "Deprecated alias for party roster reference material used only for disambiguation.",
|
||||
}
|
||||
|
||||
var _ contracts.Extractor = (*Extractor)(nil)
|
||||
var _ contracts.LegacyRawExtractor = (*Extractor)(nil)
|
||||
|
||||
type Extractor struct{}
|
||||
|
||||
@@ -164,7 +164,7 @@ func ModuleSpec() pipeline.ModuleSpec {
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ExtractorRegistry) error {
|
||||
return registry.RegisterWithSpec(ModuleSpec(), func() (contracts.Extractor, error) {
|
||||
return registry.RegisterLegacyRawWithSpec(ModuleSpec(), func() (contracts.LegacyRawExtractor, error) {
|
||||
return New(), nil
|
||||
})
|
||||
}
|
||||
|
||||
@@ -74,7 +74,7 @@ func TestRegisterMakesExtractorBuildable(t *testing.T) {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
|
||||
extractor, err := registry.Build(Key)
|
||||
extractor, err := registry.BuildLegacyRaw(Key)
|
||||
if err != nil {
|
||||
t.Fatalf("Build() error = %v, want nil", err)
|
||||
}
|
||||
|
||||
@@ -11,7 +11,7 @@ import (
|
||||
const Key = "extract/dnd/spells/shape"
|
||||
const ReasonCode = "invalid_spell_shape"
|
||||
|
||||
var _ contracts.Validator = (*Validator)(nil)
|
||||
var _ contracts.LegacyRawValidator = (*Validator)(nil)
|
||||
|
||||
type Validator struct{}
|
||||
|
||||
@@ -46,7 +46,7 @@ func Spec() pipeline.ValidatorSpec {
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ValidatorRegistry) error {
|
||||
return registry.RegisterWithSpec(Spec(), func() (contracts.Validator, error) {
|
||||
return registry.RegisterLegacyRawWithSpec(Spec(), func() (contracts.LegacyRawValidator, error) {
|
||||
return New(), nil
|
||||
})
|
||||
}
|
||||
|
||||
@@ -49,7 +49,7 @@ func TestSpecAndRegister(t *testing.T) {
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
validator, err := registry.Build(Key)
|
||||
validator, err := registry.BuildLegacyRaw(Key)
|
||||
if err != nil {
|
||||
t.Fatalf("Build(%q) error = %v, want nil", Key, err)
|
||||
}
|
||||
|
||||
@@ -13,7 +13,7 @@ import (
|
||||
const Key = "extract/dnd/spells/source_refs"
|
||||
const ReasonCode = "invalid_source_refs"
|
||||
|
||||
var _ contracts.Validator = (*Validator)(nil)
|
||||
var _ contracts.LegacyRawValidator = (*Validator)(nil)
|
||||
|
||||
type Validator struct{}
|
||||
|
||||
@@ -55,7 +55,7 @@ func Spec() pipeline.ValidatorSpec {
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ValidatorRegistry) error {
|
||||
return registry.RegisterWithSpec(Spec(), func() (contracts.Validator, error) {
|
||||
return registry.RegisterLegacyRawWithSpec(Spec(), func() (contracts.LegacyRawValidator, error) {
|
||||
return New(), nil
|
||||
})
|
||||
}
|
||||
|
||||
@@ -50,7 +50,7 @@ func TestSpecAndRegister(t *testing.T) {
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
validator, err := registry.Build(Key)
|
||||
validator, err := registry.BuildLegacyRaw(Key)
|
||||
if err != nil {
|
||||
t.Fatalf("Build(%q) error = %v, want nil", Key, err)
|
||||
}
|
||||
|
||||
@@ -14,7 +14,7 @@ import (
|
||||
const Key = "extract/dnd/spells/source_relatedness"
|
||||
const WarningReasonCode = "spell_not_near_source"
|
||||
|
||||
var _ contracts.Validator = (*Validator)(nil)
|
||||
var _ contracts.LegacyRawValidator = (*Validator)(nil)
|
||||
|
||||
type Validator struct{}
|
||||
|
||||
@@ -60,7 +60,7 @@ func Spec() pipeline.ValidatorSpec {
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ValidatorRegistry) error {
|
||||
return registry.RegisterWithSpec(Spec(), func() (contracts.Validator, error) {
|
||||
return registry.RegisterLegacyRawWithSpec(Spec(), func() (contracts.LegacyRawValidator, error) {
|
||||
return New(), nil
|
||||
})
|
||||
}
|
||||
|
||||
@@ -53,7 +53,7 @@ func TestSpecAndRegister(t *testing.T) {
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
validator, err := registry.Build(Key)
|
||||
validator, err := registry.BuildLegacyRaw(Key)
|
||||
if err != nil {
|
||||
t.Fatalf("Build(%q) error = %v, want nil", Key, err)
|
||||
}
|
||||
|
||||
@@ -14,7 +14,7 @@ import (
|
||||
|
||||
const Key = "appendorder"
|
||||
|
||||
var _ contracts.Merger = (*Merger)(nil)
|
||||
var _ contracts.LegacyRawMerger = (*Merger)(nil)
|
||||
|
||||
type Merger struct{}
|
||||
|
||||
@@ -81,7 +81,7 @@ func ModuleSpec() pipeline.ModuleSpec {
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.MergerRegistry) error {
|
||||
return registry.RegisterWithSpec(ModuleSpec(), func() (contracts.Merger, error) {
|
||||
return registry.RegisterLegacyRawWithSpec(ModuleSpec(), func() (contracts.LegacyRawMerger, error) {
|
||||
return New(), nil
|
||||
})
|
||||
}
|
||||
|
||||
@@ -10,7 +10,7 @@ import (
|
||||
|
||||
const Key = "noop"
|
||||
|
||||
var _ contracts.Normalizer = (*Normalizer)(nil)
|
||||
var _ contracts.LegacyRawNormalizer = (*Normalizer)(nil)
|
||||
|
||||
type Normalizer struct{}
|
||||
|
||||
@@ -57,7 +57,7 @@ func ModuleSpec() pipeline.ModuleSpec {
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.NormalizerRegistry) error {
|
||||
return registry.RegisterWithSpec(ModuleSpec(), func() (contracts.Normalizer, error) {
|
||||
return registry.RegisterLegacyRawWithSpec(ModuleSpec(), func() (contracts.LegacyRawNormalizer, error) {
|
||||
return New(), nil
|
||||
})
|
||||
}
|
||||
|
||||
@@ -31,7 +31,7 @@ func TestModuleSpecAndRegister(t *testing.T) {
|
||||
if !reflect.DeepEqual(spec, want) {
|
||||
t.Fatalf("registered spec = %#v, want %#v", spec, want)
|
||||
}
|
||||
normalizer, err := registry.Build(Key)
|
||||
normalizer, err := registry.BuildLegacyRaw(Key)
|
||||
if err != nil {
|
||||
t.Fatalf("Build(%q) error = %v, want nil", Key, err)
|
||||
}
|
||||
|
||||
@@ -9,7 +9,7 @@ import (
|
||||
|
||||
const Key = "generic/always_accept"
|
||||
|
||||
var _ contracts.Validator = (*Validator)(nil)
|
||||
var _ contracts.LegacyRawValidator = (*Validator)(nil)
|
||||
|
||||
type Validator struct{}
|
||||
|
||||
@@ -37,7 +37,7 @@ func Spec() pipeline.ValidatorSpec {
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ValidatorRegistry) error {
|
||||
return registry.RegisterWithSpec(Spec(), func() (contracts.Validator, error) {
|
||||
return registry.RegisterLegacyRawWithSpec(Spec(), func() (contracts.LegacyRawValidator, error) {
|
||||
return New(), nil
|
||||
})
|
||||
}
|
||||
|
||||
@@ -30,7 +30,7 @@ func TestSpecAndRegister(t *testing.T) {
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
validator, err := registry.Build(Key)
|
||||
validator, err := registry.BuildLegacyRaw(Key)
|
||||
if err != nil {
|
||||
t.Fatalf("Build(%q) error = %v, want nil", Key, err)
|
||||
}
|
||||
|
||||
@@ -10,7 +10,7 @@ import (
|
||||
const Key = "generic/always_reject"
|
||||
const ReasonCode = "always_reject"
|
||||
|
||||
var _ contracts.Validator = (*Validator)(nil)
|
||||
var _ contracts.LegacyRawValidator = (*Validator)(nil)
|
||||
|
||||
type Validator struct{}
|
||||
|
||||
@@ -42,7 +42,7 @@ func Spec() pipeline.ValidatorSpec {
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ValidatorRegistry) error {
|
||||
return registry.RegisterWithSpec(Spec(), func() (contracts.Validator, error) {
|
||||
return registry.RegisterLegacyRawWithSpec(Spec(), func() (contracts.LegacyRawValidator, error) {
|
||||
return New(), nil
|
||||
})
|
||||
}
|
||||
|
||||
@@ -33,7 +33,7 @@ func TestSpecAndRegister(t *testing.T) {
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
validator, err := registry.Build(Key)
|
||||
validator, err := registry.BuildLegacyRaw(Key)
|
||||
if err != nil {
|
||||
t.Fatalf("Build(%q) error = %v, want nil", Key, err)
|
||||
}
|
||||
|
||||
@@ -11,7 +11,7 @@ import (
|
||||
const Key = "generic/valid_json"
|
||||
const ReasonCodeInvalidJSON = "invalid_json"
|
||||
|
||||
var _ contracts.Validator = (*Validator)(nil)
|
||||
var _ contracts.LegacyRawValidator = (*Validator)(nil)
|
||||
|
||||
type Validator struct{}
|
||||
|
||||
@@ -46,7 +46,7 @@ func Spec() pipeline.ValidatorSpec {
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ValidatorRegistry) error {
|
||||
return registry.RegisterWithSpec(Spec(), func() (contracts.Validator, error) {
|
||||
return registry.RegisterLegacyRawWithSpec(Spec(), func() (contracts.LegacyRawValidator, error) {
|
||||
return New(), nil
|
||||
})
|
||||
}
|
||||
|
||||
@@ -47,7 +47,7 @@ func TestSpecAndRegister(t *testing.T) {
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
validator, err := registry.Build(Key)
|
||||
validator, err := registry.BuildLegacyRaw(Key)
|
||||
if err != nil {
|
||||
t.Fatalf("Build(%q) error = %v, want nil", Key, err)
|
||||
}
|
||||
|
||||
@@ -15,7 +15,7 @@ const Key = "generic/valid_json_schema"
|
||||
const ReasonCodeInvalidJSON = "invalid_json"
|
||||
const ReasonCodeSchemaInvalid = "json_schema_invalid"
|
||||
|
||||
var _ contracts.Validator = (*Validator)(nil)
|
||||
var _ contracts.LegacyRawValidator = (*Validator)(nil)
|
||||
|
||||
type Validator struct{}
|
||||
|
||||
@@ -75,7 +75,7 @@ func Spec() pipeline.ValidatorSpec {
|
||||
}
|
||||
|
||||
func Register(registry *pipeline.ValidatorRegistry) error {
|
||||
return registry.RegisterWithSpec(Spec(), func() (contracts.Validator, error) {
|
||||
return registry.RegisterLegacyRawWithSpec(Spec(), func() (contracts.LegacyRawValidator, error) {
|
||||
return New(), nil
|
||||
})
|
||||
}
|
||||
|
||||
@@ -74,7 +74,7 @@ func TestSpecAndRegister(t *testing.T) {
|
||||
if err := Register(registry); err != nil {
|
||||
t.Fatalf("Register() error = %v, want nil", err)
|
||||
}
|
||||
validator, err := registry.Build(Key)
|
||||
validator, err := registry.BuildLegacyRaw(Key)
|
||||
if err != nil {
|
||||
t.Fatalf("Build(%q) error = %v, want nil", Key, err)
|
||||
}
|
||||
|
||||
@@ -180,25 +180,25 @@ func dndSpellsTestCatalog(t *testing.T, specs dndSpellsCatalogSpecs) pipeline.Mo
|
||||
if err := spells.Register(extractors); err != nil {
|
||||
t.Fatalf("register dnd spells extractor: %v", err)
|
||||
}
|
||||
} else if err := extractors.RegisterWithSpec(specs.extractor, func() (contracts.Extractor, error) {
|
||||
} else if err := extractors.RegisterLegacyRawWithSpec(specs.extractor, func() (contracts.LegacyRawExtractor, error) {
|
||||
return spells.New(), nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register dnd spells extractor override: %v", err)
|
||||
}
|
||||
|
||||
if err := mergers.RegisterWithSpec(pipeline.ModuleSpec{
|
||||
if err := mergers.RegisterLegacyRawWithSpec(pipeline.ModuleSpec{
|
||||
Key: pipeline.DefaultMergeModule,
|
||||
Stage: pipeline.StageMerge,
|
||||
Requires: []string{"dnd.spell_casts"},
|
||||
}, func() (contracts.Merger, error) {
|
||||
}, func() (contracts.LegacyRawMerger, error) {
|
||||
return appendorder.New(), nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register merger: %v", err)
|
||||
}
|
||||
if err := normalizers.RegisterWithSpec(pipeline.ModuleSpec{
|
||||
if err := normalizers.RegisterLegacyRawWithSpec(pipeline.ModuleSpec{
|
||||
Key: pipeline.DefaultNormalizeModule,
|
||||
Stage: pipeline.StageNormalize,
|
||||
}, func() (contracts.Normalizer, error) {
|
||||
}, func() (contracts.LegacyRawNormalizer, error) {
|
||||
return noop.New(), nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register normalizer: %v", err)
|
||||
|
||||
@@ -173,7 +173,7 @@ func mustRegisterChunker(t *testing.T, registry *pipeline.ChunkerRegistry, spec
|
||||
|
||||
func mustRegisterExtractor(t *testing.T, registry *pipeline.ExtractorRegistry, spec pipeline.ModuleSpec) {
|
||||
t.Helper()
|
||||
if err := registry.RegisterWithSpec(spec, func() (contracts.Extractor, error) {
|
||||
if err := registry.RegisterLegacyRawWithSpec(spec, func() (contracts.LegacyRawExtractor, error) {
|
||||
return fakeExtractor{}, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register extractor: %v", err)
|
||||
@@ -182,7 +182,7 @@ func mustRegisterExtractor(t *testing.T, registry *pipeline.ExtractorRegistry, s
|
||||
|
||||
func mustRegisterMerger(t *testing.T, registry *pipeline.MergerRegistry, spec pipeline.ModuleSpec) {
|
||||
t.Helper()
|
||||
if err := registry.RegisterWithSpec(spec, func() (contracts.Merger, error) {
|
||||
if err := registry.RegisterLegacyRawWithSpec(spec, func() (contracts.LegacyRawMerger, error) {
|
||||
return appendorder.New(), nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register merger: %v", err)
|
||||
@@ -191,7 +191,7 @@ func mustRegisterMerger(t *testing.T, registry *pipeline.MergerRegistry, spec pi
|
||||
|
||||
func mustRegisterNormalizer(t *testing.T, registry *pipeline.NormalizerRegistry, spec pipeline.ModuleSpec) {
|
||||
t.Helper()
|
||||
if err := registry.RegisterWithSpec(spec, func() (contracts.Normalizer, error) {
|
||||
if err := registry.RegisterLegacyRawWithSpec(spec, func() (contracts.LegacyRawNormalizer, error) {
|
||||
return noop.New(), nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register normalizer: %v", err)
|
||||
@@ -246,7 +246,7 @@ func withoutCapability(capabilities []string, capability string) []string {
|
||||
}
|
||||
|
||||
var (
|
||||
_ contracts.Chunker = fakeChunker{}
|
||||
_ contracts.Extractor = fakeExtractor{}
|
||||
_ contracts.OutputEncoder = fakeOutput{}
|
||||
_ contracts.Chunker = fakeChunker{}
|
||||
_ contracts.LegacyRawExtractor = fakeExtractor{}
|
||||
_ contracts.OutputEncoder = fakeOutput{}
|
||||
)
|
||||
|
||||
@@ -108,7 +108,7 @@ func configResolveInput(t *testing.T) config.ResolveInput {
|
||||
}
|
||||
}
|
||||
|
||||
func seriatimRunnerRegistries(t *testing.T, extractor contracts.Extractor) pipeline.Registries {
|
||||
func seriatimRunnerRegistries(t *testing.T, extractor contracts.LegacyRawExtractor) pipeline.Registries {
|
||||
t.Helper()
|
||||
|
||||
inputs := pipeline.NewInputAdapterRegistry()
|
||||
@@ -126,17 +126,17 @@ func seriatimRunnerRegistries(t *testing.T, extractor contracts.Extractor) pipel
|
||||
}); err != nil {
|
||||
t.Fatalf("register chunker: %v", err)
|
||||
}
|
||||
if err := extractors.Register("fake/extract", func() (contracts.Extractor, error) {
|
||||
if err := extractors.RegisterLegacyRaw("fake/extract", func() (contracts.LegacyRawExtractor, error) {
|
||||
return extractor, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register extractor: %v", err)
|
||||
}
|
||||
if err := mergers.Register(pipeline.DefaultMergeModule, func() (contracts.Merger, error) {
|
||||
if err := mergers.RegisterLegacyRaw(pipeline.DefaultMergeModule, func() (contracts.LegacyRawMerger, error) {
|
||||
return appendorder.New(), nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register merger: %v", err)
|
||||
}
|
||||
if err := normalizers.Register(pipeline.DefaultNormalizeModule, func() (contracts.Normalizer, error) {
|
||||
if err := normalizers.RegisterLegacyRaw(pipeline.DefaultNormalizeModule, func() (contracts.LegacyRawNormalizer, error) {
|
||||
return noop.New(), nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register normalizer: %v", err)
|
||||
@@ -285,7 +285,7 @@ func equalInts(a, b []int) bool {
|
||||
}
|
||||
|
||||
var (
|
||||
_ contracts.Chunker = runnerSeriatimChunker{}
|
||||
_ contracts.Extractor = (*runnerSeriatimExtractor)(nil)
|
||||
_ contracts.OutputEncoder = runnerSeriatimOutput{}
|
||||
_ contracts.Chunker = runnerSeriatimChunker{}
|
||||
_ contracts.LegacyRawExtractor = (*runnerSeriatimExtractor)(nil)
|
||||
_ contracts.OutputEncoder = runnerSeriatimOutput{}
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user