Compose production modules through family registrars

This commit is contained in:
2026-07-17 04:53:09 +00:00
parent c99bad19ae
commit 0327659355
12 changed files with 524 additions and 96 deletions

View File

@@ -28,8 +28,8 @@ without exposing Scriptorium types through stage contracts.
`internal/cli` constructs the production runtime by: `internal/cli` constructs the production runtime by:
1. collecting embedded prompt and response-schema assets from production module 1. allocating the asset registry populated by the generic, Seriatim, and D&D
packages; package-family registrars;
2. creating a `ScriptoriumClient` from the effective profile source; 2. creating a `ScriptoriumClient` from the effective profile source;
3. attaching an `LLMProfileRecorder`; 3. attaching an `LLMProfileRecorder`;
4. creating a scheduler from the effective concurrency limit; 4. creating a scheduler from the effective concurrency limit;

View File

@@ -12,7 +12,9 @@ Configuration.
A stage module package provides a stable key, constructor, contract A stage module package provides a stable key, constructor, contract
implementation, `ModuleSpec`, `Register`, and focused behavior and registration implementation, `ModuleSpec`, `Register`, and focused behavior and registration
tests. A validator package follows the same pattern with `ValidatorSpec` and the tests. A validator package follows the same pattern with `ValidatorSpec` and the
validator registry. validator registry. Package-family registrars compose those leaf registrations
into the production catalog and own family-level policy such as default
validator chains and prompt asset collection.
Specs expose capability and execution metadata without constructing an Specs expose capability and execution metadata without constructing an
implementation. Chunk, extract, merge, and normalize modules that accept implementation. Chunk, extract, merge, and normalize modules that accept
@@ -154,10 +156,13 @@ payload rules are defined in the
## Production Registration ## Production Registration
`internal/cli/catalog.go` builds the production registries, registers module and The CLI allocates one complete framework registry set and one LLM asset
validator constructors, installs default validator-chain mappings, and exposes registry. It invokes `internal/modules/generic/register`,
the matching catalog for resolution. It also collects prompt assets from `internal/modules/seriatim/register`, and `internal/modules/dnd/register` in
LLM-backed packages before constructing the production client. that order, then exposes the matching catalog for resolution. The generic and
Seriatim registrars own their production leaf registrations. The D&D registrar
owns D&D leaf registrations, the spell default-validator chain, and D&D
prompt/schema asset collection.
Framework packages must not import production extensions. Tests may compose Framework packages must not import production extensions. Tests may compose
registries and catalogs directly with fakes. registries and catalogs directly with fakes.
@@ -170,8 +175,8 @@ When adding a production module or validator:
2. expose and test its spec, constructor, and registration function; 2. expose and test its spec, constructor, and registration function;
3. keep format or domain parsing inside the concrete package; 3. keep format or domain parsing inside the concrete package;
4. add package-owned prompt/schema assets when the extension is LLM-backed; 4. add package-owned prompt/schema assets when the extension is LLM-backed;
5. register it in `internal/cli/catalog.go` and add a default chain only when 5. register it through its package-family registrar and add a default chain
production policy requires one; there only when production policy requires one;
6. add resolution and composition coverage for capabilities, options, 6. add resolution and composition coverage for capabilities, options,
references, and validation behavior; references, and validation behavior;
7. update the selectable-key catalog in [Configuration](../config.md), the 7. update the selectable-key catalog in [Configuration](../config.md), the

View File

@@ -22,7 +22,7 @@ a sorted set of artifact lanes before the runner constructs any stage module.
| Package | Implemented responsibility | | Package | Implemented responsibility |
| --- | --- | | --- | --- |
| `cmd/notarius` | Executable entry point and process exit delegation. | | `cmd/notarius` | Executable entry point and process exit delegation. |
| `internal/cli` | Command parsing, config discovery, production registration, prompt asset collection, LLM client construction, reference materialization, workspace collaborator setup, durable writes, and user-facing results. | | `internal/cli` | Command parsing, config discovery, package-family registrar invocation, LLM client construction, reference materialization, workspace collaborator setup, durable writes, and user-facing results. |
## Core Packages ## Core Packages
@@ -75,7 +75,19 @@ Concrete validators live under `internal/validators`. Generic packages provide
unconditional test decisions, JSON syntax validation, and JSON Schema unconditional test decisions, JSON syntax validation, and JSON Schema
validation. D&D spell packages provide shape, source-reference, and validation. D&D spell packages provide shape, source-reference, and
source-relatedness decisions, with `spellpayload` holding their shared parser source-relatedness decisions, with `spellpayload` holding their shared parser
and lookup helpers. Production chain composition is owned by `internal/cli`. and lookup helpers.
Production composition is grouped behind package-family registrars while the
implementations remain in their current stage-oriented packages:
| Package | Implemented responsibility |
| --- | --- |
| `internal/modules/generic/register` | Registers domain-neutral chunk, merge, normalize, output, and validator implementations. |
| `internal/modules/seriatim/register` | Registers the Seriatim input adapter. |
| `internal/modules/dnd/register` | Registers D&D modules, validators, default validator policy, and prompt/schema assets. |
The CLI allocates the framework registries and asset registry, then invokes
these registrars in generic, Seriatim, and D&D order.
Implementation details for all production extensions are in Implementation details for all production extensions are in
[Module Internals](modules.md). [Module Internals](modules.md).

View File

@@ -9,23 +9,17 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts" "gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm" "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline" "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/chunk/dnd/scenes" dndregister "gitea.maximumdirect.net/eric/notarius/internal/modules/dnd/register"
"gitea.maximumdirect.net/eric/notarius/internal/modules/chunk/generic" genericregister "gitea.maximumdirect.net/eric/notarius/internal/modules/generic/register"
"gitea.maximumdirect.net/eric/notarius/internal/modules/extract/dnd/spells" seriatimregister "gitea.maximumdirect.net/eric/notarius/internal/modules/seriatim/register"
"gitea.maximumdirect.net/eric/notarius/internal/modules/input/seriatim"
"gitea.maximumdirect.net/eric/notarius/internal/modules/merge/appendorder"
"gitea.maximumdirect.net/eric/notarius/internal/modules/normalize/noop"
jsonoutput "gitea.maximumdirect.net/eric/notarius/internal/modules/output/json"
spellshape "gitea.maximumdirect.net/eric/notarius/internal/validators/extract/dnd/spells/shape"
spellsourcerefs "gitea.maximumdirect.net/eric/notarius/internal/validators/extract/dnd/spells/source_refs"
spellrelatedness "gitea.maximumdirect.net/eric/notarius/internal/validators/extract/dnd/spells/source_relatedness"
alwaysaccept "gitea.maximumdirect.net/eric/notarius/internal/validators/generic/always_accept"
alwaysreject "gitea.maximumdirect.net/eric/notarius/internal/validators/generic/always_reject"
validjson "gitea.maximumdirect.net/eric/notarius/internal/validators/generic/valid_json"
validjsonschema "gitea.maximumdirect.net/eric/notarius/internal/validators/generic/valid_json_schema"
) )
func productionRegistries() (pipeline.Registries, error) { type productionComponents struct {
registries pipeline.Registries
assets *llm.AssetRegistry
}
func newProductionComponents() (productionComponents, error) {
registries := pipeline.Registries{ registries := pipeline.Registries{
Inputs: pipeline.NewInputAdapterRegistry(), Inputs: pipeline.NewInputAdapterRegistry(),
Chunkers: pipeline.NewChunkerRegistry(), Chunkers: pipeline.NewChunkerRegistry(),
@@ -36,72 +30,26 @@ func productionRegistries() (pipeline.Registries, error) {
ValidatorChains: pipeline.NewValidatorChainRegistry(), ValidatorChains: pipeline.NewValidatorChainRegistry(),
Outputs: pipeline.NewOutputEncoderRegistry(), Outputs: pipeline.NewOutputEncoderRegistry(),
} }
if err := seriatim.Register(registries.Inputs); err != nil { assets := llm.NewAssetRegistry()
return pipeline.Registries{}, fmt.Errorf("register seriatim input: %w", err) registrars := []struct {
}
if err := generic.Register(registries.Chunkers); err != nil {
return pipeline.Registries{}, fmt.Errorf("register generic chunker: %w", err)
}
if err := scenes.Register(registries.Chunkers); err != nil {
return pipeline.Registries{}, fmt.Errorf("register dnd scenes chunker: %w", err)
}
if err := spells.Register(registries.Extractors); err != nil {
return pipeline.Registries{}, fmt.Errorf("register dnd spells extractor: %w", err)
}
if err := appendorder.Register(registries.Mergers); err != nil {
return pipeline.Registries{}, fmt.Errorf("register appendorder merger: %w", err)
}
if err := noop.Register(registries.Normalizers); err != nil {
return pipeline.Registries{}, fmt.Errorf("register noop normalizer: %w", err)
}
if err := registerProductionValidators(registries.Validators); err != nil {
return pipeline.Registries{}, err
}
if err := registerProductionValidatorChains(registries.ValidatorChains); err != nil {
return pipeline.Registries{}, err
}
if err := jsonoutput.Register(registries.Outputs); err != nil {
return pipeline.Registries{}, fmt.Errorf("register json output encoder: %w", err)
}
return registries, nil
}
func registerProductionValidators(registry *pipeline.ValidatorRegistry) error {
registrations := []struct {
name string name string
register func(*pipeline.ValidatorRegistry) error register func(pipeline.Registries, *llm.AssetRegistry) error
}{ }{
{name: "generic always accept validator", register: alwaysaccept.Register}, {name: "generic", register: genericregister.Register},
{name: "generic always reject validator", register: alwaysreject.Register}, {name: "seriatim", register: seriatimregister.Register},
{name: "generic valid json validator", register: validjson.Register}, {name: "dnd", register: dndregister.Register},
{name: "generic valid json schema validator", register: validjsonschema.Register},
{name: "dnd spell shape validator", register: spellshape.Register},
{name: "dnd spell source references validator", register: spellsourcerefs.Register},
{name: "dnd spell source relatedness validator", register: spellrelatedness.Register},
} }
for _, registration := range registrations { for _, registrar := range registrars {
if err := registration.register(registry); err != nil { if err := registrar.register(registries, assets); err != nil {
return fmt.Errorf("register %s: %w", registration.name, err) return productionComponents{}, fmt.Errorf("register %s module family: %w", registrar.name, err)
} }
} }
return nil return productionComponents{registries: registries, assets: assets}, nil
} }
func registerProductionValidatorChains(registry *pipeline.ValidatorChainRegistry) error { func productionRegistries() (pipeline.Registries, error) {
if err := registry.Register(pipeline.ValidatorChainMapping{ components, err := newProductionComponents()
Stage: pipeline.StageExtract, return components.registries, err
Module: spells.Key,
Validators: []pipeline.ModuleBinding{
pipeline.Binding(validjson.Key),
pipeline.Binding(validjsonschema.Key),
pipeline.Binding(spellshape.Key),
pipeline.Binding(spellsourcerefs.Key),
pipeline.Binding(spellrelatedness.Key),
},
}); err != nil {
return fmt.Errorf("register dnd spells validator chain: %w", err)
}
return nil
} }
func productionCatalog() (pipeline.ModuleCatalog, error) { func productionCatalog() (pipeline.ModuleCatalog, error) {
@@ -113,14 +61,8 @@ func productionCatalog() (pipeline.ModuleCatalog, error) {
} }
func productionPromptAssets() (*llm.AssetRegistry, error) { func productionPromptAssets() (*llm.AssetRegistry, error) {
registry := llm.NewAssetRegistry() components, err := newProductionComponents()
if err := scenes.RegisterPromptAssets(registry); err != nil { return components.assets, err
return nil, fmt.Errorf("register dnd scenes prompt assets: %w", err)
}
if err := spells.RegisterPromptAssets(registry); err != nil {
return nil, fmt.Errorf("register dnd spells prompt assets: %w", err)
}
return registry, nil
} }
func effectiveCatalog(opts Options) (pipeline.ModuleCatalog, error) { func effectiveCatalog(opts Options) (pipeline.ModuleCatalog, error) {
@@ -199,6 +141,22 @@ func productionLLMClientFactory(ctx context.Context, cfg config.Config, profileI
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
return buildProductionLLMClient(ctx, cfg, profileID, assets)
}
func productionLLMClientFactoryWithAssets(assets *llm.AssetRegistry) LLMClientFactory {
return func(ctx context.Context, cfg config.Config, profileID string) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error) {
return buildProductionLLMClient(ctx, cfg, profileID, assets)
}
}
func buildProductionLLMClient(ctx context.Context, cfg config.Config, profileID string, assets *llm.AssetRegistry) (contracts.StructuredLLMClient, []artifacts.LLMProfileManifest, error) {
if err := ctx.Err(); err != nil {
return nil, nil, err
}
if assets == nil {
return nil, nil, fmt.Errorf("production asset registry must not be nil")
}
recorder := llm.NewLLMProfileRecorder() recorder := llm.NewLLMProfileRecorder()
client, err := llm.NewScriptoriumClient(llm.ScriptoriumClientConfig{ client, err := llm.NewScriptoriumClient(llm.ScriptoriumClientConfig{
ProfileDir: cfg.Scriptorium.ProfileDir, ProfileDir: cfg.Scriptorium.ProfileDir,

View File

@@ -28,6 +28,24 @@ import (
) )
func TestProductionCompatibilitySnapshot(t *testing.T) { func TestProductionCompatibilitySnapshot(t *testing.T) {
normalizedOptions, err := normalizeOptions(Options{})
if err != nil {
t.Fatalf("normalizeOptions() error = %v, want nil", err)
}
if normalizedOptions.Catalog.Inputs != normalizedOptions.Registries.Inputs ||
normalizedOptions.Catalog.Chunkers != normalizedOptions.Registries.Chunkers ||
normalizedOptions.Catalog.Extractors != normalizedOptions.Registries.Extractors ||
normalizedOptions.Catalog.Mergers != normalizedOptions.Registries.Mergers ||
normalizedOptions.Catalog.Normalizers != normalizedOptions.Registries.Normalizers ||
normalizedOptions.Catalog.Validators != normalizedOptions.Registries.Validators ||
normalizedOptions.Catalog.ValidatorChains != normalizedOptions.Registries.ValidatorChains ||
normalizedOptions.Catalog.Outputs != normalizedOptions.Registries.Outputs {
t.Fatal("production catalog and execution registries do not share one composition")
}
if normalizedOptions.LLMClientFactory == nil {
t.Fatal("production LLM client factory is nil")
}
registries, err := productionRegistries() registries, err := productionRegistries()
if err != nil { if err != nil {
t.Fatalf("productionRegistries() error = %v, want nil", err) t.Fatalf("productionRegistries() error = %v, want nil", err)

View File

@@ -51,7 +51,12 @@ func Run(args []string, stdout, stderr io.Writer) int {
} }
func RunWithOptions(args []string, stdout, stderr io.Writer, opts Options) int { func RunWithOptions(args []string, stdout, stderr io.Writer, opts Options) int {
opts = normalizeOptions(opts) var err error
opts, err = normalizeOptions(opts)
if err != nil {
fmt.Fprintf(stderr, "notarius: %v\n", err)
return 1
}
if len(args) == 0 { if len(args) == 0 {
writeUsage(stdout) writeUsage(stdout)
return 0 return 0
@@ -78,17 +83,28 @@ func writeUsage(w io.Writer) {
fmt.Fprint(w, usage) fmt.Fprint(w, usage)
} }
func normalizeOptions(opts Options) Options { func normalizeOptions(opts Options) (Options, error) {
if opts.LookupEnv == nil { if opts.LookupEnv == nil {
opts.LookupEnv = os.LookupEnv opts.LookupEnv = os.LookupEnv
} }
if opts.Now == nil { if opts.Now == nil {
opts.Now = time.Now opts.Now = time.Now
} }
if isEmptyCatalog(opts.Catalog) && isEmptyRegistries(opts.Registries) {
components, err := newProductionComponents()
if err != nil {
return Options{}, err
}
opts.Registries = components.registries
opts.Catalog = catalogFromRegistries(components.registries)
if opts.LLMClientFactory == nil {
opts.LLMClientFactory = productionLLMClientFactoryWithAssets(components.assets)
}
}
if opts.LLMClientFactory == nil { if opts.LLMClientFactory == nil {
opts.LLMClientFactory = productionLLMClientFactory opts.LLMClientFactory = productionLLMClientFactory
} }
return opts return opts, nil
} }
func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) int { func runPipelineCommand(args []string, stdout, stderr io.Writer, opts Options) int {

View File

@@ -0,0 +1,71 @@
// Package register composes the production D&D module family.
package register
import (
"fmt"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/chunk/dnd/scenes"
"gitea.maximumdirect.net/eric/notarius/internal/modules/extract/dnd/spells"
spellshape "gitea.maximumdirect.net/eric/notarius/internal/validators/extract/dnd/spells/shape"
spellsourcerefs "gitea.maximumdirect.net/eric/notarius/internal/validators/extract/dnd/spells/source_refs"
spellrelatedness "gitea.maximumdirect.net/eric/notarius/internal/validators/extract/dnd/spells/source_relatedness"
validjson "gitea.maximumdirect.net/eric/notarius/internal/validators/generic/valid_json"
validjsonschema "gitea.maximumdirect.net/eric/notarius/internal/validators/generic/valid_json_schema"
)
// Register adds all production D&D modules, validators, policy, and assets.
func Register(registries pipeline.Registries, assets *llm.AssetRegistry) error {
if err := validateRegistries(registries, assets); err != nil {
return err
}
registrations := []struct {
name string
register func() error
}{
{name: "scenes chunker", register: func() error { return scenes.Register(registries.Chunkers) }},
{name: "spells extractor", register: func() error { return spells.Register(registries.Extractors) }},
{name: "spell shape validator", register: func() error { return spellshape.Register(registries.Validators) }},
{name: "spell source references validator", register: func() error { return spellsourcerefs.Register(registries.Validators) }},
{name: "spell source relatedness validator", register: func() error { return spellrelatedness.Register(registries.Validators) }},
{name: "scenes prompt assets", register: func() error { return scenes.RegisterPromptAssets(assets) }},
{name: "spells prompt assets", register: func() error { return spells.RegisterPromptAssets(assets) }},
}
for _, registration := range registrations {
if err := registration.register(); err != nil {
return fmt.Errorf("register dnd %s: %w", registration.name, err)
}
}
if err := registries.ValidatorChains.Register(pipeline.ValidatorChainMapping{
Stage: pipeline.StageExtract,
Module: spells.Key,
Validators: []pipeline.ModuleBinding{
pipeline.Binding(validjson.Key),
pipeline.Binding(validjsonschema.Key),
pipeline.Binding(spellshape.Key),
pipeline.Binding(spellsourcerefs.Key),
pipeline.Binding(spellrelatedness.Key),
},
}); err != nil {
return fmt.Errorf("register dnd spells validator chain: %w", err)
}
return nil
}
func validateRegistries(registries pipeline.Registries, assets *llm.AssetRegistry) error {
switch {
case registries.Chunkers == nil:
return fmt.Errorf("dnd registrar: chunker registry must not be nil")
case registries.Extractors == nil:
return fmt.Errorf("dnd registrar: extractor registry must not be nil")
case registries.Validators == nil:
return fmt.Errorf("dnd registrar: validator registry must not be nil")
case registries.ValidatorChains == nil:
return fmt.Errorf("dnd registrar: validator chain registry must not be nil")
case assets == nil:
return fmt.Errorf("dnd registrar: asset registry must not be nil")
default:
return nil
}
}

View File

@@ -0,0 +1,138 @@
package register
import (
"io/fs"
"reflect"
"sort"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/extract/dnd/spells"
)
func TestRegisterAddsDNDFamily(t *testing.T) {
registries := completeRegistries()
assets := llm.NewAssetRegistry()
if err := Register(registries, assets); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
assertKeys(t, "chunkers", registries.Chunkers.RegisteredKeys(), []string{"dnd/scenes"})
assertKeys(t, "extractors", registries.Extractors.RegisteredKeys(), []string{"dnd/spells"})
assertKeys(t, "validators", registries.Validators.RegisteredKeys(), []string{
"extract/dnd/spells/shape",
"extract/dnd/spells/source_refs",
"extract/dnd/spells/source_relatedness",
})
wantChain := []pipeline.ModuleBinding{
pipeline.Binding("generic/valid_json"),
pipeline.Binding("generic/valid_json_schema"),
pipeline.Binding("extract/dnd/spells/shape"),
pipeline.Binding("extract/dnd/spells/source_refs"),
pipeline.Binding("extract/dnd/spells/source_relatedness"),
}
if got := registries.ValidatorChains.Validators(pipeline.StageExtract, spells.Key); !reflect.DeepEqual(got, wantChain) {
t.Fatalf("spell validator chain = %#v, want %#v", got, wantChain)
}
assertAssetNames(t, assets.PromptFS, []string{
"dnd.scenes/dnd.scenes.yaml",
"dnd.scenes/instructions.md",
"dnd.scenes/sharedassets/common-dnd-references.md",
"dnd.scenes/sharedassets/common-dnd-system.md",
"dnd.scenes/sharedassets/common-dnd-transcript.md",
"dnd.scenes/task.md",
"dnd.spells/dnd.spells.yaml",
"dnd.spells/instructions.md",
"dnd.spells/sharedassets/common-dnd-references.md",
"dnd.spells/sharedassets/common-dnd-system.md",
"dnd.spells/sharedassets/common-dnd-transcript.md",
"dnd.spells/task.md",
})
assertAssetNames(t, assets.SchemaFS, []string{
"dnd_scenes.v1.json",
"dnd_spells.v1.json",
"dnd_spells_llm.v1.json",
})
}
func TestRegisterRejectsMissingDNDDependenciesBeforeMutation(t *testing.T) {
tests := []struct {
name string
remove func(*pipeline.Registries, **llm.AssetRegistry)
wantErr string
}{
{name: "chunkers", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.Chunkers = nil }, wantErr: "chunker registry"},
{name: "extractors", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.Extractors = nil }, wantErr: "extractor registry"},
{name: "validators", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.Validators = nil }, wantErr: "validator registry"},
{name: "validator chains", remove: func(r *pipeline.Registries, _ **llm.AssetRegistry) { r.ValidatorChains = nil }, wantErr: "validator chain registry"},
{name: "assets", remove: func(_ *pipeline.Registries, assets **llm.AssetRegistry) { *assets = nil }, wantErr: "asset registry"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
registries := completeRegistries()
assets := llm.NewAssetRegistry()
test.remove(&registries, &assets)
err := Register(registries, assets)
if err == nil || !strings.Contains(err.Error(), test.wantErr) {
t.Fatalf("Register() error = %v, want %q", err, test.wantErr)
}
if got := registries.Chunkers; got != nil && len(got.RegisteredKeys()) != 0 {
t.Fatalf("chunker keys = %#v, want validation before mutation", got.RegisteredKeys())
}
})
}
}
func TestRegisterReportsDuplicateDNDRegistration(t *testing.T) {
registries := completeRegistries()
assets := llm.NewAssetRegistry()
if err := Register(registries, assets); err != nil {
t.Fatalf("first Register() error = %v, want nil", err)
}
err := Register(registries, assets)
if err == nil || !strings.Contains(err.Error(), "register dnd scenes chunker") || !strings.Contains(err.Error(), "already registered") {
t.Fatalf("second Register() error = %v, want contextual duplicate error", err)
}
}
func completeRegistries() pipeline.Registries {
return pipeline.Registries{
Inputs: pipeline.NewInputAdapterRegistry(),
Chunkers: pipeline.NewChunkerRegistry(),
Extractors: pipeline.NewExtractorRegistry(),
Mergers: pipeline.NewMergerRegistry(),
Normalizers: pipeline.NewNormalizerRegistry(),
Validators: pipeline.NewValidatorRegistry(),
ValidatorChains: pipeline.NewValidatorChainRegistry(),
Outputs: pipeline.NewOutputEncoderRegistry(),
}
}
func assertKeys(t *testing.T, name string, got, want []string) {
t.Helper()
if !reflect.DeepEqual(got, want) {
t.Fatalf("%s keys = %#v, want %#v", name, got, want)
}
}
func assertAssetNames(t *testing.T, getFS func() (fs.FS, error), want []string) {
t.Helper()
fSys, err := getFS()
if err != nil {
t.Fatalf("asset filesystem error = %v, want nil", err)
}
var got []string
if err := fs.WalkDir(fSys, ".", func(path string, entry fs.DirEntry, err error) error {
if err == nil && !entry.IsDir() {
got = append(got, path)
}
return err
}); err != nil {
t.Fatalf("walk assets: %v", err)
}
sort.Strings(got)
if !reflect.DeepEqual(got, want) {
t.Fatalf("asset names = %#v, want %#v", got, want)
}
}

View File

@@ -0,0 +1,61 @@
// Package register composes the production domain-neutral module family.
package register
import (
"fmt"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/chunk/generic"
"gitea.maximumdirect.net/eric/notarius/internal/modules/merge/appendorder"
"gitea.maximumdirect.net/eric/notarius/internal/modules/normalize/noop"
jsonoutput "gitea.maximumdirect.net/eric/notarius/internal/modules/output/json"
alwaysaccept "gitea.maximumdirect.net/eric/notarius/internal/validators/generic/always_accept"
alwaysreject "gitea.maximumdirect.net/eric/notarius/internal/validators/generic/always_reject"
validjson "gitea.maximumdirect.net/eric/notarius/internal/validators/generic/valid_json"
validjsonschema "gitea.maximumdirect.net/eric/notarius/internal/validators/generic/valid_json_schema"
)
// Register adds all production domain-neutral modules and validators.
func Register(registries pipeline.Registries, assets *llm.AssetRegistry) error {
_ = assets
if err := validateRegistries(registries); err != nil {
return err
}
registrations := []struct {
name string
register func() error
}{
{name: "generic chunker", register: func() error { return generic.Register(registries.Chunkers) }},
{name: "appendorder merger", register: func() error { return appendorder.Register(registries.Mergers) }},
{name: "noop normalizer", register: func() error { return noop.Register(registries.Normalizers) }},
{name: "always accept validator", register: func() error { return alwaysaccept.Register(registries.Validators) }},
{name: "always reject validator", register: func() error { return alwaysreject.Register(registries.Validators) }},
{name: "valid json validator", register: func() error { return validjson.Register(registries.Validators) }},
{name: "valid json schema validator", register: func() error { return validjsonschema.Register(registries.Validators) }},
{name: "json output encoder", register: func() error { return jsonoutput.Register(registries.Outputs) }},
}
for _, registration := range registrations {
if err := registration.register(); err != nil {
return fmt.Errorf("register %s: %w", registration.name, err)
}
}
return nil
}
func validateRegistries(registries pipeline.Registries) error {
switch {
case registries.Chunkers == nil:
return fmt.Errorf("generic registrar: chunker registry must not be nil")
case registries.Mergers == nil:
return fmt.Errorf("generic registrar: merger registry must not be nil")
case registries.Normalizers == nil:
return fmt.Errorf("generic registrar: normalizer registry must not be nil")
case registries.Validators == nil:
return fmt.Errorf("generic registrar: validator registry must not be nil")
case registries.Outputs == nil:
return fmt.Errorf("generic registrar: output registry must not be nil")
default:
return nil
}
}

View File

@@ -0,0 +1,90 @@
package register
import (
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestRegisterAddsGenericFamily(t *testing.T) {
registries := completeRegistries()
if err := Register(registries, nil); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
assertKeys(t, "chunkers", registries.Chunkers.RegisteredKeys(), []string{"generic"})
assertKeys(t, "mergers", registries.Mergers.RegisteredKeys(), []string{"appendorder"})
assertKeys(t, "normalizers", registries.Normalizers.RegisteredKeys(), []string{"noop"})
assertKeys(t, "validators", registries.Validators.RegisteredKeys(), []string{
"generic/always_accept",
"generic/always_reject",
"generic/valid_json",
"generic/valid_json_schema",
})
assertKeys(t, "outputs", registries.Outputs.RegisteredKeys(), []string{"json"})
if got := registries.Inputs.RegisteredKeys(); len(got) != 0 {
t.Fatalf("input keys = %#v, want generic registrar to leave inputs unchanged", got)
}
if got := registries.Extractors.RegisteredKeys(); len(got) != 0 {
t.Fatalf("extractor keys = %#v, want generic registrar to leave extractors unchanged", got)
}
}
func TestRegisterRejectsMissingGenericRegistriesBeforeMutation(t *testing.T) {
tests := []struct {
name string
remove func(*pipeline.Registries)
wantErr string
}{
{name: "chunkers", remove: func(r *pipeline.Registries) { r.Chunkers = nil }, wantErr: "chunker registry"},
{name: "mergers", remove: func(r *pipeline.Registries) { r.Mergers = nil }, wantErr: "merger registry"},
{name: "normalizers", remove: func(r *pipeline.Registries) { r.Normalizers = nil }, wantErr: "normalizer registry"},
{name: "validators", remove: func(r *pipeline.Registries) { r.Validators = nil }, wantErr: "validator registry"},
{name: "outputs", remove: func(r *pipeline.Registries) { r.Outputs = nil }, wantErr: "output registry"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
registries := completeRegistries()
test.remove(&registries)
err := Register(registries, nil)
if err == nil || !strings.Contains(err.Error(), test.wantErr) {
t.Fatalf("Register() error = %v, want %q", err, test.wantErr)
}
if got := registries.Chunkers; got != nil && len(got.RegisteredKeys()) != 0 {
t.Fatalf("chunker keys = %#v, want validation before mutation", got.RegisteredKeys())
}
})
}
}
func TestRegisterReportsDuplicateGenericRegistration(t *testing.T) {
registries := completeRegistries()
if err := Register(registries, nil); err != nil {
t.Fatalf("first Register() error = %v, want nil", err)
}
err := Register(registries, nil)
if err == nil || !strings.Contains(err.Error(), "register generic chunker") || !strings.Contains(err.Error(), "already registered") {
t.Fatalf("second Register() error = %v, want contextual duplicate error", err)
}
}
func completeRegistries() pipeline.Registries {
return pipeline.Registries{
Inputs: pipeline.NewInputAdapterRegistry(),
Chunkers: pipeline.NewChunkerRegistry(),
Extractors: pipeline.NewExtractorRegistry(),
Mergers: pipeline.NewMergerRegistry(),
Normalizers: pipeline.NewNormalizerRegistry(),
Validators: pipeline.NewValidatorRegistry(),
ValidatorChains: pipeline.NewValidatorChainRegistry(),
Outputs: pipeline.NewOutputEncoderRegistry(),
}
}
func assertKeys(t *testing.T, name string, got, want []string) {
t.Helper()
if !reflect.DeepEqual(got, want) {
t.Fatalf("%s keys = %#v, want %#v", name, got, want)
}
}

View File

@@ -0,0 +1,22 @@
// Package register composes the production Seriatim module family.
package register
import (
"fmt"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/input/seriatim"
)
// Register adds all production Seriatim modules.
func Register(registries pipeline.Registries, assets *llm.AssetRegistry) error {
_ = assets
if registries.Inputs == nil {
return fmt.Errorf("seriatim registrar: input registry must not be nil")
}
if err := seriatim.Register(registries.Inputs); err != nil {
return fmt.Errorf("register transcript input: %w", err)
}
return nil
}

View File

@@ -0,0 +1,37 @@
package register
import (
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestRegisterAddsSeriatimFamily(t *testing.T) {
registries := pipeline.Registries{Inputs: pipeline.NewInputAdapterRegistry()}
if err := Register(registries, nil); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
if got, want := registries.Inputs.RegisteredKeys(), []string{"seriatim"}; !reflect.DeepEqual(got, want) {
t.Fatalf("input keys = %#v, want %#v", got, want)
}
}
func TestRegisterRejectsNilInputRegistry(t *testing.T) {
err := Register(pipeline.Registries{}, nil)
if err == nil || !strings.Contains(err.Error(), "input registry must not be nil") {
t.Fatalf("Register() error = %v, want nil input registry error", err)
}
}
func TestRegisterReportsDuplicateSeriatimRegistration(t *testing.T) {
registries := pipeline.Registries{Inputs: pipeline.NewInputAdapterRegistry()}
if err := Register(registries, nil); err != nil {
t.Fatalf("first Register() error = %v, want nil", err)
}
err := Register(registries, nil)
if err == nil || !strings.Contains(err.Error(), "register transcript input") || !strings.Contains(err.Error(), "already registered") {
t.Fatalf("second Register() error = %v, want contextual duplicate error", err)
}
}