Refactor feedkit boundaries ahead of v1

Remove global Postgres schema registration in favor of explicit schema-aware sink factory wiring, and update weatherfeeder to register the Postgres sink explicitly. Add optional per-source HTTP timeout and response body limit overrides while keeping feedkit defaults. Remove remaining legacy source/config compatibility surfaces, including singular kind support and old source registry/type aliases, and migrate weatherfeeder sources to plural `Kinds()` metadata. Clean up related docs, tests, and sample config to match the new Postgres, HTTP, and NATS configuration model.

README.md

@@ -1,127 +1,92 @@
 # feedkit
 
-`feedkit` provides domain-agnostic plumbing for feed-processing daemons.
+`feedkit` is a small Go toolkit for building feed-processing daemons.
 
-A daemon built on feedkit typically:
+It gives you the reusable plumbing around collection, processing, routing, and
-- ingests upstream input (polling APIs or consuming streams)
+emission, while leaving domain concepts, schemas, and application wiring in
+your daemon. The intended shape is a family of sibling applications such as
+`weatherfeeder`, `newsfeeder`, or `earthquakefeeder` that all share the same
+infrastructure patterns without sharing domain logic.
+
+## What It Does
+
+A daemon built on `feedkit` typically:
+- ingests upstream input by polling HTTP APIs or consuming streams
 - emits domain-agnostic `event.Event` values
-- applies optional processing (normalization, dedupe, policy)
+- optionally processes those events with stages like dedupe or normalization
-- routes events to sinks (stdout, NATS, files, databases, etc.)
+- routes events to one or more sinks such as stdout, NATS, or Postgres
+
+Conceptually, the pipeline is:
+
+`Collect -> Process -> Route -> Emit`
 
 ## Philosophy
 
-feedkit is not a framework. It provides small composable packages and leaves
+`feedkit` is intentionally not a framework.
-lifecycle, domain schemas, and domain-specific validation in your daemon.
 
-## Conceptual pipeline
+It does not try to own:
+- your domain payload schemas
+- your domain event kinds
+- your daemon lifecycle or `main.go`
+- your observability stack or deployment model
 
-Collect -> Process (optional stages, including dedupe + normalize) -> Route -> Emit
+Instead, it provides small composable packages that are easy to wire together in
+different daemons.
 
-| Stage | Package(s) |
+## When To Use It
-|---|---|
-| Collect | `sources`, `scheduler` |
-| Process | `pipeline`, `processors`, `processors/dedupe`, `processors/normalize` (optional stages) |
-| Route | `dispatch` |
-| Emit | `sinks` |
-| Configure | `config` |
 
-## Core packages
+`feedkit` is a good fit when you want:
+- multiple small ingestion daemons with shared infrastructure patterns
+- clear separation between raw upstream payloads and normalized canonical models
+- reusable routing and sink behavior across domains
+- strong config and event-envelope conventions without centralizing domain rules
 
-### `config`
+It is a poor fit if you want a monolithic framework that dictates application
+structure end-to-end.
 
-Loads YAML config with strict decoding and domain-agnostic validation.
+## Built-In Capabilities
 
-`SourceConfig` supports both source modes:
+`feedkit` currently includes:
-- `mode: poll` requires `every`
+- strict YAML config loading and validation
-- `mode: stream` forbids `every`
+- polling and streaming source abstractions
-- omitted `mode` means auto (inferred from the registered driver type)
+- scheduler orchestration for configured sources
+- optional pipeline processors
+- built-in dedupe and normalization processors
+- route compilation and sink fanout
+- built-in sinks for `stdout`, `nats`, and `postgres`
 
-It also supports optional expected source kinds:
+The Postgres sink is intentionally split between feedkit-owned infrastructure
-- `kinds: ["observation", "alert"]` (preferred)
+and daemon-owned schema mapping. `feedkit` manages connection setup, DDL,
-- `kind: "observation"` (legacy fallback)
+writes, and pruning; downstream applications define the schema and event mapper.
 
-### `event`
+## Typical Wiring
 
-Defines the domain-agnostic event envelope (`event.Event`) used across the system.
+At a high level, a daemon built on `feedkit` does this:
-
-### `sources`
-
-Defines source interfaces and driver registry:
-
-```go
-type Input interface {
-    Name() string
-}
-
-type PollSource interface {
-    Input
-    Poll(ctx context.Context) ([]event.Event, error)
-}
-
-type StreamSource interface {
-    Input
-    Run(ctx context.Context, out chan<- event.Event) error
-}
-```
-
-Notes:
-- a poll can emit `0..N` events
-- stream sources emit events continuously
-- a single source may emit multiple event kinds
-- driver implementations live in downstream daemons and are registered via `sources.Registry`
-
-### `scheduler`
-
-Runs one goroutine per source job:
-- poll sources: cadence driven (`every` + jitter)
-- stream sources: continuous run loop
-
-### `pipeline`
-
-Optional processing chain between collection and dispatch.
-Processors can transform, drop, or reject events.
-
-### `processors`
-
-Defines the generic processor interface and a named-driver registry used by
-daemons to build ordered processor chains.
-
-### `processors/dedupe`
-
-Built-in in-memory LRU dedupe processor that drops repeated events by `Event.ID`.
-
-### `processors/normalize`
-
-Concrete normalization processor implementation. Typical use: sources emit raw
-payload events, then a normalize stage maps them to canonical schemas.
-
-### `dispatch`
-
-Compiles routes and fans out events to sinks with per-sink queue/worker isolation.
-
-### `sinks`
-
-Defines sink interface and sink registry. Built-ins include:
-- `stdout`
-- `nats`
-- `postgres`
-
-Detailed Postgres configuration and wiring examples live in package docs:
-`sinks/doc.go`.
-
-## Typical wiring
 
 1. Load config.
-2. Register/build sources from `cfg.Sources`.
+2. Register domain-specific source drivers.
-3. Register/build sinks from `cfg.Sinks`.
+3. Register built-in and/or custom sinks.
-4. Compile routes.
+4. Build sources, sinks, and optional processor chain from config.
-5. Start scheduler (`sources -> bus`).
+5. Compile routes.
-6. Start dispatcher (`bus -> pipeline -> sinks`).
+6. Start the scheduler and dispatcher.
 
-## Non-goals
+The package docs are the better source of truth for code-level details. In
+particular, each subpackage `doc.go` describes its external API surface and any
+optional helper APIs in `helpers.go`.
 
-feedkit intentionally does not:
+## Package Layout
-- define domain payload schemas
+
-- enforce domain-specific event kinds
+The major packages are:
-- own application lifecycle
+- `config`: config loading and validation
-- prescribe observability stack choices
+- `event`: the domain-agnostic event envelope
+- `sources`: source interfaces and reusable source helpers
+- `scheduler`: source execution and cadence management
+- `processors`: processor interfaces and registry
+- `processors/dedupe`: built-in in-memory dedupe processor
+- `processors/normalize`: built-in normalization processor and helpers
+- `pipeline`: optional processor chain
+- `dispatch`: route compilation and fanout
+- `sinks`: sink interfaces, built-ins, and explicit Postgres factory helpers
+
+The root package docs in `doc.go` provide a concise package-by-package map for
+Go documentation consumers.

config/config.go

@@ -69,19 +69,13 @@ type SourceConfig struct {
 	// If set, it describes the expected emitted event kinds for this source.
 	Kinds []string `yaml:"kinds"`
 
-	// Kind is the legacy singular form. Prefer "kinds".
-	// If both kind and kinds are set, validation fails.
-	Kind string `yaml:"kind"`
-
 	// Params are driver-specific settings (URL, headers, station IDs, API keys, etc.).
 	// The driver implementation is responsible for reading/validating these.
 	Params map[string]any `yaml:"params"`
 }
 
 // ExpectedKinds returns normalized expected kinds from config.
-// "kinds" takes precedence; "kind" is used as a legacy fallback.
 func (cfg SourceConfig) ExpectedKinds() []string {
-	if len(cfg.Kinds) > 0 {
 	out := make([]string, 0, len(cfg.Kinds))
 	for _, k := range cfg.Kinds {
 		k = strings.TrimSpace(k)
@@ -90,18 +84,16 @@ func (cfg SourceConfig) ExpectedKinds() []string {
 		}
 		out = append(out, k)
 	}
-		return out
+	if len(out) == 0 {
-	}
-	if k := strings.TrimSpace(cfg.Kind); k != "" {
-		return []string{k}
-	}
 		return nil
 	}
+	return out
+}
 
 // SinkConfig describes one output sink adapter.
 type SinkConfig struct {
 	Name   string         `yaml:"name"`
-	Driver string         `yaml:"driver"` // "stdout", "file", "postgres", "rabbitmq", ...
+	Driver string         `yaml:"driver"` // "stdout", "nats", "postgres", ...
 	Params map[string]any `yaml:"params"` // sink-specific settings
 }
 

config/config_test.go

@@ -12,20 +12,12 @@ func TestSourceConfigExpectedKinds(t *testing.T) {
 		want []string
 	}{
 		{
-			name: "plural kinds preferred",
+			name: "plural kinds normalized",
 			cfg: SourceConfig{
 				Kinds: []string{" observation ", "forecast"},
-				Kind:  "alert",
 			},
 			want: []string{"observation", "forecast"},
 		},
-		{
-			name: "legacy singular fallback",
-			cfg: SourceConfig{
-				Kind: " alert ",
-			},
-			want: []string{"alert"},
-		},
 		{
 			name: "empty kinds",
 			cfg:  SourceConfig{},

config/load.go

@@ -105,13 +105,7 @@ func (c *Config) Validate() error {
 				}
 			}
 
-			// Kind/Kinds (optional)
+			// Kinds (optional)
-			if s.Kind != "" && len(s.Kinds) > 0 {
-				m.Add(fieldErr(path+".kind", `cannot be set when "kinds" is provided (use only "kinds")`))
-			}
-			if s.Kind != "" && strings.TrimSpace(s.Kind) == "" {
-				m.Add(fieldErr(path+".kind", "cannot be blank (omit it entirely, or provide a non-empty string)"))
-			}
 			for j, k := range s.Kinds {
 				kpath := fmt.Sprintf("%s.kinds[%d]", path, j)
 				if strings.TrimSpace(k) == "" {
@@ -141,7 +135,7 @@ func (c *Config) Validate() error {
 			}
 
 			if strings.TrimSpace(s.Driver) == "" {
-				m.Add(fieldErr(path+".driver", "is required (stdout|file|postgres|rabbitmq|...)"))
+				m.Add(fieldErr(path+".driver", "is required (stdout|nats|postgres|...)"))
 			}
 
 			// Params can be nil; that's fine.

config/validate_test.go

@@ -114,31 +114,6 @@ func TestValidate_SourceModeRejectsUnknownValue(t *testing.T) {
 	}
 }
 
-func TestValidate_SourceKindAndKindsConflict(t *testing.T) {
-	cfg := &Config{
-		Sources: []SourceConfig{
-			{
-				Name:   "src1",
-				Driver: "driver1",
-				Every:  Duration{Duration: time.Minute},
-				Kind:   "observation",
-				Kinds:  []string{"forecast"},
-			},
-		},
-		Sinks: []SinkConfig{
-			{Name: "sink1", Driver: "stdout"},
-		},
-	}
-
-	err := cfg.Validate()
-	if err == nil {
-		t.Fatalf("expected error, got nil")
-	}
-	if !strings.Contains(err.Error(), `sources[0].kind`) {
-		t.Fatalf("expected error to mention sources[0].kind, got: %v", err)
-	}
-}
-
 func TestValidate_SourceKindsRejectBlankEntries(t *testing.T) {
 	cfg := &Config{
 		Sources: []SourceConfig{

doc.go

@@ -1,130 +1,56 @@
-// Package feedkit provides domain-agnostic plumbing for feed-processing daemons.
+// Package feedkit provides a high-level map of the feedkit package set.
 //
-// A feed daemon ingests upstream input, turns it into event.Event values, applies
+// Most real applications do not import the root package directly. Instead, they
-// optional processing, and emits to sinks.
+// compose the subpackages that handle configuration, collection, processing,
+// routing, and sinks.
 //
-// Conceptual flow:
+// The usual flow through feedkit is:
 //
-//	Collect -> Process (optional stages, including dedupe + normalize) -> Route -> Emit
+//	Collect -> Process -> Route -> Emit
 //
-// In feedkit this maps to:
+// That flow maps to packages like this:
-//
-//	Collect: sources + scheduler
-//	Process: pipeline + processors + processors/dedupe + processors/normalize (optional stages)
-//	Route:   dispatch
-//	Emit:    sinks
-//	Config:  config
-//
-// feedkit intentionally does not define domain payload schemas or domain-specific
-// validation rules. Those belong in each concrete daemon.
-//
-// Public packages
 //
 //   - config
-//     YAML config loading/validation (strict decode + domain-agnostic checks).
+//     Loads and validates daemon config. This package owns domain-agnostic
-//
+//     config shape and consistency checks.
-//     SourceConfig supports both polling and streaming sources:
-//
-//   - mode: "poll" | "stream" | omitted (auto by driver type)
-//
-//   - every: poll interval (required for mode="poll")
-//
-//   - kinds: optional expected emitted kinds
-//
-//   - kind: legacy singular fallback
-//
-//   - params: driver-specific settings
 //
 //   - event
-//     Domain-agnostic event envelope (ID, Kind, Source, EmittedAt, Schema, Payload).
+//     Defines the event.Event envelope shared across sources, processors,
+//     dispatch, and sinks.
 //
 //   - sources
-//     Source abstractions and source-driver registry.
+//     Defines polling and streaming source interfaces, the source registry, and
-//
+//     reusable source helpers.
-//     There are two source interfaces:
-//
-//   - PollSource: Poll(ctx) ([]event.Event, error)
-//
-//   - StreamSource: Run(ctx, out) error
-//
-//     Both share Input{Name()}. A source may emit 0..N events per poll/run step,
-//     and may emit multiple event kinds.
-//
-//     For HTTP-backed polling sources, sources.NewHTTPSource provides a shared
-//     helper for generic params:
-//
-//   - params.url
-//
-//   - params.user_agent
-//
-//   - params.conditional (optional, default true)
-//
-//     When conditional polling is enabled, feedkit opportunistically uses ETag
-//     and Last-Modified validators. A 304 Not Modified response is treated as a
-//     successful poll that emits no events.
 //
 //   - scheduler
-//     Runs one goroutine per job:
+//     Runs configured sources on a cadence or as long-lived stream workers.
-//
-//   - PollSource jobs run on Every (+ jitter)
-//
-//   - StreamSource jobs run continuously
-//
-//   - pipeline
-//     Processor chain between scheduler and dispatch.
-//     Processors can transform, drop, or reject events.
 //
 //   - processors
-//     Generic processor interface and named factory registry for wiring chains.
+//     Defines the generic processor interface and registry used to build
+//     ordered processor chains.
 //
 //   - processors/dedupe
-//     Built-in in-memory LRU dedupe processor keyed by Event.ID.
+//     Built-in in-memory dedupe processor keyed by Event.ID.
 //
 //   - processors/normalize
-//     Concrete pipeline processor for raw->canonical mapping.
+//     Built-in normalization processor plus helper APIs for raw-to-canonical
-//     If no normalizer matches, the event passes through unchanged by default.
+//     event mapping.
+//
+//   - pipeline
+//     Applies an ordered processor chain between collection and dispatch.
 //
 //   - dispatch
-//     Routes events to sinks and isolates slow sinks via per-sink queues/workers.
+//     Compiles routes and fans events out to sinks with per-sink isolation.
 //
 //   - sinks
-//     Sink abstractions + sink registry.
+//     Defines sink interfaces, the sink registry, schema-free built-in sinks,
-//     Built-ins include stdout, NATS, and Postgres. For Postgres, downstream
+//     and explicit Postgres factory helpers.
-//     code registers table schemas/mappers while feedkit manages DDL, writes,
-//     optional automatic retention pruning (via sink params.prune), and
-//     manual prune helpers. Postgres table schemas must declare PruneColumn.
 //
-// Typical wiring (daemon main.go)
+// feedkit is intentionally domain-agnostic. Domain schemas, domain event kinds,
+// upstream-specific parsing, and daemon lifecycle remain the responsibility of
+// each concrete application.
 //
-//  1. Load config.
+// For repository-level overview and usage narrative, see README.md. For
-//  2. Register source drivers and build sources from config.Sources.
+// code-level details, each subpackage doc.go is the source of truth for that
-//  3. Register sink drivers and build sinks from config.Sinks.
+// package's public API surface and optional helpers.
-//  4. Compile routes.
-//  5. Start scheduler (sources -> bus) and dispatcher (bus -> pipeline -> sinks).
-//
-// Sketch:
-//
-//	cfg, _ := config.Load("config.yml")
-//	srcReg := sources.NewRegistry()
-//	// domain registers poll/stream drivers...
-//
-//	var jobs []scheduler.Job
-//	for _, sc := range cfg.Sources {
-//	    src, _ := srcReg.BuildInput(sc)
-//	    jobs = append(jobs, scheduler.Job{
-//	        Source: src,
-//	        Every:  sc.Every.Duration,
-//	    })
-//	}
-//
-//	bus := make(chan event.Event, 256)
-//	s := &scheduler.Scheduler{Jobs: jobs, Out: bus, Logf: logf}
-//	// start dispatcher similarly...
-//
-// # Context and cancellation
-//
-// All blocking work should honor context cancellation:
-//   - source polling/streaming I/O
-//   - sink consumption
-//   - any expensive processor work
 package feedkit

pipeline/ratelimit.go

@@ -1,5 +0,0 @@
-package pipeline
-
-// Placeholder for rate limit processor:
-// - per source/kind sink routing limits
-// - cooldown windows

processors/normalize/doc.go

@@ -1,16 +1,18 @@
-// Package normalize provides a concrete normalization processor for feedkit pipelines.
+// Package normalize provides the feedkit normalization processor and related
+// helper APIs for raw-to-canonical event mapping.
 //
-// Motivation:
+// External API surface:
-// Many daemons have sources that:
+//   - Processor: concrete processors.Processor implementation
-//  1. fetch raw upstream data (often JSON), and
+//   - Normalizer / Func: normalization interface and ergonomic function adapter
-//  2. transform it into a domain's normalized payload format.
 //
-// Doing both steps inside Source.Poll works, but tends to make sources large and
+// Optional helpers from helpers.go:
-// encourages duplication (unit conversions, common mapping helpers, etc.).
+//   - PayloadJSONBytes: extract supported JSON-shaped payloads into bytes
+//   - DecodeJSONPayload: decode an event payload into a typed struct
+//   - FinalizeEvent: copy the input event envelope onto a normalized output
 //
-// This package lets a source emit a "raw" event (e.g., Schema="raw.openweather.current.v1",
+// Typical usage:
-// Payload=json.RawMessage), and then a normalize.Processor can convert it into a
+// sources emit raw events (often with json.RawMessage payloads), then a
-// normalized event (e.g., Schema="weather.observation.v1", Payload=WeatherObservation{}).
+// normalize.Processor converts matching raw schemas into canonical payloads.
 //
 // Key property: normalization is optional.
 // If no Normalizer matches an event, Processor passes it through unchanged by default.

scheduler/worker.go

@@ -1,7 +0,0 @@
-package scheduler
-
-// Placeholder for per-source worker logic:
-// - ticker loop
-// - jitter
-// - backoff on errors
-// - emits events into scheduler.Out

sinks/builtins.go

@@ -1,11 +1,6 @@
 package sinks
 
-import (
+import "gitea.maximumdirect.net/ejr/feedkit/config"
-	"fmt"
-	"strings"
-
-	"gitea.maximumdirect.net/ejr/feedkit/config"
-)
 
 // RegisterBuiltins registers sink drivers included in this binary.
 //
@@ -17,39 +12,8 @@ func RegisterBuiltins(r *Registry) {
 		return NewStdoutSink(cfg.Name), nil
 	})
 
-	// File sink: writes/archives events somewhere on disk.
-	r.Register("file", func(cfg config.SinkConfig) (Sink, error) {
-		return NewFileSinkFromConfig(cfg)
-	})
-
-	// Postgres sink: persists events durably.
-	r.Register("postgres", func(cfg config.SinkConfig) (Sink, error) {
-		return NewPostgresSinkFromConfig(cfg)
-	})
-
 	// NATS sink: publishes events to a broker for downstream consumers.
 	r.Register("nats", func(cfg config.SinkConfig) (Sink, error) {
 		return NewNATSSinkFromConfig(cfg)
 	})
 }
-
-// ---- helpers for validating sink params ----
-//
-// These helpers live in sinks (not config) on purpose:
-// - config is domain-agnostic and should not embed driver-specific validation helpers.
-// - sinks are adapters; validating their own params here keeps the logic near the driver.
-
-func requireStringParam(cfg config.SinkConfig, key string) (string, error) {
-	v, ok := cfg.Params[key]
-	if !ok {
-		return "", fmt.Errorf("sink %q: params.%s is required", cfg.Name, key)
-	}
-	s, ok := v.(string)
-	if !ok {
-		return "", fmt.Errorf("sink %q: params.%s must be a string", cfg.Name, key)
-	}
-	if strings.TrimSpace(s) == "" {
-		return "", fmt.Errorf("sink %q: params.%s cannot be empty", cfg.Name, key)
-	}
-	return s, nil
-}

sinks/doc.go

@@ -1,18 +1,27 @@
-// Package sinks provides sink abstractions, a sink driver registry, and several
+// Package sinks defines the feedkit sink interface, sink driver registry, and
-// built-in sink drivers.
+// built-in infrastructure sinks.
 //
-// Built-in drivers:
+// External API surface:
+//   - Sink: adapter interface that consumes event.Event values
+//   - Registry / NewRegistry: named sink factory registry
+//   - RegisterBuiltins: registers the schema-free built-in sink drivers
+//
+// Built-in sink implementations:
 //   - stdout
 //   - nats
 //   - postgres
 //
+// Optional helpers from helpers.go:
+//   - PostgresFactory: returns a sink factory for the built-in Postgres sink
+//     using a provided downstream schema
+//
 // # NATS built-in overview
 //
 // The NATS sink publishes each event as JSON to a configured subject.
 //
 // Required params:
 //   - url: NATS server URL (for example, nats://localhost:4222)
-//   - exchange: NATS subject to publish to
+//   - subject: NATS subject to publish to
 //
 // Example config:
 //
@@ -21,7 +30,7 @@
 //	    driver: nats
 //	    params:
 //	      url: nats://localhost:4222
-//	      exchange: feedkit.events
+//	      subject: feedkit.events
 //
 // # Postgres built-in overview
 //
@@ -50,7 +59,9 @@
 //
 // Example downstream wiring:
 //
-//	sinks.MustRegisterPostgresSchema("pg_main", sinks.PostgresSchema{
+//	sinkReg := sinks.NewRegistry()
+//	sinks.RegisterBuiltins(sinkReg)
+//	sinkReg.Register("postgres", sinks.PostgresFactory(sinks.PostgresSchema{
 //	    Tables: []sinks.PostgresTable{
 //	        {
 //	            Name: "events",
@@ -79,7 +90,7 @@
 //	            },
 //	        }, nil
 //	    },
-//	})
+//	}))
 //
 // Manual pruning via type assertion (administrative helpers):
 //

sinks/file.go

@@ -1,30 +0,0 @@
-package sinks
-
-import (
-	"context"
-	"fmt"
-
-	"gitea.maximumdirect.net/ejr/feedkit/config"
-	"gitea.maximumdirect.net/ejr/feedkit/event"
-)
-
-type FileSink struct {
-	name string
-	path string
-}
-
-func NewFileSinkFromConfig(cfg config.SinkConfig) (Sink, error) {
-	path, err := requireStringParam(cfg, "path")
-	if err != nil {
-		return nil, err
-	}
-	return &FileSink{name: cfg.Name, path: path}, nil
-}
-
-func (s *FileSink) Name() string { return s.name }
-
-func (s *FileSink) Consume(ctx context.Context, e event.Event) error {
-	_ = ctx
-	_ = e
-	return fmt.Errorf("file sink: TODO implement (path=%s)", s.path)
-}

sinks/helpers.go

@@ -7,21 +7,29 @@ import (
 	"gitea.maximumdirect.net/ejr/feedkit/config"
 )
 
-// RegisterPostgresSchemaForConfiguredSinks registers one Postgres schema for each
+// requireStringParam returns a non-empty string sink param.
-// configured sink using driver=postgres.
+//
-func RegisterPostgresSchemaForConfiguredSinks(cfg *config.Config, schema PostgresSchema) error {
+// This helper is intentionally local to sinks rather than config so
-	if cfg == nil {
+// driver-specific validation stays close to the adapters that use it.
-		return fmt.Errorf("register postgres schemas: config is nil")
+func requireStringParam(cfg config.SinkConfig, key string) (string, error) {
+	v, ok := cfg.Params[key]
+	if !ok {
+		return "", fmt.Errorf("sink %q: params.%s is required", cfg.Name, key)
+	}
+	s, ok := v.(string)
+	if !ok {
+		return "", fmt.Errorf("sink %q: params.%s must be a string", cfg.Name, key)
+	}
+	if strings.TrimSpace(s) == "" {
+		return "", fmt.Errorf("sink %q: params.%s cannot be empty", cfg.Name, key)
+	}
+	return s, nil
 }
 
-	for i, sk := range cfg.Sinks {
+// PostgresFactory returns a sink factory that builds the built-in Postgres sink
-		if !strings.EqualFold(strings.TrimSpace(sk.Driver), "postgres") {
+// using the provided downstream schema definition.
-			continue
+func PostgresFactory(schema PostgresSchema) Factory {
-		}
+	return func(cfg config.SinkConfig) (Sink, error) {
-		if err := RegisterPostgresSchema(sk.Name, schema); err != nil {
+		return NewPostgresSinkFromConfig(cfg, schema)
-			return fmt.Errorf("register postgres schema for sinks[%d] name=%q: %w", i, sk.Name, err)
 	}
 }
-
-	return nil
-}

sinks/helpers_test.go

@@ -2,55 +2,19 @@ package sinks
 
 import (
 	"context"
-	"fmt"
-	"strings"
 	"testing"
-	"time"
 
 	"gitea.maximumdirect.net/ejr/feedkit/config"
 	"gitea.maximumdirect.net/ejr/feedkit/event"
 )
 
-func TestRegisterPostgresSchemaForConfiguredSinksNilConfig(t *testing.T) {
+func TestPostgresFactoryReturnsWorkingFactory(t *testing.T) {
-	err := RegisterPostgresSchemaForConfiguredSinks(nil, testPostgresSchema())
+	factory := PostgresFactory(testPostgresSchema())
-	if err == nil {
+	if factory == nil {
-		t.Fatalf("RegisterPostgresSchemaForConfiguredSinks(nil) expected error")
+		t.Fatalf("PostgresFactory() returned nil")
 	}
-	if !strings.Contains(err.Error(), "config is nil") {
+	if _, err := factory(config.SinkConfig{}); err == nil {
-		t.Fatalf("error = %q, want config is nil", err)
+		t.Fatalf("factory(config) expected parameter validation error")
-	}
-}
-
-func TestRegisterPostgresSchemaForConfiguredSinksNonPostgresNoOp(t *testing.T) {
-	cfg := &config.Config{
-		Sinks: []config.SinkConfig{
-			{Name: uniqueSinkName("stdout"), Driver: "stdout"},
-			{Name: uniqueSinkName("nats"), Driver: "nats"},
-		},
-	}
-
-	if err := RegisterPostgresSchemaForConfiguredSinks(cfg, testPostgresSchema()); err != nil {
-		t.Fatalf("RegisterPostgresSchemaForConfiguredSinks(non-postgres) error = %v", err)
-	}
-}
-
-func TestRegisterPostgresSchemaForConfiguredSinksDuplicateRegistrationFails(t *testing.T) {
-	cfg := &config.Config{
-		Sinks: []config.SinkConfig{
-			{Name: uniqueSinkName("pg"), Driver: "postgres"},
-		},
-	}
-
-	if err := RegisterPostgresSchemaForConfiguredSinks(cfg, testPostgresSchema()); err != nil {
-		t.Fatalf("first RegisterPostgresSchemaForConfiguredSinks() error = %v", err)
-	}
-
-	err := RegisterPostgresSchemaForConfiguredSinks(cfg, testPostgresSchema())
-	if err == nil {
-		t.Fatalf("second RegisterPostgresSchemaForConfiguredSinks() expected duplicate error")
-	}
-	if !strings.Contains(err.Error(), "already registered") {
-		t.Fatalf("error = %q, want already registered", err)
 	}
 }
 
@@ -80,7 +44,3 @@ func testPostgresSchema() PostgresSchema {
 		},
 	}
 }
-
-func uniqueSinkName(prefix string) string {
-	return fmt.Sprintf("%s_%d", prefix, time.Now().UnixNano())
-}

sinks/nats.go

@@ -15,7 +15,7 @@ import (
 type NATSSink struct {
 	name    string
 	url     string
-	exchange string
+	subject string
 
 	mu   sync.Mutex
 	conn *nats.Conn
@@ -26,11 +26,11 @@ func NewNATSSinkFromConfig(cfg config.SinkConfig) (Sink, error) {
 	if err != nil {
 		return nil, err
 	}
-	ex, err := requireStringParam(cfg, "exchange")
+	subject, err := requireStringParam(cfg, "subject")
 	if err != nil {
 		return nil, err
 	}
-	return &NATSSink{name: cfg.Name, url: url, exchange: ex}, nil
+	return &NATSSink{name: cfg.Name, url: url, subject: subject}, nil
 }
 
 func (r *NATSSink) Name() string { return r.name }
@@ -59,7 +59,7 @@ func (r *NATSSink) Consume(ctx context.Context, e event.Event) error {
 	if err := ctx.Err(); err != nil {
 		return err
 	}
-	if err := conn.Publish(r.exchange, b); err != nil {
+	if err := conn.Publish(r.subject, b); err != nil {
 		return fmt.Errorf("NATS sink: publish: %w", err)
 	}
 	return nil

sinks/nats_test.go

@@ -0,0 +1,47 @@
+package sinks
+
+import (
+	"strings"
+	"testing"
+
+	"gitea.maximumdirect.net/ejr/feedkit/config"
+)
+
+func TestNewNATSSinkFromConfigRequiresSubject(t *testing.T) {
+	sink, err := NewNATSSinkFromConfig(config.SinkConfig{
+		Name:   "nats-main",
+		Driver: "nats",
+		Params: map[string]any{
+			"url":     "nats://localhost:4222",
+			"subject": "feedkit.events",
+		},
+	})
+	if err != nil {
+		t.Fatalf("NewNATSSinkFromConfig() error = %v", err)
+	}
+
+	natsSink, ok := sink.(*NATSSink)
+	if !ok {
+		t.Fatalf("sink type = %T, want *NATSSink", sink)
+	}
+	if natsSink.subject != "feedkit.events" {
+		t.Fatalf("subject = %q, want feedkit.events", natsSink.subject)
+	}
+}
+
+func TestNewNATSSinkFromConfigRejectsLegacyExchange(t *testing.T) {
+	_, err := NewNATSSinkFromConfig(config.SinkConfig{
+		Name:   "nats-main",
+		Driver: "nats",
+		Params: map[string]any{
+			"url":      "nats://localhost:4222",
+			"exchange": "feedkit.events",
+		},
+	})
+	if err == nil {
+		t.Fatalf("NewNATSSinkFromConfig() expected error")
+	}
+	if !strings.Contains(err.Error(), "params.subject is required") {
+		t.Fatalf("error = %q, want params.subject is required", err)
+	}
+}

sinks/postgres.go

@@ -88,7 +88,7 @@ type PostgresSink struct {
 	pruneWindow time.Duration
 }
 
-func NewPostgresSinkFromConfig(cfg config.SinkConfig) (Sink, error) {
+func NewPostgresSinkFromConfig(cfg config.SinkConfig, schemaDef PostgresSchema) (Sink, error) {
 	uri, err := requireStringParam(cfg, "uri")
 	if err != nil {
 		return nil, err
@@ -106,9 +106,9 @@ func NewPostgresSinkFromConfig(cfg config.SinkConfig) (Sink, error) {
 		return nil, err
 	}
 
-	schema, ok := lookupPostgresSchema(cfg.Name)
+	schema, err := compilePostgresSchema(schemaDef)
-	if !ok {
+	if err != nil {
-		return nil, fmt.Errorf("postgres sink %q: no schema registered (call sinks.RegisterPostgresSchema before building sinks)", cfg.Name)
+		return nil, fmt.Errorf("postgres sink %q: compile schema: %w", cfg.Name, err)
 	}
 
 	dsn, err := buildPostgresDSN(uri, username, password)

sinks/postgres_schema.go

@@ -4,7 +4,6 @@ import (
 	"context"
 	"fmt"
 	"strings"
-	"sync"
 	"time"
 
 	"gitea.maximumdirect.net/ejr/feedkit/event"
@@ -72,51 +71,6 @@ type postgresTableCompiled struct {
 	indexes     []PostgresIndex
 }
 
-var (
-	postgresSchemaRegistryMu sync.RWMutex
-	postgresSchemaRegistry   = map[string]postgresSchemaCompiled{}
-)
-
-// RegisterPostgresSchema registers one downstream schema by sink name.
-//
-// This should be called by downstream daemon wiring code before sink
-// construction. Duplicate sink-name registrations are rejected.
-func RegisterPostgresSchema(sinkName string, schema PostgresSchema) error {
-	sinkName = strings.TrimSpace(sinkName)
-	if sinkName == "" {
-		return fmt.Errorf("postgres schema: sink name cannot be empty")
-	}
-
-	compiled, err := compilePostgresSchema(schema)
-	if err != nil {
-		return err
-	}
-
-	postgresSchemaRegistryMu.Lock()
-	defer postgresSchemaRegistryMu.Unlock()
-
-	if _, exists := postgresSchemaRegistry[sinkName]; exists {
-		return fmt.Errorf("postgres schema: sink %q already registered", sinkName)
-	}
-
-	postgresSchemaRegistry[sinkName] = compiled
-	return nil
-}
-
-func MustRegisterPostgresSchema(sinkName string, schema PostgresSchema) {
-	if err := RegisterPostgresSchema(sinkName, schema); err != nil {
-		panic(err)
-	}
-}
-
-func lookupPostgresSchema(sinkName string) (postgresSchemaCompiled, bool) {
-	postgresSchemaRegistryMu.RLock()
-	defer postgresSchemaRegistryMu.RUnlock()
-
-	s, ok := postgresSchemaRegistry[sinkName]
-	return s, ok
-}
-
 func compilePostgresSchema(schema PostgresSchema) (postgresSchemaCompiled, error) {
 	if schema.MapEvent == nil {
 		return postgresSchemaCompiled{}, fmt.Errorf("postgres schema: map function is required")

sinks/postgres_test.go

@@ -96,20 +96,12 @@ func (d *fakeDB) Close() error {
 	return nil
 }
 
-func resetPostgresSchemaRegistryForTest() {
-	postgresSchemaRegistryMu.Lock()
-	defer postgresSchemaRegistryMu.Unlock()
-	postgresSchemaRegistry = map[string]postgresSchemaCompiled{}
-}
-
 func withPostgresTestState(t *testing.T) {
 	t.Helper()
 
-	resetPostgresSchemaRegistryForTest()
 	oldOpen := openPostgresDB
 	t.Cleanup(func() {
 		openPostgresDB = oldOpen
-		resetPostgresSchemaRegistryForTest()
 	})
 }
 
@@ -183,35 +175,8 @@ func mustCompileSchema(t *testing.T, s PostgresSchema) postgresSchemaCompiled {
 	return compiled
 }
 
-func TestRegisterPostgresSchema(t *testing.T) {
+func TestCompilePostgresSchemaRejectsInvalidSchema(t *testing.T) {
-	withPostgresTestState(t)
+	_, err := compilePostgresSchema(PostgresSchema{
-
-	err := RegisterPostgresSchema("pg", schemaOneTable(func(_ context.Context, _ event.Event) ([]PostgresWrite, error) {
-		return nil, nil
-	}))
-	if err != nil {
-		t.Fatalf("register schema: %v", err)
-	}
-
-	if _, ok := lookupPostgresSchema("pg"); !ok {
-		t.Fatalf("expected schema registration")
-	}
-
-	err = RegisterPostgresSchema("pg", schemaOneTable(func(_ context.Context, _ event.Event) ([]PostgresWrite, error) {
-		return nil, nil
-	}))
-	if err == nil {
-		t.Fatalf("expected duplicate registration error")
-	}
-	if !strings.Contains(err.Error(), "already registered") {
-		t.Fatalf("unexpected duplicate error: %v", err)
-	}
-}
-
-func TestRegisterPostgresSchema_RejectsInvalidSchema(t *testing.T) {
-	withPostgresTestState(t)
-
-	err := RegisterPostgresSchema("pg", PostgresSchema{
 		Tables: []PostgresTable{
 			{
 				Name: "events",
@@ -230,7 +195,7 @@ func TestRegisterPostgresSchema_RejectsInvalidSchema(t *testing.T) {
 		t.Fatalf("unexpected schema validation error: %v", err)
 	}
 
-	err = RegisterPostgresSchema("pg2", PostgresSchema{
+	_, err = compilePostgresSchema(PostgresSchema{
 		Tables: []PostgresTable{
 			{
 				Name: "events",
@@ -254,7 +219,50 @@ func TestRegisterPostgresSchema_RejectsInvalidSchema(t *testing.T) {
 	}
 }
 
-func TestNewPostgresSinkFromConfig_MissingParams(t *testing.T) {
+func TestPostgresFactoryBuildsMultipleSinksWithSameSchema(t *testing.T) {
+	withPostgresTestState(t)
+
+	dbs := []*fakeDB{{}, {}}
+	var gotDSNs []string
+	openPostgresDB = func(dsn string) (postgresDB, error) {
+		gotDSNs = append(gotDSNs, dsn)
+		db := dbs[len(gotDSNs)-1]
+		return db, nil
+	}
+
+	factory := PostgresFactory(schemaOneTable(func(_ context.Context, _ event.Event) ([]PostgresWrite, error) {
+		return nil, nil
+	}))
+
+	for _, name := range []string{"pg_a", "pg_b"} {
+		sink, err := factory(config.SinkConfig{
+			Name:   name,
+			Driver: "postgres",
+			Params: map[string]any{
+				"uri":      "postgres://localhost/db",
+				"username": "user",
+				"password": "pass",
+			},
+		})
+		if err != nil {
+			t.Fatalf("factory(%q) error = %v", name, err)
+		}
+		if sink == nil {
+			t.Fatalf("factory(%q) returned nil sink", name)
+		}
+	}
+
+	if len(gotDSNs) != 2 {
+		t.Fatalf("len(gotDSNs) = %d, want 2", len(gotDSNs))
+	}
+	for i, db := range dbs {
+		if db.pingCalls != 1 {
+			t.Fatalf("db[%d] pingCalls = %d, want 1", i, db.pingCalls)
+		}
+	}
+}
+
+func TestNewPostgresSinkFromConfigMissingParams(t *testing.T) {
 	withPostgresTestState(t)
 
 	tests := []struct {
@@ -273,7 +281,7 @@ func TestNewPostgresSinkFromConfig_MissingParams(t *testing.T) {
 				Name:   "pg",
 				Driver: "postgres",
 				Params: tc.params,
-			})
+			}, schemaOneTable(func(_ context.Context, _ event.Event) ([]PostgresWrite, error) { return nil, nil }))
 			if err == nil {
 				t.Fatalf("expected error")
 			}
@@ -284,7 +292,7 @@ func TestNewPostgresSinkFromConfig_MissingParams(t *testing.T) {
 	}
 }
 
-func TestNewPostgresSinkFromConfig_MissingSchemaRegistration(t *testing.T) {
+func TestNewPostgresSinkFromConfigRejectsInvalidSchema(t *testing.T) {
 	withPostgresTestState(t)
 
 	_, err := NewPostgresSinkFromConfig(config.SinkConfig{
@@ -295,25 +303,29 @@ func TestNewPostgresSinkFromConfig_MissingSchemaRegistration(t *testing.T) {
 			"username": "user",
 			"password": "pass",
 		},
+	}, PostgresSchema{
+		Tables: []PostgresTable{
+			{
+				Name: "events",
+				Columns: []PostgresColumn{
+					{Name: "id", Type: "TEXT", Nullable: false},
+				},
+				PruneColumn: "missing_col",
+			},
+		},
+		MapEvent: func(_ context.Context, _ event.Event) ([]PostgresWrite, error) { return nil, nil },
 	})
 	if err == nil {
-		t.Fatalf("expected error")
+		t.Fatalf("expected invalid schema error")
 	}
-	if !strings.Contains(err.Error(), "no schema registered") {
+	if !strings.Contains(err.Error(), "compile schema") {
 		t.Fatalf("unexpected error: %v", err)
 	}
 }
 
-func TestNewPostgresSinkFromConfig_EagerInit(t *testing.T) {
+func TestNewPostgresSinkFromConfigEagerInit(t *testing.T) {
 	withPostgresTestState(t)
 
-	err := RegisterPostgresSchema("pg", schemaOneTable(func(_ context.Context, _ event.Event) ([]PostgresWrite, error) {
-		return nil, nil
-	}))
-	if err != nil {
-		t.Fatalf("register schema: %v", err)
-	}
-
 	db := &fakeDB{}
 	var gotDSN string
 	openPostgresDB = func(dsn string) (postgresDB, error) {
@@ -329,7 +341,7 @@ func TestNewPostgresSinkFromConfig_EagerInit(t *testing.T) {
 			"username": "app_user",
 			"password": "app_pass",
 		},
-	})
+	}, schemaOneTable(func(_ context.Context, _ event.Event) ([]PostgresWrite, error) { return nil, nil }))
 	if err != nil {
 		t.Fatalf("new postgres sink: %v", err)
 	}
@@ -363,22 +375,15 @@ func TestNewPostgresSinkFromConfig_EagerInit(t *testing.T) {
 	}
 }
 
-func TestNewPostgresSinkFromConfig_InitFailureClosesDB(t *testing.T) {
+func TestNewPostgresSinkFromConfigInitFailureClosesDB(t *testing.T) {
 	withPostgresTestState(t)
 
-	err := RegisterPostgresSchema("pg", schemaOneTable(func(_ context.Context, _ event.Event) ([]PostgresWrite, error) {
-		return nil, nil
-	}))
-	if err != nil {
-		t.Fatalf("register schema: %v", err)
-	}
-
 	db := &fakeDB{execErrOnCall: 1, execErr: errors.New("ddl failed")}
 	openPostgresDB = func(_ string) (postgresDB, error) {
 		return db, nil
 	}
 
-	_, err = NewPostgresSinkFromConfig(config.SinkConfig{
+	_, err := NewPostgresSinkFromConfig(config.SinkConfig{
 		Name:   "pg",
 		Driver: "postgres",
 		Params: map[string]any{
@@ -386,7 +391,7 @@ func TestNewPostgresSinkFromConfig_InitFailureClosesDB(t *testing.T) {
 			"username": "user",
 			"password": "pass",
 		},
-	})
+	}, schemaOneTable(func(_ context.Context, _ event.Event) ([]PostgresWrite, error) { return nil, nil }))
 	if err == nil {
 		t.Fatalf("expected init error")
 	}
@@ -395,7 +400,7 @@ func TestNewPostgresSinkFromConfig_InitFailureClosesDB(t *testing.T) {
 	}
 }
 
-func TestNewPostgresSinkFromConfig_PruneParamAccepted(t *testing.T) {
+func TestNewPostgresSinkFromConfigPruneParamAccepted(t *testing.T) {
 	tests := []struct {
 		name string
 		in   string
@@ -410,13 +415,6 @@ func TestNewPostgresSinkFromConfig_PruneParamAccepted(t *testing.T) {
 		t.Run(tc.name, func(t *testing.T) {
 			withPostgresTestState(t)
 
-			err := RegisterPostgresSchema("pg", schemaOneTable(func(_ context.Context, _ event.Event) ([]PostgresWrite, error) {
-				return nil, nil
-			}))
-			if err != nil {
-				t.Fatalf("register schema: %v", err)
-			}
-
 			openPostgresDB = func(_ string) (postgresDB, error) {
 				return &fakeDB{}, nil
 			}
@@ -430,7 +428,7 @@ func TestNewPostgresSinkFromConfig_PruneParamAccepted(t *testing.T) {
 					"password": "pass",
 					"prune":    tc.in,
 				},
-			})
+			}, schemaOneTable(func(_ context.Context, _ event.Event) ([]PostgresWrite, error) { return nil, nil }))
 			if err != nil {
 				t.Fatalf("new postgres sink: %v", err)
 			}
@@ -446,7 +444,7 @@ func TestNewPostgresSinkFromConfig_PruneParamAccepted(t *testing.T) {
 	}
 }
 
-func TestNewPostgresSinkFromConfig_PruneParamRejected(t *testing.T) {
+func TestNewPostgresSinkFromConfigPruneParamRejected(t *testing.T) {
 	withPostgresTestState(t)
 
 	tests := []struct {
@@ -472,7 +470,7 @@ func TestNewPostgresSinkFromConfig_PruneParamRejected(t *testing.T) {
 					"password": "pass",
 					"prune":    tc.in,
 				},
-			})
+			}, schemaOneTable(func(_ context.Context, _ event.Event) ([]PostgresWrite, error) { return nil, nil }))
 			if err == nil {
 				t.Fatalf("expected error")
 			}
@@ -483,7 +481,7 @@ func TestNewPostgresSinkFromConfig_PruneParamRejected(t *testing.T) {
 	}
 }
 
-func TestPostgresSinkConsume_InvalidEvent(t *testing.T) {
+func TestPostgresSinkConsumeInvalidEvent(t *testing.T) {
 	db := &fakeDB{}
 	called := 0
 	sink := &PostgresSink{
@@ -507,7 +505,7 @@ func TestPostgresSinkConsume_InvalidEvent(t *testing.T) {
 	}
 }
 
-func TestPostgresSinkConsume_UnmappedEventIsNoOp(t *testing.T) {
+func TestPostgresSinkConsumeUnmappedEventIsNoOp(t *testing.T) {
 	db := &fakeDB{}
 	sink := &PostgresSink{
 		name: "pg",
@@ -525,7 +523,7 @@ func TestPostgresSinkConsume_UnmappedEventIsNoOp(t *testing.T) {
 	}
 }
 
-func TestPostgresSinkConsume_OneEventWritesMultipleTablesAtomically(t *testing.T) {
+func TestPostgresSinkConsumeOneEventWritesMultipleTablesAtomically(t *testing.T) {
 	tx := &fakeTx{}
 	db := &fakeDB{tx: tx}
 	sink := &PostgresSink{
@@ -556,7 +554,7 @@ func TestPostgresSinkConsume_OneEventWritesMultipleTablesAtomically(t *testing.T
 	}
 }
 
-func TestPostgresSinkConsume_InsertFailureRollsBack(t *testing.T) {
+func TestPostgresSinkConsumeInsertFailureRollsBack(t *testing.T) {
 	tx := &fakeTx{execErrOnCall: 2, execErr: errors.New("duplicate key")}
 	db := &fakeDB{tx: tx}
 	sink := &PostgresSink{
@@ -585,7 +583,7 @@ func TestPostgresSinkConsume_InsertFailureRollsBack(t *testing.T) {
 	}
 }
 
-func TestPostgresSinkConsume_AutoPruneRunsInSameTransaction(t *testing.T) {
+func TestPostgresSinkConsumeAutoPruneRunsInSameTransaction(t *testing.T) {
 	tx := &fakeTx{}
 	db := &fakeDB{tx: tx}
 	sink := &PostgresSink{
@@ -620,7 +618,7 @@ func TestPostgresSinkConsume_AutoPruneRunsInSameTransaction(t *testing.T) {
 	}
 }
 
-func TestPostgresSinkConsume_AutoPruneFailureRollsBack(t *testing.T) {
+func TestPostgresSinkConsumeAutoPruneFailureRollsBack(t *testing.T) {
 	tx := &fakeTx{execErrOnCall: 3, execErr: errors.New("prune failed")}
 	db := &fakeDB{tx: tx}
 	sink := &PostgresSink{
@@ -650,7 +648,7 @@ func TestPostgresSinkConsume_AutoPruneFailureRollsBack(t *testing.T) {
 	}
 }
 
-func TestPostgresSinkPrune_PerTable(t *testing.T) {
+func TestPostgresSinkPrunePerTable(t *testing.T) {
 	db := &fakeDB{execRows: 7}
 	sink := &PostgresSink{
 		name: "pg",
@@ -693,7 +691,7 @@ func TestPostgresSinkPrune_PerTable(t *testing.T) {
 	}
 }
 
-func TestPostgresSinkPrune_AllTables(t *testing.T) {
+func TestPostgresSinkPruneAllTables(t *testing.T) {
 	db := &fakeDB{execRows: 3}
 	sink := &PostgresSink{
 		name: "pg",
@@ -724,7 +722,7 @@ func TestPostgresSinkPrune_AllTables(t *testing.T) {
 	}
 }
 
-func TestPostgresSinkPrune_Errors(t *testing.T) {
+func TestPostgresSinkPruneErrors(t *testing.T) {
 	db := &fakeDB{}
 	sink := &PostgresSink{
 		name: "pg",

sinks/registry.go

@@ -2,6 +2,7 @@ package sinks
 
 import (
 	"fmt"
+	"strings"
 
 	"gitea.maximumdirect.net/ejr/feedkit/config"
 )
@@ -21,13 +22,40 @@ func NewRegistry() *Registry {
 }
 
 func (r *Registry) Register(driver string, f Factory) {
+	if r == nil {
+		panic("sinks.Registry.Register: registry cannot be nil")
+	}
+	driver = strings.TrimSpace(driver)
+	if driver == "" {
+		panic("sinks.Registry.Register: driver cannot be empty")
+	}
+	if f == nil {
+		panic(fmt.Sprintf("sinks.Registry.Register: factory cannot be nil (driver=%q)", driver))
+	}
+	if r.byDriver == nil {
+		r.byDriver = map[string]Factory{}
+	}
+	if _, exists := r.byDriver[driver]; exists {
+		panic(fmt.Sprintf("sinks.Registry.Register: driver %q already registered", driver))
+	}
 	r.byDriver[driver] = f
 }
 
 func (r *Registry) Build(cfg config.SinkConfig) (Sink, error) {
-	f, ok := r.byDriver[cfg.Driver]
+	if r == nil {
+		return nil, fmt.Errorf("sinks registry is nil")
+	}
+	driver := strings.TrimSpace(cfg.Driver)
+	f, ok := r.byDriver[driver]
 	if !ok {
-		return nil, fmt.Errorf("unknown sink driver: %q", cfg.Driver)
+		return nil, fmt.Errorf("unknown sink driver: %q", driver)
 	}
-	return f(cfg)
+	sink, err := f(cfg)
+	if err != nil {
+		return nil, fmt.Errorf("build sink %q: %w", driver, err)
+	}
+	if sink == nil {
+		return nil, fmt.Errorf("build sink %q: factory returned nil sink", driver)
+	}
+	return sink, nil
 }

sinks/registry_test.go

@@ -0,0 +1,126 @@
+package sinks
+
+import (
+	"context"
+	"errors"
+	"strings"
+	"testing"
+
+	"gitea.maximumdirect.net/ejr/feedkit/config"
+	"gitea.maximumdirect.net/ejr/feedkit/event"
+)
+
+type testSink struct{ name string }
+
+func (s testSink) Name() string { return s.name }
+
+func (s testSink) Consume(context.Context, event.Event) error { return nil }
+
+func TestRegistryRegisterPanicsOnNilRegistry(t *testing.T) {
+	var r *Registry
+	defer func() {
+		if recover() == nil {
+			t.Fatalf("Register() expected panic on nil registry")
+		}
+	}()
+	r.Register("stdout", func(config.SinkConfig) (Sink, error) { return testSink{name: "stdout"}, nil })
+}
+
+func TestRegistryRegisterPanicsOnEmptyDriver(t *testing.T) {
+	r := NewRegistry()
+	defer func() {
+		if recover() == nil {
+			t.Fatalf("Register() expected panic on empty driver")
+		}
+	}()
+	r.Register("   ", func(config.SinkConfig) (Sink, error) { return testSink{name: "x"}, nil })
+}
+
+func TestRegistryRegisterPanicsOnNilFactory(t *testing.T) {
+	r := NewRegistry()
+	defer func() {
+		if recover() == nil {
+			t.Fatalf("Register() expected panic on nil factory")
+		}
+	}()
+	r.Register("stdout", nil)
+}
+
+func TestRegistryRegisterPanicsOnDuplicateDriver(t *testing.T) {
+	r := NewRegistry()
+	r.Register("stdout", func(config.SinkConfig) (Sink, error) { return testSink{name: "a"}, nil })
+
+	defer func() {
+		if recover() == nil {
+			t.Fatalf("Register() expected panic on duplicate driver")
+		}
+	}()
+	r.Register("stdout", func(config.SinkConfig) (Sink, error) { return testSink{name: "b"}, nil })
+}
+
+func TestRegistryBuildNilRegistryFails(t *testing.T) {
+	var r *Registry
+	_, err := r.Build(config.SinkConfig{Driver: "stdout"})
+	if err == nil {
+		t.Fatalf("Build() expected error for nil registry")
+	}
+	if !strings.Contains(err.Error(), "registry is nil") {
+		t.Fatalf("Build() error = %q, want registry is nil", err)
+	}
+}
+
+func TestRegistryBuildTrimsDriver(t *testing.T) {
+	r := NewRegistry()
+	r.Register("stdout", func(config.SinkConfig) (Sink, error) { return testSink{name: "stdout"}, nil })
+
+	sink, err := r.Build(config.SinkConfig{Name: "sink1", Driver: " stdout "})
+	if err != nil {
+		t.Fatalf("Build() error = %v", err)
+	}
+	if sink.Name() != "stdout" {
+		t.Fatalf("Build() sink name = %q, want stdout", sink.Name())
+	}
+}
+
+func TestRegistryBuildWrapsFactoryError(t *testing.T) {
+	r := NewRegistry()
+	r.Register("broken", func(config.SinkConfig) (Sink, error) { return nil, errors.New("boom") })
+
+	_, err := r.Build(config.SinkConfig{Driver: "broken"})
+	if err == nil {
+		t.Fatalf("Build() expected error")
+	}
+	if !strings.Contains(err.Error(), `build sink "broken": boom`) {
+		t.Fatalf("Build() error = %q", err)
+	}
+}
+
+func TestRegistryBuildRejectsNilSink(t *testing.T) {
+	r := NewRegistry()
+	r.Register("nil_sink", func(config.SinkConfig) (Sink, error) { return nil, nil })
+
+	_, err := r.Build(config.SinkConfig{Driver: "nil_sink"})
+	if err == nil {
+		t.Fatalf("Build() expected nil sink error")
+	}
+	if !strings.Contains(err.Error(), "factory returned nil sink") {
+		t.Fatalf("Build() error = %q", err)
+	}
+}
+
+func TestRegisterBuiltinsExposesExpectedDrivers(t *testing.T) {
+	r := NewRegistry()
+	RegisterBuiltins(r)
+
+	if len(r.byDriver) != 2 {
+		t.Fatalf("len(byDriver) = %d, want 2", len(r.byDriver))
+	}
+	for _, driver := range []string{"stdout", "nats"} {
+		if _, ok := r.byDriver[driver]; !ok {
+			t.Fatalf("builtins missing driver %q", driver)
+		}
+	}
+	if _, ok := r.byDriver["postgres"]; ok {
+		t.Fatalf("builtins unexpectedly registered postgres driver")
+	}
+}

sources/doc.go

@@ -1,10 +1,12 @@
-// Package sources defines feedkit's input-source abstraction.
+// Package sources defines feedkit's input-source abstractions and source
+// registry.
 //
-// A source ingests upstream input and emits one or more event.Event values.
+// External API surface:
-//
+//   - Input: common source identity surface
-// feedkit supports two source modes:
+//   - PollSource: polling source interface
-//   - PollSource: scheduler invokes Poll on a cadence.
+//   - StreamSource: streaming source interface
-//   - StreamSource: source runs continuously and pushes events as input arrives.
+//   - Registry / NewRegistry: source driver registry and builders
+//   - HTTPSource / NewHTTPSource: reusable HTTP polling helper
 //
 // Source drivers are domain-specific and registered into Registry by driver name.
 // Registry can then build configured sources from config.SourceConfig.
@@ -12,11 +14,20 @@
 // A single source may emit 0..N events per poll or stream iteration, and those
 // events may span multiple event kinds.
 //
+// Optional helpers from helpers.go:
+//   - DefaultEventID: default event ID policy for source implementations
+//   - SingleEvent: construct and validate a one-element event slice
+//   - ValidateExpectedKinds: compare configured expected kinds against source
+//     advertised kinds when metadata is available
+//
 // HTTP-backed polling sources can share NewHTTPSource for generic HTTP config
 // parsing and conditional GET behavior. The helper understands:
 //   - params.url
 //   - params.user_agent
 //   - params.conditional (optional, default true)
+//   - params.http_timeout (optional, default transport.DefaultHTTPTimeout)
+//   - params.http_response_body_limit_bytes (optional, default
+//     transport.DefaultHTTPResponseBodyLimitBytes)
 //
 // When validators are available, NewHTTPSource prefers ETag/If-None-Match and
 // falls back to Last-Modified/If-Modified-Since. A 304 Not Modified response is

sources/helpers.go

@@ -127,11 +127,6 @@ func advertisedSourceKinds(in Input) map[event.Kind]bool {
 		return kinds
 	}
 
-	if ks, ok := in.(KindSource); ok {
-		kinds[ks.Kind()] = true
-		return kinds
-	}
-
 	return nil
 }
 

sources/helpers_test.go

@@ -15,13 +15,6 @@ type testInput struct {
 
 func (s testInput) Name() string { return s.name }
 
-type testKindSource struct {
-	testInput
-	kind event.Kind
-}
-
-func (s testKindSource) Kind() event.Kind { return s.kind }
-
 type testKindsSource struct {
 	testInput
 	kinds []event.Kind
@@ -29,18 +22,6 @@ type testKindsSource struct {
 
 func (s testKindsSource) Kinds() []event.Kind { return s.kinds }
 
-func TestValidateExpectedKindsLegacyKindFallback(t *testing.T) {
-	cfg := config.SourceConfig{Kind: "observation"}
-	in := testKindSource{
-		testInput: testInput{name: "test"},
-		kind:      event.Kind("observation"),
-	}
-
-	if err := ValidateExpectedKinds(cfg, in); err != nil {
-		t.Fatalf("ValidateExpectedKinds() unexpected error: %v", err)
-	}
-}
-
 func TestValidateExpectedKindsSubsetAllowed(t *testing.T) {
 	cfg := config.SourceConfig{Kinds: []string{"observation"}}
 	in := testKindsSource{

sources/http.go

@@ -5,7 +5,6 @@ import (
 	"encoding/json"
 	"fmt"
 	"net/http"
-	"strconv"
 	"strings"
 	"sync"
 
@@ -26,6 +25,7 @@ type HTTPSource struct {
 	UserAgent              string
 	Accept                 string
 	Conditional            bool
+	ResponseBodyLimitBytes int64
 	Client                 *http.Client
 
 	mu         sync.Mutex
@@ -60,9 +60,31 @@ func NewHTTPSource(driver string, cfg config.SourceConfig, accept string) (*HTTP
 		return nil, fmt.Errorf("%s %q: params.user_agent is required", driver, cfg.Name)
 	}
 
-	conditional, err := parseConditionalParam(cfg)
+	conditional := true
-	if err != nil {
+	if _, exists := cfg.Params["conditional"]; exists {
-		return nil, err
+		var ok bool
+		conditional, ok = cfg.ParamBool("conditional")
+		if !ok {
+			return nil, fmt.Errorf("source %q: params.conditional must be a boolean", cfg.Name)
+		}
+	}
+
+	timeout := transport.DefaultHTTPTimeout
+	if _, exists := cfg.Params["http_timeout"]; exists {
+		var ok bool
+		timeout, ok = cfg.ParamDuration("http_timeout")
+		if !ok || timeout <= 0 {
+			return nil, fmt.Errorf("source %q: params.http_timeout must be a positive duration", cfg.Name)
+		}
+	}
+
+	bodyLimit := transport.DefaultHTTPResponseBodyLimitBytes
+	if _, exists := cfg.Params["http_response_body_limit_bytes"]; exists {
+		rawLimit, ok := cfg.ParamInt("http_response_body_limit_bytes")
+		if !ok || rawLimit <= 0 {
+			return nil, fmt.Errorf("source %q: params.http_response_body_limit_bytes must be a positive integer", cfg.Name)
+		}
+		bodyLimit = int64(rawLimit)
 	}
 
 	return &HTTPSource{
@@ -72,7 +94,8 @@ func NewHTTPSource(driver string, cfg config.SourceConfig, accept string) (*HTTP
 		UserAgent:              userAgent,
 		Accept:                 accept,
 		Conditional:            conditional,
-		Client:      transport.NewHTTPClient(transport.DefaultHTTPTimeout),
+		ResponseBodyLimitBytes: bodyLimit,
+		Client:                 transport.NewHTTPClient(timeout),
 	}, nil
 }
 
@@ -89,7 +112,12 @@ func (s *HTTPSource) FetchBytesIfChanged(ctx context.Context) ([]byte, bool, err
 	validators := s.validators
 	s.mu.Unlock()
 
-	body, changed, next, err := transport.FetchBodyIfChanged(
+	bodyLimit := s.ResponseBodyLimitBytes
+	if bodyLimit <= 0 {
+		bodyLimit = transport.DefaultHTTPResponseBodyLimitBytes
+	}
+
+	body, changed, next, err := transport.FetchBodyIfChangedWithLimit(
 		ctx,
 		client,
 		s.URL,
@@ -97,6 +125,7 @@ func (s *HTTPSource) FetchBytesIfChanged(ctx context.Context) ([]byte, bool, err
 		s.Accept,
 		s.Conditional,
 		validators,
+		bodyLimit,
 	)
 	if err != nil {
 		return nil, false, fmt.Errorf("%s %q: %w", s.Driver, s.Name, err)
@@ -121,27 +150,3 @@ func (s *HTTPSource) FetchJSONIfChanged(ctx context.Context) (json.RawMessage, b
 	}
 	return json.RawMessage(body), true, nil
 }
-
-func parseConditionalParam(cfg config.SourceConfig) (bool, error) {
-	raw, ok := cfg.Params["conditional"]
-	if !ok || raw == nil {
-		return true, nil
-	}
-
-	switch v := raw.(type) {
-	case bool:
-		return v, nil
-	case string:
-		s := strings.TrimSpace(v)
-		if s == "" {
-			return false, fmt.Errorf("source %q: params.conditional must be a boolean", cfg.Name)
-		}
-		parsed, err := strconv.ParseBool(s)
-		if err != nil {
-			return false, fmt.Errorf("source %q: params.conditional must be a boolean", cfg.Name)
-		}
-		return parsed, nil
-	default:
-		return false, fmt.Errorf("source %q: params.conditional must be a boolean", cfg.Name)
-	}
-}

sources/http_test.go

@@ -4,9 +4,12 @@ import (
 	"context"
 	"net/http"
 	"net/http/httptest"
+	"strings"
 	"testing"
+	"time"
 
 	"gitea.maximumdirect.net/ejr/feedkit/config"
+	"gitea.maximumdirect.net/ejr/feedkit/transport"
 )
 
 func TestNewHTTPSourceConditionalDefaultsTrue(t *testing.T) {
@@ -39,6 +42,102 @@ func TestNewHTTPSourceRejectsInvalidConditional(t *testing.T) {
 	if err == nil {
 		t.Fatalf("NewHTTPSource() error = nil, want error")
 	}
+	if !strings.Contains(err.Error(), "params.conditional must be a boolean") {
+		t.Fatalf("NewHTTPSource() error = %q, want params.conditional must be a boolean", err)
+	}
+}
+
+func TestNewHTTPSourceConditionalCanBeExplicitlyFalse(t *testing.T) {
+	src, err := NewHTTPSource("test_driver", config.SourceConfig{
+		Name:   "test-source",
+		Driver: "test_driver",
+		Params: map[string]any{
+			"url":         "https://example.invalid",
+			"user_agent":  "test-agent",
+			"conditional": false,
+		},
+	}, "application/json")
+	if err != nil {
+		t.Fatalf("NewHTTPSource() error = %v", err)
+	}
+	if src.Conditional {
+		t.Fatalf("Conditional = true, want false")
+	}
+}
+
+func TestNewHTTPSourceHTTPTimeoutOverride(t *testing.T) {
+	src, err := NewHTTPSource("test_driver", config.SourceConfig{
+		Name:   "test-source",
+		Driver: "test_driver",
+		Params: map[string]any{
+			"url":          "https://example.invalid",
+			"user_agent":   "test-agent",
+			"http_timeout": "250ms",
+		},
+	}, "application/json")
+	if err != nil {
+		t.Fatalf("NewHTTPSource() error = %v", err)
+	}
+	if src.Client == nil {
+		t.Fatalf("Client = nil")
+	}
+	if src.Client.Timeout != 250*time.Millisecond {
+		t.Fatalf("Client.Timeout = %s, want 250ms", src.Client.Timeout)
+	}
+}
+
+func TestNewHTTPSourceBodyLimitOverride(t *testing.T) {
+	src, err := NewHTTPSource("test_driver", config.SourceConfig{
+		Name:   "test-source",
+		Driver: "test_driver",
+		Params: map[string]any{
+			"url":                            "https://example.invalid",
+			"user_agent":                     "test-agent",
+			"http_response_body_limit_bytes": 12345,
+		},
+	}, "application/json")
+	if err != nil {
+		t.Fatalf("NewHTTPSource() error = %v", err)
+	}
+	if src.ResponseBodyLimitBytes != 12345 {
+		t.Fatalf("ResponseBodyLimitBytes = %d, want 12345", src.ResponseBodyLimitBytes)
+	}
+}
+
+func TestNewHTTPSourceRejectsInvalidHTTPTimeout(t *testing.T) {
+	_, err := NewHTTPSource("test_driver", config.SourceConfig{
+		Name:   "test-source",
+		Driver: "test_driver",
+		Params: map[string]any{
+			"url":          "https://example.invalid",
+			"user_agent":   "test-agent",
+			"http_timeout": "soon",
+		},
+	}, "application/json")
+	if err == nil {
+		t.Fatalf("NewHTTPSource() error = nil, want error")
+	}
+	if !strings.Contains(err.Error(), "params.http_timeout must be a positive duration") {
+		t.Fatalf("NewHTTPSource() error = %q", err)
+	}
+}
+
+func TestNewHTTPSourceRejectsInvalidBodyLimit(t *testing.T) {
+	_, err := NewHTTPSource("test_driver", config.SourceConfig{
+		Name:   "test-source",
+		Driver: "test_driver",
+		Params: map[string]any{
+			"url":                            "https://example.invalid",
+			"user_agent":                     "test-agent",
+			"http_response_body_limit_bytes": "abc",
+		},
+	}, "application/json")
+	if err == nil {
+		t.Fatalf("NewHTTPSource() error = nil, want error")
+	}
+	if !strings.Contains(err.Error(), "params.http_response_body_limit_bytes must be a positive integer") {
+		t.Fatalf("NewHTTPSource() error = %q", err)
+	}
 }
 
 func TestHTTPSourceFetchJSONIfChanged(t *testing.T) {
@@ -94,3 +193,68 @@ func TestHTTPSourceFetchJSONIfChanged(t *testing.T) {
 		t.Fatalf("second FetchJSONIfChanged() body = %q, want nil", string(raw))
 	}
 }
+
+func TestHTTPSourceFetchJSONIfChangedHonorsBodyLimitOverride(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		_, _ = w.Write([]byte(`{"ok":true}`))
+	}))
+	defer srv.Close()
+
+	src, err := NewHTTPSource("test_driver", config.SourceConfig{
+		Name:   "test-source",
+		Driver: "test_driver",
+		Params: map[string]any{
+			"url":                            srv.URL,
+			"user_agent":                     "test-agent",
+			"http_response_body_limit_bytes": 4,
+		},
+	}, "application/json")
+	if err != nil {
+		t.Fatalf("NewHTTPSource() error = %v", err)
+	}
+
+	_, _, err = src.FetchJSONIfChanged(context.Background())
+	if err == nil {
+		t.Fatalf("FetchJSONIfChanged() error = nil, want limit error")
+	}
+	if !strings.Contains(err.Error(), "response body too large") {
+		t.Fatalf("FetchJSONIfChanged() error = %q", err)
+	}
+}
+
+func TestNewHTTPSourceUsesDefaultBodyLimitWhenUnset(t *testing.T) {
+	src, err := NewHTTPSource("test_driver", config.SourceConfig{
+		Name:   "test-source",
+		Driver: "test_driver",
+		Params: map[string]any{
+			"url":        "https://example.invalid",
+			"user_agent": "test-agent",
+		},
+	}, "application/json")
+	if err != nil {
+		t.Fatalf("NewHTTPSource() error = %v", err)
+	}
+	if src.ResponseBodyLimitBytes != transport.DefaultHTTPResponseBodyLimitBytes {
+		t.Fatalf("ResponseBodyLimitBytes = %d, want %d", src.ResponseBodyLimitBytes, transport.DefaultHTTPResponseBodyLimitBytes)
+	}
+}
+
+func TestNewHTTPSourceUsesDefaultTimeoutWhenUnset(t *testing.T) {
+	src, err := NewHTTPSource("test_driver", config.SourceConfig{
+		Name:   "test-source",
+		Driver: "test_driver",
+		Params: map[string]any{
+			"url":        "https://example.invalid",
+			"user_agent": "test-agent",
+		},
+	}, "application/json")
+	if err != nil {
+		t.Fatalf("NewHTTPSource() error = %v", err)
+	}
+	if src.Client == nil {
+		t.Fatalf("Client = nil")
+	}
+	if src.Client.Timeout != transport.DefaultHTTPTimeout {
+		t.Fatalf("Client.Timeout = %s, want %s", src.Client.Timeout, transport.DefaultHTTPTimeout)
+	}
+}

sources/registry.go

@@ -15,9 +15,6 @@ import (
 type PollFactory func(cfg config.SourceConfig) (PollSource, error)
 type StreamFactory func(cfg config.SourceConfig) (StreamSource, error)
 
-// Factory is the legacy alias for poll source factories.
-type Factory = PollFactory
-
 type Registry struct {
 	byPollDriver   map[string]PollFactory
 	byStreamDriver map[string]StreamFactory
@@ -30,13 +27,6 @@ func NewRegistry() *Registry {
 	}
 }
 
-// Register associates a driver name (e.g. "openmeteo_observation") with a factory.
-//
-// The driver string is the "lookup key" used by config.sources[].driver.
-func (r *Registry) Register(driver string, f PollFactory) {
-	r.RegisterPoll(driver, f)
-}
-
 // RegisterPoll associates a driver name with a polling-source factory.
 func (r *Registry) RegisterPoll(driver string, f PollFactory) {
 	driver = strings.TrimSpace(driver)
@@ -75,11 +65,6 @@ func (r *Registry) RegisterStream(driver string, f StreamFactory) {
 	r.byStreamDriver[driver] = f
 }
 
-// Build constructs a polling source from a SourceConfig by looking up cfg.Driver.
-func (r *Registry) Build(cfg config.SourceConfig) (PollSource, error) {
-	return r.BuildPoll(cfg)
-}
-
 // BuildPoll constructs a polling source from a SourceConfig by looking up cfg.Driver.
 func (r *Registry) BuildPoll(cfg config.SourceConfig) (PollSource, error) {
 	driver := strings.TrimSpace(cfg.Driver)

sources/source.go

@@ -31,9 +31,6 @@ type PollSource interface {
 	Poll(ctx context.Context) ([]event.Event, error)
 }
 
-// Source is a compatibility alias for the legacy polling-source name.
-type Source = PollSource
-
 // StreamSource is an event-driven source (NATS/RabbitMQ/MQTT/etc).
 //
 // Run should block, producing events into `out` until ctx is cancelled or a fatal error occurs.
@@ -43,12 +40,6 @@ type StreamSource interface {
 	Run(ctx context.Context, out chan<- event.Event) error
 }
 
-// KindSource is an optional interface for sources that advertise one "primary" kind.
-// This is legacy-friendly but no longer required.
-type KindSource interface {
-	Kind() event.Kind
-}
-
 // KindsSource is an optional interface for sources that advertise multiple kinds.
 type KindsSource interface {
 	Kinds() []event.Kind

transport/http.go

@@ -10,10 +10,10 @@ import (
 	"time"
 )
 
-// maxResponseBodyBytes is a hard safety limit on HTTP response bodies.
+// DefaultHTTPResponseBodyLimitBytes is a hard safety limit on HTTP response bodies.
 // API responses should be small, so this protects us from accidental
 // or malicious large responses.
-const maxResponseBodyBytes = 2 << 21 // 4 MiB
+const DefaultHTTPResponseBodyLimitBytes int64 = 2 << 21 // 4 MiB
 
 // DefaultHTTPTimeout is the standard timeout used by HTTP sources.
 // Individual drivers may override this if they have a specific need.
@@ -29,6 +29,10 @@ func NewHTTPClient(timeout time.Duration) *http.Client {
 }
 
 func FetchBody(ctx context.Context, client *http.Client, url, userAgent, accept string) ([]byte, error) {
+	return FetchBodyWithLimit(ctx, client, url, userAgent, accept, DefaultHTTPResponseBodyLimitBytes)
+}
+
+func FetchBodyWithLimit(ctx context.Context, client *http.Client, url, userAgent, accept string, bodyLimitBytes int64) ([]byte, error) {
 	res, err := doRequest(ctx, client, http.MethodGet, url, userAgent, accept, "", "")
 	if err != nil {
 		return nil, err
@@ -39,7 +43,7 @@ func FetchBody(ctx context.Context, client *http.Client, url, userAgent, accept
 		return nil, fmt.Errorf("HTTP %s", res.Status)
 	}
 
-	return readValidatedBody(res.Body)
+	return readValidatedBody(res.Body, bodyLimitBytes)
 }
 
 // HTTPValidators are cache validators learned from prior successful GET responses.
@@ -68,6 +72,17 @@ func FetchBodyIfChanged(
 	url, userAgent, accept string,
 	conditional bool,
 	validators HTTPValidators,
+) ([]byte, bool, HTTPValidators, error) {
+	return FetchBodyIfChangedWithLimit(ctx, client, url, userAgent, accept, conditional, validators, DefaultHTTPResponseBodyLimitBytes)
+}
+
+func FetchBodyIfChangedWithLimit(
+	ctx context.Context,
+	client *http.Client,
+	url, userAgent, accept string,
+	conditional bool,
+	validators HTTPValidators,
+	bodyLimitBytes int64,
 ) ([]byte, bool, HTTPValidators, error) {
 	headerName, headerValue := conditionalHeader(conditional, validators)
 
@@ -89,7 +104,7 @@ func FetchBodyIfChanged(
 		}
 	}
 
-	b, err := readValidatedBody(res.Body)
+	b, err := readValidatedBody(res.Body, bodyLimitBytes)
 	if err != nil {
 		return nil, false, validators, err
 	}
@@ -150,9 +165,13 @@ func refreshValidators(current HTTPValidators, header http.Header) HTTPValidator
 	return current
 }
 
-func readValidatedBody(r io.Reader) ([]byte, error) {
+func readValidatedBody(r io.Reader, bodyLimitBytes int64) ([]byte, error) {
-	// Read at most maxResponseBodyBytes + 1 so we can detect overflow.
+	if bodyLimitBytes <= 0 {
-	limited := io.LimitReader(r, maxResponseBodyBytes+1)
+		bodyLimitBytes = DefaultHTTPResponseBodyLimitBytes
+	}
+
+	// Read at most bodyLimitBytes + 1 so we can detect overflow.
+	limited := io.LimitReader(r, bodyLimitBytes+1)
 
 	b, err := io.ReadAll(limited)
 	if err != nil {
@@ -163,8 +182,8 @@ func readValidatedBody(r io.Reader) ([]byte, error) {
 		return nil, fmt.Errorf("empty response body")
 	}
 
-	if len(b) > maxResponseBodyBytes {
+	if int64(len(b)) > bodyLimitBytes {
-		return nil, fmt.Errorf("response body too large (>%d bytes)", maxResponseBodyBytes)
+		return nil, fmt.Errorf("response body too large (>%d bytes)", bodyLimitBytes)
 	}
 
 	return b, nil