Eric Rakestraw
eb9a7cb349
Remove global Postgres schema registration in favor of explicit schema-aware sink factory wiring, and update weatherfeeder to register the Postgres sink explicitly. Add optional per-source HTTP timeout and response body limit overrides while keeping feedkit defaults. Remove remaining legacy source/config compatibility surfaces, including singular kind support and old source registry/type aliases, and migrate weatherfeeder sources to plural `Kinds()` metadata. Clean up related docs, tests, and sample config to match the new Postgres, HTTP, and NATS configuration model.
36 lines
1.6 KiB
Go
36 lines
1.6 KiB
Go
// Package sources defines feedkit's input-source abstractions and source
|
|
// registry.
|
|
//
|
|
// External API surface:
|
|
// - Input: common source identity surface
|
|
// - PollSource: polling source interface
|
|
// - StreamSource: streaming source interface
|
|
// - Registry / NewRegistry: source driver registry and builders
|
|
// - HTTPSource / NewHTTPSource: reusable HTTP polling helper
|
|
//
|
|
// Source drivers are domain-specific and registered into Registry by driver name.
|
|
// Registry can then build configured sources from config.SourceConfig.
|
|
//
|
|
// A single source may emit 0..N events per poll or stream iteration, and those
|
|
// events may span multiple event kinds.
|
|
//
|
|
// Optional helpers from helpers.go:
|
|
// - DefaultEventID: default event ID policy for source implementations
|
|
// - SingleEvent: construct and validate a one-element event slice
|
|
// - ValidateExpectedKinds: compare configured expected kinds against source
|
|
// advertised kinds when metadata is available
|
|
//
|
|
// HTTP-backed polling sources can share NewHTTPSource for generic HTTP config
|
|
// parsing and conditional GET behavior. The helper understands:
|
|
// - params.url
|
|
// - params.user_agent
|
|
// - params.conditional (optional, default true)
|
|
// - params.http_timeout (optional, default transport.DefaultHTTPTimeout)
|
|
// - params.http_response_body_limit_bytes (optional, default
|
|
// transport.DefaultHTTPResponseBodyLimitBytes)
|
|
//
|
|
// When validators are available, NewHTTPSource prefers ETag/If-None-Match and
|
|
// falls back to Last-Modified/If-Modified-Since. A 304 Not Modified response is
|
|
// treated as a successful unchanged poll.
|
|
package sources
|