Eric Rakestraw
eb9a7cb349
Remove global Postgres schema registration in favor of explicit schema-aware sink factory wiring, and update weatherfeeder to register the Postgres sink explicitly. Add optional per-source HTTP timeout and response body limit overrides while keeping feedkit defaults. Remove remaining legacy source/config compatibility surfaces, including singular kind support and old source registry/type aliases, and migrate weatherfeeder sources to plural `Kinds()` metadata. Clean up related docs, tests, and sample config to match the new Postgres, HTTP, and NATS configuration model.
57 lines
1.9 KiB
Go
57 lines
1.9 KiB
Go
// Package feedkit provides a high-level map of the feedkit package set.
|
|
//
|
|
// Most real applications do not import the root package directly. Instead, they
|
|
// compose the subpackages that handle configuration, collection, processing,
|
|
// routing, and sinks.
|
|
//
|
|
// The usual flow through feedkit is:
|
|
//
|
|
// Collect -> Process -> Route -> Emit
|
|
//
|
|
// That flow maps to packages like this:
|
|
//
|
|
// - config
|
|
// Loads and validates daemon config. This package owns domain-agnostic
|
|
// config shape and consistency checks.
|
|
//
|
|
// - event
|
|
// Defines the event.Event envelope shared across sources, processors,
|
|
// dispatch, and sinks.
|
|
//
|
|
// - sources
|
|
// Defines polling and streaming source interfaces, the source registry, and
|
|
// reusable source helpers.
|
|
//
|
|
// - scheduler
|
|
// Runs configured sources on a cadence or as long-lived stream workers.
|
|
//
|
|
// - processors
|
|
// Defines the generic processor interface and registry used to build
|
|
// ordered processor chains.
|
|
//
|
|
// - processors/dedupe
|
|
// Built-in in-memory dedupe processor keyed by Event.ID.
|
|
//
|
|
// - processors/normalize
|
|
// Built-in normalization processor plus helper APIs for raw-to-canonical
|
|
// event mapping.
|
|
//
|
|
// - pipeline
|
|
// Applies an ordered processor chain between collection and dispatch.
|
|
//
|
|
// - dispatch
|
|
// Compiles routes and fans events out to sinks with per-sink isolation.
|
|
//
|
|
// - sinks
|
|
// Defines sink interfaces, the sink registry, schema-free built-in sinks,
|
|
// and explicit Postgres factory helpers.
|
|
//
|
|
// feedkit is intentionally domain-agnostic. Domain schemas, domain event kinds,
|
|
// upstream-specific parsing, and daemon lifecycle remain the responsibility of
|
|
// each concrete application.
|
|
//
|
|
// For repository-level overview and usage narrative, see README.md. For
|
|
// code-level details, each subpackage doc.go is the source of truth for that
|
|
// package's public API surface and optional helpers.
|
|
package feedkit
|