Added a dedupe processor, and moved processor packages under processors/*

This commit is contained in:
2026-03-16 18:17:53 -05:00
parent 4572c53580
commit 215afe1acf
11 changed files with 297 additions and 13 deletions

View File

@@ -0,0 +1,17 @@
// Package normalize provides a concrete normalization processor for feedkit pipelines.
//
// Motivation:
// Many daemons have sources that:
// 1. fetch raw upstream data (often JSON), and
// 2. transform it into a domain's normalized payload format.
//
// Doing both steps inside Source.Poll works, but tends to make sources large and
// encourages duplication (unit conversions, common mapping helpers, etc.).
//
// This package lets a source emit a "raw" event (e.g., Schema="raw.openweather.current.v1",
// Payload=json.RawMessage), and then a normalize.Processor can convert it into a
// normalized event (e.g., Schema="weather.observation.v1", Payload=WeatherObservation{}).
//
// Key property: normalization is optional.
// If no Normalizer matches an event, Processor passes it through unchanged by default.
package normalize