Added a dedupe processor, and moved processor packages under processors/*
This commit is contained in:
17
processors/normalize/doc.go
Normal file
17
processors/normalize/doc.go
Normal file
@@ -0,0 +1,17 @@
|
||||
// Package normalize provides a concrete normalization processor for feedkit pipelines.
|
||||
//
|
||||
// Motivation:
|
||||
// Many daemons have sources that:
|
||||
// 1. fetch raw upstream data (often JSON), and
|
||||
// 2. transform it into a domain's normalized payload format.
|
||||
//
|
||||
// Doing both steps inside Source.Poll works, but tends to make sources large and
|
||||
// encourages duplication (unit conversions, common mapping helpers, etc.).
|
||||
//
|
||||
// This package lets a source emit a "raw" event (e.g., Schema="raw.openweather.current.v1",
|
||||
// Payload=json.RawMessage), and then a normalize.Processor can convert it into a
|
||||
// normalized event (e.g., Schema="weather.observation.v1", Payload=WeatherObservation{}).
|
||||
//
|
||||
// Key property: normalization is optional.
|
||||
// If no Normalizer matches an event, Processor passes it through unchanged by default.
|
||||
package normalize
|
||||
Reference in New Issue
Block a user