// Package normalizers defines weatherfeeder’s **prescriptive** conventions for // writing feedkit normalizers and provides the recommended project layout. // // Summary // ------- // weatherfeeder ingests multiple upstream providers whose payloads differ. // Sources should focus on polling/fetching. Normalizers should focus on // transforming provider-specific raw payloads into canonical internal models. // // This package is domain code (weatherfeeder). feedkit’s normalize package is // infrastructure (registry + processor). // // Directory layout (required) // --------------------------- // Normalizers are organized by provider: // // internal/normalizers// // // Example: // // internal/normalizers/nws/observation.go // internal/normalizers/nws/common.go // internal/normalizers/openweather/observation.go // internal/normalizers/openmeteo/observation.go // internal/normalizers/common/units.go // // Rules: // // 1. One normalizer per file. // Each file contains exactly one Normalizer implementation (one type). // // 2. Provider-level shared helpers live in: // internal/normalizers//common.go // // 3. Cross-provider helpers live in: // internal/normalizers/common/ // // 4. Matching is standardized on Event.Schema. // (Do not match on event.Source or event.Kind in weatherfeeder normalizers.) // // Schema conventions (required) // ----------------------------- // Sources emit RAW events with provider-specific schemas. // Normalizers convert RAW -> CANONICAL schemas. // // Raw schemas: // // raw...vN // // Canonical schemas: // // weather..vN // // weatherfeeder centralizes schema strings in internal/standards/schemas.go. // Always use those constants (do not inline schema strings). // // Example mappings: // // standards.SchemaRawOpenWeatherCurrentV1 -> standards.SchemaWeatherObservationV1 // standards.SchemaRawOpenMeteoCurrentV1 -> standards.SchemaWeatherObservationV1 // standards.SchemaRawNWSObservationV1 -> standards.SchemaWeatherObservationV1 // // Normalizer structure (required template) // ---------------------------------------- // Each normalizer file must follow this structure (with helpful comments): // // type OpenWeatherObservationNormalizer struct{} // // func (OpenWeatherObservationNormalizer) Match(e event.Event) bool { ... } // // func (OpenWeatherObservationNormalizer) Normalize(ctx context.Context, in event.Event) (*event.Event, error) { // // 1) Decode raw payload (recommended: json.RawMessage) // // 2) Parse into provider structs // // 3) Map provider -> canonical internal/model types // // 4) Build output event (copy input, modify intentionally) // // 5) Set EffectiveAt if applicable // // 6) Validate out.Validate() // } // // Required doc comment content // ---------------------------- // Every normalizer type must have a doc comment that states: // // - what it converts (e.g., “OpenWeather current -> WeatherObservation”) // - which raw schema it matches (constant name + value) // - which canonical schema it produces (constant name + value) // - any special caveats (units, day/night inference, missing fields, etc.) // // Event field handling (strong defaults) // -------------------------------------- // Normalizers should treat the incoming event envelope as stable identity and // should only change fields intentionally. // // Default behavior: // // - Keep: ID, Kind, Source, EmittedAt // - Set: Schema to the canonical schema // - Set: Payload to the canonical payload (internal/model/*) // - Optional: EffectiveAt (often derived from observation timestamp) // - Avoid changing Kind unless you have a clear “raw kind vs canonical kind” design. // // Always validate the output event: // // if err := out.Validate(); err != nil { ... } // // Payload representation for RAW events // ------------------------------------- // weatherfeeder recommends RAW payloads be stored as json.RawMessage for JSON APIs. // This keeps sources small and defers schema-specific decoding to normalizers. // // If a source already decodes into typed provider structs, it can still emit the // raw event; it should simply re-marshal to json.RawMessage (or better: decode // once in the normalizer instead to keep “fetch” separate from “transform”). // // Registration pattern // -------------------- // feedkit normalization uses a match-driven registry (“first match wins”). // // Provider subpackages should expose: // // func Register(reg *normalize.Registry) // // And internal/normalizers/builtins.go should provide one entrypoint: // // func RegisterBuiltins(reg *normalize.Registry) // // which calls each provider’s Register() in a stable order. // // Testing guidance (recommended) // ------------------------------ // Add a unit test per normalizer: // // internal/normalizers/openweather/observation_test.go // // Tests should: // // - build a RAW event with Schema=standards.SchemaRaw... and Payload=json.RawMessage // - run the normalizer // - assert canonical Schema + key payload fields + EffectiveAt // - assert out.Validate() passes package normalizers