102 lines
3.6 KiB
Markdown
102 lines
3.6 KiB
Markdown
# Normalizer Internals
|
|
|
|
## Purpose
|
|
|
|
Normalizers convert raw provider events into canonical weather events. They are
|
|
weather-domain mapping code and should stay independent of runtime wiring,
|
|
source polling, and sink persistence.
|
|
|
|
Detailed package conventions live in `internal/normalizers/doc.go`.
|
|
|
|
## Inputs And Outputs
|
|
|
|
Inputs are raw feed events whose schemas identify provider payload shape.
|
|
Outputs are canonical feed events using `model` payloads and `weather.*`
|
|
schemas.
|
|
|
|
Current mappings:
|
|
|
|
| Raw schema | Canonical schema |
|
|
| --- | --- |
|
|
| `raw.nws.observation.v1` | `weather.observation.v1` |
|
|
| `raw.openmeteo.current.v1` | `weather.observation.v1` |
|
|
| `raw.openweather.current.v1` | `weather.observation.v1` |
|
|
| `raw.nws.hourly.forecast.v1` | `weather.forecast.v1` |
|
|
| `raw.nws.narrative.forecast.v1` | `weather.forecast.v1` |
|
|
| `raw.openmeteo.hourly.forecast.v1` | `weather.forecast.v1` |
|
|
| `raw.nws.forecast_discussion.v1` | `weather.forecast_discussion.v1` |
|
|
| `raw.nws.weatherstories.v1` | `weather.weather_story.v1` |
|
|
| `raw.nws.alerts.v1` | `weather.alert.v1` |
|
|
| `raw.spc.convective_outlook.v1` | `weather.outlook.v2` |
|
|
|
|
## Boundaries
|
|
|
|
- Normalizers match by `Event.Schema`.
|
|
- Normalizers decode raw payloads into provider structs.
|
|
- Normalizers map provider data into canonical `model` payloads.
|
|
- Normalizers do not fetch network data, read config, route events, or write
|
|
sinks.
|
|
- Shared cross-provider behavior belongs in `internal/normalizers/common`.
|
|
- Provider-specific helper logic shared with sources belongs in
|
|
`internal/providers/<provider>`.
|
|
|
|
## Config Fields Used
|
|
|
|
Normalizers do not read config. They operate only on incoming events.
|
|
|
|
## External Adapters Used
|
|
|
|
Runtime composition creates feedkit's normalize processor with
|
|
`RequireMatch=false`. Events without a matching normalizer pass through
|
|
unchanged.
|
|
|
|
Weatherfeeder registers normalizers in a stable order:
|
|
|
|
1. NWS
|
|
2. Open-Meteo
|
|
3. OpenWeather
|
|
4. SPC
|
|
|
|
The current normalizers avoid ambiguous matches by using schema equality.
|
|
|
|
The SPC outlook normalizer decodes the raw multi-document bundle, maps
|
|
location-containing GeoJSON features to canonical outlooks, and adds one
|
|
run-level print-page discussion per retained outlook day. It preserves compact
|
|
GeoJSON feature geometry and computes `containsLocation` with
|
|
`internal/geo.ContainsPoint` using the source-configured point. Boundary points
|
|
count as contained. Polygons that do not contain the point are omitted from the
|
|
canonical run.
|
|
|
|
## State
|
|
|
|
Normalizers should be stateless. Shared helpers should be deterministic and free
|
|
of I/O.
|
|
|
|
## Failure Behavior
|
|
|
|
Malformed required raw payload data should produce contextual errors from the
|
|
owning normalizer. Successful normalization validates the output event before it
|
|
continues through the pipeline.
|
|
|
|
`internal/normalizers/common.Finalize` preserves the input event envelope except
|
|
for schema, payload, and effective time. It also rounds canonical float values
|
|
to four digits after the decimal point.
|
|
|
|
## Tests To Inspect
|
|
|
|
- `internal/normalizers/builtins_test.go`
|
|
- provider normalizer tests under `internal/normalizers/nws`
|
|
- provider normalizer tests under `internal/normalizers/openmeteo`
|
|
- provider normalizer tests under `internal/normalizers/openweather`
|
|
- provider normalizer tests under `internal/normalizers/spc`
|
|
- common helper tests under `internal/normalizers/common`
|
|
|
|
## Invariants
|
|
|
|
- Match by schema constants from `standards`.
|
|
- Preserve the event envelope except for intentional canonical changes.
|
|
- Produce canonical payload structs from `model`.
|
|
- Validate normalized events before returning them.
|
|
- Keep normalizers independent of sources, sinks, config loading, and runtime
|
|
composition.
|