Document provider integration contracts
This commit is contained in:
119
docs/integrations/nws.md
Normal file
119
docs/integrations/nws.md
Normal file
@@ -0,0 +1,119 @@
|
||||
# NWS Integration Notes
|
||||
|
||||
## Purpose
|
||||
|
||||
This document describes the NWS products that `weatherfeeder` currently polls
|
||||
and normalizes. It is for developers and operators maintaining NWS source URLs,
|
||||
normalizers, fixtures, and tests.
|
||||
|
||||
General config syntax belongs in [configuration](../config.md). Emitted JSON
|
||||
events are documented in [event wire contract](events.md).
|
||||
|
||||
## Implemented Drivers
|
||||
|
||||
| Driver | Kind | Raw schema | Canonical schema |
|
||||
| --- | --- | --- | --- |
|
||||
| `nws_observation` | `observation` | `raw.nws.observation.v1` | `weather.observation.v1` |
|
||||
| `nws_alerts` | `alert` | `raw.nws.alerts.v1` | `weather.alert.v1` |
|
||||
| `nws_forecast_hourly` | `forecast` | `raw.nws.hourly.forecast.v1` | `weather.forecast.v1` |
|
||||
| `nws_forecast_narrative` | `forecast` | `raw.nws.narrative.forecast.v1` | `weather.forecast.v1` |
|
||||
| `nws_forecast_discussion` | `forecast_discussion` | `raw.nws.forecast_discussion.v1` | `weather.forecast_discussion.v1` |
|
||||
| `nws_weatherstories` | `weather_story` | `raw.nws.weatherstories.v1` | `weather.weather_story.v1` |
|
||||
|
||||
## Config Requirements
|
||||
|
||||
All NWS drivers require HTTP source params:
|
||||
|
||||
- `url`
|
||||
- `user_agent`
|
||||
|
||||
The shared HTTP helper also accepts `conditional`, `http_timeout`, and
|
||||
`http_response_body_limit_bytes`. Conditional requests are enabled by default;
|
||||
an upstream `304 Not Modified` response emits no event for that poll.
|
||||
|
||||
NWS expects a descriptive `User-Agent`. Do not use anonymous or placeholder
|
||||
contact values in production configs.
|
||||
|
||||
## Upstream Shapes Used
|
||||
|
||||
`nws_observation` expects the latest station observation GeoJSON shape. The
|
||||
normalizer uses fields under `properties` such as `stationId`, `stationName`,
|
||||
`timestamp`, `textDescription`, measured values, `presentWeather`, and
|
||||
`cloudLayers`, plus point geometry for day/night inference.
|
||||
|
||||
`nws_alerts` expects an alerts FeatureCollection. The normalizer uses the
|
||||
collection `updated` timestamp, `title`, each feature ID, alert classification
|
||||
fields, narrative fields, timing fields, sender fields, and references.
|
||||
|
||||
`nws_forecast_hourly` and `nws_forecast_narrative` expect gridpoint forecast
|
||||
GeoJSON with `properties.generatedAt`, `properties.updateTime`, elevation,
|
||||
polygon geometry, and ordered `periods`.
|
||||
|
||||
`nws_forecast_discussion` expects an HTML page containing the discussion text in
|
||||
a `<pre>` block. The provider helper extracts office identity, product, issue
|
||||
time, update time, key messages, and short/long term sections.
|
||||
|
||||
`nws_weatherstories` expects a JSON response with a `stories` array. The
|
||||
normalizer uses office ID, start/end/update times, title, description, alt text,
|
||||
priority, order, and download URL.
|
||||
|
||||
## Accept Headers
|
||||
|
||||
NWS JSON sources request:
|
||||
|
||||
```text
|
||||
application/geo+json, application/json
|
||||
```
|
||||
|
||||
The forecast discussion source requests:
|
||||
|
||||
```text
|
||||
text/html, application/xhtml+xml
|
||||
```
|
||||
|
||||
## Effective Time
|
||||
|
||||
Source events set `effective_at` from the best metadata available:
|
||||
|
||||
- observations: `properties.timestamp`;
|
||||
- alerts: collection `updated`, otherwise latest per-alert timestamp;
|
||||
- hourly and narrative forecasts: `properties.generatedAt`, otherwise update
|
||||
time;
|
||||
- forecast discussions: parsed issue time;
|
||||
- weather stories: latest story update time, otherwise latest story start time.
|
||||
|
||||
Normalizers use canonical payload time as the normalized event effective time.
|
||||
Alerts and weather stories fall back to the incoming event envelope when the
|
||||
payload does not provide a better snapshot time.
|
||||
|
||||
## Mapping Notes
|
||||
|
||||
Observations preserve raw `presentWeather` fragments and infer WMO condition
|
||||
codes from METAR phenomena, provider text, and cloud-layer fallback. Sea-level
|
||||
pressure is preferred over barometric pressure when present.
|
||||
|
||||
Hourly forecasts infer WMO condition codes from `shortForecast` and icon tokens.
|
||||
Narrative forecasts preserve text but intentionally leave period condition codes
|
||||
unset. Forecast temperatures are converted to Celsius when NWS supplies
|
||||
Fahrenheit, and wind speed strings are converted to kilometers per hour.
|
||||
|
||||
Alert timing fields are parsed best-effort. Invalid per-alert timestamps are
|
||||
left unset rather than failing the whole alert run. Missing alert IDs are
|
||||
synthesized from the run snapshot time and array position.
|
||||
|
||||
Forecast discussion parsing requires an issue time. Weather story entries require
|
||||
start time, end time, and update time.
|
||||
|
||||
## Failure Behavior
|
||||
|
||||
Constructor validation failures stop daemon startup. Polling failures are
|
||||
returned to the scheduler. JSON sources still emit raw payloads when only
|
||||
minimal metadata decoding fails. Forecast discussion polling fails if the HTML
|
||||
cannot be parsed enough to determine the issue time.
|
||||
|
||||
## Tests To Inspect
|
||||
|
||||
- `internal/sources/nws/*_test.go`
|
||||
- `internal/normalizers/nws/*_test.go`
|
||||
- `internal/providers/nws/*_test.go`
|
||||
- fixtures under `internal/providers/nws/testdata`
|
||||
Reference in New Issue
Block a user