NWS Integration Notes

Purpose

This document describes the NWS products that weatherfeeder currently polls and normalizes. It is for developers and operators maintaining NWS source URLs, normalizers, fixtures, and tests.

General config syntax belongs in configuration. Emitted JSON events are documented in event wire contract.

Implemented Drivers

Driver	Kind	Raw schema	Canonical schema
`nws_observation`	`observation`	`raw.nws.observation.v1`	`weather.observation.v1`
`nws_alerts`	`alert`	`raw.nws.alerts.v1`	`weather.alert.v1`
`nws_forecast_hourly`	`forecast`	`raw.nws.hourly.forecast.v1`	`weather.forecast.v1`
`nws_forecast_narrative`	`forecast`	`raw.nws.narrative.forecast.v1`	`weather.forecast.v1`
`nws_forecast_discussion`	`forecast_discussion`	`raw.nws.forecast_discussion.v1`	`weather.forecast_discussion.v1`
`nws_weatherstories`	`weather_story`	`raw.nws.weatherstories.v1`	`weather.weather_story.v1`

Config Requirements

All NWS drivers require HTTP source params:

url
user_agent

The shared HTTP helper also accepts conditional, http_timeout, and http_response_body_limit_bytes. Conditional requests are enabled by default; an upstream 304 Not Modified response emits no event for that poll.

NWS expects a descriptive User-Agent. Do not use anonymous or placeholder contact values in production configs.

Upstream Shapes Used

nws_observation expects the latest station observation GeoJSON shape. The normalizer uses fields under properties such as stationId, stationName, timestamp, textDescription, measured values, presentWeather, and cloudLayers, plus point geometry for day/night inference.

nws_alerts expects an alerts FeatureCollection. The normalizer uses the collection updated timestamp, title, each feature ID, alert classification fields, narrative fields, timing fields, sender fields, and references. properties.onset and properties.ends map to the canonical alert period start and end. properties.expires maps only to canonical expires provider metadata and is not treated as the alert period end.

nws_forecast_hourly and nws_forecast_narrative expect gridpoint forecast GeoJSON with properties.generatedAt, properties.updateTime, elevation, polygon geometry, and ordered periods.

nws_forecast_discussion expects an HTML page containing the discussion text in a <pre> block. The provider helper extracts office identity, product, issue time, update time, key messages, and short/long term sections.

nws_weatherstories expects a JSON response with a stories array. The normalizer uses office ID, start/end/update times, title, description, alt text, priority, order, and download URL.

Accept Headers

NWS JSON sources request:

application/geo+json, application/json

The forecast discussion source requests:

text/html, application/xhtml+xml

Effective Time

Source events set effective_at from the best metadata available:

observations: properties.timestamp;
alerts: collection updated, otherwise latest per-alert timestamp;
hourly and narrative forecasts: properties.generatedAt, otherwise update time;
forecast discussions: parsed issue time;
weather stories: latest story update time, otherwise latest story start time.

Normalizers use canonical payload time as the normalized event effective time. Alerts and weather stories fall back to the incoming event envelope when the payload does not provide a better snapshot time.

Mapping Notes

Observations preserve raw presentWeather fragments and infer WMO condition codes from METAR phenomena, provider text, and cloud-layer fallback. Sea-level pressure is preferred over barometric pressure when present.

Hourly forecasts infer WMO condition codes from shortForecast and icon tokens. Narrative forecasts preserve text but intentionally leave period condition codes unset. Forecast temperatures are converted to Celsius when NWS supplies Fahrenheit, and wind speed strings are converted to kilometers per hour.

Alert timing fields are parsed best-effort. Invalid per-alert timestamps are left unset rather than failing the whole alert run. NWS ends is preserved separately from expires; expires does not fall back to ends. Missing alert IDs are synthesized from the run snapshot time and array position.

Forecast discussion parsing requires an issue time. Weather story entries require start time, end time, and update time.

Failure Behavior

Constructor validation failures stop daemon startup. Polling failures are returned to the scheduler. JSON sources still emit raw payloads when only minimal metadata decoding fails. Forecast discussion polling fails if the HTML cannot be parsed enough to determine the issue time.

Tests To Inspect

internal/sources/nws/*_test.go
internal/normalizers/nws/*_test.go
internal/providers/nws/*_test.go
fixtures under internal/providers/nws/testdata

5.0 KiB Raw Blame History