# NWS Integration Notes

## Purpose

This document describes the NWS products that `weatherfeeder` currently polls
and normalizes. It is for developers and operators maintaining NWS source URLs,
normalizers, fixtures, and tests.

General config syntax belongs in [configuration](../config.md). Emitted JSON
events are documented in [event wire contract](events.md).

## Implemented Drivers

| Driver | Kind | Raw schema | Canonical schema |
| --- | --- | --- | --- |
| `nws_observation` | `observation` | `raw.nws.observation.v1` | `weather.observation.v1` |
| `nws_alerts` | `alert` | `raw.nws.alerts.v1` | `weather.alert.v1` |
| `nws_forecast_hourly` | `forecast` | `raw.nws.hourly.forecast.v1` | `weather.forecast.v1` |
| `nws_forecast_narrative` | `forecast` | `raw.nws.narrative.forecast.v1` | `weather.forecast.v1` |
| `nws_forecast_discussion` | `forecast_discussion` | `raw.nws.forecast_discussion.v1` | `weather.forecast_discussion.v1` |
| `nws_weatherstories` | `weather_story` | `raw.nws.weatherstories.v1` | `weather.weather_story.v1` |

## Config Requirements

All NWS drivers require HTTP source params:

- `url`
- `user_agent`

The shared HTTP helper also accepts `conditional`, `http_timeout`, and
`http_response_body_limit_bytes`. Conditional requests are enabled by default;
an upstream `304 Not Modified` response emits no event for that poll.

NWS expects a descriptive `User-Agent`. Do not use anonymous or placeholder
contact values in production configs.

## Upstream Shapes Used

`nws_observation` expects the latest station observation GeoJSON shape. The
normalizer uses fields under `properties` such as `stationId`, `stationName`,
`timestamp`, `textDescription`, measured values, `presentWeather`, and
`cloudLayers`, plus point geometry for day/night inference.

`nws_alerts` expects an alerts FeatureCollection. The normalizer uses the
collection `updated` timestamp, `title`, each feature ID, alert classification
fields, narrative fields, timing fields, sender fields, and references.
`properties.onset` and `properties.ends` map to the canonical alert period
start and end. `properties.expires` maps only to canonical `expires` provider
metadata and is not treated as the alert period end.

`nws_forecast_hourly` and `nws_forecast_narrative` expect gridpoint forecast
GeoJSON with `properties.generatedAt`, `properties.updateTime`, elevation,
polygon geometry, and ordered `periods`.

`nws_forecast_discussion` expects an HTML page containing the discussion text in
a `<pre>` block. The provider helper extracts office identity, product, issue
time, update time, key messages, and short/long term sections.

`nws_weatherstories` expects a JSON response with a `stories` array. The
normalizer uses office ID, start/end/update times, title, description, alt text,
priority, order, and download URL.

## Accept Headers

NWS JSON sources request:

```text
application/geo+json, application/json
```

The forecast discussion source requests:

```text
text/html, application/xhtml+xml
```

## Effective Time

Source events set `effective_at` from the best metadata available:

- observations: `properties.timestamp`;
- alerts: collection `updated`, otherwise latest per-alert timestamp;
- hourly and narrative forecasts: `properties.generatedAt`, otherwise update
  time;
- forecast discussions: parsed issue time;
- weather stories: latest story update time, otherwise latest story start time.

Normalizers use canonical payload time as the normalized event effective time.
Alerts and weather stories fall back to the incoming event envelope when the
payload does not provide a better snapshot time.

## Mapping Notes

Observations preserve raw `presentWeather` fragments and infer WMO condition
codes from METAR phenomena, provider text, and cloud-layer fallback. Sea-level
pressure is preferred over barometric pressure when present.

Hourly forecasts infer WMO condition codes from `shortForecast` and icon tokens.
Narrative forecasts preserve text but intentionally leave period condition codes
unset. Forecast temperatures are converted to Celsius when NWS supplies
Fahrenheit, and wind speed strings are converted to kilometers per hour.

Alert timing fields are parsed best-effort. Invalid per-alert timestamps are
left unset rather than failing the whole alert run. NWS `ends` is preserved
separately from `expires`; `expires` does not fall back to `ends`. Missing alert
IDs are synthesized from the run snapshot time and array position.

Forecast discussion parsing requires an issue time. Weather story entries require
start time, end time, and update time.

## Failure Behavior

Constructor validation failures stop daemon startup. Polling failures are
returned to the scheduler. JSON sources still emit raw payloads when only
minimal metadata decoding fails. Forecast discussion polling fails if the HTML
cannot be parsed enough to determine the issue time.

## Tests To Inspect

- `internal/sources/nws/*_test.go`
- `internal/normalizers/nws/*_test.go`
- `internal/providers/nws/*_test.go`
- fixtures under `internal/providers/nws/testdata`