Document SPC outlook support

This commit is contained in:
2026-06-11 00:27:23 +00:00
parent e966276c40
commit f91a185f9d
8 changed files with 225 additions and 18 deletions

View File

@@ -27,6 +27,7 @@ Current mappings:
| `raw.nws.forecast_discussion.v1` | `weather.forecast_discussion.v1` |
| `raw.nws.weatherstories.v1` | `weather.weather_story.v1` |
| `raw.nws.alerts.v1` | `weather.alert.v1` |
| `raw.spc.convective_outlook.v1` | `weather.outlook.v1` |
## Boundaries
@@ -54,9 +55,18 @@ Weatherfeeder registers normalizers in a stable order:
1. NWS
2. Open-Meteo
3. OpenWeather
4. SPC
The current normalizers avoid ambiguous matches by using schema equality.
The SPC outlook normalizer decodes the raw multi-document bundle, maps each
GeoJSON feature to a canonical outlook, and enriches all outlooks for a day with
the matching print-page headline, summary, and discussion. It preserves compact
GeoJSON feature geometry and computes `containsLocation` with
`internal/geo.ContainsPoint` using the source-configured point. Boundary points
count as contained, and all polygons are preserved whether or not they contain
the point.
## State
Normalizers should be stateless. Shared helpers should be deterministic and free
@@ -78,6 +88,7 @@ to four digits after the decimal point.
- provider normalizer tests under `internal/normalizers/nws`
- provider normalizer tests under `internal/normalizers/openmeteo`
- provider normalizer tests under `internal/normalizers/openweather`
- provider normalizer tests under `internal/normalizers/spc`
- common helper tests under `internal/normalizers/common`
## Invariants

View File

@@ -19,6 +19,7 @@ Inputs are canonical feed events. The mapper currently handles these schemas:
- `weather.forecast_discussion.v1`
- `weather.weather_story.v1`
- `weather.alert.v1`
- `weather.outlook.v1`
Outputs are feedkit `PostgresWrite` values for weatherfeeder-owned tables.
Unsupported schemas produce no writes and no error.
@@ -80,6 +81,7 @@ Child rows use positional indexes to preserve canonical array order:
- `story_index`
- `alert_index`
- `reference_index`
- `outlook_index`
Required canonical fields are validated before writes are returned:
@@ -89,10 +91,14 @@ Required canonical fields are validated before writes are returned:
- forecast discussions require `issuedAt` and `product`;
- weather story runs require `asOf`, and each story requires `startTime`,
`endTime`, and `updatedAt`;
- alert runs require `asOf`, and each alert requires `id`.
- alert runs require `asOf`, and each alert requires `id`;
- outlook runs require `asOf`, and each outlook requires `id`, `provider`,
`product`, `day`, `outlookType`, `label`, `validFrom`, `validTo`, `issuedAt`,
`expiresAt`, and `geometry`.
Nullable canonical values are converted to SQL nulls by mapper helpers.
Observation present-weather raw values are stored as compact JSON text.
Observation present-weather raw values and outlook geometry values are stored as
compact JSON text.
## Failure Behavior

View File

@@ -28,6 +28,7 @@ Current drivers:
| `openmeteo_observation` | `observation` | `raw.openmeteo.current.v1` |
| `openmeteo_forecast` | `forecast` | `raw.openmeteo.hourly.forecast.v1` |
| `openweather_observation` | `observation` | `raw.openweather.current.v1` |
| `spc_convective_outlook` | `outlook` | `raw.spc.convective_outlook.v1` |
## Boundaries
@@ -40,7 +41,7 @@ Current drivers:
## Config Fields Used
All current source drivers use feedkit `HTTPSource`.
Most source drivers use feedkit `HTTPSource`.
Required params:
@@ -56,12 +57,18 @@ Optional params:
OpenWeather observation sources additionally require the configured URL to use
metric units. This is enforced by `internal/providers/openweather`.
The SPC convective outlook source is a multi-document poll source rather than a
single-URL `HTTPSource`. It requires `latitude`, `longitude`, and `user_agent`;
accepts optional `location_id`, `location_name`, `geojson_urls`,
`discussion_urls`, and `rss_url`; and supports `http_timeout` and
`http_response_body_limit_bytes`.
Source-level `kinds`, when configured, are validated against the source's
advertised `Kinds()`.
## External Adapters Used
Sources use feedkit's HTTP helper for:
Most sources use feedkit's HTTP helper for:
- request construction;
- `User-Agent` and `Accept` headers;
@@ -71,11 +78,15 @@ Sources use feedkit's HTTP helper for:
NWS helpers parse NWS timestamps. Open-Meteo helpers parse provider-local times
with timezone or UTC-offset data. OpenWeather helpers enforce metric-unit URLs.
SPC helpers define Day 1-3 product metadata, parse GeoJSON timestamps, extract
cleaned print-page discussion text, and parse optional RSS metadata.
## State
HTTP conditional validators are held in each source instance. They are not
persisted across process restarts.
HTTP conditional validators are held in each single-document HTTP source
instance. They are not persisted across process restarts. The SPC outlook source
keeps only a source-local hash of the most recent complete required product
bundle and emits no event when a later complete bundle is unchanged.
## Failure Behavior
@@ -88,12 +99,18 @@ to default ID/effective-time behavior from feedkit source helpers.
Unchanged conditional responses return no events and no error.
The SPC outlook source fetches all configured Day 1-3 GeoJSON products and print
pages atomically. If any required GeoJSON or print-page request fails, the poll
returns an error and emits no partial bundle. RSS is fetched only when `rss_url`
is configured.
## Tests To Inspect
- `internal/sources/builtins_test.go`
- provider source tests under `internal/sources/nws`
- provider source tests under `internal/sources/openmeteo`
- provider source tests under `internal/sources/openweather`
- provider source tests under `internal/sources/spc`
- provider helper tests under `internal/providers`
## Invariants