Document SPC outlook support
This commit is contained in:
@@ -27,6 +27,7 @@ Current mappings:
|
||||
| `raw.nws.forecast_discussion.v1` | `weather.forecast_discussion.v1` |
|
||||
| `raw.nws.weatherstories.v1` | `weather.weather_story.v1` |
|
||||
| `raw.nws.alerts.v1` | `weather.alert.v1` |
|
||||
| `raw.spc.convective_outlook.v1` | `weather.outlook.v1` |
|
||||
|
||||
## Boundaries
|
||||
|
||||
@@ -54,9 +55,18 @@ Weatherfeeder registers normalizers in a stable order:
|
||||
1. NWS
|
||||
2. Open-Meteo
|
||||
3. OpenWeather
|
||||
4. SPC
|
||||
|
||||
The current normalizers avoid ambiguous matches by using schema equality.
|
||||
|
||||
The SPC outlook normalizer decodes the raw multi-document bundle, maps each
|
||||
GeoJSON feature to a canonical outlook, and enriches all outlooks for a day with
|
||||
the matching print-page headline, summary, and discussion. It preserves compact
|
||||
GeoJSON feature geometry and computes `containsLocation` with
|
||||
`internal/geo.ContainsPoint` using the source-configured point. Boundary points
|
||||
count as contained, and all polygons are preserved whether or not they contain
|
||||
the point.
|
||||
|
||||
## State
|
||||
|
||||
Normalizers should be stateless. Shared helpers should be deterministic and free
|
||||
@@ -78,6 +88,7 @@ to four digits after the decimal point.
|
||||
- provider normalizer tests under `internal/normalizers/nws`
|
||||
- provider normalizer tests under `internal/normalizers/openmeteo`
|
||||
- provider normalizer tests under `internal/normalizers/openweather`
|
||||
- provider normalizer tests under `internal/normalizers/spc`
|
||||
- common helper tests under `internal/normalizers/common`
|
||||
|
||||
## Invariants
|
||||
|
||||
@@ -19,6 +19,7 @@ Inputs are canonical feed events. The mapper currently handles these schemas:
|
||||
- `weather.forecast_discussion.v1`
|
||||
- `weather.weather_story.v1`
|
||||
- `weather.alert.v1`
|
||||
- `weather.outlook.v1`
|
||||
|
||||
Outputs are feedkit `PostgresWrite` values for weatherfeeder-owned tables.
|
||||
Unsupported schemas produce no writes and no error.
|
||||
@@ -80,6 +81,7 @@ Child rows use positional indexes to preserve canonical array order:
|
||||
- `story_index`
|
||||
- `alert_index`
|
||||
- `reference_index`
|
||||
- `outlook_index`
|
||||
|
||||
Required canonical fields are validated before writes are returned:
|
||||
|
||||
@@ -89,10 +91,14 @@ Required canonical fields are validated before writes are returned:
|
||||
- forecast discussions require `issuedAt` and `product`;
|
||||
- weather story runs require `asOf`, and each story requires `startTime`,
|
||||
`endTime`, and `updatedAt`;
|
||||
- alert runs require `asOf`, and each alert requires `id`.
|
||||
- alert runs require `asOf`, and each alert requires `id`;
|
||||
- outlook runs require `asOf`, and each outlook requires `id`, `provider`,
|
||||
`product`, `day`, `outlookType`, `label`, `validFrom`, `validTo`, `issuedAt`,
|
||||
`expiresAt`, and `geometry`.
|
||||
|
||||
Nullable canonical values are converted to SQL nulls by mapper helpers.
|
||||
Observation present-weather raw values are stored as compact JSON text.
|
||||
Observation present-weather raw values and outlook geometry values are stored as
|
||||
compact JSON text.
|
||||
|
||||
## Failure Behavior
|
||||
|
||||
|
||||
@@ -28,6 +28,7 @@ Current drivers:
|
||||
| `openmeteo_observation` | `observation` | `raw.openmeteo.current.v1` |
|
||||
| `openmeteo_forecast` | `forecast` | `raw.openmeteo.hourly.forecast.v1` |
|
||||
| `openweather_observation` | `observation` | `raw.openweather.current.v1` |
|
||||
| `spc_convective_outlook` | `outlook` | `raw.spc.convective_outlook.v1` |
|
||||
|
||||
## Boundaries
|
||||
|
||||
@@ -40,7 +41,7 @@ Current drivers:
|
||||
|
||||
## Config Fields Used
|
||||
|
||||
All current source drivers use feedkit `HTTPSource`.
|
||||
Most source drivers use feedkit `HTTPSource`.
|
||||
|
||||
Required params:
|
||||
|
||||
@@ -56,12 +57,18 @@ Optional params:
|
||||
OpenWeather observation sources additionally require the configured URL to use
|
||||
metric units. This is enforced by `internal/providers/openweather`.
|
||||
|
||||
The SPC convective outlook source is a multi-document poll source rather than a
|
||||
single-URL `HTTPSource`. It requires `latitude`, `longitude`, and `user_agent`;
|
||||
accepts optional `location_id`, `location_name`, `geojson_urls`,
|
||||
`discussion_urls`, and `rss_url`; and supports `http_timeout` and
|
||||
`http_response_body_limit_bytes`.
|
||||
|
||||
Source-level `kinds`, when configured, are validated against the source's
|
||||
advertised `Kinds()`.
|
||||
|
||||
## External Adapters Used
|
||||
|
||||
Sources use feedkit's HTTP helper for:
|
||||
Most sources use feedkit's HTTP helper for:
|
||||
|
||||
- request construction;
|
||||
- `User-Agent` and `Accept` headers;
|
||||
@@ -71,11 +78,15 @@ Sources use feedkit's HTTP helper for:
|
||||
|
||||
NWS helpers parse NWS timestamps. Open-Meteo helpers parse provider-local times
|
||||
with timezone or UTC-offset data. OpenWeather helpers enforce metric-unit URLs.
|
||||
SPC helpers define Day 1-3 product metadata, parse GeoJSON timestamps, extract
|
||||
cleaned print-page discussion text, and parse optional RSS metadata.
|
||||
|
||||
## State
|
||||
|
||||
HTTP conditional validators are held in each source instance. They are not
|
||||
persisted across process restarts.
|
||||
HTTP conditional validators are held in each single-document HTTP source
|
||||
instance. They are not persisted across process restarts. The SPC outlook source
|
||||
keeps only a source-local hash of the most recent complete required product
|
||||
bundle and emits no event when a later complete bundle is unchanged.
|
||||
|
||||
## Failure Behavior
|
||||
|
||||
@@ -88,12 +99,18 @@ to default ID/effective-time behavior from feedkit source helpers.
|
||||
|
||||
Unchanged conditional responses return no events and no error.
|
||||
|
||||
The SPC outlook source fetches all configured Day 1-3 GeoJSON products and print
|
||||
pages atomically. If any required GeoJSON or print-page request fails, the poll
|
||||
returns an error and emits no partial bundle. RSS is fetched only when `rss_url`
|
||||
is configured.
|
||||
|
||||
## Tests To Inspect
|
||||
|
||||
- `internal/sources/builtins_test.go`
|
||||
- provider source tests under `internal/sources/nws`
|
||||
- provider source tests under `internal/sources/openmeteo`
|
||||
- provider source tests under `internal/sources/openweather`
|
||||
- provider source tests under `internal/sources/spc`
|
||||
- provider helper tests under `internal/providers`
|
||||
|
||||
## Invariants
|
||||
|
||||
Reference in New Issue
Block a user