135 lines
4.0 KiB
Markdown
135 lines
4.0 KiB
Markdown
# SPC Integration Notes
|
|
|
|
## Purpose
|
|
|
|
This document describes the Storm Prediction Center convective outlook usage
|
|
implemented by `weatherfeeder`. It is for developers and operators maintaining
|
|
SPC source configuration, provider helpers, fixtures, and tests.
|
|
|
|
General config syntax belongs in [configuration](../config.md). Emitted JSON
|
|
events are documented in [event wire contract](events.md).
|
|
|
|
## Implemented Driver
|
|
|
|
| Driver | Kind | Raw schema | Canonical schema |
|
|
| --- | --- | --- | --- |
|
|
| `spc_convective_outlook` | `outlook` | `raw.spc.convective_outlook.v1` | `weather.outlook.v1` |
|
|
|
|
## Config Requirements
|
|
|
|
The driver requires:
|
|
|
|
- `latitude`
|
|
- `longitude`
|
|
- `user_agent`
|
|
|
|
Optional params are:
|
|
|
|
- `location_id`
|
|
- `location_name`
|
|
- `geojson_urls`
|
|
- `discussion_urls`
|
|
- `rss_url`
|
|
- `http_timeout`
|
|
- `http_response_body_limit_bytes`
|
|
|
|
RSS is not fetched unless `rss_url` is configured. URL override maps are intended
|
|
for tests and upstream URL changes; the default driver configuration owns the
|
|
current Day 1-3 SPC product URLs.
|
|
|
|
## Upstream Products Used
|
|
|
|
The source fetches nine required GeoJSON products every poll:
|
|
|
|
- Day 1 categorical, tornado, hail, and wind
|
|
- Day 2 categorical, tornado, hail, and wind
|
|
- Day 3 categorical
|
|
|
|
It also fetches three required print pages:
|
|
|
|
- Day 1 convective outlook print page
|
|
- Day 2 convective outlook print page
|
|
- Day 3 convective outlook print page
|
|
|
|
GeoJSON products are authoritative for outlook polygons, valid windows, issue
|
|
times, labels, and severity rank. Print pages are authoritative for headline,
|
|
summary, and discussion text.
|
|
|
|
## Accept Headers
|
|
|
|
GeoJSON requests use:
|
|
|
|
```text
|
|
application/geo+json, application/json
|
|
```
|
|
|
|
Print-page requests use:
|
|
|
|
```text
|
|
text/html, application/xhtml+xml
|
|
```
|
|
|
|
RSS requests, when configured, use:
|
|
|
|
```text
|
|
application/rss+xml, application/xml, text/xml
|
|
```
|
|
|
|
## Polling And Raw Events
|
|
|
|
The source polls all required GeoJSON and print-page products as one bundle. If
|
|
any required request fails or returns a non-2xx response, the poll returns an
|
|
error and emits no partial event.
|
|
|
|
The raw payload contains fetched bodies plus configured location metadata and
|
|
per-product metadata. The source parses only the timestamp metadata needed for
|
|
event effective time selection; canonical mapping belongs to the normalizer.
|
|
|
|
The source emits no event when a complete fetched bundle is unchanged from the
|
|
previous successful poll. It does this with a source-local hash of the fetched
|
|
document bodies.
|
|
|
|
## Time Handling
|
|
|
|
Raw source `effective_at` prefers:
|
|
|
|
1. the latest valid GeoJSON `ISSUE_ISO`;
|
|
2. the latest print-page `Updated:` timestamp;
|
|
3. RSS `lastBuildDate` when RSS was fetched and parseable;
|
|
4. fetch time.
|
|
|
|
The normalizer sets canonical `asOf` and normalized event `effective_at` from
|
|
the latest valid outlook feature `issuedAt`, with fallback to print-page update
|
|
time and then the incoming event envelope.
|
|
|
|
## Mapping Notes
|
|
|
|
Each GeoJSON feature becomes one canonical outlook. Products are ordered by day,
|
|
then categorical, tornado, hail, and wind. Feature order is preserved within
|
|
each product.
|
|
|
|
The normalizer computes `containsLocation` with the configured latitude and
|
|
longitude against compact GeoJSON `Polygon` or `MultiPolygon` geometry.
|
|
Coordinates use GeoJSON order, `[longitude, latitude]`, and boundary points
|
|
count as contained.
|
|
|
|
All outlook polygons are preserved, including polygons that do not contain the
|
|
configured point. Matching print-page headline, summary, and discussion text is
|
|
attached to every outlook for the same day.
|
|
|
|
## Failure Behavior
|
|
|
|
Constructor validation failures stop daemon startup. Polling failures are
|
|
returned to the scheduler and emit no event for that poll.
|
|
|
|
Normalization fails when required GeoJSON timestamps, labels, geometry, or
|
|
configured coordinates are missing or invalid. Print-page extraction failures
|
|
also fail normalization because print pages are required inputs.
|
|
|
|
## Tests To Inspect
|
|
|
|
- `internal/providers/spc/*_test.go`
|
|
- `internal/sources/spc/*_test.go`
|
|
- `internal/normalizers/spc/*_test.go`
|
|
- fixtures under `internal/providers/spc/testdata`
|