Update documentation for SPC outlook support

README.md

@@ -32,6 +32,7 @@ current working directory.
 - [Event wire contract](docs/integrations/events.md)
 - [Postgres table contract](docs/integrations/postgres.md)
 - [NWS integration notes](docs/integrations/nws.md)
+- [SPC integration notes](docs/integrations/spc.md)
 - [Open-Meteo integration notes](docs/integrations/openmeteo.md)
 - [OpenWeather integration notes](docs/integrations/openweather.md)
 - [Architecture policy](docs/policy/architecture.md)

docs/config.md

@@ -52,6 +52,18 @@ sources:
       url: "https://api.weather.gov/alerts?point=38.6239,-90.3571&limit=20"
       user_agent: "Example weatherfeeder operator (ops@example.com)"
 
+  - name: SPCConvectiveOutlookLocal
+    mode: poll
+    driver: spc_convective_outlook
+    every: 30m
+    kinds: ["outlook"]
+    params:
+      latitude: 38.6239
+      longitude: -90.3571
+      location_id: "local"
+      location_name: "Configured point"
+      user_agent: "Example weatherfeeder operator (ops@example.com)"
+
 sinks:
   - name: nats_weather
     driver: nats

docs/integrations/spc.md

@@ -0,0 +1,134 @@
+# SPC Integration Notes
+
+## Purpose
+
+This document describes the Storm Prediction Center convective outlook usage
+implemented by `weatherfeeder`. It is for developers and operators maintaining
+SPC source configuration, provider helpers, fixtures, and tests.
+
+General config syntax belongs in [configuration](../config.md). Emitted JSON
+events are documented in [event wire contract](events.md).
+
+## Implemented Driver
+
+| Driver | Kind | Raw schema | Canonical schema |
+| --- | --- | --- | --- |
+| `spc_convective_outlook` | `outlook` | `raw.spc.convective_outlook.v1` | `weather.outlook.v1` |
+
+## Config Requirements
+
+The driver requires:
+
+- `latitude`
+- `longitude`
+- `user_agent`
+
+Optional params are:
+
+- `location_id`
+- `location_name`
+- `geojson_urls`
+- `discussion_urls`
+- `rss_url`
+- `http_timeout`
+- `http_response_body_limit_bytes`
+
+RSS is not fetched unless `rss_url` is configured. URL override maps are intended
+for tests and upstream URL changes; the default driver configuration owns the
+current Day 1-3 SPC product URLs.
+
+## Upstream Products Used
+
+The source fetches twelve required GeoJSON products every poll:
+
+- Day 1 categorical, tornado, hail, and wind
+- Day 2 categorical, tornado, hail, and wind
+- Day 3 categorical, tornado, hail, and wind
+
+It also fetches three required print pages:
+
+- Day 1 convective outlook print page
+- Day 2 convective outlook print page
+- Day 3 convective outlook print page
+
+GeoJSON products are authoritative for outlook polygons, valid windows, issue
+times, labels, and severity rank. Print pages are authoritative for headline,
+summary, and discussion text.
+
+## Accept Headers
+
+GeoJSON requests use:
+
+```text
+application/geo+json, application/json
+```
+
+Print-page requests use:
+
+```text
+text/html, application/xhtml+xml
+```
+
+RSS requests, when configured, use:
+
+```text
+application/rss+xml, application/xml, text/xml
+```
+
+## Polling And Raw Events
+
+The source polls all required GeoJSON and print-page products as one bundle. If
+any required request fails or returns a non-2xx response, the poll returns an
+error and emits no partial event.
+
+The raw payload contains fetched bodies plus configured location metadata and
+per-product metadata. The source parses only the timestamp metadata needed for
+event effective time selection; canonical mapping belongs to the normalizer.
+
+The source emits no event when a complete fetched bundle is unchanged from the
+previous successful poll. It does this with a source-local hash of the fetched
+document bodies.
+
+## Time Handling
+
+Raw source `effective_at` prefers:
+
+1. the latest valid GeoJSON `ISSUE_ISO`;
+2. the latest print-page `Updated:` timestamp;
+3. RSS `lastBuildDate` when RSS was fetched and parseable;
+4. fetch time.
+
+The normalizer sets canonical `asOf` and normalized event `effective_at` from
+the latest valid outlook feature `issuedAt`, with fallback to print-page update
+time and then the incoming event envelope.
+
+## Mapping Notes
+
+Each GeoJSON feature becomes one canonical outlook. Products are ordered by day,
+then categorical, tornado, hail, and wind. Feature order is preserved within
+each product.
+
+The normalizer computes `containsLocation` with the configured latitude and
+longitude against compact GeoJSON `Polygon` or `MultiPolygon` geometry.
+Coordinates use GeoJSON order, `[longitude, latitude]`, and boundary points
+count as contained.
+
+All outlook polygons are preserved, including polygons that do not contain the
+configured point. Matching print-page headline, summary, and discussion text is
+attached to every outlook for the same day.
+
+## Failure Behavior
+
+Constructor validation failures stop daemon startup. Polling failures are
+returned to the scheduler and emit no event for that poll.
+
+Normalization fails when required GeoJSON timestamps, labels, geometry, or
+configured coordinates are missing or invalid. Print-page extraction failures
+also fail normalization because print pages are required inputs.
+
+## Tests To Inspect
+
+- `internal/providers/spc/*_test.go`
+- `internal/sources/spc/*_test.go`
+- `internal/normalizers/spc/*_test.go`
+- fixtures under `internal/providers/spc/testdata`

docs/internal/runtime.md

@@ -74,7 +74,7 @@ Weatherfeeder-owned runtime state is in process:
 
 - event channel contents;
 - the bounded dedupe key set;
-- source instances and their HTTP conditional validators;
+- source instances and their in-memory unchanged-content state;
 - scheduler and dispatcher goroutines.
 
 There is no weatherfeeder-owned durable scheduler state, checkpoint, replay log,

docs/operations.md

@@ -64,16 +64,20 @@ and before each interval tick. If no jitter is configured in code, feedkit uses
 Poll failures are logged and do not stop the daemon. A failed poll emits no
 events for that source until a subsequent poll succeeds.
 
-## Conditional HTTP Fetches
+## Unchanged Upstream Content
 
-All current sources use feedkit's HTTP polling helper. By default,
-`params.conditional` is `true`, so the helper keeps ETag and Last-Modified
-validators in memory for each source instance.
+Most sources use feedkit's single-document HTTP polling helper. By default,
+`params.conditional` is `true` for those sources, so the helper keeps ETag and
+Last-Modified validators in memory for each source instance.
 
 If the upstream returns `304 Not Modified`, the source emits no events for that
 poll. Validator state is in memory only; restarting the process starts with no
 cached validators.
 
+The SPC convective outlook source polls multiple required documents as one
+bundle. It emits no event when a later complete bundle has the same fetched
+document bodies as the previous successful poll.
+
 ## Processing And Dedupe
 
 Every event passes through normalization first and dedupe second.
@@ -128,6 +132,7 @@ table in that same transaction. See the
 
 - scheduler goroutines and timers;
 - HTTP conditional request validators;
+- source-local unchanged-content state;
 - event channel buffers;
 - per-sink fanout queues;
 - the dedupe ID set.

docs/troubleshooting.md

@@ -58,7 +58,8 @@ Diagnostic: compare the configured driver with the source driver table in
 Safe fix: correct the driver name. Current drivers include `nws_observation`,
 `nws_alerts`, `nws_forecast_hourly`, `nws_forecast_narrative`,
 `nws_forecast_discussion`, `nws_weatherstories`, `openmeteo_observation`,
-`openmeteo_forecast`, and `openweather_observation`.
+`openmeteo_forecast`, `openweather_observation`, and
+`spc_convective_outlook`.
 
 ## `unknown sink driver`
 
@@ -94,6 +95,20 @@ Diagnostic: inspect the named source in the error and check its `params`.
 Safe fix: add non-empty `url` and `user_agent` values. See
 [HTTP source params](config.md#http-source-params).
 
+SPC convective outlook sources do not use `params.url`; they require
+`latitude`, `longitude`, and `user_agent`. See
+[SPC convective outlook params](config.md#spc-convective-outlook-params).
+
+## `params.latitude is required` Or `params.longitude is required`
+
+Symptom: startup exits for an `spc_convective_outlook` source.
+
+Likely cause: the source is missing one of the configured point coordinates.
+
+Diagnostic: inspect the named SPC source in the error and check its `params`.
+
+Safe fix: add numeric `latitude` and `longitude` values in decimal degrees.
+
 ## `url must include units=metric`
 
 Symptom: startup exits for an `openweather_observation` source.

examples/config.nats.yml

@@ -18,6 +18,18 @@ sources:
       url: "https://api.weather.gov/alerts?point=38.6239,-90.3571&limit=20"
       user_agent: "weatherfeeder example (operator@example.com)"
 
+  - name: SPCConvectiveOutlookSTL
+    mode: poll
+    kinds: ["outlook"]
+    driver: spc_convective_outlook
+    every: 30m
+    params:
+      latitude: 38.6239
+      longitude: -90.3571
+      location_id: "stl"
+      location_name: "St. Louis, MO"
+      user_agent: "weatherfeeder example (operator@example.com)"
+
 sinks:
   - name: nats_weather
     driver: nats
@@ -27,4 +39,4 @@ sinks:
 
 routes:
   - sink: nats_weather
-    kinds: ["observation", "alert"]
+    kinds: ["observation", "alert", "outlook"]

examples/config.postgres.yml

@@ -18,6 +18,18 @@ sources:
       url: "https://api.open-meteo.com/v1/forecast?latitude=38.6239&longitude=-90.3571&hourly=temperature_2m,relative_humidity_2m,dew_point_2m,apparent_temperature,precipitation_probability,precipitation,snowfall,weather_code,surface_pressure,wind_speed_10m,wind_direction_10m&forecast_days=3"
       user_agent: "weatherfeeder example (operator@example.com)"
 
+  - name: SPCConvectiveOutlookSTL
+    mode: poll
+    kinds: ["outlook"]
+    driver: spc_convective_outlook
+    every: 30m
+    params:
+      latitude: 38.6239
+      longitude: -90.3571
+      location_id: "stl"
+      location_name: "St. Louis, MO"
+      user_agent: "weatherfeeder example (operator@example.com)"
+
 sinks:
   - name: pg_weather
     driver: postgres
@@ -29,4 +41,4 @@ sinks:
 
 routes:
   - sink: pg_weather
-    kinds: ["observation", "forecast"]
+    kinds: ["observation", "forecast", "outlook"]