eric/weatherreporter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Fetch SPC convective outlook data

Browse Source
This commit is contained in:
Eric Rakestraw
2026-06-12 14:51:54 +00:00
parent 3bcccb4a7b
commit 0041845935
8 changed files with 216 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/config.md
View File

@@ -130,7 +130,9 @@ name the variable only; they should not contain the token value.
- `sources`: optional map of source-specific overrides, using the same policy values.
Hourly forecast data is required for generated reports. Optional sources use
the missing-source policy.
the missing-source policy. Source override keys include `observations`,
`current`, `narrative`, `alerts`, `discussion`, `weather_story`, and
`spc_convective_outlooks`.
### `scriptorium`

18
docs/integrations/weatherapi.md
View File

@@ -37,6 +37,10 @@ generation.
means the endpoint was checked and there are no current active alerts. The
adapter records a non-missing alerts source and an empty alert run.
For `/outlooks/convective`, `data: null` means no latest run is available and
follows missing-source policy. A non-null run with empty `outlooks` and
`discussions` arrays is checked empty data, not a missing source.
Malformed JSON envelopes, non-2xx statuses, and response read failures include
endpoint context in returned errors. Decode errors include source context when
they fail the fetch; optional malformed sources follow the missing-source policy.
@@ -49,11 +53,12 @@ The adapter sends these query parameters:
- `units`: from `weather_api.units`
- `precision`: from `weather_api.precision` on observations, current
conditions, hourly forecast, and narrative forecast requests
- `tz`: from `weather_api.timezone` on hourly forecast, narrative forecast, and
discussion requests
- `tz`: from `weather_api.timezone` on hourly forecast, narrative forecast,
discussion, and SPC convective outlook requests
Alerts do not receive `precision` or `tz`. Weather story requests receive only
`format=json`.
`format=json`. SPC convective outlook requests receive only `format=json` and
`tz`; they do not receive `units` or `precision`.
## Endpoints Used
@@ -66,6 +71,7 @@ The adapter fetches these endpoints once per bundle:
- `/alerts/active`
- `/discussion`
- `/weatherstories/latest`
- `/outlooks/convective`
`weatherreporter` does not call day-slice forecast endpoints or discussion
subsection endpoints. Report-period selection and daypart summarization happen
@@ -88,6 +94,7 @@ source-specific `missing_source.sources` policy:
- `alerts` for `/alerts/active`
- `discussion` for `/discussion`
- `weather_story` for `/weatherstories/latest`
- `spc_convective_outlooks` for `/outlooks/convective`
Policy behavior:
@@ -99,6 +106,9 @@ For `/alerts/active`, an HTTP error or missing `data` field still fails or
follows the relevant error path, but explicit `data: null` is not a
missing-source condition.
For `/outlooks/convective`, a non-null data object with empty outlook and
discussion arrays is accepted as checked empty data.
## Source Identity
For source payloads accepted into the bundle, including the explicit `null`
@@ -126,6 +136,8 @@ types in `internal/forecast/bundle.go`, including:
- discussion metadata, key messages, and short/long-term section text
- latest weather story title, description, timing, priority, order, alt text,
and download URL
- SPC convective outlook run metadata, outlooks, discussions, and GeoJSON
geometry
The adapter intentionally keeps upstream transport and envelope details inside
`internal/adapters/weatherapi`; downstream packages consume the normalized

18
docs/internal/weather-data.md
View File

@@ -4,10 +4,10 @@ This document describes Weather API ingestion into `weatherdata.Bundle`.
## Purpose
`internal/adapters/weatherapi` fetches normalized weather data from one
configured Weather API endpoint and assembles the bundle consumed by forecast
derivation and module builders. Module builders expose normalized current
conditions and weather story context when those sources are available.
`internal/adapters/weatherapi` fetches normalized weather data from the
configured Weather API and assembles the bundle consumed by forecast derivation
and module builders. Module builders expose normalized current conditions and
weather story context when those sources are available.
## Inputs And Outputs
@@ -21,8 +21,8 @@ Outputs:
- `weatherdata.Bundle` with observation, current conditions, hourly forecast,
narrative forecast, active alerts, discussion, latest weather story, source
records, source warnings, and an optional typed SPC convective outlook field
when that source has been populated
records, source warnings, and typed SPC convective outlook data when that
optional source is available
- optional saved bundle JSON through app fetch helpers
## Boundaries
@@ -55,7 +55,9 @@ contract used by this project.
The adapter records source name, endpoint, query, fetch time, source timestamps
when available, SHA-256 hash over compact raw `data` JSON, missing status, and
source warnings. Successful `data: null` responses from `/alerts/active`
represent a checked empty active-alert list, not a missing source.
represent a checked empty active-alert list, not a missing source. Successful
non-null `/outlooks/convective` responses with empty outlook and discussion
arrays represent checked empty outlook data.
`app.FetchAndSaveBundle` can write bundle JSON atomically for inspection.
## Skip And Resume Behavior
@@ -73,6 +75,8 @@ data is required and cannot be skipped.
- Optional sources follow missing-source policy.
- Explicit `data: null` from `/alerts/active` produces an empty, non-missing
alert run.
- Explicit `data: null` from `/outlooks/convective` follows optional
missing-source policy.
## Tests

31
internal/adapters/weatherapi/client.go
View File

@@ -21,6 +21,11 @@ import (
"gitea.maximumdirect.net/eric/weatherreporter/internal/weatherdata"
)
const (
convectiveOutlooksEndpoint = "/outlooks/convective"
sourceSPCConvectiveOutlooks = "spc_convective_outlooks"
)
type Client struct {
baseURL *url.URL
httpClient *http.Client
@@ -112,6 +117,9 @@ func (c *Client) FetchBundle(ctx context.Context) (*weatherdata.Bundle, error) {
if err := builder.fetchWeatherStory(ctx); err != nil {
return nil, err
}
if err := builder.fetchSPCConvectiveOutlooks(ctx); err != nil {
return nil, err
}
return builder.bundle, nil
}
@@ -264,6 +272,29 @@ func (b *bundleBuilder) fetchWeatherStory(ctx context.Context) error {
return nil
}
func (b *bundleBuilder) fetchSPCConvectiveOutlooks(ctx context.Context) error {
raw, source, err := b.client.fetch(ctx, sourceSPCConvectiveOutlooks, convectiveOutlooksEndpoint, queryOptions{timezone: true, omitUnits: true})
if err != nil {
return err
}
if raw == nil {
return b.handleMissing(&source, "SPC convective outlook data is missing", false)
}
var run weatherdata.ConvectiveOutlookRun
if err := decodeSource(raw, &run); err != nil {
return b.handleMalformed(&source, err, false)
}
if run.IssuedAt != nil {
source.IssuedAt = run.IssuedAt
} else {
source.IssuedAt = run.AsOf
}
source.UpdatedAt = run.UpdatedAt
b.bundle.SPCConvectiveOutlooks = &run
b.addSource(source)
return nil
}
func (b *bundleBuilder) handleMissing(source *weatherdata.Source, message string, required bool) error {
source.Missing = true
if required {

109
internal/adapters/weatherapi/client_test.go
View File

@@ -55,8 +55,17 @@ func TestFetchBundleFromFixtures(t *testing.T) {
if bundle.WeatherStory.UpdatedAt == nil {
t.Fatalf("WeatherStory.UpdatedAt = nil, want update timestamp")
}
if len(bundle.Sources) != 7 {
t.Fatalf("Sources length = %d, want 7", len(bundle.Sources))
if bundle.SPCConvectiveOutlooks == nil || len(bundle.SPCConvectiveOutlooks.Outlooks) != 1 {
t.Fatalf("SPCConvectiveOutlooks = %#v, want one outlook", bundle.SPCConvectiveOutlooks)
}
if len(bundle.SPCConvectiveOutlooks.Outlooks[0].Geometry) == 0 {
t.Fatalf("SPCConvectiveOutlooks.Outlooks[0].Geometry is empty, want GeoJSON")
}
if len(bundle.SPCConvectiveOutlooks.Discussions) != 1 || bundle.SPCConvectiveOutlooks.Discussions[0].Headline != "Severe storms possible" {
t.Fatalf("SPCConvectiveOutlooks.Discussions = %#v, want one discussion", bundle.SPCConvectiveOutlooks.Discussions)
}
if len(bundle.Sources) != 8 {
t.Fatalf("Sources length = %d, want 8", len(bundle.Sources))
}
if len(bundle.Warnings) != 0 {
t.Fatalf("Warnings length = %d, want no warnings", len(bundle.Warnings))
@@ -70,6 +79,9 @@ func TestFetchBundleFromFixtures(t *testing.T) {
if !containsPath(requested, "/weatherstories/latest") {
t.Fatalf("requested paths = %v, want weather story endpoint", requested)
}
if !containsPath(requested, convectiveOutlooksEndpoint) {
t.Fatalf("requested paths = %v, want convective outlook endpoint", requested)
}
}
func TestFetchBundleBuildsExpectedQueries(t *testing.T) {
@@ -92,6 +104,12 @@ func TestFetchBundleBuildsExpectedQueries(t *testing.T) {
}
continue
}
if strings.HasPrefix(rawURL, convectiveOutlooksEndpoint) {
if strings.Contains(rawURL, "units=") || strings.Contains(rawURL, "precision=") || !strings.Contains(rawURL, "tz=America%2FChicago") {
t.Fatalf("convective outlook request %q should use format and tz only", rawURL)
}
continue
}
if !strings.Contains(rawURL, "units=us") {
t.Fatalf("request %q missing units=us", rawURL)
}
@@ -127,6 +145,25 @@ func TestFetchBundleRecordsSourceHash(t *testing.T) {
if story.IssuedAt == nil || story.UpdatedAt == nil {
t.Fatalf("weather story source timestamps = issued %#v updated %#v, want both", story.IssuedAt, story.UpdatedAt)
}
outlooks := sourceByName(t, bundle.Sources, sourceSPCConvectiveOutlooks)
if outlooks.Endpoint != convectiveOutlooksEndpoint {
t.Fatalf("convective outlook endpoint = %q, want %s", outlooks.Endpoint, convectiveOutlooksEndpoint)
}
if outlooks.Query["format"] != "json" || outlooks.Query["tz"] != "America/Chicago" || outlooks.Query["units"] != "" || outlooks.Query["precision"] != "" {
t.Fatalf("convective outlook query = %#v, want format and tz only", outlooks.Query)
}
if outlooks.DataSHA256 != hashFixtureData(t, "convective_outlooks.json") {
t.Fatalf("convective outlook DataSHA256 = %q, want fixture hash", outlooks.DataSHA256)
}
if outlooks.Missing {
t.Fatal("convective outlook source Missing = true, want false")
}
if outlooks.IssuedAt == nil || outlooks.IssuedAt.Format(time.RFC3339) != "2026-05-29T15:45:00Z" {
t.Fatalf("convective outlook IssuedAt = %#v, want run issuedAt", outlooks.IssuedAt)
}
if outlooks.UpdatedAt == nil || outlooks.UpdatedAt.Format(time.RFC3339) != "2026-05-29T16:05:00Z" {
t.Fatalf("convective outlook UpdatedAt = %#v, want run updatedAt", outlooks.UpdatedAt)
}
}
func TestHTTPErrorIsActionable(t *testing.T) {
@@ -189,6 +226,59 @@ func TestNullAlertsMeansNoActiveAlerts(t *testing.T) {
}
}
func TestMissingSPCConvectiveOutlooksUsesPolicy(t *testing.T) {
server := fixtureServer(t, map[string]handlerOverride{
convectiveOutlooksEndpoint: {status: http.StatusOK, body: `{"data": null}`},
}, nil)
client := newTestClient(t, server.URL+"/", map[string]config.MissingSourcePolicy{
sourceSPCConvectiveOutlooks: config.MissingSourceWarn,
})
bundle, err := client.FetchBundle(context.Background())
if err != nil {
t.Fatalf("FetchBundle() error = %v", err)
}
if bundle.SPCConvectiveOutlooks != nil {
t.Fatalf("SPCConvectiveOutlooks = %#v, want nil for missing source", bundle.SPCConvectiveOutlooks)
}
source := sourceByName(t, bundle.Sources, sourceSPCConvectiveOutlooks)
if !source.Missing || len(source.Warnings) != 1 {
t.Fatalf("convective outlook source = %#v, want missing source warning", source)
}
}
func TestEmptySPCConvectiveOutlooksAreCheckedData(t *testing.T) {
server := fixtureServer(t, map[string]handlerOverride{
convectiveOutlooksEndpoint: {status: http.StatusOK, body: `{"data":{"asOf":"2026-05-29T16:00:00Z","outlooks":[],"discussions":[]}}`},
}, nil)
client := newTestClient(t, server.URL+"/", map[string]config.MissingSourcePolicy{
sourceSPCConvectiveOutlooks: config.MissingSourceWarn,
})
bundle, err := client.FetchBundle(context.Background())
if err != nil {
t.Fatalf("FetchBundle() error = %v", err)
}
if bundle.SPCConvectiveOutlooks == nil {
t.Fatal("SPCConvectiveOutlooks = nil, want checked empty run")
}
if len(bundle.SPCConvectiveOutlooks.Outlooks) != 0 || len(bundle.SPCConvectiveOutlooks.Discussions) != 0 {
t.Fatalf("SPCConvectiveOutlooks = %#v, want empty arrays", bundle.SPCConvectiveOutlooks)
}
source := sourceByName(t, bundle.Sources, sourceSPCConvectiveOutlooks)
if source.Missing || len(source.Warnings) != 0 {
t.Fatalf("convective outlook source = %#v, want non-missing source without warnings", source)
}
if source.IssuedAt == nil || source.IssuedAt.Format(time.RFC3339) != "2026-05-29T16:00:00Z" {
t.Fatalf("convective outlook IssuedAt = %#v, want fallback to asOf", source.IssuedAt)
}
for _, warning := range bundle.Warnings {
if warning.Source == sourceSPCConvectiveOutlooks {
t.Fatalf("warnings = %#v, want no convective outlook warning", bundle.Warnings)
}
}
}
func TestMissingSourcePolicyWarnNoneError(t *testing.T) {
tests := []struct {
name string
@@ -363,13 +453,14 @@ type handlerOverride struct {
func fixtureServer(t *testing.T, overrides map[string]handlerOverride, requested *[]string) *httptest.Server {
t.Helper()
fixtures := map[string]string{
"/observations": "observations.json",
"/conditions/current": "current.json",
"/forecast/hourly": "hourly.json",
"/forecast/narrative": "narrative.json",
"/alerts/active": "alerts.json",
"/discussion": "discussion.json",
"/weatherstories/latest": "weather_story.json",
"/observations": "observations.json",
"/conditions/current": "current.json",
"/forecast/hourly": "hourly.json",
"/forecast/narrative": "narrative.json",
"/alerts/active": "alerts.json",
"/discussion": "discussion.json",
"/weatherstories/latest": "weather_story.json",
convectiveOutlooksEndpoint: "convective_outlooks.json",
}
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if requested != nil {

50
internal/adapters/weatherapi/testdata/convective_outlooks.json vendored Normal file
View File

@@ -0,0 +1,50 @@
{
"data": {
"locationId": "nws-lsx-grid-90-74",
"locationName": "St. Louis, MO",
"asOf": "2026-05-29T16:00:00Z",
"issuedAt": "2026-05-29T15:45:00Z",
"updatedAt": "2026-05-29T16:05:00Z",
"product": "convective_outlook",
"outlooks": [
{
"id": "day1-categorical-slight",
"provider": "spc",
"product": "convective_outlook",
"day": 1,
"outlookType": "categorical",
"label": "SLGT",
"labelText": "Slight Risk",
"forecaster": "Smith",
"severityRank": 3,
"validFrom": "2026-05-29T13:00:00-05:00",
"validTo": "2026-05-30T07:00:00-05:00",
"issuedAt": "2026-05-29T15:45:00Z",
"expiresAt": "2026-05-30T07:00:00-05:00",
"sourceUrl": "https://www.spc.noaa.gov/products/outlook/day1otlk.html",
"imageUrl": "https://www.spc.noaa.gov/products/outlook/day1probotlk_2000_torn.gif",
"containsLocation": true,
"geometry": {
"type": "Polygon",
"coordinates": [
[
[-91.0, 38.0],
[-90.0, 38.5],
[-89.5, 37.8],
[-91.0, 38.0]
]
]
}
}
],
"discussions": [
{
"day": 1,
"headline": "Severe storms possible",
"summary": "Scattered severe storms are possible.",
"discussion": "A few storms may become severe during the afternoon.",
"updatedAt": "2026-05-29T16:05:00Z"
}
]
}
}

4
internal/app/app_test.go
View File

@@ -39,6 +39,8 @@ func TestFetchAndSaveBundle(t *testing.T) {
_, _ = w.Write([]byte(`{"data":{"product":"discussion","issuedAt":"2026-05-29T09:25:00-05:00","keyMessages":[],"shortTerm":{"qualifier":"(Short Term)","text":"Short-term AFD narrative for saved bundle."},"longTerm":{"qualifier":"(Long Term)","text":"Long-term AFD narrative for saved bundle."}}}`))
case "/weatherstories/latest":
_, _ = w.Write([]byte(`{"data":{"officeId":"LSX","startTime":"2026-05-30T08:46:00Z","endTime":"2026-05-31T11:00:00Z","updatedAt":"2026-05-30T09:00:34Z","title":"Several Chances for Rain Through Monday","description":"Scattered showers and thunderstorms remain possible.","altText":"Forecast weather story graphic.","priority":false,"order":1,"downloadUrl":"https://api.weather.gov/offices/LSX/weatherstories/download/test"}}`))
case "/outlooks/convective":
_, _ = w.Write([]byte(`{"data":{"asOf":"2026-05-29T16:00:00Z","outlooks":[],"discussions":[]}}`))
default:
http.NotFound(w, r)
}
@@ -1134,6 +1136,8 @@ func dailyBundleServer(t *testing.T) *httptest.Server {
_, _ = w.Write([]byte(`{"data":{"product":"discussion","issuedAt":"2026-05-29T09:25:00-05:00","keyMessages":["Storms are most likely during the morning."],"shortTerm":{"qualifier":"(Short Term)","text":"Short-term AFD narrative for generated report."},"longTerm":{"qualifier":"(Long Term)","text":"Long-term AFD narrative for generated report."}}}`))
case "/weatherstories/latest":
_, _ = w.Write([]byte(`{"data":{"officeId":"LSX","startTime":"2026-05-30T08:46:00Z","endTime":"2026-05-31T11:00:00Z","updatedAt":"2026-05-30T09:00:34Z","title":"Several Chances for Rain Through Monday","description":"Scattered showers and thunderstorms remain possible.","altText":"Forecast weather story graphic.","priority":false,"order":1,"downloadUrl":"https://api.weather.gov/offices/LSX/weatherstories/download/test"}}`))
case "/outlooks/convective":
_, _ = w.Write([]byte(`{"data":{"asOf":"2026-05-29T16:00:00Z","outlooks":[],"discussions":[]}}`))
default:
http.NotFound(w, r)
}

2
internal/cli/root_test.go
View File

@@ -996,6 +996,8 @@ func dailyServer(t *testing.T) *httptest.Server {
_, _ = w.Write([]byte(`{"data":{"product":"discussion","issuedAt":"2026-05-29T09:25:00-05:00","keyMessages":["Storms are most likely during the morning."]}}`))
case "/weatherstories/latest":
_, _ = w.Write([]byte(`{"data":{"officeId":"LSX","startTime":"2026-05-30T08:46:00Z","endTime":"2026-05-31T11:00:00Z","updatedAt":"2026-05-30T09:00:34Z","title":"Several Chances for Rain Through Monday","description":"Scattered showers and thunderstorms remain possible.","altText":"Forecast weather story graphic.","priority":false,"order":1,"downloadUrl":"https://api.weather.gov/offices/LSX/weatherstories/download/test"}}`))
case "/outlooks/convective":
_, _ = w.Write([]byte(`{"data":{"asOf":"2026-05-29T16:00:00Z","outlooks":[],"discussions":[]}}`))
default:
http.NotFound(w, r)
}