ejr/weatherapi
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add roadmap and implementation plan to support the new outlook schema
Some checks failed
ci/woodpecker/push/build-image Pipeline failed
Details

Browse Source
This commit is contained in:
Eric Rakestraw
2026-06-12 07:47:01 -05:00
parent b4149f00c5
commit 0135eb1153
2 changed files with 831 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

507
docs/roadmap/implementation.md
View File

@@ -1,14 +1,501 @@
# SPC Convective Outlook API Roadmap
# Implement Weatherfeeder Outlook V2 Support
There are no active roadmap items for the SPC convective outlook API.
## Summary
Current behavior is documented in:
Implement `weatherapi` support for the `weatherfeeder` SPC outlook v2 contract described in `docs/roadmap/outlook.md`.
- [`docs/api.md`](../api.md)
- [`docs/internal/http-adapter.md`](../internal/http-adapter.md)
- [`docs/internal/presenters.md`](../internal/presenters.md)
- [`docs/internal/postgres-repository.md`](../internal/postgres-repository.md)
- [`docs/integrations/weatherfeeder-postgres.md`](../integrations/weatherfeeder-postgres.md)
This is a compatibility update for the existing convective outlook route family. Do not add or remove public routes. Update dependency, repository reads, app filtering, presenters, templates, tests, and current-behavior docs so `weatherapi` reads `weather.outlook.v2` data from the new weatherfeeder Postgres table shape.
Future changes to convective outlook behavior should be proposed in a new
roadmap entry before implementation.
## Required End State
- `go.mod` depends on the released `weatherfeeder` version containing `weather.outlook.v2` support.
- Existing public routes remain available:
- `GET /outlooks/convective`
- `GET /outlooks/convective/active`
- `GET /outlooks/convective/location`
- `weatherapi` remains read-only and does not create, migrate, or repair weatherfeeder tables.
- The Postgres repository reads `outlook_runs`, `outlooks`, and `outlook_discussions`.
- The repository no longer expects polygon-level `headline`, `summary`, or `discussion` columns on `outlooks`.
- Returned `WeatherOutlookRun` values include run-level `Discussions` loaded from `outlook_discussions`.
- App-level filtering trims both `Outlooks` and `Discussions` so discussions are present only for days with retained outlooks.
- Presenters copy and timezone-convert run-level discussion `updatedAt` values.
- JSON/XML/text responses expose run-level `discussions` and do not expose polygon-level outlook prose fields.
- `data: null` semantics for no latest run are preserved.
- Filtered no-match responses return a non-null run with `outlooks: []` and `discussions: []`.
- Public query behavior remains compatible: `format`, `units`, `tz` / `TZ`, `day`, `outlookType`, and allowed `containsLocation` behavior are preserved; `precision` and unknown params remain rejected.
## Guardrails
- Do not implement weatherfeeder table creation or migration in weatherapi.
- Do not call SPC or any upstream weather provider from weatherapi.
- Do not introduce historical browsing or cross-run accumulation in this change.
- Do not remove `/outlooks/convective/location`; keep it as an API-compatible active local-outlook route.
- Do not keep stale v1 SQL reads for `outlooks.headline`, `outlooks.summary`, or `outlooks.discussion`.
- Do not commit a `replace` directive for `weatherfeeder`.
- Keep SQL in `*_queries.go`, row DTOs in `*_rows.go`, row mapping in `*_mapper.go`, and request/presentation policy in HTTP/presenter packages.
## Stage 1: Dependency And Compile Boundary
### Changes
1. Update `go.mod` to the released `gitea.maximumdirect.net/ejr/weatherfeeder` version that includes:
- `standards.SchemaWeatherOutlookV2`;
- `model.WeatherOutlookRun.Discussions`;
- `model.WeatherOutlookDiscussion`;
- `model.WeatherOutlook` without `Headline`, `Summary`, or `Discussion`.
2. Run `go mod tidy`.
3. Fix compile errors from removed polygon-level outlook prose fields.
4. Search for outlook v1 and polygon prose references:
```sh
rg "SchemaWeatherOutlookV1|weather\.outlook\.v1|\.Headline|\.Summary|\.Discussion|headline|summary|discussion" internal docs templates README.md
```
5. Treat matches carefully:
- Keep unrelated alert and forecast-discussion uses.
- Remove or update outlook polygon-level prose references.
- Replace outlook-specific schema references with v2 where applicable.
### Expected Files
- `go.mod`
- `go.sum`
- `internal/adapters/outbound/postgres/outlooks_rows.go`
- `internal/adapters/outbound/postgres/outlooks_mapper.go`
- `internal/adapters/outbound/postgres/outlooks_queries.go`
- `internal/adapters/outbound/postgres/outlooks_read.go`
- `internal/adapters/inbound/httpapi/presenter/outlook.go`
- `internal/app/service.go`
- outlook-related tests under `internal/app`, `internal/adapters/outbound/postgres`, `internal/adapters/inbound/httpapi`, and presenter tests
### Verification
At this stage, run a compile-oriented focused set. It may fail until later stages are complete, but failures should identify remaining v1/prose references.
```sh
go test ./internal/app ./internal/adapters/outbound/postgres ./internal/adapters/inbound/httpapi/presenter
```
## Stage 2: Postgres Query, Row, And Mapper Updates
### Query Changes
Update `internal/adapters/outbound/postgres/outlooks_queries.go`.
Parent query:
- Keep latest run ordering:
```sql
ORDER BY as_of DESC, event_emitted_at DESC
LIMIT 1
```
- Continue selecting parent fields needed by `WeatherOutlookRun`.
- Optionally select `discussion_count` for test/sanity visibility, but do not expose it in the API response model.
Outlook child query:
- Remove these v1 columns from the `SELECT` list:
- `headline`
- `summary`
- `discussion`
- Continue selecting:
- `outlook_index`
- `outlook_id`
- `provider`
- `product`
- `day`
- `outlook_type`
- `label`
- `label_text`
- `severity_rank`
- `valid_from`
- `valid_to`
- `issued_at`
- `expires_at`
- `forecaster`
- `source_url`
- `image_url`
- `contains_location`
- `geometry_json`
- Keep `WHERE run_event_id = $1`.
- Keep `ORDER BY outlook_index ASC`.
Discussion child query:
- Add `queryOutlookDiscussionsForRun`:
```sql
SELECT
discussion_index,
day,
headline,
summary,
discussion,
updated_at
FROM outlook_discussions
WHERE run_event_id = $1
ORDER BY discussion_index ASC
```
### Row DTO Changes
Update `internal/adapters/outbound/postgres/outlooks_rows.go`:
- Remove `Headline`, `Summary`, and `Discussion` from `outlookRow`.
- Add `DiscussionCount` to `outlookRunParentRow` only if selected by parent query.
- Add `outlookDiscussionRow`:
- `DiscussionIndex int`
- `Day int`
- `Headline sql.NullString`
- `Summary sql.NullString`
- `Discussion sql.NullString`
- `UpdatedAt sql.NullTime`
### Mapper Changes
Update `internal/adapters/outbound/postgres/outlooks_mapper.go`:
- `mapOutlookRow` maps only v2 polygon fields.
- Keep geometry validation with `json.Valid` and return contextual mapper errors for invalid JSON.
- Continue copying geometry bytes with `append([]byte(nil), geometry...)`.
- Continue normalizing outlook timestamps to UTC.
- Add `mapOutlookDiscussionRow` returning `model.WeatherOutlookDiscussion`:
- `Day` from row day;
- string fields via existing `stringValue` helper;
- `UpdatedAt` via existing `timePtr` helper, ensuring UTC normalization.
### Read Flow Changes
Update `internal/adapters/outbound/postgres/outlooks_read.go`:
- `LatestConvectiveOutlookRun` loads parent row as today.
- After parent row:
- call `loadOutlooks(ctx, row.EventID)`;
- call `loadOutlookDiscussions(ctx, row.EventID)`;
- attach both to the run.
- Add `loadOutlookDiscussions` mirroring `loadOutlooks` style:
- query with context;
- scan rows;
- map rows;
- return iteration errors;
- wrap query, scan, map, and iteration errors with operation context.
- Preserve `nil, nil` on missing parent row.
### Tests
Update or add tests in `internal/adapters/outbound/postgres`:
- `mapOutlookRow` maps v2 polygon fields and no longer expects polygon-level prose.
- `mapOutlookRow` rejects invalid `geometry_json`.
- nullable label/forecaster/source/image/severity fields map correctly.
- `mapOutlookDiscussionRow` maps day/headline/summary/discussion/updatedAt and normalizes time to UTC.
- nullable discussion fields map to empty/omitted canonical values.
- latest run read loads outlooks in `outlook_index ASC` order.
- latest run read loads discussions in `discussion_index ASC` order.
- missing latest parent returns `nil, nil`.
- query/scan/iteration failures remain context-wrapped.
### Verification
```sh
go test ./internal/adapters/outbound/postgres
```
## Stage 3: Application Filtering And Copy Semantics
### Changes
Update `internal/app/service.go`.
Clone behavior:
- `cloneOutlookRun` must deep-copy `Discussions` in addition to `Outlooks`.
- Ensure outlook severity pointers, geometry bytes, latitude/longitude pointers, and issuedAt pointers remain copied.
- Add a helper such as `cloneOutlookDiscussion` if useful.
Filtering behavior:
- Filter `Outlooks` as today using `OutlookFilter`.
- After filtering outlooks, filter `Discussions` to only days represented by retained outlooks.
- Preserve discussion order from the repository result.
- If no outlooks remain, set discussions to an empty slice, not stale unfiltered discussions.
- Preserve nil/missing run behavior: repository `nil, nil` still returns `nil, nil`.
Recommended helper shape:
```go
func filterOutlookDiscussions(discussions []model.WeatherOutlookDiscussion, outlooks []model.WeatherOutlook) []model.WeatherOutlookDiscussion
```
Rules:
- Build a set of retained outlook days.
- If the set is empty, return `[]model.WeatherOutlookDiscussion{}` when discussions was non-nil or when consistent empty-array output is desired.
- Include each discussion only if `discussion.Day` is in the set.
- Do not synthesize discussions.
### Query Compatibility
Do not change query binder behavior in this stage unless tests reveal compile fallout from the model update. Public query behavior remains:
- `day` accepted on all outlook routes.
- `outlookType` accepted on all outlook routes.
- `containsLocation` accepted on `/outlooks/convective` and `/outlooks/convective/active`.
- `containsLocation` rejected on `/outlooks/convective/location`.
- `format`, `units`, and `tz` / `TZ` accepted.
- `precision` and unknown params rejected.
### Tests
Update `internal/app/service_test.go`:
- Delegation test still asserts one repository call.
- Day filter retains matching outlooks and only matching day discussions.
- Outlook type filter retains discussions only for days with retained outlooks.
- Active filter retains discussions only for days with active retained outlooks.
- `containsLocation=false` returns an empty outlook/discussion run for v2-style test data.
- Filtered no-match result has empty `Outlooks` and empty `Discussions`.
- Mutating returned outlooks/discussions does not mutate repository-owned data.
- Geometry byte copy and pointer copy behavior remains covered.
### Verification
```sh
go test ./internal/app
```
## Stage 4: Presenter And Text Template Updates
### Presenter Changes
Update `internal/adapters/inbound/httpapi/presenter/outlook.go`:
- Include `Discussions` when constructing the presented `model.WeatherOutlookRun`.
- Convert `WeatherOutlookDiscussion.UpdatedAt` using `inLocationTimePtr`.
- Deep-copy discussions so presenter output mutation cannot affect repository/app input.
- Remove `Headline`, `Summary`, and `Discussion` assignments from `copyOutlook`.
- Continue converting:
- run `AsOf`;
- run `IssuedAt`;
- outlook `ValidFrom`;
- outlook `ValidTo`;
- outlook `IssuedAt`;
- outlook `ExpiresAt`.
- Continue preserving geometry copy behavior.
- Continue returning `nil` for nil input.
Recommended helper:
```go
func copyOutlookDiscussion(d model.WeatherOutlookDiscussion, tz *time.Location) model.WeatherOutlookDiscussion
```
### Template Changes
Update `templates/outlooks_convective.txt.tmpl`:
- Render run-level discussions from `.Data.Discussions`.
- Do not reference outlook-level `.Headline`, `.Summary`, or `.Discussion`.
- Keep no-data output when `.Data` is nil.
- Keep useful output when `.Data` exists and `.Data.Outlooks` is empty.
- Include discussion day/headline/summary/discussion fields in a readable, conditional way.
- Avoid printing zero timestamps or `<nil>` values for absent `updatedAt`.
### Presenter Tests
Update `internal/adapters/inbound/httpapi/presenter/payload_test.go`:
- `OutlookRunPayload(nil, ...)` returns nil.
- Output includes copied `Discussions`.
- Discussion `UpdatedAt` is converted to requested timezone.
- Mutating output discussions does not mutate input discussions.
- Mutating output geometry does not mutate input geometry.
- Outlook timestamps still convert to requested timezone.
- Polygon-level prose fields are not expected in output structs/tests.
### Verification
```sh
go test ./internal/adapters/inbound/httpapi/presenter
```
## Stage 5: HTTP Endpoint Behavior And Tests
### Handler Behavior
Keep route registration and handler structure unchanged unless compile updates require mechanical edits.
Ensure endpoint responses reflect the updated presenter/app behavior:
- `/outlooks/convective` returns latest run with v2 shape.
- `/outlooks/convective/active` applies active filter and trims discussions.
- `/outlooks/convective/location` applies active local filter and trims discussions.
- `data: null` remains unchanged when no run exists.
- Filtered no-match response returns data object with empty outlook and discussion arrays.
### Endpoint Tests
Update `internal/adapters/inbound/httpapi/endpoints_test.go`:
- test fakes build `model.WeatherOutlookRun` with `Discussions`.
- JSON success asserts run-level `discussions` are present.
- JSON success asserts outlook polygons do not include polygon-level prose.
- XML success renders run-level discussions without errors.
- Text success renders run-level discussion content.
- Timezone test checks conversion of:
- run `asOf`;
- run `issuedAt`;
- outlook times;
- discussion `updatedAt`.
- Null data test remains unchanged.
- Filtered no-match test expects both `outlooks` and `discussions` empty.
- Query acceptance still covers `units`, `format`, `tz` / `TZ`, `day`, `outlookType`, and allowed `containsLocation`.
- Query rejection still covers `precision`, unknown params, invalid day, invalid outlookType, invalid containsLocation, invalid timezone, conflicting `tz` / `TZ`, and `containsLocation` on `/location`.
- Active/location route tests continue using injectable `outlookNow`.
### Verification
```sh
go test ./internal/adapters/inbound/httpapi
```
## Stage 6: Documentation Updates
After code behavior is updated, update current-behavior documentation. Do not leave v2 behavior only in roadmap files.
### Public Docs
Update `docs/api.md`:
- State outlook endpoints serve weatherfeeder `weather.outlook.v2` data.
- Run fields include `discussions`.
- Add discussion field table:
- `day`;
- `headline`;
- `summary`;
- `discussion`;
- `updatedAt`.
- Remove polygon-level `headline`, `summary`, and `discussion` from outlook fields.
- State v2 outlooks are already location-filtered by weatherfeeder.
- State `containsLocation` is expected to be true for v2 outlooks.
- Explain `/outlooks/convective/location` remains an active local-outlook compatibility route under v2.
- State endpoint filters also filter `discussions` to days represented by retained outlooks.
- State current endpoints use latest-run semantics and do not accumulate historical active outlook rows.
- Update examples to show run-level `discussions`.
- Ensure examples use valid GeoJSON polygon or multipolygon, not a point, if demonstrating SPC geometry.
Update `README.md` only if its overview implies all-polygons or old response shape.
### Integration/Internal Docs
Update `docs/integrations/weatherfeeder-postgres.md`:
- Update dependency version.
- Table family for convective outlook includes `outlook_runs`, `outlooks`, and `outlook_discussions`.
- Latest run selection remains `as_of DESC, event_emitted_at DESC`.
- Child ordering includes:
- outlooks by `outlook_index ASC`;
- outlook discussions by `discussion_index ASC`.
- `outlook_runs` includes `discussion_count` if listed.
- `outlooks` no longer includes `headline`, `summary`, or `discussion`.
- Add `outlook_discussions` columns.
- State weatherapi assumes weatherfeeder's outlook v2 table reset has already been applied.
Update `docs/internal/postgres-repository.md`:
- `LatestConvectiveOutlookRun` loads parent, outlook children, and discussion children.
- Add outlook discussion child ordering.
Update `docs/internal/presenters.md`:
- Outlook presenter copies and timezone-converts run-level discussions.
- Outlook presenter no longer handles polygon-level prose.
Update `docs/internal/http-adapter.md`:
- Outlook filters trim discussions to retained outlook days.
- `/location` is retained for compatibility and active local-outlook behavior.
Update `docs/operations.md` and `docs/troubleshooting.md` only if they mention the old weatherfeeder table shape or should warn operators that weatherapi requires weatherfeeder v2 outlook tables.
### Roadmap Docs
After implementation is complete, replace or update `docs/roadmap/outlook.md` and `docs/roadmap/implementation.md` according to repository convention:
- Either mark the roadmap complete and point to current docs.
- Or move any remaining deferred ideas to a future roadmap file.
Do not leave roadmap files claiming unimplemented work after the implementation has shipped.
### Documentation Checks
Run searches for stale outlook v1/current behavior:
```sh
rg "weather\.outlook\.v1|outlooks\.headline|outlooks\.summary|outlooks\.discussion|polygon-level|all stored outlook|all polygons|headline, summary, discussion" docs README.md internal templates
```
Allowed matches:
- explicit historical/legacy notes, if any;
- unrelated alert or forecast discussion prose fields;
- roadmap files that are intentionally historical.
## Stage 7: Full Verification
Run focused tests:
```sh
go test ./internal/app
go test ./internal/adapters/outbound/postgres
go test ./internal/adapters/inbound/httpapi/presenter
go test ./internal/adapters/inbound/httpapi
```
Run the full suite:
```sh
go test ./...
```
Optional build check:
```sh
go build ./cmd/weatherapi
```
Manual verification against a database populated by weatherfeeder outlook v2 tables:
```http
GET /outlooks/convective
GET /outlooks/convective/active
GET /outlooks/convective/location
GET /outlooks/convective?day=1
GET /outlooks/convective?outlookType=tornado
GET /outlooks/convective?containsLocation=false
GET /outlooks/convective?format=text&tz=America/Chicago
```
Expected manual results:
- Latest run includes run-level `discussions`.
- Active route filters outlooks and discussions together.
- Location route returns active local outlooks and remains valid.
- `containsLocation=false` returns an empty run for v2 data.
- Text format renders discussions without template errors.
- XML format renders without errors.
- No endpoint queries historical active rows across older runs.
## Assumptions
- A weatherfeeder release containing outlook v2 exists before final implementation is committed.
- Operators have reset/recreated weatherfeeder outlook tables according to weatherfeeder transition docs.
- Existing weatherapi route names remain stable for external consumers.
- Keeping `containsLocation` query support on non-location routes is useful backward compatibility even though v2 data should always be local.
- Latest-run semantics are correct for current convective outlook endpoints.
## Open Questions
None. This plan fixes the route, dependency, storage, filtering, presentation, and documentation decisions required to support weatherfeeder outlook v2 while preserving weatherapi's existing public route family.