weatherapi/docs/roadmap/implementation.md

Implement Weatherfeeder Outlook V2 Support

Summary

Implement weatherapi support for the weatherfeeder SPC outlook v2 contract described in docs/roadmap/outlook.md.

This is a compatibility update for the existing convective outlook route family. Do not add or remove public routes. Update dependency, repository reads, app filtering, presenters, templates, tests, and current-behavior docs so weatherapi reads weather.outlook.v2 data from the new weatherfeeder Postgres table shape.

Required End State

• go.mod depends on the released weatherfeeder version containing weather.outlook.v2 support.
• Existing public routes remain available:
  • GET /outlooks/convective
  • GET /outlooks/convective/active
  • GET /outlooks/convective/location
• weatherapi remains read-only and does not create, migrate, or repair weatherfeeder tables.
• The Postgres repository reads outlook_runs, outlooks, and outlook_discussions.
• The repository no longer expects polygon-level headline, summary, or discussion columns on outlooks.
• Returned WeatherOutlookRun values include run-level Discussions loaded from outlook_discussions.
• App-level filtering trims both Outlooks and Discussions so discussions are present only for days with retained outlooks.
• Presenters copy and timezone-convert run-level discussion updatedAt values.
• JSON/XML/text responses expose run-level discussions and do not expose polygon-level outlook prose fields.
• data: null semantics for no latest run are preserved.
• Filtered no-match responses return a non-null run with outlooks: [] and discussions: [].
• Public query behavior remains compatible: format, units, tz / TZ, day, outlookType, and allowed containsLocation behavior are preserved; precision and unknown params remain rejected.

Guardrails

• Do not implement weatherfeeder table creation or migration in weatherapi.
• Do not call SPC or any upstream weather provider from weatherapi.
• Do not introduce historical browsing or cross-run accumulation in this change.
• Do not remove /outlooks/convective/location; keep it as an API-compatible active local-outlook route.
• Do not keep stale v1 SQL reads for outlooks.headline, outlooks.summary, or outlooks.discussion.
• Do not commit a replace directive for weatherfeeder.
• Keep SQL in *_queries.go, row DTOs in *_rows.go, row mapping in *_mapper.go, and request/presentation policy in HTTP/presenter packages.

Stage 1: Dependency And Compile Boundary

Changes

1. Update go.mod to the released gitea.maximumdirect.net/ejr/weatherfeeder version that includes:
   • standards.SchemaWeatherOutlookV2;
   • model.WeatherOutlookRun.Discussions;
   • model.WeatherOutlookDiscussion;
   • model.WeatherOutlook without Headline, Summary, or Discussion.
2. Run go mod tidy.
3. Fix compile errors from removed polygon-level outlook prose fields.
4. Search for outlook v1 and polygon prose references:

rg "SchemaWeatherOutlookV1|weather\.outlook\.v1|\.Headline|\.Summary|\.Discussion|headline|summary|discussion" internal docs templates README.md

5. Treat matches carefully:
   • Keep unrelated alert and forecast-discussion uses.
   • Remove or update outlook polygon-level prose references.
   • Replace outlook-specific schema references with v2 where applicable.

Expected Files

• go.mod
• go.sum
• internal/adapters/outbound/postgres/outlooks_rows.go
• internal/adapters/outbound/postgres/outlooks_mapper.go
• internal/adapters/outbound/postgres/outlooks_queries.go
• internal/adapters/outbound/postgres/outlooks_read.go
• internal/adapters/inbound/httpapi/presenter/outlook.go
• internal/app/service.go
• outlook-related tests under internal/app, internal/adapters/outbound/postgres, internal/adapters/inbound/httpapi, and presenter tests

Verification

At this stage, run a compile-oriented focused set. It may fail until later stages are complete, but failures should identify remaining v1/prose references.

go test ./internal/app ./internal/adapters/outbound/postgres ./internal/adapters/inbound/httpapi/presenter

Stage 2: Postgres Query, Row, And Mapper Updates

Query Changes

Update internal/adapters/outbound/postgres/outlooks_queries.go.

Parent query:

• Keep latest run ordering:

ORDER BY as_of DESC, event_emitted_at DESC
LIMIT 1

• Continue selecting parent fields needed by WeatherOutlookRun.
• Optionally select discussion_count for test/sanity visibility, but do not expose it in the API response model.

Outlook child query:

• Remove these v1 columns from the SELECT list:
  • headline
  • summary
  • discussion
• Continue selecting:
  • outlook_index
  • outlook_id
  • provider
  • product
  • day
  • outlook_type
  • label
  • label_text
  • severity_rank
  • valid_from
  • valid_to
  • issued_at
  • expires_at
  • forecaster
  • source_url
  • image_url
  • contains_location
  • geometry_json
• Keep WHERE run_event_id = $1.
• Keep ORDER BY outlook_index ASC.

Discussion child query:

• Add queryOutlookDiscussionsForRun:

SELECT
  discussion_index,
  day,
  headline,
  summary,
  discussion,
  updated_at
FROM outlook_discussions
WHERE run_event_id = $1
ORDER BY discussion_index ASC

Row DTO Changes

Update internal/adapters/outbound/postgres/outlooks_rows.go:

• Remove Headline, Summary, and Discussion from outlookRow.
• Add DiscussionCount to outlookRunParentRow only if selected by parent query.
• Add outlookDiscussionRow:
  • DiscussionIndex int
  • Day int
  • Headline sql.NullString
  • Summary sql.NullString
  • Discussion sql.NullString
  • UpdatedAt sql.NullTime

Mapper Changes

Update internal/adapters/outbound/postgres/outlooks_mapper.go:

• mapOutlookRow maps only v2 polygon fields.
• Keep geometry validation with json.Valid and return contextual mapper errors for invalid JSON.
• Continue copying geometry bytes with append([]byte(nil), geometry...).
• Continue normalizing outlook timestamps to UTC.
• Add mapOutlookDiscussionRow returning model.WeatherOutlookDiscussion:
  • Day from row day;
  • string fields via existing stringValue helper;
  • UpdatedAt via existing timePtr helper, ensuring UTC normalization.

Read Flow Changes

Update internal/adapters/outbound/postgres/outlooks_read.go:

• LatestConvectiveOutlookRun loads parent row as today.
• After parent row:
  • call loadOutlooks(ctx, row.EventID);
  • call loadOutlookDiscussions(ctx, row.EventID);
  • attach both to the run.
• Add loadOutlookDiscussions mirroring loadOutlooks style:
  • query with context;
  • scan rows;
  • map rows;
  • return iteration errors;
  • wrap query, scan, map, and iteration errors with operation context.
• Preserve nil, nil on missing parent row.

Tests

Update or add tests in internal/adapters/outbound/postgres:

• mapOutlookRow maps v2 polygon fields and no longer expects polygon-level prose.
• mapOutlookRow rejects invalid geometry_json.
• nullable label/forecaster/source/image/severity fields map correctly.
• mapOutlookDiscussionRow maps day/headline/summary/discussion/updatedAt and normalizes time to UTC.
• nullable discussion fields map to empty/omitted canonical values.
• latest run read loads outlooks in outlook_index ASC order.
• latest run read loads discussions in discussion_index ASC order.
• missing latest parent returns nil, nil.
• query/scan/iteration failures remain context-wrapped.

Verification

go test ./internal/adapters/outbound/postgres

Stage 3: Application Filtering And Copy Semantics

Changes

Update internal/app/service.go.

Clone behavior:

• cloneOutlookRun must deep-copy Discussions in addition to Outlooks.
• Ensure outlook severity pointers, geometry bytes, latitude/longitude pointers, and issuedAt pointers remain copied.
• Add a helper such as cloneOutlookDiscussion if useful.

Filtering behavior:

• Filter Outlooks as today using OutlookFilter.
• After filtering outlooks, filter Discussions to only days represented by retained outlooks.
• Preserve discussion order from the repository result.
• If no outlooks remain, set discussions to an empty slice, not stale unfiltered discussions.
• Preserve nil/missing run behavior: repository nil, nil still returns nil, nil.

Recommended helper shape:

func filterOutlookDiscussions(discussions []model.WeatherOutlookDiscussion, outlooks []model.WeatherOutlook) []model.WeatherOutlookDiscussion

Rules:

• Build a set of retained outlook days.
• If the set is empty, return []model.WeatherOutlookDiscussion{} when discussions was non-nil or when consistent empty-array output is desired.
• Include each discussion only if discussion.Day is in the set.
• Do not synthesize discussions.

Query Compatibility

Do not change query binder behavior in this stage unless tests reveal compile fallout from the model update. Public query behavior remains:

• day accepted on all outlook routes.
• outlookType accepted on all outlook routes.
• containsLocation accepted on /outlooks/convective and /outlooks/convective/active.
• containsLocation rejected on /outlooks/convective/location.
• format, units, and tz / TZ accepted.
• precision and unknown params rejected.

Tests

Update internal/app/service_test.go:

• Delegation test still asserts one repository call.
• Day filter retains matching outlooks and only matching day discussions.
• Outlook type filter retains discussions only for days with retained outlooks.
• Active filter retains discussions only for days with active retained outlooks.
• containsLocation=false returns an empty outlook/discussion run for v2-style test data.
• Filtered no-match result has empty Outlooks and empty Discussions.
• Mutating returned outlooks/discussions does not mutate repository-owned data.
• Geometry byte copy and pointer copy behavior remains covered.

Verification

go test ./internal/app

Stage 4: Presenter And Text Template Updates

Presenter Changes

Update internal/adapters/inbound/httpapi/presenter/outlook.go:

• Include Discussions when constructing the presented model.WeatherOutlookRun.
• Convert WeatherOutlookDiscussion.UpdatedAt using inLocationTimePtr.
• Deep-copy discussions so presenter output mutation cannot affect repository/app input.
• Remove Headline, Summary, and Discussion assignments from copyOutlook.
• Continue converting:
  • run AsOf;
  • run IssuedAt;
  • outlook ValidFrom;
  • outlook ValidTo;
  • outlook IssuedAt;
  • outlook ExpiresAt.
• Continue preserving geometry copy behavior.
• Continue returning nil for nil input.

Recommended helper:

func copyOutlookDiscussion(d model.WeatherOutlookDiscussion, tz *time.Location) model.WeatherOutlookDiscussion

Template Changes

Update templates/outlooks_convective.txt.tmpl:

• Render run-level discussions from .Data.Discussions.
• Do not reference outlook-level .Headline, .Summary, or .Discussion.
• Keep no-data output when .Data is nil.
• Keep useful output when .Data exists and .Data.Outlooks is empty.
• Include discussion day/headline/summary/discussion fields in a readable, conditional way.
• Avoid printing zero timestamps or <nil> values for absent updatedAt.

Presenter Tests

Update internal/adapters/inbound/httpapi/presenter/payload_test.go:

• OutlookRunPayload(nil, ...) returns nil.
• Output includes copied Discussions.
• Discussion UpdatedAt is converted to requested timezone.
• Mutating output discussions does not mutate input discussions.
• Mutating output geometry does not mutate input geometry.
• Outlook timestamps still convert to requested timezone.
• Polygon-level prose fields are not expected in output structs/tests.

Verification

go test ./internal/adapters/inbound/httpapi/presenter

Stage 5: HTTP Endpoint Behavior And Tests

Handler Behavior

Keep route registration and handler structure unchanged unless compile updates require mechanical edits.

Ensure endpoint responses reflect the updated presenter/app behavior:

• /outlooks/convective returns latest run with v2 shape.
• /outlooks/convective/active applies active filter and trims discussions.
• /outlooks/convective/location applies active local filter and trims discussions.
• data: null remains unchanged when no run exists.
• Filtered no-match response returns data object with empty outlook and discussion arrays.

Endpoint Tests

Update internal/adapters/inbound/httpapi/endpoints_test.go:

• test fakes build model.WeatherOutlookRun with Discussions.
• JSON success asserts run-level discussions are present.
• JSON success asserts outlook polygons do not include polygon-level prose.
• XML success renders run-level discussions without errors.
• Text success renders run-level discussion content.
• Timezone test checks conversion of:
  • run asOf;
  • run issuedAt;
  • outlook times;
  • discussion updatedAt.
• Null data test remains unchanged.
• Filtered no-match test expects both outlooks and discussions empty.
• Query acceptance still covers units, format, tz / TZ, day, outlookType, and allowed containsLocation.
• Query rejection still covers precision, unknown params, invalid day, invalid outlookType, invalid containsLocation, invalid timezone, conflicting tz / TZ, and containsLocation on /location.
• Active/location route tests continue using injectable outlookNow.

Verification

go test ./internal/adapters/inbound/httpapi

Stage 6: Documentation Updates

After code behavior is updated, update current-behavior documentation. Do not leave v2 behavior only in roadmap files.

Public Docs

Update docs/api.md:

• State outlook endpoints serve weatherfeeder weather.outlook.v2 data.
• Run fields include discussions.
• Add discussion field table:
  • day;
  • headline;
  • summary;
  • discussion;
  • updatedAt.
• Remove polygon-level headline, summary, and discussion from outlook fields.
• State v2 outlooks are already location-filtered by weatherfeeder.
• State containsLocation is expected to be true for v2 outlooks.
• Explain /outlooks/convective/location remains an active local-outlook compatibility route under v2.
• State endpoint filters also filter discussions to days represented by retained outlooks.
• State current endpoints use latest-run semantics and do not accumulate historical active outlook rows.
• Update examples to show run-level discussions.
• Ensure examples use valid GeoJSON polygon or multipolygon, not a point, if demonstrating SPC geometry.

Update README.md only if its overview implies all-polygons or old response shape.

Integration/Internal Docs

Update docs/integrations/weatherfeeder-postgres.md:

• Update dependency version.
• Table family for convective outlook includes outlook_runs, outlooks, and outlook_discussions.
• Latest run selection remains as_of DESC, event_emitted_at DESC.
• Child ordering includes:
  • outlooks by outlook_index ASC;
  • outlook discussions by discussion_index ASC.
• outlook_runs includes discussion_count if listed.
• outlooks no longer includes headline, summary, or discussion.
• Add outlook_discussions columns.
• State weatherapi assumes weatherfeeder's outlook v2 table reset has already been applied.

Update docs/internal/postgres-repository.md:

• LatestConvectiveOutlookRun loads parent, outlook children, and discussion children.
• Add outlook discussion child ordering.

Update docs/internal/presenters.md:

• Outlook presenter copies and timezone-converts run-level discussions.
• Outlook presenter no longer handles polygon-level prose.

Update docs/internal/http-adapter.md:

• Outlook filters trim discussions to retained outlook days.
• /location is retained for compatibility and active local-outlook behavior.

Update docs/operations.md and docs/troubleshooting.md only if they mention the old weatherfeeder table shape or should warn operators that weatherapi requires weatherfeeder v2 outlook tables.

Roadmap Docs

After implementation is complete, replace or update docs/roadmap/outlook.md and docs/roadmap/implementation.md according to repository convention:

• Either mark the roadmap complete and point to current docs.
• Or move any remaining deferred ideas to a future roadmap file.

Do not leave roadmap files claiming unimplemented work after the implementation has shipped.

Documentation Checks

Run searches for stale outlook v1/current behavior:

rg "weather\.outlook\.v1|outlooks\.headline|outlooks\.summary|outlooks\.discussion|polygon-level|all stored outlook|all polygons|headline, summary, discussion" docs README.md internal templates

Allowed matches:

• explicit historical/legacy notes, if any;
• unrelated alert or forecast discussion prose fields;
• roadmap files that are intentionally historical.

Stage 7: Full Verification

Run focused tests:

go test ./internal/app
go test ./internal/adapters/outbound/postgres
go test ./internal/adapters/inbound/httpapi/presenter
go test ./internal/adapters/inbound/httpapi

Run the full suite:

go test ./...

Optional build check:

go build ./cmd/weatherapi

Manual verification against a database populated by weatherfeeder outlook v2 tables:

GET /outlooks/convective
GET /outlooks/convective/active
GET /outlooks/convective/location
GET /outlooks/convective?day=1
GET /outlooks/convective?outlookType=tornado
GET /outlooks/convective?containsLocation=false
GET /outlooks/convective?format=text&tz=America/Chicago

Expected manual results:

• Latest run includes run-level discussions.
• Active route filters outlooks and discussions together.
• Location route returns active local outlooks and remains valid.
• containsLocation=false returns an empty run for v2 data.
• Text format renders discussions without template errors.
• XML format renders without errors.
• No endpoint queries historical active rows across older runs.

Assumptions

• A weatherfeeder release containing outlook v2 exists before final implementation is committed.
• Operators have reset/recreated weatherfeeder outlook tables according to weatherfeeder transition docs.
• Existing weatherapi route names remain stable for external consumers.
• Keeping containsLocation query support on non-location routes is useful backward compatibility even though v2 data should always be local.
• Latest-run semantics are correct for current convective outlook endpoints.

Open Questions

None. This plan fixes the route, dependency, storage, filtering, presentation, and documentation decisions required to support weatherfeeder outlook v2 while preserving weatherapi's existing public route family.