weatherapi/docs/roadmap/implementation.md

# Implement Weatherfeeder Outlook V2 Support

## Status

Implementation, current-behavior documentation, and automated verification are
complete. This file now serves as the checklist and verification record for the
outlook v2 compatibility update. Authoritative implemented behavior is
documented in `docs/api.md`, `docs/integrations/weatherfeeder-postgres.md`, and
`docs/internal/`.

## Verification Record

Completed on 2026-06-12:

- `go test ./internal/app`
- `go test ./internal/adapters/outbound/postgres`
- `go test ./internal/adapters/inbound/httpapi/presenter`
- `go test ./internal/adapters/inbound/httpapi`
- `go test ./...`
- `go build ./cmd/weatherapi`

The database-backed endpoint checklist requires a reachable PostgreSQL database
populated by weatherfeeder outlook v2 tables. It was not run in this workspace
because no such populated database was available.

## Summary

Implement `weatherapi` support for the `weatherfeeder` SPC outlook v2 contract described in `docs/roadmap/outlook.md`.

This is a compatibility update for the existing convective outlook route family. Do not add or remove public routes. Update dependency, repository reads, app filtering, presenters, templates, tests, and current-behavior docs so `weatherapi` reads `weather.outlook.v2` data from the new weatherfeeder Postgres table shape.

## Required End State

- `go.mod` depends on the released `weatherfeeder` version containing `weather.outlook.v2` support.
- Existing public routes remain available:
  - `GET /outlooks/convective`
  - `GET /outlooks/convective/active`
  - `GET /outlooks/convective/location`
- `weatherapi` remains read-only and does not create, migrate, or repair weatherfeeder tables.
- The Postgres repository reads `outlook_runs`, `outlooks`, and `outlook_discussions`.
- The repository no longer expects polygon-level `headline`, `summary`, or `discussion` columns on `outlooks`.
- Returned `WeatherOutlookRun` values include run-level `Discussions` loaded from `outlook_discussions`.
- App-level filtering trims both `Outlooks` and `Discussions` so discussions are present only for days with retained outlooks.
- Presenters copy and timezone-convert run-level discussion `updatedAt` values.
- JSON/XML/text responses expose run-level `discussions` and do not expose polygon-level outlook prose fields.
- `data: null` semantics for no latest run are preserved.
- Filtered no-match responses return a non-null run with `outlooks: []` and `discussions: []`.
- Public query behavior remains compatible: `format`, `units`, `tz` / `TZ`, `day`, `outlookType`, and allowed `containsLocation` behavior are preserved; `precision` and unknown params remain rejected.

## Guardrails

- Do not implement weatherfeeder table creation or migration in weatherapi.
- Do not call SPC or any upstream weather provider from weatherapi.
- Do not introduce historical browsing or cross-run accumulation in this change.
- Do not remove `/outlooks/convective/location`; keep it as an API-compatible active local-outlook route.
- Do not keep stale v1 SQL reads for `outlooks.headline`, `outlooks.summary`, or `outlooks.discussion`.
- Do not commit a `replace` directive for `weatherfeeder`.
- Keep SQL in `*_queries.go`, row DTOs in `*_rows.go`, row mapping in `*_mapper.go`, and request/presentation policy in HTTP/presenter packages.

## Stage 1: Dependency And Compile Boundary

### Changes

1. Update `go.mod` to the released `gitea.maximumdirect.net/ejr/weatherfeeder` version that includes:
   - `standards.SchemaWeatherOutlookV2`;
   - `model.WeatherOutlookRun.Discussions`;
   - `model.WeatherOutlookDiscussion`;
   - `model.WeatherOutlook` without `Headline`, `Summary`, or `Discussion`.
2. Run `go mod tidy`.
3. Fix compile errors from removed polygon-level outlook prose fields.
4. Search for outlook v1 and polygon prose references:

```sh
rg "SchemaWeatherOutlookV1|weather\.outlook\.v1|\.Headline|\.Summary|\.Discussion|headline|summary|discussion" internal docs templates README.md
```

5. Treat matches carefully:
   - Keep unrelated alert and forecast-discussion uses.
   - Remove or update outlook polygon-level prose references.
   - Replace outlook-specific schema references with v2 where applicable.

### Expected Files

- `go.mod`
- `go.sum`
- `internal/adapters/outbound/postgres/outlooks_rows.go`
- `internal/adapters/outbound/postgres/outlooks_mapper.go`
- `internal/adapters/outbound/postgres/outlooks_queries.go`
- `internal/adapters/outbound/postgres/outlooks_read.go`
- `internal/adapters/inbound/httpapi/presenter/outlook.go`
- `internal/app/service.go`
- outlook-related tests under `internal/app`, `internal/adapters/outbound/postgres`, `internal/adapters/inbound/httpapi`, and presenter tests

### Verification

At this stage, run a compile-oriented focused set. It may fail until later stages are complete, but failures should identify remaining v1/prose references.

```sh
go test ./internal/app ./internal/adapters/outbound/postgres ./internal/adapters/inbound/httpapi/presenter
```

## Stage 2: Postgres Query, Row, And Mapper Updates

### Query Changes

Update `internal/adapters/outbound/postgres/outlooks_queries.go`.

Parent query:

- Keep latest run ordering:

```sql
ORDER BY as_of DESC, event_emitted_at DESC
LIMIT 1
```

- Continue selecting parent fields needed by `WeatherOutlookRun`.
- Optionally select `discussion_count` for test/sanity visibility, but do not expose it in the API response model.

Outlook child query:

- Remove these v1 columns from the `SELECT` list:
  - `headline`
  - `summary`
  - `discussion`
- Continue selecting:
  - `outlook_index`
  - `outlook_id`
  - `provider`
  - `product`
  - `day`
  - `outlook_type`
  - `label`
  - `label_text`
  - `severity_rank`
  - `valid_from`
  - `valid_to`
  - `issued_at`
  - `expires_at`
  - `forecaster`
  - `source_url`
  - `image_url`
  - `contains_location`
  - `geometry_json`
- Keep `WHERE run_event_id = $1`.
- Keep `ORDER BY outlook_index ASC`.

Discussion child query:

- Add `queryOutlookDiscussionsForRun`:

```sql
SELECT
  discussion_index,
  day,
  headline,
  summary,
  discussion,
  updated_at
FROM outlook_discussions
WHERE run_event_id = $1
ORDER BY discussion_index ASC
```

### Row DTO Changes

Update `internal/adapters/outbound/postgres/outlooks_rows.go`:

- Remove `Headline`, `Summary`, and `Discussion` from `outlookRow`.
- Add `DiscussionCount` to `outlookRunParentRow` only if selected by parent query.
- Add `outlookDiscussionRow`:
  - `DiscussionIndex int`
  - `Day int`
  - `Headline sql.NullString`
  - `Summary sql.NullString`
  - `Discussion sql.NullString`
  - `UpdatedAt sql.NullTime`

### Mapper Changes

Update `internal/adapters/outbound/postgres/outlooks_mapper.go`:

- `mapOutlookRow` maps only v2 polygon fields.
- Keep geometry validation with `json.Valid` and return contextual mapper errors for invalid JSON.
- Continue copying geometry bytes with `append([]byte(nil), geometry...)`.
- Continue normalizing outlook timestamps to UTC.
- Add `mapOutlookDiscussionRow` returning `model.WeatherOutlookDiscussion`:
  - `Day` from row day;
  - string fields via existing `stringValue` helper;
  - `UpdatedAt` via existing `timePtr` helper, ensuring UTC normalization.

### Read Flow Changes

Update `internal/adapters/outbound/postgres/outlooks_read.go`:

- `LatestConvectiveOutlookRun` loads parent row as today.
- After parent row:
  - call `loadOutlooks(ctx, row.EventID)`;
  - call `loadOutlookDiscussions(ctx, row.EventID)`;
  - attach both to the run.
- Add `loadOutlookDiscussions` mirroring `loadOutlooks` style:
  - query with context;
  - scan rows;
  - map rows;
  - return iteration errors;
  - wrap query, scan, map, and iteration errors with operation context.
- Preserve `nil, nil` on missing parent row.

### Tests

Update or add tests in `internal/adapters/outbound/postgres`:

- `mapOutlookRow` maps v2 polygon fields and no longer expects polygon-level prose.
- `mapOutlookRow` rejects invalid `geometry_json`.
- nullable label/forecaster/source/image/severity fields map correctly.
- `mapOutlookDiscussionRow` maps day/headline/summary/discussion/updatedAt and normalizes time to UTC.
- nullable discussion fields map to empty/omitted canonical values.
- latest run read loads outlooks in `outlook_index ASC` order.
- latest run read loads discussions in `discussion_index ASC` order.
- missing latest parent returns `nil, nil`.
- query/scan/iteration failures remain context-wrapped.

### Verification

```sh
go test ./internal/adapters/outbound/postgres
```

## Stage 3: Application Filtering And Copy Semantics

### Changes

Update `internal/app/service.go`.

Clone behavior:

- `cloneOutlookRun` must deep-copy `Discussions` in addition to `Outlooks`.
- Ensure outlook severity pointers, geometry bytes, latitude/longitude pointers, and issuedAt pointers remain copied.
- Add a helper such as `cloneOutlookDiscussion` if useful.

Filtering behavior:

- Filter `Outlooks` as today using `OutlookFilter`.
- After filtering outlooks, filter `Discussions` to only days represented by retained outlooks.
- Preserve discussion order from the repository result.
- If no outlooks remain, set discussions to an empty slice, not stale unfiltered discussions.
- Preserve nil/missing run behavior: repository `nil, nil` still returns `nil, nil`.

Recommended helper shape:

```go
func filterOutlookDiscussions(discussions []model.WeatherOutlookDiscussion, outlooks []model.WeatherOutlook) []model.WeatherOutlookDiscussion
```

Rules:

- Build a set of retained outlook days.
- If the set is empty, return `[]model.WeatherOutlookDiscussion{}` when discussions was non-nil or when consistent empty-array output is desired.
- Include each discussion only if `discussion.Day` is in the set.
- Do not synthesize discussions.

### Query Compatibility

Do not change query binder behavior in this stage unless tests reveal compile fallout from the model update. Public query behavior remains:

- `day` accepted on all outlook routes.
- `outlookType` accepted on all outlook routes.
- `containsLocation` accepted on `/outlooks/convective` and `/outlooks/convective/active`.
- `containsLocation` rejected on `/outlooks/convective/location`.
- `format`, `units`, and `tz` / `TZ` accepted.
- `precision` and unknown params rejected.

### Tests

Update `internal/app/service_test.go`:

- Delegation test still asserts one repository call.
- Day filter retains matching outlooks and only matching day discussions.
- Outlook type filter retains discussions only for days with retained outlooks.
- Active filter retains discussions only for days with active retained outlooks.
- `containsLocation=false` returns an empty outlook/discussion run for v2-style test data.
- Filtered no-match result has empty `Outlooks` and empty `Discussions`.
- Mutating returned outlooks/discussions does not mutate repository-owned data.
- Geometry byte copy and pointer copy behavior remains covered.

### Verification

```sh
go test ./internal/app
```

## Stage 4: Presenter And Text Template Updates

### Presenter Changes

Update `internal/adapters/inbound/httpapi/presenter/outlook.go`:

- Include `Discussions` when constructing the presented `model.WeatherOutlookRun`.
- Convert `WeatherOutlookDiscussion.UpdatedAt` using `inLocationTimePtr`.
- Deep-copy discussions so presenter output mutation cannot affect repository/app input.
- Remove `Headline`, `Summary`, and `Discussion` assignments from `copyOutlook`.
- Continue converting:
  - run `AsOf`;
  - run `IssuedAt`;
  - outlook `ValidFrom`;
  - outlook `ValidTo`;
  - outlook `IssuedAt`;
  - outlook `ExpiresAt`.
- Continue preserving geometry copy behavior.
- Continue returning `nil` for nil input.

Recommended helper:

```go
func copyOutlookDiscussion(d model.WeatherOutlookDiscussion, tz *time.Location) model.WeatherOutlookDiscussion
```

### Template Changes

Update `templates/outlooks_convective.txt.tmpl`:

- Render run-level discussions from `.Data.Discussions`.
- Do not reference outlook-level `.Headline`, `.Summary`, or `.Discussion`.
- Keep no-data output when `.Data` is nil.
- Keep useful output when `.Data` exists and `.Data.Outlooks` is empty.
- Include discussion day/headline/summary/discussion fields in a readable, conditional way.
- Avoid printing zero timestamps or `<nil>` values for absent `updatedAt`.

### Presenter Tests

Update `internal/adapters/inbound/httpapi/presenter/payload_test.go`:

- `OutlookRunPayload(nil, ...)` returns nil.
- Output includes copied `Discussions`.
- Discussion `UpdatedAt` is converted to requested timezone.
- Mutating output discussions does not mutate input discussions.
- Mutating output geometry does not mutate input geometry.
- Outlook timestamps still convert to requested timezone.
- Polygon-level prose fields are not expected in output structs/tests.

### Verification

```sh
go test ./internal/adapters/inbound/httpapi/presenter
```

## Stage 5: HTTP Endpoint Behavior And Tests

### Handler Behavior

Keep route registration and handler structure unchanged unless compile updates require mechanical edits.

Ensure endpoint responses reflect the updated presenter/app behavior:

- `/outlooks/convective` returns latest run with v2 shape.
- `/outlooks/convective/active` applies active filter and trims discussions.
- `/outlooks/convective/location` applies active local filter and trims discussions.
- `data: null` remains unchanged when no run exists.
- Filtered no-match response returns data object with empty outlook and discussion arrays.

### Endpoint Tests

Update `internal/adapters/inbound/httpapi/endpoints_test.go`:

- test fakes build `model.WeatherOutlookRun` with `Discussions`.
- JSON success asserts run-level `discussions` are present.
- JSON success asserts outlook polygons do not include polygon-level prose.
- XML success renders run-level discussions without errors.
- Text success renders run-level discussion content.
- Timezone test checks conversion of:
  - run `asOf`;
  - run `issuedAt`;
  - outlook times;
  - discussion `updatedAt`.
- Null data test remains unchanged.
- Filtered no-match test expects both `outlooks` and `discussions` empty.
- Query acceptance still covers `units`, `format`, `tz` / `TZ`, `day`, `outlookType`, and allowed `containsLocation`.
- Query rejection still covers `precision`, unknown params, invalid day, invalid outlookType, invalid containsLocation, invalid timezone, conflicting `tz` / `TZ`, and `containsLocation` on `/location`.
- Active/location route tests continue using injectable `outlookNow`.

### Verification

```sh
go test ./internal/adapters/inbound/httpapi
```

## Stage 6: Documentation Updates

After code behavior is updated, update current-behavior documentation. Do not leave v2 behavior only in roadmap files.

### Public Docs

Update `docs/api.md`:

- State outlook endpoints serve weatherfeeder `weather.outlook.v2` data.
- Run fields include `discussions`.
- Add discussion field table:
  - `day`;
  - `headline`;
  - `summary`;
  - `discussion`;
  - `updatedAt`.
- Remove polygon-level `headline`, `summary`, and `discussion` from outlook fields.
- State v2 outlooks are already location-filtered by weatherfeeder.
- State `containsLocation` is expected to be true for v2 outlooks.
- Explain `/outlooks/convective/location` remains an active local-outlook compatibility route under v2.
- State endpoint filters also filter `discussions` to days represented by retained outlooks.
- State current endpoints use latest-run semantics and do not accumulate historical active outlook rows.
- Update examples to show run-level `discussions`.
- Ensure examples use valid GeoJSON polygon or multipolygon, not a point, if demonstrating SPC geometry.

Update `README.md` only if its overview implies all-polygons or old response shape.

### Integration/Internal Docs

Update `docs/integrations/weatherfeeder-postgres.md`:

- Update dependency version.
- Table family for convective outlook includes `outlook_runs`, `outlooks`, and `outlook_discussions`.
- Latest run selection remains `as_of DESC, event_emitted_at DESC`.
- Child ordering includes:
  - outlooks by `outlook_index ASC`;
  - outlook discussions by `discussion_index ASC`.
- `outlook_runs` includes `discussion_count` if listed.
- `outlooks` no longer includes `headline`, `summary`, or `discussion`.
- Add `outlook_discussions` columns.
- State weatherapi assumes weatherfeeder's outlook v2 table reset has already been applied.

Update `docs/internal/postgres-repository.md`:

- `LatestConvectiveOutlookRun` loads parent, outlook children, and discussion children.
- Add outlook discussion child ordering.

Update `docs/internal/presenters.md`:

- Outlook presenter copies and timezone-converts run-level discussions.
- Outlook presenter no longer handles polygon-level prose.

Update `docs/internal/http-adapter.md`:

- Outlook filters trim discussions to retained outlook days.
- `/location` is retained for compatibility and active local-outlook behavior.

Update `docs/operations.md` and `docs/troubleshooting.md` only if they mention the old weatherfeeder table shape or should warn operators that weatherapi requires weatherfeeder v2 outlook tables.

### Roadmap Docs

After implementation is complete, replace or update `docs/roadmap/outlook.md` and `docs/roadmap/implementation.md` according to repository convention:

- Either mark the roadmap complete and point to current docs.
- Or move any remaining deferred ideas to a future roadmap file.

Do not leave roadmap files claiming unimplemented work after the implementation has shipped.

### Documentation Checks

Run searches for stale outlook v1/current behavior:

```sh
rg "weather\.outlook\.v1|outlooks\.headline|outlooks\.summary|outlooks\.discussion|polygon-level|all stored outlook|all polygons|headline, summary, discussion" docs README.md internal templates
```

Allowed matches:

- explicit historical/legacy notes, if any;
- unrelated alert or forecast discussion prose fields;
- roadmap files that are intentionally historical.

## Stage 7: Full Verification

Run focused tests:

```sh
go test ./internal/app
go test ./internal/adapters/outbound/postgres
go test ./internal/adapters/inbound/httpapi/presenter
go test ./internal/adapters/inbound/httpapi
```

Run the full suite:

```sh
go test ./...
```

Optional build check:

```sh
go build ./cmd/weatherapi
```

Manual verification against a database populated by weatherfeeder outlook v2 tables:

```http
GET /outlooks/convective
GET /outlooks/convective/active
GET /outlooks/convective/location
GET /outlooks/convective?day=1
GET /outlooks/convective?outlookType=tornado
GET /outlooks/convective?containsLocation=false
GET /outlooks/convective?format=text&tz=America/Chicago
```

Expected manual results:

- Latest run includes run-level `discussions`.
- Active route filters outlooks and discussions together.
- Location route returns active local outlooks and remains valid.
- `containsLocation=false` returns an empty run for v2 data.
- Text format renders discussions without template errors.
- XML format renders without errors.
- No endpoint queries historical active rows across older runs.

## Assumptions

- A weatherfeeder release containing outlook v2 exists before final implementation is committed.
- Operators have reset/recreated weatherfeeder outlook tables according to weatherfeeder transition docs.
- Existing weatherapi route names remain stable for external consumers.
- Keeping `containsLocation` query support on non-location routes is useful backward compatibility even though v2 data should always be local.
- Latest-run semantics are correct for current convective outlook endpoints.

## Open Questions

None. This plan fixes the route, dependency, storage, filtering, presentation, and documentation decisions required to support weatherfeeder outlook v2 while preserving weatherapi's existing public route family.