Add roadmap and staged plan for implementing SPC convective outlook support

2026-06-12 09:42:43 -05:00
parent f149563c68
commit 2aba52f552
4 changed files with 784 additions and 3 deletions
--- a/docs/roadmap/implementation.md
+++ b/docs/roadmap/implementation.md
@@ -0,0 +1,591 @@
+# SPC Convective Outlook Implementation Roadmap
+
+## Purpose
+
+This roadmap defines the concrete implementation sequence for
+`docs/roadmap/outlook.md`. It is written for an LLM coding agent that will
+implement the stages in order.
+
+This is a future-work roadmap. Until a stage is implemented, non-roadmap docs
+must not describe SPC convective outlook behavior as available.
+
+## Source Feature Roadmap
+
+Use `docs/roadmap/outlook.md` as the authoritative feature roadmap for intent,
+target shape, and policy decisions. This implementation roadmap is the
+step-by-step work plan. If the two documents conflict, update the feature
+roadmap first when the target behavior changes, then update this implementation
+plan.
+
+## Locked Implementation Decisions
+
+- Preserve existing CLI syntax, generated artifact paths, report IDs, prompt
+  IDs, distributor behavior, and Scriptorium invocation.
+- Use source key and module IDs based on `spc_convective_outlooks`.
+- Add two modules:
+  - `spc_convective_outlooks` in `applicable_risk_products`;
+  - `spc_convective_discussion` in `narrative_products`.
+- Fetch the upstream Weather API source once in
+  `internal/adapters/weatherapi`; module builders must not fetch upstream data.
+- Use a Weather API adapter constant for the path:
+  `/outlooks/convective`.
+- Initial query parameters for the outlook endpoint are `format=json` and the
+  configured `tz`; do not send `precision`.
+- Treat the source as optional under existing missing-source policy.
+- Treat `data: null` as no latest run and therefore missing/unavailable
+  optional source data.
+- Treat a non-null run with empty `outlooks` and `discussions` arrays as
+  checked, non-missing empty data.
+- Preserve GeoJSON geometry in collected facts and persisted bundle artifacts,
+  but omit geometry from prompt-facing module output.
+- Filter outlooks by overlap with the resolved report valid period in Go.
+- Do not depend on `/outlooks/convective/active` or
+  `/outlooks/convective/location` for initial behavior.
+- Include SPC discussion text only when at least one retained report-period
+  outlook has `severity_rank >= 3`.
+- Define the discussion threshold as an internal constant, initially `3`, not a
+  config field.
+
+## Stage 1: Weather Data Contract
+
+Goal: add typed collected SPC outlook facts without changing fetching or prompt
+output yet.
+
+Files to inspect:
+
+- `internal/weatherdata/bundle.go`
+- `internal/facts/facts.go`
+- `internal/module/module.go`
+- `internal/briefing/modules.go`
+- `internal/weatherdata` and `internal/facts` tests
+
+Implementation:
+
+- Add `ConvectiveOutlookRun` to `internal/weatherdata`.
+- Add `ConvectiveOutlook` with fields matching the consumed Weather API
+  outlook fields:
+  - `id`
+  - `provider`
+  - `product`
+  - `day`
+  - `outlookType`
+  - `label`
+  - `labelText`
+  - `forecaster`
+  - `severityRank`
+  - `validFrom`
+  - `validTo`
+  - `issuedAt`
+  - `expiresAt`
+  - `sourceUrl`
+  - `imageUrl`
+  - `containsLocation`
+  - `geometry`
+- Store `geometry` as `json.RawMessage` or an equivalent JSON-preserving type;
+  do not introduce a GeoJSON dependency.
+- Add `ConvectiveOutlookDiscussion` with:
+  - `day`
+  - `headline`
+  - `summary`
+  - `discussion`
+  - `updatedAt`
+- Add `SPCConvectiveOutlooks *ConvectiveOutlookRun` to
+  `weatherdata.Bundle`.
+- Add matching fields to `facts.CollectedFacts`, `BuildCollected`, and
+  `CollectedFacts.Bundle`.
+- Add module constants in `internal/module`:
+  - `SPCConvectiveOutlooks ID = "spc_convective_outlooks"`
+  - `SPCConvectiveDiscussion ID = "spc_convective_discussion"`
+- Add option structs:
+  - `SPCConvectiveOutlooksOptions struct{}`
+  - `SPCConvectiveDiscussionOptions struct{}`
+- Add fact requirement constants:
+  - `CollectedSPCConvectiveOutlooks`
+  - `RequiresDerivedSPCConvectiveOutlooks`
+- Wire the new collected requirement into
+  `briefing.collectedFactAvailable`.
+
+Acceptance criteria:
+
+- No Weather API request is added in this stage.
+- No report default module list changes in this stage.
+- No prompt package shape changes in this stage.
+- Existing tests pass.
+
+Suggested validation:
+
+```bash
+go test ./internal/weatherdata ./internal/facts ./internal/module ./internal/briefing
+```
+
+This stage is small enough for one implementation prompt.
+
+## Stage 2: Weather API Adapter Fetch
+
+Goal: fetch `/outlooks/convective`, decode it into the new weather data
+contract, and preserve source provenance.
+
+Files to inspect:
+
+- `internal/adapters/weatherapi/client.go`
+- `internal/adapters/weatherapi/client_test.go`
+- `internal/adapters/weatherapi/testdata/`
+- `docs/roadmap/outlook.md`
+- upstream Weather API docs under the Convective Outlooks section
+
+Implementation:
+
+- Add a package-level adapter constant, for example:
+  `convectiveOutlooksEndpoint = "/outlooks/convective"`.
+- Add source-name constant or narrowly scoped source key for
+  `spc_convective_outlooks`.
+- Add fan-out fetch for the new optional source.
+- Build the query as:
+  - `format=json`
+  - `tz=<configured weather_api.timezone>`
+- Do not send `precision` to this endpoint.
+- Do not send `units` to this endpoint. If existing query helpers add units by
+  default, add a route-specific option such as `omitUnits` so the final request
+  remains `format` plus `tz`.
+- Decode the Weather API envelope and payload into
+  `weatherdata.ConvectiveOutlookRun`.
+- Preserve `data: null` as optional missing/unavailable source data through
+  existing missing-source policy.
+- Preserve a non-null run with empty arrays as checked non-missing source data.
+- Record source provenance:
+  - source name `spc_convective_outlooks`;
+  - endpoint constant path;
+  - exact query parameters sent;
+  - fetched time;
+  - issued time from run `issuedAt` when present, otherwise `asOf`;
+  - source hash over compact raw `data` JSON;
+  - source warnings when missing-source policy emits them.
+- Add `internal/adapters/weatherapi/testdata/convective_outlooks.json`.
+
+Acceptance criteria:
+
+- Complete fixture fetch includes `bundle.SPCConvectiveOutlooks`.
+- Source record is present and non-missing when the payload has a non-null run.
+- `data: null` follows optional missing-source policy.
+- Non-null empty arrays do not produce a missing-source warning.
+- Adapter tests assert request path and query, including absence of
+  `precision`.
+- Existing hourly required-source behavior is unchanged.
+
+Suggested validation:
+
+```bash
+go test ./internal/adapters/weatherapi ./internal/weatherdata ./internal/facts
+```
+
+This stage is small enough for one implementation prompt.
+
+## Stage 3: Report-Period Outlook Filtering
+
+Goal: derive report-scoped SPC outlook facts by valid-period overlap.
+
+Files to inspect:
+
+- `internal/facts/facts.go`
+- `internal/forecast/derive.go`
+- `internal/timeutil/periods.go`
+- `internal/report/*_report.go`
+- existing forecast/facts tests for period slicing
+
+Implementation:
+
+- Add a report-scoped derived value for retained outlooks and discussions.
+  Recommended shape:
+  - `DerivedFacts.SPCConvectiveOutlooks []weatherdata.ConvectiveOutlook`
+  - `DerivedFacts.SPCConvectiveDiscussions []weatherdata.ConvectiveOutlookDiscussion`
+- Add a small deterministic helper under `internal/facts`; this first
+  implementation is report-scoped selection of already-collected facts, not a
+  broader meteorological derivation.
+- Select outlooks whose half-open valid interval overlaps the resolved report
+  valid period.
+- Treat missing `severityRank` as lower than the discussion threshold, but do
+  not drop the outlook from the risk-product module solely because rank is
+  missing.
+- Retain discussion records only for days represented by retained outlooks.
+- Sort retained outlooks deterministically by:
+  - day;
+  - outlook type;
+  - severity rank descending when present;
+  - valid start;
+  - label;
+  - id.
+- Sort retained discussions by day, then updated time when present.
+- Keep empty retained slices distinct from a missing collected source.
+
+Acceptance criteria:
+
+- Daily Today, Daily Tomorrow, 3-Day, Weekend, and Storm valid periods select
+  expected outlooks by overlap.
+- Tomorrow and multi-day reports do not depend on server-current active
+  filtering.
+- Empty retained results are still available to modules as checked empty data
+  when the collected source exists.
+
+Suggested validation:
+
+```bash
+go test ./internal/facts ./internal/forecast ./internal/report
+```
+
+This stage is small enough for one implementation prompt.
+
+## Stage 4: Prompt Category Plumbing
+
+Goal: prepare prompt-package category placement without registering
+builderless modules.
+
+Files to inspect:
+
+- `internal/module/module.go`
+- `internal/promptinput/package.go`
+- `internal/promptinput/package_test.go`
+
+Implementation:
+
+- Add prompt input category mapping:
+  - `spc_convective_outlooks` -> `applicable_risk_products`;
+  - `spc_convective_discussion` -> `narrative_products`.
+- Use synthetic module snapshots in tests if needed; do not add module
+  definitions to `defaultModuleDefinitions` until the real builders are added
+  in Stages 5 and 6.
+
+Acceptance criteria:
+
+- Prompt category tests prove both new stanzas route to the intended groups.
+- The module registry still rejects unknown or builderless modules.
+- No report default includes the new modules yet.
+
+Suggested validation:
+
+```bash
+go test ./internal/module ./internal/promptinput
+```
+
+This stage is small enough for one implementation prompt.
+
+## Stage 5: SPC Convective Outlooks Module
+
+Goal: add the prompt-facing risk-product module.
+
+Files to inspect:
+
+- `internal/briefing/alert_digest_module.go`
+- `internal/briefing/weather_story_module.go`
+- `internal/briefing/module_format_helpers.go`
+- `internal/briefing/base_modules_test.go`
+- `docs/roadmap/outlook.md`
+
+Implementation:
+
+- Add `internal/briefing/spc_convective_outlooks_module.go`.
+- Add the `SPCConvectiveOutlooks` module definition to
+  `defaultModuleDefinitions` in the same change as its real builder.
+- Register stanza `spc_convective_outlooks`.
+- Require:
+  - `CollectedSPCConvectiveOutlooks`;
+  - `RequiresDerivedSPCConvectiveOutlooks`.
+- Use `MissingDataEmpty` so checked empty data can emit an explicit empty
+  risk-product stanza.
+- Build from collected source metadata plus derived retained outlooks.
+- Emit concise prompt-facing fields:
+  - `checked`;
+  - `as_of`;
+  - `issued_at`;
+  - `location_id`;
+  - `location_name`;
+  - `outlook_count`;
+  - `outlooks`.
+- For each outlook, emit:
+  - `day`;
+  - `outlook_type`;
+  - `label`;
+  - `label_text`;
+  - `severity_rank`;
+  - `valid_start`;
+  - `valid_end`;
+  - `issued_at`;
+  - `expires_at`;
+  - `contains_location`;
+  - `source_url`;
+  - `image_url`.
+- Use human-readable local time helpers consistent with current modules.
+- Do not emit GeoJSON geometry.
+- If the source was checked and no retained outlooks overlap the report
+  period, emit `checked: true`, `outlook_count: 0`, and an empty or omitted
+  `outlooks` list according to the existing YAML style for empty lists.
+
+Acceptance criteria:
+
+- Module output is deterministic and omits geometry.
+- Checked empty data produces an explicit checked-empty stanza.
+- Missing collected source follows registry missing-data behavior.
+- Module tests cover populated, checked-empty, and missing cases.
+
+Suggested validation:
+
+```bash
+go test ./internal/briefing ./internal/module ./internal/promptinput
+```
+
+This stage is small enough for one implementation prompt.
+
+## Stage 6: SPC Convective Discussion Module
+
+Goal: add optional SPC discussion narrative context with a severity threshold.
+
+Files to inspect:
+
+- `internal/briefing/area_forecast_discussion_module.go`
+- `internal/briefing/weather_story_module.go`
+- `internal/briefing/module_format_helpers.go`
+- `internal/briefing/base_modules_test.go`
+
+Implementation:
+
+- Add `internal/briefing/spc_convective_discussion_module.go`.
+- Add the `SPCConvectiveDiscussion` module definition to
+  `defaultModuleDefinitions` in the same change as its real builder.
+- Register stanza `spc_convective_discussion`.
+- Require:
+  - `CollectedSPCConvectiveOutlooks`;
+  - `RequiresDerivedSPCConvectiveOutlooks`.
+- Use `MissingDataOmit` so unavailable or below-threshold discussion text is
+  omitted.
+- Define a package-private constant near the module, for example:
+  `defaultSPCConvectiveDiscussionMinimumSeverityRank = 3`.
+- Build from derived retained outlooks and discussions.
+- Include discussion text only when at least one retained outlook has
+  `severityRank >= defaultSPCConvectiveDiscussionMinimumSeverityRank`.
+- When the threshold is not met, return `nil` output so the stanza is omitted.
+- When threshold is met, include discussions for retained outlook days with:
+  - `day`;
+  - `headline`;
+  - `summary`;
+  - `discussion`;
+  - `updated_at`.
+- Include a concise reason field such as:
+  `included_because: "severity_rank >= 3"`.
+
+Acceptance criteria:
+
+- Slight Risk or higher retained outlooks include matching discussion records
+  when available.
+- Lower-risk retained outlooks still appear in `spc_convective_outlooks` but
+  do not emit `spc_convective_discussion`.
+- Missing discussion text omits the stanza without failing report generation.
+- Tests cover threshold below, threshold equal, threshold above, and missing
+  discussion cases.
+
+Suggested validation:
+
+```bash
+go test ./internal/briefing ./internal/promptinput
+```
+
+This stage is small enough for one implementation prompt.
+
+## Stage 7: Report Composition And Config Examples
+
+Goal: add the implemented modules to default report definitions and maintained
+examples.
+
+Files to inspect:
+
+- `internal/report/daily_report.go`
+- `internal/report/three_day_report.go`
+- `internal/report/weekend_report.go`
+- `internal/report/storm_report.go`
+- `internal/config/reports.go`
+- `internal/config/config_test.go`
+- `examples/config.yml`
+
+Implementation:
+
+- Add `spc_convective_outlooks` to default report module lists for:
+  - Daily Today;
+  - Daily Tomorrow;
+  - 3-Day;
+  - Weekend;
+  - Storm.
+- Place `spc_convective_outlooks` immediately after `alert_digest` when
+  `alert_digest` is present.
+- Add `spc_convective_discussion` immediately after
+  `area_forecast_discussion` when `area_forecast_discussion` is present.
+- Update maintained example config module overrides if they enumerate module
+  lists.
+- Keep CLI syntax and config field names unchanged.
+
+Acceptance criteria:
+
+- All default report module compositions validate.
+- Example config loads successfully.
+- Config override tests can include both new module IDs.
+- Existing report IDs, prompt IDs, output names, and valid-period behavior are
+  unchanged.
+
+Suggested validation:
+
+```bash
+go test ./internal/report ./internal/config ./internal/briefing ./internal/app
+```
+
+This stage is small enough for one implementation prompt.
+
+## Stage 8: App And Prompt Workflow Coverage
+
+Goal: prove the end-to-end generated data package contains the new stanzas in
+the intended categories when fixture data warrants them.
+
+Files to inspect:
+
+- `internal/app/app_test.go`
+- `internal/promptinput/package_test.go`
+- Weather API test server fixtures in `internal/adapters/weatherapi`
+- `internal/state` artifact path behavior if tests inspect saved files
+
+Implementation:
+
+- Extend app-level Weather API fixtures to serve convective outlook data.
+- Add or update workflow tests so saved YAML contains:
+  - `briefing.applicable_risk_products.spc_convective_outlooks`;
+  - `briefing.narrative_products.spc_convective_discussion` when severity rank
+    is at least `3`;
+  - no geometry in prompt-facing YAML.
+- Add a workflow case where outlook data is checked but empty and the
+  risk-product stanza remains explicit.
+- Add a workflow case where severity rank is below `3` and the discussion
+  stanza is omitted.
+- Keep existing Recent Changes behavior unchanged unless a later roadmap adds
+  comparisons for SPC outlooks.
+
+Acceptance criteria:
+
+- Module snapshot and YAML data package remain deterministic.
+- Prompt category grouping preserves module order within categories.
+- No Scriptorium or distributor behavior changes are required.
+
+Suggested validation:
+
+```bash
+go test ./internal/app ./internal/promptinput ./internal/state
+```
+
+This stage is small enough for one implementation prompt.
+
+## Stage 9: Implemented Documentation
+
+Goal: update non-roadmap docs after the feature is implemented.
+
+Files to inspect and update:
+
+- `docs/integrations/weatherapi.md`
+- `docs/internal/weather-data.md`
+- `docs/internal/facts.md`
+- `docs/internal/module.md`
+- `docs/internal/briefing.md`
+- `docs/internal/prompt-input.md`
+- `docs/config.md` if example module override behavior changes
+- `examples/config.yml` if not already updated in Stage 7
+
+Documentation requirements:
+
+- Describe only the implemented SPC behavior outside `docs/roadmap/`.
+- In the Weather API integration doc, include only the route, query, response
+  fields, and missing/empty semantics used by weatherreporter.
+- In internal docs, distinguish:
+  - collected source facts and geometry/provenance;
+  - derived report-period filtering;
+  - prompt-facing module output that omits geometry.
+- In prompt-input docs, list:
+  - `spc_convective_outlooks` under `applicable_risk_products`;
+  - `spc_convective_discussion` under `narrative_products`.
+- Keep deferred route choices, geometry presentation, and user-configurable
+  threshold ideas under roadmap docs only.
+
+Acceptance criteria:
+
+- Non-roadmap docs do not describe deferred SPC behavior as current behavior.
+- Maintained examples load.
+- Documentation links and module ID lists are consistent with code.
+
+Suggested validation:
+
+```bash
+go test ./internal/config
+git diff --check
+```
+
+This stage is small enough for one implementation prompt.
+
+## Stage 10: Final Validation
+
+Goal: validate the complete feature and guard against regressions.
+
+Run:
+
+```bash
+go test ./internal/adapters/weatherapi ./internal/weatherdata ./internal/facts ./internal/forecast ./internal/briefing ./internal/module ./internal/report ./internal/config ./internal/app ./internal/promptinput
+go test ./...
+go run ./cmd/weatherreporter --help
+git diff --check
+```
+
+Manual review:
+
+- Confirm `/outlooks/convective` is referenced through an adapter constant.
+- Confirm Weather API outlook requests do not send `precision`.
+- Confirm no module builder performs Weather API calls.
+- Confirm prompt YAML omits GeoJSON geometry.
+- Confirm checked-empty outlook data is not represented as missing data.
+- Confirm SPC discussion text appears only at severity rank `3` or higher.
+- Confirm public CLI syntax, output paths, distributor upload behavior, and
+  Scriptorium argv remain unchanged.
+
+This stage is small enough for one implementation prompt.
+
+## Deferred Work
+
+Out of scope for the initial implementation:
+
+- use of `/outlooks/convective/active`;
+- use of `/outlooks/convective/location`;
+- per-report Weather API filters such as `day` or `outlookType`;
+- user-configurable SPC discussion severity threshold;
+- prompt-facing GeoJSON geometry;
+- polygon distance, area, map summaries, or rendered images;
+- Mesoscale Discussions, watches, WPC outlooks, radar, QPF, or other risk
+  products;
+- Recent Changes comparisons for SPC outlook changes;
+- module-owned upstream fetching.
+
+## Global Validation Checklist
+
+Before considering the feature complete:
+
+- all focused package tests pass;
+- `go test ./...` passes;
+- `go run ./cmd/weatherreporter --help` still matches documented CLI syntax;
+- `git diff --check` passes;
+- examples load through config tests;
+- non-roadmap docs describe only implemented behavior;
+- no secret values or large raw geometry are introduced into prompt-facing
+  output;
+- source provenance and warnings remain inspectable through existing metadata
+  and state artifacts.
+
+## Open Questions
+
+No question blocks implementation.
+
+The recommended approach is to implement the locked decisions exactly as
+described above. The main viable alternative is to query
+`/outlooks/convective/active` or `/outlooks/convective/location`, but that
+would make tomorrow, multi-day, weekend, and event reports depend on
+server-current active filtering rather than report valid periods. That
+alternative should be deferred unless fixture payloads from the base latest-run
+route prove too large or too irrelevant for prompt use.