weatherreporter/docs/roadmap/initial.md

# Weatherreporter Implementation Roadmap

This roadmap defines a staged implementation plan for `weatherreporter`, a Go application that prepares human-facing weather reports from normalized weather data collected by `weatherfeeder` and rendered through `scriptorium`.

The goal is to build the application in stable layers. Each stage should leave the repository in a working, testable state. Early stages should prioritize inspectable intermediate artifacts over complete automation.

## Guiding Implementation Principles

- Build deterministic data preparation before LLM rendering.
- Keep CLI, adapters, domain logic, state, and prompt-variable construction separate.
- Use fixture-driven tests for forecast processing and briefing builders.
- Persist intermediate artifacts so failed or low-quality reports can be inspected.
- Add one report type fully before generalizing to all report types.
- Treat `scriptorium` as an external adapter during the prototype.
- Do not build the storm-monitoring agent until scheduled report generation is reliable.

## Stage 0: Repository Skeleton and Architecture Baseline

### Goal

Create the project skeleton, commit the architecture documents, and establish the package boundaries before adding substantial logic.

### Packages Introduced

- `cmd/weatherreporter`
- `internal/cli`
- `internal/app`
- `internal/config`

### Work Items

1. Initialize the Go module.
2. Add the architecture policy document.
3. Add the package layout document.
4. Add this implementation roadmap.
5. Create minimal package directories and placeholder files where useful.
6. Add basic build/test tooling.
7. Add a minimal `weatherreporter --help` command.

### Deliverables

- Go module builds successfully.
- `go test ./...` passes.
- Basic CLI entrypoint exists.
- Documentation is present under the appropriate docs directory.

### Done Criteria

- The repository has a clear skeleton matching the intended architecture.
- The binary can be built.
- The CLI can display help without loading external services.

## Stage 1: Configuration and CLI Foundation

### Goal

Implement configuration loading and a stable command shape before integrating external systems.

### Packages Introduced or Expanded

- `internal/config`
- `internal/cli`
- `internal/app`
- `internal/timeutil`

### Work Items

1. Define configuration structs for:
   - Weather API settings.
   - Locations.
   - Location time zones.
   - `scriptorium` settings.
   - Workspace paths.
   - Report output paths.
   - Daypart definitions.
   - Recent-change thresholds.
2. Implement built-in defaults in `internal/config/defaults.go`.
3. Implement YAML config loading from:
   - `/usr/local/etc/weatherreporter/config.yml`
   - CLI override via `--config`
4. Implement config validation.
5. Implement basic command structure:
   - `generate daily`
   - `generate tomorrow`
   - `generate three-day`
   - `generate weekend`
   - `generate storm`
   - `run morning`
   - `run evening`
6. Commands may initially return “not implemented” after config and request resolution.
7. Add time-zone and clock helpers.

### Deliverables

- Config can be loaded, validated, and inspected in tests.
- CLI commands parse expected flags.
- App-layer request structs exist for report generation and scheduled batches.

### Tests

- Config defaults load successfully.
- Example config file load test.
- Invalid config produces actionable errors.
- CLI parser tests for major commands.
- Time-zone resolution tests.

### Done Criteria

- CLI request handling is stable enough that later stages can attach real behavior without reshaping commands.
- Configuration can represent at least one location and the default daypart set.

## Stage 2: Weather API Adapter and Forecast Bundle

### Goal

Fetch normalized weather data from the internal weather API and represent it as a stable forecast bundle inside the application.

### Key References

- `docs/integrations/weatherapi.md` describes the weatherapi public API
- The local API endpoint is available at `https://weather.api.rakestrawhome.com/` and will return live data

### Packages Introduced or Expanded

- `internal/adapters/weatherapi`
- `internal/forecast`
- `internal/app`

### Work Items

1. Define the internal `forecast.Bundle` type.
2. Define source substructures for:
   - Hourly forecast data.
   - Daily forecast data (NOTE: not yet implemented upstream in weatherapi, so this can remain a stub in the initial implementation).
   - NWS narrative forecast periods.
   - NWS alerts.
   - NWS forecast discussion.
   - NWS weather story (NOTE: not yet implemented upstream in weatherapi, so this can remain a stub in the initial implementation).
3. Implement the weather API client.
4. Add context-aware HTTP calls and timeouts.
5. Add actionable errors for failed API calls and decode failures.
6. Add fixture support for tests.
7. Optionally add a debug command or app method to fetch and save the raw normalized bundle.

### Deliverables

- Weather API adapter can fetch a bundle for a configured location.
- Forecast bundle type is available to downstream packages.
- Tests can use fixtures without real API calls.

### Tests

- Decode representative API fixture into `forecast.Bundle`.
- HTTP error handling.
- Timeout/cancellation behavior.
- Missing or malformed source sections.

### Done Criteria

- The app can fetch current weather data and save or log a concise confirmation.
- No report generation is required yet.

## Stage 3: Forecast Derivation and Daypart Processing

### Goal

Implement deterministic forecast processing needed by the Daily Report.

### Packages Introduced or Expanded

- `internal/forecast`
- `internal/timeutil`

### Work Items

1. Implement configurable daypart definitions.
2. Implement daypart overlap logic, including overnight periods.
3. Group hourly forecast records into dayparts.
4. Compute daypart summaries:
   - Temperature range.
   - Apparent-temperature range, if available.
   - Max precipitation probability and associated hour.
   - Peak wind speed.
   - Peak wind gust.
   - Dominant or notable conditions.
   - Thunder, snow, ice, fog, heat, cold, or wind indicators where supported by source data.
5. Implement alert overlap with report periods and dayparts.
6. Implement basic threshold helpers.
7. Implement source selection helpers for NWS narrative periods and broader context.

### Deliverables

- A deterministic Daily Report forecast summary can be built from a fixture bundle.
- Daypart outputs are inspectable as JSON.

### Tests

- Morning/midday/afternoon/evening grouping.
- Overnight grouping across midnight.
- Missing hourly data behavior.
- Boundary timestamps at daypart edges.
- Max/min and threshold calculations.
- Alert-period overlap behavior.

### Done Criteria

- Forecast derivation is reliable enough to support a first report briefing package.
- No LLM or `scriptorium` integration is required yet.

## Stage 4: Report Registry and Valid-Period Resolution

### Goal

Centralize report definitions and valid-period behavior before building report-specific briefings.

### Packages Introduced or Expanded

- `internal/report`
- `internal/timeutil`
- `internal/app`

### Work Items

1. Define report IDs and variants:
   - `daily_today`
   - `daily_tomorrow`
   - `three_day`
   - `weekend`
   - `storm`
2. Define report metadata structures.
3. Define a report `Definition` contract.
4. Implement a registry.
5. Implement valid-period resolvers:
   - Today Daily Report.
   - Tomorrow Planning Brief.
   - 3-Day Outlook.
   - Weekend Outlook.
   - Storm Report placeholder.
6. Define default prompt IDs:
   - `weather.daily_report`
   - `weather.tomorrow_report`, or reuse `weather.daily_report` if preferred.
   - `weather.three_day_outlook`
   - `weather.weekend_outlook`
   - `weather.storm_report`
7. Implement batch membership rules:
   - Morning batch: Daily Today, 3-Day Outlook, Weekend Outlook except Sunday.
   - Evening batch: Daily Tomorrow.

### Deliverables

- App layer can resolve which reports should run for a command.
- Each report has a valid period independent of generation time.
- Report definitions map to prompt IDs.

### Tests

- Daily valid period for different generation times.
- Tomorrow valid period from evening generation.
- 3-day period calculation.
- Weekend period calculation on Monday, Friday, Saturday, and Sunday.
- Morning batch skips Weekend Outlook on Sunday.
- Registry lookup errors are actionable.

### Done Criteria

- Report identity and period behavior are stable.
- Later stages can add builders without changing CLI semantics.

## Stage 5: Daily Briefing Builder

### Goal

Build the first complete report-specific briefing package without invoking the LLM.

### Packages Introduced or Expanded

- `internal/briefing`
- `internal/report`
- `internal/forecast`
- `internal/app`

### Work Items

1. Define common briefing metadata:
   - Report type.
   - Variant.
   - Location.
   - Generation time.
   - Valid start/end.
   - Source timestamps or hashes.
2. Define the Daily Report briefing schema.
3. Build Daily Report briefing content:
   - Bottom-line inputs.
   - Daypart summaries.
   - Active or relevant alerts.
   - Best/worst outdoor window inputs, if derivable.
   - NWS narrative periods relevant to the day.
   - Forecast discussion summary or selected text from the API data.
   - Weather story summary or selected text from the API data.
4. Add JSON output for the briefing package.
5. Add an app workflow that can generate the Daily briefing and write it to disk for inspection.

### Deliverables

- `weatherreporter generate daily` can produce a Daily briefing JSON artifact without calling `scriptorium`.
- Fixture-based output is stable enough for review.

### Tests

- Daily briefing from representative fixture.
- Alerts included/excluded correctly.
- Source context selection.
- Empty or quiet-weather behavior.
- Snapshot metadata completeness.

### Done Criteria

- The Daily briefing package is useful as prompt input.
- The app can produce the briefing artifact from real or fixture data.

## Stage 6: Prompt Variable Builder

### Goal

Convert a briefing package into the structured variable payload expected by `scriptorium` prompts.

### Packages Introduced or Expanded

- `internal/promptvars`
- `internal/briefing`
- `internal/report`
- `internal/app`

### Work Items

1. Define prompt variable schema structures.
2. Build variables from report metadata and briefing content.
3. Include placeholders for Recent Changes, initially empty.
4. Validate required fields before rendering.
5. Write vars JSON to the workspace.
6. Ensure vars output is stable and inspectable.

### Deliverables

- Daily Report prompt vars can be generated and written to a file.
- The vars file is suitable for `scriptorium run --vars-file`.

### Tests

- Prompt vars generated from Daily briefing fixture.
- Missing required fields fail validation.
- JSON output is deterministic where practical.

### Done Criteria

- The app can prepare a complete vars file for a Daily Report.
- LLM rendering is the only missing step for the first end-to-end report.

## Stage 7: Scriptorium Adapter and First End-to-End Daily Report

### Goal

Invoke `scriptorium` as a subprocess and produce the first rendered Markdown report.

### Key References
 - `docs/integrations/scriptorium.md` describes the CLI contract for running `scriptorium` as a subprocess.

### Packages Introduced or Expanded

- `internal/adapters/scriptorium`
- `internal/app`
- `internal/state`, minimally if needed for output paths

### Work Items

1. Define `scriptorium.Runner` interface and request/result types.
2. Implement subprocess execution with `exec.CommandContext`.
3. Pass arguments without shell interpolation.
4. Prefer `--vars-file` for prompt variables.
5. Capture stderr and stdout with reasonable limits.
6. Apply timeout and cancellation.
7. Return actionable errors for nonzero exits.
8. Wire Daily Report generation end-to-end:
   - Fetch bundle.
   - Build briefing.
   - Build vars.
   - Run `scriptorium`.
   - Write Markdown report.

### Deliverables

- `weatherreporter generate daily --location home --out ./daily.md` produces a Markdown report.
- Failures include useful context.

### Tests

- Adapter command construction using a fake command runner or fake executable.
- Nonzero exit handling.
- Timeout behavior.
- App workflow test using fake weather client and fake `scriptorium` runner.

### Done Criteria

- The first report can be generated end-to-end.
- `scriptorium` is isolated behind the adapter package.

## Stage 8: Filesystem State Store and Metadata Persistence

### Goal

Persist report artifacts and metadata in a durable, inspectable structure.

### Packages Introduced or Expanded

- `internal/state`
- `internal/app`

### Work Items

1. Define state store interface.
2. Implement filesystem-backed store.
3. Persist briefing snapshots.
4. Persist prompt variable files.
5. Persist report metadata.
6. Persist rendered Markdown reports when output path is managed by the app.
7. Use atomic writes where practical.
8. Implement lookup for prior comparable snapshots.

### Deliverables

- Each generated report has associated metadata and briefing snapshot.
- Prior comparable snapshot lookup works for Daily Reports.

### Tests

- Atomic write behavior where feasible.
- Metadata round-trip.
- Snapshot path generation.
- Prior snapshot lookup.
- Narrow-path safety behavior.

### Done Criteria

- Daily reports leave enough state for inspection and future Recent Changes.
- State layout is predictable and documented.

## Stage 9: Recent Changes for Daily Reports

### Goal

Add structured comparison of Daily Report briefing snapshots and include meaningful changes in prompt variables.

### Packages Introduced or Expanded

- `internal/changes`
- `internal/state`
- `internal/promptvars`
- `internal/app`

### Work Items

1. Define change summary structures.
2. Define threshold configuration.
3. Implement Daily briefing comparison.
4. Compare current snapshot to prior comparable snapshot.
5. Detect meaningful changes, such as:
   - Temperature shifts.
   - Precipitation timing shifts.
   - Precipitation probability category changes.
   - Alert changes.
   - Wind gust changes.
   - Snow/ice/thunder risk changes.
6. Add Recent Changes to prompt vars.
7. Omit or minimize Recent Changes when no meaningful changes exist.

### Deliverables

- Daily Report vars include Recent Changes when appropriate.
- Daily Report generation persists current snapshot after comparison.

### Tests

- No prior snapshot behavior.
- No meaningful changes behavior.
- Temperature threshold crossing.
- Precipitation timing shift.
- Alert added/removed behavior.
- Comparison uses valid period, not just generation time.

### Done Criteria

- The Daily Report can say what changed relative to the prior report covering the same forecast period.
- Markdown report text is not used as the comparison source.

## Stage 10: Tomorrow Planning Brief

### Goal

Add the evening Tomorrow Planning Brief using the Daily Report machinery where practical.

### Packages Introduced or Expanded

- `internal/report`
- `internal/briefing`
- `internal/changes`
- `internal/app`

### Work Items

1. Implement the `daily_tomorrow` report definition fully.
2. Reuse or specialize the Daily briefing builder for tomorrow’s valid date.
3. Add any tomorrow-specific planning fields, such as:
   - Morning readiness note inputs.
   - Commute/school/workday concerns.
   - What may change overnight.
4. Ensure comparison can find a prior report covering the same valid day where appropriate.
5. Implement `run evening` as Daily Tomorrow.

### Deliverables

- `weatherreporter generate tomorrow --location home` works end-to-end.
- `weatherreporter run evening --location home` works.

### Tests

- Tomorrow valid-period calculation.
- Tomorrow briefing uses the correct date.
- Recent Changes can compare against prior 3-Day or prior Tomorrow snapshot if configured.
- Evening batch includes only the expected report.

### Done Criteria

- Evening look-ahead generation is reliable.
- Daily Today and Daily Tomorrow share logic without muddling their identities.

## Stage 11: 3-Day Outlook

### Goal

Add the 3-Day Outlook report using the same architecture.

### Packages Introduced or Expanded

- `internal/report`
- `internal/briefing`
- `internal/forecast`
- `internal/changes`
- `internal/app`

### Work Items

1. Implement 3-Day valid-period resolution.
2. Build a 3-Day briefing package.
3. Summarize each day:
   - Overall character.
   - Temperature range.
   - Precipitation/storm/winter/heat/wind risks.
   - Best/worst windows if derivable.
   - Relevant alerts.
4. Attach broader NWS context, especially forecast discussion and weather story inputs.
5. Implement 3-Day Recent Changes strategy.
6. Add end-to-end generation.

### Deliverables

- `weatherreporter generate three-day --location home` produces Markdown.
- Morning batch can include the 3-Day Outlook.

### Tests

- Three-day valid period.
- Daily aggregation across three days.
- Alert overlap across multi-day period.
- Recent Changes across multi-day snapshots.
- Quiet-weather behavior.

### Done Criteria

- The 3-Day Outlook is generated using the same registry, briefing, vars, state, and rendering pipeline as the Daily Report.

## Stage 12: Weekend Outlook

### Goal

Add the Weekend Outlook with day-of-week-sensitive period behavior.

### Packages Introduced or Expanded

- `internal/report`
- `internal/briefing`
- `internal/forecast`
- `internal/changes`
- `internal/app`

### Work Items

1. Implement Weekend valid-period resolution:
   - Monday through Thursday: upcoming Saturday/Sunday, optionally Friday evening if configured.
   - Friday: Friday evening through Sunday night.
   - Saturday: remaining weekend.
   - Sunday: normally not generated by the scheduled morning batch.
2. Build Weekend briefing package.
3. Emphasize planning fields:
   - Best outdoor windows.
   - Worst weather windows.
   - Rain/storm timing.
   - Heat/cold/wind comfort.
   - Confidence and uncertainty inputs.
4. Implement Weekend Recent Changes strategy.
5. Add morning batch inclusion except Sunday.

### Deliverables

- `weatherreporter generate weekend --location home` produces Markdown.
- `weatherreporter run morning --location home` includes Weekend Outlook except Sunday.

### Tests

- Weekend period on each day of the week.
- Saturday remaining-weekend behavior.
- Sunday skip behavior in morning batch.
- Recent Changes across narrowed valid periods.
- Outdoor-window derivation behavior.

### Done Criteria

- Weekend Outlook works end-to-end and follows expected scheduling behavior.

## Stage 13: Morning and Evening Batch Hardening

### Goal

Make scheduled workflows reliable enough for unattended execution by cron, systemd timers, or another orchestrator.

### Packages Introduced or Expanded

- `internal/app`
- `internal/cli`
- `internal/state`
- `internal/config`

### Work Items

1. Finalize `run morning` workflow.
2. Finalize `run evening` workflow.
3. Decide failure behavior:
   - Continue remaining reports after one report fails, or fail fast.
   - Return aggregate status.
4. Add structured run summaries.
5. Ensure each report run records enough metadata for troubleshooting.
6. Add CLI flags for output directory, location, and optional dry-run/vars-only mode if desired.
7. Add logging suitable for scheduled execution.

### Deliverables

- Morning batch can generate Daily, 3-Day, and Weekend reports.
- Evening batch can generate Tomorrow Planning Brief.
- Failures are understandable from logs and metadata.

### Tests

- Morning batch report selection.
- Evening batch report selection.
- Partial failure behavior.
- Output path behavior.
- Dry-run or vars-only behavior, if implemented.

### Done Criteria

- The scheduled report system is ready for real daily use.

## Stage 14: Manual Storm Report

### Goal

Add manual Storm Report generation without building the automatic monitoring agent yet.

### Packages Introduced or Expanded

- `internal/report`
- `internal/briefing`
- `internal/forecast`
- `internal/app`
- `internal/changes`, if needed

### Work Items

1. Implement Storm Report definition.
2. Define storm valid-period behavior.
3. Build storm briefing package from:
   - Active alerts.
   - Forecast discussion.
   - Weather story.
   - Relevant hourly/daily periods.
   - NWS narrative periods.
4. Include storm-specific fields:
   - Event headline inputs.
   - Timing window.
   - Hazards.
   - Most likely scenario inputs.
   - Reasonable worst-case inputs, if supported by source context.
   - Confidence and uncertainty inputs.
   - What to watch next.
5. Add manual command:
   - `weatherreporter generate storm --location home`

### Deliverables

- Manual Storm Report generation works end-to-end.
- No automatic agent behavior is required.

### Tests

- Storm briefing with active alerts.
- Storm briefing with forecast discussion but no active alert.
- Quiet/no-storm behavior.
- Relevant source selection.

### Done Criteria

- A user can manually generate a focused Storm Report when desired.
- The implementation reuses the same pipeline rather than creating a separate flow.

## Stage 15: Inspection and Debugging Tools

### Goal

Make generated artifacts easy to inspect and debug.

### Packages Introduced or Expanded

- `internal/cli`
- `internal/app`
- `internal/state`

### Work Items

1. Add inspection commands as needed:
   - List recent reports.
   - Show metadata for a report.
   - Show prior comparable snapshot chosen for Recent Changes.
   - Emit briefing JSON without rendering.
   - Emit prompt vars without rendering.
2. Add clear paths to generated artifacts in command output.
3. Ensure logs do not dump large weather payloads by default.

### Deliverables

- Developers can inspect why a report was generated a certain way.
- Recent Changes comparison inputs are discoverable.

### Tests

- Inspection command behavior with fixture state.
- Missing artifact errors.
- Metadata lookup behavior.

### Done Criteria

- Debugging a bad report does not require stepping through the whole workflow manually.

## Stage 16: Future Storm Monitoring Agent

### Goal

Add automatic storm-event evaluation only after scheduled reports and manual storm reports are stable.

### Packages Introduced or Expanded

Potentially:

- `internal/storm` or expanded `internal/report`/`internal/briefing`
- `internal/adapters/scriptorium` or a separate evaluator prompt adapter
- `internal/state`
- `internal/app`

### Work Items

1. Implement deterministic candidate detection.
2. Define storm candidate structures.
3. Use source signals such as:
   - Active alerts.
   - Forecast discussion hazard wording.
   - Weather Story emphasis.
   - Hourly/daily threshold crossings.
   - Material forecast changes toward higher impact.
4. Add an LLM event evaluator through `scriptorium` or a future native LLM adapter.
5. Track storm lifecycle state:
   - `none`
   - `monitoring`
   - `active_report`
   - `escalated`
   - `deescalating`
   - `resolved`
6. Generate or update Storm Reports only when warranted.
7. Avoid noisy report generation for ordinary low-impact thunder chances.

### Deliverables

- `weatherreporter evaluate storm --location home` can decide whether a Storm Report is warranted.
- Event lifecycle state is persisted.
- Storm Report updates are generated only for meaningful changes.

### Tests

- Candidate detection thresholds.
- Noisy/non-event suppression.
- Alert-triggered escalation.
- Lifecycle transitions.
- Evaluator failure behavior.

### Done Criteria

- Automatic storm monitoring is useful and not spammy.
- Manual Storm Report generation remains available.

## Suggested First Implementation Milestone

The first meaningful milestone should be:

```text
weatherreporter generate daily --location home --out ./daily.md
```

This command should:

1. Load config.
2. Fetch weather data.
3. Build a Daily briefing package.
4. Build prompt variables.
5. Invoke `scriptorium`.
6. Write Markdown output.
7. Persist metadata and snapshots.

Do not implement all report types before this milestone. One complete vertical slice will reveal schema, state, prompt-variable, and adapter issues earlier than a broad but shallow implementation.

## Suggested Second Implementation Milestone

The second milestone should be:

```text
weatherreporter run morning --location home
weatherreporter run evening --location home
```

At this point, the app should support:

- Daily Today.
- Daily Tomorrow.
- 3-Day Outlook.
- Weekend Outlook.
- Recent Changes for all scheduled report types.
- Filesystem state and metadata.
- Reliable scheduled execution.

## Suggested Third Implementation Milestone

The third milestone should be:

```text
weatherreporter generate storm --location home
```

At this point, Storm Reports can be generated manually using the same pipeline. Automatic storm monitoring should remain future work until manual Storm Reports are useful and stable.

## Non-Goals for the Initial Prototype

Do not implement these in the initial prototype unless required by real use:

- Native LLM client inside `weatherreporter`.
- Daemon mode.
- Automatic storm-monitoring agent.
- Database-backed state.
- Multi-user authorization.
- Public HTTP API.
- Complex plugin system.
- Markdown diffing for Recent Changes.
- Raw unbounded weather payloads sent directly to prompts.

## Final Implementation Notes

The most important early design decision is to make the briefing package the central artifact. Once the briefing package is stable, every report follows the same basic path:

```text
report definition -> valid period -> forecast selection -> briefing package -> Recent Changes -> prompt vars -> scriptorium -> report metadata
```

This keeps the application modular, testable, and easy to extend with future report types.