From 05b56d6ea64aa46a84ba63e32b9b1afc632b7d2d Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 29 May 2026 11:41:50 -0500 Subject: [PATCH] Finalized the initial development roadmap --- docs/policy/development.md | 212 +++++++++++------- docs/roadmap/initial.md | 438 +++++++++++++++++++++++-------------- 2 files changed, 413 insertions(+), 237 deletions(-) diff --git a/docs/policy/development.md b/docs/policy/development.md index 980e68f..da1fc94 100644 --- a/docs/policy/development.md +++ b/docs/policy/development.md @@ -8,12 +8,12 @@ The application should remain a small, explicit, dependency-light Go program. Do `weatherreporter` is a deterministic weather briefing and report-preparation application. It should: -1. Fetch normalized weather data from an internal weather API backed by `weatherfeeder`. +1. Fetch normalized weather data from a single configured internal weather API endpoint backed by `weatherfeeder`. 2. Derive report-specific briefing packages from the normalized forecast bundle. 3. Compare current briefing snapshots against prior comparable snapshots to produce optional Recent Changes. -4. Build structured prompt variables for a specific report type. +4. Build structured prompt input data packages for a specific report type. 5. Invoke `scriptorium` as an external prompt runner. -6. Persist the rendered Markdown report, briefing snapshot, and generation metadata. +6. Persist the rendered Markdown report, briefing snapshot, prompt input package, and generation metadata. The preferred data flow is: @@ -23,7 +23,7 @@ weatherfeeder-backed internal API -> forecast bundle -> report-specific briefing builder -> recent-change comparison - -> prompt variable package + -> prompt input data package -> scriptorium subprocess adapter -> Markdown report + metadata + stored snapshot ``` @@ -96,7 +96,7 @@ internal/state/ filesystem.go metadata.go -internal/promptvars/ +internal/promptinput/ build.go schema.go @@ -167,7 +167,8 @@ Responsibilities: - `weatherreporter run morning` - `weatherreporter run evening` - `weatherreporter inspect snapshot` -- Parse CLI flags and convert them into app-layer request structs. +- Use the Go standard library for CLI parsing unless future complexity justifies a dependency. +- Parse flags such as `--config`, `--units`, `--tz`, `--out`, optional Daily `--date`, and storm `--start`/`--end`, then convert them into app-layer request structs. - Load configuration through `internal/config`. - Present concise user-facing errors. @@ -181,15 +182,19 @@ Non-responsibilities: Suggested command shape: ```text -weatherreporter generate daily --location home --date today --out ./daily.md -weatherreporter generate tomorrow --location home --out ./tomorrow.md -weatherreporter generate three-day --location home --out ./three_day.md -weatherreporter generate weekend --location home --out ./weekend.md -weatherreporter generate storm --location home --out ./storm.md -weatherreporter run morning --location home -weatherreporter run evening --location home +weatherreporter generate daily --date 2026-05-29 --out ./daily.md +weatherreporter generate tomorrow --out ./tomorrow.md +weatherreporter generate three-day --out ./three_day.md +weatherreporter generate weekend --out ./weekend.md +weatherreporter generate storm --start 2026-05-29T18:00 --end 2026-05-30T06:00 --out ./storm.md +weatherreporter run morning +weatherreporter run evening ``` +The MVP should not expose location selection. Source `locationId` and `locationName` values returned by the weather API may be retained as provenance. + +For `generate daily`, `--date` is optional. When provided, it must use `YYYY-MM-DD`; when omitted, it resolves to the current local date in the configured timezone. + ### `internal/config` Owns configuration structures, defaults, loading, precedence, and validation. @@ -199,20 +204,29 @@ Responsibilities: - Define application configuration structs. - Provide built-in defaults in `defaults.go`. - Load YAML configuration from `/usr/local/etc/weatherreporter/config.yml` or a CLI-supplied path. +- Use `gopkg.in/yaml.v3` for YAML parsing. - Apply precedence rules. - Validate required settings. -- Normalize paths, durations, report settings, locations, and daypart definitions. +- Normalize paths, durations, report settings, weather API units/timezone, missing-source policy, and daypart definitions. Suggested configuration areas: -- Weather API base URL, timeout, and location endpoints. -- Locations and time zones. -- `scriptorium` binary, profile, timeout, and optional extra arguments. +- Weather API base URL, timeout, units, timezone, precision, and missing-source policy. +- `scriptorium` binary, config path, profile, timeout, and optional extra arguments. - Workspace and output directories. - Report enablement and output naming. - Daypart definitions. - Recent-change thresholds. +Initial defaults: + +- Weather API units: `us`. +- Weather API timezone: `Chicago`. +- Weather API format: `json`. +- Missing-source policy: `warn`. + +Missing-source policy should support a global default and per-source overrides. Valid policy values are `error`, `warn`, and `none`. + Non-responsibilities: - No command execution. @@ -230,7 +244,7 @@ Responsibilities: - Run the morning batch. - Run the evening batch. - Generate a manual storm report. -- Coordinate config, weather API adapter, report registry, briefing builders, state store, change comparison, prompt variable builder, and `scriptorium` runner. +- Coordinate config, weather API adapter, report registry, briefing builders, state store, change comparison, prompt input builder, and `scriptorium` runner. - Enforce workflow order. - Ensure each generation run persists enough artifacts for inspection and future comparison. @@ -238,15 +252,16 @@ The core generation workflow should be approximately: ```text resolve report definition -resolve location and valid period +resolve valid period fetch current weather bundle build current briefing package load prior comparable briefing snapshot compute recent changes -build prompt variables -write vars file -invoke scriptorium -persist report metadata and briefing snapshot +build prompt input data package +write data package +run scriptorium render preflight +invoke scriptorium run +persist report metadata, briefing snapshot, data package, preflight output, and rendered report ``` Non-responsibilities: @@ -261,25 +276,37 @@ HTTP adapter for the internal weather API backed by `weatherfeeder`. Responsibilities: -- Fetch normalized weather data for a configured location. +- Fetch normalized weather data from the configured API base URL. +- Fan out to multiple weather API endpoints and assemble one internal `forecast.Bundle`. - Decode API responses into adapter-owned DTOs or directly into stable internal types if those types are intentionally owned by `weatherreporter`. - Apply request timeouts and context cancellation. +- Apply configured query defaults, including `format=json`, `units=us`, and `tz=Chicago` unless overridden. +- Fetch full `/forecast/hourly` and `/forecast/narrative` products, not day-slice endpoints, so Go domain code owns report-period selection. +- Record per-source provenance: endpoint, query, fetch time, issued/updated time when available, SHA-256 over canonical/minified raw `data` JSON, warnings, and missing-source status. +- Represent source warnings as first-class records with source name, code, severity, message, endpoint, and completeness impact. +- Require hourly forecast data for normal scheduled reports. +- Apply missing-source policy for `data:null`, malformed non-required sections, or unavailable upstream products. - Return actionable errors containing endpoint and operation context. -Expected data categories: +Initial data categories: +- Latest observation. +- Current conditions. - Hourly forecast data. -- Daily forecast data. - NWS narrative forecast periods. - NWS alerts. - NWS forecast discussion. + +Stubbed source slots until upstream support exists: + +- Daily forecast data. - NWS weather story. Non-responsibilities: - No daypart grouping. - No Recent Changes comparison. -- No prompt variable construction. +- No prompt input construction. - No `scriptorium` calls. ### `internal/adapters/scriptorium` @@ -292,24 +319,32 @@ Responsibilities: ```go type Runner interface { + Render(ctx context.Context, req RenderRequest) (*RenderResult, error) Run(ctx context.Context, req RunRequest) (*RunResult, error) } ``` -- Execute `scriptorium run` with `exec.CommandContext`. +- Execute `scriptorium render` for preflight/debug output without LLM generation. +- Execute `scriptorium run` for report generation. +- Run `scriptorium render` as an always-on preflight before `scriptorium run` for MVP generated reports. - Pass arguments as an argv slice, not through a shell. -- Prefer a vars file path over large inline JSON. +- Pass large prompt input as `--input data_package=`. - Capture stdout/stderr with reasonable size limits. -- Treat nonzero exits as actionable errors. +- Treat nonzero exits as actionable errors, including exit code `2` from `run`, which may still produce output. - Keep all `scriptorium`-specific flag details inside the adapter. -Suggested command form: +Suggested command forms: ```text +scriptorium render \ + --prompt weather.daily_report \ + --input data_package=./workspace/data-packages/daily/2026-05-29T050000-0500.data_package.json \ + --format json + scriptorium run \ --prompt weather.daily_report \ - --vars-file ./workspace/daily.vars.json \ - --out ./workspace/daily_report.md + --input data_package=./workspace/data-packages/daily/2026-05-29T050000-0500.data_package.json \ + --out ./workspace/reports/daily/2026-05-29T050000-0500.md ``` Non-responsibilities: @@ -365,12 +400,14 @@ Suggested report definitions: ```text daily_today -> prompt weather.daily_report -daily_tomorrow -> prompt weather.daily_report or weather.tomorrow_report +daily_tomorrow -> prompt weather.daily_report three_day -> prompt weather.three_day_outlook weekend -> prompt weather.weekend_outlook storm -> prompt weather.storm_report ``` +`weather.daily_report` should be the standard prompt for one local civil day, regardless of whether that day is today or tomorrow. + A report definition should describe: - Report ID. @@ -395,9 +432,10 @@ Builds report-specific briefing packages from forecast bundles and report defini Responsibilities: - Convert a forecast bundle into a report-specific structured briefing package. -- Keep each report’s briefing shape explicit and testable. -- Attach relevant NWS narrative periods, alerts, forecast discussion context, and weather story context. -- Provide inputs suitable for LLM prompt variables. +- Keep each report's briefing shape explicit and testable. +- Attach relevant NWS narrative periods, alerts, forecast discussion context, and weather story context when available. +- Include metadata such as schema version, configured units/timezone, source warnings, and source provenance. +- Provide inputs suitable for `scriptorium` data packages. Report-specific builders should exist for: @@ -425,9 +463,11 @@ Responsibilities: - Compare current briefing packages against prior comparable snapshots. - Apply meaningful-change thresholds. -- Produce compact structured change summaries for prompt variables. +- Produce compact structured change summaries for prompt input data packages. - Avoid comparison of rendered Markdown report text. +Comparable snapshot matching should be declared by each report definition. Daily Today, Daily Tomorrow, and compatible date slices from multi-day reports may compare by same valid local date when the report registry marks them compatible. Weekend compares by same weekend window. Storm compares by explicit event window. + Meaningful changes may include: - Temperature changes crossing configured thresholds. @@ -447,14 +487,17 @@ Non-responsibilities: ### `internal/state` -Durable state store for reports, snapshots, metadata, and comparison lookup. +Durable state store for reports, snapshots, data packages, preflight output, metadata, and comparison lookup. Responsibilities: - Persist generated report metadata. - Persist briefing snapshots. -- Persist prompt variable files when useful for inspection. +- Persist prompt input data packages. +- Persist `scriptorium render` preflight output for generated reports. - Locate prior comparable snapshots for Recent Changes. +- Track RunID as generation timestamp plus report ID. +- Use timestamped managed report names to avoid overwriting prior runs for the same valid period. - Use atomic writes where practical. - Keep filesystem layout narrow and predictable. @@ -470,25 +513,26 @@ Suggested state layout: ```text workspace/ - locations/ - home/ - snapshots/ - daily/ - 2026-05-30/ - 2026-05-29T050000-0500.briefing.json - 2026-05-29T050000-0500.metadata.json - three-day/ - weekend/ - storm/ - reports/ - daily/ - 2026-05-30.md - three-day/ - weekend/ - storm/ - vars/ - daily/ - 2026-05-29T050000-0500.vars.json + snapshots/ + daily/ + 2026-05-30/ + 2026-05-29T050000-0500.briefing.json + 2026-05-29T050000-0500.metadata.json + three-day/ + weekend/ + storm/ + reports/ + daily/ + 2026-05-29T050000-0500.md + three-day/ + weekend/ + storm/ + data-packages/ + daily/ + 2026-05-29T050000-0500.data_package.json + preflight/ + daily/ + 2026-05-29T050000-0500.render.json ``` Non-responsibilities: @@ -497,16 +541,16 @@ Non-responsibilities: - No report prose generation. - No CLI formatting decisions. -### `internal/promptvars` +### `internal/promptinput` -Builds the final variable payload passed to `scriptorium`. +Builds the final data package passed to `scriptorium`. Responsibilities: -- Combine report metadata, briefing package, Recent Changes, and selected source context into a prompt variable document. -- Validate required prompt variables before invoking `scriptorium`. -- Keep prompt variable schemas explicit enough to test. -- Write vars files to the workspace when requested by the app layer. +- Combine report metadata, briefing package, Recent Changes, selected source context, and source warnings into a prompt input document. +- Validate required data package fields before invoking `scriptorium`. +- Keep data package schemas explicit enough to test. +- Write data package files to the workspace when requested by the app layer. Non-responsibilities: @@ -521,7 +565,7 @@ Time, clock, and period helpers. Responsibilities: - Provide an injectable clock for deterministic tests. -- Resolve local dates using the configured location time zone. +- Resolve local dates using the configured report timezone. - Handle daypart spans, including overnight windows. - Normalize valid periods. - Provide helpers for recurring scheduled batches. @@ -535,24 +579,38 @@ Non-responsibilities: Each generated report must be associated with explicit metadata: +- RunID. - Report type. - Report variant, if applicable. -- Location ID. - Generation time. +- Configured report timezone. - Valid period start. - Valid period end. -- Source product timestamps and/or hashes. +- Source location ID/name when provided by upstream. +- Source product timestamps and/or SHA-256 hashes. +- Source warnings. - Briefing snapshot path. -- Prompt variable path. +- Prompt input data package path. +- Preflight output path. - Rendered report path. +All valid periods should use the configured local timezone, default `Chicago`, and half-open `[start,end)` intervals. + +Initial valid-period rules: + +- Daily Today: current local civil day, `[00:00, next 00:00)`. +- Daily Tomorrow: next local civil day. +- 3-Day Outlook: generation time through local midnight after the second following local civil day. +- Weekend Outlook: Monday through Thursday covers Saturday 00:00 to Monday 00:00; Friday and Saturday cover `max(generation time, Friday 18:00)` to Monday 00:00; scheduled Sunday morning skips Weekend Outlook. +- Manual Storm Report: requires explicit `--start` and `--end`; accept `YYYY-MM-DDTHH:MM` interpreted in the configured timezone and RFC3339 timestamps with explicit offsets. + The valid period should identify what weather period the report covers, independent of when the report was generated. Examples: - A 5 PM Tomorrow Planning Brief for Saturday and a 5 AM Saturday Daily Report both cover the same valid date. - A Saturday Weekend Outlook covers the remaining weekend, while a Friday Weekend Outlook may cover Friday evening through Sunday night. -- A Storm Report covers a forecast event window, not a fixed calendar day. +- A Storm Report covers an explicit forecast event window, not a fixed calendar day. This identity is required for reliable Recent Changes behavior. @@ -572,6 +630,8 @@ evening: - daily_tomorrow ``` +Scheduled batches should continue independent reports after a report failure. The CLI should return nonzero if any report failed and should emit an aggregate run summary. + External scheduling should be handled by systemd timers, cron, or another orchestrator. `weatherreporter` should simply provide deterministic commands that can be scheduled. ## Storm Report Direction @@ -579,7 +639,7 @@ External scheduling should be handled by systemd timers, cron, or another orches The initial version should support manual Storm Report generation: ```text -weatherreporter generate storm --location home +weatherreporter generate storm --start 2026-05-29T18:00 --end 2026-05-30T06:00 ``` Future storm monitoring should use a staged design: @@ -606,14 +666,16 @@ Core tests should not require real external services. Priority test areas: -- Configuration loading and validation. +- Configuration loading and validation, including defaults for `units=us`, `tz=Chicago`, and missing-source policy `warn`. +- Standard-library CLI command parsing, including `--units`, `--tz`, and storm `--start`/`--end`. +- Weather API fan-out, source provenance, `data:null`, and missing-source policy behavior. - Daypart grouping, especially overnight periods. - Valid-period resolution for each report type. - Briefing package construction from fixtures. -- Recent Changes threshold behavior. +- Recent Changes threshold behavior and compatible snapshot matching. - Prior snapshot lookup. -- `scriptorium` adapter behavior using a fake executable or command runner. -- CLI command parsing for major workflows. +- `scriptorium` adapter behavior using a fake executable or command runner, including both `render` and `run` with `--input data_package=`. +- Batch partial-failure behavior and aggregate exit status. ## Design Invariants diff --git a/docs/roadmap/initial.md b/docs/roadmap/initial.md index e0e80c6..5b35389 100644 --- a/docs/roadmap/initial.md +++ b/docs/roadmap/initial.md @@ -7,12 +7,26 @@ The goal is to build the application in stable layers. Each stage should leave t ## Guiding Implementation Principles - Build deterministic data preparation before LLM rendering. -- Keep CLI, adapters, domain logic, state, and prompt-variable construction separate. +- Keep CLI, adapters, domain logic, state, and prompt input construction separate. - Use fixture-driven tests for forecast processing and briefing builders. - Persist intermediate artifacts so failed or low-quality reports can be inspected. - Add one report type fully before generalizing to all report types. - Treat `scriptorium` as an external adapter during the prototype. - Do not build the storm-monitoring agent until scheduled report generation is reliable. +- Do not support multiple weatherreporter locations in the MVP; use one configured Weather API endpoint. + +## MVP Decisions Locked + +- Use one configured Weather API endpoint; do not expose MVP location selection. +- Use the Go standard library for CLI parsing. +- Use `gopkg.in/yaml.v3` for YAML configuration. +- Default Weather API query values are `format=json`, `units=us`, and `tz=Chicago`. +- Default missing-source policy is `warn`. +- Fetch full `/forecast/hourly` and `/forecast/narrative` products; Go owns report-period selection. +- Require hourly forecast data for normal scheduled reports. +- Use `scriptorium render` as an always-on preflight before `scriptorium run` for generated reports. +- Pass prompt input to `scriptorium` with `--input data_package=`. +- Identify source payloads with SHA-256 over canonical/minified raw `data` JSON. ## Stage 0: Repository Skeleton and Architecture Baseline @@ -35,7 +49,7 @@ Create the project skeleton, commit the architecture documents, and establish th 4. Add this implementation roadmap. 5. Create minimal package directories and placeholder files where useful. 6. Add basic build/test tooling. -7. Add a minimal `weatherreporter --help` command. +7. Add a minimal `weatherreporter --help` command using the Go standard library. ### Deliverables @@ -66,20 +80,24 @@ Implement configuration loading and a stable command shape before integrating ex ### Work Items 1. Define configuration structs for: - - Weather API settings. - - Locations. - - Location time zones. + - Weather API base URL, timeout, precision, units, and timezone. + - Missing-source behavior, with a global default and optional per-source overrides. - `scriptorium` settings. - Workspace paths. - Report output paths. - Daypart definitions. - Recent-change thresholds. -2. Implement built-in defaults in `internal/config/defaults.go`. +2. Implement built-in defaults in `internal/config/defaults.go`: + - `units=us` + - `tz=Chicago` + - `format=json` + - missing-source policy `warn` 3. Implement YAML config loading from: - `/usr/local/etc/weatherreporter/config.yml` - CLI override via `--config` -4. Implement config validation. -5. Implement basic command structure: +4. Use `gopkg.in/yaml.v3` for YAML parsing. +5. Implement config validation. +6. Implement basic command structure with the Go standard library: - `generate daily` - `generate tomorrow` - `generate three-day` @@ -87,27 +105,35 @@ Implement configuration loading and a stable command shape before integrating ex - `generate storm` - `run morning` - `run evening` -6. Commands may initially return “not implemented” after config and request resolution. -7. Add time-zone and clock helpers. +7. Add CLI flag parsing for: + - `--config` + - `--units` + - `--tz` + - `--out` + - optional `--date` for Daily Today, accepting `YYYY-MM-DD` and defaulting to the current local date in the configured timezone + - `--start` and `--end` for manual Storm Reports +8. Commands may initially return "not implemented" after config and request resolution. +9. Add time-zone and clock helpers. ### Deliverables - Config can be loaded, validated, and inspected in tests. -- CLI commands parse expected flags. +- CLI commands parse expected flags without a CLI framework dependency. - App-layer request structs exist for report generation and scheduled batches. ### Tests -- Config defaults load successfully. +- Config defaults load successfully, including `units=us`, `tz=Chicago`, and missing-source policy `warn`. - Example config file load test. - Invalid config produces actionable errors. -- CLI parser tests for major commands. +- CLI parser tests for major commands and shared flags. +- Storm `--start`/`--end` parsing tests. - Time-zone resolution tests. ### Done Criteria - CLI request handling is stable enough that later stages can attach real behavior without reshaping commands. -- Configuration can represent at least one location and the default daypart set. +- Configuration can represent the single Weather API endpoint, default units/timezone, default daypart set, and missing-source policy. ## Stage 2: Weather API Adapter and Forecast Bundle @@ -117,8 +143,8 @@ Fetch normalized weather data from the internal weather API and represent it as ### Key References -- `docs/integrations/weatherapi.md` describes the weatherapi public API -- The local API endpoint is available at `https://weather.api.rakestrawhome.com/` and will return live data +- `docs/integrations/weatherapi.md` describes the weatherapi public API. +- The local API endpoint is available at `https://weather.api.rakestrawhome.com/` and will return live data. ### Packages Introduced or Expanded @@ -130,30 +156,57 @@ Fetch normalized weather data from the internal weather API and represent it as 1. Define the internal `forecast.Bundle` type. 2. Define source substructures for: + - Latest observation. + - Current conditions. - Hourly forecast data. - - Daily forecast data (NOTE: not yet implemented upstream in weatherapi, so this can remain a stub in the initial implementation). - NWS narrative forecast periods. - NWS alerts. - NWS forecast discussion. - - NWS weather story (NOTE: not yet implemented upstream in weatherapi, so this can remain a stub in the initial implementation). -3. Implement the weather API client. -4. Add context-aware HTTP calls and timeouts. -5. Add actionable errors for failed API calls and decode failures. -6. Add fixture support for tests. -7. Optionally add a debug command or app method to fetch and save the raw normalized bundle. + - Daily forecast data as a stub source slot until upstream support exists. + - NWS weather story as a stub source slot until upstream support exists. +3. Implement the weather API client as a fan-out adapter that assembles one bundle from multiple endpoints. +4. Fetch full `/forecast/hourly` and `/forecast/narrative` products, not day-slice endpoints, so Go domain code owns report-period selection. +5. Apply configured query defaults to requests: + - `format=json` + - `units=us`, unless overridden + - `tz=Chicago`, unless overridden on endpoints that support timezone +6. Add context-aware HTTP calls and timeouts. +7. Add source provenance to the bundle: + - endpoint + - query + - fetched time + - issued/updated time when available + - SHA-256 over canonical/minified raw `data` JSON + - warnings +8. Represent source warnings as first-class records with source name, code, severity, message, endpoint, and completeness impact. +9. Require hourly forecast data for normal scheduled reports. +10. Apply configured missing-source policy to missing observations, current conditions, alerts, discussion, daily forecast stub, weather story stub, and malformed non-required source sections: + - `error`: fail the bundle fetch + - `warn`: include a source warning and continue + - `none`: omit the warning and continue +11. Add actionable errors for failed API calls and decode failures. +12. Add fixture support for tests. +13. Add a debug command or app method to fetch and save the raw normalized bundle for fixture capture and inspection. ### Deliverables -- Weather API adapter can fetch a bundle for a configured location. +- Weather API adapter can fetch and assemble a bundle from the configured endpoint. - Forecast bundle type is available to downstream packages. +- Bundle source provenance and warnings are inspectable. - Tests can use fixtures without real API calls. ### Tests -- Decode representative API fixture into `forecast.Bundle`. +- Decode representative API fixtures into `forecast.Bundle`. +- Fan-out success across source endpoints. - HTTP error handling. - Timeout/cancellation behavior. +- `data:null` behavior under `error`, `warn`, and `none`. - Missing or malformed source sections. +- Query construction for `units=us` and `tz=Chicago`. +- Full-product forecast endpoint selection. +- Required hourly forecast behavior. +- SHA-256 source identity behavior. ### Done Criteria @@ -211,7 +264,7 @@ Implement deterministic forecast processing needed by the Daily Report. ### Goal -Centralize report definitions and valid-period behavior before building report-specific briefings. +Centralize report definitions, compatible comparison strategies, and valid-period behavior before building report-specific briefings. ### Packages Introduced or Expanded @@ -230,19 +283,24 @@ Centralize report definitions and valid-period behavior before building report-s 2. Define report metadata structures. 3. Define a report `Definition` contract. 4. Implement a registry. -5. Implement valid-period resolvers: - - Today Daily Report. - - Tomorrow Planning Brief. - - 3-Day Outlook. - - Weekend Outlook. - - Storm Report placeholder. -6. Define default prompt IDs: +5. Implement valid-period resolvers using the configured local timezone, default `Chicago`, and half-open `[start,end)` intervals: + - Today Daily Report: current local civil day, `[00:00, next 00:00)`. + - Tomorrow Planning Brief: next local civil day. + - 3-Day Outlook: generation time through local midnight after the second following local civil day. + - Weekend Outlook: Monday through Thursday covers Saturday 00:00 to Monday 00:00; Friday and Saturday cover `max(generation time, Friday 18:00)` to Monday 00:00; scheduled Sunday morning skips Weekend Outlook. + - Storm Report: explicit `--start` and `--end`. +6. Parse manual storm period bounds from `YYYY-MM-DDTHH:MM` in the configured timezone or RFC3339 timestamps with explicit offsets. +7. Define default prompt IDs: - `weather.daily_report` - - `weather.tomorrow_report`, or reuse `weather.daily_report` if preferred. + - `weather.daily_report` for `daily_tomorrow`; this prompt covers one civil day. - `weather.three_day_outlook` - `weather.weekend_outlook` - `weather.storm_report` -7. Implement batch membership rules: +8. Define Recent Changes matching strategies in report definitions: + - Daily Today, Daily Tomorrow, and compatible date slices from multi-day reports may compare by same valid local date when marked compatible. + - Weekend compares by same weekend window. + - Storm compares by explicit event window. +9. Implement batch membership rules: - Morning batch: Daily Today, 3-Day Outlook, Weekend Outlook except Sunday. - Evening batch: Daily Tomorrow. @@ -250,20 +308,21 @@ Centralize report definitions and valid-period behavior before building report-s - App layer can resolve which reports should run for a command. - Each report has a valid period independent of generation time. -- Report definitions map to prompt IDs. +- Report definitions map to prompt IDs and comparison strategies. ### Tests - Daily valid period for different generation times. - Tomorrow valid period from evening generation. -- 3-day period calculation. +- 3-Day period calculation from generation time. - Weekend period calculation on Monday, Friday, Saturday, and Sunday. +- Storm manual period parsing and validation. - Morning batch skips Weekend Outlook on Sunday. - Registry lookup errors are actionable. ### Done Criteria -- Report identity and period behavior are stable. +- Report identity, period behavior, and compatible comparison rules are stable. - Later stages can add builders without changing CLI semantics. ## Stage 5: Daily Briefing Builder @@ -282,12 +341,15 @@ Build the first complete report-specific briefing package without invoking the L ### Work Items 1. Define common briefing metadata: + - Schema version. + - RunID. - Report type. - Variant. - - Location. - Generation time. + - Configured units and timezone. - Valid start/end. - - Source timestamps or hashes. + - Source location ID/name when provided by upstream. + - Source timestamps, SHA-256 hashes, and source warnings. 2. Define the Daily Report briefing schema. 3. Build Daily Report briefing content: - Bottom-line inputs. @@ -296,7 +358,7 @@ Build the first complete report-specific briefing package without invoking the L - Best/worst outdoor window inputs, if derivable. - NWS narrative periods relevant to the day. - Forecast discussion summary or selected text from the API data. - - Weather story summary or selected text from the API data. + - Weather story summary or selected text when upstream support exists. 4. Add JSON output for the briefing package. 5. Add an app workflow that can generate the Daily briefing and write it to disk for inspection. @@ -310,6 +372,8 @@ Build the first complete report-specific briefing package without invoking the L - Daily briefing from representative fixture. - Alerts included/excluded correctly. - Source context selection. +- Source warnings included correctly. +- RunID included correctly. - Empty or quiet-weather behavior. - Snapshot metadata completeness. @@ -318,97 +382,69 @@ Build the first complete report-specific briefing package without invoking the L - The Daily briefing package is useful as prompt input. - The app can produce the briefing artifact from real or fixture data. -## Stage 6: Prompt Variable Builder +## Stage 6: Prompt Input Package and Scriptorium Render Preflight ### Goal -Convert a briefing package into the structured variable payload expected by `scriptorium` prompts. - -### Packages Introduced or Expanded - -- `internal/promptvars` -- `internal/briefing` -- `internal/report` -- `internal/app` - -### Work Items - -1. Define prompt variable schema structures. -2. Build variables from report metadata and briefing content. -3. Include placeholders for Recent Changes, initially empty. -4. Validate required fields before rendering. -5. Write vars JSON to the workspace. -6. Ensure vars output is stable and inspectable. - -### Deliverables - -- Daily Report prompt vars can be generated and written to a file. -- The vars file is suitable for `scriptorium run --vars-file`. - -### Tests - -- Prompt vars generated from Daily briefing fixture. -- Missing required fields fail validation. -- JSON output is deterministic where practical. - -### Done Criteria - -- The app can prepare a complete vars file for a Daily Report. -- LLM rendering is the only missing step for the first end-to-end report. - -## Stage 7: Scriptorium Adapter and First End-to-End Daily Report - -### Goal - -Invoke `scriptorium` as a subprocess and produce the first rendered Markdown report. +Convert a briefing package into the structured `data_package` input expected by `scriptorium` prompts and validate prompt wiring without LLM generation. ### Key References - - `docs/integrations/scriptorium.md` describes the CLI contract for running `scriptorium` as a subprocess. + +- `docs/integrations/scriptorium.md` describes the CLI contract for `scriptorium render`. ### Packages Introduced or Expanded +- `internal/promptinput` +- `internal/briefing` +- `internal/report` - `internal/adapters/scriptorium` - `internal/app` -- `internal/state`, minimally if needed for output paths ### Work Items -1. Define `scriptorium.Runner` interface and request/result types. -2. Implement subprocess execution with `exec.CommandContext`. -3. Pass arguments without shell interpolation. -4. Prefer `--vars-file` for prompt variables. -5. Capture stderr and stdout with reasonable limits. -6. Apply timeout and cancellation. -7. Return actionable errors for nonzero exits. -8. Wire Daily Report generation end-to-end: - - Fetch bundle. - - Build briefing. - - Build vars. - - Run `scriptorium`. - - Write Markdown report. +1. Define prompt input data package schema structures. +2. Build `data_package` JSON from report metadata, briefing content, source warnings, RunID, and an initially empty Recent Changes section. +3. Validate required data package fields before rendering. +4. Write data package JSON to the workspace. +5. Implement `scriptorium render` support in the adapter. +6. Invoke preflight as: + +```text +scriptorium render \ + --prompt \ + --input data_package= \ + --format json +``` + +7. Capture stdout and stderr separately. +8. Persist render/preflight output for inspection. +9. Treat render preflight as always-on before `scriptorium run` for MVP generated reports. ### Deliverables -- `weatherreporter generate daily --location home --out ./daily.md` produces a Markdown report. -- Failures include useful context. +- Daily Report data package can be generated and written to a file. +- The data package is suitable for `scriptorium render --input data_package=`. +- Prompt/input wiring can be checked without LLM execution. ### Tests -- Adapter command construction using a fake command runner or fake executable. -- Nonzero exit handling. -- Timeout behavior. -- App workflow test using fake weather client and fake `scriptorium` runner. +- Data package generated from Daily briefing fixture. +- Missing required fields fail validation. +- JSON output is deterministic where practical. +- Adapter command construction for `scriptorium render`. +- Nonzero render exit handling. +- Always-on preflight behavior in the generation workflow. ### Done Criteria -- The first report can be generated end-to-end. -- `scriptorium` is isolated behind the adapter package. +- The app can prepare and preflight a complete data package for a Daily Report. +- LLM rendering is the only missing step for the first end-to-end report. -## Stage 8: Filesystem State Store and Metadata Persistence +## Stage 7: Filesystem State Store and Metadata Baseline ### Goal -Persist report artifacts and metadata in a durable, inspectable structure. +Persist report artifacts and metadata in a durable, inspectable structure before the first LLM-generated report. ### Packages Introduced or Expanded @@ -420,15 +456,19 @@ Persist report artifacts and metadata in a durable, inspectable structure. 1. Define state store interface. 2. Implement filesystem-backed store. 3. Persist briefing snapshots. -4. Persist prompt variable files. -5. Persist report metadata. -6. Persist rendered Markdown reports when output path is managed by the app. -7. Use atomic writes where practical. -8. Implement lookup for prior comparable snapshots. +4. Persist prompt input data package files. +5. Persist `scriptorium render` preflight output. +6. Persist report metadata. +7. Add RunID as an explicit metadata concept based on generation timestamp plus report ID. +8. Name managed report files with the generation timestamp to avoid overwriting previous runs for the same valid period. +9. Link valid period, RunID, briefing snapshot, data package, preflight output, rendered report, source warnings, and source hashes in metadata. +10. Use atomic writes where practical. +11. Implement lookup for prior comparable Daily snapshots. +12. Keep paths narrow and predictable. ### Deliverables -- Each generated report has associated metadata and briefing snapshot. +- Each Daily preflight run has associated metadata, RunID, briefing snapshot, data package, and render output. - Prior comparable snapshot lookup works for Daily Reports. ### Tests @@ -436,25 +476,92 @@ Persist report artifacts and metadata in a durable, inspectable structure. - Atomic write behavior where feasible. - Metadata round-trip. - Snapshot path generation. +- Data package path generation. +- Preflight output path generation. +- Timestamped managed report path generation. +- RunID metadata behavior. - Prior snapshot lookup. - Narrow-path safety behavior. ### Done Criteria -- Daily reports leave enough state for inspection and future Recent Changes. +- Daily report preparation leaves enough state for inspection and future Recent Changes. - State layout is predictable and documented. +## Stage 8: Scriptorium Run Adapter and First End-to-End Daily Report + +### Goal + +Invoke `scriptorium run` as a subprocess and produce the first rendered Markdown report. + +### Key References + +- `docs/integrations/scriptorium.md` describes the CLI contract for running `scriptorium` as a subprocess. + +### Packages Introduced or Expanded + +- `internal/adapters/scriptorium` +- `internal/app` +- `internal/state` + +### Work Items + +1. Define `scriptorium.Run` request/result types or extend the runner interface introduced for render preflight. +2. Implement subprocess execution with `exec.CommandContext`. +3. Pass arguments without shell interpolation. +4. Pass the prompt input data package with `--input data_package=`. +5. Invoke generation as: + +```text +scriptorium run \ + --prompt \ + --input data_package= \ + --out +``` + +6. Capture stderr and stdout with reasonable limits. +7. Apply timeout and cancellation. +8. Return actionable errors for nonzero exits, including exit code `2`, which can still produce output. +9. Wire Daily Report generation end-to-end: + - Fetch bundle. + - Build briefing. + - Build data package. + - Run render preflight. + - Run `scriptorium`. + - Write Markdown report. + - Persist metadata and snapshots. + +### Deliverables + +- `weatherreporter generate daily --out ./daily.md` produces a Markdown report. +- Failures include useful context. +- The first end-to-end report already has durable briefing, data package, metadata, and source provenance. + +### Tests + +- Adapter command construction using a fake command runner or fake executable. +- Nonzero exit handling. +- Exit code `2` handling. +- Timeout behavior. +- Always-on preflight before run. +- App workflow test using fake weather client and fake `scriptorium` runner. + +### Done Criteria + +- The first report can be generated end-to-end. +- `scriptorium` is isolated behind the adapter package. + ## Stage 9: Recent Changes for Daily Reports ### Goal -Add structured comparison of Daily Report briefing snapshots and include meaningful changes in prompt variables. +Add structured comparison of Daily Report briefing snapshots and include meaningful changes in prompt input data packages. ### Packages Introduced or Expanded - `internal/changes` - `internal/state` -- `internal/promptvars` +- `internal/promptinput` - `internal/app` ### Work Items @@ -470,12 +577,12 @@ Add structured comparison of Daily Report briefing snapshots and include meaning - Alert changes. - Wind gust changes. - Snow/ice/thunder risk changes. -6. Add Recent Changes to prompt vars. +6. Add Recent Changes to the data package. 7. Omit or minimize Recent Changes when no meaningful changes exist. ### Deliverables -- Daily Report vars include Recent Changes when appropriate. +- Daily Report data packages include Recent Changes when appropriate. - Daily Report generation persists current snapshot after comparison. ### Tests @@ -485,11 +592,11 @@ Add structured comparison of Daily Report briefing snapshots and include meaning - Temperature threshold crossing. - Precipitation timing shift. - Alert added/removed behavior. -- Comparison uses valid period, not just generation time. +- Comparison uses valid period and compatible strategy, not just generation time. ### Done Criteria -- The Daily Report can say what changed relative to the prior report covering the same forecast period. +- The Daily Report can say what changed relative to a prior compatible report covering the same forecast period. - Markdown report text is not used as the comparison source. ## Stage 10: Tomorrow Planning Brief @@ -508,24 +615,24 @@ Add the evening Tomorrow Planning Brief using the Daily Report machinery where p ### Work Items 1. Implement the `daily_tomorrow` report definition fully. -2. Reuse or specialize the Daily briefing builder for tomorrow’s valid date. +2. Reuse or specialize the Daily briefing builder for tomorrow's valid date. 3. Add any tomorrow-specific planning fields, such as: - Morning readiness note inputs. - Commute/school/workday concerns. - What may change overnight. -4. Ensure comparison can find a prior report covering the same valid day where appropriate. +4. Ensure comparison can find a prior compatible report covering the same valid local date. 5. Implement `run evening` as Daily Tomorrow. ### Deliverables -- `weatherreporter generate tomorrow --location home` works end-to-end. -- `weatherreporter run evening --location home` works. +- `weatherreporter generate tomorrow` works end-to-end. +- `weatherreporter run evening` works. ### Tests - Tomorrow valid-period calculation. - Tomorrow briefing uses the correct date. -- Recent Changes can compare against prior 3-Day or prior Tomorrow snapshot if configured. +- Recent Changes can compare against a compatible prior Tomorrow, Daily, or 3-Day snapshot when configured by the registry. - Evening batch includes only the expected report. ### Done Criteria @@ -551,32 +658,32 @@ Add the 3-Day Outlook report using the same architecture. 1. Implement 3-Day valid-period resolution. 2. Build a 3-Day briefing package. -3. Summarize each day: +3. Summarize each day or partial day: - Overall character. - Temperature range. - Precipitation/storm/winter/heat/wind risks. - Best/worst windows if derivable. - Relevant alerts. -4. Attach broader NWS context, especially forecast discussion and weather story inputs. +4. Attach broader NWS context, especially forecast discussion and weather story inputs when available. 5. Implement 3-Day Recent Changes strategy. 6. Add end-to-end generation. ### Deliverables -- `weatherreporter generate three-day --location home` produces Markdown. +- `weatherreporter generate three-day` produces Markdown. - Morning batch can include the 3-Day Outlook. ### Tests -- Three-day valid period. -- Daily aggregation across three days. +- Three-day valid period from generation time. +- Daily aggregation across the 3-Day window. - Alert overlap across multi-day period. - Recent Changes across multi-day snapshots. - Quiet-weather behavior. ### Done Criteria -- The 3-Day Outlook is generated using the same registry, briefing, vars, state, and rendering pipeline as the Daily Report. +- The 3-Day Outlook is generated using the same registry, briefing, data package, state, and rendering pipeline as the Daily Report. ## Stage 12: Weekend Outlook @@ -595,9 +702,9 @@ Add the Weekend Outlook with day-of-week-sensitive period behavior. ### Work Items 1. Implement Weekend valid-period resolution: - - Monday through Thursday: upcoming Saturday/Sunday, optionally Friday evening if configured. - - Friday: Friday evening through Sunday night. - - Saturday: remaining weekend. + - Monday through Thursday: upcoming Saturday 00:00 through Monday 00:00. + - Friday: `max(generation time, Friday 18:00)` through Monday 00:00. + - Saturday: generation time through Monday 00:00. - Sunday: normally not generated by the scheduled morning batch. 2. Build Weekend briefing package. 3. Emphasize planning fields: @@ -611,8 +718,8 @@ Add the Weekend Outlook with day-of-week-sensitive period behavior. ### Deliverables -- `weatherreporter generate weekend --location home` produces Markdown. -- `weatherreporter run morning --location home` includes Weekend Outlook except Sunday. +- `weatherreporter generate weekend` produces Markdown. +- `weatherreporter run morning` includes Weekend Outlook except Sunday. ### Tests @@ -643,27 +750,27 @@ Make scheduled workflows reliable enough for unattended execution by cron, syste 1. Finalize `run morning` workflow. 2. Finalize `run evening` workflow. -3. Decide failure behavior: - - Continue remaining reports after one report fails, or fail fast. - - Return aggregate status. -4. Add structured run summaries. -5. Ensure each report run records enough metadata for troubleshooting. -6. Add CLI flags for output directory, location, and optional dry-run/vars-only mode if desired. -7. Add logging suitable for scheduled execution. +3. Continue remaining independent reports after one report fails. +4. Return nonzero from the CLI if any report failed. +5. Add structured aggregate run summaries. +6. Ensure each report run records enough metadata for troubleshooting. +7. Add CLI flags for output directory and optional dry-run/data-package-only mode if desired. +8. Add logging suitable for scheduled execution. ### Deliverables - Morning batch can generate Daily, 3-Day, and Weekend reports. - Evening batch can generate Tomorrow Planning Brief. -- Failures are understandable from logs and metadata. +- Failures are understandable from logs, metadata, and aggregate summaries. ### Tests - Morning batch report selection. - Evening batch report selection. -- Partial failure behavior. +- Partial failure continues independent reports. +- Aggregate nonzero exit behavior. - Output path behavior. -- Dry-run or vars-only behavior, if implemented. +- Dry-run or data-package-only behavior, if implemented. ### Done Criteria @@ -686,11 +793,11 @@ Add manual Storm Report generation without building the automatic monitoring age ### Work Items 1. Implement Storm Report definition. -2. Define storm valid-period behavior. +2. Require explicit storm valid-period bounds via `--start` and `--end`. 3. Build storm briefing package from: - Active alerts. - Forecast discussion. - - Weather story. + - Weather story when available. - Relevant hourly/daily periods. - NWS narrative periods. 4. Include storm-specific fields: @@ -702,7 +809,7 @@ Add manual Storm Report generation without building the automatic monitoring age - Confidence and uncertainty inputs. - What to watch next. 5. Add manual command: - - `weatherreporter generate storm --location home` + - `weatherreporter generate storm --start