weatherreporter/docs/policy/development.md

Weatherreporter Package Layout

This document defines the proposed package layout for weatherreporter, a Go application that prepares human-facing weather reports from normalized weather data collected by weatherfeeder and rendered through scriptorium.

The application should remain a small, explicit, dependency-light Go program. Domain logic should live outside CLI, transport, and external-adapter packages. External systems should be isolated behind narrow adapters. Report-specific behavior should be selected through a registry or equivalent mechanism rather than scattered conditionals.

Architectural Summary

weatherreporter is a deterministic weather briefing and report-preparation application. It should:

1. Fetch normalized weather data from a single configured internal weather API endpoint backed by weatherfeeder.
2. Derive report-specific briefing packages from the normalized forecast bundle.
3. Compare current briefing snapshots against prior comparable snapshots to produce optional Recent Changes.
4. Build structured prompt input data packages for a specific report type.
5. Invoke scriptorium as an external prompt runner.
6. Persist the rendered Markdown report, briefing snapshot, prompt input package, and generation metadata.

The preferred data flow is:

weatherfeeder-backed internal API
    -> weather API adapter
    -> forecast bundle
    -> report-specific briefing builder
    -> recent-change comparison
    -> prompt input data package
    -> scriptorium subprocess adapter
    -> Markdown report + metadata + stored snapshot

The application should not treat the LLM as the source of weather facts. The Go code should select the relevant data, compute daypart and period summaries, attach alerts and NWS context, identify meaningful changes, and send the LLM a curated briefing package. The LLM should synthesize and phrase the report for humans.

Proposed Directory Layout

cmd/weatherreporter/
  main.go

internal/app/
  generate.go
  scheduled.go
  storm.go

internal/cli/
  root.go
  generate.go
  run.go
  inspect.go

internal/config/
  config.go
  defaults.go
  load.go
  validate.go

internal/adapters/weatherapi/
  client.go
  types.go

internal/adapters/scriptorium/
  runner.go
  types.go

internal/forecast/
  bundle.go
  dayparts.go
  derive.go
  select.go
  thresholds.go

internal/report/
  definition.go
  registry.go
  period.go
  daily.go
  tomorrow.go
  three_day.go
  weekend.go
  storm.go

internal/briefing/
  package.go
  daily.go
  tomorrow.go
  three_day.go
  weekend.go
  storm.go

internal/changes/
  compare.go
  thresholds.go
  summary.go

internal/state/
  store.go
  filesystem.go
  metadata.go

internal/promptinput/
  build.go
  schema.go

internal/timeutil/
  clock.go
  periods.go

This layout can be simplified during early prototyping if a package has only one file, but the package boundaries should remain conceptually stable.

Dependency Direction

The intended dependency direction is:

cmd/weatherreporter
    -> internal/cli
    -> internal/app
    -> internal/config
    -> internal/report
    -> internal/briefing
    -> internal/forecast
    -> internal/changes
    -> internal/state
    -> internal/adapters/*

Rules:

• cmd/weatherreporter should only bootstrap the CLI.
• internal/cli should parse commands and flags, then call internal/app.
• internal/app should orchestrate workflows but avoid embedding detailed forecast logic.
• internal/adapters/* should not contain domain policy.
• internal/forecast, internal/report, internal/briefing, and internal/changes should be testable without real external services.
• internal/state should expose a storage interface so filesystem state can later be replaced or supplemented.
• scriptorium details should not leak outside internal/adapters/scriptorium.

Package Responsibilities

cmd/weatherreporter

Entry point for the compiled binary.

Responsibilities:

• Construct the root command from internal/cli.
• Execute the command.
• Handle final process exit behavior.

Non-responsibilities:

• No configuration loading details.
• No forecast logic.
• No direct calls to weather APIs, state stores, or scriptorium.

internal/cli

Defines the user-facing command tree, flags, arguments, and command wiring.

Responsibilities:

• Define commands such as:
  • weatherreporter generate daily
  • weatherreporter generate tomorrow
  • weatherreporter generate three-day
  • weatherreporter generate weekend
  • weatherreporter generate storm
  • weatherreporter run morning
  • weatherreporter run evening
  • weatherreporter inspect snapshot
• Use the Go standard library for CLI parsing unless future complexity justifies a dependency.
• Parse flags such as --config, --units, --tz, --out, optional Daily --date, and storm --start/--end, then convert them into app-layer request structs.
• Load configuration through internal/config.
• Present concise user-facing errors.

Non-responsibilities:

• No report-building logic.
• No direct subprocess execution.
• No direct weather API calls.
• No state comparison logic.

Suggested command shape:

weatherreporter generate daily --date 2026-05-29 --out ./daily.md
weatherreporter generate tomorrow --out ./tomorrow.md
weatherreporter generate three-day --out ./three_day.md
weatherreporter generate weekend --out ./weekend.md
weatherreporter generate storm --start 2026-05-29T18:00 --end 2026-05-30T06:00 --out ./storm.md
weatherreporter run morning
weatherreporter run evening

The MVP should not expose location selection. Source locationId and locationName values returned by the weather API may be retained as provenance.

For generate daily, --date is optional. When provided, it must use YYYY-MM-DD; when omitted, it resolves to the current local date in the configured timezone.

internal/config

Owns configuration structures, defaults, loading, precedence, and validation.

Responsibilities:

• Define application configuration structs.
• Provide built-in defaults in defaults.go.
• Load YAML configuration from /usr/local/etc/weatherreporter/config.yml or a CLI-supplied path.
• Use gopkg.in/yaml.v3 for YAML parsing.
• Apply precedence rules.
• Validate required settings.
• Normalize paths, durations, report settings, weather API units/timezone, missing-source policy, and daypart definitions.

Suggested configuration areas:

• Weather API base URL, timeout, units, timezone, precision, and missing-source policy.
• scriptorium binary, config path, profile, timeout, and optional extra arguments.
• Workspace and output directories.
• Report enablement and output naming.
• Daypart definitions.
• Recent-change thresholds.

Initial defaults:

• Weather API units: us.
• Weather API timezone: Chicago.
• Weather API format: json.
• Missing-source policy: warn.

Missing-source policy should support a global default and per-source overrides. Valid policy values are error, warn, and none.

Non-responsibilities:

• No command execution.
• No HTTP calls.
• No report-building logic.

internal/app

Application orchestration and top-level use cases.

Responsibilities:

• Implement use cases such as:
  • Generate one report.
  • Run the morning batch.
  • Run the evening batch.
  • Generate a manual storm report.
• Coordinate config, weather API adapter, report registry, briefing builders, state store, change comparison, prompt input builder, and scriptorium runner.
• Enforce workflow order.
• Ensure each generation run persists enough artifacts for inspection and future comparison.

The core generation workflow should be approximately:

resolve report definition
resolve valid period
fetch current weather bundle
build current briefing package
load prior comparable briefing snapshot
compute recent changes
build prompt input data package
write data package
run scriptorium render preflight
invoke scriptorium run
persist report metadata, briefing snapshot, data package, preflight output, and rendered report

Non-responsibilities:

• No detailed daypart calculations.
• No direct parsing of NWS text unless delegated to domain packages.
• No direct shell command construction outside the scriptorium adapter.

internal/adapters/weatherapi

HTTP adapter for the internal weather API backed by weatherfeeder.

Responsibilities:

• Fetch normalized weather data from the configured API base URL.
• Fan out to multiple weather API endpoints and assemble one internal forecast.Bundle.
• Decode API responses into adapter-owned DTOs or directly into stable internal types if those types are intentionally owned by weatherreporter.
• Apply request timeouts and context cancellation.
• Apply configured query defaults, including format=json, units=us, and tz=Chicago unless overridden.
• Fetch full /forecast/hourly and /forecast/narrative products, not day-slice endpoints, so Go domain code owns report-period selection.
• Record per-source provenance: endpoint, query, fetch time, issued/updated time when available, SHA-256 over canonical/minified raw data JSON, warnings, and missing-source status.
• Represent source warnings as first-class records with source name, code, severity, message, endpoint, and completeness impact.
• Require hourly forecast data for normal scheduled reports.
• Apply missing-source policy for data:null, malformed non-required sections, or unavailable upstream products.
• Return actionable errors containing endpoint and operation context.

Initial data categories:

• Latest observation.
• Current conditions.
• Hourly forecast data.
• NWS narrative forecast periods.
• NWS alerts.
• NWS forecast discussion.

Stubbed source slots until upstream support exists:

• Daily forecast data.
• NWS weather story.

Non-responsibilities:

• No daypart grouping.
• No Recent Changes comparison.
• No prompt input construction.
• No scriptorium calls.

internal/adapters/scriptorium

Subprocess adapter for invoking scriptorium.

Responsibilities:

• Provide a narrow runner interface, such as:

type Runner interface {
    Render(ctx context.Context, req RenderRequest) (*RenderResult, error)
    Run(ctx context.Context, req RunRequest) (*RunResult, error)
}

• Execute scriptorium render for preflight/debug output without LLM generation.
• Execute scriptorium run for report generation.
• Run scriptorium render as an always-on preflight before scriptorium run for MVP generated reports.
• Pass arguments as an argv slice, not through a shell.
• Pass large prompt input as --input data_package=<path>.
• Capture stdout/stderr with reasonable size limits.
• Treat nonzero exits as actionable errors, including exit code 2 from run, which may still produce output.
• Keep all scriptorium-specific flag details inside the adapter.

Suggested command forms:

scriptorium render \
  --prompt weather.daily_report \
  --input data_package=./workspace/data-packages/daily/2026-05-29T050000-0500.data_package.json \
  --format json

scriptorium run \
  --prompt weather.daily_report \
  --input data_package=./workspace/data-packages/daily/2026-05-29T050000-0500.data_package.json \
  --out ./workspace/reports/daily/2026-05-29T050000-0500.md

Non-responsibilities:

• No weather logic.
• No report registry logic.
• No decision about which prompt to run.

Future note:

• A native LLM client can later replace or supplement this adapter behind a similar interface.

internal/forecast

Core forecast-domain processing.

Responsibilities:

• Define the normalized Bundle consumed by report builders.
• Group hourly forecast data into configured dayparts.
• Compute derived facts, including:
  • Temperature ranges.
  • Apparent-temperature ranges, if available.
  • Max precipitation probability.
  • Peak wind and wind gusts.
  • Precipitation windows.
  • Thunder mentions.
  • Snow/ice/freezing risk indicators.
  • Alert overlap with relevant periods.
• Select forecast elements relevant to a report period.
• Provide threshold helpers for impact detection.

Non-responsibilities:

• No CLI behavior.
• No external API calls.
• No rendered prose.
• No direct scriptorium calls.

internal/report

Report definitions, registry, period resolution, and report-level contracts.

Responsibilities:

• Define report IDs and report definition contracts.
• Register report types and variants.
• Resolve valid periods for each report.
• Associate report types with prompt IDs.
• Define comparison strategies and output naming behavior.

Suggested report definitions:

daily_today       -> prompt weather.daily_report
daily_tomorrow    -> prompt weather.daily_report
three_day         -> prompt weather.three_day_outlook
weekend           -> prompt weather.weekend_outlook
storm             -> prompt weather.storm_report

weather.daily_report should be the standard prompt for one local civil day, regardless of whether that day is today or tomorrow.

A report definition should describe:

• Report ID.
• Human-readable name.
• Prompt ID.
• Valid-period resolver.
• Briefing builder ID or function.
• Recent-change comparison strategy.
• Default output naming pattern.
• Whether the report participates in morning or evening scheduled batches.

Non-responsibilities:

• No detailed forecast computation.
• No state storage.
• No subprocess execution.

internal/briefing

Builds report-specific briefing packages from forecast bundles and report definitions.

Responsibilities:

• Convert a forecast bundle into a report-specific structured briefing package.
• Keep each report's briefing shape explicit and testable.
• Attach relevant NWS narrative periods, alerts, forecast discussion context, and weather story context when available.
• Include metadata such as schema version, configured units/timezone, source warnings, and source provenance.
• Provide inputs suitable for scriptorium data packages.

Report-specific builders should exist for:

• Daily Report.
• Tomorrow Planning Brief.
• 3-Day Outlook.
• Weekend Outlook.
• Storm Report.

Non-responsibilities:

• No external API fetching.
• No final prose rendering.
• No state persistence, except through app orchestration.

Design note:

• This package is the architectural center of the application. A clean briefing package makes scriptorium a renderer rather than a source of weather reasoning.

internal/changes

Structured comparison of current and prior briefing snapshots.

Responsibilities:

• Compare current briefing packages against prior comparable snapshots.
• Apply meaningful-change thresholds.
• Produce compact structured change summaries for prompt input data packages.
• Avoid comparison of rendered Markdown report text.

Comparable snapshot matching should be declared by each report definition. Daily Today, Daily Tomorrow, and compatible date slices from multi-day reports may compare by same valid local date when the report registry marks them compatible. Weekend compares by same weekend window. Storm compares by explicit event window.

Meaningful changes may include:

• Temperature changes crossing configured thresholds.
• Precipitation probability changes by category.
• Precipitation timing shifts.
• New, canceled, extended, upgraded, or expanded alerts.
• Wind gust threshold crossings.
• Snow/ice/freezing risk changes.
• Severe-weather wording or risk changes.
• Confidence or uncertainty changes, if represented in structured briefing data.

Non-responsibilities:

• No fetching prior state directly unless mediated through app/state contracts.
• No final report prose.
• No external calls.

internal/state

Durable state store for reports, snapshots, data packages, preflight output, metadata, and comparison lookup.

Responsibilities:

• Persist generated report metadata.
• Persist briefing snapshots.
• Persist prompt input data packages.
• Persist scriptorium render preflight output for generated reports.
• Locate prior comparable snapshots for Recent Changes.
• Track RunID as generation timestamp plus report ID.
• Use timestamped managed report names to avoid overwriting prior runs for the same valid period.
• Use atomic writes where practical.
• Keep filesystem layout narrow and predictable.

Initial backend:

• Filesystem state.

Potential future backend:

• SQLite or another state database, behind the same store interface.

Suggested state layout:

workspace/
  snapshots/
    daily/
      2026-05-30/
        2026-05-29T050000-0500.briefing.json
        2026-05-29T050000-0500.metadata.json
    three-day/
    weekend/
    storm/
  reports/
    daily/
      2026-05-29T050000-0500.md
    three-day/
    weekend/
    storm/
  data-packages/
    daily/
      2026-05-29T050000-0500.data_package.json
  preflight/
    daily/
      2026-05-29T050000-0500.render.json

Non-responsibilities:

• No weather derivation.
• No report prose generation.
• No CLI formatting decisions.

internal/promptinput

Builds the final data package passed to scriptorium.

Responsibilities:

• Combine report metadata, briefing package, Recent Changes, selected source context, and source warnings into a prompt input document.
• Validate required data package fields before invoking scriptorium.
• Keep data package schemas explicit enough to test.
• Write data package files to the workspace when requested by the app layer.

Non-responsibilities:

• No weather API calls.
• No forecast derivation.
• No subprocess execution.

internal/timeutil

Time, clock, and period helpers.

Responsibilities:

• Provide an injectable clock for deterministic tests.
• Resolve local dates using the configured report timezone.
• Handle daypart spans, including overnight windows.
• Normalize valid periods.
• Provide helpers for recurring scheduled batches.

Non-responsibilities:

• No report-specific forecast logic unless delegated by internal/report.
• No external calls.

Report Types and Valid-Period Identity

Each generated report must be associated with explicit metadata:

• RunID.
• Report type.
• Report variant, if applicable.
• Generation time.
• Configured report timezone.
• Valid period start.
• Valid period end.
• Source location ID/name when provided by upstream.
• Source product timestamps and/or SHA-256 hashes.
• Source warnings.
• Briefing snapshot path.
• Prompt input data package path.
• Preflight output path.
• Rendered report path.

All valid periods should use the configured local timezone, default Chicago, and half-open [start,end) intervals.

Initial valid-period rules:

• Daily Today: current local civil day, [00:00, next 00:00).
• Daily Tomorrow: next local civil day.
• 3-Day Outlook: generation time through local midnight after the second following local civil day.
• Weekend Outlook: Monday through Thursday covers Saturday 00:00 to Monday 00:00; Friday and Saturday cover max(generation time, Friday 18:00) to Monday 00:00; scheduled Sunday morning skips Weekend Outlook.
• Manual Storm Report: requires explicit --start and --end; accept YYYY-MM-DDTHH:MM interpreted in the configured timezone and RFC3339 timestamps with explicit offsets.

The valid period should identify what weather period the report covers, independent of when the report was generated.

Examples:

• A 5 PM Tomorrow Planning Brief for Saturday and a 5 AM Saturday Daily Report both cover the same valid date.
• A Saturday Weekend Outlook covers the remaining weekend, while a Friday Weekend Outlook may cover Friday evening through Sunday night.
• A Storm Report covers an explicit forecast event window, not a fixed calendar day.

This identity is required for reliable Recent Changes behavior.

Scheduled Batch Semantics

The app should support scheduled batches but should not need to be a daemon in the initial version.

Suggested batches:

morning:
  - daily_today
  - three_day
  - weekend, except Sunday

evening:
  - daily_tomorrow

Scheduled batches should continue independent reports after a report failure. The CLI should return nonzero if any report failed and should emit an aggregate run summary.

External scheduling should be handled by systemd timers, cron, or another orchestrator. weatherreporter should simply provide deterministic commands that can be scheduled.

Storm Report Direction

The initial version should support manual Storm Report generation:

weatherreporter generate storm --start 2026-05-29T18:00 --end 2026-05-30T06:00

Future storm monitoring should use a staged design:

incoming weather data
    -> deterministic candidate detector
    -> LLM event evaluator
    -> storm lifecycle state
    -> storm report generation or skip decision

Potential storm lifecycle states:

none -> monitoring -> active_report -> escalated -> deescalating -> resolved

This future behavior should not be built before the core scheduled reports are stable, but the package layout should leave room for it.

Testing Expectations

Core tests should not require real external services.

Priority test areas:

• Configuration loading and validation, including defaults for units=us, tz=Chicago, and missing-source policy warn.
• Standard-library CLI command parsing, including --units, --tz, and storm --start/--end.
• Weather API fan-out, source provenance, data:null, and missing-source policy behavior.
• Daypart grouping, especially overnight periods.
• Valid-period resolution for each report type.
• Briefing package construction from fixtures.
• Recent Changes threshold behavior and compatible snapshot matching.
• Prior snapshot lookup.
• scriptorium adapter behavior using a fake executable or command runner, including both render and run with --input data_package=<path>.
• Batch partial-failure behavior and aggregate exit status.

Design Invariants

Preserve these invariants as the project evolves:

• Weather facts come from normalized source data, not from the LLM.
• The LLM receives curated briefing packages, not unbounded raw weather payloads.
• Recent Changes are based on structured snapshot comparison, not Markdown diffing.
• Report types are registered or otherwise centrally defined.
• External integrations are thin adapters.
• CLI code wires workflows but does not own domain logic.
• The first durable state backend is filesystem-based and inspectable.
• scriptorium is an adapter boundary, not an application dependency that leaks across packages.