weatherreporter/docs/roadmap/implementation.md

GeneratedText Hourly Report Implementation Roadmap

Purpose

This roadmap defines the concrete implementation sequence for docs/roadmap/generated-text.md. It is written for an LLM coding agent that will implement each stage in order.

This is a future-work roadmap. Until a stage is implemented, non-roadmap docs must not describe GeneratedText, embedded report templates, or other unimplemented GeneratedText behavior as available.

Source Feature Roadmap

Use docs/roadmap/generated-text.md as the authoritative feature roadmap for intent, terminology, target behavior, and policy choices. This document is the step-by-step implementation plan. If target behavior changes, update generated-text.md first, then update this plan.

Future implementation agents should read generated-text.md before starting Stage 1. Treat that file as the conceptual contract for why the feature exists and what final shape it should have; treat this file as the ordered work plan for reaching that shape.

Locked Implementation Decisions

• Implement the first GeneratedText path for the rolling next-hours report, named hourly.
• Use hourly as the only planned public name for the rolling next-hours report; do not keep compatibility aliases from retired roadmap terminology.
• Use report ID hourly, artifact group hourly, batch output name hourly.md, and CLI command weatherreporter generate hourly.
• Keep the rolling report explicit-generation only; do not add scheduled batch membership in this implementation.
• Keep the valid-period length as a package-owned constant, initially 6 hours.
• Add report generation modes:
  • scriptorium_markdown for existing reports;
  • generated_text_template for hourly.
• Keep existing reports on the current Scriptorium Markdown path.
• For generated-text reports, keep Definition.PromptID as the Scriptorium prompt ID and set the hourly value to weather.hourly_generated_text.
• Add TemplateID and GeneratedTextSchemaID to report definitions.
• Use embedded assets under internal/reporttemplate.
• Use standard-library text/template.
• Let Scriptorium enforce its prompt-associated JSON Schema and retries; weatherreporter validates by unmarshalling into a typed GeneratedText struct and checking required fields. Do not add a Go JSON Schema dependency in this implementation.
• Keep the existing preflight artifact for prompt/input inspection.
• Add managed artifacts for raw GeneratedText JSON, structured Scriptorium run result, validated GeneratedText JSON, and render context JSON.
• Distributor notification continues to upload the managed Markdown report only.
• Preserve current CLI behavior, artifact paths, and generated output behavior for daily, tomorrow, three-day, weekend, and storm reports.

Stage 1: Roadmap Reconciliation And Hourly Identity

Goal: remove conflicting future instructions and establish hourly as the only planned public name for the rolling next-hours report.

Files to inspect:

• docs/roadmap/generated-text.md
• the superseded roadmap file for the retired rolling-report name
• docs/roadmap/future.md
• docs/roadmap/implementation.md
• internal/app/app.go
• internal/report/definition.go
• internal/report/registry.go

Implementation:

• Replace the superseded roadmap file for the retired rolling-report name with a short note pointing to docs/roadmap/generated-text.md, or delete it if the repository policy at implementation time prefers removing obsolete roadmap files.
• Ensure no roadmap instructs a future agent to implement retired public identifiers for the rolling report.
• If partial code using retired identifiers already exists, rename it to hourly rather than adding aliases:
  • app report kind -> ReportHourly;
  • report ID constant -> report.Hourly;
  • CLI command -> hourly;
  • artifact group -> hourly;
  • prompt IDs and tests updated to hourly names.
• Do not add compatibility aliases for retired command or config names.

Acceptance criteria:

• Active roadmap docs use hourly for the rolling report.
• No active roadmap describes a retired identifier as the target public report ID.
• Existing code compiles after any rename work.

Suggested validation:

rg -n 'near[_-]term|Near''Term' docs internal
go test ./internal/app ./internal/report ./internal/cli

This stage is small enough for one implementation prompt.

Stage 2: Report Definition Generation Mode

Goal: add generation-mode metadata to report definitions without changing the current report generation path.

Files to inspect:

• internal/report/definition.go
• internal/report/registry.go
• internal/report/*_report.go
• internal/report/period_test.go
• internal/state/filesystem.go

Implementation:

• Add a GenerationMode type in internal/report.
• Add constants:
  • GenerationModeScriptoriumMarkdown
  • GenerationModeGeneratedTextTemplate
• Add fields to report.Definition:
  • GenerationMode GenerationMode
  • TemplateID string
  • GeneratedTextSchemaID string
• Default or explicitly set existing reports to GenerationModeScriptoriumMarkdown.
• Update report definition tests so every generated report declares a generation mode.
• Do not wire app behavior in this stage.
• Keep existing PromptID semantics: it is the Scriptorium prompt ID for the report's generation mode.

Acceptance criteria:

• Existing report definitions retain current prompt IDs and behavior.
• Registry tests fail if a generated report omits generation mode.
• Generated-text-only fields are empty for existing Markdown reports.

Suggested validation:

go test ./internal/report ./internal/state

This stage is small enough for one implementation prompt.

Stage 3: Hourly Report Definition, Modules, Facts, And CLI Shell

Goal: add the hourly report identity, rolling valid period, module composition, facts derivation, config override aliases, and CLI command shell.

Files to inspect:

• internal/report/definition.go
• internal/report/registry.go
• new internal/report/hourly_report.go
• internal/report/period_test.go
• internal/briefing/modules.go
• internal/facts/facts.go
• internal/facts/facts_test.go
• internal/config/reports.go
• internal/config/config_test.go
• internal/app/app.go
• internal/cli/root.go
• internal/cli/root_test.go

Implementation:

• Add report ID Hourly ID = "hourly".

• Add comparison strategy CompareRollingWindow = "rolling_window".

• Add hourlyReportHours constant, initially 6.

• Add hourlyDefinition():

  • Name: "Hourly Report"
  • PromptID: "weather.hourly_generated_text"
  • GenerationMode: GenerationModeGeneratedTextTemplate
  • TemplateID: "hourly"
  • GeneratedTextSchemaID: "hourly"
  • ComparisonStrategy: CompareRollingWindow
  • ArtifactGroup: "hourly"
  • BatchOutputName: "hourly.md"
  • Generated: true
  • CompatiblePriorIDs: []ID{Hourly}
  • no morning/evening batch membership

• Resolve valid period as [generation_time, generation_time + 6h) in the effective timezone.

• Add default hourly modules in order:

  1. metadata
  2. current_conditions
  3. hourly_forecast
  4. precip_timing
  5. alert_digest
  6. spc_convective_outlooks
  7. area_forecast_discussion
  8. spc_convective_discussion
  9. weather_story

• Configure hourly area_forecast_discussion options to include only:

  • key_messages
  • short_term

• Update module supported-report lists:

  • include report.Hourly in all broadly compatible modules;
  • include report.Hourly for hourly_forecast;
  • keep daily/daypart modules incompatible.

• Add facts.BuildDerived support for hourly:

  • valid-period hourly slice;
  • precipitation timing from that slice;
  • alert overlap filtering;
  • SPC outlook/discussion overlap filtering;
  • no daily summaries, daypart summaries, or storm summary.

• Add config report aliases:

  • hourly
  • reject or do not recognize retired rolling-report config keys.

• Add app report kind ReportHourly = "hourly" and map it to report.Hourly.

• Add CLI parsing and help for:

  weatherreporter generate hourly [--config PATH] [--units VALUE] [--tz NAME] [--out PATH]
  

• Do not add --date, --start, --end, or duration flags.

• It is acceptable for generate hourly to return an explicit "generation mode not implemented" error until the app workflow stage, but parser and resolution tests should pass.

Acceptance criteria:

• report.DefaultRegistry().Lookup(report.Hourly) succeeds.
• hourly valid period is exactly six hours and not civil-day truncated.
• hourly default module composition validates.
• daily-only modules reject hourly.
• facts derivation supports the hourly module list.
• CLI help includes generate hourly.
• Existing command behavior is unchanged.

Suggested validation:

go test ./internal/report ./internal/briefing ./internal/facts
go test ./internal/config ./internal/app ./internal/cli
go run ./cmd/weatherreporter --help

This stage is large but still suitable for one implementation prompt if kept strictly to report identity/facts/CLI shell. If it becomes unwieldy, split it after report/facts and implement CLI/config aliases in a follow-up prompt.

Stage 4: Embedded Template And Schema Assets

Goal: create the embedded asset package and first hourly template/schema.

Files to create:

• internal/reporttemplate/reporttemplate.go
• internal/reporttemplate/templates/hourly.md.tmpl
• internal/reporttemplate/schemas/hourly.generated_text.schema.json
• internal/reporttemplate/reporttemplate_test.go

Implementation:

• Use go:embed for templates and schemas.
• Expose a narrow API:
  • Template(id string) (string, error) or equivalent;
  • Schema(id string) ([]byte, error) or equivalent;
  • Render(id string, data any) ([]byte, error).
• Use text/template.
• Register only hourly assets initially.
• Missing or unknown asset IDs must return actionable errors.
• Keep template functions minimal. Prefer prepared render-context strings over complex template logic.
• Add an initial hourly JSON Schema with:
  • required summary;
  • required timing;
  • required impacts;
  • optional confidence;
  • all fields strings;
  • additionalProperties: false.
• Add an initial hourly Markdown template that is readable and deterministic. It should include fixed headings, report valid period, selected deterministic facts, and GeneratedText slots.

Acceptance criteria:

• schema asset can be looked up for tests, local validation, and out-of-band Scriptorium registration;
• template asset can be looked up;
• template renders from a test render context;
• missing template/schema IDs fail clearly;
• no generated schema or template content is inline in Go.

Suggested validation:

go test ./internal/reporttemplate

This stage is small enough for one implementation prompt.

Stage 5: GeneratedText Contract And Render Context

Goal: add typed GeneratedText validation and a typed hourly render context.

Files to create or inspect:

• new internal/generatedtext package, or another narrow package if the implementation chooses a better local name
• internal/reporttemplate
• internal/module
• internal/briefing
• internal/promptinput

Implementation:

• Add typed hourly GeneratedText:

  type Hourly struct {
      Summary    string `json:"summary"`
      Timing     string `json:"timing"`
      Impacts    string `json:"impacts"`
      Confidence string `json:"confidence,omitempty"`
  }
  

• Add validation:

  • malformed JSON fails;
  • empty required fields fail after trimming whitespace;
  • unknown fields fail by using json.Decoder.DisallowUnknownFields;
  • validated output is normalized to stable JSON.

• Do not add a Go JSON Schema dependency in this implementation.

• Add typed hourly render context.

• Build render context from:

  • report metadata and location;
  • selected module outputs from the module snapshot;
  • validated hourly GeneratedText.

• Use module.StanzaValue or equivalent typed decoding rather than ad hoc map traversal where practical.

• Keep render context curated; do not pass the full raw data package to the template.

Acceptance criteria:

• valid GeneratedText passes and round-trips to stable JSON;
• missing required fields fail;
• unknown fields fail;
• render context has enough deterministic fields for the hourly template;
• render context construction fails clearly when required module stanzas are missing.

Suggested validation:

go test ./internal/generatedtext ./internal/reporttemplate ./internal/module

This stage is small enough for one implementation prompt.

Stage 6: State Artifacts For GeneratedText Reports

Goal: add explicit managed paths and save/load helpers for generated-text artifacts.

Files to inspect:

• internal/state/filesystem.go
• internal/state/store.go
• internal/state/metadata.go
• internal/state/filesystem_test.go
• internal/fileutil

Implementation:

• Extend state.ArtifactPaths with:
  • GeneratedTextRaw
  • GeneratedTextResult
  • GeneratedText
  • RenderContext
• Store all four under the existing snapshots tree:
  • snapshots/<artifact_group>/<YYYY-MM-DD>/<run_id>.generated_text.raw.json
  • snapshots/<artifact_group>/<YYYY-MM-DD>/<run_id>.generated_text.run.json
  • snapshots/<artifact_group>/<YYYY-MM-DD>/<run_id>.generated_text.json
  • snapshots/<artifact_group>/<YYYY-MM-DD>/<run_id>.render_context.json
• Do not add new workspace config directories.
• Extend state.Metadata with optional path fields for those artifacts. Also record GeneratedTextSchemaID as metadata for auditability without copying schema contents into every run.
• Add store methods as needed, for example:
  • SaveGeneratedTextRaw
  • SaveGeneratedTextResult
  • SaveGeneratedText
  • SaveRenderContext
• Use atomic writes through existing fileutil helpers.
• Keep existing metadata JSON shape for Markdown-path reports, with new fields omitted when empty.

Acceptance criteria:

• path tests cover all new artifact paths;
• save helpers create parent directories and write atomically;
• metadata links generated-text paths for hourly reports;
• existing report metadata tests still pass.

Suggested validation:

go test ./internal/state ./internal/fileutil

This stage is small enough for one implementation prompt.

Stage 7: Scriptorium GeneratedText Run Adapter

Goal: add a GeneratedText run helper behind the Scriptorium adapter while preserving the existing scriptorium run command shape.

Files to inspect:

• internal/adapters/scriptorium/runner.go
• internal/adapters/scriptorium/runner_test.go
• docs/integrations/scriptorium.md after implementation

Implementation:

• Add weatherreporter-owned request/result types:

  type StructuredRunRequest struct {
      PromptID        string
      DataPackagePath string
      OutputPath      string
  }
  

• Add StructuredRun(ctx, req) to the runner.

• Validate prompt ID, data package path, and output path before subprocess execution.

• Preserve current command result capture behavior:

  • full argv;
  • stdout;
  • stderr;
  • truncation flags;
  • exit code;
  • output path.

• Do not change current Render or Run methods.

• Scriptorium selects the structured output schema from the backend prompt configuration associated with PromptID; do not pass a schema path at runtime.

• Use the same scriptorium run command form as Markdown report generation. The only material difference is that --out points to the raw GeneratedText JSON artifact path rather than a Markdown report path.

• Use this argv shape:

  scriptorium run \
    --prompt <prompt_id> \
    --input data_package=<path> \
    --out <generated_text_raw_path>
  

• Do not add --format json or any schema flag. Structured output is selected by Scriptorium's internal prompt configuration.

• If local Scriptorium documentation needs updating after implementation, update docs/integrations/scriptorium.md to describe this contract.

Acceptance criteria:

• argv construction tests prove scriptorium run is invoked without --format or schema flags and with --out <generated_text_raw_path>;
• nonzero exits return captured result plus error;
• missing required fields fail before subprocess execution;
• current Scriptorium render/run tests still pass;
• tokens/secrets are not added to argv or errors.

Suggested validation:

go test ./internal/adapters/scriptorium

This stage is small enough for one implementation prompt.

Stage 8: GeneratedText App Workflow Success Path

Goal: wire the successful generated-text/template path into app orchestration for hourly while preserving the existing path for all current reports.

Files to inspect:

• internal/app/app.go
• internal/app/app_test.go
• internal/adapters/scriptorium
• internal/reporttemplate
• internal/generatedtext
• internal/state
• internal/promptinput

Implementation:

• Extend the app Renderer interface, or introduce a second narrow interface, so tests can fake structured generation.
• Branch report generation by req.Resolved.Definition.GenerationMode.
• For GenerationModeScriptoriumMarkdown, keep the existing flow unchanged.
• For the successful GenerationModeGeneratedTextTemplate path:
  1. run existing preflight and save preflight artifact;
  2. call Scriptorium structured generation with raw output path GeneratedTextRaw;
  3. save structured Scriptorium command result to GeneratedTextResult;
  4. validate raw JSON into typed GeneratedText;
  5. save normalized GeneratedText to GeneratedText;
  6. build hourly render context from metadata, module snapshot, and GeneratedText;
  7. save render context JSON;
  8. render Markdown via embedded template to managed report path;
  9. save final metadata with all generated-text paths and the schema ID;
• Defer distributor notification and optional output copy regression coverage to Stage 10.
• Keep failure-specific persistence tests minimal in this stage; Stage 9 owns failure artifact behavior.
• Keep Recent Changes empty for hourly.

Acceptance criteria:

• hourly generation reaches Scriptorium structured generation, validates GeneratedText, renders Markdown, and persists final metadata;
• daily/tomorrow/three-day/weekend/storm generation still uses existing Markdown Scriptorium path;
• generated-text artifacts are linked from metadata for successful hourly runs;
• app tests cover the successful hourly workflow with fake structured Scriptorium output.

Suggested validation:

go test ./internal/app ./internal/state ./internal/reporttemplate ./internal/generatedtext

This stage is small enough for one implementation prompt if failure behavior is kept to the later stages.

Stage 9: GeneratedText Failure Artifact Persistence

Goal: make generated-text failures inspectable without changing successful behavior from Stage 8.

Files to inspect:

• internal/app/app.go
• internal/app/app_test.go
• internal/adapters/scriptorium
• internal/generatedtext
• internal/reporttemplate
• internal/state

Implementation:

• Add focused app tests for failures at each generated-text step:
  • preflight failure;
  • structured Scriptorium generation failure;
  • GeneratedText validation failure;
  • render context construction failure;
  • Markdown template rendering failure.
• Ensure failures preserve available artifacts:
  • failed preflight saves preflight output when possible;
  • failed structured generation saves command result when available;
  • validation failures preserve raw GeneratedText output when available;
  • render-context failures preserve validated GeneratedText when already available;
  • template failures preserve GeneratedText and render context when already built.
• Return actionable errors that include report ID, RunID, and operation context without exposing data package contents or secrets.
• Do not run distributor notification after any generated-text failure.
• Do not create optional --out copies after any generated-text failure.

Acceptance criteria:

• every generated-text failure point has an app-level regression test;
• partial artifacts are persisted where available and linked from metadata when metadata can be safely written;
• failed hourly runs do not notify distributor;
• existing Markdown-generation failure behavior is unchanged.

Suggested validation:

go test ./internal/app ./internal/state ./internal/generatedtext ./internal/reporttemplate

This stage is small enough for one implementation prompt.

Stage 10: GeneratedText Output Copy And Notification Regression Coverage

Goal: preserve existing user-facing output behavior and distributor boundaries for generated-text reports.

Files to inspect:

• internal/app/app.go
• internal/app/app_test.go
• internal/adapters/distributor
• internal/state

Implementation:

• Keep --out copy behavior unchanged after the managed Markdown report exists.
• Keep --out-dir behavior unchanged for batch paths if hourly is ever run through a batch helper during tests.
• Run distributor notification, if enabled, only after:
  • structured generation succeeds;
  • GeneratedText validates;
  • Markdown template rendering succeeds;
  • final metadata is saved.
• Use the managed Markdown report path as the distributor source.
• Never use optional --out or --out-dir copies as distributor source files.
• Keep distributor request construction template-driven; do not add hourly-specific path branching.
• Keep Recent Changes empty for hourly in this implementation.

Acceptance criteria:

• --out copies the managed generated-text Markdown report after success;
• distributor notification receives the managed Markdown report path;
• distributor notification is not called on any failure from Stage 9;
• enabled distributor failure makes hourly generation fail with the same semantics as existing generated reports;
• existing daily/tomorrow/three-day/weekend/storm distributor behavior is unchanged.

Suggested validation:

go test ./internal/app ./internal/adapters/distributor ./internal/state

This stage is small enough for one implementation prompt.

Stage 11: End-To-End CLI, Config, And Prompt Package Coverage

Goal: prove the public hourly workflow and configured module overrides work through the CLI/app boundary.

Files to inspect:

• internal/cli/root.go
• internal/cli/root_test.go
• internal/config/reports.go
• internal/config/config_test.go
• internal/app/app_test.go
• examples/config.yml

Implementation:

• Add CLI command tests for generate hourly.
• Reject the retired rolling-report command name.
• Reject hourly --date, --start, --end, and duration-like flags.
• Add config override tests for reports.hourly.deterministic_modules.
• Ensure retired rolling-report config keys are rejected.
• Add app/CLI workflow tests using fake Scriptorium structured output.
• Update examples only if they enumerate report module overrides.

Acceptance criteria:

• weatherreporter --help lists generate hourly;
• the retired rolling-report command name is not accepted;
• hourly config overrides validate compatible modules;
• incompatible daily/daypart modules fail clearly for hourly;
• examples load successfully.

Suggested validation:

go test ./internal/cli ./internal/config ./internal/app
go run ./cmd/weatherreporter --help

This stage is small enough for one implementation prompt.

Stage 12: Implemented Documentation

Goal: document implemented behavior after the code exists.

Files to inspect and update:

• docs/cli.md
• docs/operations.md
• docs/config.md
• docs/internal/app-orchestration.md
• docs/internal/report-registry.md
• docs/internal/briefing.md
• docs/internal/prompt-input.md
• docs/internal/scriptorium-adapter.md
• docs/internal/state.md
• new docs/internal/reporttemplate.md
• docs/integrations/scriptorium.md
• docs/roadmap/future.md
• the superseded roadmap file for the retired rolling-report name
• examples/config.yml, only if examples changed

Documentation requirements:

• Non-roadmap docs describe only implemented hourly and GeneratedText behavior.
• docs/cli.md documents generate hourly, not the retired command name.
• docs/config.md documents reports.hourly only if override support is implemented.
• Internal docs distinguish:
  • CollectedFacts;
  • DerivedFacts;
  • ModuleOutput;
  • GeneratedText;
  • RenderContext.
• Scriptorium integration docs include the structured-output command actually used by the adapter.
• State docs list generated-text artifacts and metadata links.
• the superseded roadmap file for the retired rolling-report name is deleted or marked superseded.
• Deferred migrations for other report types stay under roadmap docs only.

Acceptance criteria:

• No non-roadmap docs describe unimplemented generated-text migrations.
• No active docs describe retired rolling-report identifiers as the target public report.
• Maintained examples remain valid.

Suggested validation:

go test ./internal/config
rg -n 'near[_-]term|Near''Term' docs internal examples
git diff --check

This stage is small enough for one implementation prompt.

Stage 13: Final Validation

Goal: run full validation and perform targeted manual checks.

Commands:

go test ./...
go run ./cmd/weatherreporter --help
git diff --check

Manual checks:

• weatherreporter --help lists generate hourly.
• weatherreporter --help does not list the retired rolling-report command.
• generated hourly Markdown has deterministic headings and section order.
• GeneratedText JSON is persisted separately from deterministic facts.
• render context JSON is persisted and readable.
• embedded hourly template is readable and easy to edit.
• embedded hourly schema is readable and easy to edit.
• existing Markdown-generation reports still run through the previous Scriptorium run --out <markdown> path.
• distributor notification, when enabled, uploads the managed Markdown report.

This stage is small enough for one implementation prompt.

Deferred Work

Do not include these in the first implementation:

• migrating daily, tomorrow, three-day, weekend, or storm to GeneratedText;
• user-configurable template or local schema source overrides;
• user-defined templates or schemas;
• a generic workflow engine;
• template hot reloading;
• dynamic module selection for templates;
• prompt/schema generation from Go structs;
• scheduled hourly batch membership;
• configurable hourly report duration;
• rolling-window Recent Changes for hourly;
• distributor-specific generated-text behavior.

Open Questions

None block implementation.