diff --git a/README.md b/README.md index ce7763d..afe931b 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,10 @@ # weatherreporter -`weatherreporter` is a Go application for preparing human-facing weather -reports from normalized forecast data. It builds JSON module snapshots, passes -YAML prompt data packages to `scriptorium`, and keeps inspectable artifacts -under a local workspace. It can also upload successfully generated managed -Markdown reports to a configured `distributor` HTTP upload endpoint. +Weatherreporter is a Go CLI that turns normalized weather data into managed, +human-facing Markdown reports. + +It provides repeatable reports with inspectable local artifacts, so operators +can review what was collected and generated for every run. ## Quickstart @@ -12,11 +12,14 @@ Markdown reports to a configured `distributor` HTTP upload endpoint. weatherreporter generate today --out ./today.md ``` +Configure a Weather API endpoint first; see the +[configuration reference](docs/config.md). + ## Documentation - [CLI reference](docs/cli.md) - [Configuration reference](docs/config.md) - [Operations guide](docs/operations.md) - [Troubleshooting](docs/troubleshooting.md) -- [Architecture policy](docs/policy/architecture.md) - [Development guide](docs/development.md) +- [Architecture policy](docs/policy/architecture.md) diff --git a/docs/operations.md b/docs/operations.md index a065e70..f7a32dd 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -1,77 +1,54 @@ # Weatherreporter Operations -This guide covers normal operation, generated artifacts, inspection, recovery, -and operational caveats. For symptom-specific diagnosis, see +This guide covers normal operation, managed workspace state, inspection, +recovery, and operational caveats. See the [CLI reference](cli.md) for complete +command syntax and the [configuration reference](config.md) for fields, +defaults, and notification templates. For symptom-based diagnosis, see [Troubleshooting](troubleshooting.md). -## Normal Workflow +## Normal Operation -Generation commands: +After configuring a Weather API endpoint, generate one report: -```text -weatherreporter generate daily --date 2026-05-29 -weatherreporter generate today -weatherreporter generate tomorrow -weatherreporter generate hourly -weatherreporter generate three-day -weatherreporter generate weekend -weatherreporter generate storm --start 2026-05-29T18:00 --end 2026-05-30T06:00 +```sh +weatherreporter generate today --out ./today.md ``` -Generation commands resolve a report period, collect a Weather API bundle, -build a rich JSON module snapshot, build a curated YAML prompt input data -package, run `scriptorium render`, and write managed artifacts under the -configured workspace. Markdown-path reports then run `scriptorium run` directly -to the managed Markdown report path. +A generation collects weather data, resolves the report period, builds and +persists the module snapshot and prompt data package, runs Scriptorium +preflight, then produces the managed Markdown report. Daily, Today, Tomorrow, +and Hourly reports additionally persist generated-text artifacts, validate the +structured generated text, and render Markdown from the validated text and +deterministic values. -`generate daily`, `generate today`, `generate tomorrow`, and `generate hourly` -use the generated-text-template workflow. They run structured `scriptorium run` -to raw GeneratedText JSON, validate the structured text, save a render context, -and render the managed Markdown report from embedded templates. `generate -daily` requires `--date YYYY-MM-DD` for the selected local civil day. -`generate today` covers the selected or current local civil day. `generate -hourly` covers the six-hour rolling period from generation time in the -effective report timezone and is not included in `run morning` or -`run evening`. +The managed report and its final metadata are saved before single-report +Distributor notification is attempted. `--out` writes an extra operator copy; +it never changes the managed report or upload source. A successful generate +command prints its summary to stdout unless `--quiet` is used. -When distributor notification is enabled, weatherreporter uploads the managed -Markdown report after report rendering succeeds and final metadata is saved. -`--out PATH` writes an extra Markdown copy for generated reports; it is not used -as the distributor upload source. Generate commands emit a compact JSON summary -to stdout by default. Use `--quiet` to suppress successful stdout for cron jobs -or other schedulers that only need nonzero exits and external logs. +Run a scheduled batch with the same configured collection: -Batch commands: - -```text -weatherreporter run morning -weatherreporter run evening +```sh +weatherreporter run morning --out-dir ./reports ``` -`run morning` generates Today Report, Tomorrow Report, and a dated Daily Report -for each later future local civil day with complete hourly forecast coverage. -`run evening` generates Tomorrow Report and the same eligible future Daily -reports. Future Daily expansion starts with the day after tomorrow. A Daily -report is eligible only when the collected hourly forecast contains every -hourly period for that local civil day; partial days are skipped. Batch commands -collect weather data once before planning, and a collection failure stops the -batch before any report is generated. +Each batch collects once before it plans reports. Morning runs Today, Tomorrow, +and every eligible dated Daily Report; evening runs Tomorrow and the same +eligible Daily Reports. Eligible Daily dates begin after tomorrow and require +complete hourly coverage for their entire local civil day. A batch continues +after an individual report fails and returns an aggregate failure when any +report or batch notification fails. -After planning succeeds, batch commands print a JSON summary to stdout, write -compact per-report status lines to stderr, continue independent reports after -one report fails, and return nonzero when any report failed. Batch commands do -not upload each report independently. When distributor notification and batch -notification are enabled, weatherreporter uploads one distributor bundle only -after every planned report succeeds. If any report fails, the batch upload is -skipped for the whole batch. `--out-dir PATH` writes extra Markdown copies -using report default filenames such as `today.md` and `tomorrow.md`; dynamic -Daily copies use `daily-YYYY-MM-DD.md`. These copies are not used as -distributor upload sources. Use `--quiet` to suppress successful batch summary -and status output; failures still return nonzero. +`--out-dir` writes extra copies such as `today.md`, `tomorrow.md`, and +`daily-YYYY-MM-DD.md`. These copies are never upload sources. Batch report +copies and notification behavior are summarized in the CLI result; use the +[CLI reference](cli.md) for its exact JSON and stderr contract. -## Filesystem Layout +## Managed Workspace -The default workspace root is `workspace`. +The default workspace root is `workspace`. Artifact paths use the report +definition's artifact group, the valid-period start date in the effective +timezone, and the RunID: ```text workspace/ @@ -91,195 +68,89 @@ workspace/ notifications/batches///distributor..json ``` -Managed artifact filenames use the artifact kind and RunID, so repeated runs -for the same valid period do not overwrite each other. The date directory is -the valid-period start date in the effective report timezone. Generated-text -artifacts are written only for Daily, Today, Tomorrow, and Hourly reports. +The generated-text and render-context artifacts are written only by Daily, +Today, Tomorrow, and Hourly reports. A report's metadata links the module +snapshot, data package, preflight artifact, managed report, and any available +generated-text or single-report notification artifact. Batch notification +artifacts are separate batch-level records under `notifications/batches`. -## RunID And Metadata - -RunIDs are based on generation time plus report ID. Reports that can be -generated more than once in a single command may append a report-specific -disambiguator. Daily appends the local valid date so multiple dynamic Daily -reports in one batch have distinct managed artifacts: - -```text -20260529T100000.123456789Z_daily_2026-05-31 -20260529T100000.123456789Z_today -``` - -Batch notification RunIDs use the batch start timestamp plus the batch command -name: - -```text -20260529T100000.123456789Z_morning -20260529T220000.123456789Z_evening -``` - -Each generated report writes metadata that links: - -- RunID, report ID, variant, and prompt ID -- generation time, timezone, and valid period -- source location, source hashes, and source warnings -- module snapshot path -- prompt input data package path -- preflight output path -- managed Markdown report path -- generated text schema ID and generated-text artifact paths for - generated-text-template reports -- distributor notification debug artifact path, when notification is attempted - -Batch summaries include report status, error text when applicable, valid -period, and known artifact paths for each attempted report. Single-report -notification fields on report items are empty for batch commands. When a batch -notification is attempted, skipped, or fails, the summary includes one -top-level `notification` object with fields such as `status`, `reason`, -`runId`, `pipelineId`, `bundleId`, `idempotencyKey`, `path`, -`includedReports`, and `error`. +RunIDs begin with the UTC generation timestamp and report ID. A Daily RunID +also contains its local valid date so multiple Daily reports in one batch have +different managed paths. Batch notification RunIDs contain the UTC batch start +timestamp and batch name. ## Distributor Notification -Distributor notification is configured with `notify.distributor` and is -disabled by default. For `generate `, weatherreporter uploads the -managed Markdown report path recorded in the report result and metadata. That -single source file is mapped to report-specific bundle paths. Extra copies -written by `--out` or `--out-dir` are operator conveniences only. +When `notify.distributor.enabled` is enabled, a successful `generate` +uploads only the managed Markdown report after final metadata has been saved. +The extra copy from `--out` is never uploaded. A notification attempt writes +a redacted debug artifact at +`notifications///distributor..json`; its +path is then recorded in report metadata. -For `run morning` and `run evening`, per-report notification is suppressed. If -`notify.distributor.enabled` and `notify.distributor.batch.enabled` are both -true, the batch uploads once after all reports finish successfully. The upload -contains one file mapping set per included report. Each mapping uses the -managed Markdown report as the source and report-specific path templates for -that report. All rendered bundle paths across the batch must be unique. If any -report fails, weatherreporter records a top-level -notification status of `skipped` with reason `one or more reports failed` and -does not call distributor. If batch notification is disabled, run commands do -not fall back to per-report uploads. +Batches suppress per-report notification. When both Distributor and its batch +notification are enabled, Weatherreporter submits one multi-report upload after +every planned report succeeds. If any report fails, it records a top-level +`skipped` notification with reason `one or more reports failed` and does not +call Distributor. If batch notification is disabled, a batch does not fall back +to individual uploads. -The rendered pipeline ID selects the configured distributor `http_upload` -workflow. The default bundle ID is a stable logical source identity derived from -producer name, location ID, and report ID: +A batch notification attempt writes +`notifications/batches///distributor..json`. +A notification failure makes the batch fail but does not change successful +individual report items into failed items. The debug artifacts contain rendered +identifiers, managed source and bundle paths, upload and status results, and +redacted errors; they do not contain tokens. -```text -weatherreporter.{location_id}.{report_id} -``` +## Inspecting Stored Runs -The default single-report idempotency key appends RunID to the rendered bundle -ID so each report generation has a distinct retry identity. The default bundle -path uses the valid-period start date, artifact group, and RunID. Batch bundle -IDs default to `weatherreporter.{location_id}.{batch}`, and batch idempotency -keys default to `{bundle_id}.{batch_run_id}`. Distributor owns destination -merge, retention, and derived snapshot behavior such as `latest`. For Daily, -the default report ID and artifact group values are both `daily`, and the -default output filename value is `daily.md`. For Today, the default report ID -and artifact group values are both `today`, and the batch output filename value -is `today.md`. +Inspection is read-only: it neither collects weather data nor invokes +Scriptorium or Distributor. Start by finding a RunID: -Single-report notification happens after final metadata save for generated -reports. Batch notification happens after all planned reports have finished and -only when all report generations succeeded. Collection, module snapshot, -data-package, render preflight, Scriptorium run, generated-text validation, -template rendering, and metadata-save failures do not trigger notification. A -single-report notification failure fails that report. A batch notification -failure makes the batch return nonzero and increments the aggregate failure -count, but individual report items remain succeeded. - -Each notification attempt writes a debug artifact under `notifications/`. -Single-report artifacts live under -`notifications///distributor..json`. Batch -artifacts live under -`notifications/batches///distributor..json`, -where the date directory is the batch start date in the effective report -timezone. The artifact records the rendered pipeline ID, bundle ID, -idempotency key, managed source paths, bundle-relative paths, bundle created -timestamp, accepted upload response, and the latest distributor run status -response when available. Weatherreporter polls status until distributor reports -`succeeded` or `failed`, or until the configured notification timeout expires. -The run status includes the distributor status, error text, and raw run report -JSON, which can show actions such as `replace_older`, `skip_same`, -`skip_destination_newer`, or `failed`. Token values are not written. - -Weatherreporter is responsible for selecting the managed Markdown report, -constructing a source bundle, and submitting it to the configured distributor -HTTP endpoint. Distributor remains responsible for destination routing, -publication, and any downstream Markdown-to-HTML transformation. Distributor -leaves destination files alone when they are not tracked by a newly uploaded -bundle, so existing uploaded dated report paths can remain available. - -## Inspection - -Inspection commands read existing workspace artifacts and emit JSON to stdout. -They do not collect weather data or run `scriptorium`. - -```text +```sh weatherreporter inspect reports --limit 10 weatherreporter inspect metadata RUN_ID -weatherreporter inspect modules RUN_ID -weatherreporter inspect data-package RUN_ID -weatherreporter inspect prior RUN_ID -weatherreporter inspect sources RUN_ID ``` -Use `inspect reports` to find RunIDs and artifact paths. Use -`inspect metadata` to see the artifact links recorded for a run. Use -`inspect modules` to review the persisted ordered module snapshot with rich -template-facing values, and `inspect data-package` to review the curated prompt -package passed to Scriptorium. Use `inspect prior` to see the prior comparable -snapshot selected for Recent Changes, or `null` when none exists. Use -`inspect sources` to review source provenance and warnings without dumping full -weather payloads. +| Command | Reads | +| --- | --- | +| `inspect reports` | Metadata files under the workspace snapshots tree. | +| `inspect metadata RUN_ID` | Metadata located by RunID. | +| `inspect modules RUN_ID` | The module snapshot path recorded in metadata. | +| `inspect data-package RUN_ID` | The data-package path recorded in metadata. | +| `inspect prior RUN_ID` | The run metadata, then compatible earlier metadata for its comparison policy. | +| `inspect sources RUN_ID` | Source provenance and warnings in the run metadata. | -## Recent Changes - -Recent Changes are computed from structured module snapshots, not rendered -Markdown or YAML text. - -Daily Report compares with prior Daily Report snapshots for the same valid -local date. Today Report compares with prior Today Report snapshots for the -same valid local date. Tomorrow Report compares with prior Tomorrow Report -snapshots for the same valid local date. 3-Day Outlook compares with prior -compatible 3-Day snapshots for the same valid local date. Weekend Outlook -compares with prior compatible Weekend snapshots for the same weekend window. -Hourly Report and Storm Report leave Recent Changes empty. - -When no prior comparable snapshot exists, or no configured threshold is crossed, -`recentChanges.items` is empty. +A missing snapshots directory produces no listed reports. An unknown or empty +RunID is an error; use `inspect reports` to obtain a valid value. ## Recovery -A failed generation run may still leave useful artifacts: +Keep the workspace when a run fails: artifacts reached before the failure +remain available where they can be safely persisted. -- If `scriptorium render` returns a result with a nonzero exit code, the - preflight JSON and metadata are written for inspection. -- If `scriptorium run` exits nonzero after writing a report, the managed report - and metadata remain available. -- Generated-text failures for Daily, Today, Tomorrow, and Hourly reports preserve - available intermediate artifacts, such as the structured run result, raw - generated-text JSON, validated generated text, and render context. Metadata - links those paths when it can be safely written. -- If single-report distributor notification fails, report artifacts and final - metadata remain available, but the report command returns nonzero. -- If batch distributor notification fails, report artifacts and final metadata - remain available, the top-level batch notification links the debug artifact, - and the batch command returns nonzero. -- For batch commands, inspect the stdout JSON summary first, then inspect the - artifact paths for each failed report or the top-level notification path. +- A preflight failure can leave the preflight artifact and metadata. +- A report-generation failure can leave the managed report, module snapshot, + data package, and metadata. +- A generated-text failure can leave raw text, the structured run result, or a + validated generated-text and render-context artifact, depending on where it + stopped. +- A single-report notification failure preserves the report and final metadata, + including its notification artifact when it was written. +- A batch notification failure preserves each report's artifacts and adds the + top-level batch notification artifact. -For a bad report, start with: - -```text -weatherreporter inspect metadata RUN_ID -weatherreporter inspect sources RUN_ID -weatherreporter inspect modules RUN_ID -weatherreporter inspect data-package RUN_ID -weatherreporter inspect prior RUN_ID -``` +Use the RunID from the action summary with the inspection commands above. For +a batch failure, inspect the summary first, then inspect the affected report +RunIDs or the batch notification path. Do not remove the whole workspace as a +first response; retain it until the failure is understood. ## Operational Caveats -- The application uses one configured Weather API endpoint. -- The application writes local filesystem state only. -- The application does not implement resume, cleanup, archive, remote storage, - daemon operation, or automatic storm monitoring. -- Generated reports and Scriptorium stderr can contain sensitive operational - context. Store workspace artifacts with appropriate filesystem permissions. +- Workspace files, generated reports, and Scriptorium stderr can contain + sensitive operational context. Set appropriate filesystem permissions and do + not publish them unintentionally. +- Weatherreporter uses one configured Weather API endpoint and local workspace + state. +- It does not provide automatic resume, cleanup, archival, remote state, daemon + operation, or automatic storm monitoring.