weatherreporter/docs/operations.md

# Weatherreporter Operations

This guide covers normal operation, generated artifacts, inspection, recovery,
and operational caveats. For symptom-specific diagnosis, see
[Troubleshooting](troubleshooting.md).

## Normal Workflow

Generation commands:

```text
weatherreporter generate daily --date 2026-05-29
weatherreporter generate today
weatherreporter generate tomorrow
weatherreporter generate hourly
weatherreporter generate three-day
weatherreporter generate weekend
weatherreporter generate storm --start 2026-05-29T18:00 --end 2026-05-30T06:00
```

Generation commands resolve a report period, collect a Weather API bundle,
build a rich JSON module snapshot, build a curated YAML prompt input data
package, run `scriptorium render`, and write managed artifacts under the
configured workspace. Markdown-path reports then run `scriptorium run` directly
to the managed Markdown report path.

`generate daily`, `generate today`, `generate tomorrow`, and `generate hourly`
use the generated-text-template workflow. They run structured `scriptorium run`
to raw GeneratedText JSON, validate the structured text, save a render context,
and render the managed Markdown report from embedded templates. `generate
daily` requires `--date YYYY-MM-DD` for the selected local civil day.
`generate today` covers the selected or current local civil day. `generate
hourly` covers the six-hour rolling period from generation time in the
effective report timezone and is not included in `run morning` or
`run evening`.

When distributor notification is enabled, weatherreporter uploads the managed
Markdown report after report rendering succeeds and final metadata is saved.
`--out PATH` writes an extra Markdown copy for generated reports; it is not used
as the distributor upload source. Generate commands emit a compact JSON summary
to stdout by default. Use `--quiet` to suppress successful stdout for cron jobs
or other schedulers that only need nonzero exits and external logs.

Batch commands:

```text
weatherreporter run morning
weatherreporter run evening
```

`run morning` generates Today Report, Tomorrow Report, and a dated Daily Report
for each later future local civil day with complete hourly forecast coverage.
`run evening` generates Tomorrow Report and the same eligible future Daily
reports. Future Daily expansion starts with the day after tomorrow. A Daily
report is eligible only when the collected hourly forecast contains every
hourly period for that local civil day; partial days are skipped. Batch commands
collect weather data once before planning, and a collection failure stops the
batch before any report is generated.

After planning succeeds, batch commands print a JSON summary to stdout, write
compact per-report status lines to stderr, continue independent reports after
one report fails, and return nonzero when any report failed. Batch commands do
not upload each report independently. When distributor notification and batch
notification are enabled, weatherreporter uploads one distributor bundle only
after every planned report succeeds. If any report fails, the batch upload is
skipped for the whole batch. `--out-dir PATH` writes extra Markdown copies
using report default filenames such as `today.md` and `tomorrow.md`; dynamic
Daily copies use `daily-YYYY-MM-DD.md`. These copies are not used as
distributor upload sources. Use `--quiet` to suppress successful batch summary
and status output; failures still return nonzero.

## Filesystem Layout

The default workspace root is `workspace`.

```text
workspace/
  reports/<artifact_group>/<YYYY-MM-DD>/report.<run_id>.md

  snapshots/<artifact_group>/<YYYY-MM-DD>/modules.<run_id>.json
  snapshots/<artifact_group>/<YYYY-MM-DD>/metadata.<run_id>.json
  snapshots/<artifact_group>/<YYYY-MM-DD>/generated_text_raw.<run_id>.json
  snapshots/<artifact_group>/<YYYY-MM-DD>/generated_text_result.<run_id>.json
  snapshots/<artifact_group>/<YYYY-MM-DD>/generated_text.<run_id>.json
  snapshots/<artifact_group>/<YYYY-MM-DD>/render_context.<run_id>.json

  data-packages/<artifact_group>/<YYYY-MM-DD>/data_package.<run_id>.yaml
  preflight/<artifact_group>/<YYYY-MM-DD>/render.<run_id>.json

  notifications/<artifact_group>/<YYYY-MM-DD>/distributor.<run_id>.json
  notifications/batches/<batch>/<YYYY-MM-DD>/distributor.<batch_run_id>.json
```

Managed artifact filenames use the artifact kind and RunID, so repeated runs
for the same valid period do not overwrite each other. The date directory is
the valid-period start date in the effective report timezone. Generated-text
artifacts are written only for Daily, Today, Tomorrow, and Hourly reports.

## RunID And Metadata

RunIDs are based on generation time plus report ID. Reports that can be
generated more than once in a single command may append a report-specific
disambiguator. Daily appends the local valid date so multiple dynamic Daily
reports in one batch have distinct managed artifacts:

```text
20260529T100000.123456789Z_daily_2026-05-31
20260529T100000.123456789Z_today
```

Batch notification RunIDs use the batch start timestamp plus the batch command
name:

```text
20260529T100000.123456789Z_morning
20260529T220000.123456789Z_evening
```

Each generated report writes metadata that links:

- RunID, report ID, variant, and prompt ID
- generation time, timezone, and valid period
- source location, source hashes, and source warnings
- module snapshot path
- prompt input data package path
- preflight output path
- managed Markdown report path
- generated text schema ID and generated-text artifact paths for
  generated-text-template reports
- distributor notification debug artifact path, when notification is attempted

Batch summaries include report status, error text when applicable, valid
period, and known artifact paths for each attempted report. Single-report
notification fields on report items are empty for batch commands. When a batch
notification is attempted, skipped, or fails, the summary includes one
top-level `notification` object with fields such as `status`, `reason`,
`runId`, `pipelineId`, `bundleId`, `idempotencyKey`, `path`,
`includedReports`, and `error`.

## Distributor Notification

Distributor notification is configured with `notify.distributor` and is
disabled by default. For `generate <report>`, weatherreporter uploads the
managed Markdown report path recorded in the report result and metadata. That
single source file is mapped to report-specific bundle paths. Extra copies
written by `--out` or `--out-dir` are operator conveniences only.

For `run morning` and `run evening`, per-report notification is suppressed. If
`notify.distributor.enabled` and `notify.distributor.batch.enabled` are both
true, the batch uploads once after all reports finish successfully. The upload
contains one file mapping set per included report. Each mapping uses the
managed Markdown report as the source and report-specific path templates for
that report. All rendered bundle paths across the batch must be unique. If any
report fails, weatherreporter records a top-level
notification status of `skipped` with reason `one or more reports failed` and
does not call distributor. If batch notification is disabled, run commands do
not fall back to per-report uploads.

The rendered pipeline ID selects the configured distributor `http_upload`
workflow. The default bundle ID is a stable logical source identity derived from
producer name, location ID, and report ID:

```text
weatherreporter.{location_id}.{report_id}
```

The default single-report idempotency key appends RunID to the rendered bundle
ID so each report generation has a distinct retry identity. The default bundle
path uses the valid-period start date, artifact group, and RunID. Batch bundle
IDs default to `weatherreporter.{location_id}.{batch}`, and batch idempotency
keys default to `{bundle_id}.{batch_run_id}`. Distributor owns destination
merge, retention, and derived snapshot behavior such as `latest`. For Daily,
the default report ID and artifact group values are both `daily`, and the
default output filename value is `daily.md`. For Today, the default report ID
and artifact group values are both `today`, and the batch output filename value
is `today.md`.

Single-report notification happens after final metadata save for generated
reports. Batch notification happens after all planned reports have finished and
only when all report generations succeeded. Collection, module snapshot,
data-package, render preflight, Scriptorium run, generated-text validation,
template rendering, and metadata-save failures do not trigger notification. A
single-report notification failure fails that report. A batch notification
failure makes the batch return nonzero and increments the aggregate failure
count, but individual report items remain succeeded.

Each notification attempt writes a debug artifact under `notifications/`.
Single-report artifacts live under
`notifications/<artifact_group>/<YYYY-MM-DD>/distributor.<run_id>.json`. Batch
artifacts live under
`notifications/batches/<batch>/<YYYY-MM-DD>/distributor.<batch_run_id>.json`,
where the date directory is the batch start date in the effective report
timezone. The artifact records the rendered pipeline ID, bundle ID,
idempotency key, managed source paths, bundle-relative paths, bundle created
timestamp, accepted upload response, and the latest distributor run status
response when available. Weatherreporter polls status until distributor reports
`succeeded` or `failed`, or until the configured notification timeout expires.
The run status includes the distributor status, error text, and raw run report
JSON, which can show actions such as `replace_older`, `skip_same`,
`skip_destination_newer`, or `failed`. Token values are not written.

Weatherreporter is responsible for selecting the managed Markdown report,
constructing a source bundle, and submitting it to the configured distributor
HTTP endpoint. Distributor remains responsible for destination routing,
publication, and any downstream Markdown-to-HTML transformation. Distributor
leaves destination files alone when they are not tracked by a newly uploaded
bundle, so existing uploaded dated report paths can remain available.

## Inspection

Inspection commands read existing workspace artifacts and emit JSON to stdout.
They do not collect weather data or run `scriptorium`.

```text
weatherreporter inspect reports --limit 10
weatherreporter inspect metadata RUN_ID
weatherreporter inspect modules RUN_ID
weatherreporter inspect data-package RUN_ID
weatherreporter inspect prior RUN_ID
weatherreporter inspect sources RUN_ID
```

Use `inspect reports` to find RunIDs and artifact paths. Use
`inspect metadata` to see the artifact links recorded for a run. Use
`inspect modules` to review the persisted ordered module snapshot with rich
template-facing values, and `inspect data-package` to review the curated prompt
package passed to Scriptorium. Use `inspect prior` to see the prior comparable
snapshot selected for Recent Changes, or `null` when none exists. Use
`inspect sources` to review source provenance and warnings without dumping full
weather payloads.

## Recent Changes

Recent Changes are computed from structured module snapshots, not rendered
Markdown or YAML text.

Daily Report compares with prior Daily Report snapshots for the same valid
local date. Today Report compares with prior Today Report snapshots for the
same valid local date. Tomorrow Report compares with prior Tomorrow Report
snapshots for the same valid local date. 3-Day Outlook compares with prior
compatible 3-Day snapshots for the same valid local date. Weekend Outlook
compares with prior compatible Weekend snapshots for the same weekend window.
Hourly Report and Storm Report leave Recent Changes empty.

When no prior comparable snapshot exists, or no configured threshold is crossed,
`recentChanges.items` is empty.

## Recovery

A failed generation run may still leave useful artifacts:

- If `scriptorium render` returns a result with a nonzero exit code, the
  preflight JSON and metadata are written for inspection.
- If `scriptorium run` exits nonzero after writing a report, the managed report
  and metadata remain available.
- Generated-text failures for Daily, Today, Tomorrow, and Hourly reports preserve
  available intermediate artifacts, such as the structured run result, raw
  generated-text JSON, validated generated text, and render context. Metadata
  links those paths when it can be safely written.
- If single-report distributor notification fails, report artifacts and final
  metadata remain available, but the report command returns nonzero.
- If batch distributor notification fails, report artifacts and final metadata
  remain available, the top-level batch notification links the debug artifact,
  and the batch command returns nonzero.
- For batch commands, inspect the stdout JSON summary first, then inspect the
  artifact paths for each failed report or the top-level notification path.

For a bad report, start with:

```text
weatherreporter inspect metadata RUN_ID
weatherreporter inspect sources RUN_ID
weatherreporter inspect modules RUN_ID
weatherreporter inspect data-package RUN_ID
weatherreporter inspect prior RUN_ID
```

## Operational Caveats

- The application uses one configured Weather API endpoint.
- The application writes local filesystem state only.
- The application does not implement resume, cleanup, archive, remote storage,
  daemon operation, or automatic storm monitoring.
- Generated reports and Scriptorium stderr can contain sensitive operational
  context. Store workspace artifacts with appropriate filesystem permissions.