eric/weatherreporter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Document batch distributor uploads

Browse Source
This commit is contained in:
Eric Rakestraw
2026-06-17 21:11:28 +00:00
parent 662db5e511
commit b3637cddd6
7 changed files with 300 additions and 97 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
docs/cli.md
View File

@@ -60,11 +60,13 @@ stops the batch before any report is generated. Batch runs continue independent
reports after a later report failure, print a JSON summary to stdout, write
compact status lines to stderr, and return nonzero when any report failed.
`--out-dir` writes extra Markdown copies for the operator; distributor
notification uses each managed report path, not the extra copies. Today and
notification uses managed report paths, not the extra copies. Today and
Tomorrow use their report default copy names, and dynamic Daily copies use
`daily-YYYY-MM-DD.md`. When notification is enabled, batch summaries and status
lines include notification status, accepted distributor run ID, or notification
error fields for each attempted report.
`daily-YYYY-MM-DD.md`. When distributor and batch notification are enabled, a
fully successful batch uploads one distributor bundle after report generation
finishes. The JSON summary exposes that upload as a top-level `notification`
object, and stderr includes one `batchNotification` status line. If any planned
report fails, the batch notification is skipped for the whole batch.
Hourly Report, 3-Day Outlook, and Weekend Outlook are explicit only; they are
not included in `run morning` or `run evening`.

38
docs/config.md
View File

@@ -78,10 +78,13 @@ missing directories, and unreadable files fail config loading.
### `notify`
`notify.distributor` controls distributor notification after successful report
generation. It is disabled by default and does not add CLI flags. When enabled,
weatherreporter uploads one distributor bundle per generated report after
report rendering succeeds and final metadata is saved.
`notify.distributor` controls distributor uploads after successful report
rendering. It is disabled by default and does not add CLI flags. When enabled,
`generate <report>` uploads one distributor bundle for the generated report
after final metadata is saved. `run morning` and `run evening` use
`notify.distributor.batch`: when batch notification is enabled and every
planned report succeeds, weatherreporter uploads one distributor bundle that
contains all managed Markdown reports from that batch.
- `enabled`: whether distributor notification config is active. Default:
`false`.
@@ -92,15 +95,18 @@ report rendering succeeds and final metadata is saved.
- `timeout`: distributor operation timeout. Must be greater than zero when
enabled. Default: `30s`.
- `failure_policy`: must be `error` when enabled. Default: `error`.
- `pipeline_id_template`: template for the distributor pipeline ID. Required
when enabled. Default: empty.
- `bundle_id_template`: template for distributor bundle IDs. Default:
`weatherreporter.{location_id}.{report_id}`.
- `idempotency_key_template`: template for distributor idempotency keys.
Default: `{bundle_id}.{run_id}`.
- `pipeline_id_template`: template for single-report distributor pipeline IDs.
Required when enabled. Default: empty.
- `bundle_id_template`: template for single-report distributor bundle IDs.
Default: `weatherreporter.{location_id}.{report_id}`.
- `idempotency_key_template`: template for single-report distributor
idempotency keys. Default: `{bundle_id}.{run_id}`.
- `report_path_templates`: ordered list of templates for Markdown report paths
inside the distributor bundle. Each rendered path maps to the same managed
Markdown report source. Default:
inside distributor bundles. For `generate <report>`, each rendered path maps
to the generated report's managed Markdown source. For `run morning` and
`run evening`, the templates are rendered once per included report and each
rendered path maps that report's managed Markdown source into the one batch
bundle. Default:
```yaml
- "{valid_start_date}/{artifact_group}/{valid_start_date}-{artifact_group}-{run_id}.md"
```
@@ -116,7 +122,7 @@ report rendering succeeds and final metadata is saved.
keys. Required when distributor notification and batch notification are
enabled. Default: `{bundle_id}.{batch_run_id}`.
Supported template variables are `location_id`, `report_id`, `run_id`,
Single-report templates support `location_id`, `report_id`, `run_id`,
`artifact_group`, `batch_output_name`, `valid_start_date`, `valid_end_date`,
`valid_start_time`, `valid_end_time`, `valid_start_stamp`, and
`valid_end_stamp`. Date values use `YYYY-MM-DD`, time values use `HHMM`, and
@@ -129,12 +135,16 @@ report stream. The rendered idempotency key is the per-run retry identity.
Batch templates support `location_id`, `batch`, `batch_run_id`, and
`batch_started_date`. Batch idempotency templates may also use `bundle_id`.
`batch_started_date` is the batch start date in the effective report timezone.
Batch bundle IDs identify a logical batch stream; batch idempotency keys
identify a specific retryable batch attempt.
Rendered report paths must be unique relative paths with `/` separators. They
must not contain backslashes, empty path segments, `.`, `..`, `manifest.json`,
or `.distributor.json`.
or `.distributor.json`. In a batch upload, uniqueness is checked across every
rendered bundle path for every included report before distributor is called.
Managed Markdown report paths are the only upload source files; copies written
with `--out` or `--out-dir` are never uploaded.
The upload token is read from the environment variable named by `token_env`
after config loading and `secrets.directory` processing. Config files should

43
docs/internal/app-orchestration.md
View File

@@ -32,7 +32,7 @@ Outputs:
result details, generated-text artifact paths when applicable, and
notification result when attempted
- batch summaries with per-report status, artifact paths, error text, and
notification outcome when attempted
one top-level batch notification result when attempted or skipped
- saved Weather API bundle JSON for explicit bundle collection workflows
- inspection JSON values for reports, metadata, module snapshots, data
packages, prior snapshots, and source provenance
@@ -74,7 +74,8 @@ metadata stay in `internal/state`.
- `workspace.*` for filesystem state
- `dayparts` for daily and outlook summarization
- `recent_change.*` for structured Recent Changes thresholds
- `notify.distributor.*` for optional notification after report generation
- `notify.distributor.*` for optional single-report and batch notification
after report generation
Output copy flags are command request fields. They are not configuration
defaults.
@@ -151,10 +152,26 @@ A collection failure stops the batch before planning or report generation.
After planning succeeds, batch generation continues independent reports after a
failure, records each result, writes compact status lines to stderr, emits a
JSON summary to stdout, and returns an aggregate error when any report failed.
When notification is enabled, each successfully generated report is notified
independently. Notification failure marks that report failed, records
notification fields in the batch result, and does not stop later reports.
`--out-dir` copies are never used as notification source files.
Batch report generation suppresses per-report distributor notification. After
all planned reports finish, app orchestration evaluates batch notification:
1. If distributor notification is disabled, the batch notification result is
omitted.
2. If batch notification is disabled, the batch notification result is omitted
and there is no per-report fallback upload.
3. If any planned report failed, the batch notification result is `skipped`
with reason `one or more reports failed`, and distributor is not called.
4. If every report succeeded, app orchestration renders batch pipeline, bundle
ID, and idempotency key templates, renders `report_path_templates` for each
included report, validates every managed source path and bundle path, checks
duplicate bundle paths across the batch, calls the notifier once with a
multi-file request, and saves a batch notification debug artifact.
Batch notification failure records a top-level failed notification, increments
the aggregate batch failure count, and returns an aggregate batch error without
marking individual report items failed. `--out-dir` copies are never used as
notification source files.
## Inspection Workflow
@@ -172,8 +189,10 @@ inspection view.
- Render and run errors preserve Scriptorium stderr and exit-code context.
- Generated-text report errors preserve available intermediate artifacts and do
not create extra output copies.
- Notification errors are wrapped with report ID, RunID, and managed report path
context and are recorded separately in batch results.
- Single-report notification errors are wrapped with report ID, RunID, and
managed report path context.
- Batch notification errors are recorded on the top-level batch notification
result and do not change individual report item status.
- Metadata and artifact path errors include filesystem context.
- Batch failures are recorded per report and surfaced through an aggregate
batch error.
@@ -201,5 +220,9 @@ Inspect:
a raw data package.
- Recent Changes are computed from structured module snapshots.
- Metadata links artifacts produced for a run.
- Distributor notification maps the managed Markdown report path to configured
bundle paths; extra output copies are not upload sources.
- Single-report distributor notification maps the managed Markdown report path
to configured bundle paths.
- Batch distributor notification maps each included managed Markdown report
path to bundle paths rendered for that report and uploads once for the
batch.
- Extra output copies are not upload sources.

33
docs/internal/distributor-adapter.md
View File

@@ -6,10 +6,10 @@ This document describes the distributor upload adapter in
## Purpose
The adapter submits generated weatherreporter Markdown reports to a configured
distributor HTTP upload endpoint. It isolates distributor package types,
token-env lookup, upload client construction, source-bundle file mapping,
timeout handling, status polling, and upload error wrapping from app
orchestration.
distributor HTTP upload endpoint. It supports one or more file mappings per
upload request. It isolates distributor package types, token-env lookup, upload
client construction, source-bundle file mapping, timeout handling, status
polling, and upload error wrapping from app orchestration.
## Inputs And Outputs
@@ -21,7 +21,7 @@ Inputs:
- pipeline ID
- bundle ID
- idempotency key
- source Markdown report path and bundle-relative path mappings
- source Markdown report paths and bundle-relative path mappings
- bundle created timestamp
- context for cancellation
@@ -40,8 +40,9 @@ Outputs:
The app layer passes weatherreporter-owned request values to the adapter. The
adapter does not choose report types, render templates, select output copies,
configure destinations, wait for downstream publication, transform Markdown, or
persist notification state.
decide whether an upload represents one report or a batch, configure
destinations, wait for downstream publication, transform Markdown, or persist
notification state.
Full upstream distributor package and HTTP contract details stay under
`docs/integrations/distributor/`.
@@ -54,14 +55,19 @@ The adapter is built from `notify.distributor` config:
- `token_env`
- `timeout`
The app layer renders pipeline ID, bundle ID, idempotency key, and bundle paths
from:
The app layer renders single-report pipeline ID, bundle ID, idempotency key,
and bundle paths from:
- `pipeline_id_template`
- `bundle_id_template`
- `idempotency_key_template`
- `report_path_templates`
For batch uploads, the app layer renders pipeline ID, bundle ID, and
idempotency key from `notify.distributor.batch.*`, renders
`report_path_templates` once per included report, and passes the resulting
multi-file request to this adapter.
The token value is read from the environment variable named by `token_env`
after config loading and `secrets.directory` processing.
@@ -70,9 +76,9 @@ after config loading and `secrets.directory` processing.
The adapter calls distributor `UploadFiles` with one or more file mappings:
- pipeline ID: the rendered distributor workflow selector
- source path: the managed Markdown report path selected by app orchestration
- bundle paths: rendered bundle-relative report paths
- created: the report generation timestamp
- source paths: managed Markdown report paths selected by app orchestration
- bundle paths: rendered bundle-relative report paths for each source
- created: the report or batch generation timestamp
The adapter creates a distributor upload client with the configured endpoint,
bearer token, and timeout-backed HTTP client. It also wraps the upload context
@@ -115,7 +121,8 @@ distributor service.
## Invariants
- Distributor package types do not leak outside the adapter.
- Only the managed Markdown report is uploaded.
- Only managed Markdown report paths selected by app orchestration are
uploaded.
- The adapter never scans the workspace.
- Token values are not included in errors, CLI output, metadata, docs, or
examples.

29
docs/internal/state.md
View File

@@ -31,6 +31,7 @@ Outputs:
- render context JSON path
- managed Markdown report path
- metadata JSON path
- distributor notification debug artifact paths
- prior comparable snapshot metadata
- loaded module snapshot, data package, generated text, generated-text run
result, or render context
@@ -74,6 +75,7 @@ valid-period start date for dated artifacts, and the RunID.
data-packages/<artifact_group>/<YYYY-MM-DD>/<run_id>.data_package.yaml
preflight/<artifact_group>/<YYYY-MM-DD>/<run_id>.render.json
notifications/<artifact_group>/<YYYY-MM-DD>/<run_id>.distributor.json
notifications/batches/<batch>/<YYYY-MM-DD>/<batch_run_id>.distributor.json
reports/<artifact_group>/<run_id>.md
```
@@ -85,6 +87,10 @@ Scriptorium run result, validated generated text, and render context artifacts.
Markdown-report metadata omits those generated-text fields. Report listing
walks metadata files under the snapshots directory.
Batch notification artifacts are stored under the notifications tree rather
than report metadata because they describe a batch-level upload. The date
directory is the batch start date in the effective report timezone.
## Prior Lookup
Prior snapshot lookup reads stored metadata through the shared lookup path and
@@ -118,6 +124,27 @@ atomically when notification is attempted and include rendered distributor
pipeline ID, bundle ID, idempotency key, bundle paths, upload status, latest
run status, and redacted errors.
Single-report notification artifacts use schema version
`weatherreporter.distributor_notification.v1` and record one managed source
path plus that source's bundle paths. Batch notification artifacts use schema
version `weatherreporter.batch_distributor_notification.v1` and record:
- `batch`
- `batchRunId`
- `attemptedAt`
- `endpoint`
- `pipelineId`
- `bundleId`
- `idempotencyKey`
- `bundleCreated`
- `includedReports`, each with `reportId`, `runId`, `sourcePath`, and
`bundlePaths`
- `status`
- `upload`
- `runStatus`
- `statusError`
- `error`
Inspection helpers read existing metadata, module snapshot, data package,
generated text, generated-text run result, and render context files. Missing
metadata directories return no inspection records or no prior snapshot rather
@@ -145,4 +172,6 @@ Inspect:
- Metadata links artifacts produced for a run.
- Generated-text artifacts live under the snapshots tree beside module
snapshots and metadata.
- Batch notification artifacts live under `notifications/batches` and are not
linked from report metadata.
- Prior lookup is based on structured metadata, not rendered report text.

128
docs/operations.md
View File

@@ -57,12 +57,14 @@ batch before any report is generated.
After planning succeeds, batch commands print a JSON summary to stdout, write
compact per-report status lines to stderr, continue independent reports after
one report fails, and return nonzero when any report failed. When notification
is configured, the summary and status lines include notification status,
accepted distributor run ID, or notification error fields for each attempted
report. `--out-dir PATH` writes extra Markdown copies using report default
filenames such as `today.md` and `tomorrow.md`; dynamic Daily copies use
`daily-YYYY-MM-DD.md`. These copies are not used as distributor upload sources.
one report fails, and return nonzero when any report failed. Batch commands do
not upload each report independently. When distributor notification and batch
notification are enabled, weatherreporter uploads one distributor bundle only
after every planned report succeeds. If any report fails, the batch upload is
skipped for the whole batch. `--out-dir PATH` writes extra Markdown copies
using report default filenames such as `today.md` and `tomorrow.md`; dynamic
Daily copies use `daily-YYYY-MM-DD.md`. These copies are not used as
distributor upload sources.
## Filesystem Layout
@@ -160,6 +162,13 @@ workspace/
YYYY-MM-DD/
<run_id>.render.json
notifications/
batches/
morning/
YYYY-MM-DD/
<batch_run_id>.distributor.json
evening/
YYYY-MM-DD/
<batch_run_id>.distributor.json
daily/
YYYY-MM-DD/
<run_id>.distributor.json
@@ -206,14 +215,21 @@ period do not overwrite each other.
RunIDs are based on generation time plus report ID. Reports that can be
generated more than once in a single command may append a report-specific
disambiguator. Daily appends the local valid date so multiple dynamic Daily
reports in one batch have distinct managed artifacts and notification
idempotency keys:
reports in one batch have distinct managed artifacts:
```text
20260529T100000.123456789Z_daily_2026-05-31
20260529T100000.123456789Z_today
```
Batch notification RunIDs use the batch start timestamp plus the batch command
name:
```text
20260529T100000.123456789Z_morning
20260529T220000.123456789Z_evening
```
Each generated report writes metadata that links:
- RunID, report ID, variant, and prompt ID
@@ -227,19 +243,33 @@ Each generated report writes metadata that links:
generated-text-template reports
- distributor notification debug artifact path, when notification is attempted
Batch summaries include report status, error text when applicable, notification
outcome when attempted, valid period, and known artifact paths for each
attempted report. Notification fields are `notificationStatus`,
`notificationRunId`, and `notificationError`.
Batch summaries include report status, error text when applicable, valid
period, and known artifact paths for each attempted report. Single-report
notification fields on report items are empty for batch commands. When a batch
notification is attempted, skipped, or fails, the summary includes one
top-level `notification` object with fields such as `status`, `reason`,
`runId`, `pipelineId`, `bundleId`, `idempotencyKey`, `path`,
`includedReports`, and `error`.
## Distributor Notification
Distributor notification is configured with `notify.distributor` and is
disabled by default. When enabled, weatherreporter uploads the managed Markdown
report path recorded in the report result and metadata. That single source file
can be mapped to one or more configured bundle paths. By default, it is mapped
to one dated report path. Extra copies written by `--out` or `--out-dir` are
operator conveniences only.
disabled by default. For `generate <report>`, weatherreporter uploads the
managed Markdown report path recorded in the report result and metadata. That
single source file can be mapped to one or more configured bundle paths. By
default, it is mapped to one dated report path. Extra copies written by `--out`
or `--out-dir` are operator conveniences only.
For `run morning` and `run evening`, per-report notification is suppressed. If
`notify.distributor.enabled` and `notify.distributor.batch.enabled` are both
true, the batch uploads once after all reports finish successfully. The upload
contains one file mapping set per included report. Each mapping uses the
managed Markdown report as the source and renders `report_path_templates` using
that report's normal template values. All rendered bundle paths across the
batch must be unique. If any report fails, weatherreporter records a top-level
notification status of `skipped` with reason `one or more reports failed` and
does not call distributor. If batch notification is disabled, run commands do
not fall back to per-report uploads.
The rendered pipeline ID selects the configured distributor `http_upload`
workflow. The default bundle ID is a stable logical source identity derived from
@@ -249,31 +279,40 @@ producer name, location ID, and report ID:
weatherreporter.{location_id}.{report_id}
```
The default idempotency key appends RunID to the rendered bundle ID so each
report generation has a distinct retry identity. The default bundle path uses
the valid-period start date, artifact group, and RunID. Distributor owns
destination merge, retention, and derived snapshot behavior such as `latest`.
For Daily, the default report ID and artifact group values are both `daily`,
and the default output filename value is `daily.md`.
For Today, the default report ID and artifact group values are both `today`,
and the batch output filename value is `today.md`.
The default single-report idempotency key appends RunID to the rendered bundle
ID so each report generation has a distinct retry identity. The default bundle
path uses the valid-period start date, artifact group, and RunID. Batch bundle
IDs default to `weatherreporter.{location_id}.{batch}`, and batch idempotency
keys default to `{bundle_id}.{batch_run_id}`. Distributor owns destination
merge, retention, and derived snapshot behavior such as `latest`. For Daily,
the default report ID and artifact group values are both `daily`, and the
default output filename value is `daily.md`. For Today, the default report ID
and artifact group values are both `today`, and the batch output filename value
is `today.md`.
Notification happens after final metadata save for generated reports.
Collection, module snapshot, data-package, render preflight, Scriptorium run,
generated-text validation, template rendering, and metadata-save failures do
not trigger notification. A notification failure fails that report.
In a batch, other reports continue, the failed report includes notification
fields in the JSON summary, and the batch returns nonzero.
Single-report notification happens after final metadata save for generated
reports. Batch notification happens after all planned reports have finished and
only when all report generations succeeded. Collection, module snapshot,
data-package, render preflight, Scriptorium run, generated-text validation,
template rendering, and metadata-save failures do not trigger notification. A
single-report notification failure fails that report. A batch notification
failure makes the batch return nonzero and increments the aggregate failure
count, but individual report items remain succeeded.
Each notification attempt writes a debug artifact under `notifications/`. The
artifact records the rendered pipeline ID, bundle ID, idempotency key, managed
source path, bundle-relative paths, bundle created timestamp, accepted upload
response, and the latest distributor run status response when available.
Weatherreporter polls status until distributor reports `succeeded` or `failed`,
or until the configured notification timeout expires. The run status includes
the distributor status, error text, and raw run report JSON, which can show
actions such as `replace_older`, `skip_same`, `skip_destination_newer`, or
`failed`. Token values are not written.
Each notification attempt writes a debug artifact under `notifications/`.
Single-report artifacts live under
`notifications/<artifact_group>/<YYYY-MM-DD>/<run_id>.distributor.json`. Batch
artifacts live under
`notifications/batches/<batch>/<YYYY-MM-DD>/<batch_run_id>.distributor.json`,
where the date directory is the batch start date in the effective report
timezone. The artifact records the rendered pipeline ID, bundle ID,
idempotency key, managed source paths, bundle-relative paths, bundle created
timestamp, accepted upload response, and the latest distributor run status
response when available. Weatherreporter polls status until distributor reports
`succeeded` or `failed`, or until the configured notification timeout expires.
The run status includes the distributor status, error text, and raw run report
JSON, which can show actions such as `replace_older`, `skip_same`,
`skip_destination_newer`, or `failed`. Token values are not written.
Weatherreporter is responsible for selecting the managed Markdown report,
constructing a source bundle, and submitting it to the configured distributor
@@ -333,10 +372,13 @@ A failed generation run may still leave useful artifacts:
available intermediate artifacts, such as the structured run result, raw
generated-text JSON, validated generated text, and render context. Metadata
links those paths when it can be safely written.
- If distributor notification fails, report artifacts and final metadata remain
available, but the report or batch command returns nonzero.
- If single-report distributor notification fails, report artifacts and final
metadata remain available, but the report command returns nonzero.
- If batch distributor notification fails, report artifacts and final metadata
remain available, the top-level batch notification links the debug artifact,
and the batch command returns nonzero.
- For batch commands, inspect the stdout JSON summary first, then inspect the
artifact paths for each failed report.
artifact paths for each failed report or the top-level notification path.
For a bad report, start with:

116
docs/troubleshooting.md
View File

@@ -236,20 +236,107 @@ Relevant docs: [Report Templates](templates.md),
Symptom: `run morning` or `run evening` returns nonzero.
Likely cause: weather collection failed before planning, or at least one
planned report failed after planning succeeded.
planned report failed after planning succeeded, or every report succeeded but
the top-level batch distributor notification failed.
Diagnostic: if stdout contains a JSON summary, inspect each failed report item
and stderr status line. If no summary was emitted, inspect the command error;
configuration, Weather API collection, or batch validation failed before any
report artifacts were created.
and the top-level `notification` object. Stderr includes one
`batchNotification` line when batch notification is attempted, skipped, or
fails. If no summary was emitted, inspect the command error; configuration,
Weather API collection, or batch validation failed before any report artifacts
were created.
Safe fix: for collection failures, fix the configuration or upstream Weather
API availability and rerun the batch. For report failures, use the failed
report's artifact paths from the summary, then inspect metadata, sources,
module snapshot, and data package for that RunID.
module snapshot, and data package for that RunID. For a batch notification
failure, inspect the notification artifact path from the top-level
`notification.path`.
Relevant docs: [CLI reference](cli.md), [Operations guide](operations.md).
## Batch Upload Skipped
Symptom: a batch JSON summary contains
`"notification":{"status":"skipped","reason":"one or more reports failed"}`.
Likely cause: at least one planned report failed, so weatherreporter did not
call distributor for the batch.
Diagnostic: inspect the failed report items in the batch JSON summary and the
matching stderr report lines. A skipped batch notification has no distributor
run ID and no notification artifact path.
Safe fix: fix the report-generation failure first, then rerun the batch. The
batch upload is all-or-nothing.
Relevant docs: [Operations guide](operations.md).
## Batch Upload Fails
Symptom: every report item in a batch summary is succeeded, but the batch
returns nonzero and the top-level notification has `status: "failed"`.
Likely cause: the distributor upload was rejected, the distributor service was
unavailable, status polling reached a terminal distributor failure, or
weatherreporter rejected the batch file mapping before upload.
Diagnostic: inspect `notification.error`, `notification.pipelineId`,
`notification.bundleId`, `notification.idempotencyKey`, and
`notification.path` in stdout. Then inspect the notification artifact; it
records included report source paths, bundle paths, upload status, distributor
run status, status lookup error, and raw status report JSON when available.
Safe fix: fix the endpoint, token, distributor pipeline, batch identity
templates, or report path templates indicated by the error, then rerun the
batch. Individual report artifacts from the failed batch notification remain
available and do not need to be regenerated for diagnosis.
Relevant docs: [Configuration reference](config.md),
[Operations guide](operations.md).
## Duplicate Batch Bundle Path
Symptom: a batch returns nonzero with duplicate bundle path context before a
distributor run ID is accepted.
Likely cause: `notify.distributor.report_path_templates` rendered the same
bundle-relative path for two included reports in the same batch.
Diagnostic: inspect the error in stdout or stderr. The validation error
includes the duplicate bundle path plus the report IDs, RunIDs, and managed
source paths involved.
Safe fix: change `notify.distributor.report_path_templates` so every report in
a batch renders a unique path. Include values such as `{artifact_group}`,
`{valid_start_date}`, `{batch_output_name}`, or `{run_id}` when needed.
Relevant docs: [Configuration reference](config.md),
[Operations guide](operations.md).
## Distributor Source Conflict
Symptom: distributor accepts or rejects an upload with conflict context for a
source, destination, digest, or idempotency key.
Likely cause: the rendered bundle ID or idempotency key does not match the
intended producer identity. A bundle ID identifies the logical source stream;
an idempotency key identifies a retry of the same upload request.
Diagnostic: inspect the report notification artifact linked from metadata or
the batch notification artifact linked from the top-level notification path.
Compare the rendered pipeline ID, bundle ID, idempotency key, included source
paths, and bundle paths with `notify.distributor.*` templates and distributor
pipeline state.
Safe fix: keep bundle ID templates stable for the source stream that should be
updated, and keep idempotency keys stable only for retries of the same generated
content. Do not reuse one idempotency key for different report or batch
content.
Relevant docs: [Operations guide](operations.md),
[Distributor adapter internals](internal/distributor-adapter.md).
## Invalid Secrets Directory
Symptom: config loading fails with `read secrets directory`, `secret file`, or
@@ -296,12 +383,14 @@ within the same distributor token and pipeline. By default the bundle ID is a
stable report-stream identity and the idempotency key appends RunID.
Diagnostic: inspect the failed batch JSON or stderr line for pipeline, bundle,
and idempotency context. Compare the configured templates with the report RunID
and report path.
and idempotency context. For batch commands, use the top-level notification
object rather than per-report notification fields. Compare the configured
templates with the report RunID or batch RunID and report path.
Also inspect the notification artifact linked from metadata. It records the
rendered pipeline ID, bundle ID, idempotency key, upload result, distributor run
status, status error, and raw run report JSON when available.
Also inspect the notification artifact linked from metadata or from the
top-level batch notification path. It records the rendered pipeline ID, bundle
ID, idempotency key, upload result, distributor run status, status error, and
raw run report JSON when available.
Safe fix: keep idempotency templates stable for retries of the same generated
report, but do not reuse the same rendered key for different generated report
@@ -319,7 +408,8 @@ Likely cause: the distributor endpoint rejected the token, pipeline ID, bundle
ID, idempotency key, source file, or one of the rendered bundle paths.
Diagnostic: inspect stdout JSON or stderr status lines for
`notificationError`. Confirm `notify.distributor.endpoint`,
`notificationError` or the top-level batch notification `error`. Confirm
`notify.distributor.endpoint`,
`notify.distributor.pipeline_id_template`,
`notify.distributor.report_path_templates`, and token configuration. Token
values are redacted from weatherreporter errors.
@@ -346,8 +436,8 @@ Likely cause: the configured distributor endpoint is unreachable, slow, or
temporarily unavailable.
Diagnostic: check network access from the weatherreporter host to
`notify.distributor.endpoint`. For batch runs, inspect which reports have
`notificationStatus: "failed"`.
`notify.distributor.endpoint`. For batch runs, inspect the top-level
notification object and the artifact linked by `notification.path`.
Safe fix: restore distributor service availability and rerun the affected
report or batch. Stable idempotency keys make retrying the same generated report