Document batch distributor uploads

docs/troubleshooting.md

@@ -236,20 +236,107 @@ Relevant docs: [Report Templates](templates.md),
 Symptom: `run morning` or `run evening` returns nonzero.
 
 Likely cause: weather collection failed before planning, or at least one
-planned report failed after planning succeeded.
+planned report failed after planning succeeded, or every report succeeded but
+the top-level batch distributor notification failed.
 
 Diagnostic: if stdout contains a JSON summary, inspect each failed report item
-and stderr status line. If no summary was emitted, inspect the command error;
-configuration, Weather API collection, or batch validation failed before any
-report artifacts were created.
+and the top-level `notification` object. Stderr includes one
+`batchNotification` line when batch notification is attempted, skipped, or
+fails. If no summary was emitted, inspect the command error; configuration,
+Weather API collection, or batch validation failed before any report artifacts
+were created.
 
 Safe fix: for collection failures, fix the configuration or upstream Weather
 API availability and rerun the batch. For report failures, use the failed
 report's artifact paths from the summary, then inspect metadata, sources,
-module snapshot, and data package for that RunID.
+module snapshot, and data package for that RunID. For a batch notification
+failure, inspect the notification artifact path from the top-level
+`notification.path`.
 
 Relevant docs: [CLI reference](cli.md), [Operations guide](operations.md).
 
+## Batch Upload Skipped
+
+Symptom: a batch JSON summary contains
+`"notification":{"status":"skipped","reason":"one or more reports failed"}`.
+
+Likely cause: at least one planned report failed, so weatherreporter did not
+call distributor for the batch.
+
+Diagnostic: inspect the failed report items in the batch JSON summary and the
+matching stderr report lines. A skipped batch notification has no distributor
+run ID and no notification artifact path.
+
+Safe fix: fix the report-generation failure first, then rerun the batch. The
+batch upload is all-or-nothing.
+
+Relevant docs: [Operations guide](operations.md).
+
+## Batch Upload Fails
+
+Symptom: every report item in a batch summary is succeeded, but the batch
+returns nonzero and the top-level notification has `status: "failed"`.
+
+Likely cause: the distributor upload was rejected, the distributor service was
+unavailable, status polling reached a terminal distributor failure, or
+weatherreporter rejected the batch file mapping before upload.
+
+Diagnostic: inspect `notification.error`, `notification.pipelineId`,
+`notification.bundleId`, `notification.idempotencyKey`, and
+`notification.path` in stdout. Then inspect the notification artifact; it
+records included report source paths, bundle paths, upload status, distributor
+run status, status lookup error, and raw status report JSON when available.
+
+Safe fix: fix the endpoint, token, distributor pipeline, batch identity
+templates, or report path templates indicated by the error, then rerun the
+batch. Individual report artifacts from the failed batch notification remain
+available and do not need to be regenerated for diagnosis.
+
+Relevant docs: [Configuration reference](config.md),
+[Operations guide](operations.md).
+
+## Duplicate Batch Bundle Path
+
+Symptom: a batch returns nonzero with duplicate bundle path context before a
+distributor run ID is accepted.
+
+Likely cause: `notify.distributor.report_path_templates` rendered the same
+bundle-relative path for two included reports in the same batch.
+
+Diagnostic: inspect the error in stdout or stderr. The validation error
+includes the duplicate bundle path plus the report IDs, RunIDs, and managed
+source paths involved.
+
+Safe fix: change `notify.distributor.report_path_templates` so every report in
+a batch renders a unique path. Include values such as `{artifact_group}`,
+`{valid_start_date}`, `{batch_output_name}`, or `{run_id}` when needed.
+
+Relevant docs: [Configuration reference](config.md),
+[Operations guide](operations.md).
+
+## Distributor Source Conflict
+
+Symptom: distributor accepts or rejects an upload with conflict context for a
+source, destination, digest, or idempotency key.
+
+Likely cause: the rendered bundle ID or idempotency key does not match the
+intended producer identity. A bundle ID identifies the logical source stream;
+an idempotency key identifies a retry of the same upload request.
+
+Diagnostic: inspect the report notification artifact linked from metadata or
+the batch notification artifact linked from the top-level notification path.
+Compare the rendered pipeline ID, bundle ID, idempotency key, included source
+paths, and bundle paths with `notify.distributor.*` templates and distributor
+pipeline state.
+
+Safe fix: keep bundle ID templates stable for the source stream that should be
+updated, and keep idempotency keys stable only for retries of the same generated
+content. Do not reuse one idempotency key for different report or batch
+content.
+
+Relevant docs: [Operations guide](operations.md),
+[Distributor adapter internals](internal/distributor-adapter.md).
+
 ## Invalid Secrets Directory
 
 Symptom: config loading fails with `read secrets directory`, `secret file`, or
@@ -296,12 +383,14 @@ within the same distributor token and pipeline. By default the bundle ID is a
 stable report-stream identity and the idempotency key appends RunID.
 
 Diagnostic: inspect the failed batch JSON or stderr line for pipeline, bundle,
-and idempotency context. Compare the configured templates with the report RunID
-and report path.
+and idempotency context. For batch commands, use the top-level notification
+object rather than per-report notification fields. Compare the configured
+templates with the report RunID or batch RunID and report path.
 
-Also inspect the notification artifact linked from metadata. It records the
-rendered pipeline ID, bundle ID, idempotency key, upload result, distributor run
-status, status error, and raw run report JSON when available.
+Also inspect the notification artifact linked from metadata or from the
+top-level batch notification path. It records the rendered pipeline ID, bundle
+ID, idempotency key, upload result, distributor run status, status error, and
+raw run report JSON when available.
 
 Safe fix: keep idempotency templates stable for retries of the same generated
 report, but do not reuse the same rendered key for different generated report
@@ -319,7 +408,8 @@ Likely cause: the distributor endpoint rejected the token, pipeline ID, bundle
 ID, idempotency key, source file, or one of the rendered bundle paths.
 
 Diagnostic: inspect stdout JSON or stderr status lines for
-`notificationError`. Confirm `notify.distributor.endpoint`,
+`notificationError` or the top-level batch notification `error`. Confirm
+`notify.distributor.endpoint`,
 `notify.distributor.pipeline_id_template`,
 `notify.distributor.report_path_templates`, and token configuration. Token
 values are redacted from weatherreporter errors.
@@ -346,8 +436,8 @@ Likely cause: the configured distributor endpoint is unreachable, slow, or
 temporarily unavailable.
 
 Diagnostic: check network access from the weatherreporter host to
-`notify.distributor.endpoint`. For batch runs, inspect which reports have
-`notificationStatus: "failed"`.
+`notify.distributor.endpoint`. For batch runs, inspect the top-level
+notification object and the artifact linked by `notification.path`.
 
 Safe fix: restore distributor service availability and rerun the affected
 report or batch. Stable idempotency keys make retrying the same generated report