Updated the distributor bundle path template

This commit is contained in:
2026-06-08 10:29:42 -05:00
parent d71c7e4d28
commit 8577fc29e4
18 changed files with 286 additions and 111 deletions

View File

@@ -98,20 +98,28 @@ weatherreporter uploads one distributor bundle per generated report after
`weatherreporter.{location_id}.{report_id}`.
- `idempotency_key_template`: template for distributor idempotency keys.
Default: `{bundle_id}.{run_id}`.
- `report_path_template`: template for the Markdown report path inside the
distributor bundle. Default: `{batch_output_name}`.
- `report_path_templates`: ordered list of templates for Markdown report paths
inside the distributor bundle. Each rendered path maps to the same managed
Markdown report source. Default:
```yaml
- "{valid_start_date}/{artifact_group}/{valid_start_date}-{artifact_group}-{run_id}.md"
- "{valid_start_date}/{artifact_group}/latest.md"
```
Supported template variables are `location_id`, `report_id`, `run_id`,
`artifact_group`, and `batch_output_name`. `pipeline_id_template` and
`idempotency_key_template` may also use `bundle_id`.
`artifact_group`, `batch_output_name`, `valid_start_date`, `valid_end_date`,
`valid_start_time`, `valid_end_time`, `valid_start_stamp`, and
`valid_end_stamp`. Date values use `YYYY-MM-DD`, time values use `HHMM`, and
stamp values use `YYYY-MM-DDTHHMM` in the effective report timezone.
`pipeline_id_template` and `idempotency_key_template` may also use `bundle_id`.
The rendered pipeline ID selects the configured distributor `http_upload`
workflow. The rendered bundle ID is the stable logical source identity for the
report stream. The rendered idempotency key is the per-run retry identity.
Rendered report paths must be relative paths with `/` separators. They must not
contain backslashes, empty path segments, `.`, `..`, `manifest.json`, or
`.distributor.json`.
Rendered report paths must be unique relative paths with `/` separators. They
must not contain backslashes, empty path segments, `.`, `..`, `manifest.json`,
or `.distributor.json`.
The upload token is read from the environment variable named by `token_env`
after config loading and `secrets.directory` processing. Config files should

View File

@@ -90,8 +90,9 @@ generation returns an error after writing output, the managed report and
metadata remain inspectable. Notification is not attempted after Weather API,
briefing, prompt input, render, Scriptorium run, or metadata-save failures.
When notification is attempted, the debug artifact records request identity,
including rendered pipeline ID, accepted upload fields, distributor status
fields, raw status report JSON when available, and redacted failure context.
including rendered pipeline ID, bundle paths, accepted upload fields,
distributor status fields, raw status report JSON when available, and redacted
failure context.
`--out` copies are never used as notification source files.
## Batch Workflow
@@ -141,5 +142,5 @@ Inspect:
- Render preflight precedes Scriptorium report generation.
- Recent Changes are computed from structured briefing snapshots.
- Metadata links artifacts produced for a run.
- Distributor notification uses the managed Markdown report path, not extra
output copies.
- Distributor notification maps the managed Markdown report path to configured
bundle paths; extra output copies are not upload sources.

View File

@@ -21,8 +21,7 @@ Inputs:
- pipeline ID
- bundle ID
- idempotency key
- source Markdown report path
- bundle-relative Markdown path
- source Markdown report path and bundle-relative path mappings
- bundle created timestamp
- context for cancellation
@@ -55,24 +54,24 @@ The adapter is built from `notify.distributor` config:
- `token_env`
- `timeout`
The app layer renders pipeline ID, bundle ID, idempotency key, and bundle path
The app layer renders pipeline ID, bundle ID, idempotency key, and bundle paths
from:
- `pipeline_id_template`
- `bundle_id_template`
- `idempotency_key_template`
- `report_path_template`
- `report_path_templates`
The token value is read from the environment variable named by `token_env`
after config loading and `secrets.directory` processing.
## Upload Behavior
The adapter calls distributor `UploadFiles` with exactly one file:
The adapter calls distributor `UploadFiles` with one or more file mappings:
- pipeline ID: the rendered distributor workflow selector
- source path: the managed Markdown report path selected by app orchestration
- bundle path: the rendered bundle-relative report path
- bundle paths: rendered bundle-relative report paths
- created: the report generation timestamp
The adapter creates a distributor upload client with the configured endpoint,
@@ -91,11 +90,11 @@ with the status report preserved.
## Failure Behavior
The adapter validates required endpoint, token env name, token value, pipeline
ID, bundle ID, idempotency key, source path, bundle path, and upload client
inputs before uploading.
ID, bundle ID, idempotency key, upload files, source paths, bundle paths, and
upload client inputs before uploading.
Upload failures include endpoint, pipeline ID, bundle ID, idempotency key,
source path, and bundle path context. Token values are redacted from adapter
source paths, and bundle paths context. Token values are redacted from adapter
errors.
Distributor idempotency conflicts are exposed as a weatherreporter-owned

View File

@@ -94,7 +94,8 @@ prepared by creating their parent directory; Scriptorium writes the report body
to the prepared path. Extra Markdown copies are handled by app orchestration.
Distributor notification debug artifacts are written atomically when
notification is attempted and include rendered distributor pipeline ID, bundle
ID, idempotency key, upload status, latest run status, and redacted errors.
ID, idempotency key, bundle paths, upload status, latest run status, and
redacted errors.
Inspection helpers read existing metadata, briefing, and data package files.
Missing metadata directories return no inspection records or no prior snapshot

View File

@@ -144,10 +144,11 @@ attempted report. Notification fields are `notificationStatus`,
## Distributor Notification
Distributor notification is configured with `notify.distributor` and is disabled
by default. When enabled, weatherreporter uploads exactly one file per
successfully generated report: the managed Markdown report path recorded in the
report result and metadata. Extra copies written by `--out` or `--out-dir` are
Distributor notification is configured with `notify.distributor` and is
disabled by default. When enabled, weatherreporter uploads the managed Markdown
report path recorded in the report result and metadata. That single source file
can be mapped to multiple configured bundle paths, such as a dated archival path
and a `latest.md` path. Extra copies written by `--out` or `--out-dir` are
operator conveniences only.
The rendered pipeline ID selects the configured distributor `http_upload`
@@ -159,9 +160,9 @@ weatherreporter.{location_id}.{report_id}
```
The default idempotency key appends RunID to the rendered bundle ID so each
report generation has a distinct retry identity. The default bundle path for the
Markdown file is the report definition's batch output name, such as `daily.md`,
`tomorrow.md`, `three-day.md`, or `weekend.md`.
report generation has a distinct retry identity. Default bundle paths use the
valid-period start date, artifact group, and RunID, and also publish
`latest.md` under the same dated artifact-group directory.
Notification happens after final metadata save. Weather API, briefing,
data-package, render preflight, Scriptorium run, and metadata-save failures do
@@ -171,7 +172,7 @@ JSON summary, and the batch returns nonzero.
Each notification attempt writes a debug artifact under `notifications/`. The
artifact records the rendered pipeline ID, bundle ID, idempotency key, managed
source path, bundle-relative path, bundle created timestamp, accepted upload
source path, bundle-relative paths, bundle created timestamp, accepted upload
response, and the latest distributor run status response when available.
Weatherreporter polls status until distributor reports `succeeded` or `failed`,
or until the configured notification timeout expires. The run status includes
@@ -182,7 +183,9 @@ actions such as `replace_older`, `skip_same`, `skip_destination_newer`, or
Weatherreporter is responsible for selecting the managed Markdown report,
constructing a source bundle, and submitting it to the configured distributor
HTTP endpoint. Distributor remains responsible for destination routing,
publication, and any downstream Markdown-to-HTML transformation.
publication, and any downstream Markdown-to-HTML transformation. Distributor
leaves destination files alone when they are not tracked by a newly uploaded
bundle, so previously uploaded dated report paths can remain available.
## Inspection

View File

@@ -265,13 +265,13 @@ Symptom: notification fails with distributor upload rejection, HTTP status, or
bundle validation context.
Likely cause: the distributor endpoint rejected the token, pipeline ID, bundle
ID, idempotency key, source file, or bundle path.
ID, idempotency key, source file, or one of the rendered bundle paths.
Diagnostic: inspect stdout JSON or stderr status lines for
`notificationError`. Confirm `notify.distributor.endpoint`,
`notify.distributor.pipeline_id_template`,
`notify.distributor.report_path_template`, and token configuration. Token values
are redacted from weatherreporter errors.
`notify.distributor.report_path_templates`, and token configuration. Token
values are redacted from weatherreporter errors.
If the upload was accepted but destination output did not change, inspect the
notification artifact's `runStatus.report`. Distributor actions such as