8 Commits

28 changed files with 2377 additions and 659 deletions

View File

@@ -3,7 +3,8 @@
`weatherreporter` is a Go application for preparing human-facing weather
reports from normalized forecast data. It builds structured briefing packages,
runs them through `scriptorium`, and keeps inspectable artifacts under a local
workspace.
workspace. It can also upload successfully generated managed Markdown reports
to a configured `distributor` HTTP upload endpoint.
## Quickstart

View File

@@ -11,7 +11,9 @@ weatherreporter generate daily --date 2026-05-29 --out ./daily.md
This loads configuration, fetches weather data, writes managed workspace
artifacts, runs `scriptorium render` as a preflight check, runs
`scriptorium run`, and writes an extra Markdown copy to `./daily.md`.
`scriptorium run`, and writes an extra Markdown copy to `./daily.md`. If
distributor notification is enabled in configuration, the command also uploads
the managed Markdown report after final metadata is saved.
## Commands
@@ -33,14 +35,20 @@ weatherreporter inspect sources [--config PATH] RUN_ID
```
`generate` commands write briefing, data package, preflight, report, and
metadata artifacts under the configured workspace. `generate storm` requires
explicit event-window bounds with `--start` and `--end`.
metadata artifacts under the configured workspace. `--out` writes an extra
Markdown copy for the operator; distributor notification uses the managed
report path, not the extra copy. `generate storm` requires explicit
event-window bounds with `--start` and `--end`.
`run morning` generates Daily Today and the 3-Day Outlook, plus Weekend Outlook
except on Sunday. `run evening` generates the Tomorrow Planning Brief. Batch
runs continue independent reports after a failure, print a JSON summary to
stdout, write compact status lines to stderr, and return nonzero when any report
failed.
failed. `--out-dir` writes extra Markdown copies for the operator; distributor
notification uses each managed report path, not the extra copies. When
notification is enabled, batch summaries and status lines include notification
status, accepted distributor run ID, or notification error fields for each
attempted report.
`inspect` commands read existing workspace artifacts and emit JSON to stdout.
They do not fetch weather data or invoke `scriptorium`.
@@ -61,6 +69,9 @@ They do not fetch weather data or invoke `scriptorium`.
Storm times accept `YYYY-MM-DDTHH:MM` in the configured timezone or RFC3339
timestamps with explicit offsets.
Distributor notification is configured only through `notify.distributor`; there
are no distributor-specific CLI flags.
## Common Workflows
```sh

View File

@@ -18,7 +18,7 @@ Precedence is:
The implemented configuration overrides are `--units` and `--tz`. Output flags
control report copies for the current command but do not change configuration
files. Environment-variable configuration is not implemented.
files. Environment variables do not override configuration fields.
## Minimal Config
@@ -64,6 +64,53 @@ multiple configured forecast locations.
The prompt-facing location object also includes `timezone`, derived from the
effective `weather_api.timezone` after CLI overrides such as `--tz`.
### `secrets`
- `directory`: optional directory of file-backed environment secrets. Default:
empty, which disables secret loading.
When configured, each regular file directly under `secrets.directory` is loaded
after config file parsing and CLI overrides. The file basename must be a valid
environment variable name matching `[A-Za-z_][A-Za-z0-9_]*`; the file contents
become the environment variable value and overwrite any existing value. One
trailing LF or CRLF is stripped. Subdirectories, symlinks, invalid filenames,
missing directories, and unreadable files fail config loading.
### `notify`
`notify.distributor` controls distributor notification after successful report
generation. It is disabled by default and does not add CLI flags. When enabled,
weatherreporter uploads one distributor bundle per generated report after
`scriptorium run` succeeds and final metadata is saved.
- `enabled`: whether distributor notification config is active. Default:
`false`.
- `endpoint`: absolute distributor endpoint URL. Required when enabled.
Default: `https://distributor.example.com`.
- `token_env`: environment variable name that will contain the distributor
upload token. Required when enabled. Default: `DISTRIBUTOR_UPLOAD_TOKEN`.
- `timeout`: distributor operation timeout. Must be greater than zero when
enabled. Default: `30s`.
- `failure_policy`: must be `error` when enabled. Default: `error`.
- `bundle_id_template`: template for distributor bundle IDs. Default:
`weatherreporter.{location_id}.{report_id}.{run_id}`.
- `idempotency_key_template`: template for distributor idempotency keys.
Default: `{bundle_id}`.
- `report_path_template`: template for the Markdown report path inside the
distributor bundle. Default: `{batch_output_name}`.
Supported template variables are `location_id`, `report_id`, `run_id`,
`artifact_group`, and `batch_output_name`. `idempotency_key_template` may also
use `bundle_id`.
Rendered report paths must be relative paths with `/` separators. They must not
contain backslashes, empty path segments, `.`, `..`, `manifest.json`, or
`.distributor.json`.
The upload token is read from the environment variable named by `token_env`
after config loading and `secrets.directory` processing. Config files should
name the variable only; they should not contain the token value.
### `missing_source`
- `default`: missing-source behavior for optional sources. One of `error`, `warn`, or `none`. Default: `warn`.
@@ -115,8 +162,13 @@ snapshot exists and a threshold is crossed.
## Secrets
Configuration files should not contain secrets. The current Weather API and
Scriptorium integration settings do not require secret fields.
Configuration files should not contain raw secrets. Use `secrets.directory` to
load secret values from files into environment variables for integrations that
read credentials from the environment. Secret file names become environment
variable names, and secret file contents become values. For distributor
notification, this allows a file such as
`<secrets.directory>/DISTRIBUTOR_UPLOAD_TOKEN` to supply the token referenced by
`notify.distributor.token_env`.
## Maintained Examples

View File

@@ -7,8 +7,9 @@ This document describes the implemented workflow coordinator in `internal/app`.
`internal/app` coordinates the top-level use cases after CLI parsing and config
loading are complete. It resolves report definitions, fetches weather data,
builds briefing and prompt-input artifacts, invokes Scriptorium through the
adapter boundary, persists managed state, runs batches, and reads existing
artifacts for inspection.
adapter boundary, optionally notifies distributor through an app-owned notifier
boundary, persists managed state, runs batches, and reads existing artifacts for
inspection.
## Inputs And Outputs
@@ -22,13 +23,15 @@ Inputs:
- resolved report definitions from `internal/report`
- forecast bundles from `internal/adapters/weatherapi`
- prior snapshots loaded from `internal/state`
- optional renderer and state-store fakes for tests
- optional renderer, notifier, and state-store fakes for tests
Outputs:
- generated report results with briefing, data package, preflight, report,
metadata, prior snapshot, Recent Changes, and Scriptorium result details
- batch summaries with per-report status, artifact paths, and error text
metadata, prior snapshot, Recent Changes, Scriptorium result details, and
notification result when attempted
- batch summaries with per-report status, artifact paths, error text, and
notification outcome when attempted
- saved Weather API bundle JSON for fetch workflows
- inspection JSON values for reports, metadata, briefings, data packages, prior
snapshots, and source provenance
@@ -42,8 +45,9 @@ Scriptorium argv.
Report selection and report identity policy come from `internal/report`.
Weather API transport stays in `internal/adapters/weatherapi`. Scriptorium
subprocess behavior stays in `internal/adapters/scriptorium`. Filesystem layout
and persisted metadata stay in `internal/state`.
subprocess behavior stays in `internal/adapters/scriptorium`. Distributor
upload behavior stays in `internal/adapters/distributor`. Filesystem layout and
persisted metadata stay in `internal/state`.
## Config Fields Used
@@ -52,6 +56,7 @@ and persisted metadata stay in `internal/state`.
- `workspace.*` for filesystem state
- `dayparts` for daily and outlook summarization
- `recent_change.*` for structured Recent Changes thresholds
- `notify.distributor.*` for optional notification after report generation
Output copy flags are command request fields. They are not configuration
defaults.
@@ -74,11 +79,15 @@ Single-report generation follows this order:
12. Run Scriptorium report generation to the managed report path.
13. Copy the managed report to the requested `--out` path when provided.
14. Save metadata with the managed report path.
15. If distributor notification is enabled, notify using the managed report
path as the source file.
If render preflight returns both a result and an error, preflight JSON and
metadata are persisted before the error is returned. If Scriptorium report
generation returns an error after writing output, the managed report and
metadata remain inspectable.
metadata remain inspectable. Notification is not attempted after Weather API,
briefing, prompt input, render, Scriptorium run, or metadata-save failures.
`--out` copies are never used as notification source files.
## Batch Workflow
@@ -87,6 +96,10 @@ on Sunday. `run evening` resolves Daily Tomorrow. Batch output copy names come
from report definitions. Batch generation continues independent reports after a
failure, records each result, writes compact status lines to stderr, emits a
JSON summary to stdout, and returns an aggregate error when any report failed.
When notification is enabled, each successfully generated report is notified
independently. Notification failure marks that report failed, records
notification fields in the batch result, and does not stop later reports.
`--out-dir` copies are never used as notification source files.
## Inspection Workflow
@@ -101,6 +114,8 @@ inspection view.
- Weather API and briefing errors stop that report before Scriptorium runs.
- Prompt input validation fails before render preflight.
- Render and run errors preserve Scriptorium stderr and exit-code context.
- Notification errors are wrapped with report ID, RunID, and managed report path
context and are recorded separately in batch results.
- Metadata and artifact path errors include filesystem context.
- Batch failures are recorded per report and surfaced through an aggregate
batch error.
@@ -121,3 +136,5 @@ Inspect:
- Render preflight precedes Scriptorium report generation.
- Recent Changes are computed from structured briefing snapshots.
- Metadata links artifacts produced for a run.
- Distributor notification uses the managed Markdown report path, not extra
output copies.

View File

@@ -0,0 +1,106 @@
# Distributor Adapter Internals
This document describes the distributor upload adapter in
`internal/adapters/distributor`.
## Purpose
The adapter submits generated weatherreporter Markdown reports to a configured
distributor HTTP upload endpoint. It isolates distributor package types,
token-env lookup, upload client construction, source-bundle file mapping,
timeout handling, and upload error wrapping from app orchestration.
## Inputs And Outputs
Inputs:
- distributor endpoint URL
- token environment variable name
- upload timeout
- bundle ID
- idempotency key
- source Markdown report path
- bundle-relative Markdown path
- context for cancellation
Outputs:
- accepted distributor run ID
- accepted distributor status
- weatherreporter-owned idempotency conflict error when applicable
## Boundaries
`internal/adapters/distributor` is the only weatherreporter package that imports
`gitea.maximumdirect.net/eric/distributor/pkg/upload` or
`gitea.maximumdirect.net/eric/distributor/pkg/bundle`.
The app layer passes weatherreporter-owned request values to the adapter. The
adapter does not choose report types, render templates, select output copies,
configure destinations, wait for downstream publication, transform Markdown, or
persist notification state.
Full upstream distributor package and HTTP contract details stay under
`docs/integrations/distributor/`.
## Config Fields Used
The adapter is built from `notify.distributor` config:
- `endpoint`
- `token_env`
- `timeout`
The app layer renders bundle ID, idempotency key, and bundle path from:
- `bundle_id_template`
- `idempotency_key_template`
- `report_path_template`
The token value is read from the environment variable named by `token_env`
after config loading and `secrets.directory` processing.
## Upload Behavior
The adapter calls distributor `UploadFiles` with exactly one file:
- source path: the managed Markdown report path selected by app orchestration
- bundle path: the rendered bundle-relative report path
The adapter creates a distributor upload client with the configured endpoint,
bearer token, and timeout-backed HTTP client. It also wraps the upload context
with the configured timeout when the timeout is greater than zero.
## Failure Behavior
The adapter validates required endpoint, token env name, token value, bundle ID,
idempotency key, source path, bundle path, and upload client inputs before
uploading.
Upload failures include endpoint, bundle ID, idempotency key, source path, and
bundle path context. Token values are redacted from adapter errors.
Distributor idempotency conflicts are exposed as a weatherreporter-owned
`IdempotencyConflictError`, so callers do not depend on upstream distributor
types.
## Tests
Inspect:
- `internal/adapters/distributor/client_test.go`
- `internal/app/app_test.go`
- `internal/cli/root_test.go`
Adapter tests use a fake upload client factory and do not require a live
distributor service.
## Invariants
- Distributor package types do not leak outside the adapter.
- Only the managed Markdown report is uploaded.
- The adapter never scans the workspace.
- Token values are not included in errors, CLI output, metadata, docs, or
examples.
- Destination routing and Markdown-to-HTML transformation belong to
distributor, not weatherreporter.

View File

@@ -19,7 +19,10 @@ weatherreporter generate storm --start 2026-05-29T18:00 --end 2026-05-30T06:00
Each command resolves a report period, fetches a Weather API bundle, builds a
briefing, builds a prompt input data package, runs `scriptorium render`, runs
`scriptorium run`, and writes managed artifacts under the configured workspace.
`--out PATH` writes an extra Markdown copy for the current generated report.
When distributor notification is enabled, weatherreporter uploads the managed
Markdown report after `scriptorium run` succeeds and final metadata is saved.
`--out PATH` writes an extra Markdown copy for the current generated report; it
is not used as the distributor upload source.
Implemented batch commands:
@@ -32,9 +35,12 @@ weatherreporter run evening
except on Sunday. `run evening` generates the Tomorrow Planning Brief. Batch
commands print a JSON summary to stdout, write compact per-report status lines
to stderr, continue independent reports after one report fails, and return
nonzero when any report failed. `--out-dir PATH` writes extra Markdown copies
using report default filenames such as `daily.md`, `three-day.md`,
`weekend.md`, and `tomorrow.md`.
nonzero when any report failed. When notification is configured, the summary and
status lines include notification status, accepted distributor run ID, or
notification error fields for each attempted report. `--out-dir PATH` writes
extra Markdown copies using report default filenames such as `daily.md`,
`three-day.md`, `weekend.md`, and `tomorrow.md`; these copies are not used as
distributor upload sources.
## Filesystem Layout
@@ -117,8 +123,40 @@ Each generated report writes metadata that links:
- preflight output path
- managed Markdown report path
Batch summaries include report status, error text when applicable, valid
period, and known artifact paths for each attempted report.
Batch summaries include report status, error text when applicable, notification
outcome when attempted, valid period, and known artifact paths for each
attempted report. Notification fields are `notificationStatus`,
`notificationRunId`, and `notificationError`.
## Distributor Notification
Distributor notification is configured with `notify.distributor` and is disabled
by default. When enabled, weatherreporter uploads exactly one file per
successfully generated report: the managed Markdown report path recorded in the
report result and metadata. Extra copies written by `--out` or `--out-dir` are
operator conveniences only.
The default bundle ID is derived from producer name, location ID, report ID, and
RunID:
```text
weatherreporter.{location_id}.{report_id}.{run_id}
```
The default idempotency key is the rendered bundle ID. The default bundle path
for the Markdown file is the report definition's batch output name, such as
`daily.md`, `tomorrow.md`, `three-day.md`, or `weekend.md`.
Notification happens after final metadata save. Weather API, briefing,
data-package, render preflight, Scriptorium run, and metadata-save failures do
not trigger notification. A notification failure fails that report. In a batch,
other reports continue, the failed report includes notification fields in the
JSON summary, and the batch returns nonzero.
Weatherreporter is responsible for selecting the managed Markdown report,
constructing a source bundle, and submitting it to the configured distributor
HTTP endpoint. Distributor remains responsible for destination routing,
publication, and any downstream Markdown-to-HTML transformation.
## Inspection
@@ -164,6 +202,8 @@ A failed generation run may still leave useful artifacts:
preflight JSON and metadata are written for inspection.
- If `scriptorium run` exits nonzero after writing a report, the managed report
and metadata remain available.
- If distributor notification fails, report artifacts and final metadata remain
available, but the report or batch command returns nonzero.
- For batch commands, inspect the stdout JSON summary first, then inspect the
artifact paths for each failed report.

View File

@@ -13,6 +13,12 @@ Generated reports must be associated with explicit metadata, including report ty
`scriptorium` is an external adapter, not domain logic. Subprocess execution must be isolated under `internal/adapters/scriptorium`, use context-aware execution, avoid shell interpolation, capture actionable stderr, and keep scriptorium-specific flags from leaking into domain packages.
`distributor` is also an external adapter. Upload behavior must be isolated
under `internal/adapters/distributor`, dependency types from the distributor
module must not leak outside that adapter, and the selected upload source must
be the managed Markdown report rather than optional output copies or broad
workspace scans.
## Project Shape
Default to a small, explicit, dependency-light Go application. Keep the design modular enough to test and change safely, but do not add abstraction unless it protects a real boundary or enables a real extension point.
@@ -54,7 +60,7 @@ Configuration precedence is:
Prefer YAML configuration unless the project has a strong reason to use another format. Config files should be discovered at `/usr/local/etc/<app_name>/config.yml`, with a CLI override via `--config`.
Configuration files should not contain raw secrets unless the application is explicitly designed for that. Prefer environment variables or secret files for secrets.
Configuration files should not contain raw secrets unless the application is explicitly designed for that. Prefer environment variables or secret files for secrets. File-backed secrets are loaded through `secrets.directory`; secret values must not be logged, persisted, or included in user-facing output.
## Adapters and External Integrations

View File

@@ -13,6 +13,7 @@ Developers and LLM coding agents should use it with
- `internal/config`: configuration structs, defaults, loading, overrides, and
validation.
- `internal/fileutil`: shared atomic filesystem write and copy helpers.
- `internal/adapters/distributor`: Distributor upload adapter.
- `internal/adapters/weatherapi`: Weather API HTTP adapter.
- `internal/adapters/scriptorium`: Scriptorium subprocess adapter.
- `internal/forecast`: normalized bundle types and deterministic forecast
@@ -45,7 +46,7 @@ Useful focused checks:
```bash
go test ./internal/cli ./internal/config
go test ./internal/app ./internal/state
go test ./internal/adapters/weatherapi ./internal/adapters/scriptorium
go test ./internal/adapters/distributor ./internal/adapters/weatherapi ./internal/adapters/scriptorium
go test ./internal/forecast ./internal/report ./internal/briefing ./internal/changes ./internal/promptinput
```
@@ -64,6 +65,8 @@ Run `gofmt -w` on changed Go files before committing.
- Use atomic writes for durable JSON artifacts where practical.
- Keep report selection and prompt IDs centralized in `internal/report`.
- Keep Scriptorium argv construction inside `internal/adapters/scriptorium`.
- Keep distributor package types and upload-client construction inside
`internal/adapters/distributor`.
- Keep Weather API transport and envelope handling inside
`internal/adapters/weatherapi`.
@@ -72,8 +75,10 @@ Run `gofmt -w` on changed Go files before committing.
Prefer the Go standard library. Add dependencies only when they materially
improve correctness, interoperability, security, or maintainability.
Current external dependency:
Current external dependencies:
- `gitea.maximumdirect.net/eric/distributor` for distributor source bundle
construction and HTTP upload client behavior.
- `gopkg.in/yaml.v3` for YAML configuration parsing.
When adding a dependency:
@@ -145,12 +150,14 @@ When an external contract changes, update the matching file under
## Tests
Core tests must not require live Weather API or Scriptorium services.
Core tests must not require live Weather API, Scriptorium, or distributor
services.
Preferred test patterns:
- fake command runners for subprocess behavior;
- `httptest.Server` for Weather API behavior;
- fake distributor upload clients for notification behavior;
- filesystem temp directories for state behavior;
- deterministic clocks for report periods and RunIDs;
- table tests for config validation, CLI parsing, period resolution, and
@@ -187,7 +194,8 @@ Update:
behavior;
- `docs/troubleshooting.md` for recurring operator-facing failure modes;
- `docs/internal/` for component contracts and invariants;
- `docs/integrations/` for external Weather API or Scriptorium contract changes;
- `docs/integrations/` for external Weather API, Scriptorium, or distributor
contract changes;
- `docs/roadmap/` only for unimplemented or deferred work.
Non-roadmap docs must describe implemented behavior only.

View File

@@ -1,336 +1,34 @@
# Distributor Integration Roadmap
# Distributor Roadmap
This roadmap describes planned work for adding `distributor` notification
support to `weatherreporter`. The feature is not implemented yet, so this plan
lives under `docs/roadmap/`; non-roadmap documentation should not describe
`notify.distributor` as available until implementation is complete.
Current distributor notification behavior is documented outside the roadmap:
## Purpose
- [Configuration reference](../config.md)
- [Operations guide](../operations.md)
- [Troubleshooting](../troubleshooting.md)
- [Distributor adapter internals](../internal/distributor-adapter.md)
`distributor` collects outputs from producer applications and publishes them
through configured downstream pipelines. For `weatherreporter`, the planned
integration should upload generated Markdown weather reports after successful
report generation, without requiring `distributor` to scan or understand the
managed `workspace/` layout.
This file tracks future distributor-related work only.
## Current Repository Facts
## Deferred Enhancements
- `weatherreporter` does not currently implement a notify hook.
- Generated Markdown reports are written to the managed report path returned as
`ReportResult.ReportPath`.
- Optional `--out` and `--out-dir` copies are user-requested extra copies and
are not canonical integration inputs.
- The managed workspace also contains briefing snapshots, metadata, prompt data
packages, and preflight artifacts; these should not be treated as a
distributor source tree.
- The provided distributor docs recommend `pkg/upload.UploadFiles` for
producers that already have generated files, so the initial integration can
upload selected report files directly.
- Add a supported warning-only notification policy.
- Include selected non-report artifacts in uploaded bundles.
- Poll distributor run status after upload acceptance.
- Persist upload retry state across process restarts.
- Add explicit CLI controls for distributor behavior.
## Decisions Locked
## Non-Goals Without A Separate Design
- Add generic configurable secrets-directory support before the distributor
adapter work. This is project infrastructure, not a distributor-only token
helper.
- When a secrets directory is configured, each regular file directly under that
directory maps filename to environment variable name and file contents to
environment variable value.
- Secret-file values overwrite pre-existing environment variables.
- Secrets loading must never log, print, persist, or include secret values in
errors, metadata, batch output, examples, or docs.
- Add a narrow `internal/adapters/distributor` adapter for the external
integration.
- Keep `gitea.maximumdirect.net/eric/distributor/pkg/upload` and
`gitea.maximumdirect.net/eric/distributor/pkg/bundle` types inside that
adapter.
- Upload one distributor bundle per successfully generated report.
- Include only the generated Markdown report in the initial bundle.
- Use the report definition's batch output filename as the default
bundle-relative Markdown path, such as `daily.md`, `tomorrow.md`,
`three-day.md`, `weekend.md`, or `storm.md`.
- Derive the default bundle ID and idempotency key from producer name, location
ID, report ID, and RunID.
- Treat distributor upload failure as a report failure when notification is
enabled; single-report and batch commands should exit nonzero for enabled
upload failures.
- Keep destination routing, Markdown-to-HTML transformation, public URLs, and
nginx directory layout in distributor configuration, not in
`weatherreporter`.
- Do not make distributor scan the weatherreporter workspace.
- Do not move destination routing into weatherreporter.
- Do not move Markdown-to-HTML transformation into weatherreporter.
- Do not store raw bearer tokens in configuration files.
## Planned Configuration
## Required Constraints For Future Work
Add future generic secrets configuration:
```yaml
secrets:
directory: ""
```
Add future distributor configuration under `notify.distributor`:
```yaml
notify:
distributor:
enabled: false
endpoint: https://distributor.example.com
token_env: DISTRIBUTOR_UPLOAD_TOKEN
timeout: 30s
failure_policy: error
bundle_id_template: "weatherreporter.{location_id}.{report_id}.{run_id}"
idempotency_key_template: "{bundle_id}"
report_path_template: "{batch_output_name}"
```
Planned behavior:
- `secrets.directory` defaults to an empty string, which disables secrets
loading.
- When `secrets.directory` is configured, load only regular files directly
under that directory.
- Each secret filename must be a valid environment variable name matching
`[A-Za-z_][A-Za-z0-9_]*`.
- Reject missing configured directory, invalid filenames, subdirectories,
symlinks, unreadable files, and empty filenames.
- Strip one trailing `\n` or `\r\n` from secret file contents for compatibility
with common mounted-secret formats. Preserve all other bytes.
- `enabled` defaults to `false`.
- `endpoint` is required only when distributor notification is enabled.
- `token_env` is required only when enabled. Raw bearer tokens should not be
stored in config files. The token value is read from normal environment state
after secrets-directory loading has been applied.
- `timeout` controls the upload call timeout and should default to `30s`.
- `failure_policy` should initially support `error`. Reserve `warn` as a
future option only if it is implemented and tested.
- Template variables should be explicit and validated:
`location_id`, `report_id`, `run_id`, `artifact_group`, and
`batch_output_name`.
- `idempotency_key_template` may reference `{bundle_id}` after the bundle ID is
rendered.
- `report_path_template` should be user-visible from the first implementation
because bundle-relative paths can affect downstream distributor routing.
- Rendered template values must produce valid distributor bundle paths: relative
slash-separated paths with no empty path, absolute path, backslash, `.`, `..`,
empty segment, `manifest.json`, or `.distributor.json`.
## Recommended Implementation Stages
### Stage 1: Secrets Directory Support
Goal: add generic file-backed environment secret loading before distributor
notification depends on environment tokens.
Implementation guidance:
- Add a generic `secrets.directory` config field under `internal/config`.
- Default the directory to an empty string so secrets loading is disabled unless
explicitly configured.
- Load secrets after config file parsing and before validation that needs
environment-backed values.
- Map each regular file directly under the configured directory to an
environment variable: filename becomes the variable name and contents become
the value.
- Use secret-file values to overwrite pre-existing environment variables.
- Strip one trailing `\n` or `\r\n`; preserve all other bytes.
- Reject missing directories, invalid environment-variable filenames,
subdirectories, symlinks, unreadable files, and empty filenames.
- Keep secret values out of errors, logs, CLI output, metadata, and examples.
Acceptance criteria:
- Empty `secrets.directory` performs no environment changes.
- Configured secrets directory values are available through normal environment
lookup after config loading.
- Secret files overwrite existing environment variables with the same name.
- Invalid directory entries fail with path/name context but without secret
values.
- Existing config loading behavior remains unchanged when secrets are disabled.
Suggested tests:
- Config defaults leave secrets loading disabled.
- A configured missing secrets directory fails.
- Regular files set environment variables and overwrite existing values.
- Invalid filenames, subdirectories, symlinks, and unreadable files fail.
- One trailing newline or CRLF is stripped; other content is preserved.
- No secret values appear in returned errors.
### Stage 2: Config And Validation
Goal: add disabled-by-default distributor notification configuration.
Implementation guidance:
- Add notify/distributor config structs under `internal/config`.
- Add defaults for disabled notification, timeout, failure policy, bundle ID
template, idempotency key template, and report path template.
- Validate enabled config: endpoint must be an absolute URL, `token_env` must be
non-empty, timeout must be positive, failure policy must be supported, and
templates must use only known variables.
- Keep `token_env` as the configured token selector. Do not add raw token config.
- Update maintained examples and config documentation only after code support is
implemented.
Acceptance criteria:
- Disabled distributor notification requires no endpoint or token env.
- Enabled distributor notification rejects invalid endpoint, missing token env,
nonpositive timeout, unsupported failure policy, unknown template variables,
and invalid rendered report bundle paths.
- Existing config loading behavior and precedence remain unchanged.
Suggested tests:
- Config defaults include disabled distributor notification.
- Example configs load successfully.
- Enabled config validation covers required fields and invalid template cases.
- A distributor token supplied through `secrets.directory` is visible through
the configured `token_env` name after config loading.
### Stage 3: Distributor Adapter
Goal: isolate distributor package usage behind an internal adapter.
Implementation guidance:
- Add `internal/adapters/distributor` with package-owned request and result
types.
- Read the bearer token from the configured environment variable after
secrets-directory loading has populated environment state.
- Use distributor `UploadFiles` with one file:
the managed Markdown report as the source path and the rendered
`report_path_template` as the bundle path.
- Return upload result information that app orchestration can record or expose
without leaking distributor dependency types.
- Wrap upload errors with endpoint, bundle ID, and report path context while
avoiding token exposure.
Acceptance criteria:
- No distributor dependency types leak outside `internal/adapters/distributor`.
- Missing source report path, missing token, invalid bundle path, and upload
failure return actionable errors.
- Idempotency conflict errors are preserved or wrapped clearly enough for
troubleshooting.
Suggested tests:
- Adapter request construction uses the configured endpoint, token, bundle ID,
idempotency key, source report path, and bundle-relative path.
- Token values do not appear in returned errors.
- Upload conflict and generic upload failure produce useful wrapped errors.
- Tokens supplied through the secrets directory are accepted via `token_env`.
### Stage 4: App Notify Hook
Goal: upload successful generated reports through the configured notifier.
Implementation guidance:
- Add an app-owned notifier interface so app tests can use fakes.
- Construct distributor notification requests from the successful
`ReportResult`, resolved report definition, configured location, and
notification config.
- Invoke notification only after Scriptorium report generation succeeds and
final metadata has been saved.
- Do not notify after Weather API, briefing, prompt input, render preflight, or
Scriptorium run failure.
- In batch mode, notify each successful report independently. If notification
fails and distributor notification is enabled, mark that report failed and
make the aggregate batch result nonzero.
Acceptance criteria:
- Disabled notification is a no-op.
- Single-report generation returns an error when enabled upload fails.
- Batch generation continues independent reports, records upload failures per
report, and exits nonzero when any enabled notification fails.
- Existing report artifact paths, optional output copies, and Scriptorium
behavior remain unchanged.
Suggested tests:
- No notifier call occurs before successful report generation.
- Successful notifier call receives the managed report path, not `--out` or
`--out-dir` copy paths.
- Enabled notifier failure affects single-report and batch command outcomes.
- Batch still continues later reports after one notification failure.
### Stage 5: CLI And Output Behavior
Goal: preserve CLI syntax while surfacing notification outcome where useful.
Implementation guidance:
- Do not add distributor-specific CLI flags in the first implementation; use
configuration only.
- Preserve existing command names, flags, help text shape, workspace paths, and
generated Markdown outputs.
- Extend batch JSON report items only if the implementation records useful
notification status, accepted distributor run ID, or notification error.
- Keep stderr status lines concise and avoid exposing secrets.
Acceptance criteria:
- `go run ./cmd/weatherreporter --help` remains accurate.
- Existing generate and run command syntax remains stable.
- Batch summaries clearly indicate notification-caused report failures if
notification status is added.
Suggested tests:
- Existing CLI parser tests continue to pass.
- Batch JSON includes notification fields only when implemented and documented.
- Secret-like token values never appear in CLI output.
### Stage 6: Documentation
Goal: document implemented distributor behavior after the feature exists.
Implementation guidance:
- Update `docs/config.md` with `secrets.directory`,
`notify.distributor` fields, and defaults.
- Update `docs/operations.md` with notification timing, failure behavior, and
the fact that managed report paths are uploaded.
- Update `docs/troubleshooting.md` with invalid secrets directory, missing
token, upload conflict, upload rejection, and distributor unavailable cases.
- Update `docs/internal/app-orchestration.md` to include notify ordering.
- Add `docs/internal/distributor-adapter.md` for the adapter contract.
- Keep distributor API details in `docs/integrations/distributor/`; link there
instead of duplicating the full upstream contract.
Acceptance criteria:
- Non-roadmap docs describe only implemented notification behavior.
- Config examples include no raw tokens.
- Secret handling documentation describes mechanisms, not secret values.
- Documentation clearly distinguishes distributor destination routing from
weatherreporter upload responsibilities.
### Stage 7: Final Validation
Goal: verify the feature without requiring live distributor service access.
Validation commands:
```bash
go test ./...
go run ./cmd/weatherreporter --help
git diff --check
```
Additional validation:
- Run focused config, adapter, app, and CLI tests.
- Confirm maintained example configs load.
- Search for accidental raw-token config examples.
- Confirm `go.mod` includes distributor only after implementation requires it.
- Confirm no test failures leak secret values in error output.
## Open Questions
No questions block this roadmap. The initial implementation should use the
locked defaults above and expose configurable templates for bundle identity and
bundle-relative report path. Secrets-directory support is generic project
infrastructure and should remain useful for future integrations beyond
distributor.
- Distributor package types stay inside `internal/adapters/distributor`.
- Weatherreporter submits explicit source bundles built from generated files.
- Optional `--out` and `--out-dir` copies remain operator conveniences, not
canonical upload sources.
- Secret values stay out of errors, logs, CLI output, metadata, examples, and
documentation.

View File

@@ -49,6 +49,26 @@ These ideas are not current behavior:
Each item needs its own design note before implementation. Non-roadmap docs
must not describe these as available behavior.
## Deferred: Distributor Notification Enhancements
Distributor notification currently uploads one managed Markdown report per
successful generated report through the configured HTTP upload endpoint.
These enhancements are not current behavior:
- `failure_policy: warn`;
- uploading metadata, briefing snapshots, data packages, or preflight artifacts;
- polling distributor status after upload acceptance;
- durable upload retry queues;
- distributor-specific CLI flags;
- making distributor scan the weatherreporter workspace;
- handling destination routing, Markdown-to-HTML transformation, public URLs,
or nginx layout inside weatherreporter.
Any distributor enhancement should preserve the existing adapter boundary:
weatherreporter selects explicit generated files and submits source bundles,
while distributor owns destination routing and publication behavior.
## Deferred: Cleanup Refactors
The initial cleanup pass intentionally left these refactors out because the

View File

@@ -1,297 +1,36 @@
# Distributor Feature Implementation Roadmap
# Distributor Follow-Up Roadmap
This roadmap turns `docs/roadmap/distributor.md` into staged implementation
work for a future coding agent. It is a planning document only. The feature is
not implemented until the stages below are completed and the implemented docs
are updated.
Current distributor notification behavior is documented in:
## Purpose
- [Configuration reference](../config.md)
- [Operations guide](../operations.md)
- [Troubleshooting](../troubleshooting.md)
- [App orchestration internals](../internal/app-orchestration.md)
- [Distributor adapter internals](../internal/distributor-adapter.md)
- [Distributor integration contracts](../integrations/distributor/api.md)
Implement generic file-backed environment secrets and distributor notification
support while preserving current CLI syntax, managed artifact paths, RunIDs,
and report output behavior. Distributor package types must stay behind a narrow
adapter boundary, and distributor should receive explicitly selected generated
Markdown files rather than scanning `workspace/`.
This roadmap tracks only future work that is not implemented.
## Locked Decisions
## Deferred Enhancements
- Implement generic `secrets.directory` support before distributor notification.
- Secret files overwrite pre-existing environment variables.
- Never expose secret values in errors, logs, CLI output, metadata, docs, or
examples.
- Add `notify.distributor` config; do not add raw token config.
- Keep public CLI commands and flags unchanged.
- Upload one distributor bundle per successfully generated report.
- Initial distributor bundles include only the managed Markdown report at
`ReportResult.ReportPath`.
- The default bundle-relative Markdown path uses the report definition's
`BatchOutputName`.
- Default bundle ID and idempotency key derive from producer, location ID,
report ID, and RunID.
- Enabled distributor upload failure is a report failure and should produce a
nonzero command exit.
- Support a non-failing distributor notification policy such as
`failure_policy: warn`.
- Upload additional generated artifacts, such as metadata, briefing snapshots,
data packages, or preflight output.
- Poll distributor status after upload acceptance and expose downstream
publication failures.
- Add durable retry queues for upload failures.
- Add distributor-specific CLI controls if operators need per-run behavior that
configuration cannot cover cleanly.
## Stage 1: Secrets Directory Support
## Boundaries To Preserve
Goal: implement generic file-backed environment secret loading in
`internal/config`.
Implementation guidance:
- Add `secrets.directory` config with a built-in default of `""`, meaning
disabled.
- Load secrets after config file parsing and CLI overrides, before validation
completes.
- Read only regular files directly under the configured directory.
- Map each file basename to an environment variable name and file contents to
the value.
- Validate names with `[A-Za-z_][A-Za-z0-9_]*`.
- Reject missing configured directory, invalid filenames, subdirectories,
symlinks, unreadable files, and empty filenames.
- Strip one trailing `\n` or `\r\n`; preserve all other bytes.
- Keep error messages actionable but do not include secret values.
Acceptance criteria:
- Empty `secrets.directory` performs no environment changes.
- Valid secret files set environment variables after config loading.
- Secret file values overwrite existing environment variables.
- Invalid entries fail before application workflows run.
- Existing config loading behavior remains unchanged when secrets are disabled.
Tests:
- Defaults leave secrets loading disabled.
- Valid file sets an environment variable.
- File value overwrites an existing environment variable.
- Missing directory, invalid filename, subdirectory, symlink, and unreadable
file cases fail.
- Newline and CRLF trimming works exactly once.
- Secret values are absent from error strings.
## Stage 2: Notify Configuration And Templates
Goal: add disabled-by-default `notify.distributor` config and template
validation.
Planned config:
```yaml
notify:
distributor:
enabled: false
endpoint: https://distributor.example.com
token_env: DISTRIBUTOR_UPLOAD_TOKEN
timeout: 30s
failure_policy: error
bundle_id_template: "weatherreporter.{location_id}.{report_id}.{run_id}"
idempotency_key_template: "{bundle_id}"
report_path_template: "{batch_output_name}"
```
Implementation guidance:
- Add notify/distributor config structs, defaults, YAML loading, and validation
under `internal/config`.
- When disabled, omit endpoint and token requirements.
- When enabled, validate absolute endpoint URL, non-empty `token_env`, positive
timeout, `failure_policy=error`, known template variables, and valid rendered
report bundle paths.
- Supported template variables are `location_id`, `report_id`, `run_id`,
`artifact_group`, and `batch_output_name`; `idempotency_key_template` may
also reference `{bundle_id}`.
- Do not add distributor-specific CLI flags.
Acceptance criteria:
- Defaults include disabled distributor notification.
- Enabled config rejects invalid endpoint, token env, timeout, failure policy,
templates, and rendered bundle paths.
- Rendered bundle paths reject absolute paths, backslashes, `.`, `..`, empty
segments, `manifest.json`, and `.distributor.json`.
- Tokens supplied by `secrets.directory` are visible through `token_env` after
config loading.
Tests:
- Config defaults and examples load.
- Disabled distributor config accepts omitted distributor fields.
- Enabled validation covers each invalid field.
- Template rendering and validation are covered with table tests.
## Stage 3: Distributor Adapter
Goal: isolate distributor upload behavior in `internal/adapters/distributor`.
Implementation guidance:
- Add weatherreporter-owned request and result types.
- Keep `gitea.maximumdirect.net/eric/distributor/pkg/upload` and
`gitea.maximumdirect.net/eric/distributor/pkg/bundle` types inside the
adapter.
- Read the bearer token from `token_env` after config/secrets loading.
- Use `UploadFiles` with exactly one file: source path is the managed Markdown
report path and bundle path is the rendered `report_path_template`.
- Return the accepted distributor run ID when available.
- Wrap endpoint, bundle ID, idempotency key, source path, and bundle path
context into errors without exposing token values.
- Preserve idempotency conflict diagnosis.
Acceptance criteria:
- No distributor dependency types leak outside the adapter.
- Missing token, source path, bundle path, or upload client inputs fail with
actionable errors.
- Upload success returns a weatherreporter-owned result.
- Upload failure errors do not include token values.
Tests:
- Fake upload client receives expected endpoint, token, bundle ID, idempotency
key, source path, and bundle path.
- Missing token/source path/bundle path errors are actionable.
- Token values are not present in errors.
- Upload conflict and generic failure are wrapped clearly.
## Stage 4: App Notification Hook
Goal: add app-level notification orchestration after successful report
generation.
Implementation guidance:
- Add an app-owned notifier interface for tests.
- Default to no-op notification when distributor notification is disabled.
- Notify only after successful `scriptorium run` and final metadata save.
- Use `ReportResult.ReportPath` as the distributor source file.
- Never use `--out` or `--out-dir` copies as distributor source files.
- Do not notify after Weather API, briefing, prompt input, render preflight, or
Scriptorium run failure.
- Single-report generation should return an error when enabled notification
fails.
- Batch generation should continue independent reports; notification failure
marks that report failed and makes the aggregate batch nonzero.
Acceptance criteria:
- Disabled notification leaves current generation behavior unchanged.
- Successful notification is recorded in the app result when useful.
- Enabled notification failure fails a single-report command.
- Batch results distinguish generation failure from notification failure enough
for operators to diagnose the failed report.
- Existing artifact paths, output copies, and Scriptorium behavior remain
unchanged.
Tests:
- No notifier call occurs on pre-generation failures.
- Notifier receives the managed report path.
- Enabled notifier success and failure are covered.
- Batch continues after one notification failure and reports aggregate failure.
## Stage 5: CLI Output And Batch Reporting
Goal: keep public CLI syntax stable while surfacing notification outcome if the
app records it.
Implementation guidance:
- Do not add distributor-specific CLI flags.
- Preserve current command names, help shape, generated paths, `--out`, and
`--out-dir`.
- If batch JSON is extended, use explicit fields such as
`notificationStatus`, `notificationRunId`, and `notificationError`.
- Keep stderr status lines concise and secret-free.
Acceptance criteria:
- `weatherreporter --help` remains accurate.
- Existing CLI parser behavior remains stable.
- Batch JSON includes notification details only when implemented and
documented.
- CLI stdout and stderr do not include token values.
Tests:
- Existing CLI parser tests pass, except intentional JSON additions.
- Batch output covers notification success and failure.
- Secret-like token values are absent from CLI output.
## Stage 6: Documentation
Goal: update implemented documentation after the feature exists.
Implementation guidance:
- Update `docs/config.md` with `secrets.directory`, `notify.distributor`,
defaults, and secret handling.
- Update `docs/operations.md` with notification timing, failure behavior, and
managed report upload source.
- Update `docs/troubleshooting.md` with invalid secrets directory, missing
token, upload conflict, upload rejection, and distributor unavailable cases.
- Update `docs/internal/app-orchestration.md` to include notify ordering.
- Add `docs/internal/distributor-adapter.md` for adapter inputs, outputs,
boundaries, config fields, failure behavior, tests, and invariants.
- Keep full upstream distributor contract details in
`docs/integrations/distributor/`.
Acceptance criteria:
- Non-roadmap docs describe only implemented behavior.
- Config examples load and contain no raw secrets.
- Secret handling docs describe mechanisms, not secret values.
- Docs distinguish weatherreporter upload responsibility from distributor
destination routing and Markdown-to-HTML transformation.
Tests and checks:
- Run config example loading tests.
- Search docs and examples for accidental raw tokens.
- Run `git diff --check`.
## Stage 7: Final Validation
Goal: verify behavior without a live distributor service.
Validation commands:
```bash
go test ./...
go run ./cmd/weatherreporter --help
git diff --check
go test ./internal/config ./internal/app ./internal/cli ./internal/adapters/distributor
```
Manual validation:
- No token values appear in errors, CLI output, docs, examples, metadata, or
batch JSON.
- `go.mod` includes distributor only after adapter implementation requires it.
- Existing report paths, RunIDs, and `--out` / `--out-dir` behavior are
unchanged.
- Distributor upload tests use fakes or local test doubles, not a live service.
## Deferred Work
These are out of scope for the initial implementation:
- `failure_policy: warn`.
- Uploading metadata, briefing, data package, or preflight artifacts.
- Distributor status polling after `202 Accepted`.
- Durable upload retry queues.
- Distributor-specific CLI flags.
- Making distributor scan `workspace/`.
- Destination routing, Markdown-to-HTML transformation, public URLs, or nginx
layout in `weatherreporter`.
## Global Validation Checklist
- Preserve current public CLI syntax.
- Preserve managed workspace artifact paths.
- Keep public CLI syntax stable unless a separate CLI design changes it.
- Keep managed workspace artifact paths and RunIDs stable.
- Keep distributor dependency types behind `internal/adapters/distributor`.
- Keep secrets out of user-facing and persisted output.
- Keep examples valid and secret-free.
- Update implemented docs in the same change as implementation.
- Run `go test ./...`, `go run ./cmd/weatherreporter --help`, and
`git diff --check` before finishing implementation.
- Keep raw token values out of errors, CLI output, metadata, docs, examples,
and persisted artifacts.
- Upload explicitly selected generated files; do not make distributor scan the
weatherreporter workspace.
- Leave destination routing, Markdown-to-HTML transformation, public URLs, and
nginx layout to distributor.

View File

@@ -199,6 +199,101 @@ metadata, sources, briefing, and data package for that RunID.
Relevant docs: [CLI reference](cli.md), [Operations guide](operations.md).
## Invalid Secrets Directory
Symptom: config loading fails with `read secrets directory`, `secret file`, or
environment variable name context.
Likely cause: `secrets.directory` points to a missing directory or contains an
invalid entry. Secret entries must be regular files directly under the
configured directory, and file basenames must match
`[A-Za-z_][A-Za-z0-9_]*`.
Diagnostic: list the configured directory and inspect entry names and file
types. Do not print secret file contents.
Safe fix: create the directory, remove subdirectories or symlinks, fix invalid
filenames, and ensure the weatherreporter process can read each secret file.
Relevant docs: [Configuration reference](config.md).
## Distributor Token Is Missing
Symptom: notification fails with a message that the distributor token
environment variable is not set.
Likely cause: `notify.distributor.enabled` is true, but the environment
variable named by `notify.distributor.token_env` was not populated directly or
through `secrets.directory`.
Diagnostic: check `notify.distributor.token_env`, then verify a matching secret
file exists under `secrets.directory` or that the process environment includes
the variable. Do not print the token value.
Safe fix: create a readable secret file whose basename matches `token_env`, or
set the environment variable through the service manager.
Relevant docs: [Configuration reference](config.md),
[Operations guide](operations.md).
## Distributor Upload Conflict
Symptom: notification fails with idempotency conflict context.
Likely cause: the same idempotency key was reused for different bundle content.
By default the key derives from bundle ID, which includes location ID, report
ID, and RunID.
Diagnostic: inspect the failed batch JSON or stderr line for bundle and
idempotency context. Compare the configured templates with the report RunID and
report path.
Safe fix: keep idempotency templates stable for retries of the same generated
report, but do not reuse the same rendered key for different generated report
content.
Relevant docs: [Operations guide](operations.md),
[Distributor adapter internals](internal/distributor-adapter.md).
## Distributor Upload Rejected
Symptom: notification fails with distributor upload rejection, HTTP status, or
bundle validation context.
Likely cause: the distributor endpoint rejected the token, bundle ID,
idempotency key, source file, or bundle path.
Diagnostic: inspect stdout JSON or stderr status lines for
`notificationError`. Confirm `notify.distributor.endpoint`,
`notify.distributor.report_path_template`, and token configuration. Token values
are redacted from weatherreporter errors.
Safe fix: fix the endpoint, token, templates, or distributor-side upload
configuration. The weatherreporter upload source is the managed Markdown report,
not `--out` or `--out-dir` copies.
Relevant docs: [Configuration reference](config.md),
[Operations guide](operations.md),
[Distributor adapter internals](internal/distributor-adapter.md).
## Distributor Unavailable
Symptom: notification fails with network, timeout, or service unavailable
context.
Likely cause: the configured distributor endpoint is unreachable, slow, or
temporarily unavailable.
Diagnostic: check network access from the weatherreporter host to
`notify.distributor.endpoint`. For batch runs, inspect which reports have
`notificationStatus: "failed"`.
Safe fix: restore distributor service availability and rerun the affected
report or batch. Stable idempotency keys make retrying the same generated report
safe unless the distributor reports a conflict.
Relevant docs: [Operations guide](operations.md).
## Unknown RunID
Symptom: an inspect command fails with `metadata for run id ... was not found`.

View File

@@ -11,6 +11,20 @@ location:
name: Brentwood
region: St. Louis Metro
secrets:
directory: ""
notify:
distributor:
enabled: false
endpoint: https://distributor.example.com
token_env: DISTRIBUTOR_UPLOAD_TOKEN
timeout: 30s
failure_policy: error
bundle_id_template: "weatherreporter.{location_id}.{report_id}.{run_id}"
idempotency_key_template: "{bundle_id}"
report_path_template: "{batch_output_name}"
missing_source:
default: warn
sources:

2
go.mod
View File

@@ -3,3 +3,5 @@ module gitea.maximumdirect.net/eric/weatherreporter
go 1.26
require gopkg.in/yaml.v3 v3.0.1
require gitea.maximumdirect.net/eric/distributor v0.4.0

48
go.sum
View File

@@ -1,3 +1,51 @@
gitea.maximumdirect.net/eric/distributor v0.4.0 h1:SRrTFjVLMv4wFZmMLwnaYtJaQlQ/wsB4OzcaSEEb524=
gitea.maximumdirect.net/eric/distributor v0.4.0/go.mod h1:G03FCFZPHpsUKC6SeMgTdbfNRpPQBdyTtDUj04e1Tu8=
github.com/aws/aws-sdk-go-v2 v1.41.9 h1:/rYeyO2+HrMztAmxAq9++XJtFMqSIpSsNA0yDGALYq4=
github.com/aws/aws-sdk-go-v2 v1.41.9/go.mod h1:+HsoOEX80qAVUitj1A2DhCNTjmb3edVyuDypb6LNEeo=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.11 h1:h5+3VT69KUBK24grGuuA5saDJTj2IIjLb9au668Fo5I=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.11/go.mod h1:dnakxebH6UwFvcvujL0LVggYQ8nEvBGjU4G/V79Nv94=
github.com/aws/aws-sdk-go-v2/config v1.32.20 h1:8VMDnWc/kEzxsI/1ngGM9mG81a8IGmIHD8KLcYGwagc=
github.com/aws/aws-sdk-go-v2/config v1.32.20/go.mod h1:PuwEpciweIXGULWeOeSTXtSbH4CW9mWdWrhdCKQI1sM=
github.com/aws/aws-sdk-go-v2/credentials v1.19.19 h1:yuFzSV1U0aRNYCQGVaTY2zW2M/L93pYHnXnrJUphYhU=
github.com/aws/aws-sdk-go-v2/credentials v1.19.19/go.mod h1:7y63L1kGzeoDlJaQ3Z578KrnmfBut96JjvJUzGwR+YE=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.25 h1:0w6dCiO8iez+YKwRhRBlL1CH/E3GTfdkuzrwj1by8vo=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.25/go.mod h1:9FDWUothyr5RCRAHc45XOiVCzUR8n/IhCYX+uVqw6vk=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.25 h1:Uii3frf9ztec/ABM2/FSH9/z7PLzxfpG8h4RpkUFflQ=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.25/go.mod h1:G6kntsA2GorAxDPbap6xgB2F+amSLUF8GJTi7PUoX44=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.25 h1:r1+/l6m+WaUJF9HISEsNOLHSNj5EXYQxK8VX6Cz9NlA=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.25/go.mod h1:cKf+D+NMDK1LndD7BowHbBZPgR9V0/5HubH0PFWvA+c=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.26 h1:A1PmWU2zfkIm9EyFlJncFXL4W4phML+h8KjltUsCvNQ=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.26/go.mod h1:dY4MRzXEizrD4hqtpKvWVGPX7QleSGGVY+EBolo1RmM=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.10 h1:d5/908OJ4bXg8lyjeMPvXetEKqoDoLi5Owy1zNue3yg=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.10/go.mod h1:a57l7Hwh+FWI+we50g5NPJHYUKeJKfXbc4w8SyXu8Ig=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.18 h1:W/EyPFl9A5rXrtoilfwHYEvzHER+K4SpBPtMXi24Mos=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.18/go.mod h1:UG50K+pvd/uy6xExbobg0rjqFBFZe6I3l75EPDZw4tg=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.25 h1:dD3dhHNglpd98gs72my22Ndqi1hqQGllFFg1F+twfxg=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.25/go.mod h1:0yAbjPfd64gG7mj85RW+fMEYdfBgCRZw8g/oWcL1pjc=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.25 h1:2pQEbwf+/6EDbiit/GcBE2K4IUpMZymaA0kOz3xK978=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.25/go.mod h1:KvT6NCcQ0EZ+ZkVRrlBMt04Po3ok23YELEp7WimhLhM=
github.com/aws/aws-sdk-go-v2/service/s3 v1.102.2 h1:ie4ElCmUKS26pzrZcIk/lmt4yWjAqLLcawstyQCh298=
github.com/aws/aws-sdk-go-v2/service/s3 v1.102.2/go.mod h1:zjsomFeX5duj+4PlMB+o4JoWTIx+G0XMyzjYrUbQkN0=
github.com/aws/aws-sdk-go-v2/service/signin v1.1.1 h1:1VwbP3qMNfxUDEXWki4rCE5iA+44VA1lokTz9HasGzw=
github.com/aws/aws-sdk-go-v2/service/signin v1.1.1/go.mod h1:vUtyoSj0OPji3kjIVSc/GlKuWEiL33f/WFxl6dmpy/A=
github.com/aws/aws-sdk-go-v2/service/sso v1.30.19 h1:N6pIsdFOW1Kd9S4KyFKXdGRBojPPxkP32+uHFWLv4Hc=
github.com/aws/aws-sdk-go-v2/service/sso v1.30.19/go.mod h1:3gt5WJArFooNmyLONS+h/R4J+o86II8du38IgCwj9dE=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.36.2 h1:hc+lBYiiTr8Zk4MTzIsQ92MeDWCIDvWGmzKUWOaBcOg=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.36.2/go.mod h1:hU6fqB3OJA6/ePheD47LQnxvjYk6br6PtQxs+Q9ojvk=
github.com/aws/aws-sdk-go-v2/service/sts v1.42.3 h1:ErklX/7uhSbkAAeyQD/Y1OoQ9hO3SJXQNEgksORW3Js=
github.com/aws/aws-sdk-go-v2/service/sts v1.42.3/go.mod h1:ULe4HCzfKPiR6R3HEurE3b1upEkuk8AkMrOKtaOxKO8=
github.com/aws/smithy-go v1.26.0 h1:9ouqbi+NyKP7fV3Te7UElCwdAb6Y8uk7LGwPE5tVe/s=
github.com/aws/smithy-go v1.26.0/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
github.com/kr/fs v0.1.0 h1:Jskdu9ieNAYnjxsi0LbQp1ulIKZV1LAFgK1tWhpZgl8=
github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
github.com/pkg/sftp v1.13.10 h1:+5FbKNTe5Z9aspU88DPIKJ9z2KZoaGCu6Sr6kKR/5mU=
github.com/pkg/sftp v1.13.10/go.mod h1:bJ1a7uDhrX/4OII+agvy28lzRvQrmIQuaHrcI1HbeGA=
github.com/yuin/goldmark v1.8.2 h1:kEGpgqJXdgbkhcOgBxkC0X0PmoPG1ZyoZ117rDVp4zE=
github.com/yuin/goldmark v1.8.2/go.mod h1:ip/1k0VRfGynBgxOz0yCqHrbZXhcjxyuS66Brc7iBKg=
golang.org/x/crypto v0.52.0 h1:RMs7fP2rXdep0CftQlK8Uf+kibLm7qkCcradZWYz988=
golang.org/x/crypto v0.52.0/go.mod h1:1QgfPxDqh0T2M/elOJtp9RvuR95kVjir0e6/BvEmGbc=
golang.org/x/sys v0.45.0 h1:dO4czNzziLiiXplLQgBCEpCvXQ3dnkn0SdaZSYdQ+FY=
golang.org/x/sys v0.45.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=

View File

@@ -0,0 +1,229 @@
// Package distributor adapts weatherreporter report artifacts to distributor uploads.
package distributor
import (
"context"
"errors"
"fmt"
"net/http"
"os"
"strings"
"time"
distributorbundle "gitea.maximumdirect.net/eric/distributor/pkg/bundle"
distributorupload "gitea.maximumdirect.net/eric/distributor/pkg/upload"
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
)
type Client struct {
Endpoint string
TokenEnv string
Timeout time.Duration
newUploadClient uploadClientFactory
}
type UploadRequest struct {
BundleID string
IdempotencyKey string
SourcePath string
BundlePath string
}
type UploadResult struct {
RunID string
Status string
}
type IdempotencyConflictError struct {
Err error
}
func (e *IdempotencyConflictError) Error() string {
if e == nil || e.Err == nil {
return "distributor idempotency conflict"
}
return e.Err.Error()
}
func (e *IdempotencyConflictError) Unwrap() error {
if e == nil {
return nil
}
return e.Err
}
type uploadClientFactory func(endpoint, token string, timeout time.Duration) (uploadClient, error)
type uploadClient interface {
UploadFiles(ctx context.Context, opts uploadFilesOptions) (uploadFilesResult, error)
}
type uploadFilesOptions struct {
BundleID string
IdempotencyKey string
SourcePath string
BundlePath string
}
type uploadFilesResult struct {
RunID string
Status string
}
func New(cfg config.DistributorNotifyConfig) *Client {
return newClient(cfg, newDistributorUploadClient)
}
func newClient(cfg config.DistributorNotifyConfig, factory uploadClientFactory) *Client {
if factory == nil {
factory = newDistributorUploadClient
}
return &Client{
Endpoint: cfg.Endpoint,
TokenEnv: cfg.TokenEnv,
Timeout: cfg.Timeout,
newUploadClient: factory,
}
}
func (c *Client) Upload(ctx context.Context, req UploadRequest) (UploadResult, error) {
if c == nil {
return UploadResult{}, fmt.Errorf("distributor client is nil")
}
if c.Endpoint == "" {
return UploadResult{}, fmt.Errorf("distributor endpoint is required")
}
if c.TokenEnv == "" {
return UploadResult{}, fmt.Errorf("distributor token environment variable is required")
}
if req.BundleID == "" {
return UploadResult{}, fmt.Errorf("distributor bundle id is required")
}
if req.IdempotencyKey == "" {
return UploadResult{}, fmt.Errorf("distributor idempotency key is required for bundle %q", req.BundleID)
}
if req.SourcePath == "" {
return UploadResult{}, fmt.Errorf("distributor source path is required for bundle %q", req.BundleID)
}
if req.BundlePath == "" {
return UploadResult{}, fmt.Errorf("distributor bundle path is required for bundle %q", req.BundleID)
}
if c.newUploadClient == nil {
return UploadResult{}, fmt.Errorf("distributor upload client factory is required for endpoint %q", c.Endpoint)
}
token := os.Getenv(c.TokenEnv)
if token == "" {
return UploadResult{}, fmt.Errorf("distributor token environment variable %q is not set", c.TokenEnv)
}
uploadClient, err := c.newUploadClient(c.Endpoint, token, c.Timeout)
if err != nil {
return UploadResult{}, fmt.Errorf("create distributor upload client for endpoint %q: %w", c.Endpoint, redactToken(err, token))
}
runCtx := ctx
if runCtx == nil {
runCtx = context.Background()
}
cancel := func() {}
if c.Timeout > 0 {
runCtx, cancel = context.WithTimeout(runCtx, c.Timeout)
}
defer cancel()
result, err := uploadClient.UploadFiles(runCtx, uploadFilesOptions{
BundleID: req.BundleID,
IdempotencyKey: req.IdempotencyKey,
SourcePath: req.SourcePath,
BundlePath: req.BundlePath,
})
if err != nil {
return UploadResult{}, wrapUploadError(err, uploadErrorContext{
Endpoint: c.Endpoint,
BundleID: req.BundleID,
IdempotencyKey: req.IdempotencyKey,
SourcePath: req.SourcePath,
BundlePath: req.BundlePath,
Token: token,
})
}
return UploadResult{
RunID: result.RunID,
Status: result.Status,
}, nil
}
type distributorUploadClient struct {
client *distributorupload.Client
}
func newDistributorUploadClient(endpoint, token string, timeout time.Duration) (uploadClient, error) {
httpClient := (*http.Client)(nil)
if timeout > 0 {
httpClient = &http.Client{Timeout: timeout}
}
client, err := distributorupload.NewClient(distributorupload.ClientOptions{
Endpoint: endpoint,
Token: token,
HTTPClient: httpClient,
})
if err != nil {
return nil, err
}
return distributorUploadClient{client: client}, nil
}
func (c distributorUploadClient) UploadFiles(ctx context.Context, opts uploadFilesOptions) (uploadFilesResult, error) {
result, err := c.client.UploadFiles(ctx, distributorupload.UploadFilesOptions{
ID: opts.BundleID,
IdempotencyKey: opts.IdempotencyKey,
Files: []distributorbundle.BundleFile{
{SourcePath: opts.SourcePath, Path: opts.BundlePath},
},
})
if err != nil {
return uploadFilesResult{}, err
}
return uploadFilesResult{
RunID: result.RunID,
Status: result.Status,
}, nil
}
type uploadErrorContext struct {
Endpoint string
BundleID string
IdempotencyKey string
SourcePath string
BundlePath string
Token string
}
func wrapUploadError(err error, ctx uploadErrorContext) error {
var conflict *distributorupload.IdempotencyConflictError
isConflict := errors.As(err, &conflict)
err = redactToken(err, ctx.Token)
if isConflict {
return &IdempotencyConflictError{
Err: fmt.Errorf("upload distributor bundle %q to endpoint %q with idempotency key %q from source %q as bundle path %q: idempotency conflict: %w", ctx.BundleID, ctx.Endpoint, ctx.IdempotencyKey, ctx.SourcePath, ctx.BundlePath, err),
}
}
return fmt.Errorf("upload distributor bundle %q to endpoint %q with idempotency key %q from source %q as bundle path %q: %w", ctx.BundleID, ctx.Endpoint, ctx.IdempotencyKey, ctx.SourcePath, ctx.BundlePath, err)
}
func redactToken(err error, token string) error {
if err == nil || token == "" {
return err
}
return errors.New(redactTokenString(err.Error(), token))
}
func redactTokenString(value, token string) string {
if token == "" {
return value
}
return strings.ReplaceAll(value, token, "[redacted]")
}

View File

@@ -0,0 +1,243 @@
package distributor
import (
"context"
"errors"
"fmt"
"strings"
"testing"
"time"
distributorupload "gitea.maximumdirect.net/eric/distributor/pkg/upload"
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
)
func TestUploadUsesConfiguredClientAndSingleFile(t *testing.T) {
cfg := config.Defaults().Notify.Distributor
cfg.Endpoint = "https://distributor.example.test"
cfg.TokenEnv = "DISTRIBUTOR_UPLOAD_TOKEN"
cfg.Timeout = 15 * time.Second
t.Setenv(cfg.TokenEnv, "secret-token")
factory := &fakeUploadFactory{
client: &fakeUploadClient{
result: uploadFilesResult{RunID: "run-123", Status: "accepted"},
},
}
client := newClient(cfg, factory.newClient)
result, err := client.Upload(context.Background(), UploadRequest{
BundleID: "weatherreporter.home.daily.run",
IdempotencyKey: "weatherreporter.home.daily.run",
SourcePath: "/tmp/report.md",
BundlePath: "daily.md",
})
if err != nil {
t.Fatalf("Upload() error = %v", err)
}
if result.RunID != "run-123" || result.Status != "accepted" {
t.Fatalf("result = %#v, want accepted run", result)
}
if factory.endpoint != cfg.Endpoint {
t.Fatalf("factory endpoint = %q, want %q", factory.endpoint, cfg.Endpoint)
}
if factory.token != "secret-token" {
t.Fatalf("factory token = %q, want secret-token", factory.token)
}
if factory.timeout != 15*time.Second {
t.Fatalf("factory timeout = %s, want 15s", factory.timeout)
}
got := factory.client.opts
if got.BundleID != "weatherreporter.home.daily.run" {
t.Fatalf("BundleID = %q, want weatherreporter.home.daily.run", got.BundleID)
}
if got.IdempotencyKey != "weatherreporter.home.daily.run" {
t.Fatalf("IdempotencyKey = %q, want weatherreporter.home.daily.run", got.IdempotencyKey)
}
if got.SourcePath != "/tmp/report.md" {
t.Fatalf("SourcePath = %q, want /tmp/report.md", got.SourcePath)
}
if got.BundlePath != "daily.md" {
t.Fatalf("BundlePath = %q, want daily.md", got.BundlePath)
}
}
func TestUploadRejectsMissingInputs(t *testing.T) {
cfg := config.Defaults().Notify.Distributor
t.Setenv(cfg.TokenEnv, "secret-token")
tests := []struct {
name string
mutate func(*Client, *UploadRequest)
wantErr string
}{
{
name: "Token",
mutate: func(c *Client, req *UploadRequest) {
t.Setenv(c.TokenEnv, "")
},
wantErr: "token environment variable",
},
{
name: "SourcePath",
mutate: func(c *Client, req *UploadRequest) {
req.SourcePath = ""
},
wantErr: "source path is required",
},
{
name: "BundlePath",
mutate: func(c *Client, req *UploadRequest) {
req.BundlePath = ""
},
wantErr: "bundle path is required",
},
{
name: "UploadClientFactory",
mutate: func(c *Client, req *UploadRequest) {
c.newUploadClient = nil
},
wantErr: "upload client factory is required",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Setenv(cfg.TokenEnv, "secret-token")
client := newClient(cfg, (&fakeUploadFactory{client: &fakeUploadClient{}}).newClient)
req := validUploadRequest()
tt.mutate(client, &req)
_, err := client.Upload(context.Background(), req)
if err == nil {
t.Fatal("Upload() error = nil, want error")
}
if !strings.Contains(err.Error(), tt.wantErr) {
t.Fatalf("error = %q, want %q", err.Error(), tt.wantErr)
}
if strings.Contains(err.Error(), "secret-token") {
t.Fatalf("error = %q, want no token value", err.Error())
}
})
}
}
func TestUploadWrapsFactoryErrorWithoutToken(t *testing.T) {
cfg := config.Defaults().Notify.Distributor
cfg.Endpoint = "https://distributor.example.test"
t.Setenv(cfg.TokenEnv, "secret-token")
factory := &fakeUploadFactory{
err: fmt.Errorf("factory failed with secret-token"),
}
client := newClient(cfg, factory.newClient)
_, err := client.Upload(context.Background(), validUploadRequest())
if err == nil {
t.Fatal("Upload() error = nil, want error")
}
if strings.Contains(err.Error(), "secret-token") {
t.Fatalf("error = %q, want no token value", err.Error())
}
if !strings.Contains(err.Error(), cfg.Endpoint) {
t.Fatalf("error = %q, want endpoint context", err.Error())
}
}
func TestUploadWrapsUploadFailureWithContextWithoutToken(t *testing.T) {
cfg := config.Defaults().Notify.Distributor
cfg.Endpoint = "https://distributor.example.test"
t.Setenv(cfg.TokenEnv, "secret-token")
factory := &fakeUploadFactory{
client: &fakeUploadClient{err: fmt.Errorf("server rejected secret-token")},
}
client := newClient(cfg, factory.newClient)
req := validUploadRequest()
_, err := client.Upload(context.Background(), req)
if err == nil {
t.Fatal("Upload() error = nil, want error")
}
for _, want := range []string{cfg.Endpoint, req.BundleID, req.IdempotencyKey, req.SourcePath, req.BundlePath} {
if !strings.Contains(err.Error(), want) {
t.Fatalf("error = %q, want context %q", err.Error(), want)
}
}
if strings.Contains(err.Error(), "secret-token") {
t.Fatalf("error = %q, want no token value", err.Error())
}
}
func TestUploadPreservesIdempotencyConflictDiagnosis(t *testing.T) {
cfg := config.Defaults().Notify.Distributor
cfg.Endpoint = "https://distributor.example.test"
t.Setenv(cfg.TokenEnv, "secret-token")
factory := &fakeUploadFactory{
client: &fakeUploadClient{
err: &distributorupload.IdempotencyConflictError{
HTTPError: distributorupload.HTTPError{
StatusCode: 409,
Status: "409 Conflict",
Message: "conflicting upload for secret-token",
},
},
},
}
client := newClient(cfg, factory.newClient)
_, err := client.Upload(context.Background(), validUploadRequest())
if err == nil {
t.Fatal("Upload() error = nil, want error")
}
var conflict *IdempotencyConflictError
if !errors.As(err, &conflict) {
t.Fatalf("Upload() error = %T %v, want IdempotencyConflictError", err, err)
}
if !strings.Contains(err.Error(), "idempotency conflict") {
t.Fatalf("error = %q, want idempotency conflict diagnosis", err.Error())
}
if strings.Contains(err.Error(), "secret-token") {
t.Fatalf("error = %q, want no token value", err.Error())
}
}
func validUploadRequest() UploadRequest {
return UploadRequest{
BundleID: "weatherreporter.home.daily.run",
IdempotencyKey: "weatherreporter.home.daily.run",
SourcePath: "/tmp/report.md",
BundlePath: "daily.md",
}
}
type fakeUploadFactory struct {
endpoint string
token string
timeout time.Duration
client *fakeUploadClient
err error
}
func (f *fakeUploadFactory) newClient(endpoint, token string, timeout time.Duration) (uploadClient, error) {
f.endpoint = endpoint
f.token = token
f.timeout = timeout
if f.err != nil {
return nil, f.err
}
return f.client, nil
}
type fakeUploadClient struct {
opts uploadFilesOptions
result uploadFilesResult
err error
}
func (c *fakeUploadClient) UploadFiles(ctx context.Context, opts uploadFilesOptions) (uploadFilesResult, error) {
c.opts = opts
if c.err != nil {
return uploadFilesResult{}, c.err
}
return c.result, nil
}

View File

@@ -3,10 +3,12 @@ package app
import (
"context"
"errors"
"fmt"
"path/filepath"
"time"
distributoradapter "gitea.maximumdirect.net/eric/weatherreporter/internal/adapters/distributor"
"gitea.maximumdirect.net/eric/weatherreporter/internal/adapters/scriptorium"
"gitea.maximumdirect.net/eric/weatherreporter/internal/adapters/weatherapi"
"gitea.maximumdirect.net/eric/weatherreporter/internal/briefing"
@@ -45,6 +47,7 @@ type GenerateRequest struct {
Date time.Time
StormStart time.Time
StormEnd time.Time
Notifier Notifier
}
type BatchRequest struct {
@@ -54,6 +57,7 @@ type BatchRequest struct {
OutputDir string
Renderer Renderer
Store state.Store
Notifier Notifier
}
type FetchBundleRequest struct {
@@ -73,6 +77,7 @@ type ReportRequest struct {
OutputPath string
Renderer Renderer
Store state.Store
Notifier Notifier
}
type BriefingResult struct {
@@ -94,6 +99,7 @@ type ReportResult struct {
RecentChanges []changes.Change
RenderResult *scriptorium.RenderResult
RunResult *scriptorium.RunResult
Notification *NotificationResult
}
type BatchResult struct {
@@ -113,6 +119,9 @@ type BatchReportResult struct {
RunID string `json:"runId"`
Status string `json:"status"`
Error string `json:"error,omitempty"`
NotificationStatus string `json:"notificationStatus,omitempty"`
NotificationRunID string `json:"notificationRunId,omitempty"`
NotificationError string `json:"notificationError,omitempty"`
GeneratedAt time.Time `json:"generatedAt"`
ValidPeriod timeutil.Period `json:"validPeriod"`
BriefingPath string `json:"briefingPath,omitempty"`
@@ -139,6 +148,45 @@ type Renderer interface {
Run(context.Context, scriptorium.RunRequest) (*scriptorium.RunResult, error)
}
type Notifier interface {
Notify(context.Context, NotificationRequest) (*NotificationResult, error)
}
type NotificationRequest struct {
ReportID report.ID
RunID string
BundleID string
IdempotencyKey string
ReportPath string
BundlePath string
}
type NotificationResult struct {
BundleID string
IdempotencyKey string
RunID string
Status string
}
type NotificationError struct {
Request NotificationRequest
Err error
}
func (e *NotificationError) Error() string {
if e == nil || e.Err == nil {
return "notification failed"
}
return e.Err.Error()
}
func (e *NotificationError) Unwrap() error {
if e == nil {
return nil
}
return e.Err
}
func Generate(ctx context.Context, req GenerateRequest) error {
now := req.Now
if now.IsZero() {
@@ -153,6 +201,7 @@ func Generate(ctx context.Context, req GenerateRequest) error {
Config: req.Config,
Resolved: resolved,
OutputPath: req.OutputPath,
Notifier: req.Notifier,
})
return err
}
@@ -211,10 +260,16 @@ func RunBatchDetailed(ctx context.Context, req BatchRequest) (*BatchResult, erro
OutputPath: outputPath,
Renderer: req.Renderer,
Store: store,
Notifier: req.Notifier,
})
if err != nil {
item.Status = "failed"
item.Error = err.Error()
var notificationErr *NotificationError
if errors.As(err, &notificationErr) {
item.NotificationStatus = "failed"
item.NotificationError = notificationErr.Error()
}
result.Failed++
} else {
item.Status = "succeeded"
@@ -224,6 +279,10 @@ func RunBatchDetailed(ctx context.Context, req BatchRequest) (*BatchResult, erro
item.ReportPath = reportResult.ReportPath
item.OutputPath = reportResult.OutputPath
item.MetadataPath = reportResult.MetadataPath
if reportResult.Notification != nil {
item.NotificationStatus = reportResult.Notification.Status
item.NotificationRunID = reportResult.Notification.RunID
}
result.Succeeded++
}
result.Reports = append(result.Reports, item)
@@ -481,6 +540,11 @@ func GenerateReport(ctx context.Context, req ReportRequest) (*ReportResult, erro
return nil, runErr
}
notification, err := notifyReport(ctx, req.Config, req.Resolved, reportPath, metadata, req.Notifier)
if err != nil {
return nil, err
}
return &ReportResult{
Briefing: briefingPackage,
BriefingPath: briefingPath,
@@ -495,6 +559,97 @@ func GenerateReport(ctx context.Context, req ReportRequest) (*ReportResult, erro
RecentChanges: recentChanges,
RenderResult: renderResult,
RunResult: runResult,
Notification: notification,
}, nil
}
func notifyReport(ctx context.Context, cfg config.Config, resolved report.Resolved, reportPath string, metadata state.Metadata, notifier Notifier) (*NotificationResult, error) {
notifier, enabled := reportNotifier(cfg, notifier)
if !enabled {
return nil, nil
}
notificationRequest, err := buildNotificationRequest(cfg, resolved, reportPath, metadata)
if err != nil {
return nil, err
}
result, err := notifier.Notify(ctx, notificationRequest)
if err != nil {
return nil, &NotificationError{
Request: notificationRequest,
Err: fmt.Errorf("notify report %q run %q from managed report %q: %w", resolved.Definition.ID, metadata.RunID, reportPath, err),
}
}
return result, nil
}
func reportNotifier(cfg config.Config, notifier Notifier) (Notifier, bool) {
if !cfg.Notify.Distributor.Enabled {
return noopNotifier{}, false
}
if notifier != nil {
return notifier, true
}
return distributorNotifier{
client: distributoradapter.New(cfg.Notify.Distributor),
}, true
}
func buildNotificationRequest(cfg config.Config, resolved report.Resolved, reportPath string, metadata state.Metadata) (NotificationRequest, error) {
values := config.DistributorTemplateValues{
LocationID: cfg.Location.ID,
ReportID: string(resolved.Definition.ID),
RunID: metadata.RunID,
ArtifactGroup: resolved.Definition.ArtifactGroup,
BatchOutputName: resolved.Definition.BatchOutputName,
}
bundleID, err := config.RenderDistributorBundleID(cfg.Notify.Distributor.BundleIDTemplate, values)
if err != nil {
return NotificationRequest{}, err
}
values.BundleID = bundleID
idempotencyKey, err := config.RenderDistributorIdempotencyKey(cfg.Notify.Distributor.IdempotencyKeyTemplate, values)
if err != nil {
return NotificationRequest{}, err
}
bundlePath, err := config.RenderDistributorReportPath(cfg.Notify.Distributor.ReportPathTemplate, values)
if err != nil {
return NotificationRequest{}, err
}
return NotificationRequest{
ReportID: resolved.Definition.ID,
RunID: metadata.RunID,
BundleID: bundleID,
IdempotencyKey: idempotencyKey,
ReportPath: reportPath,
BundlePath: bundlePath,
}, nil
}
type noopNotifier struct{}
func (noopNotifier) Notify(context.Context, NotificationRequest) (*NotificationResult, error) {
return nil, nil
}
type distributorNotifier struct {
client *distributoradapter.Client
}
func (n distributorNotifier) Notify(ctx context.Context, req NotificationRequest) (*NotificationResult, error) {
result, err := n.client.Upload(ctx, distributoradapter.UploadRequest{
BundleID: req.BundleID,
IdempotencyKey: req.IdempotencyKey,
SourcePath: req.ReportPath,
BundlePath: req.BundlePath,
})
if err != nil {
return nil, err
}
return &NotificationResult{
BundleID: req.BundleID,
IdempotencyKey: req.IdempotencyKey,
RunID: result.RunID,
Status: result.Status,
}, nil
}

View File

@@ -255,6 +255,209 @@ func TestGenerateReportWritesReportAndPreflight(t *testing.T) {
}
}
func TestGenerateReportDisabledNotificationDoesNotCallNotifier(t *testing.T) {
server := dailyBundleServer(t)
cfg := dailyTestConfig(t, server)
cfg.Workspace.Root = t.TempDir()
resolved, err := ResolveGenerate(GenerateRequest{
Config: cfg,
Report: ReportDaily,
Date: mustParse("2026-05-29T12:00:00-05:00"),
}, mustParse("2026-05-29T05:00:00-05:00"))
if err != nil {
t.Fatalf("ResolveGenerate() error = %v", err)
}
notifier := &recordingNotifier{}
_, err = GenerateReport(context.Background(), ReportRequest{
Config: cfg,
Resolved: resolved,
Renderer: successfulRenderer("# Daily Report\n"),
Notifier: notifier,
})
if err != nil {
t.Fatalf("GenerateReport() error = %v", err)
}
if len(notifier.requests) != 0 {
t.Fatalf("notification requests = %#v, want none when disabled", notifier.requests)
}
}
func TestGenerateReportNotifiesManagedReportPath(t *testing.T) {
server := dailyBundleServer(t)
cfg := dailyTestConfig(t, server)
cfg.Workspace.Root = t.TempDir()
cfg.Notify.Distributor.Enabled = true
resolved, err := ResolveGenerate(GenerateRequest{
Config: cfg,
Report: ReportDaily,
Date: mustParse("2026-05-29T12:00:00-05:00"),
}, mustParse("2026-05-29T05:00:00-05:00"))
if err != nil {
t.Fatalf("ResolveGenerate() error = %v", err)
}
notifier := &recordingNotifier{
result: &NotificationResult{RunID: "distributor-run", Status: "accepted"},
}
outputPath := filepath.Join(t.TempDir(), "daily-copy.md")
result, err := GenerateReport(context.Background(), ReportRequest{
Config: cfg,
Resolved: resolved,
OutputPath: outputPath,
Renderer: successfulRenderer("# Daily Report\n"),
Notifier: notifier,
})
if err != nil {
t.Fatalf("GenerateReport() error = %v", err)
}
if result.Notification == nil {
t.Fatal("Notification = nil, want notification result")
}
if result.Notification.RunID != "distributor-run" || result.Notification.Status != "accepted" {
t.Fatalf("Notification = %#v, want accepted distributor run", result.Notification)
}
if len(notifier.requests) != 1 {
t.Fatalf("notification requests = %d, want 1", len(notifier.requests))
}
req := notifier.requests[0]
if req.ReportPath != result.ReportPath {
t.Fatalf("notification ReportPath = %q, want managed path %q", req.ReportPath, result.ReportPath)
}
if req.ReportPath == outputPath {
t.Fatalf("notification used output copy %q, want managed report path", outputPath)
}
if req.BundlePath != "daily.md" {
t.Fatalf("notification BundlePath = %q, want daily.md", req.BundlePath)
}
if req.BundleID != "weatherreporter.home.daily_today."+result.Metadata.RunID {
t.Fatalf("notification BundleID = %q, want default template", req.BundleID)
}
if req.IdempotencyKey != req.BundleID {
t.Fatalf("IdempotencyKey = %q, want bundle id %q", req.IdempotencyKey, req.BundleID)
}
if req.RunID != result.Metadata.RunID {
t.Fatalf("notification RunID = %q, want report run id %q", req.RunID, result.Metadata.RunID)
}
}
func TestGenerateReportNotificationFailureFailsReport(t *testing.T) {
server := dailyBundleServer(t)
cfg := dailyTestConfig(t, server)
cfg.Workspace.Root = t.TempDir()
cfg.Notify.Distributor.Enabled = true
resolved, err := ResolveGenerate(GenerateRequest{
Config: cfg,
Report: ReportDaily,
Date: mustParse("2026-05-29T12:00:00-05:00"),
}, mustParse("2026-05-29T05:00:00-05:00"))
if err != nil {
t.Fatalf("ResolveGenerate() error = %v", err)
}
notifier := &recordingNotifier{err: errors.New("upload rejected")}
_, err = GenerateReport(context.Background(), ReportRequest{
Config: cfg,
Resolved: resolved,
Renderer: successfulRenderer("# Daily Report\n"),
Notifier: notifier,
})
if err == nil {
t.Fatal("GenerateReport() error = nil, want notification error")
}
if !strings.Contains(err.Error(), "notify report") || !strings.Contains(err.Error(), "upload rejected") {
t.Fatalf("error = %q, want notification context", err.Error())
}
if len(notifier.requests) != 1 {
t.Fatalf("notification requests = %d, want one attempted notification", len(notifier.requests))
}
}
func TestGenerateReportDoesNotNotifyAfterRenderOrRunFailure(t *testing.T) {
server := dailyBundleServer(t)
tests := []struct {
name string
renderer Renderer
}{
{
name: "Render",
renderer: &recordingRenderer{
renderResult: &scriptorium.RenderResult{ExitCode: 1, Stderr: "render failed"},
err: errors.New("render failed"),
},
},
{
name: "Run",
renderer: &recordingRenderer{
renderResult: &scriptorium.RenderResult{ExitCode: 0},
runResult: &scriptorium.RunResult{ExitCode: 2, Stderr: "run failed"},
runErr: errors.New("run failed"),
runBody: "# Daily Report\n",
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := dailyTestConfig(t, server)
cfg.Workspace.Root = t.TempDir()
cfg.Notify.Distributor.Enabled = true
resolved, err := ResolveGenerate(GenerateRequest{
Config: cfg,
Report: ReportDaily,
Date: mustParse("2026-05-29T12:00:00-05:00"),
}, mustParse("2026-05-29T05:00:00-05:00"))
if err != nil {
t.Fatalf("ResolveGenerate() error = %v", err)
}
notifier := &recordingNotifier{}
_, err = GenerateReport(context.Background(), ReportRequest{
Config: cfg,
Resolved: resolved,
Renderer: tt.renderer,
Notifier: notifier,
})
if err == nil {
t.Fatal("GenerateReport() error = nil, want generation error")
}
if len(notifier.requests) != 0 {
t.Fatalf("notification requests = %#v, want none after generation failure", notifier.requests)
}
})
}
}
func TestGenerateReportDoesNotNotifyAfterFetchFailure(t *testing.T) {
cfg := config.Defaults()
cfg.WeatherAPI.BaseURL = ""
cfg.WeatherAPI.Timezone = "America/Chicago"
cfg.Workspace.Root = t.TempDir()
cfg.Notify.Distributor.Enabled = true
resolved, err := ResolveGenerate(GenerateRequest{
Config: cfg,
Report: ReportDaily,
Date: mustParse("2026-05-29T12:00:00-05:00"),
}, mustParse("2026-05-29T05:00:00-05:00"))
if err != nil {
t.Fatalf("ResolveGenerate() error = %v", err)
}
notifier := &recordingNotifier{}
_, err = GenerateReport(context.Background(), ReportRequest{
Config: cfg,
Resolved: resolved,
Renderer: successfulRenderer("# Daily Report\n"),
Notifier: notifier,
})
if err == nil {
t.Fatal("GenerateReport() error = nil, want fetch setup error")
}
if len(notifier.requests) != 0 {
t.Fatalf("notification requests = %#v, want none after fetch failure", notifier.requests)
}
}
func TestGenerateReportPersistsFailedPreflight(t *testing.T) {
server := dailyBundleServer(t)
cfg := config.Defaults()
@@ -1008,6 +1211,62 @@ func TestRunBatchContinuesAfterReportFailure(t *testing.T) {
}
}
func TestRunBatchContinuesAfterNotificationFailure(t *testing.T) {
server := dailyBundleServer(t)
cfg := config.Defaults()
cfg.WeatherAPI.BaseURL = server.URL + "/"
cfg.WeatherAPI.Timezone = "America/Chicago"
cfg.Workspace.Root = t.TempDir()
cfg.Notify.Distributor.Enabled = true
notifier := &recordingNotifier{
errByReport: map[report.ID]error{
report.ThreeDay: errors.New("distributor unavailable"),
},
}
result, err := RunBatchDetailed(context.Background(), BatchRequest{
Config: cfg,
Batch: BatchMorning,
Now: mustParse("2026-05-29T05:00:00-05:00"),
Renderer: &selectiveRenderer{runBody: "# Batch Report\n"},
Notifier: notifier,
})
if err != nil {
t.Fatalf("RunBatchDetailed() error = %v", err)
}
if result.Total != 3 || result.Succeeded != 2 || result.Failed != 1 {
t.Fatalf("summary total/succeeded/failed = %d/%d/%d, want 3/2/1", result.Total, result.Succeeded, result.Failed)
}
if len(notifier.requests) != 3 {
t.Fatalf("notification requests = %d, want one per generated report", len(notifier.requests))
}
var failedThreeDay bool
for _, item := range result.Reports {
if item.ReportID == report.ThreeDay {
if item.Status == "failed" && strings.Contains(item.Error, "notify report") && strings.Contains(item.Error, "distributor unavailable") {
failedThreeDay = true
}
if item.NotificationStatus != "failed" {
t.Fatalf("3-day notification status = %q, want failed", item.NotificationStatus)
}
if !strings.Contains(item.NotificationError, "distributor unavailable") {
t.Fatalf("3-day notification error = %q, want distributor unavailable", item.NotificationError)
}
continue
}
if item.Status != "succeeded" {
t.Fatalf("report %s status = %s, want succeeded", item.ReportID, item.Status)
}
if item.NotificationStatus != "accepted" {
t.Fatalf("report %s notification status = %q, want accepted", item.ReportID, item.NotificationStatus)
}
}
if !failedThreeDay {
t.Fatalf("reports = %#v, want notification failure on 3-day item", result.Reports)
}
}
func TestRunBatchUsesOutputDirectory(t *testing.T) {
server := dailyBundleServer(t)
cfg := config.Defaults()
@@ -1164,6 +1423,14 @@ type recordingRenderer struct {
runBody string
}
func successfulRenderer(body string) *recordingRenderer {
return &recordingRenderer{
renderResult: &scriptorium.RenderResult{ExitCode: 0},
runResult: &scriptorium.RunResult{ExitCode: 0},
runBody: body,
}
}
type selectiveRenderer struct {
renderCalls int
runCalls int
@@ -1171,6 +1438,38 @@ type selectiveRenderer struct {
runBody string
}
type recordingNotifier struct {
requests []NotificationRequest
result *NotificationResult
err error
errByReport map[report.ID]error
}
func (n *recordingNotifier) Notify(_ context.Context, req NotificationRequest) (*NotificationResult, error) {
n.requests = append(n.requests, req)
if err := n.errByReport[req.ReportID]; err != nil {
return nil, err
}
if n.err != nil {
return nil, n.err
}
if n.result != nil {
result := *n.result
if result.BundleID == "" {
result.BundleID = req.BundleID
}
if result.IdempotencyKey == "" {
result.IdempotencyKey = req.IdempotencyKey
}
return &result, nil
}
return &NotificationResult{
BundleID: req.BundleID,
IdempotencyKey: req.IdempotencyKey,
Status: "accepted",
}, nil
}
func (r *selectiveRenderer) Render(_ context.Context, req scriptorium.RenderRequest) (*scriptorium.RenderResult, error) {
r.renderCalls++
if req.PromptID == r.failRenderPrompt {

View File

@@ -352,11 +352,21 @@ func writeRunLogs(stderr io.Writer, result *app.BatchResult) {
return
}
for _, item := range result.Reports {
notificationFields := ""
if item.NotificationStatus != "" {
notificationFields += fmt.Sprintf(" notificationStatus=%q", item.NotificationStatus)
}
if item.NotificationRunID != "" {
notificationFields += fmt.Sprintf(" notificationRunId=%q", item.NotificationRunID)
}
if item.NotificationError != "" {
notificationFields += fmt.Sprintf(" notificationError=%q", item.NotificationError)
}
if item.Status == "failed" {
_, _ = fmt.Fprintf(stderr, "report=%s status=failed error=%q\n", item.ReportID, item.Error)
_, _ = fmt.Fprintf(stderr, "report=%s status=failed error=%q%s\n", item.ReportID, item.Error, notificationFields)
continue
}
_, _ = fmt.Fprintf(stderr, "report=%s status=succeeded output=%q\n", item.ReportID, item.OutputPath)
_, _ = fmt.Fprintf(stderr, "report=%s status=succeeded output=%q%s\n", item.ReportID, item.OutputPath, notificationFields)
}
_, _ = fmt.Fprintf(stderr, "batch=%s total=%d succeeded=%d failed=%d\n", result.Batch, result.Total, result.Succeeded, result.Failed)
}

View File

@@ -346,6 +346,88 @@ func TestRunMorningReportsPartialFailureAndContinues(t *testing.T) {
}
}
func TestBatchOutputIncludesNotificationDetails(t *testing.T) {
result := &app.BatchResult{
Batch: app.BatchMorning,
Total: 2,
Succeeded: 1,
Failed: 1,
Reports: []app.BatchReportResult{
{
ReportID: "daily_today",
Status: "succeeded",
OutputPath: "/tmp/daily.md",
NotificationStatus: "accepted",
NotificationRunID: "distributor-run-1",
},
{
ReportID: "three_day",
Status: "failed",
Error: "notify report three_day: upload failed",
NotificationStatus: "failed",
NotificationError: "notify report three_day: upload failed",
},
},
}
var stdout bytes.Buffer
var stderr bytes.Buffer
if err := writeJSON(&stdout, result); err != nil {
t.Fatalf("writeJSON() error = %v", err)
}
writeRunLogs(&stderr, result)
var decoded app.BatchResult
if err := json.Unmarshal(stdout.Bytes(), &decoded); err != nil {
t.Fatalf("decode batch JSON: %v\n%s", err, stdout.String())
}
if decoded.Reports[0].NotificationStatus != "accepted" || decoded.Reports[0].NotificationRunID != "distributor-run-1" {
t.Fatalf("success notification fields = %#v", decoded.Reports[0])
}
if decoded.Reports[1].NotificationStatus != "failed" || !strings.Contains(decoded.Reports[1].NotificationError, "upload failed") {
t.Fatalf("failure notification fields = %#v", decoded.Reports[1])
}
if !strings.Contains(stderr.String(), `notificationStatus="accepted"`) || !strings.Contains(stderr.String(), `notificationRunId="distributor-run-1"`) {
t.Fatalf("stderr missing success notification fields:\n%s", stderr.String())
}
if !strings.Contains(stderr.String(), `notificationStatus="failed"`) || !strings.Contains(stderr.String(), `notificationError="notify report three_day: upload failed"`) {
t.Fatalf("stderr missing failure notification fields:\n%s", stderr.String())
}
}
func TestBatchOutputDoesNotExposeSecretLikeNotificationErrors(t *testing.T) {
result := &app.BatchResult{
Batch: app.BatchMorning,
Total: 1,
Failed: 1,
Reports: []app.BatchReportResult{
{
ReportID: "daily_today",
Status: "failed",
Error: "notify report daily_today: upload failed: [redacted]",
NotificationStatus: "failed",
NotificationError: "notify report daily_today: upload failed: [redacted]",
},
},
}
var stdout bytes.Buffer
var stderr bytes.Buffer
if err := writeJSON(&stdout, result); err != nil {
t.Fatalf("writeJSON() error = %v", err)
}
writeRunLogs(&stderr, result)
for _, output := range []string{stdout.String(), stderr.String()} {
if strings.Contains(output, "DISTRIBUTOR_SECRET_TOKEN") {
t.Fatalf("output contains token value:\n%s", output)
}
if !strings.Contains(output, "[redacted]") {
t.Fatalf("output missing redacted marker:\n%s", output)
}
}
}
func TestRunEveningUsesOutputDirectoryAndSummary(t *testing.T) {
server := dailyServer(t)
tempDir := t.TempDir()
@@ -384,6 +466,103 @@ func TestRunEveningUsesOutputDirectoryAndSummary(t *testing.T) {
}
}
func TestRunEveningReportsNotificationSuccess(t *testing.T) {
server := dailyServer(t)
distributorServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/upload" {
http.NotFound(w, r)
return
}
w.WriteHeader(http.StatusAccepted)
_, _ = w.Write([]byte(`{"run_id":"distributor-run-1","status":"accepted"}`))
}))
t.Cleanup(distributorServer.Close)
tempDir := t.TempDir()
scriptoriumPath := writeFakeScriptorium(t, tempDir)
configPath := filepath.Join(tempDir, "config.yml")
workspaceRoot := filepath.Join(tempDir, "workspace")
configBody := "weather_api:\n base_url: " + server.URL + "/\n timezone: America/Chicago\nscriptorium:\n binary: " + scriptoriumPath + "\nworkspace:\n root: " + workspaceRoot + "\nnotify:\n distributor:\n enabled: true\n endpoint: " + distributorServer.URL + "\n token_env: CLI_DISTRIBUTOR_TOKEN\n"
if err := os.WriteFile(configPath, []byte(configBody), 0o600); err != nil {
t.Fatalf("write config: %v", err)
}
t.Setenv("CLI_DISTRIBUTOR_TOKEN", "cli-secret-token")
var stdout bytes.Buffer
var stderr bytes.Buffer
runner := Runner{Clock: fixedClock()}
err := runner.Run(context.Background(), []string{
"run", "evening",
"--config", configPath,
}, &stdout, &stderr)
if err != nil {
t.Fatalf("Run() error = %v", err)
}
var summary app.BatchResult
if decodeErr := json.Unmarshal(stdout.Bytes(), &summary); decodeErr != nil {
t.Fatalf("decode summary: %v\n%s", decodeErr, stdout.String())
}
if len(summary.Reports) != 1 {
t.Fatalf("reports = %#v, want one report", summary.Reports)
}
if summary.Reports[0].NotificationStatus != "accepted" || summary.Reports[0].NotificationRunID != "distributor-run-1" {
t.Fatalf("notification fields = %#v", summary.Reports[0])
}
if !strings.Contains(stderr.String(), `notificationStatus="accepted"`) || !strings.Contains(stderr.String(), `notificationRunId="distributor-run-1"`) {
t.Fatalf("stderr missing notification fields:\n%s", stderr.String())
}
if strings.Contains(stdout.String(), "cli-secret-token") || strings.Contains(stderr.String(), "cli-secret-token") {
t.Fatalf("output contains token value\nstdout=%s\nstderr=%s", stdout.String(), stderr.String())
}
}
func TestRunEveningReportsNotificationFailureWithoutToken(t *testing.T) {
server := dailyServer(t)
distributorServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusBadRequest)
_, _ = w.Write([]byte(`{"error":"rejected cli-secret-token","retryable":false}`))
}))
t.Cleanup(distributorServer.Close)
tempDir := t.TempDir()
scriptoriumPath := writeFakeScriptorium(t, tempDir)
configPath := filepath.Join(tempDir, "config.yml")
workspaceRoot := filepath.Join(tempDir, "workspace")
configBody := "weather_api:\n base_url: " + server.URL + "/\n timezone: America/Chicago\nscriptorium:\n binary: " + scriptoriumPath + "\nworkspace:\n root: " + workspaceRoot + "\nnotify:\n distributor:\n enabled: true\n endpoint: " + distributorServer.URL + "\n token_env: CLI_DISTRIBUTOR_TOKEN\n"
if err := os.WriteFile(configPath, []byte(configBody), 0o600); err != nil {
t.Fatalf("write config: %v", err)
}
t.Setenv("CLI_DISTRIBUTOR_TOKEN", "cli-secret-token")
var stdout bytes.Buffer
var stderr bytes.Buffer
runner := Runner{Clock: fixedClock()}
err := runner.Run(context.Background(), []string{
"run", "evening",
"--config", configPath,
}, &stdout, &stderr)
if err == nil {
t.Fatal("Run() error = nil, want notification failure")
}
var summary app.BatchResult
if decodeErr := json.Unmarshal(stdout.Bytes(), &summary); decodeErr != nil {
t.Fatalf("decode summary: %v\n%s", decodeErr, stdout.String())
}
if len(summary.Reports) != 1 || summary.Reports[0].NotificationStatus != "failed" {
t.Fatalf("summary reports = %#v, want failed notification", summary.Reports)
}
for _, output := range []string{stdout.String(), stderr.String(), err.Error()} {
if strings.Contains(output, "cli-secret-token") {
t.Fatalf("output contains token value:\n%s", output)
}
}
for _, output := range []string{stdout.String(), stderr.String()} {
if !strings.Contains(output, "[redacted]") {
t.Fatalf("output missing redaction marker:\n%s", output)
}
}
}
func TestRunMorningGeneratesDailyAndThreeDayOnSunday(t *testing.T) {
server := dailyServer(t)
tempDir := t.TempDir()

View File

@@ -5,16 +5,21 @@ package config
import "time"
type MissingSourcePolicy string
type NotifyFailurePolicy string
const (
MissingSourceError MissingSourcePolicy = "error"
MissingSourceWarn MissingSourcePolicy = "warn"
MissingSourceNone MissingSourcePolicy = "none"
NotifyFailureError NotifyFailurePolicy = "error"
)
type Config struct {
WeatherAPI WeatherAPIConfig `yaml:"weather_api"`
Location LocationConfig `yaml:"location"`
Secrets SecretsConfig `yaml:"secrets"`
Notify NotifyConfig `yaml:"notify"`
MissingSource MissingSourceConfig `yaml:"missing_source"`
Scriptorium ScriptoriumConfig `yaml:"scriptorium"`
Workspace WorkspaceConfig `yaml:"workspace"`
@@ -37,6 +42,25 @@ type LocationConfig struct {
Region string `yaml:"region"`
}
type SecretsConfig struct {
Directory string `yaml:"directory"`
}
type NotifyConfig struct {
Distributor DistributorNotifyConfig `yaml:"distributor"`
}
type DistributorNotifyConfig struct {
Enabled bool `yaml:"enabled"`
Endpoint string `yaml:"endpoint"`
TokenEnv string `yaml:"token_env"`
Timeout time.Duration `yaml:"timeout"`
FailurePolicy NotifyFailurePolicy `yaml:"failure_policy"`
BundleIDTemplate string `yaml:"bundle_id_template"`
IdempotencyKeyTemplate string `yaml:"idempotency_key_template"`
ReportPathTemplate string `yaml:"report_path_template"`
}
type MissingSourceConfig struct {
Default MissingSourcePolicy `yaml:"default"`
Sources map[string]MissingSourcePolicy `yaml:"sources"`

View File

@@ -26,6 +26,33 @@ func TestDefaults(t *testing.T) {
if cfg.Location.ID != "home" || cfg.Location.Name != "Brentwood" || cfg.Location.Region != "St. Louis Metro" {
t.Fatalf("Location = %#v, want home/Brentwood/St. Louis Metro", cfg.Location)
}
if cfg.Secrets.Directory != "" {
t.Fatalf("Secrets.Directory = %q, want empty", cfg.Secrets.Directory)
}
if cfg.Notify.Distributor.Enabled {
t.Fatalf("Notify.Distributor.Enabled = true, want false")
}
if cfg.Notify.Distributor.Endpoint != "https://distributor.example.com" {
t.Fatalf("Notify.Distributor.Endpoint = %q, want default endpoint", cfg.Notify.Distributor.Endpoint)
}
if cfg.Notify.Distributor.TokenEnv != "DISTRIBUTOR_UPLOAD_TOKEN" {
t.Fatalf("Notify.Distributor.TokenEnv = %q, want DISTRIBUTOR_UPLOAD_TOKEN", cfg.Notify.Distributor.TokenEnv)
}
if cfg.Notify.Distributor.Timeout != 30*time.Second {
t.Fatalf("Notify.Distributor.Timeout = %s, want 30s", cfg.Notify.Distributor.Timeout)
}
if cfg.Notify.Distributor.FailurePolicy != NotifyFailureError {
t.Fatalf("Notify.Distributor.FailurePolicy = %q, want error", cfg.Notify.Distributor.FailurePolicy)
}
if cfg.Notify.Distributor.BundleIDTemplate != "weatherreporter.{location_id}.{report_id}.{run_id}" {
t.Fatalf("Notify.Distributor.BundleIDTemplate = %q, want default", cfg.Notify.Distributor.BundleIDTemplate)
}
if cfg.Notify.Distributor.IdempotencyKeyTemplate != "{bundle_id}" {
t.Fatalf("Notify.Distributor.IdempotencyKeyTemplate = %q, want default", cfg.Notify.Distributor.IdempotencyKeyTemplate)
}
if cfg.Notify.Distributor.ReportPathTemplate != "{batch_output_name}" {
t.Fatalf("Notify.Distributor.ReportPathTemplate = %q, want default", cfg.Notify.Distributor.ReportPathTemplate)
}
if cfg.MissingSource.Default != MissingSourceWarn {
t.Fatalf("MissingSource.Default = %q, want warn", cfg.MissingSource.Default)
}
@@ -112,3 +139,417 @@ func TestLoadAppliesOverrides(t *testing.T) {
t.Fatalf("Timezone = %q, want +09:30", cfg.WeatherAPI.Timezone)
}
}
func TestDisabledDistributorNotifyAcceptsOmittedFields(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "config.yml")
if err := os.WriteFile(path, []byte("notify:\n distributor:\n enabled: false\n"), 0o600); err != nil {
t.Fatalf("write config fixture: %v", err)
}
cfg, err := LoadFile(path)
if err != nil {
t.Fatalf("LoadFile() error = %v", err)
}
if cfg.Notify.Distributor.Enabled {
t.Fatalf("Notify.Distributor.Enabled = true, want false")
}
}
func TestEnabledDistributorNotifyValidation(t *testing.T) {
tests := []struct {
name string
mutate func(*Config)
wantErr string
}{
{
name: "Endpoint",
mutate: func(cfg *Config) {
cfg.Notify.Distributor.Endpoint = "distributor.example.com"
},
wantErr: "notify.distributor.endpoint",
},
{
name: "TokenEnvEmpty",
mutate: func(cfg *Config) {
cfg.Notify.Distributor.TokenEnv = ""
},
wantErr: "notify.distributor.token_env",
},
{
name: "TokenEnvInvalid",
mutate: func(cfg *Config) {
cfg.Notify.Distributor.TokenEnv = "1TOKEN"
},
wantErr: "notify.distributor.token_env",
},
{
name: "Timeout",
mutate: func(cfg *Config) {
cfg.Notify.Distributor.Timeout = 0
},
wantErr: "notify.distributor.timeout",
},
{
name: "FailurePolicy",
mutate: func(cfg *Config) {
cfg.Notify.Distributor.FailurePolicy = "warn"
},
wantErr: "notify.distributor.failure_policy",
},
{
name: "BundleTemplate",
mutate: func(cfg *Config) {
cfg.Notify.Distributor.BundleIDTemplate = "{unknown}"
},
wantErr: "notify.distributor.bundle_id_template",
},
{
name: "IdempotencyTemplate",
mutate: func(cfg *Config) {
cfg.Notify.Distributor.IdempotencyKeyTemplate = "{unknown}"
},
wantErr: "notify.distributor.idempotency_key_template",
},
{
name: "ReportPathTemplateUnknown",
mutate: func(cfg *Config) {
cfg.Notify.Distributor.ReportPathTemplate = "{bundle_id}"
},
wantErr: "notify.distributor.report_path_template",
},
{
name: "ReportPathTemplateInvalidPath",
mutate: func(cfg *Config) {
cfg.Notify.Distributor.ReportPathTemplate = "/{batch_output_name}"
},
wantErr: "notify.distributor.report_path_template",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := Defaults()
cfg.Notify.Distributor.Enabled = true
tt.mutate(&cfg)
err := Validate(cfg)
if err == nil {
t.Fatal("Validate() error = nil, want error")
}
if !strings.Contains(err.Error(), tt.wantErr) {
t.Fatalf("error = %q, want %q", err.Error(), tt.wantErr)
}
})
}
}
func TestDistributorTemplateRendering(t *testing.T) {
values := DistributorTemplateValues{
LocationID: "home",
ReportID: "daily",
RunID: "20260607T120000Z",
ArtifactGroup: "daily",
BatchOutputName: "daily.md",
BundleID: "weatherreporter.home.daily.20260607T120000Z",
}
bundleID, err := RenderDistributorBundleID("weatherreporter.{location_id}.{report_id}.{run_id}", values)
if err != nil {
t.Fatalf("RenderDistributorBundleID() error = %v", err)
}
if bundleID != "weatherreporter.home.daily.20260607T120000Z" {
t.Fatalf("bundleID = %q, want rendered value", bundleID)
}
idempotencyKey, err := RenderDistributorIdempotencyKey("{bundle_id}", values)
if err != nil {
t.Fatalf("RenderDistributorIdempotencyKey() error = %v", err)
}
if idempotencyKey != "weatherreporter.home.daily.20260607T120000Z" {
t.Fatalf("idempotencyKey = %q, want rendered bundle ID", idempotencyKey)
}
reportPath, err := RenderDistributorReportPath("reports/{batch_output_name}", values)
if err != nil {
t.Fatalf("RenderDistributorReportPath() error = %v", err)
}
if reportPath != "reports/daily.md" {
t.Fatalf("reportPath = %q, want reports/daily.md", reportPath)
}
}
func TestDistributorTemplateRejectsUnknownAndMalformedVariables(t *testing.T) {
tests := []struct {
name string
template string
}{
{name: "Unknown", template: "{unknown}"},
{name: "Unclosed", template: "{location_id"},
{name: "Unopened", template: "location_id}"},
{name: "Empty", template: "{}"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := RenderDistributorBundleID(tt.template, DistributorTemplateValues{})
if err == nil {
t.Fatal("RenderDistributorBundleID() error = nil, want error")
}
})
}
}
func TestDistributorReportPathValidation(t *testing.T) {
tests := []struct {
name string
path string
ok bool
}{
{name: "Simple", path: "daily.md", ok: true},
{name: "Nested", path: "reports/daily.md", ok: true},
{name: "Empty", path: "", ok: false},
{name: "Absolute", path: "/reports/daily.md", ok: false},
{name: "WindowsAbsolute", path: "C:/reports/daily.md", ok: false},
{name: "Backslash", path: `reports\daily.md`, ok: false},
{name: "CurrentSegment", path: "reports/./daily.md", ok: false},
{name: "ParentSegment", path: "reports/../daily.md", ok: false},
{name: "EmptySegment", path: "reports//daily.md", ok: false},
{name: "Manifest", path: "reports/manifest.json", ok: false},
{name: "DistributorMetadata", path: "reports/.distributor.json", ok: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := ValidateDistributorReportPath(tt.path)
if tt.ok && err != nil {
t.Fatalf("ValidateDistributorReportPath() error = %v", err)
}
if !tt.ok && err == nil {
t.Fatal("ValidateDistributorReportPath() error = nil, want error")
}
})
}
}
func TestDistributorReportPathRenderingRejectsInvalidValues(t *testing.T) {
tests := []struct {
name string
batchOutputName string
}{
{name: "Absolute", batchOutputName: "/daily.md"},
{name: "Backslash", batchOutputName: `reports\daily.md`},
{name: "CurrentSegment", batchOutputName: "./daily.md"},
{name: "ParentSegment", batchOutputName: "../daily.md"},
{name: "EmptySegment", batchOutputName: "reports//daily.md"},
{name: "Manifest", batchOutputName: "manifest.json"},
{name: "DistributorMetadata", batchOutputName: ".distributor.json"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := RenderDistributorReportPath("{batch_output_name}", DistributorTemplateValues{
BatchOutputName: tt.batchOutputName,
})
if err == nil {
t.Fatal("RenderDistributorReportPath() error = nil, want error")
}
})
}
}
func TestLoadFileLoadsSecretsBeforeReturningNotifyConfig(t *testing.T) {
dir := t.TempDir()
secretsDir := filepath.Join(dir, "secrets")
if err := os.Mkdir(secretsDir, 0o700); err != nil {
t.Fatalf("create secrets directory: %v", err)
}
if err := os.WriteFile(filepath.Join(secretsDir, "DISTRIBUTOR_UPLOAD_TOKEN"), []byte("loaded-token"), 0o600); err != nil {
t.Fatalf("write secret: %v", err)
}
path := filepath.Join(dir, "config.yml")
configYAML := "secrets:\n" +
" directory: " + secretsDir + "\n" +
"notify:\n" +
" distributor:\n" +
" enabled: true\n"
if err := os.WriteFile(path, []byte(configYAML), 0o600); err != nil {
t.Fatalf("write config fixture: %v", err)
}
t.Setenv("DISTRIBUTOR_UPLOAD_TOKEN", "")
cfg, err := LoadFile(path)
if err != nil {
t.Fatalf("LoadFile() error = %v", err)
}
if cfg.Notify.Distributor.TokenEnv != "DISTRIBUTOR_UPLOAD_TOKEN" {
t.Fatalf("TokenEnv = %q, want DISTRIBUTOR_UPLOAD_TOKEN", cfg.Notify.Distributor.TokenEnv)
}
if got := os.Getenv(cfg.Notify.Distributor.TokenEnv); got != "loaded-token" {
t.Fatalf("environment value = %q, want loaded-token", got)
}
}
func TestLoadSecretsDisabledLeavesEnvironmentUnchanged(t *testing.T) {
t.Setenv("WEATHERREPORTER_DISABLED_SECRET", "original")
if err := loadSecrets(SecretsConfig{}); err != nil {
t.Fatalf("loadSecrets() error = %v", err)
}
if got := os.Getenv("WEATHERREPORTER_DISABLED_SECRET"); got != "original" {
t.Fatalf("environment value = %q, want original", got)
}
}
func TestLoadFileLoadsSecretsDirectory(t *testing.T) {
dir := t.TempDir()
secretsDir := filepath.Join(dir, "secrets")
if err := os.Mkdir(secretsDir, 0o700); err != nil {
t.Fatalf("create secrets directory: %v", err)
}
if err := os.WriteFile(filepath.Join(secretsDir, "WEATHERREPORTER_SECRET"), []byte("from-file"), 0o600); err != nil {
t.Fatalf("write secret: %v", err)
}
path := filepath.Join(dir, "config.yml")
if err := os.WriteFile(path, []byte("secrets:\n directory: "+secretsDir+"\n"), 0o600); err != nil {
t.Fatalf("write config fixture: %v", err)
}
t.Setenv("WEATHERREPORTER_SECRET", "")
if _, err := LoadFile(path); err != nil {
t.Fatalf("LoadFile() error = %v", err)
}
if got := os.Getenv("WEATHERREPORTER_SECRET"); got != "from-file" {
t.Fatalf("environment value = %q, want from-file", got)
}
}
func TestLoadSecretsOverwritesExistingEnvironment(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "WEATHERREPORTER_SECRET"), []byte("from-file"), 0o600); err != nil {
t.Fatalf("write secret: %v", err)
}
t.Setenv("WEATHERREPORTER_SECRET", "existing")
if err := loadSecrets(SecretsConfig{Directory: dir}); err != nil {
t.Fatalf("loadSecrets() error = %v", err)
}
if got := os.Getenv("WEATHERREPORTER_SECRET"); got != "from-file" {
t.Fatalf("environment value = %q, want from-file", got)
}
}
func TestLoadSecretsTrimsOneTrailingLineEnding(t *testing.T) {
tests := []struct {
name string
input string
want string
}{
{name: "LF", input: "value\n", want: "value"},
{name: "CRLF", input: "value\r\n", want: "value"},
{name: "TwoLF", input: "value\n\n", want: "value\n"},
{name: "LoneCR", input: "value\r", want: "value\r"},
{name: "NoNewline", input: "value", want: "value"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "WEATHERREPORTER_SECRET"), []byte(tt.input), 0o600); err != nil {
t.Fatalf("write secret: %v", err)
}
t.Setenv("WEATHERREPORTER_SECRET", "")
if err := loadSecrets(SecretsConfig{Directory: dir}); err != nil {
t.Fatalf("loadSecrets() error = %v", err)
}
if got := os.Getenv("WEATHERREPORTER_SECRET"); got != tt.want {
t.Fatalf("environment value = %q, want %q", got, tt.want)
}
})
}
}
func TestLoadSecretsRejectsInvalidDirectoryEntries(t *testing.T) {
tests := []struct {
name string
setup func(t *testing.T, dir string)
wantErr string
}{
{
name: "InvalidFilename",
setup: func(t *testing.T, dir string) {
if err := os.WriteFile(filepath.Join(dir, "1INVALID"), []byte("secret-value"), 0o600); err != nil {
t.Fatalf("write invalid secret: %v", err)
}
},
wantErr: "invalid environment variable name",
},
{
name: "Subdirectory",
setup: func(t *testing.T, dir string) {
if err := os.Mkdir(filepath.Join(dir, "SUBDIR"), 0o700); err != nil {
t.Fatalf("create subdirectory: %v", err)
}
},
wantErr: "not a directory",
},
{
name: "Symlink",
setup: func(t *testing.T, dir string) {
target := filepath.Join(dir, "TARGET")
if err := os.WriteFile(target, []byte("secret-value"), 0o600); err != nil {
t.Fatalf("write target: %v", err)
}
if err := os.Symlink(target, filepath.Join(dir, "SYMLINK")); err != nil {
t.Fatalf("create symlink: %v", err)
}
},
wantErr: "not a symlink",
},
{
name: "Unreadable",
setup: func(t *testing.T, dir string) {
path := filepath.Join(dir, "UNREADABLE")
if err := os.WriteFile(path, []byte("secret-value"), 0o600); err != nil {
t.Fatalf("write unreadable secret: %v", err)
}
if err := os.Chmod(path, 0o000); err != nil {
t.Fatalf("chmod unreadable secret: %v", err)
}
t.Cleanup(func() {
_ = os.Chmod(path, 0o600)
})
},
wantErr: "read secret file",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
dir := t.TempDir()
tt.setup(t, dir)
err := loadSecrets(SecretsConfig{Directory: dir})
if err == nil {
t.Fatal("loadSecrets() error = nil, want error")
}
if !strings.Contains(err.Error(), tt.wantErr) {
t.Fatalf("error = %q, want %q", err.Error(), tt.wantErr)
}
if strings.Contains(err.Error(), "secret-value") {
t.Fatalf("error = %q, want no secret value", err.Error())
}
})
}
}
func TestLoadSecretsRejectsMissingDirectory(t *testing.T) {
err := loadSecrets(SecretsConfig{Directory: filepath.Join(t.TempDir(), "missing")})
if err == nil {
t.Fatal("loadSecrets() error = nil, want missing directory error")
}
if !strings.Contains(err.Error(), "read secrets directory") {
t.Fatalf("error = %q, want read secrets directory context", err.Error())
}
}

View File

@@ -18,6 +18,21 @@ func Defaults() Config {
Name: "Brentwood",
Region: "St. Louis Metro",
},
Secrets: SecretsConfig{
Directory: "",
},
Notify: NotifyConfig{
Distributor: DistributorNotifyConfig{
Enabled: false,
Endpoint: "https://distributor.example.com",
TokenEnv: "DISTRIBUTOR_UPLOAD_TOKEN",
Timeout: 30 * time.Second,
FailurePolicy: NotifyFailureError,
BundleIDTemplate: "weatherreporter.{location_id}.{report_id}.{run_id}",
IdempotencyKeyTemplate: "{bundle_id}",
ReportPathTemplate: "{batch_output_name}",
},
},
MissingSource: MissingSourceConfig{
Default: MissingSourceWarn,
Sources: map[string]MissingSourcePolicy{},

View File

@@ -35,6 +35,10 @@ func Load(opts LoadOptions) (Config, error) {
cfg.WeatherAPI.Timezone = opts.Timezone
}
if err := loadSecrets(cfg.Secrets); err != nil {
return Config{}, err
}
if err := Validate(cfg); err != nil {
return Config{}, err
}

View File

@@ -0,0 +1,145 @@
package config
import (
"fmt"
"path/filepath"
"strings"
)
type DistributorTemplateValues struct {
LocationID string
ReportID string
RunID string
ArtifactGroup string
BatchOutputName string
BundleID string
}
var distributorTemplateVariables = map[string]struct{}{
"location_id": {},
"report_id": {},
"run_id": {},
"artifact_group": {},
"batch_output_name": {},
}
var distributorIdempotencyTemplateVariables = map[string]struct{}{
"location_id": {},
"report_id": {},
"run_id": {},
"artifact_group": {},
"batch_output_name": {},
"bundle_id": {},
}
func RenderDistributorBundleID(template string, values DistributorTemplateValues) (string, error) {
return renderDistributorTemplate("notify.distributor.bundle_id_template", template, values, distributorTemplateVariables)
}
func RenderDistributorIdempotencyKey(template string, values DistributorTemplateValues) (string, error) {
return renderDistributorTemplate("notify.distributor.idempotency_key_template", template, values, distributorIdempotencyTemplateVariables)
}
func RenderDistributorReportPath(template string, values DistributorTemplateValues) (string, error) {
rendered, err := renderDistributorTemplate("notify.distributor.report_path_template", template, values, distributorTemplateVariables)
if err != nil {
return "", err
}
if err := ValidateDistributorReportPath(rendered); err != nil {
return "", err
}
return rendered, nil
}
func validateDistributorTemplate(name, template string, allowed map[string]struct{}) error {
_, err := renderDistributorTemplate(name, template, DistributorTemplateValues{}, allowed)
return err
}
func renderDistributorTemplate(name, template string, values DistributorTemplateValues, allowed map[string]struct{}) (string, error) {
var rendered strings.Builder
for i := 0; i < len(template); {
switch template[i] {
case '{':
end := strings.IndexByte(template[i+1:], '}')
if end < 0 {
return "", fmt.Errorf("%s contains an unclosed template variable", name)
}
variable := template[i+1 : i+1+end]
if variable == "" {
return "", fmt.Errorf("%s contains an empty template variable", name)
}
if _, ok := allowed[variable]; !ok {
return "", fmt.Errorf("%s contains unknown template variable %q", name, variable)
}
rendered.WriteString(distributorTemplateValue(variable, values))
i += end + 2
case '}':
return "", fmt.Errorf("%s contains an unopened template variable", name)
default:
rendered.WriteByte(template[i])
i++
}
}
return rendered.String(), nil
}
func distributorTemplateValue(variable string, values DistributorTemplateValues) string {
switch variable {
case "location_id":
return values.LocationID
case "report_id":
return values.ReportID
case "run_id":
return values.RunID
case "artifact_group":
return values.ArtifactGroup
case "batch_output_name":
return values.BatchOutputName
case "bundle_id":
return values.BundleID
default:
return ""
}
}
func ValidateDistributorReportPath(path string) error {
if path == "" {
return fmt.Errorf("notify.distributor.report_path_template renders an empty path")
}
if isDistributorAbsolutePath(path) {
return fmt.Errorf("notify.distributor.report_path_template must render a relative path")
}
if strings.Contains(path, "\\") {
return fmt.Errorf("notify.distributor.report_path_template must not render backslashes")
}
segments := strings.Split(path, "/")
for _, segment := range segments {
if segment == "" {
return fmt.Errorf("notify.distributor.report_path_template must not render empty path segments")
}
if segment == "." || segment == ".." {
return fmt.Errorf("notify.distributor.report_path_template must not render . or .. path segments")
}
if segment == "manifest.json" || segment == ".distributor.json" {
return fmt.Errorf("notify.distributor.report_path_template must not render reserved path segment %q", segment)
}
}
return nil
}
func isDistributorAbsolutePath(path string) bool {
if filepath.IsAbs(path) || strings.HasPrefix(path, "/") {
return true
}
if len(path) >= 3 && isASCIIAlpha(path[0]) && path[1] == ':' && (path[2] == '/' || path[2] == '\\') {
return true
}
return false
}
func isASCIIAlpha(ch byte) bool {
return (ch >= 'A' && ch <= 'Z') || (ch >= 'a' && ch <= 'z')
}

View File

@@ -0,0 +1,63 @@
package config
import (
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
)
var secretNamePattern = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
func loadSecrets(cfg SecretsConfig) error {
if cfg.Directory == "" {
return nil
}
entries, err := os.ReadDir(cfg.Directory)
if err != nil {
return fmt.Errorf("read secrets directory %q: %w", cfg.Directory, err)
}
for _, entry := range entries {
name := entry.Name()
if name == "" {
return fmt.Errorf("secrets directory %q contains an empty filename", cfg.Directory)
}
if !secretNamePattern.MatchString(name) {
return fmt.Errorf("secret file %q has invalid environment variable name", name)
}
if entry.Type()&os.ModeSymlink != 0 {
return fmt.Errorf("secret file %q must be a regular file, not a symlink", name)
}
if entry.IsDir() {
return fmt.Errorf("secret file %q must be a regular file, not a directory", name)
}
info, err := entry.Info()
if err != nil {
return fmt.Errorf("inspect secret file %q: %w", name, err)
}
if !info.Mode().IsRegular() {
return fmt.Errorf("secret file %q must be a regular file", name)
}
path := filepath.Join(cfg.Directory, name)
data, err := os.ReadFile(path)
if err != nil {
return fmt.Errorf("read secret file %q: %w", name, err)
}
value := string(data)
if strings.HasSuffix(value, "\r\n") {
value = strings.TrimSuffix(value, "\r\n")
} else {
value = strings.TrimSuffix(value, "\n")
}
if err := os.Setenv(name, value); err != nil {
return fmt.Errorf("set environment variable from secret file %q: %w", name, err)
}
}
return nil
}

View File

@@ -49,6 +49,10 @@ func Validate(cfg Config) error {
}
}
if err := validateDistributorNotify(cfg.Notify.Distributor); err != nil {
return err
}
if cfg.Scriptorium.Binary == "" {
return fmt.Errorf("scriptorium.binary is required")
}
@@ -75,6 +79,56 @@ func Validate(cfg Config) error {
return nil
}
func validateDistributorNotify(cfg DistributorNotifyConfig) error {
if !cfg.Enabled {
return nil
}
parsed, err := url.Parse(cfg.Endpoint)
if err != nil || parsed.Scheme == "" || parsed.Host == "" {
return fmt.Errorf("notify.distributor.endpoint must be an absolute URL when enabled")
}
if cfg.TokenEnv == "" {
return fmt.Errorf("notify.distributor.token_env is required when enabled")
}
if !secretNamePattern.MatchString(cfg.TokenEnv) {
return fmt.Errorf("notify.distributor.token_env must be a valid environment variable name")
}
if cfg.Timeout <= 0 {
return fmt.Errorf("notify.distributor.timeout must be greater than zero when enabled")
}
if cfg.FailurePolicy != NotifyFailureError {
return fmt.Errorf("notify.distributor.failure_policy must be error when enabled")
}
if cfg.BundleIDTemplate == "" {
return fmt.Errorf("notify.distributor.bundle_id_template is required when enabled")
}
if err := validateDistributorTemplate("notify.distributor.bundle_id_template", cfg.BundleIDTemplate, distributorTemplateVariables); err != nil {
return err
}
if cfg.IdempotencyKeyTemplate == "" {
return fmt.Errorf("notify.distributor.idempotency_key_template is required when enabled")
}
if err := validateDistributorTemplate("notify.distributor.idempotency_key_template", cfg.IdempotencyKeyTemplate, distributorIdempotencyTemplateVariables); err != nil {
return err
}
if cfg.ReportPathTemplate == "" {
return fmt.Errorf("notify.distributor.report_path_template is required when enabled")
}
values := DistributorTemplateValues{
LocationID: "location",
ReportID: "report",
RunID: "run",
ArtifactGroup: "artifact",
BatchOutputName: "report.md",
}
if _, err := RenderDistributorReportPath(cfg.ReportPathTemplate, values); err != nil {
return err
}
return nil
}
func validatePolicy(name string, policy MissingSourcePolicy) error {
switch policy {
case MissingSourceError, MissingSourceWarn, MissingSourceNone: