eric/weatherreporter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Refresh distributor documentation

Browse Source
This commit is contained in:
Eric Rakestraw
2026-06-07 23:50:55 +00:00
parent e2529cfdf2
commit 138bc4e7e4
7 changed files with 106 additions and 627 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
README.md
View File

@@ -3,7 +3,8 @@
`weatherreporter` is a Go application for preparing human-facing weather `weatherreporter` is a Go application for preparing human-facing weather
reports from normalized forecast data. It builds structured briefing packages, reports from normalized forecast data. It builds structured briefing packages,
runs them through `scriptorium`, and keeps inspectable artifacts under a local runs them through `scriptorium`, and keeps inspectable artifacts under a local
workspace. workspace. It can also upload successfully generated managed Markdown reports
to a configured `distributor` HTTP upload endpoint.
## Quickstart ## Quickstart

21
docs/cli.md
View File

@@ -11,7 +11,9 @@ weatherreporter generate daily --date 2026-05-29 --out ./daily.md
This loads configuration, fetches weather data, writes managed workspace This loads configuration, fetches weather data, writes managed workspace
artifacts, runs `scriptorium render` as a preflight check, runs artifacts, runs `scriptorium render` as a preflight check, runs
`scriptorium run`, and writes an extra Markdown copy to `./daily.md`. `scriptorium run`, and writes an extra Markdown copy to `./daily.md`. If
distributor notification is enabled in configuration, the command also uploads
the managed Markdown report after final metadata is saved.
## Commands ## Commands
@@ -33,16 +35,20 @@ weatherreporter inspect sources [--config PATH] RUN_ID
``` ```
`generate` commands write briefing, data package, preflight, report, and `generate` commands write briefing, data package, preflight, report, and
metadata artifacts under the configured workspace. `generate storm` requires metadata artifacts under the configured workspace. `--out` writes an extra
explicit event-window bounds with `--start` and `--end`. Markdown copy for the operator; distributor notification uses the managed
report path, not the extra copy. `generate storm` requires explicit
event-window bounds with `--start` and `--end`.
`run morning` generates Daily Today and the 3-Day Outlook, plus Weekend Outlook `run morning` generates Daily Today and the 3-Day Outlook, plus Weekend Outlook
except on Sunday. `run evening` generates the Tomorrow Planning Brief. Batch except on Sunday. `run evening` generates the Tomorrow Planning Brief. Batch
runs continue independent reports after a failure, print a JSON summary to runs continue independent reports after a failure, print a JSON summary to
stdout, write compact status lines to stderr, and return nonzero when any report stdout, write compact status lines to stderr, and return nonzero when any report
failed. When notification is configured, batch summaries and status lines include failed. `--out-dir` writes extra Markdown copies for the operator; distributor
notification status, accepted distributor run ID, or notification error fields notification uses each managed report path, not the extra copies. When
for each attempted report. notification is enabled, batch summaries and status lines include notification
status, accepted distributor run ID, or notification error fields for each
attempted report.
`inspect` commands read existing workspace artifacts and emit JSON to stdout. `inspect` commands read existing workspace artifacts and emit JSON to stdout.
They do not fetch weather data or invoke `scriptorium`. They do not fetch weather data or invoke `scriptorium`.
@@ -63,6 +69,9 @@ They do not fetch weather data or invoke `scriptorium`.
Storm times accept `YYYY-MM-DDTHH:MM` in the configured timezone or RFC3339 Storm times accept `YYYY-MM-DDTHH:MM` in the configured timezone or RFC3339
timestamps with explicit offsets. timestamps with explicit offsets.
Distributor notification is configured only through `notify.distributor`; there
are no distributor-specific CLI flags.
## Common Workflows ## Common Workflows
```sh ```sh

8
docs/policy/architecture.md
View File

@@ -13,6 +13,12 @@ Generated reports must be associated with explicit metadata, including report ty
`scriptorium` is an external adapter, not domain logic. Subprocess execution must be isolated under `internal/adapters/scriptorium`, use context-aware execution, avoid shell interpolation, capture actionable stderr, and keep scriptorium-specific flags from leaking into domain packages. `scriptorium` is an external adapter, not domain logic. Subprocess execution must be isolated under `internal/adapters/scriptorium`, use context-aware execution, avoid shell interpolation, capture actionable stderr, and keep scriptorium-specific flags from leaking into domain packages.
`distributor` is also an external adapter. Upload behavior must be isolated
under `internal/adapters/distributor`, dependency types from the distributor
module must not leak outside that adapter, and the selected upload source must
be the managed Markdown report rather than optional output copies or broad
workspace scans.
## Project Shape ## Project Shape
Default to a small, explicit, dependency-light Go application. Keep the design modular enough to test and change safely, but do not add abstraction unless it protects a real boundary or enables a real extension point. Default to a small, explicit, dependency-light Go application. Keep the design modular enough to test and change safely, but do not add abstraction unless it protects a real boundary or enables a real extension point.
@@ -54,7 +60,7 @@ Configuration precedence is:
Prefer YAML configuration unless the project has a strong reason to use another format. Config files should be discovered at `/usr/local/etc/<app_name>/config.yml`, with a CLI override via `--config`. Prefer YAML configuration unless the project has a strong reason to use another format. Config files should be discovered at `/usr/local/etc/<app_name>/config.yml`, with a CLI override via `--config`.
Configuration files should not contain raw secrets unless the application is explicitly designed for that. Prefer environment variables or secret files for secrets. Configuration files should not contain raw secrets unless the application is explicitly designed for that. Prefer environment variables or secret files for secrets. File-backed secrets are loaded through `secrets.directory`; secret values must not be logged, persisted, or included in user-facing output.
## Adapters and External Integrations ## Adapters and External Integrations

12
docs/policy/development.md
View File

@@ -13,6 +13,7 @@ Developers and LLM coding agents should use it with
- `internal/config`: configuration structs, defaults, loading, overrides, and - `internal/config`: configuration structs, defaults, loading, overrides, and
validation. validation.
- `internal/fileutil`: shared atomic filesystem write and copy helpers. - `internal/fileutil`: shared atomic filesystem write and copy helpers.
- `internal/adapters/distributor`: Distributor upload adapter.
- `internal/adapters/weatherapi`: Weather API HTTP adapter. - `internal/adapters/weatherapi`: Weather API HTTP adapter.
- `internal/adapters/scriptorium`: Scriptorium subprocess adapter. - `internal/adapters/scriptorium`: Scriptorium subprocess adapter.
- `internal/forecast`: normalized bundle types and deterministic forecast - `internal/forecast`: normalized bundle types and deterministic forecast
@@ -45,7 +46,7 @@ Useful focused checks:
```bash ```bash
go test ./internal/cli ./internal/config go test ./internal/cli ./internal/config
go test ./internal/app ./internal/state go test ./internal/app ./internal/state
go test ./internal/adapters/weatherapi ./internal/adapters/scriptorium go test ./internal/adapters/distributor ./internal/adapters/weatherapi ./internal/adapters/scriptorium
go test ./internal/forecast ./internal/report ./internal/briefing ./internal/changes ./internal/promptinput go test ./internal/forecast ./internal/report ./internal/briefing ./internal/changes ./internal/promptinput
``` ```
@@ -64,6 +65,8 @@ Run `gofmt -w` on changed Go files before committing.
- Use atomic writes for durable JSON artifacts where practical. - Use atomic writes for durable JSON artifacts where practical.
- Keep report selection and prompt IDs centralized in `internal/report`. - Keep report selection and prompt IDs centralized in `internal/report`.
- Keep Scriptorium argv construction inside `internal/adapters/scriptorium`. - Keep Scriptorium argv construction inside `internal/adapters/scriptorium`.
- Keep distributor package types and upload-client construction inside
`internal/adapters/distributor`.
- Keep Weather API transport and envelope handling inside - Keep Weather API transport and envelope handling inside
`internal/adapters/weatherapi`. `internal/adapters/weatherapi`.
@@ -147,12 +150,14 @@ When an external contract changes, update the matching file under
## Tests ## Tests
Core tests must not require live Weather API or Scriptorium services. Core tests must not require live Weather API, Scriptorium, or distributor
services.
Preferred test patterns: Preferred test patterns:
- fake command runners for subprocess behavior; - fake command runners for subprocess behavior;
- `httptest.Server` for Weather API behavior; - `httptest.Server` for Weather API behavior;
- fake distributor upload clients for notification behavior;
- filesystem temp directories for state behavior; - filesystem temp directories for state behavior;
- deterministic clocks for report periods and RunIDs; - deterministic clocks for report periods and RunIDs;
- table tests for config validation, CLI parsing, period resolution, and - table tests for config validation, CLI parsing, period resolution, and
@@ -189,7 +194,8 @@ Update:
behavior; behavior;
- `docs/troubleshooting.md` for recurring operator-facing failure modes; - `docs/troubleshooting.md` for recurring operator-facing failure modes;
- `docs/internal/` for component contracts and invariants; - `docs/internal/` for component contracts and invariants;
- `docs/integrations/` for external Weather API or Scriptorium contract changes; - `docs/integrations/` for external Weather API, Scriptorium, or distributor
contract changes;
- `docs/roadmap/` only for unimplemented or deferred work. - `docs/roadmap/` only for unimplemented or deferred work.
Non-roadmap docs must describe implemented behavior only. Non-roadmap docs must describe implemented behavior only.

352
docs/roadmap/distributor.md
View File

@@ -1,336 +1,34 @@
# Distributor Integration Roadmap # Distributor Roadmap
This roadmap describes planned work for adding `distributor` notification Current distributor notification behavior is documented outside the roadmap:
support to `weatherreporter`. The feature is not implemented yet, so this plan
lives under `docs/roadmap/`; non-roadmap documentation should not describe
`notify.distributor` as available until implementation is complete.
## Purpose - [Configuration reference](../config.md)
- [Operations guide](../operations.md)
- [Troubleshooting](../troubleshooting.md)
- [Distributor adapter internals](../internal/distributor-adapter.md)
`distributor` collects outputs from producer applications and publishes them This file tracks future distributor-related work only.
through configured downstream pipelines. For `weatherreporter`, the planned
integration should upload generated Markdown weather reports after successful
report generation, without requiring `distributor` to scan or understand the
managed `workspace/` layout.
## Current Repository Facts ## Deferred Enhancements
- `weatherreporter` does not currently implement a notify hook. - Add a supported warning-only notification policy.
- Generated Markdown reports are written to the managed report path returned as - Include selected non-report artifacts in uploaded bundles.
`ReportResult.ReportPath`. - Poll distributor run status after upload acceptance.
- Optional `--out` and `--out-dir` copies are user-requested extra copies and - Persist upload retry state across process restarts.
are not canonical integration inputs. - Add explicit CLI controls for distributor behavior.
- The managed workspace also contains briefing snapshots, metadata, prompt data
packages, and preflight artifacts; these should not be treated as a
distributor source tree.
- The provided distributor docs recommend `pkg/upload.UploadFiles` for
producers that already have generated files, so the initial integration can
upload selected report files directly.
## Decisions Locked ## Non-Goals Without A Separate Design
- Add generic configurable secrets-directory support before the distributor - Do not make distributor scan the weatherreporter workspace.
adapter work. This is project infrastructure, not a distributor-only token - Do not move destination routing into weatherreporter.
helper. - Do not move Markdown-to-HTML transformation into weatherreporter.
- When a secrets directory is configured, each regular file directly under that - Do not store raw bearer tokens in configuration files.
directory maps filename to environment variable name and file contents to
environment variable value.
- Secret-file values overwrite pre-existing environment variables.
- Secrets loading must never log, print, persist, or include secret values in
errors, metadata, batch output, examples, or docs.
- Add a narrow `internal/adapters/distributor` adapter for the external
integration.
- Keep `gitea.maximumdirect.net/eric/distributor/pkg/upload` and
`gitea.maximumdirect.net/eric/distributor/pkg/bundle` types inside that
adapter.
- Upload one distributor bundle per successfully generated report.
- Include only the generated Markdown report in the initial bundle.
- Use the report definition's batch output filename as the default
bundle-relative Markdown path, such as `daily.md`, `tomorrow.md`,
`three-day.md`, `weekend.md`, or `storm.md`.
- Derive the default bundle ID and idempotency key from producer name, location
ID, report ID, and RunID.
- Treat distributor upload failure as a report failure when notification is
enabled; single-report and batch commands should exit nonzero for enabled
upload failures.
- Keep destination routing, Markdown-to-HTML transformation, public URLs, and
nginx directory layout in distributor configuration, not in
`weatherreporter`.
## Planned Configuration ## Required Constraints For Future Work
Add future generic secrets configuration: - Distributor package types stay inside `internal/adapters/distributor`.
- Weatherreporter submits explicit source bundles built from generated files.
```yaml - Optional `--out` and `--out-dir` copies remain operator conveniences, not
secrets: canonical upload sources.
directory: "" - Secret values stay out of errors, logs, CLI output, metadata, examples, and
``` documentation.
Add future distributor configuration under `notify.distributor`:
```yaml
notify:
distributor:
enabled: false
endpoint: https://distributor.example.com
token_env: DISTRIBUTOR_UPLOAD_TOKEN
timeout: 30s
failure_policy: error
bundle_id_template: "weatherreporter.{location_id}.{report_id}.{run_id}"
idempotency_key_template: "{bundle_id}"
report_path_template: "{batch_output_name}"
```
Planned behavior:
- `secrets.directory` defaults to an empty string, which disables secrets
loading.
- When `secrets.directory` is configured, load only regular files directly
under that directory.
- Each secret filename must be a valid environment variable name matching
`[A-Za-z_][A-Za-z0-9_]*`.
- Reject missing configured directory, invalid filenames, subdirectories,
symlinks, unreadable files, and empty filenames.
- Strip one trailing `\n` or `\r\n` from secret file contents for compatibility
with common mounted-secret formats. Preserve all other bytes.
- `enabled` defaults to `false`.
- `endpoint` is required only when distributor notification is enabled.
- `token_env` is required only when enabled. Raw bearer tokens should not be
stored in config files. The token value is read from normal environment state
after secrets-directory loading has been applied.
- `timeout` controls the upload call timeout and should default to `30s`.
- `failure_policy` should initially support `error`. Reserve `warn` as a
future option only if it is implemented and tested.
- Template variables should be explicit and validated:
`location_id`, `report_id`, `run_id`, `artifact_group`, and
`batch_output_name`.
- `idempotency_key_template` may reference `{bundle_id}` after the bundle ID is
rendered.
- `report_path_template` should be user-visible from the first implementation
because bundle-relative paths can affect downstream distributor routing.
- Rendered template values must produce valid distributor bundle paths: relative
slash-separated paths with no empty path, absolute path, backslash, `.`, `..`,
empty segment, `manifest.json`, or `.distributor.json`.
## Recommended Implementation Stages
### Stage 1: Secrets Directory Support
Goal: add generic file-backed environment secret loading before distributor
notification depends on environment tokens.
Implementation guidance:
- Add a generic `secrets.directory` config field under `internal/config`.
- Default the directory to an empty string so secrets loading is disabled unless
explicitly configured.
- Load secrets after config file parsing and before validation that needs
environment-backed values.
- Map each regular file directly under the configured directory to an
environment variable: filename becomes the variable name and contents become
the value.
- Use secret-file values to overwrite pre-existing environment variables.
- Strip one trailing `\n` or `\r\n`; preserve all other bytes.
- Reject missing directories, invalid environment-variable filenames,
subdirectories, symlinks, unreadable files, and empty filenames.
- Keep secret values out of errors, logs, CLI output, metadata, and examples.
Acceptance criteria:
- Empty `secrets.directory` performs no environment changes.
- Configured secrets directory values are available through normal environment
lookup after config loading.
- Secret files overwrite existing environment variables with the same name.
- Invalid directory entries fail with path/name context but without secret
values.
- Existing config loading behavior remains unchanged when secrets are disabled.
Suggested tests:
- Config defaults leave secrets loading disabled.
- A configured missing secrets directory fails.
- Regular files set environment variables and overwrite existing values.
- Invalid filenames, subdirectories, symlinks, and unreadable files fail.
- One trailing newline or CRLF is stripped; other content is preserved.
- No secret values appear in returned errors.
### Stage 2: Config And Validation
Goal: add disabled-by-default distributor notification configuration.
Implementation guidance:
- Add notify/distributor config structs under `internal/config`.
- Add defaults for disabled notification, timeout, failure policy, bundle ID
template, idempotency key template, and report path template.
- Validate enabled config: endpoint must be an absolute URL, `token_env` must be
non-empty, timeout must be positive, failure policy must be supported, and
templates must use only known variables.
- Keep `token_env` as the configured token selector. Do not add raw token config.
- Update maintained examples and config documentation only after code support is
implemented.
Acceptance criteria:
- Disabled distributor notification requires no endpoint or token env.
- Enabled distributor notification rejects invalid endpoint, missing token env,
nonpositive timeout, unsupported failure policy, unknown template variables,
and invalid rendered report bundle paths.
- Existing config loading behavior and precedence remain unchanged.
Suggested tests:
- Config defaults include disabled distributor notification.
- Example configs load successfully.
- Enabled config validation covers required fields and invalid template cases.
- A distributor token supplied through `secrets.directory` is visible through
the configured `token_env` name after config loading.
### Stage 3: Distributor Adapter
Goal: isolate distributor package usage behind an internal adapter.
Implementation guidance:
- Add `internal/adapters/distributor` with package-owned request and result
types.
- Read the bearer token from the configured environment variable after
secrets-directory loading has populated environment state.
- Use distributor `UploadFiles` with one file:
the managed Markdown report as the source path and the rendered
`report_path_template` as the bundle path.
- Return upload result information that app orchestration can record or expose
without leaking distributor dependency types.
- Wrap upload errors with endpoint, bundle ID, and report path context while
avoiding token exposure.
Acceptance criteria:
- No distributor dependency types leak outside `internal/adapters/distributor`.
- Missing source report path, missing token, invalid bundle path, and upload
failure return actionable errors.
- Idempotency conflict errors are preserved or wrapped clearly enough for
troubleshooting.
Suggested tests:
- Adapter request construction uses the configured endpoint, token, bundle ID,
idempotency key, source report path, and bundle-relative path.
- Token values do not appear in returned errors.
- Upload conflict and generic upload failure produce useful wrapped errors.
- Tokens supplied through the secrets directory are accepted via `token_env`.
### Stage 4: App Notify Hook
Goal: upload successful generated reports through the configured notifier.
Implementation guidance:
- Add an app-owned notifier interface so app tests can use fakes.
- Construct distributor notification requests from the successful
`ReportResult`, resolved report definition, configured location, and
notification config.
- Invoke notification only after Scriptorium report generation succeeds and
final metadata has been saved.
- Do not notify after Weather API, briefing, prompt input, render preflight, or
Scriptorium run failure.
- In batch mode, notify each successful report independently. If notification
fails and distributor notification is enabled, mark that report failed and
make the aggregate batch result nonzero.
Acceptance criteria:
- Disabled notification is a no-op.
- Single-report generation returns an error when enabled upload fails.
- Batch generation continues independent reports, records upload failures per
report, and exits nonzero when any enabled notification fails.
- Existing report artifact paths, optional output copies, and Scriptorium
behavior remain unchanged.
Suggested tests:
- No notifier call occurs before successful report generation.
- Successful notifier call receives the managed report path, not `--out` or
`--out-dir` copy paths.
- Enabled notifier failure affects single-report and batch command outcomes.
- Batch still continues later reports after one notification failure.
### Stage 5: CLI And Output Behavior
Goal: preserve CLI syntax while surfacing notification outcome where useful.
Implementation guidance:
- Do not add distributor-specific CLI flags in the first implementation; use
configuration only.
- Preserve existing command names, flags, help text shape, workspace paths, and
generated Markdown outputs.
- Extend batch JSON report items only if the implementation records useful
notification status, accepted distributor run ID, or notification error.
- Keep stderr status lines concise and avoid exposing secrets.
Acceptance criteria:
- `go run ./cmd/weatherreporter --help` remains accurate.
- Existing generate and run command syntax remains stable.
- Batch summaries clearly indicate notification-caused report failures if
notification status is added.
Suggested tests:
- Existing CLI parser tests continue to pass.
- Batch JSON includes notification fields only when implemented and documented.
- Secret-like token values never appear in CLI output.
### Stage 6: Documentation
Goal: document implemented distributor behavior after the feature exists.
Implementation guidance:
- Update `docs/config.md` with `secrets.directory`,
`notify.distributor` fields, and defaults.
- Update `docs/operations.md` with notification timing, failure behavior, and
the fact that managed report paths are uploaded.
- Update `docs/troubleshooting.md` with invalid secrets directory, missing
token, upload conflict, upload rejection, and distributor unavailable cases.
- Update `docs/internal/app-orchestration.md` to include notify ordering.
- Add `docs/internal/distributor-adapter.md` for the adapter contract.
- Keep distributor API details in `docs/integrations/distributor/`; link there
instead of duplicating the full upstream contract.
Acceptance criteria:
- Non-roadmap docs describe only implemented notification behavior.
- Config examples include no raw tokens.
- Secret handling documentation describes mechanisms, not secret values.
- Documentation clearly distinguishes distributor destination routing from
weatherreporter upload responsibilities.
### Stage 7: Final Validation
Goal: verify the feature without requiring live distributor service access.
Validation commands:
```bash
go test ./...
go run ./cmd/weatherreporter --help
git diff --check
```
Additional validation:
- Run focused config, adapter, app, and CLI tests.
- Confirm maintained example configs load.
- Search for accidental raw-token config examples.
- Confirm `go.mod` includes distributor only after implementation requires it.
- Confirm no test failures leak secret values in error output.
## Open Questions
No questions block this roadmap. The initial implementation should use the
locked defaults above and expose configurable templates for bundle identity and
bundle-relative report path. Secrets-directory support is generic project
infrastructure and should remain useful for future integrations beyond
distributor.

20
docs/roadmap/future.md
View File

@@ -49,6 +49,26 @@ These ideas are not current behavior:
Each item needs its own design note before implementation. Non-roadmap docs Each item needs its own design note before implementation. Non-roadmap docs
must not describe these as available behavior. must not describe these as available behavior.
## Deferred: Distributor Notification Enhancements
Distributor notification currently uploads one managed Markdown report per
successful generated report through the configured HTTP upload endpoint.
These enhancements are not current behavior:
- `failure_policy: warn`;
- uploading metadata, briefing snapshots, data packages, or preflight artifacts;
- polling distributor status after upload acceptance;
- durable upload retry queues;
- distributor-specific CLI flags;
- making distributor scan the weatherreporter workspace;
- handling destination routing, Markdown-to-HTML transformation, public URLs,
or nginx layout inside weatherreporter.
Any distributor enhancement should preserve the existing adapter boundary:
weatherreporter selects explicit generated files and submits source bundles,
while distributor owns destination routing and publication behavior.
## Deferred: Cleanup Refactors ## Deferred: Cleanup Refactors
The initial cleanup pass intentionally left these refactors out because the The initial cleanup pass intentionally left these refactors out because the

317
docs/roadmap/implementation.md
View File

@@ -1,297 +1,36 @@
# Distributor Feature Implementation Roadmap # Distributor Follow-Up Roadmap
This roadmap turns `docs/roadmap/distributor.md` into staged implementation Current distributor notification behavior is documented in:
work for a future coding agent. It is a planning document only. The feature is
not implemented until the stages below are completed and the implemented docs
are updated.
## Purpose - [Configuration reference](../config.md)
- [Operations guide](../operations.md)
- [Troubleshooting](../troubleshooting.md)
- [App orchestration internals](../internal/app-orchestration.md)
- [Distributor adapter internals](../internal/distributor-adapter.md)
- [Distributor integration contracts](../integrations/distributor/api.md)
Implement generic file-backed environment secrets and distributor notification This roadmap tracks only future work that is not implemented.
support while preserving current CLI syntax, managed artifact paths, RunIDs,
and report output behavior. Distributor package types must stay behind a narrow
adapter boundary, and distributor should receive explicitly selected generated
Markdown files rather than scanning `workspace/`.
## Locked Decisions ## Deferred Enhancements
- Implement generic `secrets.directory` support before distributor notification. - Support a non-failing distributor notification policy such as
- Secret files overwrite pre-existing environment variables. `failure_policy: warn`.
- Never expose secret values in errors, logs, CLI output, metadata, docs, or - Upload additional generated artifacts, such as metadata, briefing snapshots,
examples. data packages, or preflight output.
- Add `notify.distributor` config; do not add raw token config. - Poll distributor status after upload acceptance and expose downstream
- Keep public CLI commands and flags unchanged. publication failures.
- Upload one distributor bundle per successfully generated report. - Add durable retry queues for upload failures.
- Initial distributor bundles include only the managed Markdown report at - Add distributor-specific CLI controls if operators need per-run behavior that
`ReportResult.ReportPath`. configuration cannot cover cleanly.
- The default bundle-relative Markdown path uses the report definition's
`BatchOutputName`.
- Default bundle ID and idempotency key derive from producer, location ID,
report ID, and RunID.
- Enabled distributor upload failure is a report failure and should produce a
nonzero command exit.
## Stage 1: Secrets Directory Support ## Boundaries To Preserve
Goal: implement generic file-backed environment secret loading in - Keep public CLI syntax stable unless a separate CLI design changes it.
`internal/config`. - Keep managed workspace artifact paths and RunIDs stable.
Implementation guidance:
- Add `secrets.directory` config with a built-in default of `""`, meaning
disabled.
- Load secrets after config file parsing and CLI overrides, before validation
completes.
- Read only regular files directly under the configured directory.
- Map each file basename to an environment variable name and file contents to
the value.
- Validate names with `[A-Za-z_][A-Za-z0-9_]*`.
- Reject missing configured directory, invalid filenames, subdirectories,
symlinks, unreadable files, and empty filenames.
- Strip one trailing `\n` or `\r\n`; preserve all other bytes.
- Keep error messages actionable but do not include secret values.
Acceptance criteria:
- Empty `secrets.directory` performs no environment changes.
- Valid secret files set environment variables after config loading.
- Secret file values overwrite existing environment variables.
- Invalid entries fail before application workflows run.
- Existing config loading behavior remains unchanged when secrets are disabled.
Tests:
- Defaults leave secrets loading disabled.
- Valid file sets an environment variable.
- File value overwrites an existing environment variable.
- Missing directory, invalid filename, subdirectory, symlink, and unreadable
file cases fail.
- Newline and CRLF trimming works exactly once.
- Secret values are absent from error strings.
## Stage 2: Notify Configuration And Templates
Goal: add disabled-by-default `notify.distributor` config and template
validation.
Planned config:
```yaml
notify:
distributor:
enabled: false
endpoint: https://distributor.example.com
token_env: DISTRIBUTOR_UPLOAD_TOKEN
timeout: 30s
failure_policy: error
bundle_id_template: "weatherreporter.{location_id}.{report_id}.{run_id}"
idempotency_key_template: "{bundle_id}"
report_path_template: "{batch_output_name}"
```
Implementation guidance:
- Add notify/distributor config structs, defaults, YAML loading, and validation
under `internal/config`.
- When disabled, omit endpoint and token requirements.
- When enabled, validate absolute endpoint URL, non-empty `token_env`, positive
timeout, `failure_policy=error`, known template variables, and valid rendered
report bundle paths.
- Supported template variables are `location_id`, `report_id`, `run_id`,
`artifact_group`, and `batch_output_name`; `idempotency_key_template` may
also reference `{bundle_id}`.
- Do not add distributor-specific CLI flags.
Acceptance criteria:
- Defaults include disabled distributor notification.
- Enabled config rejects invalid endpoint, token env, timeout, failure policy,
templates, and rendered bundle paths.
- Rendered bundle paths reject absolute paths, backslashes, `.`, `..`, empty
segments, `manifest.json`, and `.distributor.json`.
- Tokens supplied by `secrets.directory` are visible through `token_env` after
config loading.
Tests:
- Config defaults and examples load.
- Disabled distributor config accepts omitted distributor fields.
- Enabled validation covers each invalid field.
- Template rendering and validation are covered with table tests.
## Stage 3: Distributor Adapter
Goal: isolate distributor upload behavior in `internal/adapters/distributor`.
Implementation guidance:
- Add weatherreporter-owned request and result types.
- Keep `gitea.maximumdirect.net/eric/distributor/pkg/upload` and
`gitea.maximumdirect.net/eric/distributor/pkg/bundle` types inside the
adapter.
- Read the bearer token from `token_env` after config/secrets loading.
- Use `UploadFiles` with exactly one file: source path is the managed Markdown
report path and bundle path is the rendered `report_path_template`.
- Return the accepted distributor run ID when available.
- Wrap endpoint, bundle ID, idempotency key, source path, and bundle path
context into errors without exposing token values.
- Preserve idempotency conflict diagnosis.
Acceptance criteria:
- No distributor dependency types leak outside the adapter.
- Missing token, source path, bundle path, or upload client inputs fail with
actionable errors.
- Upload success returns a weatherreporter-owned result.
- Upload failure errors do not include token values.
Tests:
- Fake upload client receives expected endpoint, token, bundle ID, idempotency
key, source path, and bundle path.
- Missing token/source path/bundle path errors are actionable.
- Token values are not present in errors.
- Upload conflict and generic failure are wrapped clearly.
## Stage 4: App Notification Hook
Goal: add app-level notification orchestration after successful report
generation.
Implementation guidance:
- Add an app-owned notifier interface for tests.
- Default to no-op notification when distributor notification is disabled.
- Notify only after successful `scriptorium run` and final metadata save.
- Use `ReportResult.ReportPath` as the distributor source file.
- Never use `--out` or `--out-dir` copies as distributor source files.
- Do not notify after Weather API, briefing, prompt input, render preflight, or
Scriptorium run failure.
- Single-report generation should return an error when enabled notification
fails.
- Batch generation should continue independent reports; notification failure
marks that report failed and makes the aggregate batch nonzero.
Acceptance criteria:
- Disabled notification leaves current generation behavior unchanged.
- Successful notification is recorded in the app result when useful.
- Enabled notification failure fails a single-report command.
- Batch results distinguish generation failure from notification failure enough
for operators to diagnose the failed report.
- Existing artifact paths, output copies, and Scriptorium behavior remain
unchanged.
Tests:
- No notifier call occurs on pre-generation failures.
- Notifier receives the managed report path.
- Enabled notifier success and failure are covered.
- Batch continues after one notification failure and reports aggregate failure.
## Stage 5: CLI Output And Batch Reporting
Goal: keep public CLI syntax stable while surfacing notification outcome if the
app records it.
Implementation guidance:
- Do not add distributor-specific CLI flags.
- Preserve current command names, help shape, generated paths, `--out`, and
`--out-dir`.
- If batch JSON is extended, use explicit fields such as
`notificationStatus`, `notificationRunId`, and `notificationError`.
- Keep stderr status lines concise and secret-free.
Acceptance criteria:
- `weatherreporter --help` remains accurate.
- Existing CLI parser behavior remains stable.
- Batch JSON includes notification details only when implemented and
documented.
- CLI stdout and stderr do not include token values.
Tests:
- Existing CLI parser tests pass, except intentional JSON additions.
- Batch output covers notification success and failure.
- Secret-like token values are absent from CLI output.
## Stage 6: Documentation
Goal: update implemented documentation after the feature exists.
Implementation guidance:
- Update `docs/config.md` with `secrets.directory`, `notify.distributor`,
defaults, and secret handling.
- Update `docs/operations.md` with notification timing, failure behavior, and
managed report upload source.
- Update `docs/troubleshooting.md` with invalid secrets directory, missing
token, upload conflict, upload rejection, and distributor unavailable cases.
- Update `docs/internal/app-orchestration.md` to include notify ordering.
- Add `docs/internal/distributor-adapter.md` for adapter inputs, outputs,
boundaries, config fields, failure behavior, tests, and invariants.
- Keep full upstream distributor contract details in
`docs/integrations/distributor/`.
Acceptance criteria:
- Non-roadmap docs describe only implemented behavior.
- Config examples load and contain no raw secrets.
- Secret handling docs describe mechanisms, not secret values.
- Docs distinguish weatherreporter upload responsibility from distributor
destination routing and Markdown-to-HTML transformation.
Tests and checks:
- Run config example loading tests.
- Search docs and examples for accidental raw tokens.
- Run `git diff --check`.
## Stage 7: Final Validation
Goal: verify behavior without a live distributor service.
Validation commands:
```bash
go test ./...
go run ./cmd/weatherreporter --help
git diff --check
go test ./internal/config ./internal/app ./internal/cli ./internal/adapters/distributor
```
Manual validation:
- No token values appear in errors, CLI output, docs, examples, metadata, or
batch JSON.
- `go.mod` includes distributor only after adapter implementation requires it.
- Existing report paths, RunIDs, and `--out` / `--out-dir` behavior are
unchanged.
- Distributor upload tests use fakes or local test doubles, not a live service.
## Deferred Work
These are out of scope for the initial implementation:
- `failure_policy: warn`.
- Uploading metadata, briefing, data package, or preflight artifacts.
- Distributor status polling after `202 Accepted`.
- Durable upload retry queues.
- Distributor-specific CLI flags.
- Making distributor scan `workspace/`.
- Destination routing, Markdown-to-HTML transformation, public URLs, or nginx
layout in `weatherreporter`.
## Global Validation Checklist
- Preserve current public CLI syntax.
- Preserve managed workspace artifact paths.
- Keep distributor dependency types behind `internal/adapters/distributor`. - Keep distributor dependency types behind `internal/adapters/distributor`.
- Keep secrets out of user-facing and persisted output. - Keep raw token values out of errors, CLI output, metadata, docs, examples,
- Keep examples valid and secret-free. and persisted artifacts.
- Update implemented docs in the same change as implementation. - Upload explicitly selected generated files; do not make distributor scan the
- Run `go test ./...`, `go run ./cmd/weatherreporter --help`, and weatherreporter workspace.
`git diff --check` before finishing implementation. - Leave destination routing, Markdown-to-HTML transformation, public URLs, and
nginx layout to distributor.