From 138bc4e7e4b8724c3b80e958e5891bc2d5c1eb06 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 7 Jun 2026 23:50:55 +0000 Subject: [PATCH] Refresh distributor documentation --- README.md | 3 +- docs/cli.md | 21 +- docs/policy/architecture.md | 8 +- docs/policy/development.md | 12 +- docs/roadmap/distributor.md | 352 +++------------------------------ docs/roadmap/future.md | 20 ++ docs/roadmap/implementation.md | 317 +++-------------------------- 7 files changed, 106 insertions(+), 627 deletions(-) diff --git a/README.md b/README.md index 723f6e5..c791359 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,8 @@ `weatherreporter` is a Go application for preparing human-facing weather reports from normalized forecast data. It builds structured briefing packages, runs them through `scriptorium`, and keeps inspectable artifacts under a local -workspace. +workspace. It can also upload successfully generated managed Markdown reports +to a configured `distributor` HTTP upload endpoint. ## Quickstart diff --git a/docs/cli.md b/docs/cli.md index f031ea5..530b731 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -11,7 +11,9 @@ weatherreporter generate daily --date 2026-05-29 --out ./daily.md This loads configuration, fetches weather data, writes managed workspace artifacts, runs `scriptorium render` as a preflight check, runs -`scriptorium run`, and writes an extra Markdown copy to `./daily.md`. +`scriptorium run`, and writes an extra Markdown copy to `./daily.md`. If +distributor notification is enabled in configuration, the command also uploads +the managed Markdown report after final metadata is saved. ## Commands @@ -33,16 +35,20 @@ weatherreporter inspect sources [--config PATH] RUN_ID ``` `generate` commands write briefing, data package, preflight, report, and -metadata artifacts under the configured workspace. `generate storm` requires -explicit event-window bounds with `--start` and `--end`. +metadata artifacts under the configured workspace. `--out` writes an extra +Markdown copy for the operator; distributor notification uses the managed +report path, not the extra copy. `generate storm` requires explicit +event-window bounds with `--start` and `--end`. `run morning` generates Daily Today and the 3-Day Outlook, plus Weekend Outlook except on Sunday. `run evening` generates the Tomorrow Planning Brief. Batch runs continue independent reports after a failure, print a JSON summary to stdout, write compact status lines to stderr, and return nonzero when any report -failed. When notification is configured, batch summaries and status lines include -notification status, accepted distributor run ID, or notification error fields -for each attempted report. +failed. `--out-dir` writes extra Markdown copies for the operator; distributor +notification uses each managed report path, not the extra copies. When +notification is enabled, batch summaries and status lines include notification +status, accepted distributor run ID, or notification error fields for each +attempted report. `inspect` commands read existing workspace artifacts and emit JSON to stdout. They do not fetch weather data or invoke `scriptorium`. @@ -63,6 +69,9 @@ They do not fetch weather data or invoke `scriptorium`. Storm times accept `YYYY-MM-DDTHH:MM` in the configured timezone or RFC3339 timestamps with explicit offsets. +Distributor notification is configured only through `notify.distributor`; there +are no distributor-specific CLI flags. + ## Common Workflows ```sh diff --git a/docs/policy/architecture.md b/docs/policy/architecture.md index 56d80f7..e0cac3f 100644 --- a/docs/policy/architecture.md +++ b/docs/policy/architecture.md @@ -13,6 +13,12 @@ Generated reports must be associated with explicit metadata, including report ty `scriptorium` is an external adapter, not domain logic. Subprocess execution must be isolated under `internal/adapters/scriptorium`, use context-aware execution, avoid shell interpolation, capture actionable stderr, and keep scriptorium-specific flags from leaking into domain packages. +`distributor` is also an external adapter. Upload behavior must be isolated +under `internal/adapters/distributor`, dependency types from the distributor +module must not leak outside that adapter, and the selected upload source must +be the managed Markdown report rather than optional output copies or broad +workspace scans. + ## Project Shape Default to a small, explicit, dependency-light Go application. Keep the design modular enough to test and change safely, but do not add abstraction unless it protects a real boundary or enables a real extension point. @@ -54,7 +60,7 @@ Configuration precedence is: Prefer YAML configuration unless the project has a strong reason to use another format. Config files should be discovered at `/usr/local/etc//config.yml`, with a CLI override via `--config`. -Configuration files should not contain raw secrets unless the application is explicitly designed for that. Prefer environment variables or secret files for secrets. +Configuration files should not contain raw secrets unless the application is explicitly designed for that. Prefer environment variables or secret files for secrets. File-backed secrets are loaded through `secrets.directory`; secret values must not be logged, persisted, or included in user-facing output. ## Adapters and External Integrations diff --git a/docs/policy/development.md b/docs/policy/development.md index 153e962..781d772 100644 --- a/docs/policy/development.md +++ b/docs/policy/development.md @@ -13,6 +13,7 @@ Developers and LLM coding agents should use it with - `internal/config`: configuration structs, defaults, loading, overrides, and validation. - `internal/fileutil`: shared atomic filesystem write and copy helpers. +- `internal/adapters/distributor`: Distributor upload adapter. - `internal/adapters/weatherapi`: Weather API HTTP adapter. - `internal/adapters/scriptorium`: Scriptorium subprocess adapter. - `internal/forecast`: normalized bundle types and deterministic forecast @@ -45,7 +46,7 @@ Useful focused checks: ```bash go test ./internal/cli ./internal/config go test ./internal/app ./internal/state -go test ./internal/adapters/weatherapi ./internal/adapters/scriptorium +go test ./internal/adapters/distributor ./internal/adapters/weatherapi ./internal/adapters/scriptorium go test ./internal/forecast ./internal/report ./internal/briefing ./internal/changes ./internal/promptinput ``` @@ -64,6 +65,8 @@ Run `gofmt -w` on changed Go files before committing. - Use atomic writes for durable JSON artifacts where practical. - Keep report selection and prompt IDs centralized in `internal/report`. - Keep Scriptorium argv construction inside `internal/adapters/scriptorium`. +- Keep distributor package types and upload-client construction inside + `internal/adapters/distributor`. - Keep Weather API transport and envelope handling inside `internal/adapters/weatherapi`. @@ -147,12 +150,14 @@ When an external contract changes, update the matching file under ## Tests -Core tests must not require live Weather API or Scriptorium services. +Core tests must not require live Weather API, Scriptorium, or distributor +services. Preferred test patterns: - fake command runners for subprocess behavior; - `httptest.Server` for Weather API behavior; +- fake distributor upload clients for notification behavior; - filesystem temp directories for state behavior; - deterministic clocks for report periods and RunIDs; - table tests for config validation, CLI parsing, period resolution, and @@ -189,7 +194,8 @@ Update: behavior; - `docs/troubleshooting.md` for recurring operator-facing failure modes; - `docs/internal/` for component contracts and invariants; -- `docs/integrations/` for external Weather API or Scriptorium contract changes; +- `docs/integrations/` for external Weather API, Scriptorium, or distributor + contract changes; - `docs/roadmap/` only for unimplemented or deferred work. Non-roadmap docs must describe implemented behavior only. diff --git a/docs/roadmap/distributor.md b/docs/roadmap/distributor.md index a912f4b..43ccfdc 100644 --- a/docs/roadmap/distributor.md +++ b/docs/roadmap/distributor.md @@ -1,336 +1,34 @@ -# Distributor Integration Roadmap +# Distributor Roadmap -This roadmap describes planned work for adding `distributor` notification -support to `weatherreporter`. The feature is not implemented yet, so this plan -lives under `docs/roadmap/`; non-roadmap documentation should not describe -`notify.distributor` as available until implementation is complete. +Current distributor notification behavior is documented outside the roadmap: -## Purpose +- [Configuration reference](../config.md) +- [Operations guide](../operations.md) +- [Troubleshooting](../troubleshooting.md) +- [Distributor adapter internals](../internal/distributor-adapter.md) -`distributor` collects outputs from producer applications and publishes them -through configured downstream pipelines. For `weatherreporter`, the planned -integration should upload generated Markdown weather reports after successful -report generation, without requiring `distributor` to scan or understand the -managed `workspace/` layout. +This file tracks future distributor-related work only. -## Current Repository Facts +## Deferred Enhancements -- `weatherreporter` does not currently implement a notify hook. -- Generated Markdown reports are written to the managed report path returned as - `ReportResult.ReportPath`. -- Optional `--out` and `--out-dir` copies are user-requested extra copies and - are not canonical integration inputs. -- The managed workspace also contains briefing snapshots, metadata, prompt data - packages, and preflight artifacts; these should not be treated as a - distributor source tree. -- The provided distributor docs recommend `pkg/upload.UploadFiles` for - producers that already have generated files, so the initial integration can - upload selected report files directly. +- Add a supported warning-only notification policy. +- Include selected non-report artifacts in uploaded bundles. +- Poll distributor run status after upload acceptance. +- Persist upload retry state across process restarts. +- Add explicit CLI controls for distributor behavior. -## Decisions Locked +## Non-Goals Without A Separate Design -- Add generic configurable secrets-directory support before the distributor - adapter work. This is project infrastructure, not a distributor-only token - helper. -- When a secrets directory is configured, each regular file directly under that - directory maps filename to environment variable name and file contents to - environment variable value. -- Secret-file values overwrite pre-existing environment variables. -- Secrets loading must never log, print, persist, or include secret values in - errors, metadata, batch output, examples, or docs. -- Add a narrow `internal/adapters/distributor` adapter for the external - integration. -- Keep `gitea.maximumdirect.net/eric/distributor/pkg/upload` and - `gitea.maximumdirect.net/eric/distributor/pkg/bundle` types inside that - adapter. -- Upload one distributor bundle per successfully generated report. -- Include only the generated Markdown report in the initial bundle. -- Use the report definition's batch output filename as the default - bundle-relative Markdown path, such as `daily.md`, `tomorrow.md`, - `three-day.md`, `weekend.md`, or `storm.md`. -- Derive the default bundle ID and idempotency key from producer name, location - ID, report ID, and RunID. -- Treat distributor upload failure as a report failure when notification is - enabled; single-report and batch commands should exit nonzero for enabled - upload failures. -- Keep destination routing, Markdown-to-HTML transformation, public URLs, and - nginx directory layout in distributor configuration, not in - `weatherreporter`. +- Do not make distributor scan the weatherreporter workspace. +- Do not move destination routing into weatherreporter. +- Do not move Markdown-to-HTML transformation into weatherreporter. +- Do not store raw bearer tokens in configuration files. -## Planned Configuration +## Required Constraints For Future Work -Add future generic secrets configuration: - -```yaml -secrets: - directory: "" -``` - -Add future distributor configuration under `notify.distributor`: - -```yaml -notify: - distributor: - enabled: false - endpoint: https://distributor.example.com - token_env: DISTRIBUTOR_UPLOAD_TOKEN - timeout: 30s - failure_policy: error - bundle_id_template: "weatherreporter.{location_id}.{report_id}.{run_id}" - idempotency_key_template: "{bundle_id}" - report_path_template: "{batch_output_name}" -``` - -Planned behavior: - -- `secrets.directory` defaults to an empty string, which disables secrets - loading. -- When `secrets.directory` is configured, load only regular files directly - under that directory. -- Each secret filename must be a valid environment variable name matching - `[A-Za-z_][A-Za-z0-9_]*`. -- Reject missing configured directory, invalid filenames, subdirectories, - symlinks, unreadable files, and empty filenames. -- Strip one trailing `\n` or `\r\n` from secret file contents for compatibility - with common mounted-secret formats. Preserve all other bytes. -- `enabled` defaults to `false`. -- `endpoint` is required only when distributor notification is enabled. -- `token_env` is required only when enabled. Raw bearer tokens should not be - stored in config files. The token value is read from normal environment state - after secrets-directory loading has been applied. -- `timeout` controls the upload call timeout and should default to `30s`. -- `failure_policy` should initially support `error`. Reserve `warn` as a - future option only if it is implemented and tested. -- Template variables should be explicit and validated: - `location_id`, `report_id`, `run_id`, `artifact_group`, and - `batch_output_name`. -- `idempotency_key_template` may reference `{bundle_id}` after the bundle ID is - rendered. -- `report_path_template` should be user-visible from the first implementation - because bundle-relative paths can affect downstream distributor routing. -- Rendered template values must produce valid distributor bundle paths: relative - slash-separated paths with no empty path, absolute path, backslash, `.`, `..`, - empty segment, `manifest.json`, or `.distributor.json`. - -## Recommended Implementation Stages - -### Stage 1: Secrets Directory Support - -Goal: add generic file-backed environment secret loading before distributor -notification depends on environment tokens. - -Implementation guidance: - -- Add a generic `secrets.directory` config field under `internal/config`. -- Default the directory to an empty string so secrets loading is disabled unless - explicitly configured. -- Load secrets after config file parsing and before validation that needs - environment-backed values. -- Map each regular file directly under the configured directory to an - environment variable: filename becomes the variable name and contents become - the value. -- Use secret-file values to overwrite pre-existing environment variables. -- Strip one trailing `\n` or `\r\n`; preserve all other bytes. -- Reject missing directories, invalid environment-variable filenames, - subdirectories, symlinks, unreadable files, and empty filenames. -- Keep secret values out of errors, logs, CLI output, metadata, and examples. - -Acceptance criteria: - -- Empty `secrets.directory` performs no environment changes. -- Configured secrets directory values are available through normal environment - lookup after config loading. -- Secret files overwrite existing environment variables with the same name. -- Invalid directory entries fail with path/name context but without secret - values. -- Existing config loading behavior remains unchanged when secrets are disabled. - -Suggested tests: - -- Config defaults leave secrets loading disabled. -- A configured missing secrets directory fails. -- Regular files set environment variables and overwrite existing values. -- Invalid filenames, subdirectories, symlinks, and unreadable files fail. -- One trailing newline or CRLF is stripped; other content is preserved. -- No secret values appear in returned errors. - -### Stage 2: Config And Validation - -Goal: add disabled-by-default distributor notification configuration. - -Implementation guidance: - -- Add notify/distributor config structs under `internal/config`. -- Add defaults for disabled notification, timeout, failure policy, bundle ID - template, idempotency key template, and report path template. -- Validate enabled config: endpoint must be an absolute URL, `token_env` must be - non-empty, timeout must be positive, failure policy must be supported, and - templates must use only known variables. -- Keep `token_env` as the configured token selector. Do not add raw token config. -- Update maintained examples and config documentation only after code support is - implemented. - -Acceptance criteria: - -- Disabled distributor notification requires no endpoint or token env. -- Enabled distributor notification rejects invalid endpoint, missing token env, - nonpositive timeout, unsupported failure policy, unknown template variables, - and invalid rendered report bundle paths. -- Existing config loading behavior and precedence remain unchanged. - -Suggested tests: - -- Config defaults include disabled distributor notification. -- Example configs load successfully. -- Enabled config validation covers required fields and invalid template cases. -- A distributor token supplied through `secrets.directory` is visible through - the configured `token_env` name after config loading. - -### Stage 3: Distributor Adapter - -Goal: isolate distributor package usage behind an internal adapter. - -Implementation guidance: - -- Add `internal/adapters/distributor` with package-owned request and result - types. -- Read the bearer token from the configured environment variable after - secrets-directory loading has populated environment state. -- Use distributor `UploadFiles` with one file: - the managed Markdown report as the source path and the rendered - `report_path_template` as the bundle path. -- Return upload result information that app orchestration can record or expose - without leaking distributor dependency types. -- Wrap upload errors with endpoint, bundle ID, and report path context while - avoiding token exposure. - -Acceptance criteria: - -- No distributor dependency types leak outside `internal/adapters/distributor`. -- Missing source report path, missing token, invalid bundle path, and upload - failure return actionable errors. -- Idempotency conflict errors are preserved or wrapped clearly enough for - troubleshooting. - -Suggested tests: - -- Adapter request construction uses the configured endpoint, token, bundle ID, - idempotency key, source report path, and bundle-relative path. -- Token values do not appear in returned errors. -- Upload conflict and generic upload failure produce useful wrapped errors. -- Tokens supplied through the secrets directory are accepted via `token_env`. - -### Stage 4: App Notify Hook - -Goal: upload successful generated reports through the configured notifier. - -Implementation guidance: - -- Add an app-owned notifier interface so app tests can use fakes. -- Construct distributor notification requests from the successful - `ReportResult`, resolved report definition, configured location, and - notification config. -- Invoke notification only after Scriptorium report generation succeeds and - final metadata has been saved. -- Do not notify after Weather API, briefing, prompt input, render preflight, or - Scriptorium run failure. -- In batch mode, notify each successful report independently. If notification - fails and distributor notification is enabled, mark that report failed and - make the aggregate batch result nonzero. - -Acceptance criteria: - -- Disabled notification is a no-op. -- Single-report generation returns an error when enabled upload fails. -- Batch generation continues independent reports, records upload failures per - report, and exits nonzero when any enabled notification fails. -- Existing report artifact paths, optional output copies, and Scriptorium - behavior remain unchanged. - -Suggested tests: - -- No notifier call occurs before successful report generation. -- Successful notifier call receives the managed report path, not `--out` or - `--out-dir` copy paths. -- Enabled notifier failure affects single-report and batch command outcomes. -- Batch still continues later reports after one notification failure. - -### Stage 5: CLI And Output Behavior - -Goal: preserve CLI syntax while surfacing notification outcome where useful. - -Implementation guidance: - -- Do not add distributor-specific CLI flags in the first implementation; use - configuration only. -- Preserve existing command names, flags, help text shape, workspace paths, and - generated Markdown outputs. -- Extend batch JSON report items only if the implementation records useful - notification status, accepted distributor run ID, or notification error. -- Keep stderr status lines concise and avoid exposing secrets. - -Acceptance criteria: - -- `go run ./cmd/weatherreporter --help` remains accurate. -- Existing generate and run command syntax remains stable. -- Batch summaries clearly indicate notification-caused report failures if - notification status is added. - -Suggested tests: - -- Existing CLI parser tests continue to pass. -- Batch JSON includes notification fields only when implemented and documented. -- Secret-like token values never appear in CLI output. - -### Stage 6: Documentation - -Goal: document implemented distributor behavior after the feature exists. - -Implementation guidance: - -- Update `docs/config.md` with `secrets.directory`, - `notify.distributor` fields, and defaults. -- Update `docs/operations.md` with notification timing, failure behavior, and - the fact that managed report paths are uploaded. -- Update `docs/troubleshooting.md` with invalid secrets directory, missing - token, upload conflict, upload rejection, and distributor unavailable cases. -- Update `docs/internal/app-orchestration.md` to include notify ordering. -- Add `docs/internal/distributor-adapter.md` for the adapter contract. -- Keep distributor API details in `docs/integrations/distributor/`; link there - instead of duplicating the full upstream contract. - -Acceptance criteria: - -- Non-roadmap docs describe only implemented notification behavior. -- Config examples include no raw tokens. -- Secret handling documentation describes mechanisms, not secret values. -- Documentation clearly distinguishes distributor destination routing from - weatherreporter upload responsibilities. - -### Stage 7: Final Validation - -Goal: verify the feature without requiring live distributor service access. - -Validation commands: - -```bash -go test ./... -go run ./cmd/weatherreporter --help -git diff --check -``` - -Additional validation: - -- Run focused config, adapter, app, and CLI tests. -- Confirm maintained example configs load. -- Search for accidental raw-token config examples. -- Confirm `go.mod` includes distributor only after implementation requires it. -- Confirm no test failures leak secret values in error output. - -## Open Questions - -No questions block this roadmap. The initial implementation should use the -locked defaults above and expose configurable templates for bundle identity and -bundle-relative report path. Secrets-directory support is generic project -infrastructure and should remain useful for future integrations beyond -distributor. +- Distributor package types stay inside `internal/adapters/distributor`. +- Weatherreporter submits explicit source bundles built from generated files. +- Optional `--out` and `--out-dir` copies remain operator conveniences, not + canonical upload sources. +- Secret values stay out of errors, logs, CLI output, metadata, examples, and + documentation. diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index 6f8ec84..cae00e2 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -49,6 +49,26 @@ These ideas are not current behavior: Each item needs its own design note before implementation. Non-roadmap docs must not describe these as available behavior. +## Deferred: Distributor Notification Enhancements + +Distributor notification currently uploads one managed Markdown report per +successful generated report through the configured HTTP upload endpoint. + +These enhancements are not current behavior: + +- `failure_policy: warn`; +- uploading metadata, briefing snapshots, data packages, or preflight artifacts; +- polling distributor status after upload acceptance; +- durable upload retry queues; +- distributor-specific CLI flags; +- making distributor scan the weatherreporter workspace; +- handling destination routing, Markdown-to-HTML transformation, public URLs, + or nginx layout inside weatherreporter. + +Any distributor enhancement should preserve the existing adapter boundary: +weatherreporter selects explicit generated files and submits source bundles, +while distributor owns destination routing and publication behavior. + ## Deferred: Cleanup Refactors The initial cleanup pass intentionally left these refactors out because the diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index d86f2c5..c22064c 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,297 +1,36 @@ -# Distributor Feature Implementation Roadmap +# Distributor Follow-Up Roadmap -This roadmap turns `docs/roadmap/distributor.md` into staged implementation -work for a future coding agent. It is a planning document only. The feature is -not implemented until the stages below are completed and the implemented docs -are updated. +Current distributor notification behavior is documented in: -## Purpose +- [Configuration reference](../config.md) +- [Operations guide](../operations.md) +- [Troubleshooting](../troubleshooting.md) +- [App orchestration internals](../internal/app-orchestration.md) +- [Distributor adapter internals](../internal/distributor-adapter.md) +- [Distributor integration contracts](../integrations/distributor/api.md) -Implement generic file-backed environment secrets and distributor notification -support while preserving current CLI syntax, managed artifact paths, RunIDs, -and report output behavior. Distributor package types must stay behind a narrow -adapter boundary, and distributor should receive explicitly selected generated -Markdown files rather than scanning `workspace/`. +This roadmap tracks only future work that is not implemented. -## Locked Decisions +## Deferred Enhancements -- Implement generic `secrets.directory` support before distributor notification. -- Secret files overwrite pre-existing environment variables. -- Never expose secret values in errors, logs, CLI output, metadata, docs, or - examples. -- Add `notify.distributor` config; do not add raw token config. -- Keep public CLI commands and flags unchanged. -- Upload one distributor bundle per successfully generated report. -- Initial distributor bundles include only the managed Markdown report at - `ReportResult.ReportPath`. -- The default bundle-relative Markdown path uses the report definition's - `BatchOutputName`. -- Default bundle ID and idempotency key derive from producer, location ID, - report ID, and RunID. -- Enabled distributor upload failure is a report failure and should produce a - nonzero command exit. +- Support a non-failing distributor notification policy such as + `failure_policy: warn`. +- Upload additional generated artifacts, such as metadata, briefing snapshots, + data packages, or preflight output. +- Poll distributor status after upload acceptance and expose downstream + publication failures. +- Add durable retry queues for upload failures. +- Add distributor-specific CLI controls if operators need per-run behavior that + configuration cannot cover cleanly. -## Stage 1: Secrets Directory Support +## Boundaries To Preserve -Goal: implement generic file-backed environment secret loading in -`internal/config`. - -Implementation guidance: - -- Add `secrets.directory` config with a built-in default of `""`, meaning - disabled. -- Load secrets after config file parsing and CLI overrides, before validation - completes. -- Read only regular files directly under the configured directory. -- Map each file basename to an environment variable name and file contents to - the value. -- Validate names with `[A-Za-z_][A-Za-z0-9_]*`. -- Reject missing configured directory, invalid filenames, subdirectories, - symlinks, unreadable files, and empty filenames. -- Strip one trailing `\n` or `\r\n`; preserve all other bytes. -- Keep error messages actionable but do not include secret values. - -Acceptance criteria: - -- Empty `secrets.directory` performs no environment changes. -- Valid secret files set environment variables after config loading. -- Secret file values overwrite existing environment variables. -- Invalid entries fail before application workflows run. -- Existing config loading behavior remains unchanged when secrets are disabled. - -Tests: - -- Defaults leave secrets loading disabled. -- Valid file sets an environment variable. -- File value overwrites an existing environment variable. -- Missing directory, invalid filename, subdirectory, symlink, and unreadable - file cases fail. -- Newline and CRLF trimming works exactly once. -- Secret values are absent from error strings. - -## Stage 2: Notify Configuration And Templates - -Goal: add disabled-by-default `notify.distributor` config and template -validation. - -Planned config: - -```yaml -notify: - distributor: - enabled: false - endpoint: https://distributor.example.com - token_env: DISTRIBUTOR_UPLOAD_TOKEN - timeout: 30s - failure_policy: error - bundle_id_template: "weatherreporter.{location_id}.{report_id}.{run_id}" - idempotency_key_template: "{bundle_id}" - report_path_template: "{batch_output_name}" -``` - -Implementation guidance: - -- Add notify/distributor config structs, defaults, YAML loading, and validation - under `internal/config`. -- When disabled, omit endpoint and token requirements. -- When enabled, validate absolute endpoint URL, non-empty `token_env`, positive - timeout, `failure_policy=error`, known template variables, and valid rendered - report bundle paths. -- Supported template variables are `location_id`, `report_id`, `run_id`, - `artifact_group`, and `batch_output_name`; `idempotency_key_template` may - also reference `{bundle_id}`. -- Do not add distributor-specific CLI flags. - -Acceptance criteria: - -- Defaults include disabled distributor notification. -- Enabled config rejects invalid endpoint, token env, timeout, failure policy, - templates, and rendered bundle paths. -- Rendered bundle paths reject absolute paths, backslashes, `.`, `..`, empty - segments, `manifest.json`, and `.distributor.json`. -- Tokens supplied by `secrets.directory` are visible through `token_env` after - config loading. - -Tests: - -- Config defaults and examples load. -- Disabled distributor config accepts omitted distributor fields. -- Enabled validation covers each invalid field. -- Template rendering and validation are covered with table tests. - -## Stage 3: Distributor Adapter - -Goal: isolate distributor upload behavior in `internal/adapters/distributor`. - -Implementation guidance: - -- Add weatherreporter-owned request and result types. -- Keep `gitea.maximumdirect.net/eric/distributor/pkg/upload` and - `gitea.maximumdirect.net/eric/distributor/pkg/bundle` types inside the - adapter. -- Read the bearer token from `token_env` after config/secrets loading. -- Use `UploadFiles` with exactly one file: source path is the managed Markdown - report path and bundle path is the rendered `report_path_template`. -- Return the accepted distributor run ID when available. -- Wrap endpoint, bundle ID, idempotency key, source path, and bundle path - context into errors without exposing token values. -- Preserve idempotency conflict diagnosis. - -Acceptance criteria: - -- No distributor dependency types leak outside the adapter. -- Missing token, source path, bundle path, or upload client inputs fail with - actionable errors. -- Upload success returns a weatherreporter-owned result. -- Upload failure errors do not include token values. - -Tests: - -- Fake upload client receives expected endpoint, token, bundle ID, idempotency - key, source path, and bundle path. -- Missing token/source path/bundle path errors are actionable. -- Token values are not present in errors. -- Upload conflict and generic failure are wrapped clearly. - -## Stage 4: App Notification Hook - -Goal: add app-level notification orchestration after successful report -generation. - -Implementation guidance: - -- Add an app-owned notifier interface for tests. -- Default to no-op notification when distributor notification is disabled. -- Notify only after successful `scriptorium run` and final metadata save. -- Use `ReportResult.ReportPath` as the distributor source file. -- Never use `--out` or `--out-dir` copies as distributor source files. -- Do not notify after Weather API, briefing, prompt input, render preflight, or - Scriptorium run failure. -- Single-report generation should return an error when enabled notification - fails. -- Batch generation should continue independent reports; notification failure - marks that report failed and makes the aggregate batch nonzero. - -Acceptance criteria: - -- Disabled notification leaves current generation behavior unchanged. -- Successful notification is recorded in the app result when useful. -- Enabled notification failure fails a single-report command. -- Batch results distinguish generation failure from notification failure enough - for operators to diagnose the failed report. -- Existing artifact paths, output copies, and Scriptorium behavior remain - unchanged. - -Tests: - -- No notifier call occurs on pre-generation failures. -- Notifier receives the managed report path. -- Enabled notifier success and failure are covered. -- Batch continues after one notification failure and reports aggregate failure. - -## Stage 5: CLI Output And Batch Reporting - -Goal: keep public CLI syntax stable while surfacing notification outcome if the -app records it. - -Implementation guidance: - -- Do not add distributor-specific CLI flags. -- Preserve current command names, help shape, generated paths, `--out`, and - `--out-dir`. -- If batch JSON is extended, use explicit fields such as - `notificationStatus`, `notificationRunId`, and `notificationError`. -- Keep stderr status lines concise and secret-free. - -Acceptance criteria: - -- `weatherreporter --help` remains accurate. -- Existing CLI parser behavior remains stable. -- Batch JSON includes notification details only when implemented and - documented. -- CLI stdout and stderr do not include token values. - -Tests: - -- Existing CLI parser tests pass, except intentional JSON additions. -- Batch output covers notification success and failure. -- Secret-like token values are absent from CLI output. - -## Stage 6: Documentation - -Goal: update implemented documentation after the feature exists. - -Implementation guidance: - -- Update `docs/config.md` with `secrets.directory`, `notify.distributor`, - defaults, and secret handling. -- Update `docs/operations.md` with notification timing, failure behavior, and - managed report upload source. -- Update `docs/troubleshooting.md` with invalid secrets directory, missing - token, upload conflict, upload rejection, and distributor unavailable cases. -- Update `docs/internal/app-orchestration.md` to include notify ordering. -- Add `docs/internal/distributor-adapter.md` for adapter inputs, outputs, - boundaries, config fields, failure behavior, tests, and invariants. -- Keep full upstream distributor contract details in - `docs/integrations/distributor/`. - -Acceptance criteria: - -- Non-roadmap docs describe only implemented behavior. -- Config examples load and contain no raw secrets. -- Secret handling docs describe mechanisms, not secret values. -- Docs distinguish weatherreporter upload responsibility from distributor - destination routing and Markdown-to-HTML transformation. - -Tests and checks: - -- Run config example loading tests. -- Search docs and examples for accidental raw tokens. -- Run `git diff --check`. - -## Stage 7: Final Validation - -Goal: verify behavior without a live distributor service. - -Validation commands: - -```bash -go test ./... -go run ./cmd/weatherreporter --help -git diff --check -go test ./internal/config ./internal/app ./internal/cli ./internal/adapters/distributor -``` - -Manual validation: - -- No token values appear in errors, CLI output, docs, examples, metadata, or - batch JSON. -- `go.mod` includes distributor only after adapter implementation requires it. -- Existing report paths, RunIDs, and `--out` / `--out-dir` behavior are - unchanged. -- Distributor upload tests use fakes or local test doubles, not a live service. - -## Deferred Work - -These are out of scope for the initial implementation: - -- `failure_policy: warn`. -- Uploading metadata, briefing, data package, or preflight artifacts. -- Distributor status polling after `202 Accepted`. -- Durable upload retry queues. -- Distributor-specific CLI flags. -- Making distributor scan `workspace/`. -- Destination routing, Markdown-to-HTML transformation, public URLs, or nginx - layout in `weatherreporter`. - -## Global Validation Checklist - -- Preserve current public CLI syntax. -- Preserve managed workspace artifact paths. +- Keep public CLI syntax stable unless a separate CLI design changes it. +- Keep managed workspace artifact paths and RunIDs stable. - Keep distributor dependency types behind `internal/adapters/distributor`. -- Keep secrets out of user-facing and persisted output. -- Keep examples valid and secret-free. -- Update implemented docs in the same change as implementation. -- Run `go test ./...`, `go run ./cmd/weatherreporter --help`, and - `git diff --check` before finishing implementation. - +- Keep raw token values out of errors, CLI output, metadata, docs, examples, + and persisted artifacts. +- Upload explicitly selected generated files; do not make distributor scan the + weatherreporter workspace. +- Leave destination routing, Markdown-to-HTML transformation, public URLs, and + nginx layout to distributor.