eric/weatherreporter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Refresh distributor documentation

Browse Source
This commit is contained in:
Eric Rakestraw
2026-06-07 23:50:55 +00:00
parent e2529cfdf2
commit 138bc4e7e4
7 changed files with 106 additions and 627 deletions
Download Patch File Download Diff File Expand all files Collapse all files

317
docs/roadmap/implementation.md
View File

@@ -1,297 +1,36 @@
# Distributor Feature Implementation Roadmap
# Distributor Follow-Up Roadmap
This roadmap turns `docs/roadmap/distributor.md` into staged implementation
work for a future coding agent. It is a planning document only. The feature is
not implemented until the stages below are completed and the implemented docs
are updated.
Current distributor notification behavior is documented in:
## Purpose
- [Configuration reference](../config.md)
- [Operations guide](../operations.md)
- [Troubleshooting](../troubleshooting.md)
- [App orchestration internals](../internal/app-orchestration.md)
- [Distributor adapter internals](../internal/distributor-adapter.md)
- [Distributor integration contracts](../integrations/distributor/api.md)
Implement generic file-backed environment secrets and distributor notification
support while preserving current CLI syntax, managed artifact paths, RunIDs,
and report output behavior. Distributor package types must stay behind a narrow
adapter boundary, and distributor should receive explicitly selected generated
Markdown files rather than scanning `workspace/`.
This roadmap tracks only future work that is not implemented.
## Locked Decisions
## Deferred Enhancements
- Implement generic `secrets.directory` support before distributor notification.
- Secret files overwrite pre-existing environment variables.
- Never expose secret values in errors, logs, CLI output, metadata, docs, or
examples.
- Add `notify.distributor` config; do not add raw token config.
- Keep public CLI commands and flags unchanged.
- Upload one distributor bundle per successfully generated report.
- Initial distributor bundles include only the managed Markdown report at
`ReportResult.ReportPath`.
- The default bundle-relative Markdown path uses the report definition's
`BatchOutputName`.
- Default bundle ID and idempotency key derive from producer, location ID,
report ID, and RunID.
- Enabled distributor upload failure is a report failure and should produce a
nonzero command exit.
- Support a non-failing distributor notification policy such as
`failure_policy: warn`.
- Upload additional generated artifacts, such as metadata, briefing snapshots,
data packages, or preflight output.
- Poll distributor status after upload acceptance and expose downstream
publication failures.
- Add durable retry queues for upload failures.
- Add distributor-specific CLI controls if operators need per-run behavior that
configuration cannot cover cleanly.
## Stage 1: Secrets Directory Support
## Boundaries To Preserve
Goal: implement generic file-backed environment secret loading in
`internal/config`.
Implementation guidance:
- Add `secrets.directory` config with a built-in default of `""`, meaning
disabled.
- Load secrets after config file parsing and CLI overrides, before validation
completes.
- Read only regular files directly under the configured directory.
- Map each file basename to an environment variable name and file contents to
the value.
- Validate names with `[A-Za-z_][A-Za-z0-9_]*`.
- Reject missing configured directory, invalid filenames, subdirectories,
symlinks, unreadable files, and empty filenames.
- Strip one trailing `\n` or `\r\n`; preserve all other bytes.
- Keep error messages actionable but do not include secret values.
Acceptance criteria:
- Empty `secrets.directory` performs no environment changes.
- Valid secret files set environment variables after config loading.
- Secret file values overwrite existing environment variables.
- Invalid entries fail before application workflows run.
- Existing config loading behavior remains unchanged when secrets are disabled.
Tests:
- Defaults leave secrets loading disabled.
- Valid file sets an environment variable.
- File value overwrites an existing environment variable.
- Missing directory, invalid filename, subdirectory, symlink, and unreadable
file cases fail.
- Newline and CRLF trimming works exactly once.
- Secret values are absent from error strings.
## Stage 2: Notify Configuration And Templates
Goal: add disabled-by-default `notify.distributor` config and template
validation.
Planned config:
```yaml
notify:
distributor:
enabled: false
endpoint: https://distributor.example.com
token_env: DISTRIBUTOR_UPLOAD_TOKEN
timeout: 30s
failure_policy: error
bundle_id_template: "weatherreporter.{location_id}.{report_id}.{run_id}"
idempotency_key_template: "{bundle_id}"
report_path_template: "{batch_output_name}"
```
Implementation guidance:
- Add notify/distributor config structs, defaults, YAML loading, and validation
under `internal/config`.
- When disabled, omit endpoint and token requirements.
- When enabled, validate absolute endpoint URL, non-empty `token_env`, positive
timeout, `failure_policy=error`, known template variables, and valid rendered
report bundle paths.
- Supported template variables are `location_id`, `report_id`, `run_id`,
`artifact_group`, and `batch_output_name`; `idempotency_key_template` may
also reference `{bundle_id}`.
- Do not add distributor-specific CLI flags.
Acceptance criteria:
- Defaults include disabled distributor notification.
- Enabled config rejects invalid endpoint, token env, timeout, failure policy,
templates, and rendered bundle paths.
- Rendered bundle paths reject absolute paths, backslashes, `.`, `..`, empty
segments, `manifest.json`, and `.distributor.json`.
- Tokens supplied by `secrets.directory` are visible through `token_env` after
config loading.
Tests:
- Config defaults and examples load.
- Disabled distributor config accepts omitted distributor fields.
- Enabled validation covers each invalid field.
- Template rendering and validation are covered with table tests.
## Stage 3: Distributor Adapter
Goal: isolate distributor upload behavior in `internal/adapters/distributor`.
Implementation guidance:
- Add weatherreporter-owned request and result types.
- Keep `gitea.maximumdirect.net/eric/distributor/pkg/upload` and
`gitea.maximumdirect.net/eric/distributor/pkg/bundle` types inside the
adapter.
- Read the bearer token from `token_env` after config/secrets loading.
- Use `UploadFiles` with exactly one file: source path is the managed Markdown
report path and bundle path is the rendered `report_path_template`.
- Return the accepted distributor run ID when available.
- Wrap endpoint, bundle ID, idempotency key, source path, and bundle path
context into errors without exposing token values.
- Preserve idempotency conflict diagnosis.
Acceptance criteria:
- No distributor dependency types leak outside the adapter.
- Missing token, source path, bundle path, or upload client inputs fail with
actionable errors.
- Upload success returns a weatherreporter-owned result.
- Upload failure errors do not include token values.
Tests:
- Fake upload client receives expected endpoint, token, bundle ID, idempotency
key, source path, and bundle path.
- Missing token/source path/bundle path errors are actionable.
- Token values are not present in errors.
- Upload conflict and generic failure are wrapped clearly.
## Stage 4: App Notification Hook
Goal: add app-level notification orchestration after successful report
generation.
Implementation guidance:
- Add an app-owned notifier interface for tests.
- Default to no-op notification when distributor notification is disabled.
- Notify only after successful `scriptorium run` and final metadata save.
- Use `ReportResult.ReportPath` as the distributor source file.
- Never use `--out` or `--out-dir` copies as distributor source files.
- Do not notify after Weather API, briefing, prompt input, render preflight, or
Scriptorium run failure.
- Single-report generation should return an error when enabled notification
fails.
- Batch generation should continue independent reports; notification failure
marks that report failed and makes the aggregate batch nonzero.
Acceptance criteria:
- Disabled notification leaves current generation behavior unchanged.
- Successful notification is recorded in the app result when useful.
- Enabled notification failure fails a single-report command.
- Batch results distinguish generation failure from notification failure enough
for operators to diagnose the failed report.
- Existing artifact paths, output copies, and Scriptorium behavior remain
unchanged.
Tests:
- No notifier call occurs on pre-generation failures.
- Notifier receives the managed report path.
- Enabled notifier success and failure are covered.
- Batch continues after one notification failure and reports aggregate failure.
## Stage 5: CLI Output And Batch Reporting
Goal: keep public CLI syntax stable while surfacing notification outcome if the
app records it.
Implementation guidance:
- Do not add distributor-specific CLI flags.
- Preserve current command names, help shape, generated paths, `--out`, and
`--out-dir`.
- If batch JSON is extended, use explicit fields such as
`notificationStatus`, `notificationRunId`, and `notificationError`.
- Keep stderr status lines concise and secret-free.
Acceptance criteria:
- `weatherreporter --help` remains accurate.
- Existing CLI parser behavior remains stable.
- Batch JSON includes notification details only when implemented and
documented.
- CLI stdout and stderr do not include token values.
Tests:
- Existing CLI parser tests pass, except intentional JSON additions.
- Batch output covers notification success and failure.
- Secret-like token values are absent from CLI output.
## Stage 6: Documentation
Goal: update implemented documentation after the feature exists.
Implementation guidance:
- Update `docs/config.md` with `secrets.directory`, `notify.distributor`,
defaults, and secret handling.
- Update `docs/operations.md` with notification timing, failure behavior, and
managed report upload source.
- Update `docs/troubleshooting.md` with invalid secrets directory, missing
token, upload conflict, upload rejection, and distributor unavailable cases.
- Update `docs/internal/app-orchestration.md` to include notify ordering.
- Add `docs/internal/distributor-adapter.md` for adapter inputs, outputs,
boundaries, config fields, failure behavior, tests, and invariants.
- Keep full upstream distributor contract details in
`docs/integrations/distributor/`.
Acceptance criteria:
- Non-roadmap docs describe only implemented behavior.
- Config examples load and contain no raw secrets.
- Secret handling docs describe mechanisms, not secret values.
- Docs distinguish weatherreporter upload responsibility from distributor
destination routing and Markdown-to-HTML transformation.
Tests and checks:
- Run config example loading tests.
- Search docs and examples for accidental raw tokens.
- Run `git diff --check`.
## Stage 7: Final Validation
Goal: verify behavior without a live distributor service.
Validation commands:
```bash
go test ./...
go run ./cmd/weatherreporter --help
git diff --check
go test ./internal/config ./internal/app ./internal/cli ./internal/adapters/distributor
```
Manual validation:
- No token values appear in errors, CLI output, docs, examples, metadata, or
batch JSON.
- `go.mod` includes distributor only after adapter implementation requires it.
- Existing report paths, RunIDs, and `--out` / `--out-dir` behavior are
unchanged.
- Distributor upload tests use fakes or local test doubles, not a live service.
## Deferred Work
These are out of scope for the initial implementation:
- `failure_policy: warn`.
- Uploading metadata, briefing, data package, or preflight artifacts.
- Distributor status polling after `202 Accepted`.
- Durable upload retry queues.
- Distributor-specific CLI flags.
- Making distributor scan `workspace/`.
- Destination routing, Markdown-to-HTML transformation, public URLs, or nginx
layout in `weatherreporter`.
## Global Validation Checklist
- Preserve current public CLI syntax.
- Preserve managed workspace artifact paths.
- Keep public CLI syntax stable unless a separate CLI design changes it.
- Keep managed workspace artifact paths and RunIDs stable.
- Keep distributor dependency types behind `internal/adapters/distributor`.
- Keep secrets out of user-facing and persisted output.
- Keep examples valid and secret-free.
- Update implemented docs in the same change as implementation.
- Run `go test ./...`, `go run ./cmd/weatherreporter --help`, and
`git diff --check` before finishing implementation.
- Keep raw token values out of errors, CLI output, metadata, docs, examples,
and persisted artifacts.
- Upload explicitly selected generated files; do not make distributor scan the
weatherreporter workspace.
- Leave destination routing, Markdown-to-HTML transformation, public URLs, and
nginx layout to distributor.