diff --git a/AGENTS.md b/AGENTS.md index ec69996..f91383c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,4 +1 @@ -Please carefully review the documents in `docs/policy` before making any changes to this repository. - - `architecture.md` provides the canonical high-level architecture policy for this repository. - - `development.md` provides more granular development policy for this repository. - - `documentation.md` provides the canonical documentation policy for this repository. \ No newline at end of file +Please review `docs/development.md` for initial orientation in this repository and follow its task-specific reading guide. diff --git a/README.md b/README.md index 954d332..ce7763d 100644 --- a/README.md +++ b/README.md @@ -19,4 +19,4 @@ weatherreporter generate today --out ./today.md - [Operations guide](docs/operations.md) - [Troubleshooting](docs/troubleshooting.md) - [Architecture policy](docs/policy/architecture.md) -- [Development policy](docs/policy/development.md) +- [Development guide](docs/development.md) diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000..c478bda --- /dev/null +++ b/docs/development.md @@ -0,0 +1,86 @@ +# Development + +This is the first-read guide for people and coding agents working on +Weatherreporter. It provides a concise repository orientation and routes each +kind of change to its canonical documentation. + +Weatherreporter is a Go CLI that collects normalized weather data, derives +deterministic report facts and module snapshots, invokes Scriptorium for +generated text, renders managed Markdown reports, and can upload completed +reports through Distributor. Start with the [README](../README.md) for product +context and the [architecture policy](policy/architecture.md) for system +boundaries and invariants. + +## What To Read + +| When working on | Read | Why | +| --- | --- | --- | +| Product behavior or the shortest useful workflow | [README](../README.md), [CLI reference](cli.md), and [operations guide](operations.md) | These own product orientation, invocation, and normal operation. | +| Application shape, package boundaries, dependency direction, safety properties, or architectural invariants | [Architecture policy](policy/architecture.md) and relevant ADRs under `docs/adr/`, when present | Architecture defines the intended system; ADRs preserve significant decision rationale. | +| Any documentation addition, revision, move, or removal | [Documentation policy](policy/documentation.md) | It defines canonical owners, audience boundaries, current-state rules, and document lifecycle. | +| Adding, changing, reviewing, or deleting tests | [Testing policy](policy/testing.md) and focused package tests | The policy defines risk-based sufficiency, durable test boundaries, doubles, and test-maintenance criteria. | +| CLI commands, flags, output, quiet mode, or command wiring | [CLI reference](cli.md) and [CLI internals](internal/cli.md) | The reference owns the user contract; the internal guide owns command composition and output flow. | +| Configuration fields, defaults, loading, overrides, validation, or secrets | [Configuration reference](config.md), [architecture policy](policy/architecture.md), and tests under `internal/config` | These separate the user-visible contract, architectural rules, and executable behavior. | +| Top-level generation, batch, collection, inspection, or notification workflow | [App orchestration internals](internal/app-orchestration.md) | It owns workflow ordering, persistence points, failure propagation, and orchestration invariants. | +| Weather API transport, source envelopes, source warnings, or collection | [Weather API integration](integrations/weatherapi.md), [weather-data internals](internal/weather-data.md), and [collection internals](internal/collect.md) | These separate the external contract, normalized source facts, and app-facing collection behavior. | +| Forecast periods, weather derivation, collected facts, or derived facts | [Forecast derivation internals](internal/forecast-derivation.md) and [fact contracts](internal/facts.md) | They own deterministic derivation and the fact boundaries used by reports. | +| Report definitions, valid periods, report IDs, output naming, or batch composition | [Report registry internals](internal/report-registry.md) and [app orchestration internals](internal/app-orchestration.md) | Report definitions own selection and period rules; orchestration owns execution. | +| Module IDs, module composition, briefing values, or prompt-facing exports | [Module contract internals](internal/module.md), [module builder internals](internal/briefing.md), and [prompt-input internals](internal/prompt-input.md) | These own module contracts, value construction, and the curated prompt-package boundary. | +| Recent Changes comparison | [Changes internals](internal/changes.md) and [operations guide](operations.md) | The internal guide owns structured comparison; operations owns user-visible artifact behavior. | +| Scriptorium commands, subprocess execution, prompt inputs, or result handling | [Scriptorium integration](integrations/scriptorium.md), [Scriptorium adapter internals](internal/scriptorium-adapter.md), and [prompt-input internals](internal/prompt-input.md) | These separate the external CLI contract, subprocess boundary, and input construction. | +| Generated-text schemas, validation, render contexts, templates, or Markdown rendering | [Generated-text internals](internal/generatedtext.md), [report-template internals](internal/reporttemplate.md), and [report template guide](templates.md) | These own structured text, renderer implementation, and the maintainer-facing template surface. | +| Workspace paths, metadata, atomic persistence, lookup, inspection, or recovery | [State internals](internal/state.md), [operations guide](operations.md), and [troubleshooting guide](troubleshooting.md) | These separate implementation, operator workflows, and symptom-based recovery. | +| Distributor bundles, uploads, notification artifacts, or failures | [Distributor adapter internals](internal/distributor-adapter.md), [Distributor integration contracts](integrations/distributor/), and [operations guide](operations.md) | These separate adapter behavior, external contracts, and operational lifecycle. | +| Maintained example configuration | [Configuration reference](config.md) and files under `examples/` | The reference owns field meaning; examples own complete copyable files. | +| Proposed, deferred, or unimplemented work | Documents under `docs/roadmap/` | Future behavior and implementation status belong only in roadmaps until implemented. | + +For an existing subsystem, inspect its focused internal document, package-local +types, and tests before changing behavior. Use the package boundaries already +present before introducing a new package or abstraction. + +## Repository Map + +| Area | Responsibility | +| --- | --- | +| `cmd/weatherreporter` | Binary entry point. | +| `internal/cli` | Command parsing, flags, help, output, and command wiring. | +| `internal/app` | Generation, batches, collection coordination, notification, and inspection orchestration. | +| `internal/config` | Configuration defaults, loading, precedence, secrets, and validation. | +| `internal/adapters` | Weather API, Scriptorium, and Distributor boundaries. | +| `internal/weatherdata`, `internal/forecast`, `internal/facts` | Normalized source facts and deterministic derivation. | +| `internal/report`, `internal/module`, `internal/briefing`, `internal/changes` | Report registry, module contracts and values, and structured comparison. | +| `internal/promptinput`, `internal/generatedtext`, `internal/reporttemplate` | Prompt packages, generated-text validation, render contexts, and Markdown templates. | +| `internal/state`, `internal/fileutil`, `internal/timeutil` | Durable artifacts, atomic file operations, clocks, dates, timezones, and periods. | +| `docs` | User, operator, integration, internal, policy, and roadmap documentation. | +| `examples` | Maintained copyable configuration. | + +The [architecture policy](policy/architecture.md) is authoritative for +normative boundaries. Focused documents under `docs/internal/` own detailed +implemented subsystem behavior. + +## Contributor Workflow + +1. Read the documents and focused tests identified by the task guide. +2. Use focused package checks while iterating. +3. Run `gofmt -w` on changed Go files. +4. Update the canonical documentation and maintained examples in the same + change when behavior changes. +5. Run repository-wide validation before considering the work complete. + +Preserve actionable error context, keep secrets out of logs and fixtures, and +avoid validation that requires live Weather API, Scriptorium, or Distributor +services. The architecture and testing policies own the detailed rules. + +## Baseline Validation + +Run: + +```sh +go test ./... +go run ./cmd/weatherreporter --help +git diff --check +``` + +Use focused package tests during development and add broader or race-enabled +checks when required by the [testing policy](policy/testing.md) and the risks of +the change. diff --git a/docs/integrations/distributor/pkg-bundle.md b/docs/integrations/distributor/pkg-bundle.md index 835df2f..7791b87 100644 --- a/docs/integrations/distributor/pkg-bundle.md +++ b/docs/integrations/distributor/pkg-bundle.md @@ -10,7 +10,8 @@ import "gitea.maximumdirect.net/eric/distributor/pkg/bundle" `pkg/bundle` builds, writes, parses, and validates local source bundles. Use it directly when a producer writes bundles for `distributor` to discover, or when a producer wants to assemble and validate a bundle before using another transport. -The canonical source bundle file-format contract is [Source Bundle Contract](../integrations/source-bundle.md). +The canonical source bundle file-format contract is +`docs/integrations/source-bundle.md` in the Distributor repository. ## Preferred Complete-Bundle Workflow diff --git a/docs/integrations/distributor/pkg-upload.md b/docs/integrations/distributor/pkg-upload.md index 6ca073e..907b672 100644 --- a/docs/integrations/distributor/pkg-upload.md +++ b/docs/integrations/distributor/pkg-upload.md @@ -16,7 +16,8 @@ import "gitea.maximumdirect.net/eric/distributor/pkg/upload" import "gitea.maximumdirect.net/eric/distributor/pkg/bundle" ``` -The canonical HTTP wire contract is [HTTP Upload API Contract](../integrations/http-upload.md). +The canonical HTTP wire contract is `docs/integrations/http-upload.md` in the +Distributor repository. ## Client Construction diff --git a/docs/policy/architecture.md b/docs/policy/architecture.md index a317d2c..b890431 100644 --- a/docs/policy/architecture.md +++ b/docs/policy/architecture.md @@ -1,125 +1,212 @@ -# Architecture +# Architecture Policy -This document defines the development principles for this Go project. It is inward-facing: developers and LLM coding agents should use it to preserve the project’s shape, boundaries, and invariants as the code evolves. +## Purpose -## weatherreporter -`weatherreporter` is a deterministic weather briefing and report-preparation application. It consumes normalized weather data from the internal weatherfeeder-backed API, derives report-specific module snapshots and prompt packages, compares module snapshots against prior runs, and invokes an external prompt runner to produce human-facing reports. +This policy defines Weatherreporter's system shape, normative ownership, +dependency direction, architectural invariants, safety properties, and +non-goals. Developers and coding agents should use it to preserve the +application's boundaries as the implementation evolves. -The application should keep meteorological data selection, daypart grouping, threshold detection, forecast-period resolution, and recent-change comparison inside Go domain packages. LLM prompts should receive curated module-based prompt packages rather than raw unbounded source payloads wherever practical. +The [development guide](../development.md) owns the current package inventory +and contributor workflow. Focused documents under `docs/internal/` own +implemented subsystem mechanics. This policy owns the rules those packages and +mechanics must preserve. -Report types must be defined through a registry or equivalent mechanism. Each report definition should declare its report ID, prompt ID, valid-period resolver, module composition, comparison strategy, and output naming behavior. Avoid scattering report-type conditionals across CLI and orchestration code. +## System Shape -Generated reports must be associated with explicit metadata, including report type, location, generation time, valid period, source product timestamps or hashes, module snapshot path, and output path. Recent Changes must be based on structured snapshot comparison rather than comparison of rendered Markdown report text. +Weatherreporter is a deterministic weather briefing and report-preparation CLI. +It consumes normalized weather data, derives report facts and module snapshots, +builds curated prompt packages, compares structured snapshots with prior runs, +invokes Scriptorium for bounded generated text, renders managed Markdown +reports, persists inspectable artifacts, and can upload completed reports +through Distributor. -`scriptorium` is an external adapter, not domain logic. Subprocess execution must be isolated under `internal/adapters/scriptorium`, use context-aware execution, avoid shell interpolation, capture actionable stderr, and keep scriptorium-specific flags from leaking into domain packages. +The application is intentionally a small, explicit, dependency-light Go +program. Add abstraction only when it protects a real boundary, makes an +important invariant testable, or supports an implemented extension point. -`distributor` is also an external adapter. Upload behavior must be isolated -under `internal/adapters/distributor`, dependency types from the distributor -module must not leak outside that adapter, and the selected upload source must -be the managed Markdown report rather than optional output copies or broad -workspace scans. +The primary flow is: -## Project Shape +1. CLI parsing and configuration resolution; +2. report or batch resolution; +3. normalized weather collection; +4. deterministic fact derivation and module construction; +5. structured prior-snapshot comparison; +6. curated prompt input and generated-text processing; +7. managed Markdown rendering and metadata persistence; and +8. optional notification using managed report artifacts. -Default to a small, explicit, dependency-light Go application. Keep the design modular enough to test and change safely, but do not add abstraction unless it protects a real boundary or enables a real extension point. +Inspection is a separate read-only flow over persisted state. It must not +collect weather data, invoke Scriptorium, or upload reports. -Business/domain logic should live outside CLI, transport, and external-adapter packages. +## Ownership And Dependency Direction + +### Entry Point And CLI + +The binary entry point should do no business work beyond constructing and +running the CLI. CLI code owns commands, arguments, flags, help, output +formatting, and conversion into application requests. + +CLI packages must not own meteorological decisions, report composition, +artifact layout, Recent Changes comparison, external transport, or subprocess +construction. + +### Configuration + +Configuration loading, built-in defaults, overrides, secret loading, and +validation belong to `internal/config`. Operational values shared across +packages must be explicit configuration or constants owned by the responsible +package, not hidden in CLI or adapter code. + +The exact configuration contract belongs in the +[configuration reference](../config.md). Other architecture documents should +state ownership and safety rules rather than repeat fields, defaults, or +precedence. + +### Application Orchestration + +`internal/app` owns top-level use cases and workflow order. It composes report +resolution, collection, domain transformations, state, rendering, and optional +notification through narrow project-owned contracts. + +The application layer may coordinate components and convert between their +contracts. It must not absorb CLI parsing, HTTP transport, subprocess argument +construction, filesystem layout, weather derivation algorithms, template +execution, or adapter-specific dependency types. + +### Domain And Report Logic + +Meteorological selection, forecast-period resolution, daypart grouping, +threshold detection, fact derivation, report composition, module construction, +generated-text validation, and Recent Changes comparison belong in deterministic +Go domain packages. + +Domain packages must not depend on CLI parsing, process execution, remote +transport, or concrete external-library types. Given the same normalized +inputs, configuration, valid period, prior snapshot, and clock, domain behavior +should be reproducible. + +Report selection must go through the report registry or an equivalent +centralized mechanism. A report definition owns its identity, prompt and +rendering mode, valid-period resolver, module composition, comparison strategy, +artifact grouping, and output naming. Do not scatter report-ID conditionals +through CLI, orchestration, or adapters. + +### External Adapters + +External integrations use adapter boundaries under `internal/adapters`. +Adapters own transport and protocol mechanics; application and domain packages +own decisions. + +- The Weather API adapter owns HTTP request construction, timeouts, retries, + response-envelope handling, decoding, and endpoint compatibility. +- The Scriptorium adapter owns argument construction, context-aware subprocess + execution, stdout and stderr capture, exit interpretation, and result + decoding. It must avoid shell interpolation. +- The Distributor adapter owns dependency-specific bundle and upload types, + client construction, request execution, status handling, and redaction. + +External dependency types must not leak beyond the adapter that integrates +them. Adapters should expose narrow project-owned inputs and outputs so an +integration can be tested or replaced without changing domain logic. + +### State And Embedded Assets + +`internal/state` owns managed workspace paths, durable metadata, atomic +artifact persistence, prior lookup, and inspection reads. Other packages should +request state operations rather than reconstruct managed paths independently. + +Schemas, prompts, Markdown templates, and partials should live as separate +repository assets and be embedded by the package that owns their execution or +lookup. Keep weather derivation and path construction out of templates. + +## Architectural Invariants + +### Weather Truth And Generated Text + +- Normalized source data and deterministic Go derivation are authoritative for + weather facts. +- LLM prompts receive curated module-based packages rather than raw, + unbounded source payloads. +- Generated text is limited to defined prose slots, validated before use, and + rendered through typed or otherwise explicit contexts. +- Templates arrange validated prose and deterministic facts; they do not + perform meteorological derivation. + +### Reports And Comparison + +- Report behavior is resolved through centralized definitions. +- Recent Changes is computed from structured module snapshots, never by + comparing rendered Markdown. +- Batch workflows collect normalized weather data once and reuse that + collection for planning and report generation. +- Report metadata links identity, generation time, valid period, source + provenance, and the managed artifacts produced for the run. + +### Managed State And Notification + +- Durable structured writes are atomic where practical. +- Managed paths remain beneath the configured workspace root. +- Operations that delete, move, overwrite, or copy files use narrow, explicit + paths; destructive cleanup is opt-in. +- Intermediate artifacts reached before a later failure remain inspectable + where practical. +- Distributor uploads use managed Markdown reports, never optional output + copies or broad workspace scans. +- Notification occurs only after the managed report and required metadata have + been successfully produced. + +### Security, Errors, And Cancellation + +- Secrets must not appear in logs, errors, persisted artifacts, examples, or + user-facing output. +- Errors preserve actionable operation, report, RunID, path, endpoint, or + subprocess context without exposing secrets or unnecessarily large payloads. +- External calls, subprocesses, storage operations, and multi-step workflows + accept or propagate `context.Context` where cancellation or timeout is + meaningful. +- Adapter failures preserve useful status, stderr, or response context at the + boundary and are translated into project-owned errors before crossing into + unrelated packages. ## Dependency Policy -Prefer the Go standard library where practical. +Prefer the Go standard library. Add an external dependency only when it +materially improves correctness, security, interoperability, or +maintainability. A dependency used for a small convenience does not justify its +lifetime upgrade and compatibility cost. -Use external dependencies only when justified by correctness, security, interoperability, or substantial complexity reduction. Good reasons include complex security-sensitive behavior, such as HTML sanitization, or widely used de facto standards, such as YAML parsing. +Keep dependency-specific types inside the package that intentionally adopts +the dependency. The application should remain understandable and testable +without requiring framework-wide abstractions or live external services. -Avoid dependencies for small conveniences. Do not let external dependency types leak across internal package boundaries unless the dependency is itself the explicit public contract of that package. +## Verification And Documentation -## Package Layout +Core behavior must be testable without live Weather API, Scriptorium, or +Distributor services. The [testing policy](testing.md) owns test philosophy, +sufficiency, boundaries, and test-double guidance. -Use this layout unless the project has a documented reason to differ: +Documentation must follow the +[documentation policy](documentation.md). Update the canonical user, +operator, integration, internal, and example documentation in the same change +as the behavior it describes. Future or proposed behavior belongs under +`docs/roadmap/`; significant durable decisions may be recorded as ADRs. -- `internal/app`: application orchestration and top-level use cases. -- `internal/cli`: CLI command definitions, flags, argument parsing, and command wiring. -- `internal/config`: configuration structs, defaults, loading, precedence, and validation. -- `internal/adapters/`: adapters for external CLIs, APIs, databases, object stores, or libraries. -- `internal/api`: HTTP API handlers and request/response types, when the application exposes an HTTP API. -- `internal/transport/http`: HTTP client code, when the application calls HTTP services. +## Non-Goals -Package-private implementation constants may live near the package that owns them, preferably in `constants.go` when useful. +Weatherreporter is not: -## Configuration +- a source weather-data ingestion or normalization service; +- a general-purpose LLM orchestration framework; +- an application in which an LLM selects authoritative weather facts or report + policy; +- a plugin framework with dynamically discovered report or module behavior; +- an HTTP service or multi-user distributed job system; +- a replacement for Scriptorium or Distributor protocol ownership; or +- a system that hides operational state exclusively inside opaque logs or + remote services. -Centralize configuration loading, processing, precedence, defaults, and validation in `internal/config`. - -The goal is to make configuration discoverable and avoid implicit or hidden operational values. User-visible defaults and cross-package operational defaults should be defined in `internal/config/defaults.go`. - -Configuration precedence is: - -1. CLI flags -2. configuration file -3. built-in defaults - -Prefer YAML configuration unless the project has a strong reason to use another format. Config files should be discovered at `/usr/local/etc//config.yml`, with a CLI override via `--config`. - -Configuration files should not contain raw secrets unless the application is explicitly designed for that. Prefer environment variables or secret files for secrets. File-backed secrets are loaded through `secrets.directory`; secret values must not be logged, persisted, or included in user-facing output. - -## Adapters and External Integrations - -Use a hexagonal architecture style for external integrations. - -External adapters belong under `internal/adapters/`. If an adapter uses an external dependency, that dependency’s interface must not leak outside the adapter package. Other packages should interact only with the adapter’s API, so the dependency can be swapped, upgraded, or removed without touching unrelated code. - -Adapters should be thin. Domain decisions belong in application/domain packages, not inside adapter glue. - -## Components and Registries - -When the application has major workflow components, each component should live -near the package that owns its contract and have explicit inputs and outputs. - -The orchestrator should compose components in an explicit order using a default -sequence, dependency graph, or documented orchestration rule. - -If users can select components, validators, renderers, or adapters, selection -should go through a registry or equivalent mechanism rather than scattered -conditionals. - -## Embedded Assets - -Store embedded JSON schemas, Markdown prompts, templates, and similar assets as separate files, not inline string literals, unless there is a strong reason otherwise. - -## Errors and Logging - -Errors should be actionable and preserve context. Wrap errors with operation and path/resource context. CLI code should convert internal errors into concise user-facing messages. - -Errors and logs must not expose secrets. - -Use structured logging where practical. Logs should describe operations, paths, external calls, retries, and failure causes, but should not include large user data by default. - -## Context, Timeouts, and Cancellation - -Long-running operations should accept `context.Context`. External calls, -subprocesses, HTTP requests, storage operations, and multi-step workflows should -respect cancellation and timeouts. - -## State, Files, and Safety - -If the application writes durable state, writes should be atomic where -practical. Multi-step workflows should preserve enough state to support -inspection and retry diagnosis after failure. - -Code that deletes, moves, or overwrites files must use narrow, explicit paths. Avoid broad parent-directory operations. Cleanup that can cause data loss must be opt-in. - -## Testing - -Core logic should be testable without real external services. Use fakes, fixtures, or local test doubles for adapters where practical. - -Config examples should be load-tested. Important CLI workflows should have -parser or command tests. Component contracts should have focused tests that do -not require running the full application unless end-to-end coverage is -intentional. - -## Documentation - -Documentation should follow the project documentation policy. Keep user docs focused on implemented behavior. Put future, planned, or aspirational work only under `docs/roadmap/`. - -When changing architecture, config, CLI behavior, adapters, or component -contracts, update the relevant docs and examples in the same change. +New requirements may justify revisiting a non-goal. A change that alters system +shape, dependency direction, a safety property, or another architectural +invariant should be recorded deliberately in this policy or an ADR rather than +introduced implicitly. diff --git a/docs/policy/development.md b/docs/policy/development.md deleted file mode 100644 index a7d9370..0000000 --- a/docs/policy/development.md +++ /dev/null @@ -1,208 +0,0 @@ -# Development Policy - -This document is the contributor workflow policy for `weatherreporter`. -Developers and LLM coding agents should use it with -`docs/policy/architecture.md` and `docs/policy/documentation.md`. - -## Repository Layout - -- `cmd/weatherreporter`: binary entry point. -- `internal/app`: orchestration for generation, batches, fetch helpers, and - inspection. -- `internal/cli`: command parsing, flag handling, help text, and JSON output. -- `internal/config`: configuration structs, defaults, loading, overrides, and - validation. -- `internal/fileutil`: shared atomic filesystem write and copy helpers. -- `internal/adapters/distributor`: Distributor upload adapter. -- `internal/adapters/weatherapi`: Weather API HTTP adapter. -- `internal/adapters/scriptorium`: Scriptorium subprocess adapter. -- `internal/weatherdata`: normalized weather source facts, source metadata, and - source warnings. -- `internal/forecast`: deterministic forecast derivation. -- `internal/facts`: collected and derived report fact contracts. -- `internal/module`: module IDs, config items, output envelopes, and snapshots. -- `internal/report`: report definitions, valid periods, batches, output names, - and comparison declarations. -- `internal/briefing`: prompt-facing module value builders and module registry. -- `internal/changes`: structured Recent Changes comparison. -- `internal/promptinput`: Scriptorium `data_package` construction and - validation. -- `internal/state`: filesystem paths, atomic JSON writes, metadata, lookup, and - inspection support. -- `internal/timeutil`: clock, date, timezone, and period helpers. -- `docs`: user, operator, developer, integration, internal, policy, and roadmap - documentation. -- `examples`: maintained copyable examples. - -## Local Validation - -Use focused checks while editing and broader checks before committing: - -```bash -go test ./... -go run ./cmd/weatherreporter --help -git diff --check -``` - -Useful focused checks: - -```bash -go test ./internal/cli ./internal/config -go test ./internal/app ./internal/state -go test ./internal/adapters/distributor ./internal/adapters/weatherapi ./internal/adapters/scriptorium -go test ./internal/forecast ./internal/report ./internal/briefing ./internal/changes ./internal/promptinput -``` - -Run `gofmt -w` on changed Go files before committing. - -## Coding Conventions - -- Keep domain logic out of `cmd`, `internal/cli`, and adapter packages. -- Prefer small explicit structs and functions over broad framework-style - abstractions. -- Keep package APIs narrow and named around implemented behavior. -- Return errors with operation, path, endpoint, report, or RunID context. -- Do not log or expose secrets. -- Use `context.Context` for external calls, subprocesses, and orchestrated - workflows that may be canceled. -- Use atomic writes for durable JSON artifacts where practical. -- Keep report selection and prompt IDs centralized in `internal/report`. -- Keep Scriptorium argv construction inside `internal/adapters/scriptorium`. -- Keep distributor package types and upload-client construction inside - `internal/adapters/distributor`. -- Keep Weather API transport and envelope handling inside - `internal/adapters/weatherapi`. - -## Dependency Policy - -Prefer the Go standard library. Add dependencies only when they materially -improve correctness, interoperability, security, or maintainability. - -Current external dependencies: - -- `gitea.maximumdirect.net/eric/distributor` for distributor source bundle - construction and HTTP upload client behavior. -- `gopkg.in/yaml.v3` for YAML configuration parsing. - -When adding a dependency: - -- explain why the standard library is not enough; -- keep dependency types from leaking across unrelated package boundaries; -- add tests for the behavior the dependency supports; -- update this policy if the dependency becomes part of contributor workflow. - -## Configuration Changes - -Configuration is owned by `internal/config`. - -When adding or changing a field: - -- update `Config` and the nested config struct in `config.go`; -- add or adjust defaults in `defaults.go` when the field has a safe default; -- update loading or CLI override behavior in `load.go` only when needed; -- validate required values and accepted ranges in `validate.go`; -- add or update config tests; -- update `docs/config.md` and maintained examples when the field is user - visible; -- keep secrets out of example config files. - -Configuration precedence is: - -1. CLI overrides supported by `config.LoadOptions`; -2. configuration file values; -3. built-in defaults. - -The default config path is `/usr/local/etc/weatherreporter/config.yml`. - -## CLI Changes - -The CLI is owned by `internal/cli`. - -When adding or changing a command or flag: - -- update help text and parser behavior together; -- declare whether the command is an action command or an inspection/data-output - command; -- convert parsed values into app-layer request structs; -- keep domain decisions in `internal/app` or domain packages; -- use the centralized output helpers in `internal/cli/output.go`; -- keep action-command summary conversion in `internal/cli/result.go`; -- add parser or command tests in `internal/cli`; -- update `docs/cli.md`; -- update `docs/operations.md` or `docs/troubleshooting.md` when behavior affects - operators. - -CLI commands should return concise actionable errors and avoid printing partial -JSON when command construction fails. - -## Components And Adapters - -Use existing package boundaries before adding a package. - -Add a new internal component only when it owns a distinct implemented contract. -Define its inputs, outputs, state behavior, failure behavior, tests, and -invariants in `docs/internal/`. - -Adapters should stay thin: - -- HTTP adapters own transport, request construction, envelope handling, and - decode boundaries. -- subprocess adapters own argv construction, timeout handling, stdout/stderr - capture, and exit-code interpretation. -- adapter packages should not own report selection, forecast summarization, - Recent Changes, or prompt input schema decisions. - -When an external contract changes, update the matching file under -`docs/integrations/`. - -## Tests - -Core tests must not require live Weather API, Scriptorium, or distributor -services. - -Preferred test patterns: - -- fake command runners for subprocess behavior; -- `httptest.Server` for Weather API behavior; -- fake distributor upload clients for notification behavior; -- filesystem temp directories for state behavior; -- deterministic clocks for report periods and RunIDs; -- table tests for config validation, CLI parsing, period resolution, and - threshold behavior. - -Add focused tests near the package that owns the behavior. Use app-level tests -for workflow ordering, persistence, and cross-package contracts. - -## Examples - -Examples under `examples/` must be real, maintained, and free of secrets. - -When updating examples: - -- use implemented config fields only; -- avoid private endpoints and credentials; -- keep comments short and operationally useful; -- add or update validation coverage when a new example file is introduced; -- link maintained examples from `docs/config.md`. - -Do not add generated report examples unless they can be kept current without -live external services. - -## Documentation Checklist - -Documentation updates are part of behavior changes. - -Update: - -- `README.md` for project orientation or quickstart changes; -- `docs/cli.md` for command and flag changes; -- `docs/config.md` for config fields, defaults, and precedence changes; -- `docs/operations.md` for state, artifact, batch, inspection, and recovery - behavior; -- `docs/troubleshooting.md` for recurring operator-facing failure modes; -- `docs/internal/` for component contracts and invariants; -- `docs/integrations/` for external Weather API, Scriptorium, or distributor - contract changes; -- `docs/roadmap/` only for unimplemented or deferred work. - -Non-roadmap docs must describe implemented behavior only. diff --git a/docs/policy/documentation.md b/docs/policy/documentation.md index d9ed7fa..809ff58 100644 --- a/docs/policy/documentation.md +++ b/docs/policy/documentation.md @@ -1,356 +1,207 @@ -# Go Project Documentation Policy +# Documentation Policy ## Purpose -Project documentation must help four audiences: - -1. users who need to run the application; -2. administrators/operators who need to configure and operate it; -3. developers who need to understand and change it safely; -4. LLM coding agents that need clear scope, boundaries, and invariants. - -Docs should be accurate, concise, task-oriented, and organized by audience. Prefer links to canonical docs over repetition. +This policy assigns each Weatherreporter documentation topic to one canonical +owner. Its goal is to keep documentation accurate, concise, discoverable, and +resistant to drift for users, operators, developers, integrators, maintainers, +and coding agents. ## Core Rules -### 1. Keep docs concise - -Each document should cover a defined scope and only the essentials for that scope. - -Avoid: -- long background explanations; -- repeated reference material; -- implementation detail in user-facing docs; -- aspirational language outside roadmap docs; -- verbose examples where one minimal example is clearer. - -### 2. Document only implemented behavior outside roadmap files - -Unimplemented, planned, aspirational, experimental, or future work may be described only under: - -- `docs/roadmap/` - -No other documentation file, including `README.md`, should describe code, features, modules, stages, commands, config fields, or behaviors that do not currently exist. - -If a feature is partial, non-roadmap docs may describe only the implemented portion and its current boundary. - -### 3. Use canonical homes - -Each type of information should have one canonical location. - -Canonical homes: - -- project purpose and quickstart: `README.md` -- development principles: `docs/policy/architecture.md` -- configuration reference: `docs/config.md` -- CLI reference: `docs/cli.md` -- operations and recovery: `docs/operations.md` -- troubleshooting: `docs/troubleshooting.md` -- implemented internals: `docs/internal/` -- future work: `docs/roadmap/` -- contributor workflow: `docs/policy/development.md` -- copyable examples: `examples/` - -Other files should summarize briefly and link to the canonical source. - -### 4. Keep examples real - -Examples should be valid, maintained, and free of secrets. - -Where practical: -- example configs should load successfully; -- example commands should match real CLI syntax; -- important examples should be covered by tests. - -## Documentation Profiles - -All projects require: - -- `README.md` -- `docs/policy/architecture.md` - -Additional docs depend on the project. - -### Small library - -Recommended: -- `docs/policy/development.md`, if contributor conventions are non-obvious - -### Simple CLI - -Required: -- `docs/cli.md` - -Recommended: -- `docs/policy/development.md` - -### Config-driven CLI - -Required: -- `docs/cli.md` -- `docs/config.md` - -Recommended: -- `examples/` -- `docs/policy/development.md` - -### Stateful or operator-facing application - -Required: -- `docs/cli.md`, if CLI-based -- `docs/config.md`, if config-driven -- `docs/operations.md` - -Recommended: -- `docs/troubleshooting.md` -- `examples/` -- `docs/policy/development.md` - -### Modular, staged, service-oriented, or orchestration application - -Required: -- `docs/cli.md`, if CLI-based -- `docs/config.md`, if config-driven -- `docs/operations.md` -- `docs/internal/` -- `docs/policy/development.md` - -Recommended: -- `docs/troubleshooting.md` -- validated examples under `examples/` - -## Required Documents - -### README.md - -**Audience:** users, administrators, operators - -The README is the outward-facing project orientation page. - -It should include, in order: - -1. concise description; -2. elevator pitch; -3. shortest useful command or usage example; -4. links to targeted docs. - -The README should be short. It is not a manual. - -The “shortest useful command” means the simplest command that performs the project’s core use case. (It does not mean `app --help`.) - -### docs/policy/architecture.md - -**Audience:** developers, LLM coding agents - -`docs/policy/architecture.md` is required for every project. - -It is an inward-facing development policy document. It should describe how the project is intended to be built and changed. - -It should include: - -- project shape; -- core design principles; -- package and boundary philosophy; -- state/persistence philosophy, if applicable; -- external integration philosophy, if applicable; -- error-handling and logging principles; -- testing expectations; -- documentation expectations; -- architectural invariants; -- explicit non-goals, if useful. - -For small projects, this file may be brief. It may simply state that the project is intentionally narrow, monolithic, and dependency-light. - -### docs/policy/development.md - -**Audience:** developers, LLM coding agents - -Required for projects maintained by humans and LLM coding agents. - -It should include: - -- repository layout; -- build/test commands; -- coding conventions; -- dependency policy; -- how to add config fields; -- how to add CLI flags; -- how to add stages/modules/adapters, if applicable; -- how to update examples; -- documentation update expectations. - -### docs/config.md - -**Audience:** administrators, operators, advanced users - -Required for applications with configuration files. - -It should include, in order: - -1. config file locations and discovery precedence; -2. minimal working config; -3. production-oriented config; -4. full configuration reference; -5. secrets handling, if applicable; -6. links to maintained examples. - -The full configuration reference should be canonical. - -### docs/cli.md - -**Audience:** users, administrators, operators - -Required for CLI applications. - -It should include, in order: - -1. shortest useful command; -2. command overview; -3. complete flag reference; -4. common workflows; -5. diagnostic or recovery commands, if applicable. - -Explain when commands are useful, not just their syntax. - -### docs/operations.md - -**Audience:** administrators, operators - -Required for applications that maintain state, support resume behavior, run multiple stages, write durable artifacts, use remote storage, or require recovery procedures. - -It should cover: - -- normal workflow; -- filesystem layout; -- remote storage layout, if applicable; -- logs and manifests; -- resume/retry behavior; -- cleanup behavior; -- archive/backup behavior; -- safe recovery procedures; -- operational caveats. - -### docs/troubleshooting.md - -**Audience:** administrators, operators - -Recommended once recurring failure modes exist. - -Each entry should include: - -- symptom; -- likely cause; -- diagnostic command or inspection step; -- safe fix; -- relevant links. - -### docs/internal/ - -**Audience:** developers, LLM coding agents - -Required for modular, staged, service-oriented, or orchestration projects. - -This directory describes implemented internal components. It is not the roadmap. - -Use one file per major component where useful. - -Each component doc should include: - -1. purpose; -2. inputs and outputs; -3. boundaries; -4. config fields used; -5. external adapters used; -6. state or manifest behavior, if applicable; -7. skip/resume behavior, if applicable; -8. failure behavior; -9. tests to inspect before changing; -10. architectural invariants. - -### docs/roadmap/ - -**Audience:** maintainers, developers, LLM coding agents - -This is the only place for planned, future, aspirational, experimental, or unimplemented work. - -Roadmap docs should clearly distinguish: - -- proposed work; -- accepted plans; -- deferred ideas; -- rejected ideas; -- implementation prompts or task breakdowns, if useful. - -Roadmap docs should not be confused with current behavior. - -### docs/integrations/ - -**Audience:** developers, LLM coding agents - -Required for projects that depend on external CLIs, APIs, services, protocols, or file formats where the integration contract is important to maintain. - -This directory contains concise, versioned reference notes for external integration contracts. It should document only the parts of the external system that this project actually uses. - -Use one file per integration where useful. - -## Examples Directory - -Projects with non-trivial configuration or workflows should include `examples/`. - -Useful examples include: - -- minimal working config; -- production-oriented config; -- full annotated config; -- local development config; -- remote/object-storage config; -- minimal session/input file. - -Examples should be valid, maintained, tested when practical, and linked from relevant docs. - -## Security and Privacy - -Docs and examples must not include: - -- real API keys; -- tokens; -- passwords; -- private keys; -- private environment dumps; -- sensitive user data; -- raw private transcripts; -- private infrastructure details unless intentionally public. - -Document secret-handling mechanisms, not actual secret values. - -## Maintenance Rules - -When docs change, verify the affected behavior. - -Where practical: - -- load example config files in tests; -- test CLI examples or command parser behavior; -- validate documented flags against real flags; -- remove stale references; -- update links after renames; -- keep roadmap content out of non-roadmap docs. - -If documentation and code disagree, fix the documentation and/or open a roadmap item; do not leave aspirational behavior in current-behavior docs. - -Documentation is complete only when it matches the current code. - -## Documentation Change Checklist - -Before merging documentation changes, verify: - -- README is concise and orientation-focused. -- `docs/policy/architecture.md` describes development principles. -- Future work appears only under `docs/roadmap/`. -- User-facing docs avoid unnecessary internals. -- Developer-facing docs preserve boundaries and invariants. -- Config examples match the schema. -- CLI examples match real commands and flags. -- Defaults appear in the canonical config reference. -- No secrets or private data are included. -- Links are accurate. +### One Canonical Documentation Owner + +Each authoritative fact belongs in one canonical document or documentation +area. A non-owning document may give a short, stable summary for orientation, +but it must link to the canonical owner instead of maintaining a second +definition. + +Volatile details include commands, flags, configuration fields and defaults, +report and module IDs, schemas, file names, paths, status and exit behavior, +retry behavior, and runtime guarantees. If readers could reasonably treat a +statement as a contract, its exact documentation belongs with the owner named +in this policy. + +Executable sources of truth and documentation owners serve different purposes. +Code, schemas, and embedded assets determine runtime behavior. The canonical +document owns the corresponding explanation or reference for readers. Both may +necessarily express the same contract, but other documentation should summarize +and link rather than create another complete reference. When implementation and +documentation disagree, verify the intended behavior and update them together. + +### Current State, Decisions, And Future Work + +Outside `docs/roadmap/`, documentation describes implemented behavior only. +Partial features may be described only to their implemented boundary. + +An accepted architecture decision may describe an approved direction before it +is implemented, but acceptance is not evidence that the behavior exists. +Current-state documents change when the implementation lands. Temporary +roadmaps own future work, sequencing, and implementation status; they do not +replace durable policies, decisions, or current contracts. + +### Audience And Detail + +Write for the document's stated audience and include only the detail needed for +its owned topic. User and operator documentation should not expose incidental +implementation detail. Developer documentation should link to user-facing and +external contracts instead of restating them. + +### Links + +Use descriptive link text and repository-relative links for repository +documents. Link to the canonical owner rather than to a duplicate summary. +Check every added or changed link, and repair or remove links when their target +moves or is retired. + +### Examples And Code Fences + +Complete copyable files belong in `examples/` when maintained examples exist. +Documentation may use the smallest illustrative snippet needed for its owned +topic, but should link to a maintained example instead of embedding a second +complete copy. + +Examples must be valid, secret-free, and tested where practical. Commands, +flags, configuration, imports, and Go snippets must match implemented behavior. +Use a language tag on fenced code blocks, and identify fragments that are +illustrative rather than directly runnable. + +### Security And Privacy + +Documentation and examples must not contain real credentials, private keys, +private environment dumps, sensitive source material, or private +infrastructure details unless intentionally public. Document secret-handling +mechanisms, not secret values. + +## Canonical Ownership + +| Topic | Canonical owner | Owned content | Content owned elsewhere | +| --- | --- | --- | --- | +| Product orientation and minimal quickstart | `README.md` | What Weatherreporter is, why it is useful, one shortest successful invocation, and links onward. | Complete command reference, configuration reference, operational procedures, architecture, and implementation detail. | +| Contributor workflow and package inventory | `docs/development.md` | Repository layout, local workflow, validation commands, coding conventions, task-specific change guidance, dependency workflow, and repository hygiene. | Architectural invariants, user-facing contracts, detailed subsystem behavior, and future work. | +| Current application architecture | `docs/policy/architecture.md` | System shape, normative ownership, dependency direction, package boundaries, invariants, safety properties, and non-goals. | Concrete implementation mechanics, contributor procedures, decision history, and future work. | +| Documentation organization | `docs/policy/documentation.md` | Documentation ownership, audience boundaries, maintenance rules, and document lifecycle. | Application architecture and runtime behavior. | +| Testing policy | `docs/policy/testing.md` | Test philosophy, risk-based sufficiency, stable test boundaries, doubles, coverage guidance, regression policy, and criteria for adding, rewriting, or deleting tests. | Subsystem behavior, application contracts, subsystem-specific test inventories, and implementation plans. | +| CLI contract | `docs/cli.md` | Commands, arguments, flags, invocation semantics, stdout and stderr behavior, summaries, and exit behavior. | Configuration field definitions, complete operating procedures, runtime filesystem layout, and command implementation. | +| Configuration contract | `docs/config.md` | Discovery and precedence, fields, defaults, secrets, validation rules, and user-selectable values. | Complete example files, CLI syntax, runtime state lifecycle, and loading implementation. | +| Operations | `docs/operations.md` | Normal workflows, physical workspace layout, artifacts and metadata, inspection, notification behavior, recovery, cleanup, permissions, and operational caveats. | Complete CLI syntax, configuration field definitions, logical external contracts, and implementation mechanics. | +| Troubleshooting | `docs/troubleshooting.md` | Recurring symptoms, likely causes, diagnostic steps, safe fixes, and links to normal-operation references. | Complete command and configuration references, routine operating procedures, and implementation detail. | +| Report template surface | `docs/templates.md` | Implemented template files and partials, render-context fields, editing rules, and maintainer-facing template examples. | Weather derivation, module implementation, generated-text validation internals, and operator procedures. | +| External and durable integration contracts | `docs/integrations/` | Weather API, Scriptorium, Distributor, external formats and protocols, durable logical paths and schemas, compatibility behavior, and upstream or downstream responsibilities. | Physical runtime placement and lifecycle, internal transformations, CLI syntax, and configuration defaults. | +| Internal subsystem behavior | `docs/internal/` | Implementation flow, internal collaborators and state transitions, package-local guarantees and failures, and relevant tests. | Global architecture invariants, user-facing contracts, external schemas, operator procedures, and future package plans. | +| Architectural decision history | `docs/adr/`, when repository-local decisions require records | Significant decisions, context, alternatives, rationale, consequences, and supersession history. | Current behavior reference, implementation status, and task sequencing. | +| Temporary feature roadmaps | `docs/roadmap/`, while planned work needs coordination | Proposed, accepted, deferred, or rejected work; sequencing; gates; implementation status; and task breakdowns. | Implemented behavior reference and durable decision rationale. | +| Complete copyable artifacts | `examples/` | Maintained configuration and other files intended to be copied or run. | Field-by-field reference, command reference, and prose explanation. | + +Conditional owners do not require placeholder files or directories. If +Weatherreporter introduces a new public API, consumer interface, release +process, or other durable documentation responsibility, update this policy to +assign its canonical owner when that responsibility is introduced. + +## Boundary Rules + +### Orientation, Architecture, And Internals + +The README owns product orientation. The development policy routes contributors +and owns the concise current package inventory. Architecture owns normative +structure and invariants. Focused internal documents own implementation +behavior. These documents may link to one another but must not maintain +parallel package or behavior references. + +### Commands, Configuration, Operations, And Troubleshooting + +CLI documentation answers how to invoke Weatherreporter and what its command +interface does. Configuration documentation answers what settings mean. +Operations answers what happens to runtime state and how to operate or recover +the application. Troubleshooting starts from a symptom and leads to diagnosis +and a safe fix. + +When a workflow crosses these topics, place the complete procedure with the +document that owns the task and link to the other contracts. Do not duplicate +complete flag, field, or path references to make a workflow self-contained. + +### Templates, Integrations, And Implementation + +Template documentation defines the maintainer-facing rendering surface. +Integration documentation defines externally observable shapes, logical paths, +protocols, and compatibility behavior. Internal documentation explains how +Weatherreporter produces, transforms, or consumes those contracts. + +Internal documents may name a command, field, template value, path, or protocol +to identify a dependency, but must link to its canonical documentation for the +complete definition. + +### Executable Authority + +CLI parsing and help generation are the executable authority for accepted +commands and flags. Configuration structs, defaults, loading, and validation +are the executable authority for configuration behavior. Schemas and embedded +assets are the executable authority for validated formats and template +execution. Tests protect selected contracts and invariants but do not become a +second documentation reference merely by asserting them. + +Canonical documentation must be checked against these authorities whenever the +corresponding behavior changes. + +### Security Topics + +This policy owns what documentation and examples may contain. Architecture owns +application security boundaries and invariants. Configuration owns +credential-supply mechanisms. Operations owns permissions and handling of +sensitive runtime artifacts. Integration documents own consumer-visible +security contracts. Internal documents own implementation mechanisms only. + +## Architecture Decision Records + +Use sequentially numbered ADR filenames such as +`0001-record-architecture-decisions.md`. Follow the lightweight Nygard format: + +1. title; +2. status; +3. date; +4. context; +5. decision; +6. alternatives considered; +7. consequences. + +Use one of these statuses: + +- **Proposed:** the decision is under consideration and may change; +- **Accepted:** the decision is approved, whether or not implementation is + complete; +- **Rejected:** the proposed decision was considered and not adopted; +- **Superseded:** a later accepted ADR replaces the accepted decision. + +A proposed ADR transitions to Accepted or Rejected. An Accepted ADR transitions +to Superseded only when a later Accepted ADR replaces it. An ADR may be created +as Accepted when the decision has already been made. + +Treat the decision content of an Accepted ADR as immutable. A changed decision +requires a later ADR rather than a rewrite of the accepted record. A Superseded +ADR must link to its replacement, and the replacement must link back. Rejected +architectural alternatives belong in the ADR; rejected feature ideas belong in +a roadmap when they need to be retained. + +## Document Lifecycle + +Create durable current-state documentation with the implementation it +describes. Update its canonical owner in the same change when behavior changes. +If ownership moves, remove the old definition and leave a link where navigation +remains useful. + +Roadmaps are temporary coordination documents. When their work is complete, +record completion, move any still-useful decisions or contracts to their +durable owners, update incoming links, and archive or remove the roadmap +according to repository practice. Do not preserve completed roadmaps as a +second current-state reference. + +Before completing documentation work: + +- verify affected behavior and examples; +- check commands, flags, fields, defaults, schemas, paths, and identifiers + against their implementation; +- keep unimplemented behavior in a roadmap, subject to the ADR exception; +- validate links and fenced examples; +- confirm non-owning documents summarize and link rather than redefine; +- remove stale or unsupported claims; and +- confirm that no secrets or sensitive private data were added. diff --git a/docs/policy/testing.md b/docs/policy/testing.md new file mode 100644 index 0000000..11bdd7d --- /dev/null +++ b/docs/policy/testing.md @@ -0,0 +1,337 @@ +# Testing Policy + +## Purpose + +Our tests exist to make **incorrect changes expensive and correct changes +cheap**. + +We do not optimize for test count, line coverage, exhaustive isolation, or the +fewest possible tests. We optimize for sufficient confidence in important +behavior while imposing as little unnecessary friction as possible on future +development. + +## Every Test Has A Cost + +Every test has an immediate cost and a continuing lifetime cost. It must be +written, reviewed, executed, understood, diagnosed when it fails, updated when +legitimate behavior changes, and maintained as fixtures and dependencies +evolve. + +Tests also create cognitive and architectural friction. They can constrain +refactoring, duplicate policy, slow feedback, add noise to failures, and cause +harmless implementation changes to require unrelated suite edits. + +A test is warranted when the confidence it provides justifies those costs. +Apply that judgment at two levels: + +1. **Per test:** What realistic defect does this test detect, how consequential + would it be, and is that protection worth the test's lifetime cost? +2. **Across the suite:** Does this collection provide materially more + confidence than a smaller, simpler suite would? + +Prefer a lean suite that provides sufficient confidence in the risks that +matter without redundant or low-value tests. Some friction is intentional: +tests should make dangerous changes, such as corrupting state, breaking +compatibility, violating security boundaries, or reintroducing subtle defects, +require deliberate review. They should not make ordinary internal changes +needlessly expensive. + +Maintenance cost is not a reason to omit testing by default. When omitting a +plausible test, be able to explain why the protected failure is low-risk, +already covered, obvious, reversible, or cheaper to detect elsewhere. Favor +testing when failure would be consequential, subtle, or difficult to observe. + +## Default Testing Style + +Use a classical or Detroit-style approach: + +- Test observable behavior, resulting state, contracts, and invariants. +- Use real internal collaborators when they are fast and deterministic. +- Use fakes, stubs, or mocks primarily at expensive, nondeterministic, + destructive, or external boundaries. +- Prefer package-level behavioral tests over tests coupled to private helpers + or internal call sequences. +- Test exact collaborator interactions only when the interaction itself is a + requirement. + +Weatherreporter's important seams include clocks, subprocesses, HTTP services, +Distributor uploads, filesystem roots, environment-backed secrets, and any +future source of randomness or nondeterminism. + +## Execution Requirements + +The [development guide](../development.md) owns baseline repository validation. +The default test suite is: + +```sh +go test ./... +``` + +Run race-enabled tests when a change affects concurrent execution, goroutine +lifecycle, shared mutable state, or cancellation coordination. Use a focused +package command while iterating and `go test -race ./...` when the risk crosses +package boundaries. + +Tests in the default suite must be deterministic, offline, and independent of +real credentials. They must not invoke live Weather API, Scriptorium, or +Distributor services or depend on other mutable external infrastructure. +Tests that require live infrastructure must be explicitly opt-in and clearly +separated from the default suite. + +Control clocks, environment variables, filesystem roots, and machine-specific +state when they affect behavior. Tests must be safe to repeat and must not +depend on execution order or state left by an earlier test. Tests that modify +process-global state may remain serial; use `t.Parallel()` only when the test +and its collaborators are actually safe to run concurrently. + +## Test Types And Assets + +Use each test type where it protects a distinct risk: + +- Unit and package tests protect focused domain behavior and invariants through + the narrowest stable boundary. +- Contract tests protect CLI behavior, configuration, durable artifacts, + schemas, templates, integration formats, compatibility, and stable error + identity. +- Integration tests use real deterministic collaborators when correctness + depends on their interaction, while replacing live or nondeterministic + external boundaries. +- App and CLI tests protect representative assembled generation, batch, + inspection, persistence, and notification workflows. +- Fixtures must be minimal, synthetic, versioned with the behavior they + exercise, and free of credentials or private data. +- Golden files are appropriate only when the complete output is intentionally + stable and semantic review of updates is practical. +- Failure-path tests should cover consequential malformed input, dependency + failure, cancellation, partial results, and recovery behavior. + +## What Deserves Tests + +Prioritize tests for: + +1. CLI, configuration, artifact, template, integration, and package contracts. +2. Meteorological domain rules and important invariants. +3. Boundary conditions and malformed input. +4. Failure handling, cancellation, retries, recovery, and partial success. +5. Serialization, schemas, compatibility, and round trips. +6. Previously observed or plausible regressions. +7. Representative app and CLI workflows. + +A package-level contract is behavior relied upon by another package or major +collaborator, not every observable implementation detail. + +For data integrity, destructive operations, compatibility, security, +concurrency, idempotency, or recovery, presume that durable tests are required +unless the behavior is already credibly protected at another layer. + +Do not add tests merely because a function, branch, or line exists. Do not add +a test when the same meaningful risk is already adequately protected +elsewhere. + +## Choose The Right Boundary + +Test through the narrowest stable boundary that expresses the behavior clearly. +That may be: + +- a small pure function when dense domain logic is clearest there; +- a package operation when several internal collaborators jointly produce the + behavior; or +- a larger integration or app boundary when correctness emerges from + interaction. + +Do not force every behavior through oversized workflow tests. Do not test every +private helper merely because it exists. Choose the boundary that provides +durable confidence with the least incidental coupling. + +## Test Behavior, Not Implementation + +A test should protect a decision, contract, or invariant, not memorialize the +current implementation. Before adding or retaining a test, ask: + +> What realistic defect would this test catch? + +A test is suspect when its main purpose is to detect that someone changed a +private constant, renamed or split a helper, reordered equivalent operations, +changed incidental formatting, replaced one correct algorithm with another, or +refactored private structure without changing behavior. + +Refactoring should normally require no test edits unless the changed structure +is itself contractual. A test can be factually correct and still have negative +value when the behavior it protects is too incidental to justify its future +cost. + +Use these expectations when evaluating failures: + +| Change | Expected effect on tests | +| --- | --- | +| Internal refactor that preserves behavior | Existing tests should normally remain unchanged and pass. | +| Internal default change with no contractual significance | Tests should normally derive expectations from configuration or relationships rather than duplicate the old value. | +| Intentional change to user-visible behavior, policy, schema, or compatibility | Relevant tests should be reviewed and changed deliberately. | +| Accidental contract or invariant violation | Tests should fail; fix production code rather than rewriting tests to accept the defect. | + +A failing test is not necessarily a test that should be edited. Many tests may +correctly fail because of one production defect. The maintenance smell is a +correct internal change that requires unrelated expectation changes throughout +the suite. + +## Separate Mechanism From Policy + +Do not duplicate configurable thresholds and defaults throughout the suite. +Test mechanisms relationally: a configured valid value is accepted, a value +outside the permitted relationship is rejected, and runtime behavior respects +the configured value. + +Test an exact default when its literal value is itself a documented user, +operational, safety, protocol, or compatibility contract. The same distinction +applies to timeouts, capacities, retry counts, ranges, thresholds, and output +limits. + +When concurrency limits are introduced, distinguish configuration enforcement +from runtime enforcement. Validate accepted and rejected settings separately +from measuring whether observed peak concurrency respects the configured +limit. + +## Avoid Semantic Duplication + +Each behavior should have a clear test owner: + +- CLI parser tests own arguments, flags, and command construction. +- Config tests own loading, precedence, defaults, secrets, and validation. +- Domain tests own weather transformations and invariants. +- Adapter tests own HTTP, subprocess, and upload boundaries. +- Orchestrator tests own workflow ordering, persistence, partial success, and + failure propagation. +- State tests own path derivation, atomic artifacts, lookup, and round trips. +- Template and generated-text tests own schemas, render contexts, and rendered + output contracts. + +Higher-level tests should not repeat every lower-level case. Tests that are +individually reasonable may still be collectively redundant; assess the +marginal protection of each additional test. + +## Use Test Doubles Deliberately + +Choose the least elaborate double that provides the required control or +observation: + +1. Prefer real collaborators when they are fast and deterministic. +2. Use small in-memory fakes when realistic stateful behavior helps. +3. Use stubs when a dependency only needs controlled responses. +4. Use mocks when the interaction itself is contractual. + +Mocks are appropriate for requirements such as uploading exactly once, saving +metadata before notification, propagating cancellation to Scriptorium, or +avoiding an external call after an earlier workflow failure. Do not use mocks +merely to isolate every object or reproduce the implementation's call graph. + +## Go-Specific Guidance + +Use: + +- table-driven tests for meaningful behavioral categories and boundaries; +- `t.TempDir()` for real filesystem behavior; +- `httptest.Server` for realistic Weather API interactions; +- test-controlled clocks for periods and RunIDs; +- fake command runners for Scriptorium behavior; +- fake upload clients for Distributor behavior; +- fuzz tests when parsers, normalization, or path handling have a broad and + consequential input space; +- golden files only when complete output stability is intentional; and +- a small number of representative app and CLI workflow tests. + +Avoid exact error-string assertions unless wording is contractual. Prefer +`errors.Is`, `errors.As`, typed errors, structured fields, or the smallest +stable semantic fragment that identifies the failure. At CLI boundaries, +prefer structured summaries, exit behavior, and stable classifications over +snapshots of complete diagnostic wording. + +Golden-file updates must require an explicit local flag. Ordinary validation +must never update golden files automatically, and maintainers must inspect the +semantic diff before accepting an update. + +Keep tests readable and direct. Helpers and fixture frameworks must earn their +maintenance cost; do not build elaborate infrastructure for small or isolated +needs. + +## Coverage + +Coverage is a diagnostic, not a target. Use it to find untested critical +branches and unexpectedly weak packages. Do not write low-value tests solely +to increase a percentage or infer quality from coverage alone. + +Pure domain logic will often warrant higher coverage than CLI wiring or thin +external adapters. Uneven coverage is acceptable when it reflects risk. + +## Regression Tests + +A bug fix should normally include a regression test that fails before the fix +and passes afterward. Prefer the narrowest durable test of the violated +contract or invariant. + +Retain the test when the defect could realistically recur and its consequences +justify the ongoing cost. Remove or consolidate it if the design makes +recurrence implausible or a stronger invariant test subsumes it. + +## Deleting Or Rewriting Tests + +Tests are maintained code, not permanent historical artifacts. Delete or +rewrite a test when its maintenance cost exceeds the confidence it provides. +Candidates include tests that: + +- require edits after harmless internal changes; +- assert private constants without protecting a real contract; +- duplicate the same policy across several layers; +- verify mock choreography rather than outcomes; +- snapshot large amounts of incidental output; +- protect risks already covered more effectively elsewhere; or +- are flaky, misleading, obsolete, or no longer correspond to a plausible + failure. + +Test removal must be deliberate and within the scope of the change. Identify +the behavior the test protected and show that the behavior is covered more +effectively elsewhere or that the failure is no longer plausible enough to +justify durable coverage. Replace several brittle tests with one stronger +behavior or invariant test when appropriate. + +Do not delete or weaken a test merely because it fails after a production +change. First determine whether the failure exposes an accidental regression, +an intentional contract change, or an implementation-coupled assertion. + +## Reviewing A Proposed Test + +When a proposed test's value or durability is not self-evident, ask: + +1. What realistic defect would it catch, and how consequential is that defect? +2. Is the behavior already protected elsewhere? +3. Which layer should own the test? +4. Does it assert a durable contract or incidental implementation detail? +5. What should cause it to fail, and what legitimate changes should not? +6. Could a smaller or more direct test protect the same risk? +7. What ongoing maintenance, execution, and diagnostic cost will it impose? + +Written answers are not required for every routine test. Do not add a test when +its expected lifetime cost exceeds its expected protective value. + +## Definition Of Sufficient + +A suite is sufficient when: + +- important contracts and invariants are protected; +- meaningful boundaries and failure modes are exercised; +- consequential regressions are credibly protected against silent recurrence; +- data integrity, destructive operations, compatibility, security, + concurrency, idempotency, and recovery receive risk-appropriate protection; +- external boundaries have realistic local integration coverage; +- representative complete workflows are tested; +- failures provide useful signal rather than redundant noise; and +- legitimate internal changes usually do not require test edits. + +Sufficiency is a risk judgment, not a coverage percentage or test count. +Reassess it as Weatherreporter, its users, and the consequences of failure +evolve. + +The governing rule is: + +> Test heavily where failure is consequential, subtle, or difficult to detect +> after the fact. Test lightly where failure is obvious, reversible, and +> inexpensive. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..92b31e1 --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,630 @@ +# Documentation Refresh Implementation Plan + +## Status + +Accepted for implementation. + +This is a temporary, staged coordination plan for refreshing Weatherreporter's +current documentation. Implement each stage in order. Do not treat this file as +a current-behavior reference. + +## Objective + +Bring every maintained document and example into agreement with: + +- the implemented Go application; +- the canonical ownership rules in `docs/policy/documentation.md`; +- the architectural boundaries in `docs/policy/architecture.md`; +- the testing rules in `docs/policy/testing.md`; and +- the contributor routing in `docs/development.md`. + +The completed documentation set must be concise, task-oriented, free of stale +or duplicate contracts, and verifiably consistent with the executable sources +of truth. + +## Scope And Constraints + +This is a documentation-only refresh. + +- Do not add or change product behavior. +- Do not modify Go code, schemas, prompts, or templates to make documentation + claims true. +- `examples/*.yml` may be corrected when they disagree with implemented + configuration, because examples are maintained documentation artifacts. +- If code and documentation disagree, document implemented and tested behavior. +- If code appears to violate an architecture or safety policy, do not conceal + the conflict with prose and do not fix it during this refresh. Record the + conflict in this plan's implementation notes and report it to the maintainer. +- Do not implement any work from `docs/roadmap/future.md` or + `docs/roadmap/promptkit.md`. +- Preserve the exact contents of `AGENTS.md`. +- Treat `docs/policy/*.md` and `docs/development.md` as the governing baseline. + Change them only to repair a link made stale by this refresh; do not reopen + their substantive decisions. +- Do not create `docs/internal/overview.md`; `docs/development.md` owns the + current concise package inventory. +- Do not introduce a release guide, public API guide, consumer guide, or ADR + unless a separate maintainer decision expands this plan. +- Preserve unrelated working-tree changes and do not commit unless explicitly + instructed. + +## Executable Authorities + +Use codebase-memory graph tools before text search for code discovery. Index the +repository first if needed, use `search_graph` to locate symbols, +`trace_path` for dependencies and impact, and `get_code_snippet` for exact +source. Use graph-augmented `search_code` or `rg` for literal flags, paths, +errors, schemas, asset names, and configuration values. + +The primary authorities for this refresh are: + +| Contract | Executable authority | +| --- | --- | +| CLI commands, flags, help, and dispatch | `internal/cli/root.go`, `internal/report/names.go`, and `internal/cli/root_test.go` | +| CLI JSON summaries and stdout/stderr behavior | `internal/cli/output.go`, `internal/cli/result.go`, and their tests | +| Configuration fields, defaults, loading, precedence, secrets, templates, and validation | `internal/config/*.go` and `internal/config/config_test.go` | +| Report IDs, modes, periods, modules, output names, comparison, and Distributor paths | `internal/report/*.go` and `internal/report/period_test.go` | +| Application generation, batch, inspection, persistence, and notification order | `internal/app/*.go`, `internal/app/*_test.go`, and `internal/cli/root_test.go` | +| Weather collection and external Weather API behavior | `internal/collect`, `internal/weatherdata`, `internal/adapters/weatherapi`, fixtures, and focused tests | +| Facts, forecast derivation, modules, and Recent Changes | `internal/facts`, `internal/forecast`, `internal/module`, `internal/briefing`, `internal/changes`, and focused tests | +| Prompt packages and Scriptorium invocation | `internal/promptinput`, `internal/adapters/scriptorium`, and focused tests | +| Generated text, render contexts, templates, and schemas | `internal/generatedtext`, `internal/reporttemplate`, embedded assets, and focused tests | +| Workspace paths, metadata, atomic writes, prior lookup, and inspection | `internal/state`, `internal/fileutil`, and focused tests | +| Distributor bundle, upload, polling, redaction, and notification behavior | `internal/adapters/distributor`, `internal/app/batch_notification.go`, `go.mod`, and focused tests | + +Sibling repositories may be used only as secondary integration references. +Weatherreporter's pinned dependency, adapter code, and tests determine the +subset this repository actually uses. Do not copy broad upstream manuals into +this repository. + +## Rules For Every Stage + +For each stage: + +1. Read the task-specific entries in `docs/development.md`. +2. Inspect the listed executable authorities and focused tests. +3. Update only the files assigned to the stage, plus this plan's stage status + if progress tracking is requested. +4. Keep exact facts in their canonical owner. Replace duplicate detail with a + short stable summary and a relative link. +5. Describe implemented behavior only outside `docs/roadmap/`. +6. Keep examples synthetic, valid, and secret-free. +7. Use language tags on every fenced block. +8. Run the stage validation before proceeding. +9. Do not reinterpret surprising behavior as intent; tests and executable + contracts take precedence over old prose. + +## Stage 1: Inventory And Mechanical Hygiene + +### Files + +- `README.md` +- all Markdown files under `docs/` +- `examples/*.yml` + +### Work + +1. Classify every document under the ownership table in + `docs/policy/documentation.md`. Do not create a separate permanent inventory + document. +2. Identify duplicated exact contracts, stale paths, broken links, untagged + code fences, and current-state claims in the wrong document. +3. Repair mechanical link and path defects that do not require content + redesign. +4. In `docs/integrations/distributor/pkg-bundle.md`, remove the broken local + link to `../integrations/source-bundle.md`. Identify the upstream canonical + file as `docs/integrations/source-bundle.md` in the Distributor repository + using prose or code formatting, not a nonexistent repository-relative link. +5. In `docs/integrations/distributor/pkg-upload.md`, do the same for the broken + `../integrations/http-upload.md` link. +6. Confirm that no document references the retired development-policy path. +7. Defer substantive rewrites to the assigned later stage; this stage should + establish a clean mechanical baseline. + +### Acceptance + +- Every repository-relative link target exists. +- All fenced blocks have language tags. +- No stale development-policy reference remains. +- No future feature is presented as current behavior outside `docs/roadmap/`. +- `git diff --check` passes. + +## Stage 2: CLI, Configuration, And Examples + +### Files + +- `docs/cli.md` +- `docs/config.md` +- `examples/minimal-config.yml` +- `examples/config.yml` + +### CLI Work + +1. Regenerate the command inventory from `go run ./cmd/weatherreporter --help` + and verify it against `internal/cli/root.go` and report command-name tests. +2. Keep this document's order: + - shortest useful command; + - command overview and complete usage; + - stdout, stderr, JSON summary, quiet-mode, and exit behavior; + - complete flag reference; + - concise invocation examples; and + - inspection commands. +3. Keep exact command syntax and output semantics here. Move or replace + operational workflow explanations with links to `operations.md`. +4. Verify all seven generate commands, both batch commands, and every inspect + command. +5. Verify command-specific restrictions: required Daily date, optional Today + date, Storm bounds, Hourly restrictions, generate output copies, batch + output directories, and inspection quiet-mode rejection. + +### Configuration Work + +1. Compare every documented field and default with `Config`, `Defaults`, + loading, secret loading, report overrides, notification templates, and + validation code. +2. Keep discovery, precedence, fields, defaults, accepted values, validation + rules, secret-supply behavior, and links to examples here. +3. Keep operational notification sequencing, filesystem lifecycle, and + recovery in `operations.md`; summarize and link rather than duplicate them. +4. Verify report-module override keys against the report registry and module + registry. +5. Verify every template placeholder accepted by Distributor notification + configuration. + +### Example Work + +1. Ensure both examples contain only implemented fields and synthetic + endpoints. +2. Keep `minimal-config.yml` genuinely minimal. +3. Keep `config.yml` production-oriented and representative without becoming a + second field reference. +4. Do not add credentials, private endpoints, or generated reports. + +### Validation + +```sh +go run ./cmd/weatherreporter --help +go test ./internal/cli ./internal/config ./internal/report ./internal/module +git diff --check +``` + +The config tests must load the maintained examples successfully. + +## Stage 3: README And Operations + +### Files + +- `README.md` +- `docs/operations.md` + +### README Work + +Keep the README short and in this order: + +1. concise product description; +2. one-sentence value proposition; +3. the shortest successful generation command; and +4. links to CLI, configuration, operations, troubleshooting, development, and + architecture documentation. + +Do not add a complete command list, configuration fields, workspace layout, or +future Promptkit behavior. + +### Operations Work + +1. Verify generation and batch workflow order against `internal/app`. +2. Verify the physical workspace tree, artifact names, RunID placement, + metadata links, notification artifact placement, and prior lookup against + `internal/state`. +3. Verify output-copy behavior and the rule that managed reports are the only + upload sources. +4. Verify single-report and batch notification sequencing, skip conditions, + failure accounting, and inspectable artifacts. +5. Verify inspection is read-only and identify exactly which artifacts each + command reads. +6. Keep normal operation, state lifecycle, inspection, recovery, and + operational caveats here. +7. Replace complete flag or config-field definitions with links to their + canonical references. +8. Keep only the minimal commands needed to illustrate an operational + procedure. + +### Validation + +```sh +go test ./internal/app ./internal/state ./internal/cli +go run ./cmd/weatherreporter --help +git diff --check +``` + +## Stage 4: Troubleshooting + +### File + +- `docs/troubleshooting.md` + +### Work + +1. Verify every documented symptom and diagnostic fragment against current + errors and failure tests. +2. Use one consistent entry shape: + - symptom; + - likely cause; + - diagnostic step or command; + - safe fix; and + - links to canonical references. +3. Keep symptom-oriented diagnosis here. Remove repeated normal workflows, + full endpoint inventories, config-field definitions, and artifact schemas. +4. Preserve the distinction between pre-run failures, collection failures, + Scriptorium preparation/run failures, generated-text validation failures, + template failures, report failures, batch partial failures, skipped batch + uploads, Distributor failures, secret loading, and state lookup failures. +5. Use exact error text only when it is a stable operator-facing contract; + otherwise use the smallest stable identifying fragment. +6. Ensure fixes are safe, narrow, and do not recommend broad workspace + deletion or expose secret values. + +### Validation + +```sh +go test ./internal/cli ./internal/config ./internal/app ./internal/state +go test ./internal/adapters/weatherapi ./internal/adapters/scriptorium ./internal/adapters/distributor +git diff --check +``` + +## Stage 5: Report Template Guide + +### File + +- `docs/templates.md` + +### Work + +1. Verify top-level templates, shared partials, schema IDs, prompt sources, and + registered template functions against `internal/reporttemplate`. +2. Verify every documented render-context field against + `internal/generatedtext`; remove fields that are absent and add implemented + fields needed by maintainers. +3. Keep the maintainer-facing editing rules, context contracts, and minimal + template examples here. +4. Clearly distinguish deterministic module values, collected/derived facts, + and validated GeneratedText prose. +5. Keep weather derivation and generated-text validation mechanics in their + internal owners; summarize and link from this guide. +6. Avoid duplicating the full JSON schema bodies or package implementation + flow. + +### Validation + +```sh +go test ./internal/reporttemplate ./internal/generatedtext ./internal/app +git diff --check +``` + +## Stage 6: Weather API Integration + +### File + +- `docs/integrations/weatherapi.md` + +### Work + +1. Verify URL joining, query parameters, timeout and retry behavior, response + envelope, endpoint set, decoding, source identity, warnings, and + required/optional source behavior against the adapter, collection layer, + fixtures, and tests. +2. Keep the external Weather API contract and compatibility assumptions here. +3. Move internal normalization and orchestration mechanics to links pointing at + the relevant internal documents. +4. Document only endpoints and fields Weatherreporter currently consumes. + +### Validation + +```sh +go test ./internal/adapters/weatherapi ./internal/collect ./internal/weatherdata +git diff --check +``` + +## Stage 7: Scriptorium Integration + +### File + +- `docs/integrations/scriptorium.md` + +### Work + +1. Verify the exact `render` and `run` invocations, argument ordering, + timeouts, output paths, result decoding, stderr handling, cancellation, and + failure behavior against the adapter and tests. +2. Keep the external CLI contract here and link to internal prompt-input, + adapter, generated-text, and orchestration mechanics. +3. Describe Scriptorium as the implemented runtime. Do not mention the proposed + Promptkit replacement outside `docs/roadmap/promptkit.md`. +4. Do not copy Scriptorium's general manual; document only the commands and + result fields Weatherreporter uses. + +### Validation + +```sh +go test ./internal/adapters/scriptorium ./internal/promptinput ./internal/app +git diff --check +``` + +## Stage 8: Distributor Integration + +### Files + +- `docs/integrations/distributor/api.md` +- `docs/integrations/distributor/pkg-bundle.md` +- `docs/integrations/distributor/pkg-upload.md` + +### Work + +1. Rewrite these as concise references to the exact Distributor surface used + by Weatherreporter: + - `api.md` owns accepted upload, authentication, idempotency, run status, + terminal failure, and retention assumptions used by the adapter; + - `pkg-bundle.md` owns the bundle file mapping and path constraints used by + Weatherreporter; + - `pkg-upload.md` owns the client construction, upload, retry, conflict, and + status operations used by the adapter. +2. Remove generic producer tutorials, broad upstream feature descriptions, + unrelated examples, and claims Weatherreporter does not rely on. +3. Verify the pinned module version in `go.mod` and the adapter's actual calls. +4. Use the sibling Distributor repository only to confirm the pinned API. Do + not make Weatherreporter documentation depend on sibling-relative links. +5. Keep internal request construction, redaction, polling decisions, and app + notification order in internal documents. +6. Ensure all local links resolve and upstream canonical paths are identified + without pretending those files exist in this repository. + +### Validation + +```sh +go test ./internal/adapters/distributor ./internal/app ./internal/config +git diff --check +``` + +## Stage 9: CLI, Collection, And App Internals + +### Files + +- `docs/internal/cli.md` +- `docs/internal/collect.md` +- `docs/internal/app-orchestration.md` + +### Work + +Verify purpose, inputs and outputs, ownership boundaries, workflow composition, +failure propagation, focused tests, and invariants. Keep public CLI syntax in +`docs/cli.md`, Weather API wire behavior in its integration document, and +physical state layout in `docs/operations.md` or `docs/internal/state.md`. + +The app document may retain explicit workflow ordering because orchestration is +its owned internal contract. It must not duplicate complete config, CLI, or +external protocol references. + +### Validation + +```sh +go test ./internal/cli ./internal/collect ./internal/app +git diff --check +``` + +## Stage 10: State, Source, And Adapter Internals + +### Files + +- `docs/internal/state.md` +- `docs/internal/weather-data.md` +- `docs/internal/scriptorium-adapter.md` +- `docs/internal/distributor-adapter.md` + +### Work + +1. Verify each document against its package and focused tests. +2. Keep state path derivation, metadata ownership, prior lookup, atomic writes, + and inspection mechanics in the state document. +3. Keep normalized bundle, source metadata, and warning semantics in the + weather-data document. +4. Keep argv/result translation in the Scriptorium adapter document and + dependency/client translation in the Distributor adapter document. +5. Link external contracts and user-facing references rather than repeating + them. + +### Validation + +```sh +go test ./internal/state ./internal/weatherdata +go test ./internal/adapters/scriptorium ./internal/adapters/distributor +git diff --check +``` + +## Stage 11: Deterministic Domain Internals + +### Files + +- `docs/internal/facts.md` +- `docs/internal/forecast-derivation.md` +- `docs/internal/changes.md` + +### Work + +Verify fact ownership, derivation inputs, period and daypart behavior, +thresholds, comparison strategies, missing-data behavior, failures, tests, and +invariants. Keep these documents deterministic and independent of CLI, +filesystem, subprocess, and transport mechanics. Link configuration defaults +instead of restating them. + +### Validation + +```sh +go test ./internal/facts ./internal/forecast ./internal/changes ./internal/timeutil +git diff --check +``` + +## Stage 12: Reports, Modules, Briefing, And Prompt Input + +### Files + +- `docs/internal/report-registry.md` +- `docs/internal/module.md` +- `docs/internal/briefing.md` +- `docs/internal/prompt-input.md` + +### Work + +1. Verify all report definitions, registry fields, valid periods, batch + membership, comparison policy, output naming, module defaults, and + Distributor path declarations. +2. Verify registered module IDs, options, rich values, prompt exports, + composition by report, and missing-source behavior. +3. Verify prompt package schema, ordering, metadata, serialization, validation, + and the boundary between rich module values and prompt-facing exports. +4. Keep exact CLI command syntax, config defaults, integration protocol, and + rendered template contracts in their canonical owners. +5. Prefer tables for report-to-module or report-to-mode mappings where they + materially reduce repeated prose. + +### Validation + +```sh +go test ./internal/report ./internal/module ./internal/briefing ./internal/promptinput +git diff --check +``` + +## Stage 13: Generated Text And Template Internals + +### Files + +- `docs/internal/generatedtext.md` +- `docs/internal/reporttemplate.md` + +### Work + +Verify catalog registration, schema selection, decoding and validation, +generated-text types, render-context construction, embedded asset lookup, +template parsing, partials, failure behavior, and invariants. Keep the complete +maintainer-facing context-field reference in `docs/templates.md`; internal +documents should explain implementation and link to that contract. + +### Validation + +```sh +go test ./internal/generatedtext ./internal/reporttemplate ./internal/app +git diff --check +``` + +## Stage 14: Roadmap Review + +### Files + +- `docs/roadmap/future.md` +- `docs/roadmap/promptkit.md` + +### Work + +1. Ensure every roadmap item is explicitly unimplemented, proposed, accepted, + deferred, or rejected. +2. Remove or rewrite work that has already been implemented. +3. Keep current-state summaries minimal and link to canonical current + documentation instead of reproducing it. +4. In `future.md`, correct the Distributor enhancement introduction so it + acknowledges implemented single-report and batch notification without + maintaining their detailed current contract. +5. Preserve the Promptkit roadmap as future migration policy. Do not implement + it, resolve its product-design open questions, or describe it as current. +6. Ensure future architecture does not leak into README, user/operator docs, + integrations, or current internal docs. + +### Validation + +```sh +rg -n "Promptkit|promptkit" README.md docs \ + --glob "*.md" --glob "!docs/roadmap/**" +git diff --check +``` + +The Promptkit search must return no current-state documentation matches. + +## Stage 15: Cross-Document Consistency And Final Validation + +### Semantic Review + +Verify these cross-document facts have one exact owner and consistent summaries +elsewhere: + +- seven generate report commands; +- four generated-text-template reports and three direct-Markdown reports; +- morning and evening batch membership and future Daily eligibility; +- configuration precedence and defaults; +- module and report IDs; +- managed workspace and notification paths; +- Recent Changes structured comparison; +- Scriptorium preflight and run behavior; +- single-report and batch Distributor behavior; +- output copies versus managed upload sources; and +- inspection's read-only behavior. + +Remove duplicate exact tables or narratives from non-owning documents. Preserve +short task-relevant summaries and relative links. + +### Link Validation + +Check every Markdown link in `README.md` and `docs/`. Repository-relative +targets and changed heading anchors must resolve. No maintained document may +depend on sibling checkout paths. + +At minimum, run a repository-relative target check equivalent to: + +```sh +find README.md docs -type f -name "*.md" -print0 | + while IFS= read -r -d "" file; do + dir=$(dirname "$file") + rg -o '\]\([^)]+\)' "$file" | + sed -E 's/^\]\(([^)#]+)(#[^)]+)?\)$/\1/' | + while IFS= read -r target; do + case "$target" in + http:*|https:*|mailto:*) continue + esac + test -z "$target" || test -e "$dir/$target" || + printf '%s -> %s\n' "$file" "$target" + done + done +``` + +Manually verify anchors on every changed link because the target check does not +validate fragments. + +### Final Commands + +```sh +go test ./... +go run ./cmd/weatherreporter --help +git diff --check +``` + +Review `git diff --name-only` and confirm the refresh changed only Markdown and +maintained YAML examples. Confirm no secrets, private endpoints, generated +artifacts, code changes, `go.work`, or local module replacements were added. + +## Completion Criteria + +The refresh is complete when: + +- every document has one clear canonical owner; +- current user and operator documentation matches implemented behavior; +- CLI and configuration references are complete and verified; +- examples load and contain no secrets; +- operation, recovery, template, and integration contracts are accurate; +- internal documents preserve package boundaries without duplicating public + references; +- roadmaps contain only future or coordination material; +- all repository-relative links and changed anchors resolve; +- all validation commands pass; and +- no unresolved architecture-policy conflict was hidden or implemented as part + of the documentation work. + +After all stages are accepted, remove this temporary implementation plan in a +final documentation-only change. Do not move its task sequencing or completion +history into current-state documentation.