Added initial policy documentation and a docs implementation roadmap

docs/policy/architecture.md

@@ -0,0 +1,196 @@
+# Architecture Policy
+
+## Purpose
+
+This document defines `weatherapi`'s development architecture and invariants for maintainers and LLM coding agents. It describes how the implemented system is built and how future changes should preserve its boundaries.
+
+This is an inward-facing policy document. Consumer-facing HTTP contract details belong in [`docs/api.md`](../api.md), and proposed or unimplemented work belongs under [`docs/roadmap/`](../roadmap/).
+
+## Project Shape
+
+`weatherapi` is a small Go HTTP API that serves weather data previously persisted by `weatherfeeder`. It does not poll upstream weather providers. It reads weatherfeeder-owned Postgres tables, maps rows back into weatherfeeder canonical model types or API-specific read models, and presents those resources through HTTP endpoints.
+
+The implemented runtime flow is:
+
+1. `cmd/weatherapi` resolves the config path from `-config`, then `WEATHERAPI_CONFIG`, then `config.yml`.
+2. Feedapi loads YAML configuration for the HTTP server, database handles, and template directory.
+3. The command opens configured databases and selects the first database as the primary weather data store.
+4. The Postgres repository is constructed against the primary database.
+5. The application service is constructed over the repository read port.
+6. HTTP endpoint definitions are registered with feedapi.
+7. Feedapi handles routing, middleware, format negotiation, template rendering, and graceful shutdown.
+8. Each request binds query parameters, calls an application use case, presents the result, and returns a `data` response envelope.
+
+Core application code lives in `internal/app`. Inbound HTTP adapters live under `internal/adapters/inbound/httpapi`. Outbound Postgres reads live under `internal/adapters/outbound/postgres`. Text response templates live in `templates/`.
+
+## Core Design Principles
+
+- Hexagonal boundaries: HTTP, configuration, database handles, SQL, rendering, and templates are adapters around application read use cases.
+- Read-only API posture: `weatherapi` reads persisted weather data; ingestion and table ownership belong to `weatherfeeder`.
+- Explicit ports: `internal/app.Repository` is the outbound read port, and `internal/adapters/inbound/httpapi.Service` is the inbound service contract used by handlers.
+- Thin composition root: `cmd/weatherapi` wires config, databases, repository, service, endpoints, and feedapi runtime; it should not contain endpoint, SQL, or presentation logic.
+- Adapter-local policy: query validation belongs at the HTTP boundary; SQL row shape and null handling belong in the Postgres adapter; unit and timezone presentation belongs in presenter helpers.
+- Current-behavior docs: outside roadmap files, document only implemented routes, config, storage assumptions, and runtime behavior.
+- Standard-library-first: use the Go standard library where reasonable, with narrow dependencies for framework/runtime integration and database access.
+
+## Architectural Boundaries
+
+Core/application logic:
+
+- `internal/app` defines read use cases, repository ports, and API-specific read models such as current conditions.
+- Application code may depend on canonical `weatherfeeder/model` types because those are the persisted weather domain contract.
+- Application code should not import HTTP, feedapi endpoint/render packages, SQL packages, or template packages.
+
+Inbound HTTP adapter:
+
+- `internal/adapters/inbound/httpapi` owns route definitions, query binding, request validation, day-slice filtering for forecast routes, and endpoint-specific handler wiring.
+- `internal/adapters/inbound/httpapi/presenter` owns response payload shaping, metric/US conversions, rounding, timestamp timezone conversion, and copy semantics.
+- Endpoint handlers should call the `Service` interface and return `response.Envelope{Data: ...}`. They should not execute SQL directly.
+
+Outbound Postgres adapter:
+
+- `internal/adapters/outbound/postgres` owns SQL text, row structs, row scanning, null conversion, UTC normalization, and reconstruction of nested canonical payloads from weatherfeeder tables.
+- Repository methods return `nil, nil` for missing latest rows so HTTP responses can render `data: null`.
+- The adapter assumes the weatherfeeder Postgres schema exists. It does not create or migrate weatherfeeder tables.
+
+Runtime composition:
+
+- `cmd/weatherapi/main.go` owns process wiring: config path selection, config loading, DB registry, primary DB selection, repository/service construction, endpoint registration, signal cancellation, and logging.
+- Feedapi owns the generic HTTP runtime: routing, renderers, middleware, database registry lifecycle, and graceful server shutdown.
+
+## Modules or Stages
+
+The implemented request path has five main stages: route matching, query binding, service call, presentation, and rendering.
+
+Query binding contract:
+
+- Normalize common `format`, `units`, and `precision` query values where supported.
+- Reject unknown query parameters on implemented endpoints.
+- Keep endpoint-specific query rules in endpoint binders.
+- Parse timezone only for route families that support `tz` / `TZ`.
+- Return typed feedapi errors so invalid input becomes a structured `400 Bad Request` response.
+
+Service contract:
+
+- Keep service methods narrow and resource-specific.
+- Delegate simple latest-resource reads directly to the repository.
+- Keep use-case defaults in `internal/app`; current conditions use `ObservationWindowMinutesDefault`.
+- Do not let transport or SQL details leak into service method signatures.
+
+Presenter contract:
+
+- Treat repository-returned models as immutable input.
+- Copy payloads before unit conversion, rounding, or timezone conversion.
+- Preserve nil pointers and optional fields so JSON/XML `omitempty` behavior remains meaningful.
+- Keep metric and US field naming explicit and tested.
+- Keep text templates conditional so absent optional values do not render misleading zero values.
+
+Repository contract:
+
+- Query the latest parent row for each resource, then load child rows in stored order where needed.
+- Map SQL nulls to nil pointers or zero-value omitted fields consistently.
+- Normalize database timestamps to UTC before returning models.
+- Wrap query, scan, iteration, and decode errors with operation context.
+
+## State, Inputs, and Outputs
+
+Inputs are HTTP requests and Postgres rows written by `weatherfeeder`.
+
+Outputs are HTTP responses in JSON, XML, or text format. All public endpoint handlers return a top-level `data` envelope; missing latest data is represented as `data: null`.
+
+`weatherapi` owns no durable weather state. Its runtime state is limited to process memory, loaded configuration, HTTP server state, template registry, renderer registry, and database connection pools. Durable weather data and schema creation are external concerns owned by `weatherfeeder` and Postgres.
+
+The API currently serves latest-resource views: latest observation, current conditions, latest active alerts run, latest hourly forecast, latest narrative forecast, latest forecast discussion, latest weather story run, and latest individual weather story. Forecast `today` and `tomorrow` routes derive filtered copies from the latest run.
+
+## Configuration and CLI Boundaries
+
+The executable supports one CLI flag:
+
+- `-config`: path to the YAML config file.
+
+If `-config` is not provided, the default comes from `WEATHERAPI_CONFIG`; if that environment variable is empty, the default is `config.yml`.
+
+Configuration shape is provided by feedapi and includes:
+
+- `server`: listen address, default format, and HTTP timeouts;
+- `databases`: named database handles;
+- `templates`: template base directory.
+
+`cmd/weatherapi` requires at least one configured database and uses the first database as the primary weather repository. Template loading is handled by feedapi; text output is available when the configured template directory is present and loaded.
+
+Do not duplicate full CLI or config reference material here. If `docs/cli.md` or `docs/config.md` are added or updated, they should be the canonical user/operator references.
+
+## Errors, Logging, and Diagnostics
+
+`cmd/weatherapi` uses the standard library `log` package with timestamps and microseconds.
+
+Startup failures are fatal and include operation context such as config loading, database opening, primary database selection, app construction, or server startup. Database close errors during shutdown are logged.
+
+Feedapi handles transport-level diagnostics:
+
+- bind errors become structured invalid-parameter responses;
+- unsupported formats become structured unsupported-format responses;
+- handler errors are normalized before response rendering;
+- recovery, request ID, and timing middleware are installed by default.
+
+Repository methods should distinguish no data from errors. `sql.ErrNoRows` maps to `nil, nil`; query, scan, iteration, and JSON decode failures should return contextual errors.
+
+The daemon handles `os.Interrupt` and `SIGTERM` through `signal.NotifyContext`. Feedapi starts `http.Server.ListenAndServe` and performs graceful shutdown when the context is canceled.
+
+## Testing Expectations
+
+When changing behavior, inspect or add focused tests in the owning package.
+
+Expected coverage by change type:
+
+- App service: repository delegation, use-case defaults, filtering or aggregation rules, and error propagation.
+- HTTP adapter: route registration, query validation, supported/rejected parameters, format negotiation, error envelopes, null `data`, day-slice behavior, and timezone behavior.
+- Presenter: metric/US conversion, rounding, timezone conversion, optional field omission, nil handling, copy semantics, and text-template-sensitive fields.
+- Postgres adapter: SQL row mapping, nullable handling, UTC normalization, child ordering, missing rows, and nested payload reconstruction.
+- Runtime/config: config path behavior, database selection, template loading, and endpoint registration when those surfaces change.
+
+Use fakes, `httptest`, local renderers/templates, and row-mapping fixtures rather than live services. Full-package tests should remain fast and deterministic.
+
+## Dependency Policy
+
+Prefer the Go standard library for HTTP helpers, time handling, logging, tests, conversions, and small utilities.
+
+Accepted project-level dependencies are narrow and purposeful:
+
+- `feedapi` provides config loading, HTTP runtime, endpoint definitions, renderers, templates, middleware, and database registry behavior.
+- `weatherfeeder` provides canonical weather model and standards types that match the persisted data contract.
+- `github.com/lib/pq` provides the Postgres driver.
+
+Do not add dependencies for small conveniences. Do not let dependency-specific types cross application boundaries unless that dependency is the explicit adapter contract.
+
+## Documentation Expectations
+
+Documentation must follow [`docs/policy/documentation.md`](documentation.md).
+
+Rules for architecture-related docs:
+
+- Current-behavior docs must describe implemented behavior only.
+- Planned work, including outlook endpoints, belongs only under `docs/roadmap/` until implemented.
+- Prefer links to canonical docs over repeated reference material.
+- Update docs in the same change when modifying public routes, query behavior, config behavior, runtime behavior, presenter payloads, or database read assumptions.
+
+## Architectural Invariants
+
+- Keep `cmd/weatherapi` as composition code, not endpoint, SQL, or presentation logic.
+- Keep application read ports in `internal/app` independent of HTTP and SQL packages.
+- Keep HTTP route definitions and query binding in the inbound adapter.
+- Keep unit conversion, rounding, timezone presentation, and payload copy behavior in presenters.
+- Keep SQL text, row structs, and null handling in the Postgres adapter.
+- Preserve `data` envelope responses and `data: null` semantics for missing latest rows.
+- Preserve UTC normalization at the repository boundary and timezone conversion at the presentation boundary.
+- Preserve strict query-parameter rejection unless an endpoint explicitly allows a parameter.
+- Preserve text templates as adapter presentation assets, not business logic.
+- Do not make `weatherapi` responsible for ingesting upstream weather data or creating weatherfeeder tables.
+
+## Non-Goals
+
+- `weatherapi` is not a weather ingester and does not call NWS, Open-Meteo, OpenWeather, or SPC provider APIs.
+- `weatherapi` does not own weatherfeeder's database schema creation or migrations.
+- `weatherapi` does not provide historical browsing except where an implemented endpoint explicitly derives a filtered view from the latest run.
+- `weatherapi` is not a general-purpose API framework; generic HTTP runtime behavior belongs to `feedapi`.
+- Architecture policy is not a public endpoint, CLI, or config reference.

docs/policy/documentation.md

@@ -0,0 +1,444 @@
+# Go Project Documentation Policy
+
+## Purpose
+
+Project documentation must help five audiences:
+
+1. users who need to run the application;
+2. administrators/operators who need to configure and operate it;
+3. developers who need to understand and change it safely;
+4. LLM coding agents that need clear scope, boundaries, and invariants;
+5. developers and LLM coding agents integrating this project from another codebase.
+
+Docs should be accurate, concise, task-oriented, and organized by audience. Prefer links to canonical docs over repetition.
+
+## Core Rules
+
+### 1. Keep docs concise
+
+Each document should cover a defined scope and only the essentials for that scope.
+
+Avoid:
+- long background explanations;
+- repeated reference material;
+- implementation detail in user-facing docs;
+- aspirational language outside roadmap docs;
+- verbose examples where one minimal example is clearer.
+
+### 2. Document only implemented behavior outside roadmap files
+
+Unimplemented, planned, aspirational, experimental, or future work may be described only under:
+
+- `docs/roadmap/`
+
+No other documentation file, including `README.md`, should describe code, features, modules, stages, commands, config fields, or behaviors that do not currently exist.
+
+If a feature is partial, non-roadmap docs may describe only the implemented portion and its current boundary.
+
+### 3. Use canonical homes
+
+Each type of information should have one canonical location.
+
+Canonical homes:
+
+- project purpose and quickstart: `README.md`
+- development principles: `docs/policy/architecture.md`
+- public HTTP API reference: `docs/api.md`
+- configuration reference: `docs/config.md`
+- CLI reference: `docs/cli.md`
+- operations and recovery: `docs/operations.md`
+- troubleshooting: `docs/troubleshooting.md`
+- public API/package consumer guidance: `docs/consumers/`
+- implemented internals: `docs/internal/`
+- external protocol, service, and file-format contracts: `docs/integrations/`
+- future work: `docs/roadmap/`
+- contributor workflow: `docs/policy/development.md`
+- copyable examples: `examples/`
+
+Other files should summarize briefly and link to the canonical source.
+
+### 4. Keep examples real
+
+Examples should be valid, maintained, and free of secrets.
+
+Where practical:
+- example configs should load successfully;
+- example commands should match real CLI syntax;
+- important examples should be covered by tests.
+
+## Documentation Profiles
+
+All projects require:
+
+- `README.md`
+- `docs/policy/architecture.md`
+
+Additional docs depend on the project.
+
+### Small library
+
+Recommended:
+- `docs/policy/development.md`, if contributor conventions are non-obvious
+
+### Simple CLI
+
+Required:
+- `docs/cli.md`
+
+Recommended:
+- `docs/policy/development.md`
+
+### Config-driven CLI
+
+Required:
+- `docs/cli.md`
+- `docs/config.md`
+
+Recommended:
+- `examples/`
+- `docs/policy/development.md`
+
+### Stateful or operator-facing application
+
+Required:
+- `docs/cli.md`, if CLI-based
+- `docs/config.md`, if config-driven
+- `docs/operations.md`
+
+Recommended:
+- `docs/troubleshooting.md`
+- `examples/`
+- `docs/policy/development.md`
+
+### Modular, service-oriented, or orchestration application
+
+Required:
+- `docs/cli.md`, if CLI-based
+- `docs/config.md`, if config-driven
+- `docs/operations.md`
+- `docs/internal/`
+- `docs/policy/development.md`
+
+Recommended:
+- `docs/troubleshooting.md`
+- validated examples under `examples/`
+
+### Public HTTP API service
+
+Required:
+- `docs/api.md`
+- `docs/cli.md`, if CLI-based
+- `docs/config.md`, if config-driven
+- `docs/operations.md`
+- `docs/internal/`
+- `docs/policy/development.md`
+
+Recommended:
+- `docs/troubleshooting.md`
+- `docs/consumers/`, for task-oriented client integration guides
+- `docs/integrations/`, for upstream/downstream service contracts
+- validated examples under `examples/`
+
+### Project with public packages or consumer APIs
+
+Required:
+- `docs/consumers/api.md`
+- one `docs/consumers/pkg-<name>.md` file per public package, if public packages exist
+
+Recommended:
+- copyable consumer examples under `examples/`, if practical
+
+## Required Documents
+
+### README.md
+
+**Audience:** users, administrators, operators
+
+The README is the outward-facing project orientation page.
+
+It should include, in order:
+
+1. concise description;
+2. elevator pitch;
+3. shortest useful command or usage example;
+4. links to targeted docs.
+
+The README should be short. It is not a manual.
+
+The “shortest useful command” means the simplest command that performs the project’s core use case. (It does not mean `app --help`.)
+
+### docs/policy/architecture.md
+
+**Audience:** developers, LLM coding agents
+
+`docs/policy/architecture.md` is required for every project.
+
+It is an inward-facing development policy document. It should describe how the project is intended to be built and changed.
+
+It should include:
+
+- project shape;
+- core design principles;
+- package and boundary philosophy;
+- state/persistence philosophy, if applicable;
+- external integration philosophy, if applicable;
+- error-handling and logging principles;
+- testing expectations;
+- documentation expectations;
+- architectural invariants;
+- explicit non-goals, if useful.
+
+For small projects, this file may be brief. It may simply state that the project is intentionally narrow, monolithic, and dependency-light.
+
+### docs/api.md
+
+**Audience:** external HTTP API consumers, developers, LLM coding agents integrating by HTTP
+
+Required for projects whose primary public interface is HTTP.
+
+`docs/api.md` is the canonical public HTTP API contract. It should be normative for external consumers and should not be duplicated by README, operations docs, consumer guides, or integration docs.
+
+It should include:
+
+1. base URL conventions;
+2. authentication and authorization behavior, if implemented;
+3. response envelope;
+4. supported media types and content negotiation behavior;
+5. shared query parameters;
+6. endpoint reference grouped by route family;
+7. request parameters and validation rules;
+8. response fields, units, nullability, and optionality;
+9. error response shape and status codes;
+10. pagination, caching, rate-limit, idempotency, and retry behavior, if implemented;
+11. compact request and response examples.
+
+It must document only implemented endpoints and behavior. Planned endpoints, proposed fields, future filters, and experimental response shapes belong only under `docs/roadmap/`.
+
+For HTTP API projects, `docs/consumers/` may provide task-oriented client integration guides, but those guides should link to `docs/api.md` for the authoritative endpoint contract.
+
+### docs/policy/development.md
+
+**Audience:** developers, LLM coding agents
+
+Required for projects maintained by humans and LLM coding agents.
+
+It should include:
+
+- repository layout;
+- build/test commands;
+- coding conventions;
+- dependency policy;
+- how to add config fields;
+- how to add CLI flags;
+- how to add modules or adapters, if applicable;
+- how to update examples;
+- documentation update expectations.
+
+### docs/config.md
+
+**Audience:** administrators, operators, advanced users
+
+Required for applications with configuration files.
+
+It should include, in order:
+
+1. config file locations and discovery precedence;
+2. minimal working config;
+3. production-oriented config;
+4. full configuration reference;
+5. secrets handling, if applicable;
+6. links to maintained examples.
+
+The full configuration reference should be canonical.
+
+### docs/cli.md
+
+**Audience:** users, administrators, operators
+
+Required for CLI applications.
+
+It should include, in order:
+
+1. shortest useful command;
+2. command overview;
+3. complete flag reference;
+4. common workflows;
+5. diagnostic or recovery commands, if applicable.
+
+Explain when commands are useful, not just their syntax.
+
+### docs/operations.md
+
+**Audience:** administrators, operators
+
+Required for applications that maintain state, support resume behavior, run multi-step workflows, write durable artifacts, use remote storage, or require recovery procedures.
+
+It should cover:
+
+- normal workflow;
+- filesystem layout;
+- remote storage layout, if applicable;
+- logs and manifests;
+- resume/retry behavior;
+- cleanup behavior;
+- archive/backup behavior;
+- safe recovery procedures;
+- operational caveats.
+
+### docs/troubleshooting.md
+
+**Audience:** administrators, operators
+
+Recommended once recurring failure modes exist.
+
+Each entry should include:
+
+- symptom;
+- likely cause;
+- diagnostic command or inspection step;
+- safe fix;
+- relevant links.
+
+### docs/consumers/
+
+**Audience:** developers and LLM coding agents integrating this project from another codebase
+
+Required for projects with public packages, SDKs, client APIs, plugin APIs, or other application-facing integration surfaces.
+
+This directory describes how an external codebase should consume the project's public API. It should be task-oriented and copyable where useful. It is not the place for internal implementation details or operator procedures.
+
+For projects whose public API is HTTP, `docs/consumers/` is not required, and it should not duplicate the endpoint reference in `docs/api.md`. If present, it may provide practical integration workflows, client-specific examples, or migration notes that link back to `docs/api.md`.
+
+`docs/consumers/api.md` should provide the consumer-facing overview and primary implementation workflow. It should include:
+
+1. intended consumer audience and use cases;
+2. required inputs supplied by operators or deployment configuration;
+3. recommended public package or API workflow;
+4. minimal copyable example;
+5. consumer responsibilities and boundaries;
+6. retry, idempotency, or status behavior, if applicable;
+7. links to package-specific docs and canonical integration contracts.
+
+Package-specific docs should be named `pkg-<name>.md` and should include:
+
+1. import path;
+2. intended use cases;
+3. primary types and functions needed by consumers;
+4. minimal examples;
+5. validation, error, retry, and boundary behavior;
+6. links to canonical file-format or wire-protocol contracts.
+
+### docs/internal/
+
+**Audience:** developers, LLM coding agents
+
+Required for modular, service-oriented, or orchestration projects.
+
+This directory describes implemented internal components. It is not the roadmap.
+
+Use one file per major component where useful.
+
+Each component doc should include:
+
+1. purpose;
+2. inputs and outputs;
+3. boundaries;
+4. config fields used;
+5. external adapters used;
+6. state or manifest behavior, if applicable;
+7. skip/resume behavior, if applicable;
+8. failure behavior;
+9. tests to inspect before changing;
+10. architectural invariants.
+
+### docs/roadmap/
+
+**Audience:** maintainers, developers, LLM coding agents
+
+This is the only place for planned, future, aspirational, experimental, or unimplemented work.
+
+Roadmap docs should clearly distinguish:
+
+- proposed work;
+- accepted plans;
+- deferred ideas;
+- rejected ideas;
+- implementation prompts or task breakdowns, if useful.
+
+Roadmap docs should not be confused with current behavior.
+
+### docs/integrations/
+
+**Audience:** developers, LLM coding agents
+
+Required for projects that depend on external CLIs, APIs, services, protocols, or file formats where the integration contract is important to maintain.
+
+This directory contains concise, versioned reference notes for external integration contracts. It should document only the parts of the external system that this project actually uses or exposes.
+
+For public HTTP API services, `docs/integrations/` should document upstream, downstream, storage, protocol, or runtime contracts that the service depends on or bridges. It should not become a second copy of the public HTTP endpoint reference; that belongs in `docs/api.md`.
+
+Use one file per integration where useful.
+
+## Examples Directory
+
+Projects with non-trivial configuration or workflows should include `examples/`.
+
+Useful examples include:
+
+- minimal working config;
+- production-oriented config;
+- full annotated config;
+- local development config;
+- remote/object-storage config;
+- minimal session/input file.
+
+Examples should be valid, maintained, tested when practical, and linked from relevant docs.
+
+## Security and Privacy
+
+Docs and examples must not include:
+
+- real API keys;
+- tokens;
+- passwords;
+- private keys;
+- private environment dumps;
+- sensitive user data;
+- raw private transcripts;
+- private infrastructure details unless intentionally public.
+
+Document secret-handling mechanisms, not actual secret values.
+
+## Maintenance Rules
+
+When docs change, verify the affected behavior.
+
+Where practical:
+
+- load example config files in tests;
+- test CLI examples or command parser behavior;
+- validate documented flags against real flags;
+- remove stale references;
+- update links after renames;
+- keep roadmap content out of non-roadmap docs.
+
+If documentation and code disagree, fix the documentation and/or open a roadmap item; do not leave aspirational behavior in current-behavior docs.
+
+Documentation is complete only when it matches the current code.
+
+## Documentation Change Checklist
+
+Before merging documentation changes, verify:
+
+- README is concise and orientation-focused.
+- `docs/policy/architecture.md` describes development principles.
+- `docs/api.md` is the canonical HTTP contract for HTTP API services.
+- Future work appears only under `docs/roadmap/`.
+- User-facing docs avoid unnecessary internals.
+- Consumer-facing docs explain public APIs without duplicating HTTP endpoint or integration contracts.
+- Developer-facing docs preserve boundaries and invariants.
+- Config examples match the schema.
+- CLI examples match real commands and flags.
+- Defaults appear in the canonical config reference.
+- No secrets or private data are included.
+- Links are accurate.