From ec027e34cc4773e51da1d4559c903e411a22935e Mon Sep 17 00:00:00 2001
From: Eric Rakestraw <eric@rakestraw.us>
Date: Thu, 11 Jun 2026 14:16:22 +0000
Subject: [PATCH] Add operations and troubleshooting documentation

---
 docs/operations.md      | 185 +++++++++++++++++++++++++++
 docs/troubleshooting.md | 275 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 460 insertions(+)
 create mode 100644 docs/operations.md
 create mode 100644 docs/troubleshooting.md

diff --git a/docs/operations.md b/docs/operations.md
new file mode 100644
index 0000000..5eae3c3
--- /dev/null
+++ b/docs/operations.md
@@ -0,0 +1,185 @@
+# weatherapi Operations
+
+`weatherapi` is a read-only HTTP service. It serves latest weather records from
+Postgres tables populated by `weatherfeeder`; it does not ingest provider data,
+create tables, or run database migrations.
+
+## Runtime Model
+
+At startup, the executable:
+
+1. resolves the config path from `-config`, `WEATHERAPI_CONFIG`, or `config.yml`;
+2. loads feedapi YAML configuration;
+3. opens all configured database handles;
+4. selects the first configured database as the primary weather data store;
+5. registers HTTP endpoints and text templates with feedapi;
+6. starts the HTTP server.
+
+See [`docs/cli.md`](cli.md) for invocation details and [`docs/config.md`](config.md)
+for configuration fields.
+
+## Prerequisites
+
+- A reachable PostgreSQL database.
+- Weatherfeeder-owned weather tables already created and populated.
+- A config file with at least one `databases` entry.
+- The `templates/` directory when `format=text` responses are needed.
+- Network access from the service process or container to Postgres.
+
+The first database entry in the config is the only database used for weather
+reads. Additional configured handles may be opened by feedapi, but `weatherapi`
+selects the first entry as primary.
+
+## Local Run
+
+Use the checked-in local sample only when its database settings match your
+environment:
+
+```sh
+go run ./cmd/weatherapi -config config.yml
+```
+
+For a local binary:
+
+```sh
+go build -o ./weatherapi ./cmd/weatherapi
+./weatherapi -config config.yml
+```
+
+Keep the process working directory aligned with `templates.base_dir`. With the
+checked-in config, run from the repository root so `templates` resolves to the
+repository's template directory.
+
+## Container Run
+
+The Docker image copies these runtime files into `/weatherapi`:
+
+- `/weatherapi/weatherapi`
+- `/weatherapi/config.yml`
+- `/weatherapi/templates`
+
+The runtime working directory is `/weatherapi`, and the entrypoint is the
+`weatherapi` binary. To use the image's bundled config and templates:
+
+```sh
+docker run --rm -p 8080:8080 weatherapi
+```
+
+To provide an external config file:
+
+```sh
+docker run --rm -p 8080:8080 \
+  -v "$PWD/examples/config.production.yml:/weatherapi/config.yml:ro" \
+  weatherapi
+```
+
+To use a different config path, append the CLI flag:
+
+```sh
+docker run --rm -p 8080:8080 \
+  -v "$PWD/config.yml:/config/weatherapi.yml:ro" \
+  weatherapi -config /config/weatherapi.yml
+```
+
+If you mount a custom template directory, make `templates.base_dir` point to the
+mounted path.
+
+The repository's Woodpecker image build uses Kaniko and passes private Gitea
+credentials as build arguments so Go can download private modules during the
+build.
+
+## Database Dependency
+
+`weatherapi` expects weatherfeeder-compatible tables for observations, current
+conditions aggregation, active alerts, forecasts, forecast discussions, and
+weather stories. It only reads those tables.
+
+Operational ownership is split:
+
+- `weatherfeeder` owns provider polling, normalization, writes, table creation,
+  and schema compatibility.
+- PostgreSQL owns durable storage, backups, replication, and restore.
+- `weatherapi` owns serving read-only HTTP responses from the configured
+  primary database.
+
+When restoring from backup or replacing the database, verify that weatherfeeder
+has resumed writes before treating stale `weatherapi` responses as an API
+problem.
+
+## Templates and Text Output
+
+JSON and XML responses do not depend on text templates. `format=text` uses the
+template named by each endpoint, with templates stored under `templates.base_dir`.
+
+The repository includes templates for all implemented endpoint families:
+observations, current conditions, active alerts, hourly forecasts, narrative
+forecasts, forecast discussions, and weather stories.
+
+If text rendering fails or returns an unsupported-format error, verify:
+
+- the process can read `templates.base_dir`;
+- expected `*.txt.tmpl` files are present;
+- the requested endpoint supports `format=text`, which all implemented
+  endpoints currently do.
+
+## Startup and Shutdown
+
+Startup failures are fatal and are logged with the prefix `weatherapi failed`.
+Common failure contexts include:
+
+- `load config`
+- `config.databases requires at least one entry`
+- `open databases`
+- `select primary database`
+- `build app`
+
+The process handles `SIGINT` and `SIGTERM`. Feedapi performs graceful HTTP
+shutdown after the runtime context is canceled. Database close errors during
+shutdown are logged, but they occur after the service has already begun stopping.
+
+## Verification
+
+There is no dedicated health endpoint in `weatherapi`. Use an implemented read
+endpoint as a practical service check:
+
+```sh
+curl -i 'http://localhost:8080/observations?format=json'
+```
+
+Interpretation:
+
+- `200 OK` with `{"data": null}` means the service is running but no latest
+  observation is available.
+- `200 OK` with a populated `data` object means the service and that read path
+  are working.
+- `400 Bad Request` indicates request validation, not service health.
+- `406 Not Acceptable` indicates format negotiation.
+- `5xx` indicates a runtime, database, or handler failure.
+
+For a broader smoke check, use the requests in
+[`examples/requests.http`](../examples/requests.http).
+
+## Logs
+
+The executable uses Go's standard logger with date, time, and microseconds.
+Startup and fatal runtime errors are written to standard error. Container
+platforms should collect stdout and stderr from the process.
+
+## Backup and Recovery Boundaries
+
+Back up and restore PostgreSQL using normal database procedures. `weatherapi`
+has no local durable weather state to back up.
+
+After a database restore or failover:
+
+1. confirm the configured database is reachable from the service;
+2. confirm weatherfeeder-owned tables exist;
+3. confirm weatherfeeder is writing fresh rows if current data is expected;
+4. restart `weatherapi` if database connection settings changed;
+5. verify with an implemented endpoint such as `/observations`.
+
+## Related Docs
+
+- [`docs/api.md`](api.md): HTTP contract and error envelope
+- [`docs/config.md`](config.md): configuration reference
+- [`docs/troubleshooting.md`](troubleshooting.md): symptom-oriented fixes
diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md
new file mode 100644
index 0000000..54d40f4
--- /dev/null
+++ b/docs/troubleshooting.md
@@ -0,0 +1,275 @@
+# weatherapi Troubleshooting
+
+Use this guide to separate startup, database, template, request-validation, and
+no-data problems. For full command and configuration reference, see
+[`docs/cli.md`](cli.md) and [`docs/config.md`](config.md).
+
+## Startup Fails with `load config`
+
+Symptom: the process exits and logs `weatherapi failed: load config: ...`.
+
+Likely causes:
+
+- the config path is wrong;
+- the process working directory is not where `config.yml` is expected;
+- YAML syntax is invalid;
+- required feedapi config fields are missing.
+
+Diagnostic:
+
+```sh
+ls -l config.yml
+python3 -c 'import yaml; yaml.safe_load(open("config.yml"))'
+```
+
+Safe fix: pass the intended file explicitly with `-config`, or set a non-blank
+`WEATHERAPI_CONFIG`. Fix YAML syntax and compare the file with
+[`examples/config.minimal.yml`](../examples/config.minimal.yml).
+
+## Startup Fails with `config.databases requires at least one entry`
+
+Symptom: the process exits before opening any database.
+
+Likely cause: `databases` is missing or empty in the YAML file.
+
+Diagnostic:
+
+```sh
+rg -n '^databases:' config.yml
+```
+
+Safe fix: add at least one database entry. The first entry is the primary
+weather data store used for reads.
+
+## Startup Fails with `open databases`
+
+Symptom: the process exits while opening configured database handles.
+
+Likely causes:
+
+- PostgreSQL host or port is unreachable;
+- credentials are invalid;
+- the database name in `uri` is wrong;
+- TLS/`sslmode` settings do not match the server;
+- the configured driver is not usable for this deployment.
+
+Diagnostic:
+
+```sh
+psql 'postgres://USER:PASSWORD@HOST:5432/DBNAME?sslmode=disable' -c 'select 1'
+```
+
+Safe fix: correct `uri`, `username`, `password`, network routing, firewall
+rules, or TLS settings. Keep production secrets out of committed config files.
+
+## Startup Fails with `select primary database`
+
+Symptom: databases open, then the process exits selecting the primary database.
+
+Likely cause: feedapi opened a registry that does not contain the first
+configured database name.
+
+Diagnostic: inspect the first `databases` item in the active config and verify
+that `name` is present and non-empty.
+
+Safe fix: give the first database entry a stable `name` and keep it as the
+weather database entry.
+
+## Startup Fails with `build app`
+
+Symptom: the process exits after database setup but before serving HTTP.
+
+Likely causes:
+
+- feedapi cannot construct the HTTP app from the config;
+- renderer or template setup failed;
+- endpoint registration failed.
+
+Diagnostic: read the error text after `build app:`. If the error mentions
+templates, inspect `templates.base_dir`.
+
+Safe fix: correct the config value mentioned by the error. For template errors,
+make the directory readable and ensure the repository's `*.txt.tmpl` files are
+present.
+
+## Requests Return `400 invalid_parameter`
+
+Symptom: JSON error body includes:
+
+```json
+{
+  "error": {
+    "code": "invalid_parameter",
+    "message": "..."
+  }
+}
+```
+
+Likely causes:
+
+- unknown query parameter;
+- `precision` outside `0` through `2`, or not an integer;
+- `precision` used on alerts, discussions, or weather stories;
+- `tz` used on observations, current conditions, or alerts;
+- invalid timezone value;
+- both `tz` and `TZ` are present with different values.
+
+Diagnostic: compare the request against the route's query parameters in
+[`docs/api.md`](api.md). Reproduce with `curl -i` to see the status and body.
+
+Safe fix: remove unsupported parameters or correct values. Use `tz=Chicago`,
+`tz=America/Chicago`, a supported US abbreviation, or an offset such as `TZ=-5`
+on routes that accept timezone selection.
+
+## Requests Return `406 unsupported_format`
+
+Symptom: the response status is `406 Not Acceptable` and the error code is
+`unsupported_format`.
+
+Likely causes:
+
+- `format` has a value other than `json`, `xml`, or `text`;
+- the `Accept` header cannot be matched to a registered renderer;
+- the configured default format is unsupported.
+
+Diagnostic:
+
+```sh
+curl -i 'http://localhost:8080/observations?format=json'
+curl -i -H 'Accept: application/json' 'http://localhost:8080/observations'
+```
+
+Safe fix: request `format=json`, `format=xml`, or `format=text`, or set
+`server.default_format` to a supported value.
+
+## Text Responses Fail or Do Not Render Expected Text
+
+Symptom: `format=text` fails, returns an error, or does not contain expected
+endpoint text.
+
+Likely causes:
+
+- `templates.base_dir` points to the wrong directory;
+- templates were not copied into the runtime container or deployment directory;
+- file permissions prevent reading templates;
+- a customized template has invalid syntax.
+
+Diagnostic:
+
+```sh
+find templates -maxdepth 1 -name '*.txt.tmpl' -print | sort
+curl -i 'http://localhost:8080/forecast/hourly?format=text'
+```
+
+Safe fix: restore the repository's `templates/` directory or update
+`templates.base_dir` to the mounted template path. In the Docker image, the
+default template directory is `/weatherapi/templates`.
+
+## Successful Response Has `data: null`
+
+Symptom: the response is `200 OK`, but JSON contains `"data": null`.
+
+Likely causes:
+
+- weatherfeeder has not populated the relevant table yet;
+- the relevant latest row does not exist after a database restore;
+- the service is pointed at an empty or wrong database;
+- current conditions have no observations in the implemented 30-minute window.
+
+Diagnostic:
+
+```sh
+curl -s 'http://localhost:8080/observations?format=json'
+curl -s 'http://localhost:8080/conditions/current?format=json'
+```
+
+Safe fix: verify weatherfeeder is running and writing to the same database that
+`weatherapi` uses as its first configured database. For current conditions,
+wait for recent observations or inspect weatherfeeder ingestion.
+
+## Forecast Day Routes Return Empty `periods`
+
+Symptom: `/forecast/hourly/today`, `/forecast/hourly/tomorrow`,
+`/forecast/narrative/today`, or `/forecast/narrative/tomorrow` returns a
+forecast object with an empty `periods` array.
+
+Likely causes:
+
+- no period `startTime` falls on that calendar day in the selected timezone;
+- the request omitted `tz`, so UTC day boundaries were used;
+- the stored forecast is stale.
+
+Diagnostic:
+
+```sh
+curl -s 'http://localhost:8080/forecast/hourly/today?tz=America/Chicago'
+curl -s 'http://localhost:8080/forecast/hourly?tz=America/Chicago'
+```
+
+Safe fix: provide the intended `tz` value and verify the unfiltered forecast
+contains periods for the expected local date.
+
+## Timezone Errors
+
+Symptom: requests with `tz` or `TZ` return `400 invalid_parameter`.
+
+Likely causes:
+
+- the timezone is not an IANA location, supported US abbreviation, supported
+  alias, or valid UTC offset;
+- an offset is outside `-14:00` through `+14:00`;
+- both `tz` and `TZ` are present but do not match case-insensitively.
+
+Diagnostic:
+
+```sh
+curl -i 'http://localhost:8080/forecast/hourly?tz=not-a-timezone'
+curl -i 'http://localhost:8080/forecast/hourly?tz=CDT&TZ=EST'
+```
+
+Safe fix: use one timezone parameter. Known-good examples include
+`tz=America/Chicago`, `tz=Chicago`, `tz=CDT`, and `TZ=-5`.
+
+## Precision Errors
+
+Symptom: requests with `precision` return `400 invalid_parameter`.
+
+Likely causes:
+
+- value is not an integer;
+- value is less than `0` or greater than `2`;
+- route does not accept `precision`.
+
+Diagnostic:
+
+```sh
+curl -i 'http://localhost:8080/conditions/current?precision=3'
+curl -i 'http://localhost:8080/alerts/active?precision=1'
+```
+
+Safe fix: use `precision=0`, `precision=1`, or `precision=2` only on
+observations, current conditions, and forecast routes.
+
+## Missing or Stale Weather Data
+
+Symptom: responses are successful but older than expected, or endpoint families
+are empty.
+
+Likely causes:
+
+- weatherfeeder is stopped or failing;
+- weatherfeeder writes to a different database than `weatherapi` reads;
+- a restore or deployment changed database credentials;
+- upstream provider ingestion is delayed outside `weatherapi`.
+
+Diagnostic: inspect weatherfeeder logs and query the configured Postgres
+database directly for recent rows in the relevant weatherfeeder tables.
+
+Safe fix: repair weatherfeeder ingestion or database routing. Restart
+`weatherapi` only when config or database connectivity changed.
+
+## Related Docs
+
+- [`docs/api.md`](api.md): endpoint contract, query parameters, errors
+- [`docs/config.md`](config.md): YAML fields and examples
+- [`docs/operations.md`](operations.md): runtime and recovery boundaries