ejr/weatherfeeder
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document weatherfeeder CLI and configuration

Browse Source
This commit is contained in:
Eric Rakestraw
2026-06-10 20:07:56 +00:00
parent 77bd59cb87
commit 4ac4e401ed
3 changed files with 305 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

48
README.md
View File

@@ -1,36 +1,30 @@
# weatherfeeder
weatherfeeder is a small daemon that polls weather observations, forecasts, and alerts from multiple upstream
providers, normalizes them into a provider-independent format, and emits them to a sink.
`weatherfeeder` is a config-driven daemon that polls weather providers, normalizes
provider-specific responses into canonical weather events, and routes those
events to configured sinks.
Today, the only implemented sink is `stdout`, which prints JSON-encoded events.
It currently supports NWS observations, alerts, hourly forecasts, narrative
forecasts, forecast discussions, and weather stories; Open-Meteo observations
and hourly forecasts; and OpenWeather observations. Implemented sinks are
stdout, NATS, and Postgres.
## What weatherfeeder emits
## Quickstart
weatherfeeder emits **feed events** encoded as JSON. Each event includes a schema identifier and a payload.
Downstream consumers should key off the `schema` value and decode the `payload` accordingly.
Run the checked-in sample config:
Canonical domain schemas emitted after normalization:
```sh
cd cmd/weatherfeeder
go run .
```
- `weather.observation.v1` → `WeatherObservation`
- `weather.forecast.v1` → `WeatherForecastRun`
- `weather.forecast_discussion.v1` → `WeatherForecastDiscussion`
- `weather.weather_story.v1` → `WeatherStoryRun`
- `weather.alert.v1` → `WeatherAlertRun`
The sample config at `cmd/weatherfeeder/config.yml` is load-tested and can be
used as a starting point. The executable always reads `config.yml` from its
current working directory.
For the complete wire contract (event envelope + payload schemas, fields, units, and compatibility rules), see:
## Documentation
- **API.md**
## Upstream providers (current MVP)
- NWS: observations, hourly forecasts, narrative forecasts, forecast discussions, weather stories, alerts
- Open-Meteo: observations, hourly forecasts
- OpenWeather: observations
## Versioning & compatibility
The JSON field names on canonical payload types are treated as part of the wire contract.
Additive changes are preferred. Renames/removals require a schema version bump.
See **API.md** for details.
- [CLI reference](docs/cli.md)
- [Configuration reference](docs/config.md)
- [Architecture policy](docs/policy/architecture.md)
- [Documentation policy](docs/policy/documentation.md)

70
docs/cli.md Normal file
View File

@@ -0,0 +1,70 @@
# CLI Reference
## Shortest Useful Command
Run `weatherfeeder` from a directory containing `config.yml`:
```sh
cd cmd/weatherfeeder
go run .
```
When using a built binary:
```sh
./weatherfeeder
```
## Command Overview
`weatherfeeder` starts a long-running polling daemon. On startup it:
1. reads `config.yml` from the current working directory;
2. builds configured sources, sinks, and routes;
3. starts polling sources on their configured intervals;
4. normalizes and deduplicates events;
5. dispatches matching events to configured sinks.
The command logs startup, runtime, and shutdown messages to stderr using the Go
standard logger.
## Flags
There are currently no CLI flags, subcommands, or environment-variable based
configuration controls.
The config path is fixed at `config.yml` relative to the process current working
directory. To run with a different config, change the working directory or place
the desired file at that path.
## Common Workflows
Run the checked-in sample config:
```sh
cd cmd/weatherfeeder
go run .
```
Build and run a local binary:
```sh
go build -o weatherfeeder ./cmd/weatherfeeder
cp cmd/weatherfeeder/config.yml .
./weatherfeeder
```
Run in the project container image with a mounted config:
```sh
docker run --rm -v "$PWD/config.yml:/weatherfeeder/config.yml:ro" weatherfeeder
```
The Docker image sets `/weatherfeeder` as the working directory, so the mounted
file must appear at `/weatherfeeder/config.yml`.
## Shutdown
Stop the daemon with `Ctrl-C` or `SIGTERM`. The process uses context-aware
shutdown for scheduler, dispatcher, processors, sources, and sinks, then logs
`shutdown complete`.

214
docs/config.md Normal file
View File

@@ -0,0 +1,214 @@
# Configuration Reference
## Config File
`weatherfeeder` reads exactly one YAML file named `config.yml` from the current
working directory. There is no config path flag and no search path.
YAML decoding is strict for config struct fields: misspelled fields such as
`sources[].drviver` fail startup. Driver-specific `params` maps are validated by
the source or sink constructor that consumes them.
The top-level file contains:
```yaml
sources:
- name: NWSObservationKSTL
mode: poll
driver: nws_observation
every: 10m
kinds: ["observation"]
params:
url: "https://api.weather.gov/stations/KSTL/observations/latest"
user_agent: "Example weatherfeeder operator (ops@example.com)"
sinks:
- name: stdout
driver: stdout
params: {}
routes:
- sink: stdout
kinds: ["observation"]
```
`sources` and `sinks` must each contain at least one entry. `routes` is optional.
When `routes` is omitted, every configured sink receives every event kind.
## Production-Oriented Shape
A typical deployment uses multiple polling sources and sends the same canonical
event stream to a broker or database:
```yaml
sources:
- name: NWSAlertsLocal
mode: poll
driver: nws_alerts
every: 1m
kinds: ["alert"]
params:
url: "https://api.weather.gov/alerts?point=38.6239,-90.3571&limit=20"
user_agent: "Example weatherfeeder operator (ops@example.com)"
sinks:
- name: nats_weather
driver: nats
params:
url: nats://nats:4222
subject: weatherfeeder
- name: pg_weather
driver: postgres
params:
uri: postgres://weatherdb:5432/weatherdb?sslmode=disable
username: weatherdb
password: "<set outside source control>"
prune: 3d
routes:
- sink: nats_weather
kinds: ["observation", "forecast", "forecast_discussion", "weather_story", "alert"]
- sink: pg_weather
kinds: ["observation", "forecast", "forecast_discussion", "weather_story", "alert"]
```
Do not commit real API keys, database passwords, or personal contact addresses in
copyable configs.
## Top-Level Fields
| Field | Required | Description |
|---|:---:|---|
| `sources` | yes | List of configured input sources. |
| `sinks` | yes | List of configured output sinks. |
| `routes` | no | List of sink routing rules. If omitted, all sinks receive all kinds. |
## Source Fields
| Field | Required | Description |
|---|:---:|---|
| `name` | yes | Unique source name. Used as the event source identifier. |
| `driver` | yes | Source driver name. |
| `mode` | no | `poll`, `stream`, or omitted for auto. Current weatherfeeder drivers are polling drivers. |
| `every` | yes | Poll interval for current weatherfeeder source drivers. |
| `kinds` | no | Expected event kinds. If present, startup verifies they match the source driver. |
| `params` | driver-specific | Driver parameters. Current source drivers require HTTP params. |
Current event kinds are `observation`, `forecast`, `forecast_discussion`,
`weather_story`, and `alert`.
## Source Drivers
| Driver | Kind | Upstream product |
|---|---|---|
| `nws_observation` | `observation` | NWS station latest observation. |
| `nws_alerts` | `alert` | NWS alerts collection. |
| `nws_forecast_hourly` | `forecast` | NWS hourly gridpoint forecast. |
| `nws_forecast_narrative` | `forecast` | NWS narrative gridpoint forecast. |
| `nws_forecast_discussion` | `forecast_discussion` | NWS forecast discussion HTML product. |
| `nws_weatherstories` | `weather_story` | NWS office weather stories. |
| `openmeteo_observation` | `observation` | Open-Meteo current conditions. |
| `openmeteo_forecast` | `forecast` | Open-Meteo hourly forecast. |
| `openweather_observation` | `observation` | OpenWeather current weather. |
## HTTP Source Params
All current source drivers use the shared HTTP polling helper.
| Param | Required | Description |
|---|:---:|---|
| `url` | yes | Full upstream request URL. `URL` is also accepted by the helper. |
| `user_agent` | yes | User-Agent sent to the upstream provider. `userAgent` is also accepted by the helper. |
| `conditional` | no | Boolean. Defaults to `true`; enables ETag and Last-Modified conditional requests. |
| `http_timeout` | no | Positive duration for the HTTP client timeout. |
| `http_response_body_limit_bytes` | no | Positive integer response body limit in bytes. |
When `conditional` is enabled and the upstream returns `304 Not Modified`, the
source emits no events for that poll.
OpenWeather observation URLs must include `units=metric`. Startup fails if the
URL omits it or sets another unit system.
## Sink Fields
| Field | Required | Description |
|---|:---:|---|
| `name` | yes | Unique sink name. Routes refer to this value. |
| `driver` | yes | Sink driver name. |
| `params` | driver-specific | Sink parameters. |
## Sink Drivers
### `stdout`
Prints each event as JSON to stdout.
```yaml
sinks:
- name: stdout
driver: stdout
params: {}
```
### `nats`
Publishes each event as JSON to a NATS subject.
| Param | Required | Description |
|---|:---:|---|
| `url` | yes | NATS server URL, such as `nats://localhost:4222`. |
| `subject` | yes | Subject to publish events to. |
### `postgres`
Writes supported canonical weather events to Postgres using weatherfeeder's
registered schema mapping.
| Param | Required | Description |
|---|:---:|---|
| `uri` | yes | PostgreSQL connection URI. |
| `username` | yes | Database username. |
| `password` | yes | Database password. |
| `prune` | no | Retention window. If set, rows older than the window are pruned on each write transaction. |
`prune` accepts Go duration strings such as `72h`, plus day and week suffixes
such as `3d` and `2w`.
## Routes
Routes connect event kinds to sinks:
```yaml
routes:
- sink: stdout
kinds: ["observation", "alert"]
```
| Field | Required | Description |
|---|:---:|---|
| `sink` | yes | Name of a configured sink. |
| `kinds` | no | Event kinds to send to that sink. Omit or use an empty list to match all kinds. |
Route `kinds` values are trimmed and lowercased by the dispatcher. Blank entries
are rejected.
## Duration Formats
Top-level source `every` accepts:
- Go duration strings such as `30s`, `10m`, or `1h`;
- integer values, interpreted as minutes;
- numeric strings such as `"15"`, also interpreted as minutes.
HTTP param durations such as `http_timeout` accept Go duration strings. Numeric
values and numeric strings are interpreted as seconds.
Postgres `prune` must be a string duration.
## Secrets
The config file is read directly from disk and has no built-in secret expansion.
Keep real credentials out of repository-tracked configs. Use deployment tooling
to render `config.yml` with the needed secret values before starting the daemon.