weatherfeeder/docs/operations.md

# Operations

This document describes how to run and observe the `weatherfeeder` daemon in its
current form. For configuration syntax, see [configuration](config.md). For the
CLI surface, see [CLI reference](cli.md).

## Normal Workflow

1. Prepare `config.yml` in the process working directory.
2. Start the daemon with `./weatherfeeder` or `go run .` from
   `cmd/weatherfeeder`.
3. Watch stderr logs for startup or runtime errors.
4. Consume events from the configured sinks.
5. Stop the process with `Ctrl-C` or `SIGTERM`.

The daemon has no admin subcommands and no runtime reload command. Change the
config file and restart the process to apply configuration changes.

Maintained copyable configs are available under [`examples/`](../examples/).

## Runtime Lifecycle

On startup, `weatherfeeder`:

1. loads `config.yml` from the current working directory;
2. registers built-in source drivers;
3. registers stdout, NATS, and weatherfeeder Postgres sink drivers;
4. builds sources and validates configured `kinds` against source metadata;
5. builds sinks and compiles routes;
6. starts the scheduler and dispatcher;
7. processes events through normalization, then in-memory dedupe;
8. routes processed events to configured sinks.

Startup errors are fatal and terminate the process. Runtime poll, pipeline, and
sink errors are logged and the process continues unless the scheduler or
dispatcher returns a fatal error.

## Logs

The process uses the Go standard logger with date, time, and microseconds. Logs
go to stderr.

Common log prefixes:

| Prefix | Meaning |
|---|---|
| `config load failed` | `config.yml` could not be read, parsed, or validated. |
| `build source failed` | A source driver or its params are invalid. |
| `source expected kinds validation failed` | Configured source `kinds` do not match the source driver. |
| `build sink failed` | A sink driver or its params are invalid, or a sink could not initialize. |
| `compile routes failed` | Routes reference invalid sinks or kinds. |
| `scheduler: poll failed` | A source poll failed; the source will be polled again on its next interval. |
| `dispatcher: pipeline error` | Normalization or dedupe failed for one event. |
| `dispatch: sink ... failed consuming event` | A sink failed to consume one event. |
| `shutdown complete` | Scheduler and dispatcher have exited. |

## Scheduling And Polling

Current weatherfeeder sources are polling sources. Each source uses its
configured `every` interval. The scheduler applies jitter before the first poll
and before each interval tick. If no jitter is configured in code, feedkit uses
`min(every/10, 30s)`, capped at half the interval.

Poll failures are logged and do not stop the daemon. A failed poll emits no
events for that source until a subsequent poll succeeds.

## Unchanged Upstream Content

Most sources use feedkit's single-document HTTP polling helper. By default,
`params.conditional` is `true` for those sources, so the helper keeps ETag and
Last-Modified validators in memory for each source instance.

If the upstream returns `304 Not Modified`, the source emits no events for that
poll. Validator state is in memory only; restarting the process starts with no
cached validators.

The SPC convective outlook source polls multiple required documents as one
bundle. It emits no event when a later complete bundle has the same fetched
document bodies as the previous successful poll.

## Processing And Dedupe

Every event passes through normalization first and dedupe second.

Normalizers match raw source schemas and produce canonical `weather.*.v1`
payloads. If an event has no matching normalizer, the normalize processor passes
it through unchanged.

Dedupe keys by event ID and stores a bounded in-memory set of 2048 recent IDs.
Duplicate IDs are dropped. Dedupe state is not persisted, so a restart starts
with an empty dedupe set.

## Routing And Sink Fanout

Routes choose sinks by event kind. If `routes` is omitted, every sink receives
every event kind. If a route omits `kinds`, that route also matches all kinds.

The dispatcher creates one queue and one worker goroutine per sink. The default
per-sink queue size is 64. `weatherfeeder` does not currently expose config
fields for sink queue size, enqueue timeout, or consume timeout.

Sink errors are logged per event. A sink failure does not stop other sinks from
receiving the same event.

## Sink Behavior

### stdout

The stdout sink validates each event and prints one JSON object per line to
stdout. This is useful for local inspection and log forwarding.

### NATS

The NATS sink connects lazily on the first event, reuses the connection while it
is open, and publishes each event as JSON to the configured subject. Connection,
marshal, and publish failures are logged by the dispatch worker.

### Postgres

The Postgres sink opens the database during startup. It creates missing tables
and indexes with `CREATE TABLE IF NOT EXISTS` and `CREATE INDEX IF NOT EXISTS`.
It does not modify existing table definitions.

Each mapped canonical event is written in one transaction. If `params.prune` is
set, the sink deletes rows older than `now - prune` from every weatherfeeder
table in that same transaction. See the
[Postgres table contract](integrations/postgres.md).

## State And Recovery

`weatherfeeder` keeps only runtime state in process memory:

- scheduler goroutines and timers;
- HTTP conditional request validators;
- source-local unchanged-content state;
- event channel buffers;
- per-sink fanout queues;
- the dedupe ID set.

Durable state is external sink state: NATS broker state outside this process and
Postgres tables managed by the configured database.

There is no internal checkpoint, replay log, or resume marker. To recover from a
process failure, fix the underlying issue and restart the daemon from a working
directory containing the desired `config.yml`.

## Shutdown

`weatherfeeder` listens for `os.Interrupt` and `SIGTERM`. On shutdown, the
shared context is canceled. Scheduler jobs stop polling, dispatch workers stop,
and the process logs `shutdown complete`.

Queued sink work may be dropped when shutdown context cancellation reaches the
fanout workers. Use external sink durability, such as Postgres or broker
retention, for durable downstream state.

## Caveats

- There is no health-check endpoint.
- There is no runtime config reload.
- There are no built-in metrics.
- Source conditional request state and dedupe state are reset by restart.
- Existing Postgres schemas are not migrated automatically.