ejr/weatherfeeder
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document event and Postgres integration contracts

Browse Source
This commit is contained in:
Eric Rakestraw
2026-06-10 20:12:27 +00:00
parent 4ac4e401ed
commit abb8f218ec
6 changed files with 658 additions and 407 deletions
Download Patch File Download Diff File Expand all files Collapse all files

406
API.md
View File

@@ -1,404 +1,4 @@
# weatherfeeder API (Wire Contract)
# Event Wire Contract
This document defines the stable, consumer-facing JSON contract emitted by weatherfeeder sinks.
weatherfeeder emits **events** encoded as JSON. Each event has:
- an **envelope** (metadata + schema identifier), and
- a **payload** whose shape is determined by `schema`.
Downstream consumers should:
1. parse the event envelope,
2. switch on `schema`, then
3. decode `payload` into the matching schema.
---
## Event envelope
All events are JSON objects with these fields:
| Field | Type | Required | Notes |
|---|---:|:---:|---|
| `id` | string | yes | Stable event identifier. Treat as opaque. |
| `schema` | string | yes | Schema identifier (e.g. `weather.observation.v1`). |
| `source` | string | yes | Provider/source identifier (stable within configuration). |
| `effectiveAt` | string (timestamp) | yes | RFC3339Nano timestamp indicating when this event is effective. |
| `payload` | object | yes | Schema-specific payload (see below). |
### Timestamp format
All timestamps are encoded as JSON strings using Go’s `time.Time` JSON encoding (RFC3339Nano).
Examples:
- `"2026-01-17T14:27:00Z"`
- `"2026-01-17T08:27:00-06:00"`
---
## Canonical schemas
weatherfeeder emits five canonical domain schemas:
- `weather.observation.v1`
- `weather.forecast.v1`
- `weather.forecast_discussion.v1`
- `weather.weather_story.v1`
- `weather.alert.v1`
Each payload is described below using the JSON field names as the contract.
### Raw upstream schemas
weatherfeeder sources also emit provider-specific raw schemas before normalization.
Relevant raw source schemas include:
- `raw.nws.forecast_discussion.v1`
- payload type: string
- payload contents: exact fetched HTML response body
- `raw.nws.weatherstories.v1`
- payload type: object
- payload contents: exact fetched JSON response body
---
## Shared Conventions
- Timestamps are JSON strings in RFC3339Nano format.
- Optional fields are omitted when unknown (`omitempty` behavior).
- Numeric measurements are normalized to metric units:
- `*C` = Celsius
- `*Kmh` = kilometers/hour
- `*Pa` = Pascals
- `*Meters` = meters
- `*Mm` = millimeters
- `*Percent` = percent (0-100)
- `conditionCode` is a WMO weather interpretation code (`int`).
- Unknown/unmappable is `-1`.
- Downstream consumers should treat unknown codes as “unknown conditions” rather than failing decoding.
- For readability and stability, weatherfeeder rounds floating-point values in canonical payloads to
**4 digits after the decimal** during normalization.
---
## Schema: `weather.observation.v1`
Payload type: `WeatherObservation`
A `WeatherObservation` represents a point-in-time observation for a station/location.
### Fields
| Field | Type | Required | Notes |
|---|---:|:---:|---|
| `stationId` | string | no | Provider station/location identifier |
| `stationName` | string | no | Human station name |
| `timestamp` | timestamp string | yes | Observation timestamp |
| `conditionCode` | int | yes | WMO code (`-1` unknown) |
| `isDay` | bool | no | Day/night hint |
| `textDescription` | string | no | Human-facing short description |
| `temperatureC` | number | no | Celsius |
| `dewpointC` | number | no | Celsius |
| `windDirectionDegrees` | number | no | Degrees |
| `windSpeedKmh` | number | no | km/h |
| `windGustKmh` | number | no | km/h |
| `barometricPressurePa` | number | no | Pascals |
| `visibilityMeters` | number | no | Meters |
| `relativeHumidityPercent` | number | no | Percent |
| `apparentTemperatureC` | number | no | Celsius |
| `presentWeather` | array | no | Provider-specific structured weather fragments |
### Nested: `presentWeather[]`
Each `presentWeather[]` element:
| Field | Type | Required | Notes |
|---|---:|:---:|---|
| `raw` | object | no | Provider-specific JSON object |
---
## Schema: `weather.forecast.v1`
Payload type: `WeatherForecastRun`
A `WeatherForecastRun` is a single issued forecast snapshot for a location and a specific product
(hourly / narrative / daily). The run contains an ordered list of forecast periods.
### `product` values
`product` is one of:
- `"hourly"`
- `"narrative"`
- `"daily"`
### Fields
| Field | Type | Required | Notes |
|---|---:|:---:|---|
| `locationId` | string | no | Provider location identifier |
| `locationName` | string | no | Human name, if available |
| `issuedAt` | string (timestamp) | yes | When this run was generated/issued |
| `updatedAt` | string (timestamp) | no | Optional later update time |
| `product` | string | yes | One of `hourly`, `narrative`, `daily` |
| `latitude` | number | no | Degrees |
| `longitude` | number | no | Degrees |
| `elevationMeters` | number | no | meters |
| `periods` | array | yes | Chronological forecast periods |
### Nested: `periods[]` (`WeatherForecastPeriod`)
A `WeatherForecastPeriod` is valid for `[startTime, endTime)`.
| Field | Type | Required | Units / Notes |
|---|---:|:---:|---|
| `startTime` | string (timestamp) | yes | Period start |
| `endTime` | string (timestamp) | yes | Period end |
| `name` | string | no | Human label (often empty for hourly) |
| `isDay` | bool | no | Day/night hint |
| `conditionCode` | int | no | WMO code when applicable (`-1` for unknown) |
| `textDescription` | string | no | Human-facing short phrase |
| `temperatureC` | number | no | °C |
| `temperatureCMin` | number | no | °C (aggregated products) |
| `temperatureCMax` | number | no | °C (aggregated products) |
| `dewpointC` | number | no | °C |
| `relativeHumidityPercent` | number | no | percent |
| `windDirectionDegrees` | number | no | degrees |
| `windSpeedKmh` | number | no | km/h |
| `windGustKmh` | number | no | km/h |
| `barometricPressurePa` | number | no | Pa |
| `visibilityMeters` | number | no | meters |
| `apparentTemperatureC` | number | no | °C |
| `cloudCoverPercent` | number | no | percent |
| `probabilityOfPrecipitationPercent` | number | no | percent |
| `precipitationAmountMm` | number | no | mm (liquid equivalent) |
| `snowfallDepthMm` | number | no | mm |
| `uvIndex` | number | no | unitless index |
---
## Schema: `weather.alert.v1`
Payload type: `WeatherAlertRun`
A `WeatherAlertRun` is a snapshot of *active* alerts for a location as-of a point in time.
A run may contain zero, one, or many alerts.
### Fields
| Field | Type | Required | Notes |
|---|---:|:---:|---|
| `locationId` | string | no | Provider location identifier |
| `locationName` | string | no | Human name, if available |
| `asOf` | string (timestamp) | yes | When the provider asserted this snapshot is current |
| `latitude` | number | no | Degrees |
| `longitude` | number | no | Degrees |
| `alerts` | array | yes | Active alerts (order provider-dependent) |
### Nested: `alerts[]` (`WeatherAlert`)
| Field | Type | Required | Notes |
|---|---:|:---:|---|
| `id` | string | yes | Provider-stable identifier (often a URL/URI) |
| `event` | string | no | Classification/event label |
| `headline` | string | no | Alert headline |
| `severity` | string | no | Example: Extreme/Severe/Moderate/Minor/Unknown |
| `urgency` | string | no | Example: Immediate/Expected/Future/Past/Unknown |
| `certainty` | string | no | Example: Observed/Likely/Possible/Unlikely/Unknown |
| `status` | string | no | Example: Actual/Exercise/Test/System/Unknown |
| `messageType` | string | no | Example: Alert/Update/Cancel |
| `category` | string | no | Example: Met/Geo/Safety/Rescue/Fire/Health/Env/Transport/Infra/CBRNE/Other |
| `response` | string | no | Example: Shelter/Evacuate/Prepare/Execute/Avoid/Monitor/Assess/AllClear/None |
| `response` | string | no | e.g. Shelter/Evacuate/Prepare/... |
| `description` | string | no | Narrative |
| `instruction` | string | no | What to do |
| `sent` | string (timestamp) | no | Provider-dependent |
| `effective` | string (timestamp) | no | Provider-dependent |
| `onset` | string (timestamp) | no | Provider-dependent |
| `expires` | string (timestamp) | no | Provider-dependent |
| `areaDescription` | string | no | Often a provider string |
| `senderName` | string | no | Provenance |
| `references` | array | no | Related alert references |
### Nested: `references[]` (`AlertReference`)
| Field | Type | Required | Notes |
|---|---:|:---:|---|
| `id` | string | no | Provider reference ID/URI |
| `identifier` | string | no | Provider identifier string, if distinct |
| `sender` | string | no | Sender |
| `sent` | string (timestamp) | no | Timestamp |
---
## Schema: `weather.weather_story.v1`
Payload type: `WeatherStoryRun`
A `WeatherStoryRun` is a snapshot of NWS weather stories for an office as-of a point in time.
The run may contain zero, one, or many stories.
### Fields
| Field | Type | Required | Notes |
|---|---:|:---:|---|
| `officeId` | string | no | NWS office identifier, e.g. `LSX` |
| `asOf` | string (timestamp) | yes | Latest story update time or source fallback |
| `stories` | array | yes | Weather stories (order provider-dependent) |
### Nested: `stories[]` (`WeatherStory`)
| Field | Type | Required | Notes |
|---|---:|:---:|---|
| `officeId` | string | no | NWS office identifier |
| `startTime` | string (timestamp) | yes | Story validity start |
| `endTime` | string (timestamp) | yes | Story validity end |
| `updatedAt` | string (timestamp) | yes | Story update time |
| `title` | string | no | Human story title |
| `description` | string | no | Story narrative text |
| `altText` | string | no | Accessibility text for the provider graphic |
| `priority` | bool | yes | Provider priority flag |
| `order` | int | yes | Provider display order |
| `downloadUrl` | string | no | Provider download URL; weatherfeeder does not fetch the asset |
---
## Schema: `weather.forecast_discussion.v1`
Payload type: `WeatherForecastDiscussion`
A `WeatherForecastDiscussion` is an issued narrative bulletin for an NWS office.
It is distinct from `weather.forecast.v1`, which is period-based.
### Fields
| Field | Type | Required | Notes |
|---|---:|:---:|---|
| `officeId` | string | no | NWS office identifier, e.g. `LSX` |
| `officeName` | string | no | Human office name |
| `product` | string | yes | Currently `afd` |
| `issuedAt` | string (timestamp) | yes | Bulletin issue time |
| `updatedAt` | string (timestamp) | no | Optional page/update timestamp |
| `keyMessages` | array | no | Ordered key-message bullet list |
| `shortTerm` | object | no | Short-term discussion section |
| `longTerm` | object | no | Long-term discussion section |
### Nested: `shortTerm` / `longTerm`
| Field | Type | Required | Notes |
|---|---:|:---:|---|
| `qualifier` | string | no | Header qualifier such as `(Through Late Sunday Night)` |
| `issuedAt` | string (timestamp) | no | Optional section-local issue time |
| `text` | string | no | Paragraph-preserved prose text |
---
## Compatibility rules
- Consumers **must** ignore unknown fields.
- Producers (weatherfeeder) prefer **additive changes** within a schema version.
- Renames/removals/semantic breaks normally require a **schema version bump** (`weather.*.v2`); pre-1.0 projects may choose in-place changes.
---
## Examples
### Observation event (`weather.observation.v1`)
```json
{
"id": "nws:KSTL:2026-01-17T14:00:00Z",
"schema": "weather.observation.v1",
"source": "nws_observation",
"effectiveAt": "2026-01-17T14:00:00Z",
"payload": {
"stationId": "KSTL",
"timestamp": "2026-01-17T14:00:00Z",
"conditionCode": 1,
"textDescription": "Mainly Sunny",
"temperatureC": 3.25,
"windSpeedKmh": 18.5
}
}
```
### Forecast event (`weather.forecast.v1`)
```json
{
"id": "openmeteo:38.63,-90.20:2026-01-17T13:00:00Z",
"schema": "weather.forecast.v1",
"source": "openmeteo_forecast",
"effectiveAt": "2026-01-17T13:00:00Z",
"payload": {
"locationName": "St. Louis, MO",
"issuedAt": "2026-01-17T13:00:00Z",
"product": "hourly",
"latitude": 38.63,
"longitude": -90.2,
"periods": [
{
"startTime": "2026-01-17T14:00:00Z",
"endTime": "2026-01-17T15:00:00Z",
"conditionCode": 2,
"textDescription": "Partly Cloudy",
"temperatureC": 3.5,
"probabilityOfPrecipitationPercent": 10
}
]
}
}
```
### Alert event (`weather.alert.v1`)
```json
{
"id": "nws:alerts:2026-01-17T14:10:00Z",
"schema": "weather.alert.v1",
"source": "nws_alerts",
"effectiveAt": "2026-01-17T14:10:00Z",
"payload": {
"asOf": "2026-01-17T14:05:00Z",
"alerts": [
{
"id": "https://api.weather.gov/alerts/abc123",
"event": "Winter Weather Advisory",
"headline": "Winter Weather Advisory issued January 17 at 8:05AM CST",
"severity": "Moderate",
"description": "Mixed precipitation expected...",
"expires": "2026-01-18T06:00:00Z"
}
]
}
}
```
### Weather story event (`weather.weather_story.v1`)
```json
{
"id": "nws:weatherstories:2026-05-30T09:00:34Z",
"schema": "weather.weather_story.v1",
"source": "nws_weatherstories",
"effectiveAt": "2026-05-30T09:00:34Z",
"payload": {
"officeId": "LSX",
"asOf": "2026-05-30T09:00:34Z",
"stories": [
{
"officeId": "LSX",
"startTime": "2026-05-30T08:46:00Z",
"endTime": "2026-05-31T11:00:00Z",
"updatedAt": "2026-05-30T09:00:34Z",
"title": "Several Chances for Rain Through Monday",
"description": "Scattered showers and thunderstorms remain possible.",
"altText": "This slide shows the forecast for today through Tuesday.",
"priority": false,
"order": 1,
"downloadUrl": "https://api.weather.gov/offices/LSX/weatherstories/download/3228e499-2aae-45a8-9ff9-1c060311026f"
}
]
}
}
```
The canonical event wire contract has moved to
[docs/integrations/events.md](docs/integrations/events.md).

2
README.md
View File

@@ -26,5 +26,7 @@ current working directory.
- [CLI reference](docs/cli.md)
- [Configuration reference](docs/config.md)
- [Event wire contract](docs/integrations/events.md)
- [Postgres table contract](docs/integrations/postgres.md)
- [Architecture policy](docs/policy/architecture.md)
- [Documentation policy](docs/policy/documentation.md)

3
docs/config.md
View File

@@ -164,7 +164,8 @@ Publishes each event as JSON to a NATS subject.
### `postgres`
Writes supported canonical weather events to Postgres using weatherfeeder's
registered schema mapping.
registered schema mapping. The table contract is documented in
[Postgres integration](integrations/postgres.md).
| Param | Required | Description |
|---|:---:|---|

238
docs/integrations/events.md Normal file
View File

@@ -0,0 +1,238 @@
# Event Wire Contract
This document is the canonical JSON contract for events emitted by
`weatherfeeder` JSON sinks, including stdout and NATS. Postgres stores the same
event envelope fields in parent table columns; see
[Postgres integration](postgres.md).
Downstream consumers should read the envelope, switch on `schema`, and decode
`payload` according to that schema.
## Envelope
Every emitted event is a JSON object with these fields:
| Field | Type | Required | Notes |
|---|---|:---:|---|
| `id` | string | yes | Stable event identifier. Treat as opaque. |
| `kind` | string | yes | Routing kind, such as `observation` or `alert`. |
| `source` | string | yes | Configured source name. |
| `emitted_at` | timestamp | yes | When the daemon emitted the event. |
| `effective_at` | timestamp | no | Timestamp the payload is about, when known. |
| `schema` | string | no | Schema identifier. Weatherfeeder sources and normalizers set this. |
| `payload` | object, array, string, or scalar | yes | Schema-specific payload. |
Timestamps are JSON strings using Go `time.Time` JSON encoding, which is
RFC3339Nano-compatible. Weatherfeeder normalizers use UTC timestamps for
canonical payloads.
## Kinds And Schemas
Canonical schemas emitted after normalization:
| Kind | Schema | Payload |
|---|---|---|
| `observation` | `weather.observation.v1` | `WeatherObservation` |
| `forecast` | `weather.forecast.v1` | `WeatherForecastRun` |
| `forecast_discussion` | `weather.forecast_discussion.v1` | `WeatherForecastDiscussion` |
| `weather_story` | `weather.weather_story.v1` | `WeatherStoryRun` |
| `alert` | `weather.alert.v1` | `WeatherAlertRun` |
Raw upstream schemas emitted by current sources:
| Kind | Schema | Payload |
|---|---|---|
| `observation` | `raw.nws.observation.v1` | NWS observation JSON |
| `observation` | `raw.openmeteo.current.v1` | Open-Meteo current JSON |
| `observation` | `raw.openweather.current.v1` | OpenWeather current JSON |
| `forecast` | `raw.nws.hourly.forecast.v1` | NWS hourly forecast JSON |
| `forecast` | `raw.nws.narrative.forecast.v1` | NWS narrative forecast JSON |
| `forecast_discussion` | `raw.nws.forecast_discussion.v1` | NWS forecast discussion HTML string |
| `weather_story` | `raw.nws.weatherstories.v1` | NWS weather stories JSON |
| `forecast` | `raw.openmeteo.hourly.forecast.v1` | Open-Meteo hourly forecast JSON |
| `alert` | `raw.nws.alerts.v1` | NWS alerts JSON |
`standards.SchemaRawOpenWeatherHourlyForecastV1` exists in code, but no current
registered source emits it.
## Shared Conventions
- Canonical numeric measurements use metric units.
- Floating-point values in canonical payloads are rounded to 4 digits after the
decimal point during normalization.
- Optional fields use JSON `omitempty`; absent fields should be treated as
unknown.
- `conditionCode` is a WMO weather interpretation code. Unknown observation
conditions use `-1`. Forecast period `conditionCode` is optional.
- Additive fields are compatible within a schema version. Removing, renaming, or
changing the meaning of a field requires a new schema identifier.
## `weather.observation.v1`
Payload type: `WeatherObservation`.
| Field | Type | Required | Notes |
|---|---|:---:|---|
| `stationId` | string | no | Provider station/location identifier. |
| `stationName` | string | no | Human station name. |
| `timestamp` | timestamp | yes | Observation timestamp. |
| `conditionCode` | integer | yes | WMO code; `-1` means unknown. |
| `isDay` | boolean | no | Day/night hint. |
| `textDescription` | string | no | Short human description. |
| `temperatureC` | number | no | Celsius. |
| `dewpointC` | number | no | Celsius. |
| `windDirectionDegrees` | number | no | Degrees. |
| `windSpeedKmh` | number | no | Kilometers per hour. |
| `windGustKmh` | number | no | Kilometers per hour. |
| `barometricPressurePa` | number | no | Pascals. |
| `visibilityMeters` | number | no | Meters. |
| `relativeHumidityPercent` | number | no | Percent from 0 to 100. |
| `apparentTemperatureC` | number | no | Celsius. |
| `presentWeather` | array | no | Provider-specific present weather fragments. |
`presentWeather[]` entries contain optional `raw` objects.
## `weather.forecast.v1`
Payload type: `WeatherForecastRun`.
| Field | Type | Required | Notes |
|---|---|:---:|---|
| `locationId` | string | no | Provider location identifier. |
| `locationName` | string | no | Human location name. |
| `issuedAt` | timestamp | yes | When the forecast run was generated or issued. |
| `updatedAt` | timestamp | no | Later provider update time. |
| `product` | string | yes | `hourly`, `narrative`, or `daily`. |
| `latitude` | number | no | Degrees. |
| `longitude` | number | no | Degrees. |
| `elevationMeters` | number | no | Meters. |
| `periods` | array | yes | Ordered forecast periods. |
`periods[]` entries:
| Field | Type | Required | Notes |
|---|---|:---:|---|
| `startTime` | timestamp | yes | Period start. |
| `endTime` | timestamp | yes | Period end. |
| `name` | string | no | Human label. |
| `isDay` | boolean | no | Day/night hint. |
| `conditionCode` | integer | no | WMO code when applicable. |
| `textDescription` | string | no | Human summary. |
| `temperatureC` | number | no | Celsius. |
| `temperatureCMin` | number | no | Celsius. |
| `temperatureCMax` | number | no | Celsius. |
| `dewpointC` | number | no | Celsius. |
| `relativeHumidityPercent` | number | no | Percent from 0 to 100. |
| `windDirectionDegrees` | number | no | Degrees. |
| `windSpeedKmh` | number | no | Kilometers per hour. |
| `windGustKmh` | number | no | Kilometers per hour. |
| `barometricPressurePa` | number | no | Pascals. |
| `visibilityMeters` | number | no | Meters. |
| `apparentTemperatureC` | number | no | Celsius. |
| `cloudCoverPercent` | number | no | Percent from 0 to 100. |
| `probabilityOfPrecipitationPercent` | number | no | Percent from 0 to 100. |
| `precipitationAmountMm` | number | no | Liquid-equivalent millimeters. |
| `snowfallDepthMm` | number | no | Millimeters. |
| `uvIndex` | number | no | Unitless index. |
## `weather.forecast_discussion.v1`
Payload type: `WeatherForecastDiscussion`.
| Field | Type | Required | Notes |
|---|---|:---:|---|
| `officeId` | string | no | NWS office identifier. |
| `officeName` | string | no | Office name. |
| `product` | string | yes | Current value is `afd`. |
| `issuedAt` | timestamp | yes | Bulletin issue time. |
| `updatedAt` | timestamp | no | Later update time. |
| `keyMessages` | array of strings | no | Extracted key messages. |
| `shortTerm` | object | no | Short-term section. |
| `longTerm` | object | no | Long-term section. |
`shortTerm` and `longTerm` sections contain optional `qualifier`, `issuedAt`,
and `text` fields.
## `weather.weather_story.v1`
Payload type: `WeatherStoryRun`.
| Field | Type | Required | Notes |
|---|---|:---:|---|
| `officeId` | string | no | NWS office identifier. |
| `asOf` | timestamp | yes | Snapshot time. |
| `stories` | array | yes | Ordered story cards. |
`stories[]` entries:
| Field | Type | Required | Notes |
|---|---|:---:|---|
| `officeId` | string | no | Office identifier. |
| `startTime` | timestamp | yes | Story start. |
| `endTime` | timestamp | yes | Story end. |
| `updatedAt` | timestamp | yes | Story update time. |
| `title` | string | no | Story title. |
| `description` | string | no | Story description. |
| `altText` | string | no | Image alternate text. |
| `priority` | boolean | yes | Provider priority flag. |
| `order` | integer | yes | Provider display order. |
| `downloadUrl` | string | no | Story image URL. |
## `weather.alert.v1`
Payload type: `WeatherAlertRun`.
| Field | Type | Required | Notes |
|---|---|:---:|---|
| `locationId` | string | no | Provider location identifier. |
| `locationName` | string | no | Human location name. |
| `asOf` | timestamp | yes | Snapshot time. |
| `latitude` | number | no | Degrees. |
| `longitude` | number | no | Degrees. |
| `alerts` | array | yes | Active alerts. |
`alerts[]` entries:
| Field | Type | Required | Notes |
|---|---|:---:|---|
| `id` | string | yes | Provider-stable alert identifier. |
| `event` | string | no | Alert event label. |
| `headline` | string | no | Alert headline. |
| `severity` | string | no | Provider severity. |
| `urgency` | string | no | Provider urgency. |
| `certainty` | string | no | Provider certainty. |
| `status` | string | no | Alert status. |
| `messageType` | string | no | Alert message type. |
| `category` | string | no | Alert category. |
| `response` | string | no | Recommended response. |
| `description` | string | no | Alert description. |
| `instruction` | string | no | Alert instruction. |
| `sent` | timestamp | no | Provider sent time. |
| `effective` | timestamp | no | Effective time. |
| `onset` | timestamp | no | Onset time. |
| `expires` | timestamp | no | Expiration time. |
| `areaDescription` | string | no | Affected area description. |
| `senderName` | string | no | Provider sender name. |
| `references` | array | no | Related alerts. |
`references[]` entries contain optional `id`, `identifier`, `sender`, and
`sent` fields.
## Compact Example
```json
{
"id": "NWSObservationKSTL:2026-06-10T12:00:00Z",
"kind": "observation",
"source": "NWSObservationKSTL",
"emitted_at": "2026-06-10T12:00:05Z",
"effective_at": "2026-06-10T12:00:00Z",
"schema": "weather.observation.v1",
"payload": {
"stationId": "KSTL",
"timestamp": "2026-06-10T12:00:00Z",
"conditionCode": 0,
"temperatureC": 22.5
}
}
```

403
docs/integrations/postgres.md Normal file
View File

@@ -0,0 +1,403 @@
# Postgres Integration
This document is the canonical table contract for the optional `postgres` sink.
It describes the schema created and written by weatherfeeder through feedkit's
Postgres sink.
Configure the sink as described in [configuration](../config.md#postgres).
## Initialization And Writes
At startup, each configured Postgres sink opens the database and runs
`CREATE TABLE IF NOT EXISTS` for every weatherfeeder table, followed by
`CREATE INDEX IF NOT EXISTS` for every configured index.
This initialization creates missing tables and indexes only. It does not alter
existing tables, migrate column definitions, drop old objects, or backfill data.
Schema changes require operator-managed database migration.
Events are mapped only for canonical weather schemas:
- `weather.observation.v1`
- `weather.forecast.v1`
- `weather.forecast_discussion.v1`
- `weather.weather_story.v1`
- `weather.alert.v1`
Unsupported schemas produce no writes for this sink. Mapped events are inserted
transactionally. Inserts use ordinary `INSERT`; duplicate primary keys fail the
write.
## Shared Envelope Columns
Parent tables store the feed event envelope:
| Column | Type | Null | Source |
|---|---|:---:|---|
| `event_id` | `TEXT` | no | `event.id` |
| `event_kind` | `TEXT` | no | `event.kind` |
| `event_source` | `TEXT` | no | `event.source` |
| `event_schema` | `TEXT` | no | `event.schema` |
| `event_emitted_at` | `TIMESTAMPTZ` | no | `event.emitted_at` |
| `event_effective_at` | `TIMESTAMPTZ` | yes | `event.effective_at` |
## Table Overview
| Table | Primary key | Prune column |
|---|---|---|
| `observations` | `event_id` | `observed_at` |
| `observation_present_weather` | `event_id`, `weather_index` | `observed_at` |
| `forecasts` | `event_id` | `issued_at` |
| `forecast_periods` | `run_event_id`, `period_index` | `issued_at` |
| `forecast_discussions` | `event_id` | `issued_at` |
| `forecast_discussion_key_messages` | `run_event_id`, `message_index` | `issued_at` |
| `weather_story_runs` | `event_id` | `as_of` |
| `weather_stories` | `run_event_id`, `story_index` | `as_of` |
| `alert_runs` | `event_id` | `as_of` |
| `alerts` | `run_event_id`, `alert_index` | `as_of` |
| `alert_references` | `run_event_id`, `alert_index`, `reference_index` | `as_of` |
## Table Contract
### `observations`
Primary key: `event_id`
Prune column: `observed_at`
Indexes:
- `idx_wf_obs_station_observed_at` on `station_id`, `observed_at`
- `idx_wf_obs_observed_at` on `observed_at`
- `idx_wf_obs_condition_code` on `condition_code`
| Column | Type | Null | Source |
|---|---|:---:|---|
| `event_id` | `TEXT` | no | `event.id` |
| `event_kind` | `TEXT` | no | `event.kind` |
| `event_source` | `TEXT` | no | `event.source` |
| `event_schema` | `TEXT` | no | `event.schema` |
| `event_emitted_at` | `TIMESTAMPTZ` | no | `event.emitted_at` |
| `event_effective_at` | `TIMESTAMPTZ` | yes | `event.effective_at` |
| `station_id` | `TEXT` | yes | `payload.stationId` |
| `station_name` | `TEXT` | yes | `payload.stationName` |
| `observed_at` | `TIMESTAMPTZ` | no | `payload.timestamp` |
| `condition_code` | `INTEGER` | no | `payload.conditionCode` |
| `is_day` | `BOOLEAN` | yes | `payload.isDay` |
| `text_description` | `TEXT` | yes | `payload.textDescription` |
| `temperature_c` | `DOUBLE PRECISION` | yes | `payload.temperatureC` |
| `dewpoint_c` | `DOUBLE PRECISION` | yes | `payload.dewpointC` |
| `wind_direction_degrees` | `DOUBLE PRECISION` | yes | `payload.windDirectionDegrees` |
| `wind_speed_kmh` | `DOUBLE PRECISION` | yes | `payload.windSpeedKmh` |
| `wind_gust_kmh` | `DOUBLE PRECISION` | yes | `payload.windGustKmh` |
| `barometric_pressure_pa` | `DOUBLE PRECISION` | yes | `payload.barometricPressurePa` |
| `visibility_meters` | `DOUBLE PRECISION` | yes | `payload.visibilityMeters` |
| `relative_humidity_percent` | `DOUBLE PRECISION` | yes | `payload.relativeHumidityPercent` |
| `apparent_temperature_c` | `DOUBLE PRECISION` | yes | `payload.apparentTemperatureC` |
### `observation_present_weather`
Primary key: `event_id`, `weather_index`
Prune column: `observed_at`
Foreign key: `event_id` references `observations(event_id)` with cascade delete.
Index: `idx_wf_obs_present_observed_at` on `observed_at`
| Column | Type | Null | Source |
|---|---|:---:|---|
| `event_id` | `TEXT REFERENCES observations(event_id) ON DELETE CASCADE` | no | Parent event ID. |
| `weather_index` | `INTEGER` | no | `payload.presentWeather[]` index. |
| `observed_at` | `TIMESTAMPTZ` | no | `payload.timestamp` |
| `raw_text` | `TEXT` | yes | Compact JSON text from `payload.presentWeather[].raw` |
### `forecasts`
Primary key: `event_id`
Prune column: `issued_at`
Indexes:
- `idx_wf_fc_location_product_issued_at` on `location_id`, `product`, `issued_at`
- `idx_wf_fc_issued_at` on `issued_at`
- `idx_wf_fc_product_issued_at` on `product`, `issued_at`
| Column | Type | Null | Source |
|---|---|:---:|---|
| `event_id` | `TEXT` | no | `event.id` |
| `event_kind` | `TEXT` | no | `event.kind` |
| `event_source` | `TEXT` | no | `event.source` |
| `event_schema` | `TEXT` | no | `event.schema` |
| `event_emitted_at` | `TIMESTAMPTZ` | no | `event.emitted_at` |
| `event_effective_at` | `TIMESTAMPTZ` | yes | `event.effective_at` |
| `location_id` | `TEXT` | yes | `payload.locationId` |
| `location_name` | `TEXT` | yes | `payload.locationName` |
| `issued_at` | `TIMESTAMPTZ` | no | `payload.issuedAt` |
| `updated_at` | `TIMESTAMPTZ` | yes | `payload.updatedAt` |
| `product` | `TEXT` | no | `payload.product` |
| `latitude` | `DOUBLE PRECISION` | yes | `payload.latitude` |
| `longitude` | `DOUBLE PRECISION` | yes | `payload.longitude` |
| `elevation_meters` | `DOUBLE PRECISION` | yes | `payload.elevationMeters` |
| `period_count` | `INTEGER` | no | `len(payload.periods)` |
### `forecast_periods`
Primary key: `run_event_id`, `period_index`
Prune column: `issued_at`
Foreign key: `run_event_id` references `forecasts(event_id)` with cascade delete.
Indexes:
- `idx_wf_fc_period_start_time` on `start_time`
- `idx_wf_fc_period_end_time` on `end_time`
- `idx_wf_fc_period_run_start` on `run_event_id`, `start_time`
| Column | Type | Null | Source |
|---|---|:---:|---|
| `run_event_id` | `TEXT REFERENCES forecasts(event_id) ON DELETE CASCADE` | no | Parent event ID. |
| `period_index` | `INTEGER` | no | `payload.periods[]` index. |
| `issued_at` | `TIMESTAMPTZ` | no | Parent `payload.issuedAt` |
| `start_time` | `TIMESTAMPTZ` | no | `payload.periods[].startTime` |
| `end_time` | `TIMESTAMPTZ` | no | `payload.periods[].endTime` |
| `name` | `TEXT` | yes | `payload.periods[].name` |
| `is_day` | `BOOLEAN` | yes | `payload.periods[].isDay` |
| `condition_code` | `INTEGER` | yes | `payload.periods[].conditionCode` |
| `text_description` | `TEXT` | yes | `payload.periods[].textDescription` |
| `temperature_c` | `DOUBLE PRECISION` | yes | `payload.periods[].temperatureC` |
| `temperature_c_min` | `DOUBLE PRECISION` | yes | `payload.periods[].temperatureCMin` |
| `temperature_c_max` | `DOUBLE PRECISION` | yes | `payload.periods[].temperatureCMax` |
| `dewpoint_c` | `DOUBLE PRECISION` | yes | `payload.periods[].dewpointC` |
| `relative_humidity_percent` | `DOUBLE PRECISION` | yes | `payload.periods[].relativeHumidityPercent` |
| `wind_direction_degrees` | `DOUBLE PRECISION` | yes | `payload.periods[].windDirectionDegrees` |
| `wind_speed_kmh` | `DOUBLE PRECISION` | yes | `payload.periods[].windSpeedKmh` |
| `wind_gust_kmh` | `DOUBLE PRECISION` | yes | `payload.periods[].windGustKmh` |
| `barometric_pressure_pa` | `DOUBLE PRECISION` | yes | `payload.periods[].barometricPressurePa` |
| `visibility_meters` | `DOUBLE PRECISION` | yes | `payload.periods[].visibilityMeters` |
| `apparent_temperature_c` | `DOUBLE PRECISION` | yes | `payload.periods[].apparentTemperatureC` |
| `cloud_cover_percent` | `DOUBLE PRECISION` | yes | `payload.periods[].cloudCoverPercent` |
| `probability_of_precipitation_percent` | `DOUBLE PRECISION` | yes | `payload.periods[].probabilityOfPrecipitationPercent` |
| `precipitation_amount_mm` | `DOUBLE PRECISION` | yes | `payload.periods[].precipitationAmountMm` |
| `snowfall_depth_mm` | `DOUBLE PRECISION` | yes | `payload.periods[].snowfallDepthMm` |
| `uv_index` | `DOUBLE PRECISION` | yes | `payload.periods[].uvIndex` |
### `forecast_discussions`
Primary key: `event_id`
Prune column: `issued_at`
Indexes:
- `idx_wf_discussion_office_product_issued_at` on `office_id`, `product`, `issued_at`
- `idx_wf_discussion_issued_at` on `issued_at`
| Column | Type | Null | Source |
|---|---|:---:|---|
| `event_id` | `TEXT` | no | `event.id` |
| `event_kind` | `TEXT` | no | `event.kind` |
| `event_source` | `TEXT` | no | `event.source` |
| `event_schema` | `TEXT` | no | `event.schema` |
| `event_emitted_at` | `TIMESTAMPTZ` | no | `event.emitted_at` |
| `event_effective_at` | `TIMESTAMPTZ` | yes | `event.effective_at` |
| `office_id` | `TEXT` | yes | `payload.officeId` |
| `office_name` | `TEXT` | yes | `payload.officeName` |
| `issued_at` | `TIMESTAMPTZ` | no | `payload.issuedAt` |
| `updated_at` | `TIMESTAMPTZ` | yes | `payload.updatedAt` |
| `product` | `TEXT` | no | `payload.product` |
| `short_term_qualifier` | `TEXT` | yes | `payload.shortTerm.qualifier` |
| `short_term_issued_at` | `TIMESTAMPTZ` | yes | `payload.shortTerm.issuedAt` |
| `short_term_text` | `TEXT` | yes | `payload.shortTerm.text` |
| `long_term_qualifier` | `TEXT` | yes | `payload.longTerm.qualifier` |
| `long_term_issued_at` | `TIMESTAMPTZ` | yes | `payload.longTerm.issuedAt` |
| `long_term_text` | `TEXT` | yes | `payload.longTerm.text` |
| `key_message_count` | `INTEGER` | no | `len(payload.keyMessages)` |
### `forecast_discussion_key_messages`
Primary key: `run_event_id`, `message_index`
Prune column: `issued_at`
Foreign key: `run_event_id` references `forecast_discussions(event_id)` with
cascade delete.
Index: `idx_wf_discussion_message_issued_at` on `issued_at`
| Column | Type | Null | Source |
|---|---|:---:|---|
| `run_event_id` | `TEXT REFERENCES forecast_discussions(event_id) ON DELETE CASCADE` | no | Parent event ID. |
| `message_index` | `INTEGER` | no | `payload.keyMessages[]` index. |
| `issued_at` | `TIMESTAMPTZ` | no | Parent `payload.issuedAt` |
| `message_text` | `TEXT` | yes | `payload.keyMessages[]` value |
### `weather_story_runs`
Primary key: `event_id`
Prune column: `as_of`
Indexes:
- `idx_wf_story_run_office_as_of` on `office_id`, `as_of`
- `idx_wf_story_run_as_of` on `as_of`
| Column | Type | Null | Source |
|---|---|:---:|---|
| `event_id` | `TEXT` | no | `event.id` |
| `event_kind` | `TEXT` | no | `event.kind` |
| `event_source` | `TEXT` | no | `event.source` |
| `event_schema` | `TEXT` | no | `event.schema` |
| `event_emitted_at` | `TIMESTAMPTZ` | no | `event.emitted_at` |
| `event_effective_at` | `TIMESTAMPTZ` | yes | `event.effective_at` |
| `office_id` | `TEXT` | yes | `payload.officeId` |
| `as_of` | `TIMESTAMPTZ` | no | `payload.asOf` |
| `story_count` | `INTEGER` | no | `len(payload.stories)` |
### `weather_stories`
Primary key: `run_event_id`, `story_index`
Prune column: `as_of`
Foreign key: `run_event_id` references `weather_story_runs(event_id)` with
cascade delete.
Indexes:
- `idx_wf_stories_start_time` on `start_time`
- `idx_wf_stories_end_time` on `end_time`
- `idx_wf_stories_updated_at` on `updated_at`
| Column | Type | Null | Source |
|---|---|:---:|---|
| `run_event_id` | `TEXT REFERENCES weather_story_runs(event_id) ON DELETE CASCADE` | no | Parent event ID. |
| `story_index` | `INTEGER` | no | `payload.stories[]` index. |
| `as_of` | `TIMESTAMPTZ` | no | Parent `payload.asOf` |
| `office_id` | `TEXT` | yes | `payload.stories[].officeId` |
| `start_time` | `TIMESTAMPTZ` | no | `payload.stories[].startTime` |
| `end_time` | `TIMESTAMPTZ` | no | `payload.stories[].endTime` |
| `updated_at` | `TIMESTAMPTZ` | no | `payload.stories[].updatedAt` |
| `title` | `TEXT` | yes | `payload.stories[].title` |
| `description` | `TEXT` | yes | `payload.stories[].description` |
| `alt_text` | `TEXT` | yes | `payload.stories[].altText` |
| `priority` | `BOOLEAN` | no | `payload.stories[].priority` |
| `story_order` | `INTEGER` | no | `payload.stories[].order` |
| `download_url` | `TEXT` | yes | `payload.stories[].downloadUrl` |
### `alert_runs`
Primary key: `event_id`
Prune column: `as_of`
Indexes:
- `idx_wf_alert_run_location_as_of` on `location_id`, `as_of`
- `idx_wf_alert_run_as_of` on `as_of`
| Column | Type | Null | Source |
|---|---|:---:|---|
| `event_id` | `TEXT` | no | `event.id` |
| `event_kind` | `TEXT` | no | `event.kind` |
| `event_source` | `TEXT` | no | `event.source` |
| `event_schema` | `TEXT` | no | `event.schema` |
| `event_emitted_at` | `TIMESTAMPTZ` | no | `event.emitted_at` |
| `event_effective_at` | `TIMESTAMPTZ` | yes | `event.effective_at` |
| `location_id` | `TEXT` | yes | `payload.locationId` |
| `location_name` | `TEXT` | yes | `payload.locationName` |
| `as_of` | `TIMESTAMPTZ` | no | `payload.asOf` |
| `latitude` | `DOUBLE PRECISION` | yes | `payload.latitude` |
| `longitude` | `DOUBLE PRECISION` | yes | `payload.longitude` |
| `alert_count` | `INTEGER` | no | `len(payload.alerts)` |
### `alerts`
Primary key: `run_event_id`, `alert_index`
Prune column: `as_of`
Foreign key: `run_event_id` references `alert_runs(event_id)` with cascade
delete.
Indexes:
- `idx_wf_alerts_alert_id` on `alert_id`
- `idx_wf_alerts_severity_expires` on `severity`, `expires`
- `idx_wf_alerts_as_of` on `as_of`
| Column | Type | Null | Source |
|---|---|:---:|---|
| `run_event_id` | `TEXT REFERENCES alert_runs(event_id) ON DELETE CASCADE` | no | Parent event ID. |
| `alert_index` | `INTEGER` | no | `payload.alerts[]` index. |
| `as_of` | `TIMESTAMPTZ` | no | Parent `payload.asOf` |
| `alert_id` | `TEXT` | no | `payload.alerts[].id` |
| `event` | `TEXT` | yes | `payload.alerts[].event` |
| `headline` | `TEXT` | yes | `payload.alerts[].headline` |
| `severity` | `TEXT` | yes | `payload.alerts[].severity` |
| `urgency` | `TEXT` | yes | `payload.alerts[].urgency` |
| `certainty` | `TEXT` | yes | `payload.alerts[].certainty` |
| `status` | `TEXT` | yes | `payload.alerts[].status` |
| `message_type` | `TEXT` | yes | `payload.alerts[].messageType` |
| `category` | `TEXT` | yes | `payload.alerts[].category` |
| `response` | `TEXT` | yes | `payload.alerts[].response` |
| `description` | `TEXT` | yes | `payload.alerts[].description` |
| `instruction` | `TEXT` | yes | `payload.alerts[].instruction` |
| `sent` | `TIMESTAMPTZ` | yes | `payload.alerts[].sent` |
| `effective` | `TIMESTAMPTZ` | yes | `payload.alerts[].effective` |
| `onset` | `TIMESTAMPTZ` | yes | `payload.alerts[].onset` |
| `expires` | `TIMESTAMPTZ` | yes | `payload.alerts[].expires` |
| `area_description` | `TEXT` | yes | `payload.alerts[].areaDescription` |
| `sender_name` | `TEXT` | yes | `payload.alerts[].senderName` |
| `reference_count` | `INTEGER` | no | `len(payload.alerts[].references)` |
### `alert_references`
Primary key: `run_event_id`, `alert_index`, `reference_index`
Prune column: `as_of`
Foreign key: `run_event_id` references `alert_runs(event_id)` with cascade
delete.
Indexes:
- `idx_wf_alert_refs_as_of` on `as_of`
- `idx_wf_alert_refs_sent` on `sent`
| Column | Type | Null | Source |
|---|---|:---:|---|
| `run_event_id` | `TEXT REFERENCES alert_runs(event_id) ON DELETE CASCADE` | no | Parent event ID. |
| `alert_index` | `INTEGER` | no | Parent alert index. |
| `reference_index` | `INTEGER` | no | `payload.alerts[].references[]` index. |
| `as_of` | `TIMESTAMPTZ` | no | Parent `payload.asOf` |
| `id` | `TEXT` | yes | `payload.alerts[].references[].id` |
| `identifier` | `TEXT` | yes | `payload.alerts[].references[].identifier` |
| `sender` | `TEXT` | yes | `payload.alerts[].references[].sender` |
| `sent` | `TIMESTAMPTZ` | yes | `payload.alerts[].references[].sent` |
## Retention
When sink param `prune` is set, every successful write transaction deletes rows
older than `now - prune` from every table using that table's prune column.
The sink also exposes manual prune helpers in code, but the `weatherfeeder`
binary does not provide CLI commands for them.
## Reconstructing Canonical Payloads
- `WeatherObservation`: read `observations`, then join
`observation_present_weather` by `event_id` ordered by `weather_index`.
- `WeatherForecastRun`: read `forecasts`, then join `forecast_periods` by
`run_event_id` ordered by `period_index`.
- `WeatherForecastDiscussion`: read `forecast_discussions`, then join
`forecast_discussion_key_messages` by `run_event_id` ordered by
`message_index`.
- `WeatherStoryRun`: read `weather_story_runs`, then join `weather_stories` by
`run_event_id` ordered by `story_index`.
- `WeatherAlertRun`: read `alert_runs`, join `alerts` by `run_event_id` ordered
by `alert_index`, then join `alert_references` by `run_event_id` and
`alert_index` ordered by `reference_index`.

13
docs/policy/architecture.md
View File

@@ -4,7 +4,9 @@
This document defines `weatherfeeder`'s development architecture and invariants for maintainers and LLM coding agents. It describes how the implemented system is built and how future changes should preserve its boundaries.
This is an inward-facing policy document. User-facing wire contracts belong in [`API.md`](../../API.md), and future work belongs under [`docs/roadmap/`](../roadmap/).
This is an inward-facing policy document. User-facing wire contracts belong in
[`docs/integrations/events.md`](../integrations/events.md), and future work
belongs under [`docs/roadmap/`](../roadmap/).
## Project Shape
@@ -98,7 +100,10 @@ The daemon's own state is in-process:
- source instances may keep HTTP conditional request state through feedkit HTTP source helpers;
- scheduler and dispatcher state is not persisted by `weatherfeeder`.
Durable persistence is an external sink concern. The Postgres table contract is documented in `internal/sinks/postgres/doc.go`; the consumer-facing event contract is documented in [`API.md`](../../API.md).
Durable persistence is an external sink concern. The Postgres table contract is
documented in [`docs/integrations/postgres.md`](../integrations/postgres.md);
the consumer-facing event contract is documented in
[`docs/integrations/events.md`](../integrations/events.md).
## Configuration and CLI Boundaries
@@ -112,7 +117,9 @@ Configuration shape is owned by feedkit's config package:
Weatherfeeder-specific config policy belongs in source and sink constructors, registry setup, and tests. Do not spread config parsing through domain model or normalizer packages.
If dedicated `docs/config.md` or `docs/cli.md` files are added later, they should become the canonical user/operator references. This policy should stay architectural and avoid duplicating those references.
[`docs/config.md`](../config.md) and [`docs/cli.md`](../cli.md) are the
canonical user/operator references. This policy should stay architectural and
avoid duplicating those references.
## Errors, Logging, and Diagnostics