weatherfeeder/docs/policy/development.md

Development Policy

Purpose

This document describes how to change weatherfeeder safely. It is for maintainers and coding agents working in the repository.

Use this alongside the architecture policy. User-facing CLI, configuration, operations, and wire-contract details belong in their canonical docs, not here.

Repository Layout

• cmd/weatherfeeder/: executable wiring, sample config.yml, and runtime composition tests.
• model/: canonical weather payload structs. JSON tags are part of the wire contract.
• standards/: schema strings, versioning conventions, WMO constants, and shared wire-format policy.
• internal/sources/: source adapters that poll upstream providers and emit raw feed events.
• internal/normalizers/: raw-to-canonical event transforms.
• internal/providers/: pure provider helper code shared by sources and normalizers.
• internal/sinks/postgres/: weatherfeeder-owned Postgres schema and canonical event mapper.
• docs/: current behavior, consumer guides, integration contracts, policies, and roadmap files.
• examples/: maintained, copyable configuration examples.

Build And Test

Run the full test suite before committing behavior or documentation changes that depend on code behavior:

go test ./...

Use narrower commands while iterating:

go test ./cmd/weatherfeeder
go test ./internal/sources/...
go test ./internal/normalizers/...
go test ./internal/sinks/postgres

Format Go code before committing:

gofmt -w <changed-go-files>

Do not require live upstream weather services, NATS, or Postgres for unit tests. Use fixtures, local test servers, and package-level tests.

Coding Conventions

• Keep cmd/weatherfeeder focused on composition: config load, registry setup, scheduler jobs, processor chain, dispatch, signal handling, and logging.
• Keep source fetching separate from normalizer mapping.
• Match normalizers by schema constants from standards, not source names.
• Keep provider-specific helper code under internal/providers/<provider> when both sources and normalizers use it.
• Keep cross-provider normalizer helpers pure and deterministic under internal/normalizers/common.
• Keep sink persistence mapping isolated under internal/sinks/<sink>.
• Wrap errors with operation context, but do not include whole upstream payloads in errors or logs by default.
• Prefer explicit registries and small package-level constructors over hidden global behavior.

Architecture-Preserving Changes

When changing weatherfeeder, preserve the split between feedkit infrastructure and weather-domain behavior.

Do:

• keep generic scheduling, dispatch, processor, config, and sink mechanics in feedkit;
• keep weather provider rules in source adapters, provider helpers, and normalizers;
• keep canonical weather payloads in model and schema/wire identifiers in standards;
• keep Postgres table and row mapping under internal/sinks/postgres;
• use explicit registries for built-in sources and normalizers.

Do not:

• move provider parsing, WMO mapping, or canonical weather policy into cmd/weatherfeeder;
• move weather-specific constants, schemas, or validation rules into feedkit;
• put database column metadata or sink-specific tags on canonical model structs;
• replace explicit registries with dynamic plugin loading;
• introduce broad abstractions when a small provider-specific helper preserves clarity.

Dependency Policy

Prefer the Go standard library unless a dependency materially improves maintainability or interoperability.

feedkit owns generic daemon infrastructure for config, HTTP source helpers, scheduling, processors, dispatch, and sinks. Weatherfeeder code should contain weather-domain behavior and narrow adapter logic rather than duplicating feedkit infrastructure.

Do not add broad dependencies for small conveniences. Do not let dependency-specific types leak across package boundaries unless that dependency is the package contract.

Adding Config Fields

Generic config shape is owned by feedkit. Weatherfeeder-specific config behavior belongs in source or sink constructors, registry setup, and tests.

When adding config behavior:

• validate required params at the adapter boundary;
• keep secrets in environment variables or placeholders, not committed values;
• update configuration docs;
• update maintained examples when the change affects normal operation;
• add or update config-load tests for example files when practical.

Adding CLI Flags

The executable currently reads config.yml from the current working directory. If CLI flags are added:

• keep parsing in cmd/weatherfeeder;
• avoid moving config policy into domain packages;
• update CLI docs;
• update tests that exercise command behavior.

Adding A Source Driver

Source drivers should fetch upstream data and emit raw events with minimal metadata decoding.

Checklist:

• implement the driver under internal/sources/<provider>;
• build from config.SourceConfig;
• validate required params in the constructor;
• use feedkit HTTP helpers for HTTP polling when applicable;
• emit raw schema constants from standards;
• advertise emitted kinds through Kinds();
• decode only metadata needed for event ID and effective time;
• register the driver in internal/sources/builtins.go;
• add constructor, kind, and polling tests;
• update config docs and examples when operators need new configuration;
• add provider integration notes when the provider contract needs maintenance context.

Adding A Normalizer

Normalizers own provider-to-canonical mapping.

Checklist:

• add one normalizer type per normalizer file;
• match using Event.Schema;
• decode raw payloads into provider structs;
• map to canonical model payloads;
• use internal/normalizers/common.Finalize so envelope handling and float rounding stay consistent;
• preserve input envelope fields except schema, payload, and effective time;
• register through the provider package and internal/normalizers/builtins.go;
• add tests for schema matching, key payload fields, effective time, malformed required data, and output validation.

Adding Canonical Models Or Schemas

Canonical event changes affect multiple contracts.

Checklist:

• update payload structs in model;
• add or update schema constants in standards;
• update event wire contract docs;
• update normalizers that produce the schema;
• update Postgres mapping if the schema is persisted;
• add tests for wire shape and mapper behavior.

Adding Postgres Mapping

Weatherfeeder owns the canonical-event-to-table mapping. Feedkit owns the generic Postgres sink mechanics.

Checklist:

• update internal/sinks/postgres/schema.go;
• update internal/sinks/postgres/map.go;
• preserve event envelope columns in parent rows when the table supports them;
• validate required canonical fields before writing;
• use positional indexes for child rows that represent arrays;
• update mapper and schema tests;
• update Postgres integration docs when the table contract changes.

Examples And Documentation

Documentation must follow the documentation policy.

When behavior changes, update the canonical docs in the same change:

• config shape: docs/config.md;
• CLI behavior: docs/cli.md;
• operations and recovery: docs/operations.md;
• troubleshooting: docs/troubleshooting.md;
• public Go package consumption: docs/consumers/;
• external contracts: docs/integrations/;
• internal component behavior: docs/internal/;
• copyable configs: examples/.

Keep roadmap content under docs/roadmap/. Current-behavior docs must describe implemented behavior only.

Review Checklist

Before committing:

• run focused tests for changed packages;
• run go test ./... for broad behavior or documentation changes tied to code;
• verify maintained examples still load when examples or config docs changed;
• check links in changed docs;
• search for stale paths, unsupported features, and secret-like values;
• keep unrelated refactors out of the change.