diff --git a/docs/internal/runtime.md b/docs/internal/runtime.md index f631fe9..97e2113 100644 --- a/docs/internal/runtime.md +++ b/docs/internal/runtime.md @@ -68,6 +68,21 @@ Runtime composition uses feedkit for: Weatherfeeder registers its own source drivers and its Postgres schema mapper. +Responsibility split: + +| Runtime concern | Owner | +| --- | --- | +| Config loading and generic validation | feedkit | +| Source, processor, and sink registries | feedkit mechanics; weatherfeeder registrations | +| Source polling and stream supervision | feedkit scheduler | +| Raw weather data fetching | weatherfeeder source adapters | +| Normalizer execution order and pass-through behavior | feedkit normalize processor | +| Weather raw-to-canonical mapping | weatherfeeder normalizers | +| Dedupe mechanics | feedkit dedupe processor | +| Route compilation and sink fanout | feedkit dispatch | +| Weather Postgres table shape and row mapping | weatherfeeder Postgres adapter | +| Postgres connection, DDL, inserts, transactions, and pruning | feedkit Postgres sink | + ## State Weatherfeeder-owned runtime state is in process: diff --git a/docs/policy/architecture.md b/docs/policy/architecture.md index 67edf3c..a086b6d 100644 --- a/docs/policy/architecture.md +++ b/docs/policy/architecture.md @@ -23,6 +23,27 @@ The implemented runtime flow is: Canonical payload structs live in `model`. Schema identifiers and cross-provider wire conventions live in `standards`. Source adapters live under `internal/sources`. Normalizers live under `internal/normalizers`. Provider-specific parsing helpers shared by sources and normalizers live under `internal/providers`. Sink-specific persistence mapping lives under `internal/sinks`. +## Architecture Style + +`weatherfeeder` uses a pragmatic ports-and-adapters architecture rather than a +formal framework. Provider APIs, config loading, scheduling, dispatch, and sinks +sit outside the weather domain model and normalization rules. + +The implementation style is: + +- Pipeline-oriented: events flow from source polling through normalization, + dedupe, routing, and sink fanout. +- Schema-routed: normalizers select raw payloads by explicit schema strings, not + source names or configured routes. +- Provider-isolated: NWS, Open-Meteo, OpenWeather, and SPC quirks stay in + provider-specific source, provider-helper, and normalizer packages. +- Registry-based: built-in source drivers, normalizers, processors, and sinks + are assembled explicitly through registries instead of dynamic plugin loading. +- Adapter-clean: persistence and external-system details stay behind source and + sink adapters, not in `model` or normalizers. +- Direct Go: prefer small package-level constructors and straightforward code + over broad abstractions. + ## Core Design Principles - Hexagonal boundaries: provider APIs, config loading, scheduling, dispatch, and sinks are external mechanisms around the weather domain model and normalization logic. @@ -57,6 +78,23 @@ Tests and examples: - The sample `cmd/weatherfeeder/config.yml` is executable test input and is load-tested. - Tests should keep exercising package contracts directly rather than relying only on full-daemon execution. +## Feedkit Boundary + +`feedkit` provides reusable daemon infrastructure. `weatherfeeder` provides the +weather-domain adapters, models, schemas, and mapping policy. + +| Area | Feedkit owns | Weatherfeeder owns | +| --- | --- | --- | +| Config | Generic YAML shape: sources, sinks, routes, modes, cadence, and params. | Driver-specific config rules such as NWS `user_agent`, OpenWeather `units=metric`, and SPC coordinates. | +| Events | Domain-agnostic event envelope: ID, kind, source, emitted/effective times, schema, and payload. | Event kind meaning, schema strings, and canonical weather payloads. | +| Sources | Source interfaces, registry, expected-kind validation, HTTP helper, and default event ID helper. | NWS/Open-Meteo/OpenWeather/SPC source drivers and raw schema emission. | +| Processing | Processor registry, normalize processor, dedupe processor, and pipeline execution. | Weather normalizers and schema-specific raw-to-canonical mapping. | +| Dispatch | Route compilation and sink fanout mechanics. | Which weather event kinds are configured and meaningful. | +| Sinks | Generic stdout, NATS, and Postgres sink mechanics. | Weather-specific Postgres schema and canonical event-to-row mapping. | + +Do not move weather-domain policy into `feedkit`, and do not duplicate generic +daemon mechanics in `weatherfeeder` when feedkit already provides the boundary. + ## Modules Or Processing Steps The implemented processing steps are source polling, normalization, dedupe, and sink dispatch. diff --git a/docs/policy/development.md b/docs/policy/development.md index 7af25d3..bfddd84 100644 --- a/docs/policy/development.md +++ b/docs/policy/development.md @@ -70,6 +70,32 @@ Use fixtures, local test servers, and package-level tests. - Prefer explicit registries and small package-level constructors over hidden global behavior. +## Architecture-Preserving Changes + +When changing `weatherfeeder`, preserve the split between feedkit +infrastructure and weather-domain behavior. + +Do: + +- keep generic scheduling, dispatch, processor, config, and sink mechanics in + feedkit; +- keep weather provider rules in source adapters, provider helpers, and + normalizers; +- keep canonical weather payloads in `model` and schema/wire identifiers in + `standards`; +- keep Postgres table and row mapping under `internal/sinks/postgres`; +- use explicit registries for built-in sources and normalizers. + +Do not: + +- move provider parsing, WMO mapping, or canonical weather policy into + `cmd/weatherfeeder`; +- move weather-specific constants, schemas, or validation rules into feedkit; +- put database column metadata or sink-specific tags on canonical model structs; +- replace explicit registries with dynamic plugin loading; +- introduce broad abstractions when a small provider-specific helper preserves + clarity. + ## Dependency Policy Prefer the Go standard library unless a dependency materially improves