weatherfeeder/docs/roadmap/cleanup.md

Cleanup Roadmap

Summary

This roadmap tracks remaining behavior-preserving cleanup opportunities for weatherfeeder. Earlier cleanup stages for event kind constants, source driver constants, registry-derived source tests, shared multi-document HTTP config, and shallow docs consistency tests have been completed and are no longer listed as future work.

The remaining work should be implemented only in small, focused changes that do not alter public event schemas, event kinds, source driver names, config keys, Postgres table contracts, or canonical JSON field names.

Guardrails

• Keep feedkit as the generic daemon infrastructure boundary.
• Keep weather-domain behavior in sources, provider helpers, normalizers, model, standards, and Postgres mapping.
• Keep current-behavior docs accurate and roadmap-only plans under docs/roadmap/.
• Do not introduce plugin systems, workflow engines, generic source frameworks, ORM-style mapping, reflection mapping, generated docs, or persistence tags on canonical model structs.
• Run focused tests after each stage and go test ./... after all stages.

Stage 1: Reduce Postgres Mapper Envelope Duplication

Centralize repeated parent event envelope mapping without changing the table contract.

Implementation requirements:

• Add small helpers inside internal/sinks/postgres for parent envelope values: event_id, event_kind, event_source, event_schema, event_emitted_at, and event_effective_at.
• Use the helper in every parent table mapper that stores event envelope columns.
• Keep explicit per-product mapper functions and product-specific validation.
• Optionally centralize envelope column declarations only if schema.go remains easy to scan.
• Preserve every table, column, nullability rule, required-field check, compact JSON behavior, UTC normalization, child positional index, and write count.

Acceptance criteria:

• Mapper output for existing valid payloads is equivalent before and after the refactor.
• Unsupported schemas still map to zero writes and no error.
• Required-field failures still include useful product/path context.

Focused tests:

go test ./internal/sinks/postgres

Stage 2: Consolidate Package-Local Fixture Helpers

Reduce low-value duplicated fixture-reading code only where it is local and obvious.

Implementation requirements:

• Consolidate duplicated fixture readers only within the same Go package.
• Keep fixtures under each package's testdata directory.
• Do not create a cross-package test utility package.
• Do not change fixture contents unless an existing test already requires it.
• Do not mix this cleanup with parser behavior changes.

Acceptance criteria:

• Local test helper duplication is reduced where multiple files in one package already share the same fixture-reading behavior.
• Tests remain easy to read locally.
• No package imports a helper solely for tests from another package.

Focused tests:

go test ./internal/providers/nws ./internal/providers/spc
go test ./internal/sources/nws ./internal/sources/spc
go test ./internal/normalizers/nws ./internal/normalizers/spc

Stage 3: Dead-Code And Literal Sweep

Perform a final cleanup sweep after mapper and fixture cleanup.

Implementation requirements:

• Search for stale internal driver, kind, and schema literals.
• Replace internal code/test literals with constants where it reduces typo or drift risk.
• Keep user-facing docs and YAML examples literal.
• Keep intentional legacy-driver negative tests.
• Do not remove compatibility tests unless they are clearly obsolete.
• Do not broaden the cleanup into unrelated refactors.

Suggested searches:

rg 'event\.Kind\("|nws_forecast|nws_weatherstories|openmeteo_|openweather_|spc_convective_outlook|weather_story|forecast_discussion|raw\.|weather\.' .
rg 'TODO|legacy|deprecated|unknown source driver' internal cmd docs examples

Acceptance criteria:

• Internal literals are reduced where constants already exist.
• Intentional literals in docs, YAML examples, raw schema docs, and negative tests remain readable.
• No behavior changes are introduced.

Focused tests:

go test ./internal/sources ./internal/normalizers/... ./internal/sinks/postgres ./cmd/weatherfeeder

Final Verification

After all stages are complete, run:

go test ./...
git status --short

Verify current-behavior docs still describe implemented behavior only and no public contracts changed unintentionally.

Refactors To Avoid

Do not perform these changes as part of this cleanup roadmap:

• Generic workflow or stage engine.
• Runtime plugin system.
• Weather-specific constants in feedkit.
• Replacing feedkit scheduler, dispatch, HTTP helpers, or sink mechanics.
• Generic source abstraction covering every source type.
• Reflection-based or generated Postgres mapper.
• ORM-style persistence layer.
• Database column metadata on canonical model structs.
• Cross-provider timestamp parser that hides provider-specific formats.
• Broad WMO mapper consolidation beyond existing common text fallback.
• Generated documentation system.