From 77bd59cb8704d642dc09c1d746d0d427f07ad438 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Wed, 10 Jun 2026 14:59:20 -0500 Subject: [PATCH] Add policy documents and a roadmap to implement a full documentation set --- docs/policy/architecture.md | 179 +++++++++++++ docs/policy/documentation.md | 356 ++++++++++++++++++++++++ docs/roadmap/documentation.md | 491 ++++++++++++++++++++++++++++++++++ 3 files changed, 1026 insertions(+) create mode 100644 docs/policy/architecture.md create mode 100644 docs/policy/documentation.md create mode 100644 docs/roadmap/documentation.md diff --git a/docs/policy/architecture.md b/docs/policy/architecture.md new file mode 100644 index 0000000..6ff9584 --- /dev/null +++ b/docs/policy/architecture.md @@ -0,0 +1,179 @@ +# Architecture Policy + +## Purpose + +This document defines `weatherfeeder`'s development architecture and invariants for maintainers and LLM coding agents. It describes how the implemented system is built and how future changes should preserve its boundaries. + +This is an inward-facing policy document. User-facing wire contracts belong in [`API.md`](../../API.md), and future work belongs under [`docs/roadmap/`](../roadmap/). + +## Project Shape + +`weatherfeeder` is a config-driven Go daemon that polls upstream weather providers, emits feed events, normalizes provider-specific raw payloads into canonical weather payloads, and routes those events to configured sinks. + +The implemented runtime flow is: + +1. `cmd/weatherfeeder` loads `config.yml` from the working directory. +2. Source drivers are built through the source registry. +3. Feedkit scheduler jobs poll sources and publish raw events onto an in-process event channel. +4. A pipeline runs normalization, then in-memory dedupe. +5. The dispatcher routes processed events to configured sinks by event kind. +6. Sinks consume events independently through feedkit fanout workers. + +Canonical payload structs live in `model`. Schema identifiers and cross-provider wire conventions live in `standards`. Source adapters live under `internal/sources`. Normalizers live under `internal/normalizers`. Provider-specific parsing helpers shared by sources and normalizers live under `internal/providers`. Sink-specific persistence mapping lives under `internal/sinks`. + +## Core Design Principles + +- Hexagonal boundaries: provider APIs, config loading, scheduling, dispatch, and sinks are external mechanisms around the weather domain model and normalization logic. +- Raw-to-canonical flow: sources should fetch and envelope raw provider payloads; normalizers should own provider-to-canonical mapping. +- Schema-based routing: normalizers match on event schema, not source name or event kind. +- Composable registries: source drivers, normalizers, processors, and sinks are assembled explicitly through registries. +- Bounded concurrency: scheduling and sink fanout are concurrent, but the application should keep queues, goroutine ownership, logging, and cancellation behavior visible. +- Standard-library-first: use the Go standard library unless a narrow dependency materially improves maintainability or interoperability. +- Current-behavior docs: outside roadmap files, document only implemented behavior. + +## Architectural Boundaries + +Core/domain logic: + +- `model` defines canonical payload structs and JSON field names. +- `standards` defines schema strings, versioning conventions, WMO constants, and shared wire policy. +- Normalizer mapping code is domain logic and should stay independent of CLI setup, filesystem paths, sink details, and runtime orchestration. + +Adapters: + +- Source adapters under `internal/sources/` fetch upstream data and produce raw feed events. +- Sink adapters under `internal/sinks/` map canonical events to external systems. +- Provider helper packages under `internal/providers/` contain pure parsing or provider-specific helper logic shared by sources and normalizers. + +Runtime composition: + +- `cmd/weatherfeeder/main.go` owns process wiring: config load, registry setup, scheduler jobs, processor chain, dispatcher, signal cancellation, and logging. +- It should remain thin. Do not move provider mapping or sink persistence rules into `cmd/weatherfeeder`. + +Tests and examples: + +- The sample `cmd/weatherfeeder/config.yml` is executable test input and is load-tested. +- Tests should keep exercising package contracts directly rather than relying only on full-daemon execution. + +## Modules or Stages + +The implemented processing stages are source polling, normalization, dedupe, and sink dispatch. + +Source contract: + +- Build from `feedkit/config.SourceConfig`. +- Validate required driver params inside the source constructor. +- Advertise emitted event kinds through `Kinds()` when possible. +- Emit raw schemas from `standards`. +- Decode only minimal metadata needed for event identity and effective time; leave full provider decoding to normalizers. +- Respect `context.Context` during network work. + +Normalizer contract: + +- One normalizer type per normalizer file. +- Match by `Event.Schema`. +- Decode raw payloads into provider structs. +- Map to canonical `model` payloads. +- Preserve the incoming event envelope except for intentional schema, payload, and effective-time changes. +- Use shared helpers in `internal/normalizers/common` for cross-provider behavior. +- Follow the detailed normalizer conventions in `internal/normalizers/doc.go`. + +Sink contract: + +- Consume canonical schemas, not provider raw schemas. +- Keep sink mapping isolated from normalizers and sources. +- Preserve event envelope fields in durable storage where the sink schema supports it. +- Validate required canonical fields before writing. + +## State, Inputs, and Outputs + +Inputs are configured source polls. The daemon currently uses feedkit's YAML config model with sources, sinks, and routes. + +Outputs are feed events sent to configured sinks. Implemented sink support comes from feedkit built-ins plus weatherfeeder's Postgres schema mapping. The sample config includes stdout and NATS routes; Postgres is configured as an optional commented example. + +The daemon's own state is in-process: + +- the event channel buffers events during runtime; +- the dedupe processor stores a bounded in-memory key set; +- source instances may keep HTTP conditional request state through feedkit HTTP source helpers; +- scheduler and dispatcher state is not persisted by `weatherfeeder`. + +Durable persistence is an external sink concern. The Postgres table contract is documented in `internal/sinks/postgres/doc.go`; the consumer-facing event contract is documented in [`API.md`](../../API.md). + +## Configuration and CLI Boundaries + +The implemented executable reads `config.yml` from the current working directory. It does not currently expose CLI flags or config path discovery. + +Configuration shape is owned by feedkit's config package: + +- `sources` define named source drivers, mode, poll cadence, expected kinds, and driver params. +- `sinks` define named sink drivers and sink params. +- `routes` connect event kinds to sinks. + +Weatherfeeder-specific config policy belongs in source and sink constructors, registry setup, and tests. Do not spread config parsing through domain model or normalizer packages. + +If dedicated `docs/config.md` or `docs/cli.md` files are added later, they should become the canonical user/operator references. This policy should stay architectural and avoid duplicating those references. + +## Errors, Logging, and Diagnostics + +`cmd/weatherfeeder` uses the standard library `log` package with timestamps and microseconds. + +Startup errors are fatal and include config index, source/sink name, driver name, or operation context where available. Runtime scheduler and dispatcher errors are reported through logs; context cancellation and deadline errors are treated as shutdown conditions. + +Normalizers and sink mappers should wrap errors with operation and payload context, for example decode, parse, map, scan, or required-field context. Avoid logging or returning whole upstream payloads by default. + +The daemon handles `os.Interrupt` and `SIGTERM` through `signal.NotifyContext`. Sources, sinks, scheduler jobs, dispatcher, and processors should respect `context.Context`. + +## Testing Expectations + +When changing behavior, inspect or add focused tests in the owning package. + +Expected coverage by change type: + +- Source drivers: constructor behavior, advertised kinds, poll event schema/kind, effective-time policy, unchanged responses, and malformed metadata handling. +- Normalizers: schema matching, canonical schema output, key field mapping, effective time, malformed required fields, and wire-shape regressions. +- Provider helpers: parsing edge cases and fixtures. +- Runtime wiring: config loading, source registry build, scheduler job creation, processor ordering, pass-through behavior, and dedupe behavior. +- Postgres sink: schema shape, mapper writes, required-field validation, nullable handling, and compact JSON behavior. +- Documentation-sensitive examples: keep sample config loadable. + +Use local test servers and fixtures rather than real upstream services. Full-package tests should remain fast and deterministic. + +## Dependency Policy + +Prefer the Go standard library for parsing, HTTP handling, time handling, logging, and tests where reasonable. + +Existing broad runtime composition is delegated to `feedkit`, which provides config, sources, scheduler, processors, dispatch, and sinks. Keep weatherfeeder-specific code from depending directly on low-level external clients when feedkit or a small adapter can contain that dependency. + +Third-party dependencies should be narrow, justified, and preferably de facto standard for their purpose. YAML parsing through feedkit is an acceptable example. Do not add dependencies for small conveniences, and do not let dependency-specific types leak across package boundaries unless that dependency is the package's explicit contract. + +## Documentation Expectations + +Documentation must follow [`docs/policy/documentation.md`](documentation.md). + +Rules for architecture-related docs: + +- Current-behavior docs must describe implemented behavior only. +- Roadmap or speculative work belongs only under `docs/roadmap/`. +- Prefer links to canonical docs over repeated reference material. +- Update docs in the same change when modifying schemas, config behavior, runtime behavior, adapters, or persistence contracts. + +## Architectural Invariants + +- Keep `cmd/weatherfeeder` as composition code, not domain logic. +- Keep source fetching separate from normalization. +- Keep normalizers matched by schema constants from `standards`. +- Keep canonical payload structs in `model` and treat JSON tags as wire contract. +- Keep provider-specific helpers under `internal/providers/` when shared by sources and normalizers. +- Keep cross-provider normalizer helpers pure and deterministic. +- Keep sink persistence mapping isolated under `internal/sinks/`. +- Preserve explicit registry-based extension points for sources and normalizers. +- Preserve context-aware shutdown and bounded in-process queues. +- Avoid broad dependencies without clear architectural value. + +## Non-Goals + +- `weatherfeeder` is not an HTTP API. API serving belongs to separate consumers such as `weatherapi`. +- `weatherfeeder` does not own long-term durable state except through configured external sinks. +- `weatherfeeder` does not provide a general plugin runtime; new built-in providers and sinks are registered in code. +- Architecture policy is not a CLI, config, or wire-contract reference. diff --git a/docs/policy/documentation.md b/docs/policy/documentation.md new file mode 100644 index 0000000..d9ed7fa --- /dev/null +++ b/docs/policy/documentation.md @@ -0,0 +1,356 @@ +# Go Project Documentation Policy + +## Purpose + +Project documentation must help four audiences: + +1. users who need to run the application; +2. administrators/operators who need to configure and operate it; +3. developers who need to understand and change it safely; +4. LLM coding agents that need clear scope, boundaries, and invariants. + +Docs should be accurate, concise, task-oriented, and organized by audience. Prefer links to canonical docs over repetition. + +## Core Rules + +### 1. Keep docs concise + +Each document should cover a defined scope and only the essentials for that scope. + +Avoid: +- long background explanations; +- repeated reference material; +- implementation detail in user-facing docs; +- aspirational language outside roadmap docs; +- verbose examples where one minimal example is clearer. + +### 2. Document only implemented behavior outside roadmap files + +Unimplemented, planned, aspirational, experimental, or future work may be described only under: + +- `docs/roadmap/` + +No other documentation file, including `README.md`, should describe code, features, modules, stages, commands, config fields, or behaviors that do not currently exist. + +If a feature is partial, non-roadmap docs may describe only the implemented portion and its current boundary. + +### 3. Use canonical homes + +Each type of information should have one canonical location. + +Canonical homes: + +- project purpose and quickstart: `README.md` +- development principles: `docs/policy/architecture.md` +- configuration reference: `docs/config.md` +- CLI reference: `docs/cli.md` +- operations and recovery: `docs/operations.md` +- troubleshooting: `docs/troubleshooting.md` +- implemented internals: `docs/internal/` +- future work: `docs/roadmap/` +- contributor workflow: `docs/policy/development.md` +- copyable examples: `examples/` + +Other files should summarize briefly and link to the canonical source. + +### 4. Keep examples real + +Examples should be valid, maintained, and free of secrets. + +Where practical: +- example configs should load successfully; +- example commands should match real CLI syntax; +- important examples should be covered by tests. + +## Documentation Profiles + +All projects require: + +- `README.md` +- `docs/policy/architecture.md` + +Additional docs depend on the project. + +### Small library + +Recommended: +- `docs/policy/development.md`, if contributor conventions are non-obvious + +### Simple CLI + +Required: +- `docs/cli.md` + +Recommended: +- `docs/policy/development.md` + +### Config-driven CLI + +Required: +- `docs/cli.md` +- `docs/config.md` + +Recommended: +- `examples/` +- `docs/policy/development.md` + +### Stateful or operator-facing application + +Required: +- `docs/cli.md`, if CLI-based +- `docs/config.md`, if config-driven +- `docs/operations.md` + +Recommended: +- `docs/troubleshooting.md` +- `examples/` +- `docs/policy/development.md` + +### Modular, staged, service-oriented, or orchestration application + +Required: +- `docs/cli.md`, if CLI-based +- `docs/config.md`, if config-driven +- `docs/operations.md` +- `docs/internal/` +- `docs/policy/development.md` + +Recommended: +- `docs/troubleshooting.md` +- validated examples under `examples/` + +## Required Documents + +### README.md + +**Audience:** users, administrators, operators + +The README is the outward-facing project orientation page. + +It should include, in order: + +1. concise description; +2. elevator pitch; +3. shortest useful command or usage example; +4. links to targeted docs. + +The README should be short. It is not a manual. + +The “shortest useful command” means the simplest command that performs the project’s core use case. (It does not mean `app --help`.) + +### docs/policy/architecture.md + +**Audience:** developers, LLM coding agents + +`docs/policy/architecture.md` is required for every project. + +It is an inward-facing development policy document. It should describe how the project is intended to be built and changed. + +It should include: + +- project shape; +- core design principles; +- package and boundary philosophy; +- state/persistence philosophy, if applicable; +- external integration philosophy, if applicable; +- error-handling and logging principles; +- testing expectations; +- documentation expectations; +- architectural invariants; +- explicit non-goals, if useful. + +For small projects, this file may be brief. It may simply state that the project is intentionally narrow, monolithic, and dependency-light. + +### docs/policy/development.md + +**Audience:** developers, LLM coding agents + +Required for projects maintained by humans and LLM coding agents. + +It should include: + +- repository layout; +- build/test commands; +- coding conventions; +- dependency policy; +- how to add config fields; +- how to add CLI flags; +- how to add stages/modules/adapters, if applicable; +- how to update examples; +- documentation update expectations. + +### docs/config.md + +**Audience:** administrators, operators, advanced users + +Required for applications with configuration files. + +It should include, in order: + +1. config file locations and discovery precedence; +2. minimal working config; +3. production-oriented config; +4. full configuration reference; +5. secrets handling, if applicable; +6. links to maintained examples. + +The full configuration reference should be canonical. + +### docs/cli.md + +**Audience:** users, administrators, operators + +Required for CLI applications. + +It should include, in order: + +1. shortest useful command; +2. command overview; +3. complete flag reference; +4. common workflows; +5. diagnostic or recovery commands, if applicable. + +Explain when commands are useful, not just their syntax. + +### docs/operations.md + +**Audience:** administrators, operators + +Required for applications that maintain state, support resume behavior, run multiple stages, write durable artifacts, use remote storage, or require recovery procedures. + +It should cover: + +- normal workflow; +- filesystem layout; +- remote storage layout, if applicable; +- logs and manifests; +- resume/retry behavior; +- cleanup behavior; +- archive/backup behavior; +- safe recovery procedures; +- operational caveats. + +### docs/troubleshooting.md + +**Audience:** administrators, operators + +Recommended once recurring failure modes exist. + +Each entry should include: + +- symptom; +- likely cause; +- diagnostic command or inspection step; +- safe fix; +- relevant links. + +### docs/internal/ + +**Audience:** developers, LLM coding agents + +Required for modular, staged, service-oriented, or orchestration projects. + +This directory describes implemented internal components. It is not the roadmap. + +Use one file per major component where useful. + +Each component doc should include: + +1. purpose; +2. inputs and outputs; +3. boundaries; +4. config fields used; +5. external adapters used; +6. state or manifest behavior, if applicable; +7. skip/resume behavior, if applicable; +8. failure behavior; +9. tests to inspect before changing; +10. architectural invariants. + +### docs/roadmap/ + +**Audience:** maintainers, developers, LLM coding agents + +This is the only place for planned, future, aspirational, experimental, or unimplemented work. + +Roadmap docs should clearly distinguish: + +- proposed work; +- accepted plans; +- deferred ideas; +- rejected ideas; +- implementation prompts or task breakdowns, if useful. + +Roadmap docs should not be confused with current behavior. + +### docs/integrations/ + +**Audience:** developers, LLM coding agents + +Required for projects that depend on external CLIs, APIs, services, protocols, or file formats where the integration contract is important to maintain. + +This directory contains concise, versioned reference notes for external integration contracts. It should document only the parts of the external system that this project actually uses. + +Use one file per integration where useful. + +## Examples Directory + +Projects with non-trivial configuration or workflows should include `examples/`. + +Useful examples include: + +- minimal working config; +- production-oriented config; +- full annotated config; +- local development config; +- remote/object-storage config; +- minimal session/input file. + +Examples should be valid, maintained, tested when practical, and linked from relevant docs. + +## Security and Privacy + +Docs and examples must not include: + +- real API keys; +- tokens; +- passwords; +- private keys; +- private environment dumps; +- sensitive user data; +- raw private transcripts; +- private infrastructure details unless intentionally public. + +Document secret-handling mechanisms, not actual secret values. + +## Maintenance Rules + +When docs change, verify the affected behavior. + +Where practical: + +- load example config files in tests; +- test CLI examples or command parser behavior; +- validate documented flags against real flags; +- remove stale references; +- update links after renames; +- keep roadmap content out of non-roadmap docs. + +If documentation and code disagree, fix the documentation and/or open a roadmap item; do not leave aspirational behavior in current-behavior docs. + +Documentation is complete only when it matches the current code. + +## Documentation Change Checklist + +Before merging documentation changes, verify: + +- README is concise and orientation-focused. +- `docs/policy/architecture.md` describes development principles. +- Future work appears only under `docs/roadmap/`. +- User-facing docs avoid unnecessary internals. +- Developer-facing docs preserve boundaries and invariants. +- Config examples match the schema. +- CLI examples match real commands and flags. +- Defaults appear in the canonical config reference. +- No secrets or private data are included. +- Links are accurate. diff --git a/docs/roadmap/documentation.md b/docs/roadmap/documentation.md new file mode 100644 index 0000000..fc0d2a2 --- /dev/null +++ b/docs/roadmap/documentation.md @@ -0,0 +1,491 @@ +# Documentation Roadmap + +## Purpose + +This roadmap defines the work required to bring `weatherfeeder` documentation into compliance with [`docs/policy/documentation.md`](../policy/documentation.md) and the current implementation described by [`docs/policy/architecture.md`](../policy/architecture.md). + +This is a planning document only. Implementation stages must update current-behavior documentation so it reflects implemented code, and keep proposed or unimplemented work under `docs/roadmap/`. + +## Repository Documentation Inventory + +- `README.md`: keep and rewrite. It is the correct project orientation file, but it is stale: it says stdout is the only implemented sink, uses “current MVP” framing, and lacks a shortest useful run command and links to targeted docs. +- `API.md`: move and rewrite. It currently acts as the event wire contract, but external integration contracts belong under `docs/integrations/`. It is also stale: it omits the emitted `kind` and `emitted_at` envelope fields, documents `effectiveAt` instead of the actual `effective_at` JSON field, lists only some raw schemas, and has duplicate alert `response` rows. +- `docs/policy/documentation.md`: keep and lightly update only if needed. It is the controlling documentation policy and should not be duplicated elsewhere. +- `docs/policy/architecture.md`: keep and lightly update after the documentation tree is migrated. Its architecture content is current, but its link to `API.md` should be updated when the wire contract moves to `docs/integrations/events.md`. +- `docs/roadmap/spc.md`: keep. It is correctly located under roadmap because it describes unimplemented SPC outlook support. +- `docs/roadmap/documentation.md`: create. This file is the current deliverable. +- `cmd/weatherfeeder/config.yml`: keep and treat as an executable sample, but do not use it as the only operator-facing config reference. It includes live endpoints and a commented OpenWeather URL with an apparent real API key; future cleanup should remove or replace that secret-like value. +- `internal/normalizers/doc.go`: keep and lightly update. It contains implemented developer guidance but incorrectly references `internal/standards/schema.go`; the actual package is `standards` at repository root. +- `internal/normalizers/common/doc.go`: keep and lightly update if needed. It is concise and current. +- `internal/providers/nws/doc.go`: keep and lightly update if needed. It documents implemented provider-helper boundaries. +- `internal/providers/openweather/doc.go`: keep and lightly update. It references future forecasts/alerts in a comment; outside roadmap docs this should be rewritten to current behavior only. +- `standards/doc.go`: keep and lightly update. It incorrectly says `internal/standards/doc.go`; actual path is `standards/doc.go`. +- `model/doc.go`: keep and lightly update. It says JSON tags are wire contract for “stdout today; others later,” which is stale because NATS and Postgres are implemented. +- `internal/sinks/postgres/doc.go`: keep and lightly update. It is the authoritative implemented Postgres table contract, but should be linked from `docs/integrations/postgres.md` instead of being the only discoverable operator/developer reference. +- `examples/`: create new. No examples directory exists. Current example material is embedded in `cmd/weatherfeeder/config.yml` only. +- `docs/config.md`: create new. Required because `weatherfeeder` is config-driven. +- `docs/cli.md`: create new. Required because `weatherfeeder` is a command-line executable, even though it currently has no flags. +- `docs/operations.md`: create new. Required because `weatherfeeder` is an operator-facing daemon with polling, concurrency, external sinks, retention/pruning, and shutdown behavior. +- `docs/troubleshooting.md`: create new. Recommended and useful because recurring operator failures are visible in constructors and config validation. +- `docs/policy/development.md`: create new. Required for a modular project maintained by humans and coding agents. +- `docs/internal/`: create new. Required because the project is modular and staged. +- `docs/integrations/`: create new. Required because the project depends on external weather APIs, NATS, Postgres, and an emitted event contract. + +## Policy Compliance Assessment + +Required missing documents: + +- `docs/cli.md` +- `docs/config.md` +- `docs/operations.md` +- `docs/internal/` +- `docs/policy/development.md` +- `docs/integrations/` entries for important external contracts + +Recommended missing documents: + +- `docs/troubleshooting.md` +- `examples/` with maintained config examples + +Existing stale or misplaced content: + +- `README.md` is not concise enough as a current quickstart and contains stale sink status. +- `API.md` is in the wrong canonical home for an external integration contract and does not match `feedkit/event.Event` JSON tags. +- Some package comments include stale path references or future-looking language outside roadmap files. +- The sample config contains live endpoint examples and an OpenWeather URL with an apparent credential-like `appid` value in a commented block; examples should be secret-free. + +Unimplemented, historical, or roadmap-only content outside roadmap: + +- README “current MVP” phrasing is development-history framing and should be removed. +- Comments such as “others later,” “forecasts/alerts later,” and “future expensive steps” should be rewritten or removed unless they describe an implemented boundary. +- Do not move SPC outlook content out of `docs/roadmap/spc.md` until it is implemented. + +Examples needing work: + +- There is no `examples/` directory. +- `cmd/weatherfeeder/config.yml` is load-tested by `cmd/weatherfeeder/main_test.go`, but it is not an ideal copyable public example because it includes local/operator-specific values and a commented API key-like OpenWeather URL. +- Future examples should be checked with `feedkit/config.Load` or a weatherfeeder-specific test. + +Links needing verification: + +- Links from `README.md` to `API.md` should be changed to the new canonical event contract path if `API.md` is moved. +- `docs/policy/architecture.md` links to `../../API.md`; update this after moving the contract. +- Internal package comments referring to `internal/standards` need correction to `standards`. + +## Target Documentation Set + +### `README.md` + +- Audience: users, administrators, operators. +- Purpose: orient readers and get them to the shortest useful run path. +- Canonical scope: project purpose, elevator pitch, shortest useful command, and links to deeper docs. +- Recommended outline: description; what it does; shortest useful command; emitted products and providers summary; links to config, CLI, operations, event contract, development policy. +- Source of truth: `cmd/weatherfeeder/main.go`, `cmd/weatherfeeder/config.yml`, `internal/sources/builtins.go`, `internal/normalizers/builtins.go`, `standards/schema.go`, `README.md` stale claims. +- Acceptance criteria: concise, no stale stdout-only claim, no full config reference, no development-history framing, links target canonical docs. + +### `docs/cli.md` + +- Audience: users, administrators, operators. +- Purpose: document the executable interface. +- Canonical scope: how to run `weatherfeeder` and what CLI flags exist. +- Recommended outline: shortest useful command; command overview; complete flag reference stating there are currently no flags; working-directory requirement for `config.yml`; common local/container workflows; shutdown signal behavior. +- Source of truth: `cmd/weatherfeeder/main.go`, `Dockerfile`, `cmd/weatherfeeder/main_test.go`. +- Acceptance criteria: accurately states that `config.yml` is loaded from current working directory and no config path flag exists; does not invent flags. + +### `docs/config.md` + +- Audience: administrators, operators, advanced users. +- Purpose: canonical configuration reference. +- Canonical scope: YAML shape, field rules, source/sink/route definitions, driver params, and defaults. +- Recommended outline: config file location; minimal config; production-oriented config; top-level reference; source drivers; source params; sink drivers; sink params; routes; duration formats; secrets handling; links to examples. +- Source of truth: `cmd/weatherfeeder/main.go`, `cmd/weatherfeeder/config.yml`, `../feedkit/config/config.go`, `../feedkit/config/load.go`, `../feedkit/sources/http.go`, `../feedkit/sinks/*.go`, `internal/sources/builtins.go`, source constructors. +- Acceptance criteria: documents strict YAML known-field behavior, required `sources` and `sinks`, optional `routes`, `mode`, `every`, `kinds`, HTTP params including `url`, `user_agent`, `conditional`, `http_timeout`, and `http_response_body_limit_bytes`; documents OpenWeather `units=metric` requirement; avoids embedding secrets. + +### `docs/operations.md` + +- Audience: administrators and operators. +- Purpose: explain how to run, observe, shut down, and recover the daemon. +- Canonical scope: runtime behavior, logs, scheduling, sink behavior, Postgres initialization/pruning, and operational caveats. +- Recommended outline: normal workflow; runtime lifecycle; logs; scheduler and polling behavior; conditional HTTP fetches; routing and sink fanout; Postgres table creation and retention; shutdown; recovery; caveats. +- Source of truth: `cmd/weatherfeeder/main.go`, `../feedkit/scheduler`, `../feedkit/dispatch`, `../feedkit/sources/http.go`, `../feedkit/sinks/postgres.go`, `internal/sinks/postgres/schema.go`, `Dockerfile`. +- Acceptance criteria: describes implemented behavior only; no unsupported admin commands; clearly states durable state is external sink state. + +### `docs/troubleshooting.md` + +- Audience: administrators and operators. +- Purpose: provide safe diagnosis for common implemented failure modes. +- Canonical scope: symptoms, likely causes, checks, fixes, and links. +- Recommended outline: missing `config.yml`; YAML parse or unknown-field errors; unknown source/sink driver; source kind mismatch; missing `params.url` or `params.user_agent`; OpenWeather missing `units=metric`; NATS connection failure; Postgres connection/table/credential failure; no events due to 304 unchanged responses; route sends no events. +- Source of truth: constructors and validation in feedkit/weatherfeeder, `cmd/weatherfeeder/main.go`, source tests, sink tests. +- Acceptance criteria: every entry has symptom, likely cause, diagnostic step, safe fix, and link to config/operations where useful. + +### `docs/policy/development.md` + +- Audience: developers and LLM coding agents. +- Purpose: contributor workflow and safe-change guidance. +- Canonical scope: repository layout, build/test commands, coding conventions, dependency policy, adding drivers/normalizers/schemas/config fields/docs. +- Recommended outline: layout; Go/test commands; style; dependency policy; adding a source; adding a normalizer; adding a canonical model/schema; adding Postgres mapping; updating sample config/examples/docs; review checklist. +- Source of truth: `docs/policy/architecture.md`, `go.mod`, `cmd/weatherfeeder/main_test.go`, package docs, builtins registries, tests. +- Acceptance criteria: no user/operator reference duplication; links to config/CLI docs; gives enough steps for coding agents to preserve architecture. + +### `docs/internal/runtime.md` + +- Audience: developers and LLM coding agents. +- Purpose: describe implemented runtime composition. +- Canonical scope: config load, registries, scheduler, processor chain, dispatcher, sink fanout, shutdown. +- Recommended outline: purpose; inputs/outputs; boundaries; config fields used; adapters used; state; failure behavior; tests; invariants. +- Source of truth: `cmd/weatherfeeder/main.go`, `cmd/weatherfeeder/main_test.go`, `../feedkit/scheduler`, `../feedkit/dispatch`, `../feedkit/processors`. +- Acceptance criteria: accurately describes normalize then dedupe, `dedupeMaxEntries = 2048`, event channel buffer 256, and context shutdown behavior. + +### `docs/internal/sources.md` + +- Audience: developers and LLM coding agents. +- Purpose: document implemented source drivers and source contracts. +- Canonical scope: source boundaries, registered drivers, kinds, raw schemas, params, effective-time policies at a summary level. +- Recommended outline: source contract; common HTTP behavior; driver table; provider notes; failure behavior; tests; invariants. +- Source of truth: `internal/sources/builtins.go`, `internal/sources/*`, `../feedkit/sources/http.go`, source tests, `standards/schema.go`. +- Acceptance criteria: includes all current drivers and no legacy `nws_forecast`; documents raw payload preservation and conditional GET behavior without duplicating full config reference. + +### `docs/internal/normalizers.md` + +- Audience: developers and LLM coding agents. +- Purpose: document raw-to-canonical mapping boundaries and registration. +- Canonical scope: normalizer contract, schema matching, provider packages, output schemas, tests. +- Recommended outline: contract; registration/order; mapping table from raw schemas to canonical schemas; common helpers; failure behavior; tests; invariants. +- Source of truth: `internal/normalizers/doc.go`, `internal/normalizers/builtins.go`, provider normalizer packages, `standards/schema.go`, normalizer tests. +- Acceptance criteria: fixes stale `internal/standards` references; avoids duplicating every payload field from event contract. + +### `docs/internal/postgres-sink.md` + +- Audience: developers and LLM coding agents. +- Purpose: explain weatherfeeder-owned Postgres mapping internals. +- Canonical scope: schema registration, mapper behavior, required-field validation, table contract link. +- Recommended outline: purpose; input schemas; writes; parent/child tables; nullable rules; pruning columns; tests; invariants. +- Source of truth: `internal/sinks/postgres/doc.go`, `schema.go`, `map.go`, sink tests, feedkit Postgres sink. +- Acceptance criteria: links to `docs/integrations/postgres.md` for operator-facing table contract; does not duplicate full table definitions unless that integration doc remains intentionally concise. + +### `docs/integrations/events.md` + +- Audience: downstream consumers and developers. +- Purpose: canonical emitted event wire contract. +- Canonical scope: feedkit event envelope JSON and canonical weather payload schemas. +- Recommended outline: envelope; timestamp rules; kinds and schemas; raw schema note; shared conventions; canonical payloads; compatibility rules; compact examples. +- Source of truth: `../feedkit/event/event.go`, `standards/schema.go`, `model/*.go`, `internal/normalizers/common/round.go`, normalizer tests. +- Acceptance criteria: documents actual JSON field names (`id`, `kind`, `source`, `emitted_at`, `effective_at`, `schema`, `payload`); includes all current raw and canonical schemas; states forecast period `conditionCode` is optional; examples validate against current structs. + +### `docs/integrations/postgres.md` + +- Audience: downstream SQL consumers, administrators, developers. +- Purpose: canonical Postgres table contract for weatherfeeder writes. +- Canonical scope: tables, columns, keys, indexes, pruning, reconstruction notes, migration caveats. +- Recommended outline: scope; sink config link; initialization behavior; table overview; full table contract; pruning; migrations; reconstruction of canonical payloads. +- Source of truth: `internal/sinks/postgres/doc.go`, `schema.go`, `map.go`, `../feedkit/sinks/postgres.go`, schema/map tests. +- Acceptance criteria: matches generated schema exactly; notes `CREATE TABLE IF NOT EXISTS` does not alter existing schemas; documents `forecast_periods.condition_code` nullable. + +### `docs/integrations/nws.md` + +- Audience: developers and operators maintaining NWS integrations. +- Purpose: concise notes for implemented NWS endpoint usage. +- Canonical scope: only endpoints and fields weatherfeeder currently uses. +- Recommended outline: supported drivers; endpoint shapes; required user agent; accept headers; effective-time policies; parser caveats; tests. +- Source of truth: `internal/sources/nws`, `internal/normalizers/nws`, `internal/providers/nws`, NWS tests and fixtures. +- Acceptance criteria: no undocumented SPC outlook content; weather stories and forecast discussions included as implemented. + +### `docs/integrations/openmeteo.md` + +- Audience: developers and operators maintaining Open-Meteo integrations. +- Purpose: concise notes for implemented Open-Meteo observation and hourly forecast usage. +- Canonical scope: current endpoint usage, time parsing, fields mapped, effective time. +- Source of truth: `internal/sources/openmeteo`, `internal/normalizers/openmeteo`, `internal/providers/openmeteo`, tests. +- Acceptance criteria: no daily forecast or unsupported products documented as current. + +### `docs/integrations/openweather.md` + +- Audience: developers and operators maintaining OpenWeather integration. +- Purpose: concise notes for implemented OpenWeather observation usage. +- Canonical scope: current observation driver, `units=metric` requirement, timestamp behavior, mapped fields. +- Source of truth: `internal/sources/openweather`, `internal/normalizers/openweather`, `internal/providers/openweather`, tests. +- Acceptance criteria: no forecast/alert OpenWeather support documented as current. + +### `examples/config.minimal.yml` + +- Audience: users and operators. +- Purpose: copyable minimal config. +- Canonical scope: one source, one stdout sink, one route. +- Source of truth: `cmd/weatherfeeder/config.yml`, config tests, source constructors. +- Acceptance criteria: loads with `config.Load`; no secrets; uses placeholder-safe or public endpoints as appropriate. + +### `examples/config.nats.yml` + +- Audience: operators. +- Purpose: copyable NATS publishing config. +- Canonical scope: selected sources, NATS sink, route examples. +- Source of truth: sample config, feedkit NATS sink. +- Acceptance criteria: loads with `config.Load`; no secrets; clearly uses replaceable NATS URL/subject. + +### `examples/config.postgres.yml` + +- Audience: operators. +- Purpose: copyable Postgres persistence config. +- Canonical scope: selected sources, Postgres sink params, pruning example, routes. +- Source of truth: sample config, feedkit Postgres sink, weatherfeeder Postgres schema. +- Acceptance criteria: loads with `config.Load`; uses placeholders for username/password/URI; links from config and operations docs. + +## File-by-File Rewrite Guidance + +### `README.md` + +Cover what the daemon does, the shortest useful command, and where to go next. Use implemented provider and product summaries from source/normalizer registries. Avoid full config tables, full wire schemas, history, “MVP” language, and stale sink limitations. Link to `docs/cli.md`, `docs/config.md`, `docs/operations.md`, `docs/integrations/events.md`, and `docs/policy/development.md`. + +Do not carry forward the claim that stdout is the only implemented sink. + +### `API.md` + +Move its canonical content to `docs/integrations/events.md` and either delete `API.md` or replace it with a short pointer only if backward-compatible repository links are required. The rewritten contract must inspect `../feedkit/event/event.go` before documenting envelope fields. Do not preserve camelCase envelope examples unless the code changes first. + +### `docs/config.md` + +Use feedkit config structs for generic fields and weatherfeeder source/sink constructors for driver-specific params. Link to examples instead of embedding all large configs. Avoid documenting feedkit stream-only params as actively used by weatherfeeder unless clearly marked as generic feedkit config accepted by the current config model; all current weatherfeeder sources are poll sources. + +### `docs/cli.md` + +State the current no-flags behavior plainly. Include that `weatherfeeder` expects `config.yml` in the current working directory. Do not document environment variables, subcommands, or alternate config paths unless implemented. + +### `docs/operations.md` + +Focus on running the daemon and operating sinks. Include signal shutdown, log behavior, conditional HTTP fetches, Postgres table creation, retention pruning, and no internal durable scheduler state. Avoid unsupported recovery commands. + +### `docs/troubleshooting.md` + +Base entries on actual validation and constructor errors. Link to `docs/config.md` for fixes. Avoid speculative upstream outages beyond generic HTTP/source errors unless tests or code reveal specific behavior. + +### `docs/policy/development.md` + +Use `docs/policy/architecture.md` as the authority. Give concrete safe-change steps for adding source drivers, normalizers, schemas, Postgres tables, config examples, and docs. Avoid repeating the architecture policy wholesale. + +### `docs/internal/*.md` + +Create concise component docs. They should guide changes, not become manuals. Each should include tests to inspect before changing. Link to package docs when those are more precise. + +### `docs/integrations/*.md` + +Document only external contracts actually used by the implementation. Keep provider docs narrow and maintenance-oriented. Do not describe unimplemented SPC support outside `docs/roadmap/spc.md`. + +### Package comments + +Fix stale path and future-language comments during the documentation migration stage that touches developer docs. Specific likely fixes: + +- `model/doc.go`: remove “stdout today; others later.” +- `standards/doc.go`: correct `internal/standards/doc.go` path. +- `internal/normalizers/doc.go`: correct `internal/standards/schema.go` path. +- `internal/providers/openweather/doc.go`: remove “forecasts/alerts later.” + +## Examples Plan + +No `examples/` directory currently exists. Create one because the project has non-trivial configuration and operator workflows. + +Recommended examples: + +- `examples/config.minimal.yml`: one public HTTP source, stdout sink, route. Validity check: `config.Load` in a new or existing config/example test. Link from README, config, CLI. +- `examples/config.nats.yml`: NATS sink publishing example. Validity check: `config.Load`; do not require a live NATS server. Link from config and operations. +- `examples/config.postgres.yml`: Postgres sink with placeholder credentials and optional `prune`. Validity check: `config.Load`; do not require a live Postgres server. Link from config, operations, Postgres integration docs. +- `examples/config.full.yml`: optional, only if the team wants one annotated config covering all implemented drivers. Validity check: `config.Load`; must use placeholders for secrets and avoid real OpenWeather API keys. Link from config docs only. + +Do not add SPC examples until SPC support is implemented. + +## Internal Documentation Plan + +### Runtime Pipeline + +- Path: `docs/internal/runtime.md` +- Purpose: explain daemon composition. +- Inputs and outputs: YAML config in; feed events through scheduler/pipeline/dispatcher to sinks out. +- Boundaries: composition belongs in `cmd/weatherfeeder`; domain mapping belongs in normalizers/model. +- Config fields used: sources, sinks, routes, source `every`, mode/kinds, sink params. +- Adapters used: feedkit scheduler, processors, dispatch, sinks. +- Failure behavior: fatal startup errors; runtime errors logged and cancel context; interrupt/SIGTERM shutdown. +- Tests to inspect: `cmd/weatherfeeder/main_test.go`, feedkit scheduler/dispatch tests. +- Invariants: normalize before dedupe; bounded event channel; context-aware shutdown. + +### Sources + +- Path: `docs/internal/sources.md` +- Purpose: document source driver architecture. +- Inputs and outputs: `config.SourceConfig` and upstream HTTP responses in; raw feed events out. +- Boundaries: sources fetch and envelope raw payloads; normalizers decode full payloads. +- Config fields used: name, driver, mode, every, kinds, params URL/user agent/HTTP options. +- Adapters used: feedkit source registry and HTTP helper. +- Failure behavior: constructor validation, HTTP errors, unchanged 304 returns no events, some malformed metadata still emits raw events. +- Tests to inspect: `internal/sources/builtins_test.go`, provider source tests. +- Invariants: drivers registered in one place; raw schemas from `standards`; no canonical mapping in sources. + +### Normalizers + +- Path: `docs/internal/normalizers.md` +- Purpose: document raw-to-canonical mapping architecture. +- Inputs and outputs: raw events matched by schema in; canonical model payload events out. +- Boundaries: no HTTP, no sink persistence, no CLI/config behavior. +- Config fields used: none directly. +- Adapters used: feedkit normalize processor. +- Failure behavior: decode/parse/missing required fields return contextual errors; no-match passthrough configured in main. +- Tests to inspect: `internal/normalizers/builtins_test.go`, provider normalizer tests, `internal/normalizers/common` tests. +- Invariants: schema matching only, stable registration order, canonical schema constants. + +### Postgres Sink Mapping + +- Path: `docs/internal/postgres-sink.md` +- Purpose: document weatherfeeder-specific Postgres mapping internals. +- Inputs and outputs: canonical events in; feedkit `PostgresWrite` rows out. +- Boundaries: mapper validates and maps; feedkit owns DB connection, DDL, transactions, pruning. +- Config fields used: sink `uri`, `username`, `password`, optional `prune` via feedkit. +- Adapters used: feedkit Postgres sink. +- Failure behavior: unsupported schemas map to no writes or errors per mapper behavior; required missing fields fail before write. +- Tests to inspect: `internal/sinks/postgres/map_test.go`, `schema_test.go`, feedkit Postgres tests. +- Invariants: consume canonical schemas only; preserve envelope columns; keep parent/child order indexes. + +## Integration Documentation Plan + +### Event Wire Contract + +- Path: `docs/integrations/events.md` +- External system or contract: JSON events emitted to stdout/NATS and represented in Postgres parent envelope columns. +- Current usage: sinks marshal feedkit events or map canonical events to tables. +- Version notes: schema identifiers use `raw.*.v1` and `weather.*.v1`; JSON tags are compatibility contract. +- Document: actual envelope fields, schema list, payload fields, units, optionality, examples. +- Do not document: weatherapi HTTP endpoints, unimplemented outlooks, or future schemas. + +### Postgres + +- Path: `docs/integrations/postgres.md` +- External system or contract: PostgreSQL schema created/written by feedkit Postgres sink using weatherfeeder schema definition. +- Current usage: optional configured sink; create-if-missing tables and indexes; transactional writes; optional pruning. +- Version notes: no migration framework is implemented; existing DBs may need manual schema changes when table definitions change. +- Document: params, table contract, pruning, reconstruction, migration caveats. +- Do not document: weatherapi query behavior except as a downstream consumer link if needed. + +### NWS + +- Path: `docs/integrations/nws.md` +- External system or contract: NWS API and NWS forecast discussion/weather story endpoints used by current drivers. +- Current usage: observations, alerts, hourly forecast, narrative forecast, forecast discussion, weather stories. +- Version notes: no explicit upstream API version in code; weatherfeeder pins its own raw schema names. +- Document: endpoint shapes used, required User-Agent, accept headers, effective time, raw schemas. +- Do not document: SPC outlooks or unsupported NWS products. + +### Open-Meteo + +- Path: `docs/integrations/openmeteo.md` +- External system or contract: Open-Meteo forecast API for current observations and hourly forecasts. +- Current usage: JSON HTTP source, time parsing from timezone/UTC offset, hourly fields mapped to canonical forecast. +- Version notes: no explicit upstream API version in code. +- Document: required URL/user agent params, expected current/hourly response pieces, effective time behavior. +- Do not document: daily forecast unless implemented. + +### OpenWeather + +- Path: `docs/integrations/openweather.md` +- External system or contract: OpenWeather current weather endpoint. +- Current usage: observation source/normalizer only. +- Version notes: source requires `units=metric` in URL. +- Document: metric units requirement, timestamp behavior, mapped fields, secret handling for API key. +- Do not document: OpenWeather forecasts or alerts. + +### NATS + +- Path: include in `docs/operations.md` and `docs/config.md`; create `docs/integrations/nats.md` only if NATS-specific maintenance grows. +- External system or contract: NATS publish subject configured by feedkit sink. +- Current usage: optional sink publishes each event as JSON. +- Document: `url`, `subject`, JSON event payload, connection failure behavior. +- Do not document: subscriptions or server administration beyond this sink contract. + +## Recommended Implementation Sequence + +### Stage 1: Current-State README, CLI, and Config Docs + +- Goal: establish accurate user/operator entry points. +- Files to create/update/delete/move: rewrite `README.md`; create `docs/cli.md`; create `docs/config.md`. +- Repository areas to inspect: `cmd/weatherfeeder/main.go`, `cmd/weatherfeeder/config.yml`, `../feedkit/config`, `../feedkit/sources/http.go`, `../feedkit/sinks`, `internal/sources/builtins.go`, source constructors. +- Acceptance criteria: README concise and current; CLI doc states no flags and cwd `config.yml`; config doc includes complete implemented config reference and no secrets. +- Suggested validation commands: `go test ./cmd/weatherfeeder`; `rg -n "only implemented sink|current MVP|--config|API.md" README.md docs/cli.md docs/config.md`. +- Size: small enough for one implementation prompt. + +### Stage 2: Event and Postgres Integration Contracts + +- Goal: move and correct external contracts. +- Files to create/update/delete/move: create `docs/integrations/events.md`; create `docs/integrations/postgres.md`; delete or replace `API.md` with a pointer; update links in `README.md` and `docs/policy/architecture.md`. +- Repository areas to inspect: `../feedkit/event/event.go`, `standards/schema.go`, `model/*.go`, `internal/sinks/postgres/doc.go`, `schema.go`, `map.go`, normalizer tests, sink tests. +- Acceptance criteria: event envelope field names match code; all current schemas listed; Postgres contract matches schema definition; no duplicate or stale API rows. +- Suggested validation commands: `go test ./internal/sinks/postgres ./internal/normalizers/...`; `rg -n "effectiveAt|emittedAt|API.md|response \| string.*response" README.md docs API.md`. +- Size: one implementation prompt if kept focused; split if examples are expanded heavily. + +### Stage 3: Operations and Troubleshooting Docs + +- Goal: document running and recovering the daemon. +- Files to create/update/delete/move: create `docs/operations.md`; create `docs/troubleshooting.md`. +- Repository areas to inspect: `cmd/weatherfeeder/main.go`, `Dockerfile`, feedkit scheduler/dispatch/source/sink implementations, config validation, source/sink constructors. +- Acceptance criteria: operational docs describe logs, shutdown, scheduling, conditional HTTP, fanout, Postgres create/prune behavior, and common failures without inventing commands. +- Suggested validation commands: `go test ./cmd/weatherfeeder`; `rg -n "TODO|future|planned|unimplemented" docs/operations.md docs/troubleshooting.md`. +- Size: small enough for one implementation prompt. + +### Stage 4: Examples Directory and Example Validation + +- Goal: add maintained copyable configs. +- Files to create/update/delete/move: create `examples/config.minimal.yml`, `examples/config.nats.yml`, `examples/config.postgres.yml`; optionally create `examples/config.full.yml`; update tests to load examples; update links in README/config/operations. +- Repository areas to inspect: `cmd/weatherfeeder/config.yml`, `cmd/weatherfeeder/main_test.go`, feedkit config validation, source/sink params. +- Acceptance criteria: examples contain no secrets, load successfully, and are linked from canonical docs. +- Suggested validation commands: `go test ./cmd/weatherfeeder`; `rg -n "appid=|password: [^<]|token|secret" examples cmd/weatherfeeder/config.yml docs`. +- Size: one implementation prompt. + +### Stage 5: Developer and Internal Docs + +- Goal: document safe change workflow and internal component boundaries. +- Files to create/update/delete/move: create `docs/policy/development.md`; create `docs/internal/runtime.md`; create `docs/internal/sources.md`; create `docs/internal/normalizers.md`; create `docs/internal/postgres-sink.md`; lightly update package comments with stale paths/future language. +- Repository areas to inspect: architecture policy, package docs, registries, source/normalizer/sink tests, `go.mod`. +- Acceptance criteria: docs are inward-facing, concise, and action-oriented; package comments no longer contain stale path or future-support language outside roadmap. +- Suggested validation commands: `go test ./...`; `rg -n "internal/standards|others later|forecasts/alerts later|future expensive steps" . -g '*.go' -g '*.md'`. +- Size: likely one implementation prompt, but split package-comment cleanup if code comment edits are considered too broad. + +### Stage 6: Provider Integration Notes + +- Goal: document external provider contracts actually used by implemented source/normalizer pairs. +- Files to create/update/delete/move: create `docs/integrations/nws.md`, `docs/integrations/openmeteo.md`, `docs/integrations/openweather.md`; optionally create `docs/integrations/nats.md` only if NATS content should stand alone. +- Repository areas to inspect: provider source packages, provider normalizer packages, `internal/providers`, provider tests/fixtures. +- Acceptance criteria: each integration doc is narrow, current, and excludes unimplemented products; NWS doc does not include SPC outlooks except linking to roadmap if needed. +- Suggested validation commands: `go test ./internal/sources/... ./internal/normalizers/... ./internal/providers/...`; `rg -n "SPC|outlook|daily forecast|OpenWeather forecast|OpenWeather alert" docs/integrations`. +- Size: one implementation prompt. + +### Stage 7: Final Documentation Consistency Pass + +- Goal: remove stale references and verify policy compliance. +- Files to create/update/delete/move: any docs touched in prior stages; remove obsolete root `API.md` if not kept as pointer. +- Repository areas to inspect: whole documentation tree, package comments, tests. +- Acceptance criteria: every non-roadmap doc describes implemented behavior only; every doc has clear audience/scope; roadmap content remains only in `docs/roadmap`; links resolve by inspection. +- Suggested validation commands: `go test ./...`; `find docs -type f -name '*.md' -print | sort`; `rg -n "MVP|only implemented sink|planned|future|later|deprecated|experimental|TODO|API.md|effectiveAt|emittedAt|internal/standards|appid=" README.md docs internal model standards cmd examples`. +- Size: small enough for one implementation prompt. + +## Validation Plan + +Automated checks available now: + +- `go test ./...` verifies code behavior and existing load-tested sample config. +- `go test ./cmd/weatherfeeder` verifies `cmd/weatherfeeder/config.yml` loads and sources build scheduler jobs. +- Focused package tests verify source drivers, normalizers, and Postgres schema/mapping. + +Recommended new checks during implementation: + +- Add tests that load all `examples/*.yml` with `feedkit/config.Load`. +- Add or extend tests to ensure documented source drivers in config examples build through `internal/sources.RegisterBuiltins`. +- Use `rg` checks for stale or prohibited terms: `only implemented sink`, `current MVP`, `future`, `later`, `planned`, `deprecated`, `experimental`, `API.md`, `effectiveAt`, `emittedAt`, `internal/standards`, and secret-like strings such as `appid=`. +- Manually verify Markdown links because no dedicated Markdown/link checker is currently present. +- Manually compare `docs/integrations/events.md` against `../feedkit/event/event.go`, `standards/schema.go`, and `model/*.go` before accepting. +- Manually compare `docs/integrations/postgres.md` against `internal/sinks/postgres/schema.go` before accepting. + +No repository-local Markdown formatter, Markdown linter, Makefile, justfile, Taskfile, or `package.json` docs tooling was found during this planning pass. + +## Open Questions + +No question blocks implementation of this documentation migration. Recommended decisions: + +- Move the canonical event contract from `API.md` to `docs/integrations/events.md` to comply with the documentation policy. +- Replace root `API.md` with a short pointer only if backward-compatible links are considered important; otherwise delete it during the migration. +- Keep `cmd/weatherfeeder/config.yml` as the executable in-repo sample used by tests, but create public copyable examples under `examples/` and remove secret-like values from examples.