28 KiB
SPC Convective Outlook Implementation Plan
Purpose
Implement weatherfeeder support for Storm Prediction Center Day 1-3 convective outlooks described in docs/roadmap/spc.md. This plan is written for an LLM coding agent and should be followed stage by stage.
This is a planning document only. The implementation must preserve the existing weatherfeeder architecture: sources emit raw provider events, normalizers map raw payloads into canonical model types, and sinks persist canonical schemas.
Decisions
The following choices are fixed for this implementation:
- Scope is Day 1-3 SPC convective outlooks only.
- Day 4-8 outlooks are out of scope.
- GeoJSON files are authoritative for polygons, validity windows, issue times, outlook labels, and severity ranking.
- Day 1-3 print pages are authoritative for discussion text.
- RSS is optional supplemental metadata only and must not be required for correctness.
- Do not fetch RSS by default. Include RSS only when an optional
rss_urlsource param is configured. - A poll is atomic for required products. If any configured GeoJSON or print-page URL fails or returns a non-2xx response, return an error and emit no event.
- Use compact GeoJSON geometry in the canonical payload for auditability and downstream display.
- Use standard-library-first parsing. Do not add an HTML parsing dependency unless string-based extraction proves unmaintainable during implementation.
- Keep all new planned behavior inside
weatherfeeder; do not makeweatherapichanges in this pass.
Public Contract
Add schema constants in standards/schema.go:
SchemaRawSPCConvectiveOutlookV1 = "raw.spc.convective_outlook.v1"SchemaWeatherOutlookV1 = "weather.outlook.v1"
Add source driver:
spc_convective_outlook
Add event kind:
outlook
Add canonical model types:
model.WeatherOutlookRunmodel.WeatherOutlook
Canonical run fields:
type WeatherOutlookRun struct {
LocationID string `json:"locationId,omitempty"`
LocationName string `json:"locationName,omitempty"`
Latitude *float64 `json:"latitude,omitempty"`
Longitude *float64 `json:"longitude,omitempty"`
AsOf time.Time `json:"asOf"`
IssuedAt *time.Time `json:"issuedAt,omitempty"`
Outlooks []WeatherOutlook `json:"outlooks"`
}
Canonical outlook fields:
type WeatherOutlook struct {
ID string `json:"id"`
Provider string `json:"provider"`
Product string `json:"product"`
Day int `json:"day"`
OutlookType string `json:"outlookType"`
Label string `json:"label"`
LabelText string `json:"labelText,omitempty"`
SeverityRank *int `json:"severityRank,omitempty"`
ValidFrom time.Time `json:"validFrom"`
ValidTo time.Time `json:"validTo"`
IssuedAt time.Time `json:"issuedAt"`
ExpiresAt time.Time `json:"expiresAt"`
Forecaster string `json:"forecaster,omitempty"`
Headline string `json:"headline,omitempty"`
Summary string `json:"summary,omitempty"`
Discussion string `json:"discussion,omitempty"`
SourceURL string `json:"sourceUrl,omitempty"`
ImageURL string `json:"imageUrl,omitempty"`
ContainsLocation bool `json:"containsLocation"`
Geometry json.RawMessage `json:"geometry"`
}
Required canonical fields:
- Run:
asOf,outlooks. - Outlook:
id,provider,product,day,outlookType,label,validFrom,validTo,issuedAt,expiresAt,containsLocation,geometry.
Canonical values:
providerisspc.productisconvective.outlookTypeis one ofcategorical,tornado,hail,wind.dayis one of1,2,3.
Source Inputs
Default required GeoJSON products:
https://www.spc.noaa.gov/products/outlook/day1otlk_cat.nolyr.geojsonhttps://www.spc.noaa.gov/products/outlook/day1otlk_torn.nolyr.geojsonhttps://www.spc.noaa.gov/products/outlook/day1otlk_hail.nolyr.geojsonhttps://www.spc.noaa.gov/products/outlook/day1otlk_wind.nolyr.geojsonhttps://www.spc.noaa.gov/products/outlook/day2otlk_cat.nolyr.geojsonhttps://www.spc.noaa.gov/products/outlook/day2otlk_torn.nolyr.geojsonhttps://www.spc.noaa.gov/products/outlook/day2otlk_hail.nolyr.geojsonhttps://www.spc.noaa.gov/products/outlook/day2otlk_wind.nolyr.geojsonhttps://www.spc.noaa.gov/products/outlook/day3otlk_cat.nolyr.geojsonhttps://www.spc.noaa.gov/products/outlook/day3otlk_torn.nolyr.geojsonhttps://www.spc.noaa.gov/products/outlook/day3otlk_hail.nolyr.geojsonhttps://www.spc.noaa.gov/products/outlook/day3otlk_wind.nolyr.geojson
Default required print-page products:
https://www.spc.noaa.gov/products/outlook/day1otlk_prt.htmlhttps://www.spc.noaa.gov/products/outlook/day2otlk_prt.htmlhttps://www.spc.noaa.gov/products/outlook/day3otlk_prt.html
Optional RSS product:
https://www.spc.noaa.gov/products/spcacrss.xml
Recommended config shape:
- name: SPCConvectiveOutlookSTL
mode: poll
kinds: ["outlook"]
driver: spc_convective_outlook
every: 30m
params:
latitude: 38.6239
longitude: -90.3571
location_id: "stl"
location_name: "St. Louis, MO"
user_agent: "HomeOps (eric@maximumdirect.net)"
Optional source params:
geojson_urls: map from product key to URL, used by tests and future upstream changes.discussion_urls: map from day key to URL, used by tests and future upstream changes.rss_url: string; when non-empty, fetch RSS as supplemental metadata.
Product keys for geojson_urls:
day1_categorical,day1_tornado,day1_hail,day1_windday2_categorical,day2_tornado,day2_hail,day2_windday3_categorical,day3_tornado,day3_hail,day3_wind
Discussion keys for discussion_urls:
day1,day2,day3
Raw Bundle Shape
Create a provider raw bundle type under internal/providers/spc or internal/normalizers/spc and use it consistently between source tests and normalizer tests. Prefer internal/providers/spc if source metadata extraction and normalizer parsing share helpers.
Suggested raw payload shape:
type RawConvectiveOutlookBundle struct {
LocationID string `json:"locationId,omitempty"`
LocationName string `json:"locationName,omitempty"`
Latitude float64 `json:"latitude"`
Longitude float64 `json:"longitude"`
FetchedAt time.Time `json:"fetchedAt"`
Products []RawOutlookProduct `json:"products"`
Discussions []RawDiscussionPage `json:"discussions"`
RSS *RawRSSFeed `json:"rss,omitempty"`
}
type RawOutlookProduct struct {
Key string `json:"key"`
Day int `json:"day"`
OutlookType string `json:"outlookType"`
URL string `json:"url"`
FetchedAt time.Time `json:"fetchedAt"`
Body json.RawMessage `json:"body"`
}
type RawDiscussionPage struct {
Key string `json:"key"`
Day int `json:"day"`
URL string `json:"url"`
FetchedAt time.Time `json:"fetchedAt"`
Body string `json:"body"`
}
type RawRSSFeed struct {
URL string `json:"url"`
FetchedAt time.Time `json:"fetchedAt"`
Body string `json:"body"`
}
Do not put parsed canonical fields into the raw bundle except configured metadata and product keys needed to identify each fetched upstream document. Source-level timestamp parsing is allowed only for event effectiveAt selection.
Stage 1: Provider Helpers And Fixtures
Goal: add deterministic SPC parsing primitives and test fixtures before wiring the source or normalizer.
Files to add:
internal/providers/spc/doc.gointernal/providers/spc/time.gointernal/providers/spc/product.gointernal/providers/spc/geojson.gointernal/providers/spc/discussion.gointernal/providers/spc/rss.go, only if optional RSS parsing is implementedinternal/providers/spc/testdata/day1_cat.geojsoninternal/providers/spc/testdata/day2_torn.geojsoninternal/providers/spc/testdata/day3_wind.geojsoninternal/providers/spc/testdata/day1_prt.htmlinternal/providers/spc/testdata/day2_prt_corr.htmlinternal/providers/spc/testdata/day3_prt.html
Provider helper behavior:
- Define stable product metadata for the 12 required GeoJSON products.
- Define stable discussion metadata for the 3 required print-page products.
- Parse SPC ISO timestamps from GeoJSON properties using
time.Parse(time.RFC3339, value)after trimming whitespace. - Decode enough GeoJSON to expose feature properties and raw geometry without owning canonical mapping.
- Preserve raw geometry as compact JSON bytes for later canonical use.
- Extract print-page product text from the first useful
<pre>block. - Strip embedded
<script>blocks and remaining tags from extracted<pre>content. - Use
html.UnescapeString, normalize CRLF to LF, and trim surrounding blank lines. - Parse print-page
Updated:timestamps when present. - Parse discussion headline/product title from product text.
- Parse
...SUMMARY...content through the next section heading. - Preserve full product text as discussion text after cleanup.
- Preserve correction markers such as
CORR 1in headline or metadata when present.
Dependency rule:
- Start with
strings,regexp,encoding/json,encoding/xml, andhtmlfrom the standard library. - Do not add
golang.org/x/net/htmlor another HTML parser unless tests show the string extraction is too brittle.
Tests:
go test ./internal/providers/spc- Product metadata contains exactly 12 GeoJSON products in day/type order.
- Discussion metadata contains exactly 3 print pages in day order.
- GeoJSON fixture decode exposes
VALID_ISO,EXPIRE_ISO,ISSUE_ISO,FORECASTER,LABEL,LABEL2,DN, and geometry. - Print-page fixture extraction returns product text without scripts or tags.
- Day 2 correction fixture preserves
CORR 1. - Summary extraction returns only the summary paragraph content.
- Updated timestamp parser returns UTC time when present and nil when absent.
Stage completion criteria:
- Provider helper tests pass.
- No source, normalizer, model, standards, sink, config, or docs current-behavior files are changed in this stage unless needed for package compilation.
Stage 2: Geometry Helper
Goal: implement point-in-polygon support independent of SPC parsing.
Files to add:
internal/geo/geojson.gointernal/geo/point.gointernal/geo/point_test.go
Behavior:
- Support GeoJSON
PolygonandMultiPolygononly. - Accept geometry as
json.RawMessageor[]byte. - Interpret GeoJSON coordinate order as
[longitude, latitude]. - Treat the first ring as the exterior ring.
- Treat subsequent rings as holes.
- Count boundary points as inside.
- Return a clear error for unsupported geometry types, malformed coordinates, empty rings, or invalid JSON.
- Use planar ray casting. This is sufficient for operational point-in-polygon checks at SPC polygon scale.
Tests:
- Point inside simple polygon returns true.
- Point outside simple polygon returns false.
- Point on polygon boundary returns true.
- Point in a hole returns false.
- Point inside one
MultiPolygonmember returns true. - Longitude/latitude ordering regression test fails if coordinates are reversed.
- Unsupported geometry returns a useful error.
Verification:
go test ./internal/geo
Stage completion criteria:
- Geometry helper has no dependency on SPC, source, normalizer, or sink packages.
Stage 3: Source Driver And Raw Schema
Goal: emit raw SPC outlook bundles from a new poll source.
Files to update:
standards/schema.gointernal/sources/builtins.gointernal/sources/builtins_test.go
Files to add:
internal/sources/spc/convective_outlook.gointernal/sources/spc/convective_outlook_test.go
Source constructor:
- Export
NewConvectiveOutlookSource(cfg config.SourceConfig) (*ConvectiveOutlookSource, error). - Register driver
spc_convective_outlookininternal/sources/builtins.go. - Validate required params
latitudeandlongitude. - Accept optional params
location_id,location_name,geojson_urls,discussion_urls, andrss_url. - Require
user_agentwhen the existing HTTP source conventions require it. - Advertise
Kinds() []event.Kind{event.Kind("outlook")}.
Fetch behavior:
- Fetch all required GeoJSON URLs and print-page URLs every poll cycle.
- Fetch optional RSS only when
rss_urlis configured and non-empty. - Use Accept
application/geo+json, application/jsonfor GeoJSON requests. - Use Accept
text/html, application/xhtml+xmlfor print pages. - Use Accept
application/rss+xml, application/xml, text/xmlfor RSS. - Respect
context.Contexton all requests. - Do not emit partial bundles.
- If the complete raw bundle is unchanged from the previous successful poll, return no events. Implement source-local change detection by hashing the combined fetched bodies if feedkit HTTP conditional helpers do not directly support multi-document polling.
- Build the raw bundle with fetched bodies and metadata.
- Emit one raw event with kind
outlookand schemastandards.SchemaRawSPCConvectiveOutlookV1. - Use
fksources.DefaultEventID("", sourceName, effectiveAt, emittedAt).
Effective time policy:
- Prefer latest valid
ISSUE_ISOacross all GeoJSON features. - Fallback to latest valid print-page
Updated:timestamp. - Fallback to RSS
lastBuildDateif RSS was fetched and parseable. - Fallback to fetch time if no provider timestamp is parseable.
- Keep fetch-time fallback as the last source effective-time option so downstream consumers still receive a stable effective time when provider metadata is absent.
Tests:
- Builtin registry builds
spc_convective_outlookas aPollSource. Kinds()returnsoutlook.- Constructor rejects missing
latitudeorlongitude. - Poll emits one raw event with schema
raw.spc.convective_outlook.v1. - Raw bundle includes 12 products and 3 discussions when defaults are overridden to test-server URLs.
- Poll effectiveAt prefers latest GeoJSON
ISSUE_ISO. - Poll effectiveAt falls back to print-page
Updated:when GeoJSON issue times are absent. - Poll includes RSS only when
rss_urlis configured. - Unchanged response emits no events on a second poll.
- A configured GeoJSON fetch failure returns error and emits no event.
- A configured print-page fetch failure returns error and emits no event.
- Tests use
httptest.Server; do not call live SPC services.
Verification:
go test ./internal/providers/spc ./internal/sources
Stage completion criteria:
- Source emits raw bundles only.
- No canonical model, normalizer, or sink mapping is added in this stage except schema constants required for compilation.
Stage 4: Canonical Model And Normalizer
Goal: convert raw SPC bundles into weather.outlook.v1 events.
Files to update:
standards/schema.gointernal/normalizers/builtins_test.go
Files to add:
model/outlook.gointernal/normalizers/spc/convective_outlook.gointernal/normalizers/spc/register.gointernal/normalizers/spc/convective_outlook_test.go
Files to update for registration:
internal/normalizers/builtins.go
Normalizer behavior:
- Add
SPCConvectiveOutlookNormalizer. - Match only
standards.SchemaRawSPCConvectiveOutlookV1. - Decode
RawConvectiveOutlookBundle. - Build one
model.WeatherOutlookRunper raw bundle. - Map each GeoJSON feature to one
model.WeatherOutlook. - Preserve feature order within each product.
- Order products by day then outlook type in this order:
categorical,tornado,hail,wind. - Compute
containsLocationwithinternal/geousing the configured bundle latitude and longitude. - Store compact feature geometry JSON in
WeatherOutlook.Geometry. - Set output schema to
standards.SchemaWeatherOutlookV1. - Set output effectiveAt to run
AsOf. - Use
internal/normalizers/common.Finalizeor existing normalizer helper patterns so envelope handling remains consistent.
GeoJSON property mapping:
VALID_ISOmaps tovalidFromand is required.EXPIRE_ISOmaps tovalidToandexpiresAtand is required.ISSUE_ISOmaps toissuedAtand is required.FORECASTERmaps toforecasterand is optional.LABELmaps tolabeland is required.LABEL2maps tolabelTextand is optional.DNmaps toseverityRankand is optional.
Derived mapping:
providerisspc.productisconvective.dayandoutlookTypecome from raw product metadata, not from brittle URL parsing when product metadata is available.idis deterministic: join day, outlook type, normalized label, issuedAt UTC in RFC3339 format, validFrom UTC in RFC3339 format, and product-local feature index. Use a stable ASCII-safe format.sourceUrlis the GeoJSON product URL for geometry/probability features.imageUrlis empty for v1.
Discussion enrichment:
- Parse each print-page discussion with
internal/providers/spchelpers. - Attach Day 1 print-page headline, summary, and discussion to Day 1 outlooks, Day 2 to Day 2 outlooks, and Day 3 to Day 3 outlooks.
- Use the same discussion text for categorical, tornado, hail, and wind outlooks for the same day.
- If a print page is fetched but discussion extraction fails, normalization should fail because print pages are required v1 inputs.
- RSS metadata must not be required for discussion enrichment.
Run timestamp policy:
AsOfis latest validissuedAtacross all outlook features.- Fallback to latest print-page
Updated:timestamp. - Fallback to input event
EffectiveAt. - Fallback to input event
EmittedAt. IssuedAtis latest validissuedAtacross all outlook features when any feature exists; otherwise nil.
Error behavior:
- Fail normalization if required GeoJSON timestamps are missing or unparseable.
- Fail normalization if required
LABELis empty. - Fail normalization if required geometry is missing or invalid.
- Fail normalization if latitude or longitude is missing or invalid in the raw bundle.
- Include product key and feature index in errors.
Tests:
- Normalizer matches only
raw.spc.convective_outlook.v1. - Builtin normalizer order includes the SPC normalizer after existing provider normalizers unless a specific order is needed.
- Canonical schema is
weather.outlook.v1. - Categorical fixture maps expected day, type, label, label text, severity rank, valid times, issue time, forecaster, source URL, and geometry.
- Probabilistic fixtures for tornado, hail, and wind map expected
outlookTypevalues. containsLocationis true for a fixture point inside a polygon.containsLocationis false for a fixture point outside a polygon.- Print-page text maps to headline, summary, and discussion.
- Day 2 correction marker
CORR 1is preserved in headline or discussion metadata. - Missing optional RSS still normalizes successfully.
- Invalid required timestamp fails with product key and feature index context.
- Invalid geometry fails with product key and feature index context.
- Output JSON uses the intended field names and does not expose raw bundle internals.
Verification:
go test ./model ./internal/geo ./internal/providers/spc ./internal/normalizers ./internal/normalizers/spc
Stage completion criteria:
- A raw SPC bundle normalizes into canonical
model.WeatherOutlookRun. - No Postgres schema or current-behavior docs are updated in this stage unless the canonical schema documentation is intentionally updated with implementation.
Stage 5: Postgres Sink
Goal: persist canonical outlook events through the weatherfeeder Postgres sink.
Files to update:
internal/sinks/postgres/schema.gointernal/sinks/postgres/map.gointernal/sinks/postgres/schema_test.gointernal/sinks/postgres/map_test.gointernal/sinks/postgres/doc.go
Tables:
outlook_runsoutlooks
Add table constants:
tableOutlookRuns = "outlook_runs"tableOutlooks = "outlooks"
outlook_runs columns:
event_id TEXT NOT NULLevent_kind TEXT NOT NULLevent_source TEXT NOT NULLevent_schema TEXT NOT NULLevent_emitted_at TIMESTAMPTZ NOT NULLevent_effective_at TIMESTAMPTZ NULLlocation_id TEXT NULLlocation_name TEXT NULLlatitude DOUBLE PRECISION NULLlongitude DOUBLE PRECISION NULLas_of TIMESTAMPTZ NOT NULLissued_at TIMESTAMPTZ NULLoutlook_count INTEGER NOT NULL
outlook_runs keys and indexes:
- Primary key:
event_id - Prune column:
as_of - Index
idx_wf_outlook_run_location_as_ofonlocation_id, as_of - Index
idx_wf_outlook_run_as_ofonas_of
outlooks columns:
run_event_id TEXT REFERENCES outlook_runs(event_id) ON DELETE CASCADE NOT NULLoutlook_index INTEGER NOT NULLas_of TIMESTAMPTZ NOT NULLproduct TEXT NOT NULLday INTEGER NOT NULLoutlook_type TEXT NOT NULLlabel TEXT NOT NULLlabel_text TEXT NULLseverity_rank INTEGER NULLvalid_from TIMESTAMPTZ NOT NULLvalid_to TIMESTAMPTZ NOT NULLissued_at TIMESTAMPTZ NOT NULLexpires_at TIMESTAMPTZ NOT NULLforecaster TEXT NULLheadline TEXT NULLsummary TEXT NULLdiscussion TEXT NULLsource_url TEXT NULLimage_url TEXT NULLcontains_location BOOLEAN NOT NULLgeometry_json TEXT NOT NULL
outlooks keys and indexes:
- Primary key:
run_event_id, outlook_index - Prune column:
as_of - Index
idx_wf_outlooks_contains_validoncontains_location, valid_from, valid_to - Index
idx_wf_outlooks_day_type_labelonday, outlook_type, label - Index
idx_wf_outlooks_validonvalid_from, valid_to
Mapper behavior:
- Extend
mapPostgresEventforstandards.SchemaWeatherOutlookV1. - Decode
model.WeatherOutlookRun. - Require run
AsOf. - Map envelope columns exactly like existing parent run tables.
- Store all times as UTC.
- Write one
outlook_runsrow and oneoutlooksrow per outlook. - Use
outlook_indexas the zero-based slice index. - Require outlook
ID,Provider,Product,Day,OutlookType,Label,ValidFrom,ValidTo,IssuedAt,ExpiresAt, and non-emptyGeometry. - Store compact geometry JSON text in
geometry_jsonusing the existing compact JSON helper or a similar helper. - Preserve all outlook polygons, not only those containing the configured point.
Tests:
- Schema includes
outlook_runsandoutlooks. - Schema includes required columns and indexes.
- Mapper writes one run row plus one row per outlook.
- Mapper stores UTC times.
- Mapper stores compact geometry JSON.
- Mapper preserves
contains_locationfalse as false, not nil. - Mapper rejects missing run
asOf. - Mapper rejects missing required outlook times.
- Mapper rejects empty geometry.
- Mapper ignores unrelated schemas unchanged.
Verification:
go test ./internal/sinks/postgres
Stage completion criteria:
- Postgres schema and mapper tests pass.
- No weatherapi read path is added in this stage.
Stage 6: Config, Runtime Wiring, And Current-Behavior Docs
Goal: expose the completed source/normalizer/sink behavior in maintained configuration and documentation after implementation exists.
Files to update:
cmd/weatherfeeder/config.ymlcmd/weatherfeeder/main_test.go, if config load expectations need updatesdocs/config.mddocs/integrations/events.mddocs/integrations/postgres.mddocs/internal/sources.mddocs/internal/normalizers.mddocs/internal/postgres-sink.mdREADME.md
Config changes:
- Add sample source
SPCConvectiveOutlookSTLwith driverspc_convective_outlook, kindoutlook,every: 30m, latitude, longitude, location metadata, anduser_agent. - Add
outlookroute examples for stdout, NATS, and Postgres where the existing config style includes kind lists. - Keep optional RSS disabled in sample config unless there is a specific operator reason to include it.
Docs changes:
- Update current-behavior docs only after the code for that behavior exists.
docs/config.mdshould document required and optional source params.docs/integrations/events.mdshould documentweather.outlook.v1, including field definitions, required fields, optional fields, geometry semantics, andcontainsLocationsemantics.docs/integrations/postgres.mdshould documentoutlook_runsandoutlooks.- Internal docs should explain provider boundaries, print-page discussion parsing, and point-in-polygon behavior where useful for maintainers.
README.mdshould only briefly list SPC convective outlook support and link to canonical docs.- Do not document weatherapi endpoints in weatherfeeder current-behavior docs.
Tests:
- Existing config load tests pass.
- Add config test coverage if the sample config is expected to build scheduler jobs for the new source.
- Documentation examples should use the real driver name and event kind.
Verification:
go test ./cmd/weatherfeeder ./internal/sources ./internal/normalizers ./internal/sinks/postgres
Stage completion criteria:
- Sample config remains loadable.
- Current-behavior docs match implemented code.
- No roadmap-only claims leak into non-roadmap docs beyond the implemented behavior.
Stage 7: Full Verification And Cleanup
Goal: validate the complete feature and remove implementation-only rough edges.
Run focused tests:
go test ./internal/providers/spc ./internal/geo ./internal/sources ./internal/normalizers/... ./internal/sinks/postgres ./model ./cmd/weatherfeeder
Run full tests:
go test ./...
Manual review checklist:
standards/schema.gocontains raw and canonical SPC schema constants.model/outlook.gouses stable JSON tags and no provider-specific names except canonical strings.- Source registry includes
spc_convective_outlook. - Normalizer registry includes the SPC normalizer.
- Source tests do not use live SPC services.
- Normalizer tests use fixtures and cover discussion parsing, geometry, and timestamp failures.
- Postgres tests cover schema shape and mapper validation.
- Docs follow
docs/policy/documentation.mdand use canonical homes. - No weatherapi files are changed.
- No broad dependency was added without clear justification.
Acceptance criteria:
- A configured SPC source emits one raw bundled
outlookevent per changed complete product snapshot. - Normalization produces
weather.outlook.v1events. - All Day 1-3 categorical, tornado, hail, and wind products are represented.
- Each outlook includes
containsLocationfor the configured latitude and longitude. - Each outlook preserves compact GeoJSON geometry.
- Day 1-3 print-page discussion text is preserved in canonical headline, summary, and discussion fields where parseable.
- RSS metadata is optional and supplemental only.
- Postgres sink persists outlook runs and outlook rows.
- Sample config and current-behavior docs describe the implemented kind, driver, schema, and storage contract.
Suggested Prompt Boundaries
This feature is too broad for a single safe implementation prompt. Use these implementation prompts in order:
- Implement Stage 1 provider helpers and fixtures only.
- Implement Stage 2 geometry helper only.
- Implement Stage 3 source driver and raw schema only.
- Implement Stage 4 canonical model and normalizer only.
- Implement Stage 5 Postgres sink mapping only.
- Implement Stage 6 config and current-behavior documentation only.
- Run Stage 7 verification and perform targeted fixes only.
Each prompt should run its stage-specific tests before moving on. Do not proceed to the next stage with failing tests unless the failure is unrelated and explicitly documented.