Add outlook schema transition guide
This commit is contained in:
87
docs/roadmap/outlook-schema-transition.md
Normal file
87
docs/roadmap/outlook-schema-transition.md
Normal file
@@ -0,0 +1,87 @@
|
|||||||
|
# SPC Outlook Postgres Schema Transition
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
This document describes how to reset existing Postgres outlook tables from the
|
||||||
|
`weather.outlook.v1` storage shape to the `weather.outlook.v2` compatible
|
||||||
|
storage shape.
|
||||||
|
|
||||||
|
Updated `weatherfeeder` versions create outlook tables with run-level
|
||||||
|
discussion storage. Existing databases that already contain the old outlook
|
||||||
|
table family need a manual reset because the Postgres sink creates tables with
|
||||||
|
`CREATE TABLE IF NOT EXISTS`.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
This reset drops only the outlook table family and lets updated `weatherfeeder`
|
||||||
|
recreate it:
|
||||||
|
|
||||||
|
- `outlook_discussions`
|
||||||
|
- `outlooks`
|
||||||
|
- `outlook_runs`
|
||||||
|
|
||||||
|
Other weather tables are not affected.
|
||||||
|
|
||||||
|
## Warning
|
||||||
|
|
||||||
|
These commands delete stored SPC outlook history. Existing `weather.outlook.v1`
|
||||||
|
outlook rows are intentionally removed. Downstream readers should be updated
|
||||||
|
intentionally for the new outlook shape.
|
||||||
|
|
||||||
|
## Deployment Order
|
||||||
|
|
||||||
|
1. Stop `weatherfeeder`.
|
||||||
|
2. Drop the existing outlook tables.
|
||||||
|
3. Deploy updated `weatherfeeder`.
|
||||||
|
4. Start `weatherfeeder` so the Postgres sink recreates the new outlook tables.
|
||||||
|
5. Deploy updated downstream consumers such as `weatherapi`.
|
||||||
|
|
||||||
|
## Reset SQL
|
||||||
|
|
||||||
|
```sql
|
||||||
|
DROP TABLE IF EXISTS outlook_discussions;
|
||||||
|
DROP TABLE IF EXISTS outlooks;
|
||||||
|
DROP TABLE IF EXISTS outlook_runs;
|
||||||
|
```
|
||||||
|
|
||||||
|
## Verification SQL
|
||||||
|
|
||||||
|
Before or after the updated daemon starts, this query shows which outlook tables
|
||||||
|
exist:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT table_name
|
||||||
|
FROM information_schema.tables
|
||||||
|
WHERE table_name IN ('outlook_runs', 'outlooks', 'outlook_discussions')
|
||||||
|
ORDER BY table_name;
|
||||||
|
```
|
||||||
|
|
||||||
|
After the updated daemon has started and recreated the tables, verify the new
|
||||||
|
run column:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT column_name, is_nullable, data_type
|
||||||
|
FROM information_schema.columns
|
||||||
|
WHERE table_name = 'outlook_runs'
|
||||||
|
AND column_name = 'discussion_count';
|
||||||
|
```
|
||||||
|
|
||||||
|
Verify the discussion table indexes:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT indexname
|
||||||
|
FROM pg_indexes
|
||||||
|
WHERE tablename = 'outlook_discussions'
|
||||||
|
ORDER BY indexname;
|
||||||
|
```
|
||||||
|
|
||||||
|
Verify that legacy polygon-level prose columns are gone from `outlooks`:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT column_name
|
||||||
|
FROM information_schema.columns
|
||||||
|
WHERE table_name = 'outlooks'
|
||||||
|
AND column_name IN ('headline', 'summary', 'discussion');
|
||||||
|
```
|
||||||
|
|
||||||
|
The final query should return zero rows.
|
||||||
Reference in New Issue
Block a user