Add outlook schema transition guide
This commit is contained in:
87
docs/roadmap/outlook-schema-transition.md
Normal file
87
docs/roadmap/outlook-schema-transition.md
Normal file
@@ -0,0 +1,87 @@
|
||||
# SPC Outlook Postgres Schema Transition
|
||||
|
||||
## Purpose
|
||||
|
||||
This document describes how to reset existing Postgres outlook tables from the
|
||||
`weather.outlook.v1` storage shape to the `weather.outlook.v2` compatible
|
||||
storage shape.
|
||||
|
||||
Updated `weatherfeeder` versions create outlook tables with run-level
|
||||
discussion storage. Existing databases that already contain the old outlook
|
||||
table family need a manual reset because the Postgres sink creates tables with
|
||||
`CREATE TABLE IF NOT EXISTS`.
|
||||
|
||||
## Scope
|
||||
|
||||
This reset drops only the outlook table family and lets updated `weatherfeeder`
|
||||
recreate it:
|
||||
|
||||
- `outlook_discussions`
|
||||
- `outlooks`
|
||||
- `outlook_runs`
|
||||
|
||||
Other weather tables are not affected.
|
||||
|
||||
## Warning
|
||||
|
||||
These commands delete stored SPC outlook history. Existing `weather.outlook.v1`
|
||||
outlook rows are intentionally removed. Downstream readers should be updated
|
||||
intentionally for the new outlook shape.
|
||||
|
||||
## Deployment Order
|
||||
|
||||
1. Stop `weatherfeeder`.
|
||||
2. Drop the existing outlook tables.
|
||||
3. Deploy updated `weatherfeeder`.
|
||||
4. Start `weatherfeeder` so the Postgres sink recreates the new outlook tables.
|
||||
5. Deploy updated downstream consumers such as `weatherapi`.
|
||||
|
||||
## Reset SQL
|
||||
|
||||
```sql
|
||||
DROP TABLE IF EXISTS outlook_discussions;
|
||||
DROP TABLE IF EXISTS outlooks;
|
||||
DROP TABLE IF EXISTS outlook_runs;
|
||||
```
|
||||
|
||||
## Verification SQL
|
||||
|
||||
Before or after the updated daemon starts, this query shows which outlook tables
|
||||
exist:
|
||||
|
||||
```sql
|
||||
SELECT table_name
|
||||
FROM information_schema.tables
|
||||
WHERE table_name IN ('outlook_runs', 'outlooks', 'outlook_discussions')
|
||||
ORDER BY table_name;
|
||||
```
|
||||
|
||||
After the updated daemon has started and recreated the tables, verify the new
|
||||
run column:
|
||||
|
||||
```sql
|
||||
SELECT column_name, is_nullable, data_type
|
||||
FROM information_schema.columns
|
||||
WHERE table_name = 'outlook_runs'
|
||||
AND column_name = 'discussion_count';
|
||||
```
|
||||
|
||||
Verify the discussion table indexes:
|
||||
|
||||
```sql
|
||||
SELECT indexname
|
||||
FROM pg_indexes
|
||||
WHERE tablename = 'outlook_discussions'
|
||||
ORDER BY indexname;
|
||||
```
|
||||
|
||||
Verify that legacy polygon-level prose columns are gone from `outlooks`:
|
||||
|
||||
```sql
|
||||
SELECT column_name
|
||||
FROM information_schema.columns
|
||||
WHERE table_name = 'outlooks'
|
||||
AND column_name IN ('headline', 'summary', 'discussion');
|
||||
```
|
||||
|
||||
The final query should return zero rows.
|
||||
Reference in New Issue
Block a user