eric/weatherreporter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Added a staged roadmap to implement the small changes and refactors identified by the audit

Browse Source
This commit is contained in:
Eric Rakestraw
2026-06-16 09:51:22 -05:00
parent a27e870522
commit 0884eb0ce5
4 changed files with 481 additions and 1366 deletions
Download Patch File Download Diff File Expand all files Collapse all files

657
docs/roadmap/implementation.md
View File

@@ -1,657 +0,0 @@
# Data Package Export Implementation Roadmap
## Purpose
This roadmap defines the staged implementation plan for
`docs/roadmap/data-package-exports.md`. It is written for an LLM coding agent
that will implement the curated data-package export feature in order.
The goal is to separate rich module values used by templates from curated
prompt-facing values serialized into Scriptorium data packages. The change
should make YAML data packages smaller and clearer without reducing deterministic
template composability.
This is a planning document only. It may describe unimplemented behavior because
it lives under `docs/roadmap/`.
## Source Roadmap
Use `docs/roadmap/data-package-exports.md` as the feature roadmap and target
state authority. That document defines the user intent, module/export boundary,
initial cleanup targets, and non-goals.
If this implementation plan and the feature roadmap conflict, update the feature
roadmap first so it remains the conceptual source of truth, then update this
file.
## Locked Decisions
- Keep rich module values available for templates, module snapshots, inspection,
Recent Changes, and render contexts.
- Serialize curated prompt-facing module values into data packages.
- Store prompt-facing values on `module.Output` as runtime-only values.
- Do not persist prompt-facing values in module snapshot JSON.
- Keep `internal/promptinput` independent from `internal/briefing`.
- Add prompt export behavior to the module registry/definition path where module
output policy already lives.
- Default modules without custom exporters to pass-through prompt values.
- Bump the data-package schema version from `weatherreporter.data_package.v2` to
`weatherreporter.data_package.v3`.
- No compatibility aliases are required for removed prompt-facing fields.
- Do not add user-configurable field selection, field allowlists, reflection
filters, or prompt-specific field profiles in this implementation.
## Implementation Principles
- Preserve existing generated Markdown output.
- Preserve existing template render-context richness.
- Preserve existing module snapshot richness and JSON shape except for any
unavoidable schema-version-only change. The preferred approach is to omit
prompt values from module snapshot JSON entirely.
- Keep exporter code near the module that owns the shape.
- Use typed prompt-facing structs for modules that need reshaping.
- Keep simple modules on default pass-through behavior.
- Make exporter errors actionable and include module ID/stanza context.
- Keep YAML category ordering unchanged.
- Treat generated-template `today`, `tomorrow`, and `daily` reports as equal
consumers of rich template module values.
- Update implemented docs only after code behavior exists.
## Stage 1: Runtime Prompt Value Contract
### Goal
Add a runtime-only prompt value to module outputs while preserving rich module
snapshot persistence.
### Files To Inspect
- `internal/module/module.go`
- `internal/module/module_test.go`
- `internal/state/filesystem.go`
- `internal/state/filesystem_test.go`
- `internal/app/app.go`
- `internal/app/app_test.go`
### Implementation
- Add a runtime-only field to `module.Output`, for example:
```go
type Output struct {
ID ID `json:"id"`
StanzaName string `json:"stanzaName"`
Value any `json:"value"`
PromptValue any `json:"-"`
}
```
- Add a small helper in `internal/module`, for example:
```go
func (o Output) DataPackageValue() any
```
The helper should return `PromptValue` when non-nil and fall back to `Value`
otherwise. This fallback keeps tests and any manually built snapshots simple.
- Do not require `PromptValue` in `Snapshot.Validate`.
- Do not persist `PromptValue` to module snapshot JSON.
- Do not change `StanzaValue`; it should continue decoding rich `Value`.
### Acceptance Criteria
- Module snapshots still serialize rich module values under `value`.
- Module snapshots do not serialize `promptValue` or any equivalent field.
- Existing rich-module snapshot lookup and `StanzaValue` behavior remain
unchanged.
- Code has a single helper for choosing the data-package value from an output.
### Tests
- Add module tests proving:
- `DataPackageValue` uses `PromptValue` when set;
- `DataPackageValue` falls back to `Value`;
- marshaled snapshot JSON omits `PromptValue`;
- `StanzaValue` continues decoding rich `Value`.
Suggested focused command:
```sh
go test ./internal/module ./internal/state
```
### Prompt Size
Small enough for one implementation prompt.
## Stage 2: Module Registry Prompt Exporters
### Goal
Teach the module registry to attach prompt-facing values to outputs as modules
are built.
### Files To Inspect
- `internal/briefing/modules.go`
- `internal/briefing/modules_test.go`
- `internal/briefing/*_module.go`
- `internal/module/module.go`
### Implementation
- Add a prompt exporter type in `internal/briefing`, for example:
```go
type ModulePromptExporter func(value any) (any, error)
```
- Add `PromptExporter ModulePromptExporter` to `ModuleDefinition`.
- In `ModuleRegistry.BuildModule`, after builder output ID and stanza validation:
- if `PromptExporter` is nil, set `output.PromptValue = output.Value`;
- if `PromptExporter` is present, call it with `output.Value`;
- set `output.PromptValue` to the returned value;
- wrap exporter errors with module ID and stanza context.
- Add a typed exporter helper if it keeps module exporters concise, for example:
```go
func promptExporter[T any](fn func(T) (any, error)) ModulePromptExporter
```
The helper should convert `any` through JSON marshal/unmarshal or direct type
assertion only if it meaningfully reduces boilerplate without hiding errors.
- Do not make `internal/promptinput` import `internal/briefing`.
- Do not move module-specific export policy into `internal/promptinput`.
### Acceptance Criteria
- Every built module output has a non-nil data-package value.
- Modules without custom exporters pass through rich values.
- Custom exporter errors identify the module and stanza.
- Registry validation remains focused on definitions, duplicate IDs/stanzas,
supported reports, options, builders, and missing-data policy.
### Tests
- Add or update module registry tests for:
- default pass-through prompt values;
- custom exporter prompt values;
- exporter error wrapping;
- output ID/stanza validation still runs before or around export behavior;
- output `PromptValue` is not persisted in snapshot JSON.
Suggested focused command:
```sh
go test ./internal/briefing ./internal/module
```
### Prompt Size
Small enough for one implementation prompt.
## Stage 3: Promptinput Uses Exported Values And Schema V3
### Goal
Make data-package construction serialize prompt-facing values and bump the data
package schema version.
### Files To Inspect
- `internal/promptinput/package.go`
- `internal/promptinput/package_test.go`
- `internal/state/filesystem_test.go`
- `internal/app/app.go`
- `internal/app/app_test.go`
- `docs/roadmap/data-package-exports.md`
### Implementation
- Change `promptinput.SchemaVersion` to:
```go
const SchemaVersion = "weatherreporter.data_package.v3"
```
- Update `stanzasFromSnapshot` to use `output.DataPackageValue()` rather than
`output.Value`.
- Keep `BriefingStanzas` category grouping and ordering unchanged.
- Keep `LoadYAML` validation strict for the current schema version.
- Update tests and fixtures that assert `weatherreporter.data_package.v2`.
### Acceptance Criteria
- Saved data packages use schema version `weatherreporter.data_package.v3`.
- Data package stanzas use `PromptValue` when present.
- Data package stanzas fall back to rich `Value` when `PromptValue` is absent,
which keeps hand-built test snapshots and loaded rich snapshots usable.
- YAML category ordering remains unchanged.
- Module snapshots remain rich and unaffected by prompt export serialization.
### Tests
- Update promptinput tests for:
- schema version `v3`;
- prompt value preferred over rich value;
- fallback to rich value;
- deterministic categorized YAML output unchanged apart from stanza values and
schema version;
- unknown/uncategorized stanza behavior unchanged.
- Update state tests that load data packages.
Suggested focused command:
```sh
go test ./internal/promptinput ./internal/state
```
### Prompt Size
Small enough for one implementation prompt.
## Stage 4: Current Conditions And Hourly Forecast Exports
### Goal
Add custom prompt exports for the clearest noisy raw-data modules:
`current_conditions` and `hourly_forecast`.
### Files To Inspect
- `internal/briefing/current_conditions_module.go`
- `internal/briefing/hourly_forecast_module.go`
- `internal/briefing/base_modules_test.go`
- `internal/generatedtext/render_context.go`
- `internal/reporttemplate/templates/*.md.tmpl`
- `internal/app/app_test.go`
### Current Conditions Export Shape
The prompt-facing `current_conditions` export should keep:
- `condition_text`
- `is_day`
- `temperature_c`
- `temperature_f`
- `apparent_temperature_c`
- `apparent_temperature_f`
- `dewpoint_c`
- `dewpoint_f`
- `relative_humidity_percent`
- `wind_speed_kmh`
- `wind_speed_mph`
- `wind_direction`
It should remove:
- `condition_text_lower`
- `wind_direction_text`
The rich `CurrentConditionsModule` should keep those helper fields for
templates.
### Hourly Forecast Export Shape
The prompt-facing `hourly_forecast` export should keep module-level fields:
- `product`
- `issued_at`
- `updated_at`
- `source_location`
- `source_location_id`
- `periods`
Each prompt-facing hourly period should keep:
- `period_begins`
- `period_ends`
- `name`
- `is_day`
- `condition_code`
- `text_description`
- temperature fields
- dewpoint fields
- wind speed and gust fields
- `wind_direction`
- pressure fields
- visibility fields
- apparent temperature fields
- `cloud_cover_percent`
- `probability_of_precipitation_percent`
- precipitation amount fields
- snowfall depth fields
- `uv_index`
- `relative_humidity_percent`
Each prompt-facing hourly period should remove:
- `hour_label`
- `text_description_lower`
- `mention_precipitation`
The rich `HourlyForecastPeriod` should keep those helper fields for templates.
### Implementation
- Add prompt export structs near each module.
- Add exporter functions near each module.
- Register exporters in the default module definitions.
- Prefer straightforward field copying over reflection.
- Preserve `omitempty` behavior.
### Acceptance Criteria
- Saved data packages no longer include removed fields for current conditions or
hourly forecast.
- Rich module snapshots and render contexts still include template helper
fields.
- Existing hourly, today, tomorrow, and daily template output remains unchanged
wherever those templates consume current conditions or hourly forecast values.
### Tests
- Update module tests to prove prompt exports omit stale fields and keep
expected factual fields.
- Update template/render-context tests to prove helper fields remain available
to templates.
- Update app workflow tests to inspect saved YAML and reject:
- `condition_text_lower`;
- `wind_direction_text`;
- `hour_label`;
- `text_description_lower`;
- `mention_precipitation`.
Suggested focused command:
```sh
go test ./internal/briefing ./internal/generatedtext ./internal/reporttemplate ./internal/app
```
### Prompt Size
Medium. Suitable for one implementation prompt.
## Stage 5: Derived Daypart Summary Export
### Goal
Add a curated prompt export for `derived_daypart_summaries` while preserving the
rich daypart fields used by Daily, Today, and Tomorrow templates.
### Files To Inspect
- `internal/briefing/derived_daypart_summaries_module.go`
- `internal/briefing/derived_modules_test.go`
- `internal/generatedtext/render_context.go`
- `internal/reporttemplate/templates/daily.md.tmpl`
- `internal/reporttemplate/templates/today.md.tmpl`
- `internal/reporttemplate/templates/tomorrow.md.tmpl`
- `internal/reporttemplate/reporttemplate_test.go`
- `internal/app/app_test.go`
### Export Shape
For each daypart, the prompt-facing export should keep:
- `date`
- `display_name`
- `period_begins`
- `period_ends`
- `temp_range_f`
- `apparent_temp_range_f`
- `max_pop_percent`
- `max_pop_time`
- `mention_precipitation`
- `max_wind_gust_mph`
- `max_wind_gust_time`
- `dominant_condition`
- `temperature_trend`
- `temperature_start_phrase_f`
- `temperature_end_phrase_f`
- `temperature_peak_phrase_f`
- `temperature_steady_phrase_f`
- `notable_conditions`
- `snow`
- `ice`
- `fog`
- `heat`
- `cold`
- `wind`
- `relevant_alert_count`
The prompt-facing export should remove:
- `temperature_phrase_f`
- `dominant_condition_lower`
- `dominant_condition_display`
- `max_pop_time_label`
For `max_pop_time`, use the most readable existing time label. Prefer
`MaxPopTimeLabel` when present, falling back to `MaxPopTime`, while keeping the
prompt-facing field name `max_pop_time`.
Keep the temperature trend phrase fields even though only some are populated for
each trend. They are not duplicates when used according to the trend:
- rising/falling use start and end phrases;
- peaking uses peak phrase;
- steady uses steady phrase.
### Implementation
- Add prompt export structs near the daypart module.
- Add an exporter for the `map[string]DerivedDaypartSummaryModule` value.
- Preserve map keys and values for all emitted dayparts.
- Register the exporter in the default module definitions.
- Do not alter rich `DerivedDaypartSummaryModule` fields used by templates.
### Acceptance Criteria
- Saved data packages no longer include the removed daypart fields.
- The prompt-facing daypart export still gives the LLM enough information to
understand condition, temperature trend, precipitation, wind, notable
conditions, and alert relevance.
- Daily, Today, and Tomorrow template output remains unchanged.
- Render contexts still expose rich daypart helper fields.
### Tests
- Update daypart module tests for:
- prompt export field presence;
- removed field absence;
- rising, falling, peaking, and steady trend values;
- max PoP time using the friendly label under the stable `max_pop_time` key.
- Update template tests to prove rich helper fields still render Daily, Today,
and Tomorrow daypart wording.
- Update app data-package tests to reject stale daypart fields.
Suggested focused command:
```sh
go test ./internal/briefing ./internal/generatedtext ./internal/reporttemplate ./internal/app
```
### Prompt Size
Medium. Suitable for one implementation prompt.
## Stage 6: Pass-Through Review And Workflow Regression Tests
### Goal
Confirm all other modules export correctly through pass-through behavior and add
workflow-level coverage proving the new boundary.
### Files To Inspect
- `internal/briefing/*_module.go`
- `internal/briefing/modules.go`
- `internal/promptinput/package_test.go`
- `internal/app/app_test.go`
- `internal/generatedtext/render_context_test.go`
- `internal/reporttemplate/reporttemplate_test.go`
### Implementation
- Review remaining modules:
- `metadata`
- `narrative_forecast`
- `derived_daily_summary`
- `precip_timing`
- `outdoor_windows`
- `alert_digest`
- `spc_convective_outlooks`
- `spc_convective_discussion`
- `area_forecast_discussion`
- `weather_story`
- planning modules
- Keep pass-through behavior for modules that are already prompt-appropriate.
- Add custom exporters only if a module contains clear template-only helpers or
confusing duplicate fields.
- Do not broaden this stage into a general prompt-schema redesign.
- Add workflow assertions that:
- module snapshots contain rich values;
- render contexts contain rich values;
- data packages contain curated values;
- Scriptorium receives the curated data package path exactly as before.
- Include workflow coverage for generated-template `today`, `tomorrow`, and
`daily` reports when asserting daypart and render-context behavior.
### Acceptance Criteria
- Every default module either has a custom exporter or intentionally uses
pass-through.
- App workflow tests prove saved data packages omit the cleaned fields.
- App workflow tests prove rich helper fields remain available where templates
use them.
- App workflow or render-context tests include `daily` alongside `today` and
`tomorrow` for daypart helper coverage.
- Existing report generation behavior remains unchanged except for data-package
YAML content and schema version.
### Tests
Suggested focused command:
```sh
go test ./internal/briefing ./internal/promptinput ./internal/generatedtext ./internal/reporttemplate ./internal/app
```
### Prompt Size
Small to medium. Suitable for one implementation prompt.
## Stage 7: Documentation And Final Validation
### Goal
Update implemented documentation after the code exists and run full validation.
### Files To Inspect
- `docs/internal/module.md`
- `docs/internal/prompt-input.md`
- `docs/templates.md`
- `docs/operations.md`
- `docs/troubleshooting.md`
- `docs/roadmap/data-package-exports.md`
- tests that assert documentation examples or data-package snippets
### Implementation
- Update `docs/internal/module.md` to describe:
- rich module output;
- runtime prompt export values;
- pass-through exporters;
- module-owned prompt export policy.
- Update `docs/internal/prompt-input.md` to document:
- schema version `weatherreporter.data_package.v3`;
- curated module export behavior;
- category ordering unchanged;
- data packages are not full template render contexts.
- Update `docs/templates.md` to clarify that templates can use richer fields
than the data package exposes.
- Update any Daily template-variable documentation alongside Today and Tomorrow
references where the same module fields are discussed.
- Update any implemented docs containing stale examples of removed data-package
fields.
- Do not document deferred configurable field profiles or reflection filters as
implemented behavior.
### Acceptance Criteria
- Non-roadmap docs describe only implemented data-package export behavior.
- Docs do not imply data packages contain template-only helper fields.
- Examples and snippets use schema version `v3` when they show data packages.
### Validation Commands
```sh
go test ./internal/module ./internal/briefing ./internal/promptinput
go test ./internal/generatedtext ./internal/reporttemplate ./internal/app
go test ./...
go run ./cmd/weatherreporter --help
git diff --check
```
### Stale Field Greps
Run focused checks against docs and expected saved data-package fixtures/snippets:
```sh
rg -n "condition_text_lower|wind_direction_text|hour_label|text_description_lower|mention_precipitation|dominant_condition_lower|dominant_condition_display|max_pop_time_label|temperature_phrase_f" docs internal/*/*_test.go
```
Review matches manually. Some matches should remain in rich module structs,
template tests, and render-context tests; they should not remain as expected
data-package fields.
### Prompt Size
Medium. Suitable for one implementation prompt.
## Deferred Work
Do not include these in this implementation sequence:
- user-configurable data-package field selection;
- per-prompt custom field profiles;
- reflection-based generic include/exclude lists;
- automatic schema generation for data-package exports;
- changing collected facts or derived facts contracts;
- changing Scriptorium invocation behavior;
- changing generated Markdown templates beyond preserving their output;
- migrating old workspace data-package artifacts.
## Open Questions
No open questions block implementation.
The implementation plan intentionally locks in the recommended choices from the
feature roadmap: schema version bump to `weatherreporter.data_package.v3`, and
runtime-only prompt export values stored on `module.Output`.
## Global Validation Checklist
- `go test ./...` passes.
- `go run ./cmd/weatherreporter --help` is accurate.
- `git diff --check` passes.
- Saved data packages use schema version `weatherreporter.data_package.v3`.
- Saved data packages serialize prompt-facing module exports.
- Saved module snapshots serialize rich module values and do not persist
prompt-facing values.
- Render contexts continue to expose rich module values.
- Hourly, Today, Tomorrow, and Daily templates continue to render the same
Markdown output.
- YAML briefing category order remains unchanged.
- `current_conditions` data packages omit `condition_text_lower` and
`wind_direction_text`.
- `hourly_forecast.periods[]` data packages omit `hour_label`,
`text_description_lower`, and `mention_precipitation`.
- `derived_daypart_summaries` data packages omit `temperature_phrase_f`,
`dominant_condition_lower`, `dominant_condition_display`, and
`max_pop_time_label`.
- Non-roadmap docs describe only implemented behavior.