weatherreporter/docs/roadmap/data-package-exports.md

Data Package Export Roadmap

Purpose

This roadmap defines the target state for cleaning up module fields exposed in YAML data packages. The goal is to keep report templates composable while making LLM prompt inputs concise, readable, and free of template-only helper fields.

This feature is implemented. Current data-package behavior is documented in docs/internal/prompt-input.md; the rich module and template boundary is documented in docs/internal/module.md and docs/templates.md.

Problem

Module output structs currently serve two different consumers:

• deterministic report templates, which benefit from presentation helpers such as lower-case text, display labels, trend phrases, and hour labels;
• Scriptorium data packages, which should expose the clearest useful weather facts to the LLM with minimal redundancy.

Those consumers now need different surfaces. Examples include:

• current_conditions exposes both condition_text and condition_text_lower, plus both abbreviated and long-form wind-direction fields.
• hourly_forecast.periods[] exposes both period_begins and hour_label, and both text_description and text_description_lower.
• derived_daypart_summaries exposes numerous temperature and condition phrase fields that are useful for deterministic template wording but noisy in the prompt data package.

The cleanup should not weaken template composability. Templates should still be able to use rich module values and helper fields.

The cleanup applies to every report that consumes these modules, including the generated-template today, tomorrow, and daily reports. The same derived_daypart_summaries prompt export should serve all three reports while their templates continue to use rich daypart helper fields.

Intent

Data packages should be curated prompt inputs, not a raw dump of every field available to Go templates.

The intended architecture is:

• module builders produce rich internal/template module values;
• each module may define a prompt-facing export value for data-package use;
• prompt input construction serializes the prompt-facing export value;
• template rendering continues to use the full rich module value.

The result should let weatherreporter optimize separately for:

• precise deterministic Markdown rendering;
• compact, readable LLM input;
• stable internal module contracts.

Locked Decisions

• Do not make the existing module structs smaller solely to clean up data packages.
• Do not use per-module string field allowlists as the primary mechanism.
• Do not rely on reflection-heavy field filtering for nested module shapes.
• Do not use json:"-" or yaml:"-" on rich template fields as the main boundary.
• Keep rich module outputs available for template rendering, inspection, tests, and internal use.
• Add an explicit prompt/data-package export layer for module outputs.
• Simple modules may use default pass-through export behavior.
• No compatibility aliases are needed for removed prompt-facing fields because the prompt schema is still pre-release.
• Bump the data-package schema version when implementing this change.
• Compute prompt export values during module snapshot construction and store the runtime-only prompt value on module.Output alongside the rich Value.
• Do not persist prompt export values in module snapshot JSON; module snapshots should continue to preserve rich module values.

Target Architecture

Each module definition should be able to declare how its output is represented in prompt data packages.

A possible shape is:

type ModuleDefinition struct {
    // existing fields...
    PromptExporter ModulePromptExporter
}

type ModulePromptExporter func(value any) (any, error)

The exact API may differ if implementation discovers a cleaner fit, but the contract should preserve these properties:

• the exporter is owned near the module definition or module builder;
• the exporter receives the rich module value and returns a prompt-facing value;
• missing exporters default to pass-through for modules whose rich value is already prompt-appropriate;
• exporter errors include module ID and stanza context;
• promptinput uses exported prompt values instead of rich values;
• render contexts and templates continue using rich values.

The preferred implementation should avoid making internal/promptinput import internal/briefing directly. If prompt export needs registry knowledge, either:

• record the prompt-facing value in module.Output when the module snapshot is built; or
• pass an explicit export map/registry into prompt-input construction without creating a package cycle.

The implementation should keep package boundaries consistent with existing architecture: module output policy belongs with module definitions, and data package serialization belongs in internal/promptinput.

Prompt Export Contract

A module prompt export should be:

• curated: include fields useful to the LLM, omit fields used only for deterministic sentence construction;
• typed: use small prompt-facing structs for modules that need reshaping;
• stable: keep field names intentional and avoid duplicating equivalent facts under multiple names;
• readable: prefer fields that explain themselves in YAML;
• loss-aware: do not omit facts that the LLM needs to reason about timing, severity, uncertainty, or practical impact;
• module-owned: keep each module responsible for its own prompt-facing contract.

Prompt-facing structs may live next to the module that owns them, for example:

type CurrentConditionsPromptExport struct {
    ConditionText           string `json:"condition_text,omitempty"`
    TemperatureF            *int   `json:"temperature_f,omitempty"`
    ApparentTemperatureF    *int   `json:"apparent_temperature_f,omitempty"`
    RelativeHumidityPercent *int   `json:"relative_humidity_percent,omitempty"`
    WindSpeedMph            *int   `json:"wind_speed_mph,omitempty"`
    WindDirection           string `json:"wind_direction,omitempty"`
}

The names do not need to include PromptExport if implementation finds a clearer convention, but they should distinguish data-package shape from template-rendering shape.

Initial Cleanup Targets

Current Conditions

Keep prompt-facing fields that express current observed conditions directly:

• condition_text
• is_day
• temperature fields
• apparent temperature fields
• dewpoint fields
• relative humidity
• wind speed
• one wind direction field

Remove prompt-facing fields that are template-only duplicates:

• condition_text_lower
• duplicate wind-direction text when an equivalent wind_direction field is present

The template surface may keep those helper fields.

Hourly Forecast

Keep prompt-facing period fields that carry facts:

• period_begins
• period_ends
• name
• is_day
• condition code, if useful
• text_description
• temperature fields
• dewpoint, apparent temperature, humidity, wind, gust, pressure, visibility, cloud cover, precipitation probability, precipitation amount, snowfall depth, and UV index when provided by upstream data

Remove prompt-facing fields that duplicate or encode template logic:

• hour_label, because period_begins already gives the time in a friendly local label;
• text_description_lower, because the LLM can interpret text_description;
• mention_precipitation, because it is a template threshold helper when the underlying precipitation probability is present.

The template surface may keep these helper fields.

Derived Daypart Summaries

Keep prompt-facing fields that describe the daypart:

• date
• display_name
• period_begins
• period_ends
• temperature range or the best single temperature phrase
• apparent temperature range when useful
• maximum precipitation probability and time
• maximum wind gust and time
• dominant condition
• temperature trend
• notable conditions
• weather indicator booleans
• relevant alert count

Remove prompt-facing fields that mainly support deterministic sentence construction:

• duplicate lower-case/display variants of the same dominant condition;
• multiple temperature phrase fragments when a smaller set can express the same trend;
• duplicate time labels where one friendly time field is enough.

The exact retained daypart temperature fields should be chosen during implementation with template needs and LLM readability in mind. The prompt export should preserve the facts needed to understand whether temperatures are rising, falling, peaking, or steady, but it does not need every phrase fragment used by the Markdown template.

Other Modules

Most existing modules may initially use pass-through export unless they expose clear template-only helpers. During implementation, review at least:

• narrative_forecast
• precip_timing
• outdoor_windows
• alert_digest
• spc_convective_outlooks
• spc_convective_discussion
• area_forecast_discussion
• weather_story
• planning modules

Do not remove fields merely because they are verbose. Remove or reshape fields when they are redundant, template-specific, or confusing in the context of LLM input.

Data Package Behavior

After implementation:

• saved YAML data packages should use prompt-facing module exports;
• saved module snapshots should continue preserving rich module output values;
• generated-text render contexts should continue preserving rich module values;
• Recent Changes should continue using structured module snapshots unless a specific comparison should intentionally move to prompt-facing fields;
• inspection commands should make clear whether they are showing rich module snapshots or prompt data packages.
• generated-template reports, including today, tomorrow, and daily, should continue rendering from rich module values.

This roadmap does not require changing source warnings, report metadata, collected facts, derived facts, or generated report artifacts.

Schema And Versioning

This is a prompt-input schema cleanup. Because the project is pre-release, the implementation may make a clean break in data-package field names without compatibility aliases.

The data-package schema version should be bumped when this feature is implemented because persisted data-package fields will be removed or renamed. This makes artifact shape changes explicit and helps inspection tooling distinguish old and new data packages.

Documentation Guidance

After implementation, update implemented documentation only:

• docs/internal/module.md: describe the distinction between rich module output and prompt-facing export values.
• docs/internal/prompt-input.md: document that data packages use curated prompt exports, not full template module structs.
• docs/templates.md: clarify that templates may have richer fields than the data package.
• Any module field examples in implemented docs should match the new prompt-facing data package shape.

Do not document future module fields or unimplemented exporters outside docs/roadmap/.

Acceptance Criteria

The feature is complete when:

• prompt data packages serialize curated module exports instead of blindly serializing rich module values;
• templates still render from rich module values without losing helper fields;
• current_conditions no longer exposes lower-case condition text or duplicate wind-direction fields in data packages;
• hourly_forecast.periods[] no longer exposes hour_label, text_description_lower, or mention_precipitation in data packages;
• derived_daypart_summaries no longer exposes redundant condition and temperature phrase variants in data packages;
• today, tomorrow, and daily rendered reports continue to have access to rich daypart helper fields for deterministic template wording;
• simple modules that do not need cleanup still export correctly through default pass-through behavior;
• exporter errors include module/stanza context;
• YAML category ordering remains unchanged;
• module snapshot artifacts remain rich enough for templates, inspection, and regression diagnosis;
• tests prove that removed prompt-facing fields are absent from saved YAML data packages and still available to templates where needed.

Testing Expectations

Implementation should add or update focused tests for:

• module registry validation for prompt exporters, if exporters are registered there;
• promptinput construction using exported prompt values;
• pass-through behavior for simple modules;
• custom exports for current conditions, hourly forecast, and daypart summaries;
• data-package YAML output rejecting stale fields;
• template render tests proving template-only helper fields remain available for today, tomorrow, and daily;
• app workflow tests proving saved data packages use curated exports while render contexts keep rich values.

Suggested validation after implementation:

go test ./internal/module ./internal/briefing ./internal/promptinput
go test ./internal/generatedtext ./internal/reporttemplate ./internal/app
go test ./...
go run ./cmd/weatherreporter --help
git diff --check

Deferred Work

Do not include these in the initial cleanup unless implementation reveals they are necessary:

• user-configurable data-package field selection;
• per-prompt custom field profiles;
• reflection-based generic include/exclude lists;
• automatic schema generation for data-package exports;
• changing collected facts or derived facts contracts;
• changing Scriptorium invocation behavior;
• changing generated Markdown templates beyond preserving their current output.

Open Questions

No open questions block implementation.

The implementation plan in docs/roadmap/implementation.md is the sequencing authority for this feature.