# Data Package Export Implementation Roadmap

## Purpose

This roadmap defines the staged implementation plan for
`docs/roadmap/data-package-exports.md`. It is written for an LLM coding agent
that will implement the curated data-package export feature in order.

The goal is to separate rich module values used by templates from curated
prompt-facing values serialized into Scriptorium data packages. The change
should make YAML data packages smaller and clearer without reducing deterministic
template composability.

This is a planning document only. It may describe unimplemented behavior because
it lives under `docs/roadmap/`.

## Source Roadmap

Use `docs/roadmap/data-package-exports.md` as the feature roadmap and target
state authority. That document defines the user intent, module/export boundary,
initial cleanup targets, and non-goals.

If this implementation plan and the feature roadmap conflict, update the feature
roadmap first so it remains the conceptual source of truth, then update this
file.

## Locked Decisions

- Keep rich module values available for templates, module snapshots, inspection,
  Recent Changes, and render contexts.
- Serialize curated prompt-facing module values into data packages.
- Store prompt-facing values on `module.Output` as runtime-only values.
- Do not persist prompt-facing values in module snapshot JSON.
- Keep `internal/promptinput` independent from `internal/briefing`.
- Add prompt export behavior to the module registry/definition path where module
  output policy already lives.
- Default modules without custom exporters to pass-through prompt values.
- Bump the data-package schema version from `weatherreporter.data_package.v2` to
  `weatherreporter.data_package.v3`.
- No compatibility aliases are required for removed prompt-facing fields.
- Do not add user-configurable field selection, field allowlists, reflection
  filters, or prompt-specific field profiles in this implementation.

## Implementation Principles

- Preserve existing generated Markdown output.
- Preserve existing template render-context richness.
- Preserve existing module snapshot richness and JSON shape except for any
  unavoidable schema-version-only change. The preferred approach is to omit
  prompt values from module snapshot JSON entirely.
- Keep exporter code near the module that owns the shape.
- Use typed prompt-facing structs for modules that need reshaping.
- Keep simple modules on default pass-through behavior.
- Make exporter errors actionable and include module ID/stanza context.
- Keep YAML category ordering unchanged.
- Treat generated-template `today`, `tomorrow`, and `daily` reports as equal
  consumers of rich template module values.
- Update implemented docs only after code behavior exists.

## Stage 1: Runtime Prompt Value Contract

### Goal

Add a runtime-only prompt value to module outputs while preserving rich module
snapshot persistence.

### Files To Inspect

- `internal/module/module.go`
- `internal/module/module_test.go`
- `internal/state/filesystem.go`
- `internal/state/filesystem_test.go`
- `internal/app/app.go`
- `internal/app/app_test.go`

### Implementation

- Add a runtime-only field to `module.Output`, for example:

```go
type Output struct {
    ID          ID     `json:"id"`
    StanzaName  string `json:"stanzaName"`
    Value       any    `json:"value"`
    PromptValue any    `json:"-"`
}
```

- Add a small helper in `internal/module`, for example:

```go
func (o Output) DataPackageValue() any
```

The helper should return `PromptValue` when non-nil and fall back to `Value`
otherwise. This fallback keeps tests and any manually built snapshots simple.

- Do not require `PromptValue` in `Snapshot.Validate`.
- Do not persist `PromptValue` to module snapshot JSON.
- Do not change `StanzaValue`; it should continue decoding rich `Value`.

### Acceptance Criteria

- Module snapshots still serialize rich module values under `value`.
- Module snapshots do not serialize `promptValue` or any equivalent field.
- Existing rich-module snapshot lookup and `StanzaValue` behavior remain
  unchanged.
- Code has a single helper for choosing the data-package value from an output.

### Tests

- Add module tests proving:
  - `DataPackageValue` uses `PromptValue` when set;
  - `DataPackageValue` falls back to `Value`;
  - marshaled snapshot JSON omits `PromptValue`;
  - `StanzaValue` continues decoding rich `Value`.

Suggested focused command:

```sh
go test ./internal/module ./internal/state
```

### Prompt Size

Small enough for one implementation prompt.

## Stage 2: Module Registry Prompt Exporters

### Goal

Teach the module registry to attach prompt-facing values to outputs as modules
are built.

### Files To Inspect

- `internal/briefing/modules.go`
- `internal/briefing/modules_test.go`
- `internal/briefing/*_module.go`
- `internal/module/module.go`

### Implementation

- Add a prompt exporter type in `internal/briefing`, for example:

```go
type ModulePromptExporter func(value any) (any, error)
```

- Add `PromptExporter ModulePromptExporter` to `ModuleDefinition`.
- In `ModuleRegistry.BuildModule`, after builder output ID and stanza validation:
  - if `PromptExporter` is nil, set `output.PromptValue = output.Value`;
  - if `PromptExporter` is present, call it with `output.Value`;
  - set `output.PromptValue` to the returned value;
  - wrap exporter errors with module ID and stanza context.
- Add a typed exporter helper if it keeps module exporters concise, for example:

```go
func promptExporter[T any](fn func(T) (any, error)) ModulePromptExporter
```

The helper should convert `any` through JSON marshal/unmarshal or direct type
assertion only if it meaningfully reduces boilerplate without hiding errors.

- Do not make `internal/promptinput` import `internal/briefing`.
- Do not move module-specific export policy into `internal/promptinput`.

### Acceptance Criteria

- Every built module output has a non-nil data-package value.
- Modules without custom exporters pass through rich values.
- Custom exporter errors identify the module and stanza.
- Registry validation remains focused on definitions, duplicate IDs/stanzas,
  supported reports, options, builders, and missing-data policy.

### Tests

- Add or update module registry tests for:
  - default pass-through prompt values;
  - custom exporter prompt values;
  - exporter error wrapping;
  - output ID/stanza validation still runs before or around export behavior;
  - output `PromptValue` is not persisted in snapshot JSON.

Suggested focused command:

```sh
go test ./internal/briefing ./internal/module
```

### Prompt Size

Small enough for one implementation prompt.

## Stage 3: Promptinput Uses Exported Values And Schema V3

### Goal

Make data-package construction serialize prompt-facing values and bump the data
package schema version.

### Files To Inspect

- `internal/promptinput/package.go`
- `internal/promptinput/package_test.go`
- `internal/state/filesystem_test.go`
- `internal/app/app.go`
- `internal/app/app_test.go`
- `docs/roadmap/data-package-exports.md`

### Implementation

- Change `promptinput.SchemaVersion` to:

```go
const SchemaVersion = "weatherreporter.data_package.v3"
```

- Update `stanzasFromSnapshot` to use `output.DataPackageValue()` rather than
  `output.Value`.
- Keep `BriefingStanzas` category grouping and ordering unchanged.
- Keep `LoadYAML` validation strict for the current schema version.
- Update tests and fixtures that assert `weatherreporter.data_package.v2`.

### Acceptance Criteria

- Saved data packages use schema version `weatherreporter.data_package.v3`.
- Data package stanzas use `PromptValue` when present.
- Data package stanzas fall back to rich `Value` when `PromptValue` is absent,
  which keeps hand-built test snapshots and loaded rich snapshots usable.
- YAML category ordering remains unchanged.
- Module snapshots remain rich and unaffected by prompt export serialization.

### Tests

- Update promptinput tests for:
  - schema version `v3`;
  - prompt value preferred over rich value;
  - fallback to rich value;
  - deterministic categorized YAML output unchanged apart from stanza values and
    schema version;
  - unknown/uncategorized stanza behavior unchanged.
- Update state tests that load data packages.

Suggested focused command:

```sh
go test ./internal/promptinput ./internal/state
```

### Prompt Size

Small enough for one implementation prompt.

## Stage 4: Current Conditions And Hourly Forecast Exports

### Goal

Add custom prompt exports for the clearest noisy raw-data modules:
`current_conditions` and `hourly_forecast`.

### Files To Inspect

- `internal/briefing/current_conditions_module.go`
- `internal/briefing/hourly_forecast_module.go`
- `internal/briefing/base_modules_test.go`
- `internal/generatedtext/render_context.go`
- `internal/reporttemplate/templates/*.md.tmpl`
- `internal/app/app_test.go`

### Current Conditions Export Shape

The prompt-facing `current_conditions` export should keep:

- `condition_text`
- `is_day`
- `temperature_c`
- `temperature_f`
- `apparent_temperature_c`
- `apparent_temperature_f`
- `dewpoint_c`
- `dewpoint_f`
- `relative_humidity_percent`
- `wind_speed_kmh`
- `wind_speed_mph`
- `wind_direction`

It should remove:

- `condition_text_lower`
- `wind_direction_text`

The rich `CurrentConditionsModule` should keep those helper fields for
templates.

### Hourly Forecast Export Shape

The prompt-facing `hourly_forecast` export should keep module-level fields:

- `product`
- `issued_at`
- `updated_at`
- `source_location`
- `source_location_id`
- `periods`

Each prompt-facing hourly period should keep:

- `period_begins`
- `period_ends`
- `name`
- `is_day`
- `condition_code`
- `text_description`
- temperature fields
- dewpoint fields
- wind speed and gust fields
- `wind_direction`
- pressure fields
- visibility fields
- apparent temperature fields
- `cloud_cover_percent`
- `probability_of_precipitation_percent`
- precipitation amount fields
- snowfall depth fields
- `uv_index`
- `relative_humidity_percent`

Each prompt-facing hourly period should remove:

- `hour_label`
- `text_description_lower`
- `mention_precipitation`

The rich `HourlyForecastPeriod` should keep those helper fields for templates.

### Implementation

- Add prompt export structs near each module.
- Add exporter functions near each module.
- Register exporters in the default module definitions.
- Prefer straightforward field copying over reflection.
- Preserve `omitempty` behavior.

### Acceptance Criteria

- Saved data packages no longer include removed fields for current conditions or
  hourly forecast.
- Rich module snapshots and render contexts still include template helper
  fields.
- Existing hourly, today, tomorrow, and daily template output remains unchanged
  wherever those templates consume current conditions or hourly forecast values.

### Tests

- Update module tests to prove prompt exports omit stale fields and keep
  expected factual fields.
- Update template/render-context tests to prove helper fields remain available
  to templates.
- Update app workflow tests to inspect saved YAML and reject:
  - `condition_text_lower`;
  - `wind_direction_text`;
  - `hour_label`;
  - `text_description_lower`;
  - `mention_precipitation`.

Suggested focused command:

```sh
go test ./internal/briefing ./internal/generatedtext ./internal/reporttemplate ./internal/app
```

### Prompt Size

Medium. Suitable for one implementation prompt.

## Stage 5: Derived Daypart Summary Export

### Goal

Add a curated prompt export for `derived_daypart_summaries` while preserving the
rich daypart fields used by Daily, Today, and Tomorrow templates.

### Files To Inspect

- `internal/briefing/derived_daypart_summaries_module.go`
- `internal/briefing/derived_modules_test.go`
- `internal/generatedtext/render_context.go`
- `internal/reporttemplate/templates/daily.md.tmpl`
- `internal/reporttemplate/templates/today.md.tmpl`
- `internal/reporttemplate/templates/tomorrow.md.tmpl`
- `internal/reporttemplate/reporttemplate_test.go`
- `internal/app/app_test.go`

### Export Shape

For each daypart, the prompt-facing export should keep:

- `date`
- `display_name`
- `period_begins`
- `period_ends`
- `temp_range_f`
- `apparent_temp_range_f`
- `max_pop_percent`
- `max_pop_time`
- `mention_precipitation`
- `max_wind_gust_mph`
- `max_wind_gust_time`
- `dominant_condition`
- `temperature_trend`
- `temperature_start_phrase_f`
- `temperature_end_phrase_f`
- `temperature_peak_phrase_f`
- `temperature_steady_phrase_f`
- `notable_conditions`
- `snow`
- `ice`
- `fog`
- `heat`
- `cold`
- `wind`
- `relevant_alert_count`

The prompt-facing export should remove:

- `temperature_phrase_f`
- `dominant_condition_lower`
- `dominant_condition_display`
- `max_pop_time_label`

For `max_pop_time`, use the most readable existing time label. Prefer
`MaxPopTimeLabel` when present, falling back to `MaxPopTime`, while keeping the
prompt-facing field name `max_pop_time`.

Keep the temperature trend phrase fields even though only some are populated for
each trend. They are not duplicates when used according to the trend:

- rising/falling use start and end phrases;
- peaking uses peak phrase;
- steady uses steady phrase.

### Implementation

- Add prompt export structs near the daypart module.
- Add an exporter for the `map[string]DerivedDaypartSummaryModule` value.
- Preserve map keys and values for all emitted dayparts.
- Register the exporter in the default module definitions.
- Do not alter rich `DerivedDaypartSummaryModule` fields used by templates.

### Acceptance Criteria

- Saved data packages no longer include the removed daypart fields.
- The prompt-facing daypart export still gives the LLM enough information to
  understand condition, temperature trend, precipitation, wind, notable
  conditions, and alert relevance.
- Daily, Today, and Tomorrow template output remains unchanged.
- Render contexts still expose rich daypart helper fields.

### Tests

- Update daypart module tests for:
  - prompt export field presence;
  - removed field absence;
  - rising, falling, peaking, and steady trend values;
  - max PoP time using the friendly label under the stable `max_pop_time` key.
- Update template tests to prove rich helper fields still render Daily, Today,
  and Tomorrow daypart wording.
- Update app data-package tests to reject stale daypart fields.

Suggested focused command:

```sh
go test ./internal/briefing ./internal/generatedtext ./internal/reporttemplate ./internal/app
```

### Prompt Size

Medium. Suitable for one implementation prompt.

## Stage 6: Pass-Through Review And Workflow Regression Tests

### Goal

Confirm all other modules export correctly through pass-through behavior and add
workflow-level coverage proving the new boundary.

### Files To Inspect

- `internal/briefing/*_module.go`
- `internal/briefing/modules.go`
- `internal/promptinput/package_test.go`
- `internal/app/app_test.go`
- `internal/generatedtext/render_context_test.go`
- `internal/reporttemplate/reporttemplate_test.go`

### Implementation

- Review remaining modules:
  - `metadata`
  - `narrative_forecast`
  - `derived_daily_summary`
  - `precip_timing`
  - `outdoor_windows`
  - `alert_digest`
  - `spc_convective_outlooks`
  - `spc_convective_discussion`
  - `area_forecast_discussion`
  - `weather_story`
  - planning modules
- Keep pass-through behavior for modules that are already prompt-appropriate.
- Add custom exporters only if a module contains clear template-only helpers or
  confusing duplicate fields.
- Do not broaden this stage into a general prompt-schema redesign.
- Add workflow assertions that:
  - module snapshots contain rich values;
  - render contexts contain rich values;
  - data packages contain curated values;
  - Scriptorium receives the curated data package path exactly as before.
- Include workflow coverage for generated-template `today`, `tomorrow`, and
  `daily` reports when asserting daypart and render-context behavior.

### Acceptance Criteria

- Every default module either has a custom exporter or intentionally uses
  pass-through.
- App workflow tests prove saved data packages omit the cleaned fields.
- App workflow tests prove rich helper fields remain available where templates
  use them.
- App workflow or render-context tests include `daily` alongside `today` and
  `tomorrow` for daypart helper coverage.
- Existing report generation behavior remains unchanged except for data-package
  YAML content and schema version.

### Tests

Suggested focused command:

```sh
go test ./internal/briefing ./internal/promptinput ./internal/generatedtext ./internal/reporttemplate ./internal/app
```

### Prompt Size

Small to medium. Suitable for one implementation prompt.

## Stage 7: Documentation And Final Validation

### Goal

Update implemented documentation after the code exists and run full validation.

### Files To Inspect

- `docs/internal/module.md`
- `docs/internal/prompt-input.md`
- `docs/templates.md`
- `docs/operations.md`
- `docs/troubleshooting.md`
- `docs/roadmap/data-package-exports.md`
- tests that assert documentation examples or data-package snippets

### Implementation

- Update `docs/internal/module.md` to describe:
  - rich module output;
  - runtime prompt export values;
  - pass-through exporters;
  - module-owned prompt export policy.
- Update `docs/internal/prompt-input.md` to document:
  - schema version `weatherreporter.data_package.v3`;
  - curated module export behavior;
  - category ordering unchanged;
  - data packages are not full template render contexts.
- Update `docs/templates.md` to clarify that templates can use richer fields
  than the data package exposes.
- Update any Daily template-variable documentation alongside Today and Tomorrow
  references where the same module fields are discussed.
- Update any implemented docs containing stale examples of removed data-package
  fields.
- Do not document deferred configurable field profiles or reflection filters as
  implemented behavior.

### Acceptance Criteria

- Non-roadmap docs describe only implemented data-package export behavior.
- Docs do not imply data packages contain template-only helper fields.
- Examples and snippets use schema version `v3` when they show data packages.

### Validation Commands

```sh
go test ./internal/module ./internal/briefing ./internal/promptinput
go test ./internal/generatedtext ./internal/reporttemplate ./internal/app
go test ./...
go run ./cmd/weatherreporter --help
git diff --check
```

### Stale Field Greps

Run focused checks against docs and expected saved data-package fixtures/snippets:

```sh
rg -n "condition_text_lower|wind_direction_text|hour_label|text_description_lower|mention_precipitation|dominant_condition_lower|dominant_condition_display|max_pop_time_label|temperature_phrase_f" docs internal/*/*_test.go
```

Review matches manually. Some matches should remain in rich module structs,
template tests, and render-context tests; they should not remain as expected
data-package fields.

### Prompt Size

Medium. Suitable for one implementation prompt.

## Deferred Work

Do not include these in this implementation sequence:

- user-configurable data-package field selection;
- per-prompt custom field profiles;
- reflection-based generic include/exclude lists;
- automatic schema generation for data-package exports;
- changing collected facts or derived facts contracts;
- changing Scriptorium invocation behavior;
- changing generated Markdown templates beyond preserving their output;
- migrating old workspace data-package artifacts.

## Open Questions

No open questions block implementation.

The implementation plan intentionally locks in the recommended choices from the
feature roadmap: schema version bump to `weatherreporter.data_package.v3`, and
runtime-only prompt export values stored on `module.Output`.

## Global Validation Checklist

- `go test ./...` passes.
- `go run ./cmd/weatherreporter --help` is accurate.
- `git diff --check` passes.
- Saved data packages use schema version `weatherreporter.data_package.v3`.
- Saved data packages serialize prompt-facing module exports.
- Saved module snapshots serialize rich module values and do not persist
  prompt-facing values.
- Render contexts continue to expose rich module values.
- Hourly, Today, Tomorrow, and Daily templates continue to render the same
  Markdown output.
- YAML briefing category order remains unchanged.
- `current_conditions` data packages omit `condition_text_lower` and
  `wind_direction_text`.
- `hourly_forecast.periods[]` data packages omit `hour_label`,
  `text_description_lower`, and `mention_precipitation`.
- `derived_daypart_summaries` data packages omit `temperature_phrase_f`,
  `dominant_condition_lower`, `dominant_condition_display`, and
  `max_pop_time_label`.
- Non-roadmap docs describe only implemented behavior.