Add roadmap and implementation plan to move towards hybrid deterministic/llm generation of reports

docs/roadmap/generated-text.md

@@ -0,0 +1,343 @@
+# GeneratedText Report Rendering Roadmap
+
+## Purpose
+
+This roadmap defines planned work to move selected reports from full Markdown
+LLM generation to deterministic Markdown rendering with structured LLM-filled
+text slots. The first target report is the rolling next-hours report, now named
+`hourly` rather than `near_term`.
+
+This feature is not implemented yet, so this document lives under
+`docs/roadmap/`.
+
+## Intent
+
+The current module and prompt-input system has reached the point where many
+report facts can be calculated deterministically. The next step is to let Go
+own report structure while using the LLM for only the prose that benefits from
+language synthesis.
+
+The intended workflow is:
+
+1. collect upstream facts once;
+2. derive deterministic report facts;
+3. build module outputs and the prompt data package;
+4. ask Scriptorium for structured JSON using the schema associated with the
+   prompt ID on the Scriptorium backend;
+5. validate and persist that structured response as `GeneratedText`;
+6. build a curated render context;
+7. render final Markdown from an embedded human-editable template.
+
+This should improve consistency for headings, dates, ordering, units, recurring
+sections, and report-specific format while preserving the LLM's role in
+summarizing and explaining weather impacts.
+
+## Terminology
+
+- `CollectedFacts`: normalized upstream inputs from Weather API sources.
+- `DerivedFacts`: deterministic Go-calculated facts sliced to a report period.
+- `ModuleOutput`: prompt-facing deterministic module stanzas.
+- `GeneratedText`: validated structured LLM output. It is not authoritative
+  weather data; it is prose or short text intended to fill named template
+  slots.
+- `RenderContext`: the final deterministic object passed to a Markdown
+  template. It may expose selected values from CollectedFacts, DerivedFacts,
+  ModuleOutput, report metadata, and GeneratedText.
+
+The report template should not receive the unbounded raw data package directly.
+It should receive a curated render context so templates remain readable and do
+not become a second application logic layer.
+
+## Locked Decisions
+
+- Use `GeneratedText` as the name for the structured LLM output layer.
+- Continue using Scriptorium behind `internal/adapters/scriptorium`.
+- Use Scriptorium structured JSON output. The Scriptorium backend associates
+  the prompt ID with the JSON schema; weatherreporter does not pass a schema
+  file at runtime.
+- Keep Scriptorium retries inside Scriptorium.
+- Also validate the returned JSON inside `weatherreporter` before rendering.
+- Use embedded, separate, human-editable Markdown template files.
+- Use embedded, separate, human-readable JSON schema files.
+- Use Go standard library `text/template` for Markdown rendering unless a
+  concrete limitation appears.
+- Let Scriptorium enforce the prompt-associated JSON Schema and perform
+  retries, then have weatherreporter validate by unmarshalling into a typed
+  GeneratedText struct and checking required fields. Do not add a Go JSON
+  Schema dependency in the first implementation.
+- Do not generate schemas or templates from inline Go.
+- Start with the rolling next-hours report only.
+- Rename the planned `near_term` report to `hourly`.
+- Do not keep both `near_term` and `hourly` public report IDs.
+- Do not add a generic workflow engine or plugin system.
+
+## Hourly Report Target
+
+The planned `near_term` report should become the `hourly` report.
+
+Target report identity:
+
+- report ID: `hourly`
+- CLI command: `weatherreporter generate hourly`
+- display name: `Hourly Report`
+- prompt ID for structured text generation: `weather.hourly_generated_text`
+- Markdown template ID: `hourly`
+- GeneratedText schema ID: `hourly`
+- artifact group: `hourly`
+- batch output name: `hourly.md`
+- default valid-period length: 6 hours
+- valid period: `[generation_time, generation_time + hourlyReportHours)`
+
+Use a package-owned constant for the valid-period length, initially 6 hours.
+The duration should not be user-configurable in the first implementation.
+
+The report remains explicit-generation only at first. Do not add it to morning
+or evening batches until cadence and downstream behavior are proven.
+
+## Embedded Asset Layout
+
+Recommended asset package:
+
+```text
+internal/reporttemplate/
+  templates/
+    hourly.md.tmpl
+  schemas/
+    hourly.generated_text.schema.json
+```
+
+`internal/reporttemplate` should embed assets with `go:embed` and expose a
+narrow API for:
+
+- looking up a template by template ID;
+- looking up a schema by schema ID;
+- rendering Markdown from a typed render context.
+
+Do not allow template or schema asset IDs to come from free-form user input in
+the initial implementation. Report definitions should declare the IDs.
+
+## Report Definition Fields
+
+Extend report definitions only as needed for this generation mode. Required
+fields:
+
+- `GenerationMode`
+- `TemplateID`
+- `GeneratedTextSchemaID`
+
+Recommended generation modes:
+
+- `GenerationModeScriptoriumMarkdown`: existing full Markdown generation path.
+- `GenerationModeGeneratedTextTemplate`: structured GeneratedText plus
+  deterministic template rendering.
+
+Existing reports should stay on the current full Markdown path until migrated.
+The `hourly` report should use the GeneratedText/template path from the start.
+For generated-text reports, the existing `PromptID` field remains the
+Scriptorium prompt ID and should be set to `weather.hourly_generated_text`.
+
+## GeneratedText Contract
+
+The hourly JSON schema should define only the text slots the template needs.
+Keep the first schema small. Example shape:
+
+```json
+{
+  "summary": "Brief near-term overview.",
+  "timing": "Plain-language timing of notable changes.",
+  "impacts": "Practical impacts for the next few hours.",
+  "confidence": "Optional confidence or uncertainty note."
+}
+```
+
+The exact schema should be locked during implementation based on the first
+hourly template. Required fields should be genuinely required by the template.
+Optional fields should be omitted from the rendered Markdown when empty.
+
+GeneratedText should be persisted as structured JSON exactly as validated by
+weatherreporter. Raw Scriptorium output should also be persisted for debugging
+when practical.
+
+## Template Contract
+
+The hourly Markdown template should be a readable file that a human can edit
+without digging into Go code. It should own:
+
+- Markdown heading structure;
+- section order;
+- deterministic fact placement;
+- conditional inclusion of optional GeneratedText slots;
+- final report layout.
+
+The template should not own:
+
+- Weather API fetching;
+- fact derivation;
+- module composition;
+- Scriptorium invocation;
+- schema validation;
+- path construction;
+- distributor upload behavior.
+
+The render context should prepare strings and simple values so template logic
+stays shallow. Prefer small conditionals over complex template functions.
+
+## Scriptorium Adapter Direction
+
+`internal/adapters/scriptorium` should expose weatherreporter-owned request and
+result types for structured JSON generation. The adapter should hide
+Scriptorium argv details from app, report, facts, module, prompt-input, and
+template packages.
+
+Scriptorium uses the prompt ID to select the backend-associated structured
+output schema. The adapter should not pass a schema path at runtime.
+
+## State And Artifacts
+
+Add managed artifacts for the new generation path. Recommended logical
+artifacts:
+
+- module snapshot JSON;
+- data package YAML;
+- raw GeneratedText JSON written by Scriptorium;
+- structured Scriptorium command result for the GeneratedText run;
+- validated GeneratedText JSON;
+- render context JSON;
+- rendered Markdown report;
+- metadata linking all of the above.
+
+The existing `preflight` artifact should remain as the rendered prompt/input
+inspection artifact before structured generation. GeneratedText reports add
+separate artifacts for raw GeneratedText, structured run result, validated
+GeneratedText, and render context. Metadata should record the
+`GeneratedTextSchemaID`; no per-run schema artifact is persisted.
+
+Artifact paths should remain under the workspace and use the report artifact
+group, valid start date, and RunID conventions already used by state.
+
+## End-To-End Behavior
+
+Reports using `GenerationModeGeneratedTextTemplate` should follow the same
+fact collection, derivation, module snapshot, prompt package, state, and
+notification boundaries as current reports. The difference is limited to the
+last rendering segment: Scriptorium returns structured GeneratedText, Go
+validates that structure, Go builds a curated render context, and an embedded
+template renders the final Markdown.
+
+The existing full-Markdown Scriptorium path should remain available for
+reports that have not migrated.
+
+## Target Architecture
+
+GeneratedText rendering adds one new generation mode, not a replacement for the
+whole report system. Reports that opt into this mode should continue to use
+the existing report registry, fact pipeline, module snapshot, data package,
+state store, and notification boundaries.
+
+The target flow is:
+
+1. `internal/report` declares whether a report uses the existing full-Markdown
+   Scriptorium path or the GeneratedText/template path.
+2. The app builds CollectedFacts, DerivedFacts, module snapshots, and the
+   prompt data package using the same boundaries as existing reports.
+3. `internal/adapters/scriptorium` asks Scriptorium for structured JSON using
+   the report's prompt ID. The Scriptorium backend owns the prompt-to-schema
+   association.
+4. Weatherreporter validates the returned JSON as typed `GeneratedText`.
+5. A report-specific render context is built from selected deterministic facts,
+   module outputs, metadata, and GeneratedText.
+6. `internal/reporttemplate` renders the final Markdown from an embedded,
+   human-editable template.
+7. The managed Markdown report remains the canonical output and the distributor
+   upload source.
+
+This structure keeps each layer understandable:
+
+- CollectedFacts and DerivedFacts remain weather-data contracts.
+- ModuleOutput remains the prompt-facing deterministic facts contract.
+- GeneratedText remains LLM-written prose slots, not weather truth.
+- RenderContext remains a curated template input, not a second raw data package.
+- Templates own presentation order and headings, not weather derivation logic.
+
+## Editing Surface
+
+The desired human editing surface is file-based and report-specific. For the
+hourly report, the important files should be easy to find and inspect:
+
+- one Markdown template file for the report layout;
+- one JSON schema file describing the expected GeneratedText payload;
+- one report definition declaring the prompt ID, template ID, schema ID, module
+  composition, valid period, artifact group, and output naming policy.
+
+The template and schema should be embedded into the binary, but they should
+remain ordinary files in the repository. A maintainer should be able to tweak
+headings, reorder sections, or inspect the structured prose contract without
+reading unrelated orchestration code.
+
+## Boundary Guidelines
+
+- Keep report identity, prompt IDs, generation mode, template IDs, schema IDs,
+  valid periods, module composition, and output naming in `internal/report`.
+- Keep Scriptorium subprocess details in `internal/adapters/scriptorium`.
+- Keep Markdown template lookup and rendering in `internal/reporttemplate`.
+- Keep GeneratedText validation in a narrow package or component with typed
+  report-specific structs.
+- Keep app orchestration responsible for sequencing and artifact persistence,
+  not for report-specific prose or template structure.
+- Keep distributor upload behavior unchanged: upload the managed Markdown
+  report after it exists.
+- Do not let templates fetch data, derive facts, perform source selection, or
+  construct workspace paths.
+- Do not let Scriptorium-specific flags, schema paths, or subprocess details
+  leak into report definitions or domain packages.
+- Do not add user-provided template or schema paths in the first
+  implementation.
+
+## Target Acceptance Criteria
+
+The feature is successful when:
+
+- `weatherreporter generate hourly` produces a deterministic Markdown report
+  whose headings, dates, and section order come from an embedded template;
+- Scriptorium fills only the report's structured GeneratedText slots;
+- weatherreporter validates GeneratedText before rendering;
+- the raw GeneratedText, validated GeneratedText, render context, rendered
+  Markdown, and metadata are inspectable after a run;
+- existing daily, tomorrow, three-day, weekend, and storm reports keep their
+  current full-Markdown Scriptorium path until explicitly migrated;
+- failures in structured generation, validation, or template rendering are
+  diagnosable from saved artifacts where practical;
+- the managed Markdown report remains the source for optional copies and
+  distributor uploads;
+- the implementation does not introduce a generic workflow engine, plugin
+  system, or broad report rewrite.
+
+## Deferred Work
+
+Do not include these in the first implementation:
+
+- migrating daily, tomorrow, three-day, weekend, or storm to GeneratedText;
+- user-configurable template or local schema source overrides;
+- user-defined templates or schemas;
+- a generic workflow engine;
+- template hot reloading;
+- dynamic module selection for templates;
+- distributor-specific generated-text behavior;
+- prompt/schema generation from Go structs;
+- scheduled hourly batch membership;
+- configurable hourly report duration.
+
+## Open Questions
+
+No questions block roadmap creation.
+
+Recommended approach: keep JSON schema enforcement in Scriptorium for retry
+behavior, and also perform weatherreporter-side typed validation before
+rendering. Weatherreporter should validate the returned JSON by unmarshalling
+into the report's GeneratedText type and checking required fields; it should
+not add a Go JSON Schema dependency in the first implementation.
+
+Viable alternative: trust Scriptorium validation and only unmarshal into typed
+Go structs without explicit required-field checks. This is simpler, but it
+weakens weatherreporter's rendering contract and may make empty or malformed
+LLM output harder to diagnose.