343 lines
14 KiB
Markdown
343 lines
14 KiB
Markdown
# GeneratedText Report Rendering Roadmap
|
|
|
|
## Purpose
|
|
|
|
This roadmap defines planned work to move selected reports from full Markdown
|
|
LLM generation to deterministic Markdown rendering with structured LLM-filled
|
|
text slots. The first target report is the rolling next-hours report, named
|
|
`hourly`.
|
|
|
|
This feature is not implemented yet, so this document lives under
|
|
`docs/roadmap/`.
|
|
|
|
## Intent
|
|
|
|
The current module and prompt-input system has reached the point where many
|
|
report facts can be calculated deterministically. The next step is to let Go
|
|
own report structure while using the LLM for only the prose that benefits from
|
|
language synthesis.
|
|
|
|
The intended workflow is:
|
|
|
|
1. collect upstream facts once;
|
|
2. derive deterministic report facts;
|
|
3. build module outputs and the prompt data package;
|
|
4. ask Scriptorium for structured JSON using the schema associated with the
|
|
prompt ID on the Scriptorium backend;
|
|
5. validate and persist that structured response as `GeneratedText`;
|
|
6. build a curated render context;
|
|
7. render final Markdown from an embedded human-editable template.
|
|
|
|
This should improve consistency for headings, dates, ordering, units, recurring
|
|
sections, and report-specific format while preserving the LLM's role in
|
|
summarizing and explaining weather impacts.
|
|
|
|
## Terminology
|
|
|
|
- `CollectedFacts`: normalized upstream inputs from Weather API sources.
|
|
- `DerivedFacts`: deterministic Go-calculated facts sliced to a report period.
|
|
- `ModuleOutput`: prompt-facing deterministic module stanzas.
|
|
- `GeneratedText`: validated structured LLM output. It is not authoritative
|
|
weather data; it is prose or short text intended to fill named template
|
|
slots.
|
|
- `RenderContext`: the final deterministic object passed to a Markdown
|
|
template. It may expose selected values from CollectedFacts, DerivedFacts,
|
|
ModuleOutput, report metadata, and GeneratedText.
|
|
|
|
The report template should not receive the unbounded raw data package directly.
|
|
It should receive a curated render context so templates remain readable and do
|
|
not become a second application logic layer.
|
|
|
|
## Locked Decisions
|
|
|
|
- Use `GeneratedText` as the name for the structured LLM output layer.
|
|
- Continue using Scriptorium behind `internal/adapters/scriptorium`.
|
|
- Use Scriptorium structured JSON output. The Scriptorium backend associates
|
|
the prompt ID with the JSON schema; weatherreporter does not pass a schema
|
|
file at runtime.
|
|
- Keep Scriptorium retries inside Scriptorium.
|
|
- Also validate the returned JSON inside `weatherreporter` before rendering.
|
|
- Use embedded, separate, human-editable Markdown template files.
|
|
- Use embedded, separate, human-readable JSON schema files.
|
|
- Use Go standard library `text/template` for Markdown rendering unless a
|
|
concrete limitation appears.
|
|
- Let Scriptorium enforce the prompt-associated JSON Schema and perform
|
|
retries, then have weatherreporter validate by unmarshalling into a typed
|
|
GeneratedText struct and checking required fields. Do not add a Go JSON
|
|
Schema dependency in the first implementation.
|
|
- Do not generate schemas or templates from inline Go.
|
|
- Start with the rolling next-hours report only.
|
|
- Use `hourly` as the only public report ID for the rolling next-hours
|
|
report.
|
|
- Do not keep compatibility aliases from retired roadmap terminology.
|
|
- Do not add a generic workflow engine or plugin system.
|
|
|
|
## Hourly Report Target
|
|
|
|
The rolling next-hours report target identity is:
|
|
|
|
- report ID: `hourly`
|
|
- CLI command: `weatherreporter generate hourly`
|
|
- display name: `Hourly Report`
|
|
- prompt ID for structured text generation: `weather.hourly_generated_text`
|
|
- Markdown template ID: `hourly`
|
|
- GeneratedText schema ID: `hourly`
|
|
- artifact group: `hourly`
|
|
- batch output name: `hourly.md`
|
|
- default valid-period length: 6 hours
|
|
- valid period: `[generation_time, generation_time + hourlyReportHours)`
|
|
|
|
Use a package-owned constant for the valid-period length, initially 6 hours.
|
|
The duration should not be user-configurable in the first implementation.
|
|
|
|
The report remains explicit-generation only at first. Do not add it to morning
|
|
or evening batches until cadence and downstream behavior are proven.
|
|
|
|
## Embedded Asset Layout
|
|
|
|
Recommended asset package:
|
|
|
|
```text
|
|
internal/reporttemplate/
|
|
templates/
|
|
hourly.md.tmpl
|
|
schemas/
|
|
hourly.generated_text.schema.json
|
|
```
|
|
|
|
`internal/reporttemplate` should embed assets with `go:embed` and expose a
|
|
narrow API for:
|
|
|
|
- looking up a template by template ID;
|
|
- looking up a schema by schema ID;
|
|
- rendering Markdown from a typed render context.
|
|
|
|
Do not allow template or schema asset IDs to come from free-form user input in
|
|
the initial implementation. Report definitions should declare the IDs.
|
|
|
|
## Report Definition Fields
|
|
|
|
Extend report definitions only as needed for this generation mode. Required
|
|
fields:
|
|
|
|
- `GenerationMode`
|
|
- `TemplateID`
|
|
- `GeneratedTextSchemaID`
|
|
|
|
Recommended generation modes:
|
|
|
|
- `GenerationModeScriptoriumMarkdown`: existing full Markdown generation path.
|
|
- `GenerationModeGeneratedTextTemplate`: structured GeneratedText plus
|
|
deterministic template rendering.
|
|
|
|
Existing reports should stay on the current full Markdown path until migrated.
|
|
The `hourly` report should use the GeneratedText/template path from the start.
|
|
For generated-text reports, the existing `PromptID` field remains the
|
|
Scriptorium prompt ID and should be set to `weather.hourly_generated_text`.
|
|
|
|
## GeneratedText Contract
|
|
|
|
The hourly JSON schema should define only the text slots the template needs.
|
|
Keep the first schema small. Example shape:
|
|
|
|
```json
|
|
{
|
|
"summary": "Brief hourly overview.",
|
|
"timing": "Plain-language timing of notable changes.",
|
|
"impacts": "Practical impacts for the next few hours.",
|
|
"confidence": "Optional confidence or uncertainty note."
|
|
}
|
|
```
|
|
|
|
The exact schema should be locked during implementation based on the first
|
|
hourly template. Required fields should be genuinely required by the template.
|
|
Optional fields should be omitted from the rendered Markdown when empty.
|
|
|
|
GeneratedText should be persisted as structured JSON exactly as validated by
|
|
weatherreporter. Raw Scriptorium output should also be persisted for debugging
|
|
when practical.
|
|
|
|
## Template Contract
|
|
|
|
The hourly Markdown template should be a readable file that a human can edit
|
|
without digging into Go code. It should own:
|
|
|
|
- Markdown heading structure;
|
|
- section order;
|
|
- deterministic fact placement;
|
|
- conditional inclusion of optional GeneratedText slots;
|
|
- final report layout.
|
|
|
|
The template should not own:
|
|
|
|
- Weather API fetching;
|
|
- fact derivation;
|
|
- module composition;
|
|
- Scriptorium invocation;
|
|
- schema validation;
|
|
- path construction;
|
|
- distributor upload behavior.
|
|
|
|
The render context should prepare strings and simple values so template logic
|
|
stays shallow. Prefer small conditionals over complex template functions.
|
|
|
|
## Scriptorium Adapter Direction
|
|
|
|
`internal/adapters/scriptorium` should expose weatherreporter-owned request and
|
|
result types for structured JSON generation. The adapter should hide
|
|
Scriptorium argv details from app, report, facts, module, prompt-input, and
|
|
template packages.
|
|
|
|
Scriptorium uses the prompt ID to select the backend-associated structured
|
|
output schema. The adapter should not pass a schema path at runtime.
|
|
|
|
## State And Artifacts
|
|
|
|
Add managed artifacts for the new generation path. Recommended logical
|
|
artifacts:
|
|
|
|
- module snapshot JSON;
|
|
- data package YAML;
|
|
- raw GeneratedText JSON written by Scriptorium;
|
|
- structured Scriptorium command result for the GeneratedText run;
|
|
- validated GeneratedText JSON;
|
|
- render context JSON;
|
|
- rendered Markdown report;
|
|
- metadata linking all of the above.
|
|
|
|
The existing `preflight` artifact should remain as the rendered prompt/input
|
|
inspection artifact before structured generation. GeneratedText reports add
|
|
separate artifacts for raw GeneratedText, structured run result, validated
|
|
GeneratedText, and render context. Metadata should record the
|
|
`GeneratedTextSchemaID`; no per-run schema artifact is persisted.
|
|
|
|
Artifact paths should remain under the workspace and use the report artifact
|
|
group, valid start date, and RunID conventions already used by state.
|
|
|
|
## End-To-End Behavior
|
|
|
|
Reports using `GenerationModeGeneratedTextTemplate` should follow the same
|
|
fact collection, derivation, module snapshot, prompt package, state, and
|
|
notification boundaries as current reports. The difference is limited to the
|
|
last rendering segment: Scriptorium returns structured GeneratedText, Go
|
|
validates that structure, Go builds a curated render context, and an embedded
|
|
template renders the final Markdown.
|
|
|
|
The existing full-Markdown Scriptorium path should remain available for
|
|
reports that have not migrated.
|
|
|
|
## Target Architecture
|
|
|
|
GeneratedText rendering adds one new generation mode, not a replacement for the
|
|
whole report system. Reports that opt into this mode should continue to use
|
|
the existing report registry, fact pipeline, module snapshot, data package,
|
|
state store, and notification boundaries.
|
|
|
|
The target flow is:
|
|
|
|
1. `internal/report` declares whether a report uses the existing full-Markdown
|
|
Scriptorium path or the GeneratedText/template path.
|
|
2. The app builds CollectedFacts, DerivedFacts, module snapshots, and the
|
|
prompt data package using the same boundaries as existing reports.
|
|
3. `internal/adapters/scriptorium` asks Scriptorium for structured JSON using
|
|
the report's prompt ID. The Scriptorium backend owns the prompt-to-schema
|
|
association.
|
|
4. Weatherreporter validates the returned JSON as typed `GeneratedText`.
|
|
5. A report-specific render context is built from selected deterministic facts,
|
|
module outputs, metadata, and GeneratedText.
|
|
6. `internal/reporttemplate` renders the final Markdown from an embedded,
|
|
human-editable template.
|
|
7. The managed Markdown report remains the canonical output and the distributor
|
|
upload source.
|
|
|
|
This structure keeps each layer understandable:
|
|
|
|
- CollectedFacts and DerivedFacts remain weather-data contracts.
|
|
- ModuleOutput remains the prompt-facing deterministic facts contract.
|
|
- GeneratedText remains LLM-written prose slots, not weather truth.
|
|
- RenderContext remains a curated template input, not a second raw data package.
|
|
- Templates own presentation order and headings, not weather derivation logic.
|
|
|
|
## Editing Surface
|
|
|
|
The desired human editing surface is file-based and report-specific. For the
|
|
hourly report, the important files should be easy to find and inspect:
|
|
|
|
- one Markdown template file for the report layout;
|
|
- one JSON schema file describing the expected GeneratedText payload;
|
|
- one report definition declaring the prompt ID, template ID, schema ID, module
|
|
composition, valid period, artifact group, and output naming policy.
|
|
|
|
The template and schema should be embedded into the binary, but they should
|
|
remain ordinary files in the repository. A maintainer should be able to tweak
|
|
headings, reorder sections, or inspect the structured prose contract without
|
|
reading unrelated orchestration code.
|
|
|
|
## Boundary Guidelines
|
|
|
|
- Keep report identity, prompt IDs, generation mode, template IDs, schema IDs,
|
|
valid periods, module composition, and output naming in `internal/report`.
|
|
- Keep Scriptorium subprocess details in `internal/adapters/scriptorium`.
|
|
- Keep Markdown template lookup and rendering in `internal/reporttemplate`.
|
|
- Keep GeneratedText validation in a narrow package or component with typed
|
|
report-specific structs.
|
|
- Keep app orchestration responsible for sequencing and artifact persistence,
|
|
not for report-specific prose or template structure.
|
|
- Keep distributor upload behavior unchanged: upload the managed Markdown
|
|
report after it exists.
|
|
- Do not let templates fetch data, derive facts, perform source selection, or
|
|
construct workspace paths.
|
|
- Do not let Scriptorium-specific flags, schema paths, or subprocess details
|
|
leak into report definitions or domain packages.
|
|
- Do not add user-provided template or schema paths in the first
|
|
implementation.
|
|
|
|
## Target Acceptance Criteria
|
|
|
|
The feature is successful when:
|
|
|
|
- `weatherreporter generate hourly` produces a deterministic Markdown report
|
|
whose headings, dates, and section order come from an embedded template;
|
|
- Scriptorium fills only the report's structured GeneratedText slots;
|
|
- weatherreporter validates GeneratedText before rendering;
|
|
- the raw GeneratedText, validated GeneratedText, render context, rendered
|
|
Markdown, and metadata are inspectable after a run;
|
|
- existing daily, tomorrow, three-day, weekend, and storm reports keep their
|
|
current full-Markdown Scriptorium path until explicitly migrated;
|
|
- failures in structured generation, validation, or template rendering are
|
|
diagnosable from saved artifacts where practical;
|
|
- the managed Markdown report remains the source for optional copies and
|
|
distributor uploads;
|
|
- the implementation does not introduce a generic workflow engine, plugin
|
|
system, or broad report rewrite.
|
|
|
|
## Deferred Work
|
|
|
|
Do not include these in the first implementation:
|
|
|
|
- migrating daily, tomorrow, three-day, weekend, or storm to GeneratedText;
|
|
- user-configurable template or local schema source overrides;
|
|
- user-defined templates or schemas;
|
|
- a generic workflow engine;
|
|
- template hot reloading;
|
|
- dynamic module selection for templates;
|
|
- distributor-specific generated-text behavior;
|
|
- prompt/schema generation from Go structs;
|
|
- scheduled hourly batch membership;
|
|
- configurable hourly report duration.
|
|
|
|
## Open Questions
|
|
|
|
No questions block roadmap creation.
|
|
|
|
Recommended approach: keep JSON schema enforcement in Scriptorium for retry
|
|
behavior, and also perform weatherreporter-side typed validation before
|
|
rendering. Weatherreporter should validate the returned JSON by unmarshalling
|
|
into the report's GeneratedText type and checking required fields; it should
|
|
not add a Go JSON Schema dependency in the first implementation.
|
|
|
|
Viable alternative: trust Scriptorium validation and only unmarshal into typed
|
|
Go structs without explicit required-field checks. This is simpler, but it
|
|
weakens weatherreporter's rendering contract and may make empty or malformed
|
|
LLM output harder to diagnose.
|