eric/weatherreporter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Persist module snapshots for generated reports

Browse Source
This commit is contained in:
Eric Rakestraw
2026-06-09 20:58:35 +00:00
parent 40639309b1
commit 2483c2362d
11 changed files with 236 additions and 82 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
docs/cli.md
View File

@@ -28,16 +28,16 @@ weatherreporter run morning [--config PATH] [--units VALUE] [--tz NAME] [--out-d
weatherreporter run evening [--config PATH] [--units VALUE] [--tz NAME] [--out-dir PATH]
weatherreporter inspect reports [--config PATH] [--limit N]
weatherreporter inspect metadata [--config PATH] RUN_ID
weatherreporter inspect briefing [--config PATH] RUN_ID
weatherreporter inspect modules [--config PATH] RUN_ID
weatherreporter inspect data-package [--config PATH] RUN_ID
weatherreporter inspect prior [--config PATH] RUN_ID
weatherreporter inspect sources [--config PATH] RUN_ID
```
`generate` commands write briefing, data package, preflight, report, and
metadata artifacts under the configured workspace. `--out` writes an extra
Markdown copy for the operator; distributor notification uses the managed
report path, not the extra copy. `generate storm` requires explicit
`generate` commands write briefing, module snapshot, data package, preflight,
report, and metadata artifacts under the configured workspace. `--out` writes
an extra Markdown copy for the operator; distributor notification uses the
managed report path, not the extra copy. `generate storm` requires explicit
event-window bounds with `--start` and `--end`.
`run morning` generates Daily Today and the 3-Day Outlook, plus Weekend Outlook
@@ -88,14 +88,15 @@ weatherreporter run evening --out-dir ./reports
```sh
weatherreporter inspect reports --limit 10
weatherreporter inspect metadata 20260529T100000.000000000Z_daily_today
weatherreporter inspect briefing 20260529T100000.000000000Z_daily_today
weatherreporter inspect modules 20260529T100000.000000000Z_daily_today
weatherreporter inspect data-package 20260529T100000.000000000Z_daily_today
weatherreporter inspect prior 20260529T100000.000000000Z_daily_today
weatherreporter inspect sources 20260529T100000.000000000Z_daily_today
```
`inspect reports` lists recent generated runs with artifact paths and source
warning counts. The other inspect commands require a RunID. `inspect prior`
warning counts. The other inspect commands require a RunID. `inspect modules`
returns the persisted ordered module snapshot for a run. `inspect prior`
returns the prior comparable snapshot metadata selected from stored metadata, or
`null` when none exists. `inspect sources` shows source provenance and source
warnings without dumping full weather payloads.

28
docs/operations.md
View File

@@ -52,18 +52,22 @@ workspace/
daily/
YYYY-MM-DD/
<run_id>.briefing.json
<run_id>.modules.json
<run_id>.metadata.json
three-day/
YYYY-MM-DD/
<run_id>.briefing.json
<run_id>.modules.json
<run_id>.metadata.json
weekend/
YYYY-MM-DD/
<run_id>.briefing.json
<run_id>.modules.json
<run_id>.metadata.json
storm/
YYYY-MM-DD/
<run_id>.briefing.json
<run_id>.modules.json
<run_id>.metadata.json
data-packages/
daily/
@@ -164,11 +168,11 @@ report generation has a distinct retry identity. The default bundle path uses
the valid-period start date, artifact group, and RunID. Distributor owns
destination merge, retention, and derived snapshot behavior such as `latest`.
Notification happens after final metadata save. Weather API, briefing,
data-package, render preflight, Scriptorium run, and metadata-save failures do
not trigger notification. A notification failure fails that report. In a batch,
other reports continue, the failed report includes notification fields in the
JSON summary, and the batch returns nonzero.
Notification happens after final metadata save. Weather API, briefing, module
snapshot, data-package, render preflight, Scriptorium run, and metadata-save
failures do not trigger notification. A notification failure fails that report.
In a batch, other reports continue, the failed report includes notification
fields in the JSON summary, and the batch returns nonzero.
Each notification attempt writes a debug artifact under `notifications/`. The
artifact records the rendered pipeline ID, bundle ID, idempotency key, managed
@@ -195,7 +199,7 @@ They do not fetch weather data or run `scriptorium`.
```text
weatherreporter inspect reports --limit 10
weatherreporter inspect metadata RUN_ID
weatherreporter inspect briefing RUN_ID
weatherreporter inspect modules RUN_ID
weatherreporter inspect data-package RUN_ID
weatherreporter inspect prior RUN_ID
weatherreporter inspect sources RUN_ID
@@ -203,11 +207,11 @@ weatherreporter inspect sources RUN_ID
Use `inspect reports` to find recent RunIDs and artifact paths. Use
`inspect metadata` to see the artifact links recorded for a run. Use
`inspect briefing` and `inspect data-package` to review the exact structured
inputs used for rendering. Use `inspect prior` to see the prior comparable
snapshot selected for Recent Changes, or `null` when none exists. Use
`inspect sources` to review source provenance and warnings without dumping full
weather payloads.
`inspect modules` to review the persisted ordered module snapshot, and
`inspect data-package` to review the current structured prompt package used for
rendering. Use `inspect prior` to see the prior comparable snapshot selected
for Recent Changes, or `null` when none exists. Use `inspect sources` to review
source provenance and warnings without dumping full weather payloads.
## Recent Changes
@@ -241,7 +245,7 @@ For a bad report, start with:
```text
weatherreporter inspect metadata RUN_ID
weatherreporter inspect sources RUN_ID
weatherreporter inspect briefing RUN_ID
weatherreporter inspect modules RUN_ID
weatherreporter inspect data-package RUN_ID
weatherreporter inspect prior RUN_ID
```

153
internal/app/app.go
View File

@@ -17,6 +17,7 @@ import (
"gitea.maximumdirect.net/eric/weatherreporter/internal/facts"
"gitea.maximumdirect.net/eric/weatherreporter/internal/fileutil"
"gitea.maximumdirect.net/eric/weatherreporter/internal/forecast"
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
"gitea.maximumdirect.net/eric/weatherreporter/internal/state"
@@ -73,6 +74,11 @@ type BriefingRequest struct {
OutputPath string
}
type ModuleSnapshotRequest struct {
Config config.Config
Resolved report.Resolved
}
type ReportRequest struct {
Config config.Config
Resolved report.Resolved
@@ -88,21 +94,23 @@ type BriefingResult struct {
}
type ReportResult struct {
Briefing briefing.Package
BriefingPath string
DataPackage promptinput.Package
DataPackagePath string
PreflightPath string
ReportPath string
OutputPath string
NotificationPath string
Metadata state.Metadata
MetadataPath string
PriorSnapshot *state.PriorSnapshot
RecentChanges []changes.Change
RenderResult *scriptorium.RenderResult
RunResult *scriptorium.RunResult
Notification *NotificationResult
Briefing briefing.Package
ModuleSnapshot module.Snapshot
BriefingPath string
ModuleSnapshotPath string
DataPackage promptinput.Package
DataPackagePath string
PreflightPath string
ReportPath string
OutputPath string
NotificationPath string
Metadata state.Metadata
MetadataPath string
PriorSnapshot *state.PriorSnapshot
RecentChanges []changes.Change
RenderResult *scriptorium.RenderResult
RunResult *scriptorium.RunResult
Notification *NotificationResult
}
type BatchResult struct {
@@ -493,11 +501,22 @@ func GenerateReport(ctx context.Context, req ReportRequest) (*ReportResult, erro
if err != nil {
return nil, err
}
moduleSnapshot, err := BuildModuleSnapshot(ModuleSnapshotRequest{
Config: req.Config,
Resolved: req.Resolved,
}, bundle)
if err != nil {
return nil, err
}
briefingPath, err := store.SaveBriefing(ctx, req.Resolved, briefingPackage)
if err != nil {
return nil, err
}
moduleSnapshotPath, err := store.SaveModuleSnapshot(ctx, req.Resolved, moduleSnapshot)
if err != nil {
return nil, err
}
recentChanges, err := recentChanges(ctx, store, priorSnapshot, briefingPackage, req.Config.RecentChange)
if err != nil {
@@ -537,6 +556,7 @@ func GenerateReport(ctx context.Context, req ReportRequest) (*ReportResult, erro
}
metadata := state.BuildMetadata(req.Resolved, briefingPackage, state.ArtifactPaths{
Briefing: briefingPath,
ModuleSnapshot: moduleSnapshotPath,
Metadata: paths.Metadata,
DataPackage: dataPackagePath,
Preflight: preflightPath,
@@ -590,21 +610,23 @@ func GenerateReport(ctx context.Context, req ReportRequest) (*ReportResult, erro
}
return &ReportResult{
Briefing: briefingPackage,
BriefingPath: briefingPath,
DataPackage: dataPackage,
DataPackagePath: dataPackagePath,
PreflightPath: preflightPath,
ReportPath: reportPath,
OutputPath: outputPath,
NotificationPath: notificationPath,
Metadata: metadata,
MetadataPath: metadataPath,
PriorSnapshot: priorSnapshot,
RecentChanges: recentChanges,
RenderResult: renderResult,
RunResult: runResult,
Notification: notification,
Briefing: briefingPackage,
ModuleSnapshot: moduleSnapshot,
BriefingPath: briefingPath,
ModuleSnapshotPath: moduleSnapshotPath,
DataPackage: dataPackage,
DataPackagePath: dataPackagePath,
PreflightPath: preflightPath,
ReportPath: reportPath,
OutputPath: outputPath,
NotificationPath: notificationPath,
Metadata: metadata,
MetadataPath: metadataPath,
PriorSnapshot: priorSnapshot,
RecentChanges: recentChanges,
RenderResult: renderResult,
RunResult: runResult,
Notification: notification,
}, nil
}
@@ -807,20 +829,7 @@ func distributorUploadFiles(sourcePath string, bundlePaths []string) []distribut
func BuildBriefing(req BriefingRequest, bundle *weatherdata.Bundle) (briefing.Package, error) {
collected := facts.BuildCollected(bundle)
dayparts := make([]forecast.DaypartDefinition, 0, len(req.Config.Dayparts))
for _, daypart := range req.Config.Dayparts {
dayparts = append(dayparts, forecast.DaypartDefinition{
Name: daypart.Name,
Start: daypart.Start,
End: daypart.End,
})
}
derived, err := facts.BuildDerived(facts.BuildDerivedRequest{
Resolved: req.Resolved,
Timezone: req.Config.WeatherAPI.Timezone,
Dayparts: dayparts,
Collected: collected,
})
derived, err := buildDerivedFacts(req.Config, req.Resolved, collected)
if err != nil {
return briefing.Package{}, err
}
@@ -846,6 +855,62 @@ func BuildBriefing(req BriefingRequest, bundle *weatherdata.Bundle) (briefing.Pa
}
}
func BuildModuleSnapshot(req ModuleSnapshotRequest, bundle *weatherdata.Bundle) (module.Snapshot, error) {
collected := facts.BuildCollected(bundle)
derived, err := buildDerivedFacts(req.Config, req.Resolved, collected)
if err != nil {
return module.Snapshot{}, err
}
registry, err := briefing.DefaultModuleRegistry()
if err != nil {
return module.Snapshot{}, err
}
moduleContext := briefing.ModuleContext{
Resolved: req.Resolved,
Collected: collected,
Derived: derived,
Units: req.Config.WeatherAPI.Units,
Timezone: req.Config.WeatherAPI.Timezone,
Location: briefingLocation(req.Config),
}
var outputs []module.Output
for _, item := range req.Resolved.Definition.Modules {
definition, err := registry.Lookup(item.ID)
if err != nil {
return module.Snapshot{}, err
}
if definition.Builder == nil {
continue
}
output, err := registry.BuildModule(moduleContext, item)
if err != nil {
return module.Snapshot{}, err
}
if output == nil {
continue
}
outputs = append(outputs, *output)
}
return module.NewSnapshot(outputs)
}
func buildDerivedFacts(cfg config.Config, resolved report.Resolved, collected facts.CollectedFacts) (facts.DerivedFacts, error) {
dayparts := make([]forecast.DaypartDefinition, 0, len(cfg.Dayparts))
for _, daypart := range cfg.Dayparts {
dayparts = append(dayparts, forecast.DaypartDefinition{
Name: daypart.Name,
Start: daypart.Start,
End: daypart.End,
})
}
return facts.BuildDerived(facts.BuildDerivedRequest{
Resolved: resolved,
Timezone: cfg.WeatherAPI.Timezone,
Dayparts: dayparts,
Collected: collected,
})
}
func briefingLocation(cfg config.Config) *briefing.LocationContext {
location := briefing.LocationContext{
ID: cfg.Location.ID,

25
internal/app/app_test.go
View File

@@ -194,7 +194,14 @@ func TestGenerateReportWritesReportAndPreflight(t *testing.T) {
if renderer.runRequest.OutputPath != result.ReportPath {
t.Fatalf("run OutputPath = %q, want managed report path %q", renderer.runRequest.OutputPath, result.ReportPath)
}
assertPathsExist(t, result.BriefingPath, result.DataPackagePath, result.PreflightPath, result.ReportPath, result.MetadataPath, outputPath)
assertPathsExist(t, result.BriefingPath, result.ModuleSnapshotPath, result.DataPackagePath, result.PreflightPath, result.ReportPath, result.MetadataPath, outputPath)
snapshotData, err := os.ReadFile(result.ModuleSnapshotPath)
if err != nil {
t.Fatalf("read module snapshot: %v", err)
}
if !strings.Contains(string(snapshotData), module.SnapshotSchemaVersion) || !strings.Contains(string(snapshotData), `"metadata"`) || !strings.Contains(string(snapshotData), `"derived_daily_summary"`) {
t.Fatalf("module snapshot missing expected stanzas:\n%s", string(snapshotData))
}
data, err := os.ReadFile(result.DataPackagePath)
if err != nil {
t.Fatalf("read data package: %v", err)
@@ -238,7 +245,7 @@ func TestGenerateReportWritesReportAndPreflight(t *testing.T) {
if result.Metadata.RunID != resolved.Metadata().RunID {
t.Fatalf("metadata RunID = %q, want %q", result.Metadata.RunID, resolved.Metadata().RunID)
}
if result.Metadata.BriefingPath != result.BriefingPath || result.Metadata.DataPackagePath != result.DataPackagePath {
if result.Metadata.BriefingPath != result.BriefingPath || result.Metadata.ModuleSnapshotPath != result.ModuleSnapshotPath || result.Metadata.DataPackagePath != result.DataPackagePath {
t.Fatalf("metadata does not link artifact paths: %#v", result.Metadata)
}
if result.Metadata.RenderedReportPath != result.ReportPath {
@@ -647,6 +654,7 @@ func TestGenerateReportIncludesRecentChangesFromPriorSnapshot(t *testing.T) {
}
_, err = store.SaveMetadata(context.Background(), state.BuildMetadata(priorResolved, priorBriefing, state.ArtifactPaths{
Briefing: priorBriefingPath,
ModuleSnapshot: priorPaths.ModuleSnapshot,
Metadata: priorPaths.Metadata,
DataPackage: priorPaths.DataPackage,
Preflight: priorPaths.Preflight,
@@ -762,6 +770,7 @@ func TestTomorrowReportCanCompareAgainstPriorDailySnapshot(t *testing.T) {
}
_, err = store.SaveMetadata(context.Background(), state.BuildMetadata(priorResolved, priorBriefing, state.ArtifactPaths{
Briefing: priorBriefingPath,
ModuleSnapshot: priorPaths.ModuleSnapshot,
Metadata: priorPaths.Metadata,
DataPackage: priorPaths.DataPackage,
Preflight: priorPaths.Preflight,
@@ -829,6 +838,7 @@ func TestGenerateThreeDayReportWritesReportAndRecentChanges(t *testing.T) {
}
_, err = store.SaveMetadata(context.Background(), state.BuildMetadata(priorResolved, priorBriefing, state.ArtifactPaths{
Briefing: priorBriefingPath,
ModuleSnapshot: priorPaths.ModuleSnapshot,
Metadata: priorPaths.Metadata,
DataPackage: priorPaths.DataPackage,
Preflight: priorPaths.Preflight,
@@ -905,6 +915,7 @@ func TestGenerateWeekendReportWritesReportAndRecentChanges(t *testing.T) {
}
_, err = store.SaveMetadata(context.Background(), state.BuildMetadata(priorResolved, priorBriefing, state.ArtifactPaths{
Briefing: priorBriefingPath,
ModuleSnapshot: priorPaths.ModuleSnapshot,
Metadata: priorPaths.Metadata,
DataPackage: priorPaths.DataPackage,
Preflight: priorPaths.Preflight,
@@ -1045,15 +1056,15 @@ func TestInspectGeneratedReportArtifacts(t *testing.T) {
if err != nil {
t.Fatalf("InspectMetadata() error = %v", err)
}
if metadata.BriefingPath != result.BriefingPath || metadata.DataPackagePath != result.DataPackagePath {
if metadata.BriefingPath != result.BriefingPath || metadata.ModuleSnapshotPath != result.ModuleSnapshotPath || metadata.DataPackagePath != result.DataPackagePath {
t.Fatalf("metadata paths = %#v, want generated artifact paths", metadata)
}
briefingPackage, err := InspectBriefing(context.Background(), InspectRunRequest{Config: cfg, RunID: result.Metadata.RunID})
moduleSnapshot, err := InspectModules(context.Background(), InspectRunRequest{Config: cfg, RunID: result.Metadata.RunID})
if err != nil {
t.Fatalf("InspectBriefing() error = %v", err)
t.Fatalf("InspectModules() error = %v", err)
}
if briefingPackage.Metadata.RunID != result.Metadata.RunID {
t.Fatalf("briefing RunID = %q, want %q", briefingPackage.Metadata.RunID, result.Metadata.RunID)
if moduleSnapshot.SchemaVersion != module.SnapshotSchemaVersion || len(moduleSnapshot.Outputs) == 0 {
t.Fatalf("module snapshot = %#v, want persisted outputs", moduleSnapshot)
}
dataPackage, err := InspectDataPackage(context.Background(), InspectRunRequest{Config: cfg, RunID: result.Metadata.RunID})
if err != nil {

7
internal/app/inspect.go
View File

@@ -6,6 +6,7 @@ import (
"gitea.maximumdirect.net/eric/weatherreporter/internal/briefing"
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
"gitea.maximumdirect.net/eric/weatherreporter/internal/state"
@@ -44,12 +45,12 @@ func InspectMetadata(ctx context.Context, req InspectRunRequest) (state.Metadata
return inspection.metadata, err
}
func InspectBriefing(ctx context.Context, req InspectRunRequest) (briefing.Package, error) {
func InspectModules(ctx context.Context, req InspectRunRequest) (module.Snapshot, error) {
inspection, err := inspectRun(ctx, req)
if err != nil {
return briefing.Package{}, err
return module.Snapshot{}, err
}
return inspection.store.LoadBriefing(ctx, inspection.metadata.BriefingPath)
return inspection.store.LoadModuleSnapshot(ctx, inspection.metadata.ModuleSnapshotPath)
}
func InspectDataPackage(ctx context.Context, req InspectRunRequest) (promptinput.Package, error) {

6
internal/cli/root.go
View File

@@ -26,7 +26,7 @@ Usage:
weatherreporter run evening [--config PATH] [--units VALUE] [--tz NAME] [--out-dir PATH]
weatherreporter inspect reports [--config PATH] [--limit N]
weatherreporter inspect metadata [--config PATH] RUN_ID
weatherreporter inspect briefing [--config PATH] RUN_ID
weatherreporter inspect modules [--config PATH] RUN_ID
weatherreporter inspect data-package [--config PATH] RUN_ID
weatherreporter inspect prior [--config PATH] RUN_ID
weatherreporter inspect sources [--config PATH] RUN_ID
@@ -118,8 +118,8 @@ var inspectRunCommands = []inspectRunCommand{
{Name: "metadata", Inspect: func(ctx context.Context, req app.InspectRunRequest) (any, error) {
return app.InspectMetadata(ctx, req)
}},
{Name: "briefing", Inspect: func(ctx context.Context, req app.InspectRunRequest) (any, error) {
return app.InspectBriefing(ctx, req)
{Name: "modules", Inspect: func(ctx context.Context, req app.InspectRunRequest) (any, error) {
return app.InspectModules(ctx, req)
}},
{Name: "data-package", Inspect: func(ctx context.Context, req app.InspectRunRequest) (any, error) {
return app.InspectDataPackage(ctx, req)

16
internal/cli/root_test.go
View File

@@ -28,6 +28,10 @@ func TestRunHelpLongFlag(t *testing.T) {
if !strings.Contains(stdout.String(), "generate daily") {
t.Fatalf("help output missing generate command:\n%s", stdout.String())
}
removedInspectCommand := "inspect " + "briefing"
if !strings.Contains(stdout.String(), "inspect modules") || strings.Contains(stdout.String(), removedInspectCommand) {
t.Fatalf("help output has wrong inspect commands:\n%s", stdout.String())
}
}
func TestRunHelpShortFlag(t *testing.T) {
@@ -729,7 +733,8 @@ func TestRunInspectGeneratedArtifacts(t *testing.T) {
t.Fatalf("inspect reports output missing run:\n%s", stdout.String())
}
for _, command := range []string{"metadata", "briefing", "data-package", "sources"} {
var sourcesOutput string
for _, command := range []string{"metadata", "modules", "data-package", "sources"} {
stdout.Reset()
err = runner.Run(context.Background(), []string{"inspect", command, "--config", configPath, runID}, &stdout, &stderr)
if err != nil {
@@ -738,9 +743,12 @@ func TestRunInspectGeneratedArtifacts(t *testing.T) {
if !strings.Contains(stdout.String(), runID) {
t.Fatalf("inspect %s output missing run id:\n%s", command, stdout.String())
}
if command == "sources" {
sourcesOutput = stdout.String()
}
}
if !strings.Contains(stdout.String(), `"warnings"`) {
t.Fatalf("inspect sources output missing warnings:\n%s", stdout.String())
if !strings.Contains(sourcesOutput, `"warnings"`) {
t.Fatalf("inspect sources output missing warnings:\n%s", sourcesOutput)
}
}
@@ -772,7 +780,7 @@ func TestRunInspectRunCommandsParseRunIDAndConfig(t *testing.T) {
t.Fatalf("write config: %v", err)
}
runner := Runner{Clock: fixedClock()}
commands := []string{"metadata", "briefing", "data-package", "prior", "sources"}
commands := []string{"metadata", "modules", "data-package", "prior", "sources"}
for _, command := range commands {
t.Run(command+" requires run id", func(t *testing.T) {

34
internal/state/filesystem.go
View File

@@ -13,6 +13,7 @@ import (
"gitea.maximumdirect.net/eric/weatherreporter/internal/briefing"
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
"gitea.maximumdirect.net/eric/weatherreporter/internal/fileutil"
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
)
@@ -28,6 +29,7 @@ type FilesystemStore struct {
type ArtifactPaths struct {
Briefing string `json:"briefing"`
ModuleSnapshot string `json:"moduleSnapshot"`
Metadata string `json:"metadata"`
DataPackage string `json:"dataPackage"`
Preflight string `json:"preflight"`
@@ -91,6 +93,7 @@ func (s *FilesystemStore) Paths(resolved report.Resolved) (ArtifactPaths, error)
filenameBase := metadata.RunID
return ArtifactPaths{
Briefing: s.join(s.snapshotsDir, group, validDate, filenameBase+".briefing.json"),
ModuleSnapshot: s.join(s.snapshotsDir, group, validDate, filenameBase+".modules.json"),
Metadata: s.join(s.snapshotsDir, group, validDate, filenameBase+".metadata.json"),
DataPackage: s.join(s.dataPackagesDir, group, validDate, filenameBase+".data_package.json"),
Preflight: s.join(s.preflightDir, group, validDate, filenameBase+".render.json"),
@@ -110,6 +113,20 @@ func (s *FilesystemStore) SaveBriefing(_ context.Context, resolved report.Resolv
return paths.Briefing, nil
}
func (s *FilesystemStore) SaveModuleSnapshot(_ context.Context, resolved report.Resolved, snapshot module.Snapshot) (string, error) {
paths, err := s.Paths(resolved)
if err != nil {
return "", err
}
if err := snapshot.Validate(); err != nil {
return "", err
}
if err := fileutil.WriteJSONAtomic(paths.ModuleSnapshot, snapshot); err != nil {
return "", err
}
return paths.ModuleSnapshot, nil
}
func (s *FilesystemStore) SaveDataPackage(_ context.Context, resolved report.Resolved, pkg promptinput.Package) (string, error) {
paths, err := s.Paths(resolved)
if err != nil {
@@ -167,6 +184,9 @@ func (s *FilesystemStore) SaveMetadata(_ context.Context, metadata Metadata) (st
if metadata.BriefingPath == "" {
return "", fmt.Errorf("metadata briefing path is required")
}
if metadata.ModuleSnapshotPath == "" {
return "", fmt.Errorf("metadata module snapshot path is required")
}
if metadata.DataPackagePath == "" {
return "", fmt.Errorf("metadata data package path is required")
}
@@ -308,6 +328,20 @@ func (s *FilesystemStore) LoadDataPackage(_ context.Context, path string) (promp
return pkg, nil
}
func (s *FilesystemStore) LoadModuleSnapshot(_ context.Context, path string) (module.Snapshot, error) {
if path == "" {
return module.Snapshot{}, fmt.Errorf("module snapshot path is required")
}
var snapshot module.Snapshot
if err := readJSON(path, &snapshot); err != nil {
return module.Snapshot{}, err
}
if err := snapshot.Validate(); err != nil {
return module.Snapshot{}, err
}
return snapshot, nil
}
func (s *FilesystemStore) reportRecord(path string) (ReportRecord, error) {
var metadata Metadata
if err := readJSON(path, &metadata); err != nil {

29
internal/state/filesystem_test.go
View File

@@ -11,6 +11,7 @@ import (
"gitea.maximumdirect.net/eric/weatherreporter/internal/briefing"
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
"gitea.maximumdirect.net/eric/weatherreporter/internal/timeutil"
@@ -27,6 +28,7 @@ func TestPathsUseRunIDAndWorkspace(t *testing.T) {
for _, want := range []string{
filepath.Join("snapshots", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.briefing.json"),
filepath.Join("snapshots", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.modules.json"),
filepath.Join("snapshots", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.metadata.json"),
filepath.Join("data-packages", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.data_package.json"),
filepath.Join("preflight", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.render.json"),
@@ -56,6 +58,14 @@ func TestSaveArtifactsAndMetadataRoundTrip(t *testing.T) {
if err != nil {
t.Fatalf("SaveDataPackage() error = %v", err)
}
snapshot, err := module.NewSnapshot([]module.Output{{ID: module.Metadata, StanzaName: "metadata", Value: map[string]string{"run_id": resolved.Metadata().RunID}}})
if err != nil {
t.Fatalf("NewSnapshot() error = %v", err)
}
moduleSnapshotPath, err := store.SaveModuleSnapshot(context.Background(), resolved, snapshot)
if err != nil {
t.Fatalf("SaveModuleSnapshot() error = %v", err)
}
preflightPath, err := store.SavePreflight(context.Background(), resolved, PreflightArtifact{Stdout: `{"ok":true}`})
if err != nil {
t.Fatalf("SavePreflight() error = %v", err)
@@ -112,6 +122,7 @@ func TestSaveArtifactsAndMetadataRoundTrip(t *testing.T) {
}
metadata := BuildMetadata(resolved, briefingPackage, ArtifactPaths{
Briefing: briefingPath,
ModuleSnapshot: moduleSnapshotPath,
Metadata: paths.Metadata,
DataPackage: dataPackagePath,
Preflight: preflightPath,
@@ -122,7 +133,7 @@ func TestSaveArtifactsAndMetadataRoundTrip(t *testing.T) {
t.Fatalf("SaveMetadata() error = %v", err)
}
for _, path := range []string{briefingPath, dataPackagePath, preflightPath, notificationPath, renderedReportPath, metadataPath} {
for _, path := range []string{briefingPath, moduleSnapshotPath, dataPackagePath, preflightPath, notificationPath, renderedReportPath, metadataPath} {
if _, err := os.Stat(path); err != nil {
t.Fatalf("expected artifact %q: %v", path, err)
}
@@ -134,6 +145,13 @@ func TestSaveArtifactsAndMetadataRoundTrip(t *testing.T) {
if loadedBriefing.Metadata.RunID != resolved.Metadata().RunID {
t.Fatalf("loaded briefing RunID = %q, want %q", loadedBriefing.Metadata.RunID, resolved.Metadata().RunID)
}
loadedSnapshot, err := store.LoadModuleSnapshot(context.Background(), moduleSnapshotPath)
if err != nil {
t.Fatalf("LoadModuleSnapshot() error = %v", err)
}
if loadedSnapshot.SchemaVersion != module.SnapshotSchemaVersion || len(loadedSnapshot.Outputs) != 1 {
t.Fatalf("loaded module snapshot = %#v, want one metadata output", loadedSnapshot)
}
var decoded Metadata
data, err := os.ReadFile(metadataPath)
if err != nil {
@@ -145,7 +163,7 @@ func TestSaveArtifactsAndMetadataRoundTrip(t *testing.T) {
if decoded.RunID != resolved.Metadata().RunID {
t.Fatalf("RunID = %q, want %q", decoded.RunID, resolved.Metadata().RunID)
}
if decoded.BriefingPath != briefingPath || decoded.DataPackagePath != dataPackagePath || decoded.PreflightPath != preflightPath {
if decoded.BriefingPath != briefingPath || decoded.ModuleSnapshotPath != moduleSnapshotPath || decoded.DataPackagePath != dataPackagePath || decoded.PreflightPath != preflightPath {
t.Fatalf("metadata paths = %#v, want saved artifact paths", decoded)
}
if decoded.RenderedReportPath != renderedReportPath {
@@ -173,6 +191,7 @@ func TestSaveMetadataUsesExplicitMetadataPath(t *testing.T) {
metadata := BuildMetadata(resolved, briefingPackage, ArtifactPaths{
Briefing: otherBriefingPath,
ModuleSnapshot: paths.ModuleSnapshot,
Metadata: paths.Metadata,
DataPackage: paths.DataPackage,
Preflight: paths.Preflight,
@@ -208,6 +227,7 @@ func TestFindPriorSnapshot(t *testing.T) {
}
_, err = store.SaveMetadata(context.Background(), BuildMetadata(first, briefingPackage, ArtifactPaths{
Briefing: briefingPath,
ModuleSnapshot: paths.ModuleSnapshot,
Metadata: paths.Metadata,
DataPackage: paths.DataPackage,
Preflight: paths.Preflight,
@@ -247,6 +267,7 @@ func TestFindPriorSnapshotUsesValidDate(t *testing.T) {
}
_, err = store.SaveMetadata(context.Background(), BuildMetadata(previousDate, briefingPackage, ArtifactPaths{
Briefing: briefingPath,
ModuleSnapshot: paths.ModuleSnapshot,
Metadata: paths.Metadata,
DataPackage: paths.DataPackage,
Preflight: paths.Preflight,
@@ -280,6 +301,7 @@ func TestFindPriorSnapshotSupportsThreeDay(t *testing.T) {
}
_, err = store.SaveMetadata(context.Background(), BuildMetadata(first, briefingPackage, ArtifactPaths{
Briefing: briefingPath,
ModuleSnapshot: paths.ModuleSnapshot,
Metadata: paths.Metadata,
DataPackage: paths.DataPackage,
Preflight: paths.Preflight,
@@ -316,6 +338,7 @@ func TestFindPriorSnapshotSupportsWeekend(t *testing.T) {
}
_, err = store.SaveMetadata(context.Background(), BuildMetadata(first, briefingPackage, ArtifactPaths{
Briefing: briefingPath,
ModuleSnapshot: paths.ModuleSnapshot,
Metadata: paths.Metadata,
DataPackage: paths.DataPackage,
Preflight: paths.Preflight,
@@ -352,6 +375,7 @@ func TestFindPriorSnapshotSupportsNarrowedWeekendPeriod(t *testing.T) {
}
_, err = store.SaveMetadata(context.Background(), BuildMetadata(first, briefingPackage, ArtifactPaths{
Briefing: briefingPath,
ModuleSnapshot: paths.ModuleSnapshot,
Metadata: paths.Metadata,
DataPackage: paths.DataPackage,
Preflight: paths.Preflight,
@@ -521,6 +545,7 @@ func pathsString(paths ArtifactPaths) string {
return strings.Join([]string{
paths.Briefing,
paths.Metadata,
paths.ModuleSnapshot,
paths.DataPackage,
paths.Preflight,
paths.Notification,

2
internal/state/metadata.go
View File

@@ -27,6 +27,7 @@ type Metadata struct {
Sources []briefing.SourceMetadata `json:"sources,omitempty"`
SourceWarnings []weatherdata.SourceWarning `json:"sourceWarnings,omitempty"`
BriefingPath string `json:"briefingPath"`
ModuleSnapshotPath string `json:"moduleSnapshotPath"`
DataPackagePath string `json:"dataPackagePath"`
PreflightPath string `json:"preflightPath"`
NotificationPath string `json:"notificationPath,omitempty"`
@@ -51,6 +52,7 @@ func BuildMetadata(resolved report.Resolved, briefingPackage briefing.Package, p
Sources: briefingPackage.Metadata.Sources,
SourceWarnings: briefingPackage.Metadata.SourceWarnings,
BriefingPath: paths.Briefing,
ModuleSnapshotPath: paths.ModuleSnapshot,
DataPackagePath: paths.DataPackage,
PreflightPath: paths.Preflight,
RenderedReportPath: paths.RenderedReport,

3
internal/state/store.go
View File

@@ -7,6 +7,7 @@ import (
"time"
"gitea.maximumdirect.net/eric/weatherreporter/internal/briefing"
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
)
@@ -14,6 +15,7 @@ import (
type Store interface {
Paths(report.Resolved) (ArtifactPaths, error)
SaveBriefing(context.Context, report.Resolved, briefing.Package) (string, error)
SaveModuleSnapshot(context.Context, report.Resolved, module.Snapshot) (string, error)
SaveDataPackage(context.Context, report.Resolved, promptinput.Package) (string, error)
SavePreflight(context.Context, report.Resolved, PreflightArtifact) (string, error)
SaveDistributorNotification(context.Context, report.Resolved, DistributorNotificationArtifact) (string, error)
@@ -21,6 +23,7 @@ type Store interface {
SaveMetadata(context.Context, Metadata) (string, error)
FindPriorSnapshot(context.Context, report.Resolved) (*PriorSnapshot, error)
LoadBriefing(context.Context, string) (briefing.Package, error)
LoadModuleSnapshot(context.Context, string) (module.Snapshot, error)
}
type PriorSnapshot struct {