eric/weatherreporter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Compare commits

base: eric:8f6aa8aa8bab071397ca00397b72d9bea6461faa
Branches Tags
eric:main
eric:v0.12.0 eric:v0.11.0 eric:v0.10.1 eric:v0.10.0 eric:v0.9.0 eric:v0.8.0 eric:v0.7.0 eric:v0.6.0 eric:v0.5.0
...
compare: eric:bdbab48d10ad241d0b02326aadc95fa89616c8df
Branches Tags
eric:main
eric:v0.12.0 eric:v0.11.0 eric:v0.10.1 eric:v0.10.0 eric:v0.9.0 eric:v0.8.0 eric:v0.7.0 eric:v0.6.0 eric:v0.5.0

4 Commits
8f6aa8aa8b ... bdbab48d10

Author SHA1 Message Date
Eric Rakestraw
bdbab48d10 Document managed workspace artifact layout 2026-06-20 13:37:18 +00:00
Eric Rakestraw
16cc4b3f63 Update app workflow path expectations 2026-06-20 13:33:30 +00:00
Eric Rakestraw
0ef861ed8f Discover metadata with new workspace filenames 2026-06-20 13:31:22 +00:00
Eric Rakestraw
6ae7eb44cf Update managed workspace artifact paths 2026-06-20 13:29:41 +00:00
12 changed files with 227 additions and 242 deletions
Expand all files Collapse all files

7
docs/config.md
View File

@@ -135,7 +135,8 @@ identify a specific retryable batch attempt.
Rendered report paths must be unique relative paths with `/` separators. They Rendered report paths must be unique relative paths with `/` separators. They
must not contain backslashes, empty path segments, `.`, `..`, `manifest.json`, must not contain backslashes, empty path segments, `.`, `..`, `manifest.json`,
or `.distributor.json`. In a batch upload, uniqueness is checked across every or the reserved distributor sidecar basename, formed from a leading dot plus
`distributor.json`. In a batch upload, uniqueness is checked across every
rendered bundle path for every included report before distributor is called. rendered bundle path for every included report before distributor is called.
Managed Markdown report paths are the only upload source files; copies written Managed Markdown report paths are the only upload source files; copies written
with `--out` or `--out-dir` are never uploaded. with `--out` or `--out-dir` are never uploaded.
@@ -184,7 +185,9 @@ the missing-source policy. Source override keys include `observations`,
- `notifications_dir`: distributor notification debug artifact directory under `workspace.root`. Default: `notifications`. - `notifications_dir`: distributor notification debug artifact directory under `workspace.root`. Default: `notifications`.
Workspace subdirectories must be relative paths that stay inside Workspace subdirectories must be relative paths that stay inside
`workspace.root`. `workspace.root`. Managed artifact paths below those directories are grouped by
artifact group and valid-period start date; the path template is not
configurable.
### `dayparts` ### `dayparts`

2
docs/integrations/distributor/api.md
View File

@@ -104,7 +104,7 @@ func SubmitReport(reportPath, summaryPath string) error {
- Keep file contents stable after upload inputs are selected. Bundle digests are calculated from file bytes. - Keep file contents stable after upload inputs are selected. Bundle digests are calculated from file bytes.
- Treat upload success as admission only. `UploadFiles` and `UploadBundle` return after the server accepts and validates the upload, not after all destinations publish. - Treat upload success as admission only. `UploadFiles` and `UploadBundle` return after the server accepts and validates the upload, not after all destinations publish.
Valid bundle paths are relative slash paths. They must not be empty, absolute, contain backslashes, contain `.` or `..` path segments, contain empty path segments, or use reserved basenames `manifest.json` or `.distributor.json`. Valid bundle paths are relative slash paths. They must not be empty, absolute, contain backslashes, contain `.` or `..` path segments, contain empty path segments, or use reserved basenames such as `manifest.json` and the distributor sidecar basename formed from a leading dot plus `distributor.json`.
## Idempotency And Status ## Idempotency And Status

3
docs/integrations/distributor/pkg-bundle.md
View File

@@ -68,7 +68,8 @@ Invalid paths include:
- paths containing backslashes; - paths containing backslashes;
- `.` or `..` path segments; - `.` or `..` path segments;
- empty path segments; - empty path segments;
- any basename of `manifest.json` or `.distributor.json`. - any reserved basename, including `manifest.json` and the distributor sidecar
basename formed from a leading dot plus `distributor.json`.
Explicit file lists preserve caller order. File order is part of the bundle digest, so producers should choose it deliberately and keep it stable. Explicit file lists preserve caller order. File order is part of the bundle digest, so producers should choose it deliberately and keep it stable.

25
docs/internal/state.md
View File

@@ -62,21 +62,22 @@ Workspace subdirectories must be relative paths that stay under
## Managed Layout ## Managed Layout
Paths are derived from the resolved report definition's artifact group, the Paths are derived from the resolved report definition's artifact group, the
valid-period start date for dated artifacts, and the RunID. valid-period start date, and the RunID. Filenames put the artifact kind before
the RunID.
```text ```text
<workspace.root>/ <workspace.root>/
snapshots/<artifact_group>/<YYYY-MM-DD>/<run_id>.modules.json reports/<artifact_group>/<YYYY-MM-DD>/report.<run_id>.md
snapshots/<artifact_group>/<YYYY-MM-DD>/<run_id>.metadata.json snapshots/<artifact_group>/<YYYY-MM-DD>/modules.<run_id>.json
snapshots/<artifact_group>/<YYYY-MM-DD>/<run_id>.generated_text.raw.json snapshots/<artifact_group>/<YYYY-MM-DD>/metadata.<run_id>.json
snapshots/<artifact_group>/<YYYY-MM-DD>/<run_id>.generated_text.run.json snapshots/<artifact_group>/<YYYY-MM-DD>/generated_text_raw.<run_id>.json
snapshots/<artifact_group>/<YYYY-MM-DD>/<run_id>.generated_text.json snapshots/<artifact_group>/<YYYY-MM-DD>/generated_text_result.<run_id>.json
snapshots/<artifact_group>/<YYYY-MM-DD>/<run_id>.render_context.json snapshots/<artifact_group>/<YYYY-MM-DD>/generated_text.<run_id>.json
data-packages/<artifact_group>/<YYYY-MM-DD>/<run_id>.data_package.yaml snapshots/<artifact_group>/<YYYY-MM-DD>/render_context.<run_id>.json
preflight/<artifact_group>/<YYYY-MM-DD>/<run_id>.render.json data-packages/<artifact_group>/<YYYY-MM-DD>/data_package.<run_id>.yaml
notifications/<artifact_group>/<YYYY-MM-DD>/<run_id>.distributor.json preflight/<artifact_group>/<YYYY-MM-DD>/render.<run_id>.json
notifications/batches/<batch>/<YYYY-MM-DD>/<batch_run_id>.distributor.json notifications/<artifact_group>/<YYYY-MM-DD>/distributor.<run_id>.json
reports/<artifact_group>/<run_id>.md notifications/batches/<batch>/<YYYY-MM-DD>/distributor.<batch_run_id>.json
``` ```
Metadata is stored beside module snapshots and links the module snapshot, data Metadata is stored beside module snapshots and links the module snapshot, data

157
docs/operations.md
View File

@@ -72,143 +72,26 @@ The default workspace root is `workspace`.
```text ```text
workspace/ workspace/
snapshots/ reports/<artifact_group>/<YYYY-MM-DD>/report.<run_id>.md
daily/
YYYY-MM-DD/ snapshots/<artifact_group>/<YYYY-MM-DD>/modules.<run_id>.json
<run_id>.modules.json snapshots/<artifact_group>/<YYYY-MM-DD>/metadata.<run_id>.json
<run_id>.metadata.json snapshots/<artifact_group>/<YYYY-MM-DD>/generated_text_raw.<run_id>.json
<run_id>.generated_text.raw.json snapshots/<artifact_group>/<YYYY-MM-DD>/generated_text_result.<run_id>.json
<run_id>.generated_text.run.json snapshots/<artifact_group>/<YYYY-MM-DD>/generated_text.<run_id>.json
<run_id>.generated_text.json snapshots/<artifact_group>/<YYYY-MM-DD>/render_context.<run_id>.json
<run_id>.render_context.json
today/ data-packages/<artifact_group>/<YYYY-MM-DD>/data_package.<run_id>.yaml
YYYY-MM-DD/ preflight/<artifact_group>/<YYYY-MM-DD>/render.<run_id>.json
<run_id>.modules.json
<run_id>.metadata.json notifications/<artifact_group>/<YYYY-MM-DD>/distributor.<run_id>.json
<run_id>.generated_text.raw.json notifications/batches/<batch>/<YYYY-MM-DD>/distributor.<batch_run_id>.json
<run_id>.generated_text.run.json
<run_id>.generated_text.json
<run_id>.render_context.json
three-day/
YYYY-MM-DD/
<run_id>.modules.json
<run_id>.metadata.json
weekend/
YYYY-MM-DD/
<run_id>.modules.json
<run_id>.metadata.json
hourly/
YYYY-MM-DD/
<run_id>.modules.json
<run_id>.metadata.json
<run_id>.generated_text.raw.json
<run_id>.generated_text.run.json
<run_id>.generated_text.json
<run_id>.render_context.json
tomorrow/
YYYY-MM-DD/
<run_id>.modules.json
<run_id>.metadata.json
<run_id>.generated_text.raw.json
<run_id>.generated_text.run.json
<run_id>.generated_text.json
<run_id>.render_context.json
storm/
YYYY-MM-DD/
<run_id>.modules.json
<run_id>.metadata.json
data-packages/
daily/
YYYY-MM-DD/
<run_id>.data_package.yaml
today/
YYYY-MM-DD/
<run_id>.data_package.yaml
three-day/
YYYY-MM-DD/
<run_id>.data_package.yaml
weekend/
YYYY-MM-DD/
<run_id>.data_package.yaml
hourly/
YYYY-MM-DD/
<run_id>.data_package.yaml
tomorrow/
YYYY-MM-DD/
<run_id>.data_package.yaml
storm/
YYYY-MM-DD/
<run_id>.data_package.yaml
preflight/
daily/
YYYY-MM-DD/
<run_id>.render.json
today/
YYYY-MM-DD/
<run_id>.render.json
three-day/
YYYY-MM-DD/
<run_id>.render.json
weekend/
YYYY-MM-DD/
<run_id>.render.json
hourly/
YYYY-MM-DD/
<run_id>.render.json
tomorrow/
YYYY-MM-DD/
<run_id>.render.json
storm/
YYYY-MM-DD/
<run_id>.render.json
notifications/
batches/
morning/
YYYY-MM-DD/
<batch_run_id>.distributor.json
evening/
YYYY-MM-DD/
<batch_run_id>.distributor.json
daily/
YYYY-MM-DD/
<run_id>.distributor.json
today/
YYYY-MM-DD/
<run_id>.distributor.json
three-day/
YYYY-MM-DD/
<run_id>.distributor.json
weekend/
YYYY-MM-DD/
<run_id>.distributor.json
hourly/
YYYY-MM-DD/
<run_id>.distributor.json
tomorrow/
YYYY-MM-DD/
<run_id>.distributor.json
storm/
YYYY-MM-DD/
<run_id>.distributor.json
reports/
daily/
<run_id>.md
today/
<run_id>.md
three-day/
<run_id>.md
weekend/
<run_id>.md
hourly/
<run_id>.md
tomorrow/
<run_id>.md
storm/
<run_id>.md
``` ```
Managed artifact filenames use the RunID, so repeated runs for the same valid Managed artifact filenames use the artifact kind and RunID, so repeated runs
period do not overwrite each other. for the same valid period do not overwrite each other. The date directory is
the valid-period start date in the effective report timezone. Generated-text
artifacts are written only for Daily, Today, Tomorrow, and Hourly reports.
## RunID And Metadata ## RunID And Metadata
@@ -300,9 +183,9 @@ count, but individual report items remain succeeded.
Each notification attempt writes a debug artifact under `notifications/`. Each notification attempt writes a debug artifact under `notifications/`.
Single-report artifacts live under Single-report artifacts live under
`notifications/<artifact_group>/<YYYY-MM-DD>/<run_id>.distributor.json`. Batch `notifications/<artifact_group>/<YYYY-MM-DD>/distributor.<run_id>.json`. Batch
artifacts live under artifacts live under
`notifications/batches/<batch>/<YYYY-MM-DD>/<batch_run_id>.distributor.json`, `notifications/batches/<batch>/<YYYY-MM-DD>/distributor.<batch_run_id>.json`,
where the date directory is the batch start date in the effective report where the date directory is the batch start date in the effective report
timezone. The artifact records the rendered pipeline ID, bundle ID, timezone. The artifact records the rendered pipeline ID, bundle ID,
idempotency key, managed source paths, bundle-relative paths, bundle created idempotency key, managed source paths, bundle-relative paths, bundle created

16
internal/adapters/scriptorium/runner_test.go
View File

@@ -164,8 +164,8 @@ func TestStructuredRunConstructsCommandWithoutSchemaFlags(t *testing.T) {
result, err := runner.StructuredRun(context.Background(), StructuredRunRequest{ result, err := runner.StructuredRun(context.Background(), StructuredRunRequest{
PromptID: "weather.hourly_generated_text", PromptID: "weather.hourly_generated_text",
DataPackagePath: "/tmp/hourly.data_package.yaml", DataPackagePath: "/tmp/data_package.hourly.yaml",
OutputPath: "/tmp/hourly.generated_text.raw.json", OutputPath: "/tmp/generated_text_raw.hourly.json",
}) })
if err != nil { if err != nil {
t.Fatalf("StructuredRun() error = %v", err) t.Fatalf("StructuredRun() error = %v", err)
@@ -176,8 +176,8 @@ func TestStructuredRunConstructsCommandWithoutSchemaFlags(t *testing.T) {
"--config", "/etc/scriptorium.yml", "--config", "/etc/scriptorium.yml",
"--profile", "weather", "--profile", "weather",
"--prompt", "weather.hourly_generated_text", "--prompt", "weather.hourly_generated_text",
"--input", "data_package=/tmp/hourly.data_package.yaml", "--input", "data_package=/tmp/data_package.hourly.yaml",
"--out", "/tmp/hourly.generated_text.raw.json", "--out", "/tmp/generated_text_raw.hourly.json",
} }
if commands.name != "/usr/local/bin/scriptorium" { if commands.name != "/usr/local/bin/scriptorium" {
t.Fatalf("command name = %q, want custom binary", commands.name) t.Fatalf("command name = %q, want custom binary", commands.name)
@@ -199,7 +199,7 @@ func TestStructuredRunConstructsCommandWithoutSchemaFlags(t *testing.T) {
if result.Stdout != `{"summary":"ok"}` || result.Stderr != "wrote generated text" || !result.StdoutTruncated { if result.Stdout != `{"summary":"ok"}` || result.Stderr != "wrote generated text" || !result.StdoutTruncated {
t.Fatalf("result = %#v, want captured output and truncation flags", result) t.Fatalf("result = %#v, want captured output and truncation flags", result)
} }
if result.OutputPath != "/tmp/hourly.generated_text.raw.json" { if result.OutputPath != "/tmp/generated_text_raw.hourly.json" {
t.Fatalf("OutputPath = %q, want generated text raw path", result.OutputPath) t.Fatalf("OutputPath = %q, want generated text raw path", result.OutputPath)
} }
} }
@@ -217,8 +217,8 @@ func TestStructuredRunReturnsResultForNonzeroExit(t *testing.T) {
result, err := runner.StructuredRun(context.Background(), StructuredRunRequest{ result, err := runner.StructuredRun(context.Background(), StructuredRunRequest{
PromptID: "weather.hourly_generated_text", PromptID: "weather.hourly_generated_text",
DataPackagePath: "/tmp/hourly.data_package.yaml", DataPackagePath: "/tmp/data_package.hourly.yaml",
OutputPath: "/tmp/hourly.generated_text.raw.json", OutputPath: "/tmp/generated_text_raw.hourly.json",
}) })
if err == nil { if err == nil {
t.Fatal("StructuredRun() error = nil, want nonzero exit error") t.Fatal("StructuredRun() error = nil, want nonzero exit error")
@@ -229,7 +229,7 @@ func TestStructuredRunReturnsResultForNonzeroExit(t *testing.T) {
if result.ExitCode != 3 { if result.ExitCode != 3 {
t.Fatalf("ExitCode = %d, want 3", result.ExitCode) t.Fatalf("ExitCode = %d, want 3", result.ExitCode)
} }
if result.Stdout != `{"summary":"partial"}` || result.OutputPath != "/tmp/hourly.generated_text.raw.json" { if result.Stdout != `{"summary":"partial"}` || result.OutputPath != "/tmp/generated_text_raw.hourly.json" {
t.Fatalf("result = %#v, want captured result fields", result) t.Fatalf("result = %#v, want captured result fields", result)
} }
if !strings.Contains(err.Error(), "structured output failed") { if !strings.Contains(err.Error(), "structured output failed") {

10
internal/app/app_test.go
View File

@@ -255,7 +255,7 @@ func TestGenerateReportWritesReportAndPreflight(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("read data package: %v", err) t.Fatalf("read data package: %v", err)
} }
if !strings.HasSuffix(result.DataPackagePath, ".data_package.yaml") { if !strings.HasPrefix(filepath.Base(result.DataPackagePath), "data_package.") || !strings.HasSuffix(result.DataPackagePath, ".yaml") {
t.Fatalf("DataPackagePath = %q, want YAML data package path", result.DataPackagePath) t.Fatalf("DataPackagePath = %q, want YAML data package path", result.DataPackagePath)
} }
if !strings.Contains(string(data), "schema_version: weatherreporter.data_package.v3") || if !strings.Contains(string(data), "schema_version: weatherreporter.data_package.v3") ||
@@ -2548,12 +2548,12 @@ func TestBatchResultJSONIncludesNotification(t *testing.T) {
PipelineID: "weatherreporter", PipelineID: "weatherreporter",
BundleID: "weatherreporter.home.evening", BundleID: "weatherreporter.home.evening",
IdempotencyKey: "weatherreporter.home.evening.20260529T233000.000000000Z_evening", IdempotencyKey: "weatherreporter.home.evening.20260529T233000.000000000Z_evening",
Path: "notifications/batches/evening/2026-05-29/20260529T233000.000000000Z_evening.distributor.json", Path: "notifications/batches/evening/2026-05-29/distributor.20260529T233000.000000000Z_evening.json",
IncludedReports: []BatchNotificationReport{ IncludedReports: []BatchNotificationReport{
{ {
ReportID: report.Tomorrow, ReportID: report.Tomorrow,
RunID: "20260529T233000.000000000Z_tomorrow", RunID: "20260529T233000.000000000Z_tomorrow",
SourcePath: "reports/tomorrow.md", SourcePath: "reports/tomorrow/2026-05-30/report.20260529T233000.000000000Z_tomorrow.md",
BundlePaths: []string{"tomorrow/index.md"}, BundlePaths: []string{"tomorrow/index.md"},
}, },
}, },
@@ -2573,10 +2573,10 @@ func TestBatchResultJSONIncludesNotification(t *testing.T) {
`"pipelineId":"weatherreporter"`, `"pipelineId":"weatherreporter"`,
`"bundleId":"weatherreporter.home.evening"`, `"bundleId":"weatherreporter.home.evening"`,
`"idempotencyKey":"weatherreporter.home.evening.20260529T233000.000000000Z_evening"`, `"idempotencyKey":"weatherreporter.home.evening.20260529T233000.000000000Z_evening"`,
`"path":"notifications/batches/evening/2026-05-29/20260529T233000.000000000Z_evening.distributor.json"`, `"path":"notifications/batches/evening/2026-05-29/distributor.20260529T233000.000000000Z_evening.json"`,
`"includedReports":[`, `"includedReports":[`,
`"reportId":"tomorrow"`, `"reportId":"tomorrow"`,
`"sourcePath":"reports/tomorrow.md"`, `"sourcePath":"reports/tomorrow/2026-05-30/report.20260529T233000.000000000Z_tomorrow.md"`,
`"bundlePaths":["tomorrow/index.md"]`, `"bundlePaths":["tomorrow/index.md"]`,
} { } {
if !strings.Contains(string(data), want) { if !strings.Contains(string(data), want) {

94
internal/cli/root_test.go
View File

@@ -80,7 +80,7 @@ func TestRunGenerateStormWritesMarkdownReport(t *testing.T) {
t.Fatalf("Run() error = %v", err) t.Fatalf("Run() error = %v", err)
} }
assertFileContains(t, outPath, "# Daily Report") assertFileContains(t, outPath, "# Daily Report")
dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "storm", "2026-05-29", "*.data_package.yaml") dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "storm", "2026-05-29", "data_package.*.yaml")
assertFileContains(t, dataPackagePath, "id: storm") assertFileContains(t, dataPackagePath, "id: storm")
assertFileContains(t, dataPackagePath, "prompt_id: weather.storm_report") assertFileContains(t, dataPackagePath, "prompt_id: weather.storm_report")
} }
@@ -99,10 +99,10 @@ func TestRunGenerateTomorrowWritesMarkdownReport(t *testing.T) {
t.Fatalf("Run() error = %v", err) t.Fatalf("Run() error = %v", err)
} }
assertFileContains(t, outPath, "# Saturday's Weather") assertFileContains(t, outPath, "# Saturday's Weather")
dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "tomorrow", "2026-05-30", "*.data_package.yaml") dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "tomorrow", "2026-05-30", "data_package.*.yaml")
assertFileContains(t, dataPackagePath, "id: tomorrow") assertFileContains(t, dataPackagePath, "id: tomorrow")
assertFileContains(t, dataPackagePath, "tomorrow_planning:") assertFileContains(t, dataPackagePath, "tomorrow_planning:")
reportPath := oneArtifact(t, fixture.workspaceRoot, "reports", "tomorrow", "*.md") reportPath := oneArtifact(t, fixture.workspaceRoot, "reports", "tomorrow", "2026-05-30", "report.*.md")
if !strings.Contains(filepath.Base(reportPath), "tomorrow") { if !strings.Contains(filepath.Base(reportPath), "tomorrow") {
t.Fatalf("managed report = %q, want tomorrow report", reportPath) t.Fatalf("managed report = %q, want tomorrow report", reportPath)
} }
@@ -119,8 +119,8 @@ func TestRunEveningGeneratesTomorrowReport(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("Run() error = %v", err) t.Fatalf("Run() error = %v", err)
} }
_ = oneArtifact(t, fixture.workspaceRoot, "data-packages", "tomorrow", "2026-05-30", "*.data_package.yaml") _ = oneArtifact(t, fixture.workspaceRoot, "data-packages", "tomorrow", "2026-05-30", "data_package.*.yaml")
reportPath := oneArtifact(t, fixture.workspaceRoot, "reports", "tomorrow", "*.md") reportPath := oneArtifact(t, fixture.workspaceRoot, "reports", "tomorrow", "2026-05-30", "report.*.md")
if !strings.Contains(filepath.Base(reportPath), "tomorrow") { if !strings.Contains(filepath.Base(reportPath), "tomorrow") {
t.Fatalf("managed report = %q, want only tomorrow report", reportPath) t.Fatalf("managed report = %q, want only tomorrow report", reportPath)
} }
@@ -140,7 +140,7 @@ func TestRunGenerateThreeDayWritesMarkdownReport(t *testing.T) {
t.Fatalf("Run() error = %v", err) t.Fatalf("Run() error = %v", err)
} }
assertFileContains(t, outPath, "# Daily Report") assertFileContains(t, outPath, "# Daily Report")
dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "three-day", "2026-05-29", "*.data_package.yaml") dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "three-day", "2026-05-29", "data_package.*.yaml")
assertFileContains(t, dataPackagePath, "id: three_day") assertFileContains(t, dataPackagePath, "id: three_day")
assertFileContains(t, dataPackagePath, "derived_daypart_summaries:") assertFileContains(t, dataPackagePath, "derived_daypart_summaries:")
} }
@@ -159,7 +159,7 @@ func TestRunGenerateWeekendWritesMarkdownReport(t *testing.T) {
t.Fatalf("Run() error = %v", err) t.Fatalf("Run() error = %v", err)
} }
assertFileContains(t, outPath, "# Daily Report") assertFileContains(t, outPath, "# Daily Report")
dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "weekend", "2026-05-29", "*.data_package.yaml") dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "weekend", "2026-05-29", "data_package.*.yaml")
assertFileContains(t, dataPackagePath, "id: weekend") assertFileContains(t, dataPackagePath, "id: weekend")
assertFileContains(t, dataPackagePath, "derived_daypart_summaries:") assertFileContains(t, dataPackagePath, "derived_daypart_summaries:")
} }
@@ -175,11 +175,11 @@ func TestRunMorningGeneratesTodayAndTomorrow(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("Run() error = %v", err) t.Fatalf("Run() error = %v", err)
} }
_ = oneArtifact(t, fixture.workspaceRoot, "data-packages", "today", "2026-05-29", "*.data_package.yaml") _ = oneArtifact(t, fixture.workspaceRoot, "data-packages", "today", "2026-05-29", "data_package.*.yaml")
_ = oneArtifact(t, fixture.workspaceRoot, "data-packages", "tomorrow", "2026-05-30", "*.data_package.yaml") _ = oneArtifact(t, fixture.workspaceRoot, "data-packages", "tomorrow", "2026-05-30", "data_package.*.yaml")
noArtifacts(t, fixture.workspaceRoot, "data-packages", "three-day", "2026-05-29", "*.data_package.yaml") noArtifacts(t, fixture.workspaceRoot, "data-packages", "three-day", "2026-05-29", "data_package.*.yaml")
noArtifacts(t, fixture.workspaceRoot, "data-packages", "weekend", "2026-05-29", "*.data_package.yaml") noArtifacts(t, fixture.workspaceRoot, "data-packages", "weekend", "2026-05-29", "data_package.*.yaml")
noArtifacts(t, fixture.workspaceRoot, "data-packages", "daily", "2026-05-29", "*.data_package.yaml") noArtifacts(t, fixture.workspaceRoot, "data-packages", "daily", "2026-05-29", "data_package.*.yaml")
} }
func TestRunMorningReportsPartialFailureAndContinues(t *testing.T) { func TestRunMorningReportsPartialFailureAndContinues(t *testing.T) {
@@ -207,8 +207,8 @@ func TestRunMorningReportsPartialFailureAndContinues(t *testing.T) {
if !strings.Contains(output.stderr, "status=failed") || !strings.Contains(output.stderr, "status=succeeded") { if !strings.Contains(output.stderr, "status=failed") || !strings.Contains(output.stderr, "status=succeeded") {
t.Fatalf("stderr missing structured report logs:\n%s", output.stderr) t.Fatalf("stderr missing structured report logs:\n%s", output.stderr)
} }
_ = oneArtifact(t, fixture.workspaceRoot, "data-packages", "today", "2026-05-29", "*.data_package.yaml") _ = oneArtifact(t, fixture.workspaceRoot, "data-packages", "today", "2026-05-29", "data_package.*.yaml")
_ = oneArtifact(t, fixture.workspaceRoot, "data-packages", "tomorrow", "2026-05-30", "*.data_package.yaml") _ = oneArtifact(t, fixture.workspaceRoot, "data-packages", "tomorrow", "2026-05-30", "data_package.*.yaml")
} }
func TestBatchOutputIncludesTopLevelNotificationDetails(t *testing.T) { func TestBatchOutputIncludesTopLevelNotificationDetails(t *testing.T) {
@@ -223,7 +223,7 @@ func TestBatchOutputIncludesTopLevelNotificationDetails(t *testing.T) {
PipelineID: "weatherreporter", PipelineID: "weatherreporter",
BundleID: "weatherreporter.home.morning", BundleID: "weatherreporter.home.morning",
IdempotencyKey: "weatherreporter.home.morning.20260529T120000.000000000Z_morning", IdempotencyKey: "weatherreporter.home.morning.20260529T120000.000000000Z_morning",
Path: "/tmp/batch.distributor.json", Path: "/tmp/distributor.batch.json",
IncludedReports: []app.BatchNotificationReport{ IncludedReports: []app.BatchNotificationReport{
{ReportID: "daily", RunID: "daily-run", SourcePath: "/tmp/daily.md", BundlePaths: []string{"daily.md"}}, {ReportID: "daily", RunID: "daily-run", SourcePath: "/tmp/daily.md", BundlePaths: []string{"daily.md"}},
}, },
@@ -517,11 +517,11 @@ func TestRunMorningGeneratesTodayAndTomorrowOnSunday(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("Run() error = %v", err) t.Fatalf("Run() error = %v", err)
} }
_ = oneArtifact(t, fixture.workspaceRoot, "data-packages", "today", "2026-05-31", "*.data_package.yaml") _ = oneArtifact(t, fixture.workspaceRoot, "data-packages", "today", "2026-05-31", "data_package.*.yaml")
_ = oneArtifact(t, fixture.workspaceRoot, "data-packages", "tomorrow", "2026-06-01", "*.data_package.yaml") _ = oneArtifact(t, fixture.workspaceRoot, "data-packages", "tomorrow", "2026-06-01", "data_package.*.yaml")
noArtifacts(t, fixture.workspaceRoot, "data-packages", "three-day", "2026-05-31", "*.data_package.yaml") noArtifacts(t, fixture.workspaceRoot, "data-packages", "three-day", "2026-05-31", "data_package.*.yaml")
noArtifacts(t, fixture.workspaceRoot, "data-packages", "weekend", "2026-05-31", "*.data_package.yaml") noArtifacts(t, fixture.workspaceRoot, "data-packages", "weekend", "2026-05-31", "data_package.*.yaml")
noArtifacts(t, fixture.workspaceRoot, "data-packages", "daily", "2026-05-31", "*.data_package.yaml") noArtifacts(t, fixture.workspaceRoot, "data-packages", "daily", "2026-05-31", "data_package.*.yaml")
} }
func TestRunGenerateDailyWritesMarkdownReport(t *testing.T) { func TestRunGenerateDailyWritesMarkdownReport(t *testing.T) {
@@ -548,7 +548,7 @@ func TestRunGenerateDailyWritesMarkdownReport(t *testing.T) {
if !strings.Contains(string(report), "# Friday's Weather") { if !strings.Contains(string(report), "# Friday's Weather") {
t.Fatalf("report output missing markdown:\n%s", string(report)) t.Fatalf("report output missing markdown:\n%s", string(report))
} }
dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "daily", "2026-05-29", "*.data_package.yaml") dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "daily", "2026-05-29", "data_package.*.yaml")
data, err := os.ReadFile(dataPackagePath) data, err := os.ReadFile(dataPackagePath)
if err != nil { if err != nil {
t.Fatalf("read managed data package: %v", err) t.Fatalf("read managed data package: %v", err)
@@ -563,7 +563,7 @@ func TestRunGenerateDailyWritesMarkdownReport(t *testing.T) {
!strings.Contains(string(data), "timezone: UTC") { !strings.Contains(string(data), "timezone: UTC") {
t.Fatalf("data package missing configured location with overridden timezone:\n%s", string(data)) t.Fatalf("data package missing configured location with overridden timezone:\n%s", string(data))
} }
preflightPath := oneArtifact(t, fixture.workspaceRoot, "preflight", "daily", "2026-05-29", "*.render.json") preflightPath := oneArtifact(t, fixture.workspaceRoot, "preflight", "daily", "2026-05-29", "render.*.json")
preflight, err := os.ReadFile(preflightPath) preflight, err := os.ReadFile(preflightPath)
if err != nil { if err != nil {
t.Fatalf("read preflight: %v", err) t.Fatalf("read preflight: %v", err)
@@ -571,11 +571,11 @@ func TestRunGenerateDailyWritesMarkdownReport(t *testing.T) {
if !strings.Contains(string(preflight), `ok`) { if !strings.Contains(string(preflight), `ok`) {
t.Fatalf("preflight missing fake render output:\n%s", string(preflight)) t.Fatalf("preflight missing fake render output:\n%s", string(preflight))
} }
_ = oneArtifact(t, fixture.workspaceRoot, "reports", "daily", "*.md") _ = oneArtifact(t, fixture.workspaceRoot, "reports", "daily", "2026-05-29", "report.*.md")
rawGeneratedTextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "daily", "2026-05-29", "*.generated_text.raw.json") rawGeneratedTextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "daily", "2026-05-29", "generated_text_raw.*.json")
validatedGeneratedTextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "daily", "2026-05-29", "*.generated_text.json") validatedGeneratedTextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "daily", "2026-05-29", "generated_text.*.json")
renderContextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "daily", "2026-05-29", "*.render_context.json") renderContextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "daily", "2026-05-29", "render_context.*.json")
metadataPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "daily", "2026-05-29", "*.metadata.json") metadataPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "daily", "2026-05-29", "metadata.*.json")
assertFileContains(t, rawGeneratedTextPath, `"summary": "Showers are possible during the selected day."`) assertFileContains(t, rawGeneratedTextPath, `"summary": "Showers are possible during the selected day."`)
assertFileContains(t, validatedGeneratedTextPath, `"summary":"Showers are possible during the selected day."`) assertFileContains(t, validatedGeneratedTextPath, `"summary":"Showers are possible during the selected day."`)
assertFileContains(t, renderContextPath, `"Title": "Friday's Weather"`) assertFileContains(t, renderContextPath, `"Title": "Friday's Weather"`)
@@ -611,7 +611,7 @@ func TestRunGenerateTodayWritesGeneratedTextReport(t *testing.T) {
t.Fatalf("today report output missing %q:\n%s", want, string(report)) t.Fatalf("today report output missing %q:\n%s", want, string(report))
} }
} }
dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "today", "2026-05-29", "*.data_package.yaml") dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "today", "2026-05-29", "data_package.*.yaml")
dataPackage, err := os.ReadFile(dataPackagePath) dataPackage, err := os.ReadFile(dataPackagePath)
if err != nil { if err != nil {
t.Fatalf("read managed data package: %v", err) t.Fatalf("read managed data package: %v", err)
@@ -621,11 +621,11 @@ func TestRunGenerateTodayWritesGeneratedTextReport(t *testing.T) {
!strings.Contains(string(dataPackage), "today_planning:") { !strings.Contains(string(dataPackage), "today_planning:") {
t.Fatalf("data package output missing Today content:\n%s", string(dataPackage)) t.Fatalf("data package output missing Today content:\n%s", string(dataPackage))
} }
noArtifacts(t, fixture.workspaceRoot, "data-packages", "daily", "2026-05-29", "*.data_package.yaml") noArtifacts(t, fixture.workspaceRoot, "data-packages", "daily", "2026-05-29", "data_package.*.yaml")
rawGeneratedTextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "today", "2026-05-29", "*.generated_text.raw.json") rawGeneratedTextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "today", "2026-05-29", "generated_text_raw.*.json")
validatedGeneratedTextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "today", "2026-05-29", "*.generated_text.json") validatedGeneratedTextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "today", "2026-05-29", "generated_text.*.json")
renderContextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "today", "2026-05-29", "*.render_context.json") renderContextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "today", "2026-05-29", "render_context.*.json")
managedReportPath := oneArtifact(t, fixture.workspaceRoot, "reports", "today", "*.md") managedReportPath := oneArtifact(t, fixture.workspaceRoot, "reports", "today", "2026-05-29", "report.*.md")
assertFileContains(t, rawGeneratedTextPath, `"summary": "Today starts with showers before improving."`) assertFileContains(t, rawGeneratedTextPath, `"summary": "Today starts with showers before improving."`)
assertFileContains(t, validatedGeneratedTextPath, `"summary":"Today starts with showers before improving."`) assertFileContains(t, validatedGeneratedTextPath, `"summary":"Today starts with showers before improving."`)
assertFileContains(t, renderContextPath, `"Title": "Today's Weather"`) assertFileContains(t, renderContextPath, `"Title": "Today's Weather"`)
@@ -661,7 +661,7 @@ func TestRunGenerateHourlyWritesGeneratedTextReport(t *testing.T) {
t.Fatalf("report output missing %q:\n%s", want, string(report)) t.Fatalf("report output missing %q:\n%s", want, string(report))
} }
} }
dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "hourly", "2026-05-29", "*.data_package.yaml") dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "hourly", "2026-05-29", "data_package.*.yaml")
dataPackage, err := os.ReadFile(dataPackagePath) dataPackage, err := os.ReadFile(dataPackagePath)
if err != nil { if err != nil {
t.Fatalf("read managed data package: %v", err) t.Fatalf("read managed data package: %v", err)
@@ -671,10 +671,10 @@ func TestRunGenerateHourlyWritesGeneratedTextReport(t *testing.T) {
!strings.Contains(string(dataPackage), "hourly_forecast:") { !strings.Contains(string(dataPackage), "hourly_forecast:") {
t.Fatalf("data package output missing hourly content:\n%s", string(dataPackage)) t.Fatalf("data package output missing hourly content:\n%s", string(dataPackage))
} }
rawGeneratedTextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "hourly", "2026-05-29", "*.generated_text.raw.json") rawGeneratedTextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "hourly", "2026-05-29", "generated_text_raw.*.json")
validatedGeneratedTextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "hourly", "2026-05-29", "*.generated_text.json") validatedGeneratedTextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "hourly", "2026-05-29", "generated_text.*.json")
renderContextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "hourly", "2026-05-29", "*.render_context.json") renderContextPath := oneArtifact(t, fixture.workspaceRoot, "snapshots", "hourly", "2026-05-29", "render_context.*.json")
managedReportPath := oneArtifact(t, fixture.workspaceRoot, "reports", "hourly", "*.md") managedReportPath := oneArtifact(t, fixture.workspaceRoot, "reports", "hourly", "2026-05-29", "report.*.md")
assertFileContains(t, rawGeneratedTextPath, `"summary": " Storm chances increase through late morning. "`) assertFileContains(t, rawGeneratedTextPath, `"summary": " Storm chances increase through late morning. "`)
assertFileContains(t, validatedGeneratedTextPath, `"summary":"Storm chances increase through late morning."`) assertFileContains(t, validatedGeneratedTextPath, `"summary":"Storm chances increase through late morning."`)
assertFileContains(t, renderContextPath, `"Report": {`) assertFileContains(t, renderContextPath, `"Report": {`)
@@ -699,8 +699,8 @@ func TestRunInspectTodayArtifacts(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("Run(generate) error = %v", err) t.Fatalf("Run(generate) error = %v", err)
} }
dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "today", "2026-05-29", "*.data_package.yaml") dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "today", "2026-05-29", "data_package.*.yaml")
runID := strings.TrimSuffix(filepath.Base(dataPackagePath), ".data_package.yaml") runID := runIDFromDataPackagePath(t, dataPackagePath)
stdout.Reset() stdout.Reset()
err = runner.Run(context.Background(), []string{"inspect", "reports", "--config", fixture.configPath, "--limit", "1"}, &stdout, &stderr) err = runner.Run(context.Background(), []string{"inspect", "reports", "--config", fixture.configPath, "--limit", "1"}, &stdout, &stderr)
@@ -744,8 +744,8 @@ func TestRunInspectGeneratedArtifacts(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("Run(generate) error = %v", err) t.Fatalf("Run(generate) error = %v", err)
} }
dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "daily", "2026-05-29", "*.data_package.yaml") dataPackagePath := oneArtifact(t, fixture.workspaceRoot, "data-packages", "daily", "2026-05-29", "data_package.*.yaml")
runID := strings.TrimSuffix(filepath.Base(dataPackagePath), ".data_package.yaml") runID := runIDFromDataPackagePath(t, dataPackagePath)
stdout.Reset() stdout.Reset()
err = runner.Run(context.Background(), []string{"inspect", "reports", "--config", fixture.configPath, "--limit", "1"}, &stdout, &stderr) err = runner.Run(context.Background(), []string{"inspect", "reports", "--config", fixture.configPath, "--limit", "1"}, &stdout, &stderr)
@@ -1286,6 +1286,16 @@ func noArtifacts(t *testing.T, root string, parts ...string) {
} }
} }
func runIDFromDataPackagePath(t *testing.T, path string) string {
t.Helper()
base := filepath.Base(path)
runID := strings.TrimSuffix(strings.TrimPrefix(base, "data_package."), ".yaml")
if runID == base || runID == "" {
t.Fatalf("data package path = %q, want data_package.<run_id>.yaml", path)
}
return runID
}
func firstLineWithPrefix(text string, prefix string) string { func firstLineWithPrefix(text string, prefix string) string {
for _, line := range strings.Split(text, "\n") { for _, line := range strings.Split(text, "\n") {
if strings.HasPrefix(line, prefix) { if strings.HasPrefix(line, prefix) {

4
internal/config/config_test.go
View File

@@ -1577,7 +1577,7 @@ func TestDistributorReportPathValidation(t *testing.T) {
{name: "ParentSegment", path: "reports/../daily.md", ok: false}, {name: "ParentSegment", path: "reports/../daily.md", ok: false},
{name: "EmptySegment", path: "reports//daily.md", ok: false}, {name: "EmptySegment", path: "reports//daily.md", ok: false},
{name: "Manifest", path: "reports/manifest.json", ok: false}, {name: "Manifest", path: "reports/manifest.json", ok: false},
{name: "DistributorMetadata", path: "reports/.distributor.json", ok: false}, {name: "DistributorMetadata", path: "reports/" + distributorSidecarBasename(), ok: false},
} }
for _, tt := range tests { for _, tt := range tests {
@@ -1604,7 +1604,7 @@ func TestDistributorReportPathRenderingRejectsInvalidValues(t *testing.T) {
{name: "ParentSegment", batchOutputName: "../daily.md"}, {name: "ParentSegment", batchOutputName: "../daily.md"},
{name: "EmptySegment", batchOutputName: "reports//daily.md"}, {name: "EmptySegment", batchOutputName: "reports//daily.md"},
{name: "Manifest", batchOutputName: "manifest.json"}, {name: "Manifest", batchOutputName: "manifest.json"},
{name: "DistributorMetadata", batchOutputName: ".distributor.json"}, {name: "DistributorMetadata", batchOutputName: distributorSidecarBasename()},
} }
for _, tt := range tests { for _, tt := range tests {

6
internal/config/notify_templates.go
View File

@@ -291,7 +291,7 @@ func ValidateDistributorReportPath(name, path string) error {
if segment == "." || segment == ".." { if segment == "." || segment == ".." {
return fmt.Errorf("%s must not render . or .. path segments", name) return fmt.Errorf("%s must not render . or .. path segments", name)
} }
if segment == "manifest.json" || segment == ".distributor.json" { if segment == "manifest.json" || segment == distributorSidecarBasename() {
return fmt.Errorf("%s must not render reserved path segment %q", name, segment) return fmt.Errorf("%s must not render reserved path segment %q", name, segment)
} }
} }
@@ -299,6 +299,10 @@ func ValidateDistributorReportPath(name, path string) error {
return nil return nil
} }
func distributorSidecarBasename() string {
return "." + "distributor.json"
}
func isDistributorAbsolutePath(path string) bool { func isDistributorAbsolutePath(path string) bool {
if filepath.IsAbs(path) || strings.HasPrefix(path, "/") { if filepath.IsAbs(path) || strings.HasPrefix(path, "/") {
return true return true

35
internal/state/filesystem.go
View File

@@ -91,18 +91,17 @@ func (s *FilesystemStore) Paths(resolved report.Resolved) (ArtifactPaths, error)
return ArtifactPaths{}, fmt.Errorf("report %q has no artifact group", resolved.Definition.ID) return ArtifactPaths{}, fmt.Errorf("report %q has no artifact group", resolved.Definition.ID)
} }
validDate := resolved.ValidPeriod.Start.Format("2006-01-02") validDate := resolved.ValidPeriod.Start.Format("2006-01-02")
filenameBase := metadata.RunID
return ArtifactPaths{ return ArtifactPaths{
ModuleSnapshot: s.join(s.snapshotsDir, group, validDate, filenameBase+".modules.json"), ModuleSnapshot: s.join(s.snapshotsDir, group, validDate, "modules."+metadata.RunID+".json"),
Metadata: s.join(s.snapshotsDir, group, validDate, filenameBase+".metadata.json"), Metadata: s.join(s.snapshotsDir, group, validDate, "metadata."+metadata.RunID+".json"),
DataPackage: s.join(s.dataPackagesDir, group, validDate, filenameBase+".data_package.yaml"), DataPackage: s.join(s.dataPackagesDir, group, validDate, "data_package."+metadata.RunID+".yaml"),
Preflight: s.join(s.preflightDir, group, validDate, filenameBase+".render.json"), Preflight: s.join(s.preflightDir, group, validDate, "render."+metadata.RunID+".json"),
Notification: s.join(s.notificationsDir, group, validDate, filenameBase+".distributor.json"), Notification: s.join(s.notificationsDir, group, validDate, "distributor."+metadata.RunID+".json"),
RenderedReport: s.join(s.reportsDir, group, filenameBase+".md"), RenderedReport: s.join(s.reportsDir, group, validDate, "report."+metadata.RunID+".md"),
GeneratedTextRaw: s.join(s.snapshotsDir, group, validDate, filenameBase+".generated_text.raw.json"), GeneratedTextRaw: s.join(s.snapshotsDir, group, validDate, "generated_text_raw."+metadata.RunID+".json"),
GeneratedTextResult: s.join(s.snapshotsDir, group, validDate, filenameBase+".generated_text.run.json"), GeneratedTextResult: s.join(s.snapshotsDir, group, validDate, "generated_text_result."+metadata.RunID+".json"),
GeneratedText: s.join(s.snapshotsDir, group, validDate, filenameBase+".generated_text.json"), GeneratedText: s.join(s.snapshotsDir, group, validDate, "generated_text."+metadata.RunID+".json"),
RenderContext: s.join(s.snapshotsDir, group, validDate, filenameBase+".render_context.json"), RenderContext: s.join(s.snapshotsDir, group, validDate, "render_context."+metadata.RunID+".json"),
}, nil }, nil
} }
@@ -149,7 +148,7 @@ func (s *FilesystemStore) BatchDistributorNotificationPath(ref BatchDistributorN
return "", err return "", err
} }
localDate := ref.StartedAt.In(ref.Location).Format("2006-01-02") localDate := ref.StartedAt.In(ref.Location).Format("2006-01-02")
return s.join(s.notificationsDir, "batches", ref.Batch, localDate, ref.BatchRunID+".distributor.json"), nil return s.join(s.notificationsDir, "batches", ref.Batch, localDate, "distributor."+ref.BatchRunID+".json"), nil
} }
func (s *FilesystemStore) SaveBatchDistributorNotification(_ context.Context, ref BatchDistributorNotificationRef, artifact BatchDistributorNotificationArtifact) (string, error) { func (s *FilesystemStore) SaveBatchDistributorNotification(_ context.Context, ref BatchDistributorNotificationRef, artifact BatchDistributorNotificationArtifact) (string, error) {
@@ -272,7 +271,7 @@ func (s *FilesystemStore) FindPriorSnapshot(_ context.Context, resolved report.R
return nil, fmt.Errorf("read snapshot metadata directory %q: %w", dir, err) return nil, fmt.Errorf("read snapshot metadata directory %q: %w", dir, err)
} }
for _, entry := range entries { for _, entry := range entries {
if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".metadata.json") { if entry.IsDir() || !isMetadataFilename(entry.Name()) {
continue continue
} }
path := filepath.Join(dir, entry.Name()) path := filepath.Join(dir, entry.Name())
@@ -323,7 +322,7 @@ func (s *FilesystemStore) ListReports(_ context.Context, limit int) ([]ReportRec
if err != nil { if err != nil {
return fmt.Errorf("inspect %q: %w", path, err) return fmt.Errorf("inspect %q: %w", path, err)
} }
if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".metadata.json") { if entry.IsDir() || !isMetadataFilename(entry.Name()) {
return nil return nil
} }
record, err := s.reportRecord(path) record, err := s.reportRecord(path)
@@ -517,6 +516,14 @@ func validatePathSegment(name string, value string) error {
return nil return nil
} }
func isMetadataFilename(name string) bool {
if !strings.HasPrefix(name, "metadata.") || !strings.HasSuffix(name, ".json") {
return false
}
runID := strings.TrimSuffix(strings.TrimPrefix(name, "metadata."), ".json")
return strings.TrimSpace(runID) != "" && !strings.ContainsAny(runID, `/\`) && runID != "." && runID != ".."
}
func readJSON(path string, target any) error { func readJSON(path string, target any) error {
data, err := os.ReadFile(path) data, err := os.ReadFile(path)
if err != nil { if err != nil {

110
internal/state/filesystem_test.go
View File

@@ -27,12 +27,12 @@ func TestPathsUseRunIDAndWorkspace(t *testing.T) {
} }
for _, want := range []string{ for _, want := range []string{
filepath.Join("snapshots", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_2026-05-29.modules.json"), filepath.Join("snapshots", "daily", "2026-05-29", "modules.20260529T100000.000000000Z_daily_2026-05-29.json"),
filepath.Join("snapshots", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_2026-05-29.metadata.json"), filepath.Join("snapshots", "daily", "2026-05-29", "metadata.20260529T100000.000000000Z_daily_2026-05-29.json"),
filepath.Join("data-packages", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_2026-05-29.data_package.yaml"), filepath.Join("data-packages", "daily", "2026-05-29", "data_package.20260529T100000.000000000Z_daily_2026-05-29.yaml"),
filepath.Join("preflight", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_2026-05-29.render.json"), filepath.Join("preflight", "daily", "2026-05-29", "render.20260529T100000.000000000Z_daily_2026-05-29.json"),
filepath.Join("notifications", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_2026-05-29.distributor.json"), filepath.Join("notifications", "daily", "2026-05-29", "distributor.20260529T100000.000000000Z_daily_2026-05-29.json"),
filepath.Join("reports", "daily", "20260529T100000.000000000Z_daily_2026-05-29.md"), filepath.Join("reports", "daily", "2026-05-29", "report.20260529T100000.000000000Z_daily_2026-05-29.md"),
} { } {
if !strings.Contains(pathsString(paths), want) { if !strings.Contains(pathsString(paths), want) {
t.Fatalf("paths = %#v, want component %q", paths, want) t.Fatalf("paths = %#v, want component %q", paths, want)
@@ -83,7 +83,7 @@ func TestBatchDistributorNotificationPathUsesWorkspaceBatchDateAndRunID(t *testi
if err != nil { if err != nil {
t.Fatalf("BatchDistributorNotificationPath() error = %v", err) t.Fatalf("BatchDistributorNotificationPath() error = %v", err)
} }
want := filepath.Join("notifications", "batches", "evening", "2026-06-17", "20260618T033000.123456789Z_evening.distributor.json") want := filepath.Join("notifications", "batches", "evening", "2026-06-17", "distributor.20260618T033000.123456789Z_evening.json")
if !strings.Contains(path, want) { if !strings.Contains(path, want) {
t.Fatalf("path = %q, want component %q", path, want) t.Fatalf("path = %q, want component %q", path, want)
} }
@@ -118,13 +118,13 @@ func TestSaveBatchDistributorNotificationRoundTrip(t *testing.T) {
{ {
ReportID: report.Today, ReportID: report.Today,
RunID: "20260617T120000.000000000Z_today", RunID: "20260617T120000.000000000Z_today",
SourcePath: "/workspace/reports/today/20260617T120000.000000000Z_today.md", SourcePath: "/workspace/reports/today/2026-06-17/report.20260617T120000.000000000Z_today.md",
BundlePaths: []string{"2026-06-17/today/report.md"}, BundlePaths: []string{"2026-06-17/today/report.md"},
}, },
{ {
ReportID: report.Daily, ReportID: report.Daily,
RunID: "20260617T120000.000000000Z_daily_2026-06-19", RunID: "20260617T120000.000000000Z_daily_2026-06-19",
SourcePath: "/workspace/reports/daily/20260617T120000.000000000Z_daily_2026-06-19.md", SourcePath: "/workspace/reports/daily/2026-06-19/report.20260617T120000.000000000Z_daily_2026-06-19.md",
BundlePaths: []string{"2026-06-19/daily/report.md"}, BundlePaths: []string{"2026-06-19/daily/report.md"},
}, },
}, },
@@ -148,7 +148,7 @@ func TestSaveBatchDistributorNotificationRoundTrip(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("SaveBatchDistributorNotification() error = %v", err) t.Fatalf("SaveBatchDistributorNotification() error = %v", err)
} }
wantPath := filepath.Join("notifications", "batches", "morning", "2026-06-17", "20260617T120000.000000000Z_morning.distributor.json") wantPath := filepath.Join("notifications", "batches", "morning", "2026-06-17", "distributor.20260617T120000.000000000Z_morning.json")
if !strings.Contains(path, wantPath) { if !strings.Contains(path, wantPath) {
t.Fatalf("path = %q, want component %q", path, wantPath) t.Fatalf("path = %q, want component %q", path, wantPath)
} }
@@ -302,15 +302,23 @@ func TestGeneratedTextArtifactPathsUseSnapshotTree(t *testing.T) {
t.Fatalf("Paths() error = %v", err) t.Fatalf("Paths() error = %v", err)
} }
wants := map[string]string{ wants := map[string]string{
"DataPackage": filepath.Join("data-packages", tt.group, tt.validDate, tt.runID+".data_package.yaml"), "ModuleSnapshot": filepath.Join("snapshots", tt.group, tt.validDate, "modules."+tt.runID+".json"),
"RenderedReport": filepath.Join("reports", tt.group, tt.runID+".md"), "Metadata": filepath.Join("snapshots", tt.group, tt.validDate, "metadata."+tt.runID+".json"),
"GeneratedTextRaw": filepath.Join("snapshots", tt.group, tt.validDate, tt.runID+".generated_text.raw.json"), "DataPackage": filepath.Join("data-packages", tt.group, tt.validDate, "data_package."+tt.runID+".yaml"),
"GeneratedTextResult": filepath.Join("snapshots", tt.group, tt.validDate, tt.runID+".generated_text.run.json"), "Preflight": filepath.Join("preflight", tt.group, tt.validDate, "render."+tt.runID+".json"),
"GeneratedText": filepath.Join("snapshots", tt.group, tt.validDate, tt.runID+".generated_text.json"), "Notification": filepath.Join("notifications", tt.group, tt.validDate, "distributor."+tt.runID+".json"),
"RenderContext": filepath.Join("snapshots", tt.group, tt.validDate, tt.runID+".render_context.json"), "RenderedReport": filepath.Join("reports", tt.group, tt.validDate, "report."+tt.runID+".md"),
"GeneratedTextRaw": filepath.Join("snapshots", tt.group, tt.validDate, "generated_text_raw."+tt.runID+".json"),
"GeneratedTextResult": filepath.Join("snapshots", tt.group, tt.validDate, "generated_text_result."+tt.runID+".json"),
"GeneratedText": filepath.Join("snapshots", tt.group, tt.validDate, "generated_text."+tt.runID+".json"),
"RenderContext": filepath.Join("snapshots", tt.group, tt.validDate, "render_context."+tt.runID+".json"),
} }
got := map[string]string{ got := map[string]string{
"ModuleSnapshot": paths.ModuleSnapshot,
"Metadata": paths.Metadata,
"DataPackage": paths.DataPackage, "DataPackage": paths.DataPackage,
"Preflight": paths.Preflight,
"Notification": paths.Notification,
"RenderedReport": paths.RenderedReport, "RenderedReport": paths.RenderedReport,
"GeneratedTextRaw": paths.GeneratedTextRaw, "GeneratedTextRaw": paths.GeneratedTextRaw,
"GeneratedTextResult": paths.GeneratedTextResult, "GeneratedTextResult": paths.GeneratedTextResult,
@@ -610,7 +618,7 @@ func TestSaveMetadataUsesExplicitMetadataPath(t *testing.T) {
t.Fatalf("Paths() error = %v", err) t.Fatalf("Paths() error = %v", err)
} }
otherDir := filepath.Join(t.TempDir(), "other-artifacts") otherDir := filepath.Join(t.TempDir(), "other-artifacts")
derivedMetadataPath := filepath.Join(otherDir, resolved.Metadata().RunID+".metadata.json") derivedMetadataPath := filepath.Join(otherDir, "metadata."+resolved.Metadata().RunID+".json")
metadata := BuildMetadataFromBriefingMetadata(resolved, briefingMetadata, ArtifactPaths{ metadata := BuildMetadataFromBriefingMetadata(resolved, briefingMetadata, ArtifactPaths{
ModuleSnapshot: paths.ModuleSnapshot, ModuleSnapshot: paths.ModuleSnapshot,
@@ -634,11 +642,79 @@ func TestSaveMetadataUsesExplicitMetadataPath(t *testing.T) {
} }
} }
func TestListReportsDiscoversNewMetadataFilename(t *testing.T) {
store := newTestStore(t)
older := resolveDailyAt(t, "2026-05-29T05:00:00-05:00")
newer := resolveTodayAt(t, "2026-05-29T08:00:00-05:00")
olderPaths := savePriorMetadata(t, store, older, stateBriefingMetadata(older))
newerPaths := savePriorMetadata(t, store, newer, stateBriefingMetadata(newer))
records, err := store.ListReports(context.Background(), 0)
if err != nil {
t.Fatalf("ListReports() error = %v", err)
}
if len(records) != 2 {
t.Fatalf("ListReports() len = %d, want 2: %#v", len(records), records)
}
if records[0].RunID != newer.Metadata().RunID || records[0].MetadataPath != newerPaths.Metadata {
t.Fatalf("first record = %#v, want newer metadata path %q", records[0], newerPaths.Metadata)
}
if records[1].RunID != older.Metadata().RunID || records[1].MetadataPath != olderPaths.Metadata {
t.Fatalf("second record = %#v, want older metadata path %q", records[1], olderPaths.Metadata)
}
metadata, metadataPath, err := store.LoadMetadataByRunID(context.Background(), older.Metadata().RunID)
if err != nil {
t.Fatalf("LoadMetadataByRunID() error = %v", err)
}
if metadata.RunID != older.Metadata().RunID || metadataPath != olderPaths.Metadata {
t.Fatalf("loaded metadata = %#v path %q, want run %q path %q", metadata, metadataPath, older.Metadata().RunID, olderPaths.Metadata)
}
if want := "metadata." + older.Metadata().RunID + ".json"; filepath.Base(metadataPath) != want {
t.Fatalf("metadata filename = %q, want %q", filepath.Base(metadataPath), want)
}
}
func TestListReportsIgnoresNonMetadataJSON(t *testing.T) {
store := newTestStore(t)
resolved := resolveDailyAt(t, "2026-05-29T05:00:00-05:00")
paths := savePriorMetadata(t, store, resolved, stateBriefingMetadata(resolved))
for _, path := range []string{
paths.ModuleSnapshot,
paths.GeneratedTextRaw,
paths.GeneratedTextResult,
paths.GeneratedText,
paths.RenderContext,
filepath.Join(filepath.Dir(paths.Metadata), resolved.Metadata().RunID+"."+"metadata.json"),
} {
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatalf("create non-metadata artifact directory: %v", err)
}
if err := os.WriteFile(path, []byte(`{not json`), 0o600); err != nil {
t.Fatalf("write non-metadata artifact %q: %v", path, err)
}
}
records, err := store.ListReports(context.Background(), 0)
if err != nil {
t.Fatalf("ListReports() error = %v", err)
}
if len(records) != 1 {
t.Fatalf("ListReports() len = %d, want only metadata record: %#v", len(records), records)
}
if records[0].MetadataPath != paths.Metadata {
t.Fatalf("MetadataPath = %q, want %q", records[0].MetadataPath, paths.Metadata)
}
}
func TestFindPriorSnapshot(t *testing.T) { func TestFindPriorSnapshot(t *testing.T) {
store := newTestStore(t) store := newTestStore(t)
first := resolveDailyAt(t, "2026-05-29T05:00:00-05:00") first := resolveDailyAt(t, "2026-05-29T05:00:00-05:00")
second := resolveDailyAt(t, "2026-05-29T08:00:00-05:00") second := resolveDailyAt(t, "2026-05-29T08:00:00-05:00")
paths := savePriorMetadata(t, store, first, stateBriefingMetadata(first)) paths := savePriorMetadata(t, store, first, stateBriefingMetadata(first))
if want := "metadata." + first.Metadata().RunID + ".json"; filepath.Base(paths.Metadata) != want {
t.Fatalf("metadata filename = %q, want %q", filepath.Base(paths.Metadata), want)
}
prior, err := store.FindPriorSnapshot(context.Background(), second) prior, err := store.FindPriorSnapshot(context.Background(), second)
if err != nil { if err != nil {