6 Commits

25 changed files with 1126 additions and 167 deletions

View File

@@ -101,15 +101,6 @@ contains all managed Markdown reports from that batch.
Default: `weatherreporter.{location_id}.{report_id}`. Default: `weatherreporter.{location_id}.{report_id}`.
- `idempotency_key_template`: template for single-report distributor - `idempotency_key_template`: template for single-report distributor
idempotency keys. Default: `{bundle_id}.{run_id}`. idempotency keys. Default: `{bundle_id}.{run_id}`.
- `report_path_templates`: ordered list of templates for Markdown report paths
inside distributor bundles. For `generate <report>`, each rendered path maps
to the generated report's managed Markdown source. For `run morning` and
`run evening`, the templates are rendered once per included report and each
rendered path maps that report's managed Markdown source into the one batch
bundle. Default:
```yaml
- "{valid_start_date}/{artifact_group}/{valid_start_date}-{artifact_group}-{run_id}.md"
```
- `batch.enabled`: whether batch distributor notification config is active - `batch.enabled`: whether batch distributor notification config is active
when distributor notification is enabled. Default: `true`. when distributor notification is enabled. Default: `true`.
- `batch.pipeline_id_template`: template for batch distributor pipeline IDs. - `batch.pipeline_id_template`: template for batch distributor pipeline IDs.
@@ -124,10 +115,13 @@ contains all managed Markdown reports from that batch.
Single-report templates support `location_id`, `report_id`, `run_id`, Single-report templates support `location_id`, `report_id`, `run_id`,
`artifact_group`, `batch_output_name`, `valid_start_date`, `valid_end_date`, `artifact_group`, `batch_output_name`, `valid_start_date`, `valid_end_date`,
`valid_start_time`, `valid_end_time`, `valid_start_stamp`, and `valid_start_time`, `valid_end_time`, `valid_start_stamp`, `valid_end_stamp`,
`valid_end_stamp`. Date values use `YYYY-MM-DD`, time values use `HHMM`, and and `storm_id`. Date values use `YYYY-MM-DD`, time values use `HHMM`, and
stamp values use `YYYY-MM-DDTHHMM` in the effective report timezone. stamp values use `YYYY-MM-DDTHHMM` in the effective report timezone.
`pipeline_id_template` and `idempotency_key_template` may also use `bundle_id`. `storm_id` is derived from the storm report valid period as
`{valid_start_stamp}-{valid_end_stamp}`; it renders empty for non-storm
reports. `pipeline_id_template` and `idempotency_key_template` may also use
`bundle_id`.
The rendered pipeline ID selects the configured distributor `http_upload` The rendered pipeline ID selects the configured distributor `http_upload`
workflow. The rendered bundle ID is the stable logical source identity for the workflow. The rendered bundle ID is the stable logical source identity for the
@@ -146,6 +140,18 @@ rendered bundle path for every included report before distributor is called.
Managed Markdown report paths are the only upload source files; copies written Managed Markdown report paths are the only upload source files; copies written
with `--out` or `--out-dir` are never uploaded. with `--out` or `--out-dir` are never uploaded.
Distributor bundle paths are report-specific. Weatherreporter uses
`reports.<report>.distributor.path_templates` when that override is configured;
otherwise it uses the report definition defaults:
- `hourly`: `hourly/index.md`
- `daily`: `daily/{valid_start_date}/{run_id}.md`, `daily/{valid_start_date}/index.md`
- `today`: `daily/{valid_start_date}/{run_id}.md`, `daily/{valid_start_date}/index.md`, `today/index.md`
- `tomorrow`: `daily/{valid_start_date}/{run_id}.md`, `daily/{valid_start_date}/index.md`, `tomorrow/index.md`
- `three_day`: `three-day/{valid_start_date}/{run_id}.md`, `three-day/{valid_start_date}/index.md`
- `weekend`: `weekend/{valid_start_date}/{run_id}.md`, `weekend/{valid_start_date}/index.md`
- `storm`: `storm/{storm_id}/{run_id}.md`, `storm/{storm_id}/index.md`
The upload token is read from the environment variable named by `token_env` The upload token is read from the environment variable named by `token_env`
after config loading and `secrets.directory` processing. Config files should after config loading and `secrets.directory` processing. Config files should
name the variable only; they should not contain the token value. name the variable only; they should not contain the token value.
@@ -220,12 +226,19 @@ Each report entry supports:
- `deterministic_modules`: ordered module list. Entries may be string module - `deterministic_modules`: ordered module list. Entries may be string module
IDs or objects with `id` and optional `options`. IDs or objects with `id` and optional `options`.
- `distributor.path_templates`: optional ordered distributor bundle path
templates for this report. If omitted, the report definition defaults are
used. If present, the list must contain at least one template.
Example: Example:
```yaml ```yaml
reports: reports:
daily: daily:
distributor:
path_templates:
- "daily/{valid_start_date}/{run_id}.md"
- "daily/{valid_start_date}/index.md"
deterministic_modules: deterministic_modules:
- metadata - metadata
- current_conditions - current_conditions

View File

@@ -163,10 +163,11 @@ all planned reports finish, app orchestration evaluates batch notification:
3. If any planned report failed, the batch notification result is `skipped` 3. If any planned report failed, the batch notification result is `skipped`
with reason `one or more reports failed`, and distributor is not called. with reason `one or more reports failed`, and distributor is not called.
4. If every report succeeded, app orchestration renders batch pipeline, bundle 4. If every report succeeded, app orchestration renders batch pipeline, bundle
ID, and idempotency key templates, renders `report_path_templates` for each ID, and idempotency key templates, renders report-specific distributor
included report, validates every managed source path and bundle path, checks paths for each included report, validates every managed source path and
duplicate bundle paths across the batch, calls the notifier once with a bundle path, checks duplicate bundle paths across the batch, calls the
multi-file request, and saves a batch notification debug artifact. notifier once with a multi-file request, and saves a batch notification
debug artifact.
Batch notification failure records a top-level failed notification, increments Batch notification failure records a top-level failed notification, increments
the aggregate batch failure count, and returns an aggregate batch error without the aggregate batch failure count, and returns an aggregate batch error without

View File

@@ -61,12 +61,16 @@ and bundle paths from:
- `pipeline_id_template` - `pipeline_id_template`
- `bundle_id_template` - `bundle_id_template`
- `idempotency_key_template` - `idempotency_key_template`
- `report_path_templates` - report-specific path templates
For batch uploads, the app layer renders pipeline ID, bundle ID, and For batch uploads, the app layer renders pipeline ID, bundle ID, and
idempotency key from `notify.distributor.batch.*`, renders idempotency key from `notify.distributor.batch.*`, resolves report-specific
`report_path_templates` once per included report, and passes the resulting path templates once per included report, and passes the resulting multi-file
multi-file request to this adapter. request to this adapter.
Report-specific path resolution happens entirely in the app layer. Explicit
`reports.<report>.distributor.path_templates` overrides take precedence over
report definition defaults.
The token value is read from the environment variable named by `token_env` The token value is read from the environment variable named by `token_env`
after config loading and `secrets.directory` processing. after config loading and `secrets.directory` processing.

View File

@@ -256,17 +256,16 @@ top-level `notification` object with fields such as `status`, `reason`,
Distributor notification is configured with `notify.distributor` and is Distributor notification is configured with `notify.distributor` and is
disabled by default. For `generate <report>`, weatherreporter uploads the disabled by default. For `generate <report>`, weatherreporter uploads the
managed Markdown report path recorded in the report result and metadata. That managed Markdown report path recorded in the report result and metadata. That
single source file can be mapped to one or more configured bundle paths. By single source file is mapped to report-specific bundle paths. Extra copies
default, it is mapped to one dated report path. Extra copies written by `--out` written by `--out` or `--out-dir` are operator conveniences only.
or `--out-dir` are operator conveniences only.
For `run morning` and `run evening`, per-report notification is suppressed. If For `run morning` and `run evening`, per-report notification is suppressed. If
`notify.distributor.enabled` and `notify.distributor.batch.enabled` are both `notify.distributor.enabled` and `notify.distributor.batch.enabled` are both
true, the batch uploads once after all reports finish successfully. The upload true, the batch uploads once after all reports finish successfully. The upload
contains one file mapping set per included report. Each mapping uses the contains one file mapping set per included report. Each mapping uses the
managed Markdown report as the source and renders `report_path_templates` using managed Markdown report as the source and report-specific path templates for
that report's normal template values. All rendered bundle paths across the that report. All rendered bundle paths across the batch must be unique. If any
batch must be unique. If any report fails, weatherreporter records a top-level report fails, weatherreporter records a top-level
notification status of `skipped` with reason `one or more reports failed` and notification status of `skipped` with reason `one or more reports failed` and
does not call distributor. If batch notification is disabled, run commands do does not call distributor. If batch notification is disabled, run commands do
not fall back to per-report uploads. not fall back to per-report uploads.

View File

@@ -300,15 +300,15 @@ Relevant docs: [Configuration reference](config.md),
Symptom: a batch returns nonzero with duplicate bundle path context before a Symptom: a batch returns nonzero with duplicate bundle path context before a
distributor run ID is accepted. distributor run ID is accepted.
Likely cause: `notify.distributor.report_path_templates` rendered the same Likely cause: report-specific distributor path templates rendered the same
bundle-relative path for two included reports in the same batch. bundle-relative path for two included reports in the same batch.
Diagnostic: inspect the error in stdout or stderr. The validation error Diagnostic: inspect the error in stdout or stderr. The validation error
includes the duplicate bundle path plus the report IDs, RunIDs, and managed includes the duplicate bundle path plus the report IDs, RunIDs, and managed
source paths involved. source paths involved.
Safe fix: change `notify.distributor.report_path_templates` so every report in Safe fix: configure a per-report distributor path override so every report in a
a batch renders a unique path. Include values such as `{artifact_group}`, batch renders a unique path. Include values such as `{artifact_group}`,
`{valid_start_date}`, `{batch_output_name}`, or `{run_id}` when needed. `{valid_start_date}`, `{batch_output_name}`, or `{run_id}` when needed.
Relevant docs: [Configuration reference](config.md), Relevant docs: [Configuration reference](config.md),
@@ -411,7 +411,7 @@ Diagnostic: inspect stdout JSON or stderr status lines for
`notificationError` or the top-level batch notification `error`. Confirm `notificationError` or the top-level batch notification `error`. Confirm
`notify.distributor.endpoint`, `notify.distributor.endpoint`,
`notify.distributor.pipeline_id_template`, `notify.distributor.pipeline_id_template`,
`notify.distributor.report_path_templates`, and token configuration. Token report-specific distributor paths, and token configuration. Token
values are redacted from weatherreporter errors. values are redacted from weatherreporter errors.
If the upload was accepted but destination output did not change, inspect the If the upload was accepted but destination output did not change, inspect the

View File

@@ -24,8 +24,6 @@ notify:
pipeline_id_template: "weatherreporter.{report_id}" pipeline_id_template: "weatherreporter.{report_id}"
bundle_id_template: "weatherreporter.{location_id}.{report_id}" bundle_id_template: "weatherreporter.{location_id}.{report_id}"
idempotency_key_template: "{bundle_id}.{run_id}" idempotency_key_template: "{bundle_id}.{run_id}"
report_path_templates:
- "{valid_start_date}/{artifact_group}/{valid_start_date}-{artifact_group}-{run_id}.md"
batch: batch:
enabled: true enabled: true
pipeline_id_template: "weatherreporter" pipeline_id_template: "weatherreporter"
@@ -74,6 +72,10 @@ recent_change:
reports: reports:
daily: daily:
distributor:
path_templates:
- "daily/{valid_start_date}/{run_id}.md"
- "daily/{valid_start_date}/index.md"
deterministic_modules: deterministic_modules:
- metadata - metadata
- current_conditions - current_conditions

View File

@@ -934,7 +934,7 @@ func buildNotificationRequest(cfg config.Config, resolved report.Resolved, repor
if err != nil { if err != nil {
return NotificationRequest{}, err return NotificationRequest{}, err
} }
bundlePaths, err := config.RenderDistributorReportPaths(cfg.Notify.Distributor.ReportPathTemplates, values) bundlePaths, err := renderDistributorReportBundlePaths(cfg, resolved, metadata.RunID, reportPath, values)
if err != nil { if err != nil {
return NotificationRequest{}, err return NotificationRequest{}, err
} }
@@ -964,9 +964,45 @@ func distributorTemplateValuesForReport(cfg config.Config, resolved report.Resol
if err := addDistributorValidPeriodValues(&values, resolved.ValidPeriod, cfg.WeatherAPI.Timezone); err != nil { if err := addDistributorValidPeriodValues(&values, resolved.ValidPeriod, cfg.WeatherAPI.Timezone); err != nil {
return config.DistributorTemplateValues{}, err return config.DistributorTemplateValues{}, err
} }
if resolved.Definition.ID == report.Storm {
values.StormID = values.ValidStartStamp + "-" + values.ValidEndStamp
}
return values, nil return values, nil
} }
func renderDistributorReportBundlePaths(cfg config.Config, resolved report.Resolved, runID string, sourcePath string, values config.DistributorTemplateValues) ([]string, error) {
templates, name, err := distributorPathTemplatesForReport(cfg, resolved.Definition)
if err != nil {
return nil, distributorReportPathError(resolved.Definition.ID, runID, sourcePath, err)
}
paths, err := config.RenderDistributorReportPaths(name, templates, values)
if err != nil {
return nil, distributorReportPathError(resolved.Definition.ID, runID, sourcePath, err)
}
return paths, nil
}
func distributorPathTemplatesForReport(cfg config.Config, definition report.Definition) ([]string, string, error) {
overrides, err := cfg.ReportDistributorPathOverrides()
if err != nil {
return nil, "", err
}
if templates, ok := overrides[definition.ID]; ok {
return append([]string(nil), templates...), fmt.Sprintf("reports.%s.distributor.path_templates", definition.ID), nil
}
if len(definition.DistributorPathTemplates) > 0 {
return append([]string(nil), definition.DistributorPathTemplates...), fmt.Sprintf("report.%s.distributor_path_templates", definition.ID), nil
}
return nil, "", fmt.Errorf("no distributor path templates configured")
}
func distributorReportPathError(id report.ID, runID string, sourcePath string, err error) error {
if sourcePath != "" {
return fmt.Errorf("report %q run %q source path %q: %w", id, runID, sourcePath, err)
}
return fmt.Errorf("report %q run %q: %w", id, runID, err)
}
func addDistributorValidPeriodValues(values *config.DistributorTemplateValues, period timeutil.Period, timezone string) error { func addDistributorValidPeriodValues(values *config.DistributorTemplateValues, period timeutil.Period, timezone string) error {
location, err := timeutil.LoadLocation(timezone) location, err := timeutil.LoadLocation(timezone)
if err != nil { if err != nil {

View File

@@ -739,7 +739,7 @@ func TestGenerateHourlyReportCopiesOutputAndNotifiesManagedReport(t *testing.T)
if req.ReportPath == outputPath { if req.ReportPath == outputPath {
t.Fatalf("notification used output copy %q, want managed report path", outputPath) t.Fatalf("notification used output copy %q, want managed report path", outputPath)
} }
wantBundlePaths := []string{"2026-05-29/hourly/hourly.md"} wantBundlePaths := []string{"hourly/index.md"}
if strings.Join(req.BundlePaths, "\n") != strings.Join(wantBundlePaths, "\n") { if strings.Join(req.BundlePaths, "\n") != strings.Join(wantBundlePaths, "\n") {
t.Fatalf("notification BundlePaths = %#v, want %#v", req.BundlePaths, wantBundlePaths) t.Fatalf("notification BundlePaths = %#v, want %#v", req.BundlePaths, wantBundlePaths)
} }
@@ -778,7 +778,6 @@ func TestGenerateTodayReportCopiesOutputAndNotifiesTodayTemplateValues(t *testin
cfg.Notify.Distributor.PipelineIDTemplate = "weatherreporter.{report_id}.{artifact_group}" cfg.Notify.Distributor.PipelineIDTemplate = "weatherreporter.{report_id}.{artifact_group}"
cfg.Notify.Distributor.BundleIDTemplate = "{artifact_group}.{batch_output_name}.{report_id}" cfg.Notify.Distributor.BundleIDTemplate = "{artifact_group}.{batch_output_name}.{report_id}"
cfg.Notify.Distributor.IdempotencyKeyTemplate = "{bundle_id}.{run_id}" cfg.Notify.Distributor.IdempotencyKeyTemplate = "{bundle_id}.{run_id}"
cfg.Notify.Distributor.ReportPathTemplates = []string{"{valid_start_date}/{artifact_group}/{batch_output_name}"}
resolved := resolveGenerateForTest(t, cfg, GenerateRequest{ resolved := resolveGenerateForTest(t, cfg, GenerateRequest{
Report: ReportToday, Report: ReportToday,
Date: mustParse("2026-05-29T12:00:00-05:00"), Date: mustParse("2026-05-29T12:00:00-05:00"),
@@ -834,7 +833,11 @@ func TestGenerateTodayReportCopiesOutputAndNotifiesTodayTemplateValues(t *testin
if req.BundleID != "today.today.md.today" { if req.BundleID != "today.today.md.today" {
t.Fatalf("BundleID = %q, want Today artifact group, output name, and report id", req.BundleID) t.Fatalf("BundleID = %q, want Today artifact group, output name, and report id", req.BundleID)
} }
wantBundlePaths := []string{"2026-05-29/today/today.md"} wantBundlePaths := []string{
"daily/2026-05-29/" + result.Metadata.RunID + ".md",
"daily/2026-05-29/index.md",
"today/index.md",
}
if strings.Join(req.BundlePaths, "\n") != strings.Join(wantBundlePaths, "\n") { if strings.Join(req.BundlePaths, "\n") != strings.Join(wantBundlePaths, "\n") {
t.Fatalf("BundlePaths = %#v, want %#v", req.BundlePaths, wantBundlePaths) t.Fatalf("BundlePaths = %#v, want %#v", req.BundlePaths, wantBundlePaths)
} }
@@ -981,7 +984,6 @@ func TestGenerateTomorrowReportNotificationUsesTomorrowTemplateValues(t *testing
cfg.Notify.Distributor.PipelineIDTemplate = "weatherreporter.{report_id}.{artifact_group}" cfg.Notify.Distributor.PipelineIDTemplate = "weatherreporter.{report_id}.{artifact_group}"
cfg.Notify.Distributor.BundleIDTemplate = "{artifact_group}.{batch_output_name}.{report_id}" cfg.Notify.Distributor.BundleIDTemplate = "{artifact_group}.{batch_output_name}.{report_id}"
cfg.Notify.Distributor.IdempotencyKeyTemplate = "{bundle_id}.{run_id}" cfg.Notify.Distributor.IdempotencyKeyTemplate = "{bundle_id}.{run_id}"
cfg.Notify.Distributor.ReportPathTemplates = []string{"{valid_start_date}/{artifact_group}/{batch_output_name}"}
resolved := resolveGenerateForTest(t, cfg, GenerateRequest{ resolved := resolveGenerateForTest(t, cfg, GenerateRequest{
Report: ReportTomorrow, Report: ReportTomorrow,
}, "2026-05-29T18:00:00-05:00") }, "2026-05-29T18:00:00-05:00")
@@ -1011,7 +1013,11 @@ func TestGenerateTomorrowReportNotificationUsesTomorrowTemplateValues(t *testing
if req.BundleID != "tomorrow.tomorrow.md.tomorrow" { if req.BundleID != "tomorrow.tomorrow.md.tomorrow" {
t.Fatalf("BundleID = %q, want artifact group, batch output name, and report id", req.BundleID) t.Fatalf("BundleID = %q, want artifact group, batch output name, and report id", req.BundleID)
} }
wantBundlePaths := []string{"2026-05-30/tomorrow/tomorrow.md"} wantBundlePaths := []string{
"daily/2026-05-30/" + result.Metadata.RunID + ".md",
"daily/2026-05-30/index.md",
"tomorrow/index.md",
}
if strings.Join(req.BundlePaths, "\n") != strings.Join(wantBundlePaths, "\n") { if strings.Join(req.BundlePaths, "\n") != strings.Join(wantBundlePaths, "\n") {
t.Fatalf("BundlePaths = %#v, want %#v", req.BundlePaths, wantBundlePaths) t.Fatalf("BundlePaths = %#v, want %#v", req.BundlePaths, wantBundlePaths)
} }
@@ -1234,7 +1240,8 @@ func TestGenerateReportNotifiesManagedReportPath(t *testing.T) {
t.Fatalf("decode notification artifact: %v", err) t.Fatalf("decode notification artifact: %v", err)
} }
wantBundlePaths := []string{ wantBundlePaths := []string{
"2026-05-29/daily/2026-05-29-daily-" + result.Metadata.RunID + ".md", "daily/2026-05-29/" + result.Metadata.RunID + ".md",
"daily/2026-05-29/index.md",
} }
if notificationArtifact.PipelineID != "weatherreporter.daily" || strings.Join(notificationArtifact.BundlePaths, "\n") != strings.Join(wantBundlePaths, "\n") || notificationArtifact.BundleCreated.IsZero() || notificationArtifact.RunStatus == nil || !strings.Contains(string(notificationArtifact.RunStatus.Report), "replace_older") { if notificationArtifact.PipelineID != "weatherreporter.daily" || strings.Join(notificationArtifact.BundlePaths, "\n") != strings.Join(wantBundlePaths, "\n") || notificationArtifact.BundleCreated.IsZero() || notificationArtifact.RunStatus == nil || !strings.Contains(string(notificationArtifact.RunStatus.Report), "replace_older") {
t.Fatalf("notification artifact = %#v, want requested pipeline, status report, and created timestamp", notificationArtifact) t.Fatalf("notification artifact = %#v, want requested pipeline, status report, and created timestamp", notificationArtifact)
@@ -2214,6 +2221,254 @@ func TestResolveGenerateStorm(t *testing.T) {
} }
} }
func TestDistributorTemplateValuesDeriveStormID(t *testing.T) {
cfg := config.Defaults()
cfg.WeatherAPI.Timezone = "America/Chicago"
now := mustParse("2026-05-29T12:00:00-05:00")
start := mustParse("2026-05-29T18:00:00-05:00")
end := mustParse("2026-05-30T06:00:00-05:00")
resolved, err := ResolveGenerate(GenerateRequest{
Config: cfg,
Report: ReportStorm,
StormStart: start,
StormEnd: end,
}, now)
if err != nil {
t.Fatalf("ResolveGenerate() error = %v", err)
}
values, err := distributorTemplateValuesForReport(cfg, resolved, "run", "")
if err != nil {
t.Fatalf("distributorTemplateValuesForReport() error = %v", err)
}
if values.StormID != "2026-05-29T1800-2026-05-30T0600" {
t.Fatalf("StormID = %q, want storm valid-period stamp", values.StormID)
}
}
func TestDistributorTemplateValuesLeaveStormIDEmptyForOtherReports(t *testing.T) {
cfg := config.Defaults()
cfg.WeatherAPI.Timezone = "America/Chicago"
location, err := timeutil.LoadLocation(cfg.WeatherAPI.Timezone)
if err != nil {
t.Fatalf("load location: %v", err)
}
resolved, err := report.Resolve(report.Today, report.ResolveRequest{
Now: mustParse("2026-05-29T12:00:00-05:00"),
Location: location,
})
if err != nil {
t.Fatalf("Resolve() error = %v", err)
}
values, err := distributorTemplateValuesForReport(cfg, resolved, "run", "")
if err != nil {
t.Fatalf("distributorTemplateValuesForReport() error = %v", err)
}
if values.StormID != "" {
t.Fatalf("StormID = %q, want empty for %s", values.StormID, resolved.Definition.ID)
}
}
func TestBuildNotificationRequestUsesReportDefaultBundlePaths(t *testing.T) {
cfg := config.Defaults()
cfg.Location.ID = "home"
cfg.WeatherAPI.Timezone = "America/Chicago"
cfg.Notify.Distributor.PipelineIDTemplate = "weatherreporter.{report_id}"
location := mustLoadTestLocation(t, cfg.WeatherAPI.Timezone)
now := mustParse("2026-05-29T12:00:00-05:00")
registry := report.DefaultRegistry()
tests := []struct {
id report.ID
req report.ResolveRequest
want func(state.Metadata) []string
source string
}{
{
id: report.Hourly,
req: report.ResolveRequest{
Now: now,
Location: location,
},
want: func(metadata state.Metadata) []string {
return []string{"hourly/index.md"}
},
source: "/managed/hourly.md",
},
{
id: report.Daily,
req: report.ResolveRequest{
Now: now,
Location: location,
Date: mustParse("2026-05-29T12:00:00-05:00"),
},
want: func(metadata state.Metadata) []string {
return []string{
"daily/2026-05-29/" + metadata.RunID + ".md",
"daily/2026-05-29/index.md",
}
},
source: "/managed/daily.md",
},
{
id: report.Today,
req: report.ResolveRequest{
Now: now,
Location: location,
},
want: func(metadata state.Metadata) []string {
return []string{
"daily/2026-05-29/" + metadata.RunID + ".md",
"daily/2026-05-29/index.md",
"today/index.md",
}
},
source: "/managed/today.md",
},
{
id: report.Tomorrow,
req: report.ResolveRequest{
Now: now,
Location: location,
},
want: func(metadata state.Metadata) []string {
return []string{
"daily/2026-05-30/" + metadata.RunID + ".md",
"daily/2026-05-30/index.md",
"tomorrow/index.md",
}
},
source: "/managed/tomorrow.md",
},
{
id: report.ThreeDay,
req: report.ResolveRequest{
Now: now,
Location: location,
},
want: func(metadata state.Metadata) []string {
return []string{
"three-day/2026-05-29/" + metadata.RunID + ".md",
"three-day/2026-05-29/index.md",
}
},
source: "/managed/three-day.md",
},
{
id: report.Weekend,
req: report.ResolveRequest{
Now: mustParse("2026-05-29T05:00:00-05:00"),
Location: location,
},
want: func(metadata state.Metadata) []string {
return []string{
"weekend/2026-05-29/" + metadata.RunID + ".md",
"weekend/2026-05-29/index.md",
}
},
source: "/managed/weekend.md",
},
{
id: report.Storm,
req: report.ResolveRequest{
Now: now,
Location: location,
StormStart: mustParse("2026-05-29T18:00:00-05:00"),
StormEnd: mustParse("2026-05-30T06:00:00-05:00"),
},
want: func(metadata state.Metadata) []string {
return []string{
"storm/2026-05-29T1800-2026-05-30T0600/" + metadata.RunID + ".md",
"storm/2026-05-29T1800-2026-05-30T0600/index.md",
}
},
source: "/managed/storm.md",
},
}
for _, tt := range tests {
t.Run(string(tt.id), func(t *testing.T) {
resolved, err := registry.Resolve(tt.id, tt.req)
if err != nil {
t.Fatalf("Resolve() error = %v", err)
}
metadata := notificationMetadataForTest(resolved)
req, err := buildNotificationRequest(cfg, resolved, tt.source, metadata)
if err != nil {
t.Fatalf("buildNotificationRequest() error = %v", err)
}
want := tt.want(metadata)
if strings.Join(req.BundlePaths, "\n") != strings.Join(want, "\n") {
t.Fatalf("BundlePaths = %#v, want %#v", req.BundlePaths, want)
}
if req.ReportPath != tt.source {
t.Fatalf("ReportPath = %q, want %q", req.ReportPath, tt.source)
}
})
}
}
func TestBuildNotificationRequestUsesReportDistributorPathOverride(t *testing.T) {
cfg := config.Defaults()
cfg.Location.ID = "home"
cfg.WeatherAPI.Timezone = "America/Chicago"
cfg.Notify.Distributor.PipelineIDTemplate = "weatherreporter.{report_id}"
applyReportDistributorPathOverrides(t, &cfg, `
reports:
daily:
distributor:
path_templates:
- "custom/{report_id}/{run_id}.md"
`)
location := mustLoadTestLocation(t, cfg.WeatherAPI.Timezone)
resolved, err := report.Resolve(report.Daily, report.ResolveRequest{
Now: mustParse("2026-05-29T12:00:00-05:00"),
Location: location,
Date: mustParse("2026-05-29T12:00:00-05:00"),
})
if err != nil {
t.Fatalf("Resolve() error = %v", err)
}
metadata := notificationMetadataForTest(resolved)
req, err := buildNotificationRequest(cfg, resolved, "/managed/daily.md", metadata)
if err != nil {
t.Fatalf("buildNotificationRequest() error = %v", err)
}
want := []string{"custom/daily/" + metadata.RunID + ".md"}
if strings.Join(req.BundlePaths, "\n") != strings.Join(want, "\n") {
t.Fatalf("BundlePaths = %#v, want override %#v", req.BundlePaths, want)
}
}
func TestBuildNotificationRequestRequiresReportBundlePaths(t *testing.T) {
cfg := config.Defaults()
cfg.Location.ID = "home"
cfg.WeatherAPI.Timezone = "America/Chicago"
cfg.Notify.Distributor.PipelineIDTemplate = "weatherreporter.{report_id}"
location := mustLoadTestLocation(t, cfg.WeatherAPI.Timezone)
resolved, err := report.Resolve(report.Hourly, report.ResolveRequest{
Now: mustParse("2026-05-29T12:00:00-05:00"),
Location: location,
})
if err != nil {
t.Fatalf("Resolve() error = %v", err)
}
resolved.Definition.DistributorPathTemplates = nil
metadata := notificationMetadataForTest(resolved)
_, err = buildNotificationRequest(cfg, resolved, "/managed/hourly.md", metadata)
if err == nil {
t.Fatal("buildNotificationRequest() error = nil, want missing path templates error")
}
for _, want := range []string{`report "hourly"`, metadata.RunID, `/managed/hourly.md`, "no distributor path templates"} {
if !strings.Contains(err.Error(), want) {
t.Fatalf("error = %q, want %q", err.Error(), want)
}
}
}
func TestBatchRunIDUsesUTCStartAndBatchName(t *testing.T) { func TestBatchRunIDUsesUTCStartAndBatchName(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
@@ -2333,10 +2588,6 @@ func TestBatchResultJSONIncludesNotification(t *testing.T) {
func TestBuildBatchNotificationRequestIncludesEveningReports(t *testing.T) { func TestBuildBatchNotificationRequestIncludesEveningReports(t *testing.T) {
server := dailyBundleServer(t) server := dailyBundleServer(t)
cfg := dailyNotificationConfig(t, server) cfg := dailyNotificationConfig(t, server)
cfg.Notify.Distributor.ReportPathTemplates = []string{
"archive/{valid_start_date}/{artifact_group}/{run_id}.md",
"latest/{batch_output_name}",
}
cfg.Notify.Distributor.Batch.PipelineIDTemplate = "weatherreporter.{batch}.{batch_started_date}" cfg.Notify.Distributor.Batch.PipelineIDTemplate = "weatherreporter.{batch}.{batch_started_date}"
cfg.Notify.Distributor.Batch.BundleIDTemplate = "weatherreporter.{location_id}.{batch}" cfg.Notify.Distributor.Batch.BundleIDTemplate = "weatherreporter.{location_id}.{batch}"
cfg.Notify.Distributor.Batch.IdempotencyKeyTemplate = "{bundle_id}.{batch_run_id}" cfg.Notify.Distributor.Batch.IdempotencyKeyTemplate = "{bundle_id}.{batch_run_id}"
@@ -2368,20 +2619,26 @@ func TestBuildBatchNotificationRequestIncludesEveningReports(t *testing.T) {
if len(req.IncludedReports) != 3 { if len(req.IncludedReports) != 3 {
t.Fatalf("IncludedReports = %d, want 3", len(req.IncludedReports)) t.Fatalf("IncludedReports = %d, want 3", len(req.IncludedReports))
} }
if len(req.Files) != 6 { if len(req.Files) != 7 {
t.Fatalf("Files = %d, want two mappings per report", len(req.Files)) t.Fatalf("Files = %d, want report-specific mappings", len(req.Files))
} }
wantBundlePaths := map[string]struct{}{} wantBundlePaths := map[string]struct{}{}
for _, plannedReport := range planned { for _, plannedReport := range planned {
resolved := plannedReport.Resolved resolved := plannedReport.Resolved
runID := resolved.Metadata().RunID runID := resolved.Metadata().RunID
outputName := plannedReport.OutputCopyName validStart := resolved.ValidPeriod.Start.In(mustLoadTestLocation(t, cfg.WeatherAPI.Timezone)).Format(timeutil.DateLayout)
if outputName == "" { switch resolved.Definition.ID {
outputName = resolved.Definition.BatchOutputName case report.Tomorrow:
wantBundlePaths["daily/"+validStart+"/"+runID+".md"] = struct{}{}
wantBundlePaths["daily/"+validStart+"/index.md"] = struct{}{}
wantBundlePaths["tomorrow/index.md"] = struct{}{}
case report.Daily:
wantBundlePaths["daily/"+validStart+"/"+runID+".md"] = struct{}{}
wantBundlePaths["daily/"+validStart+"/index.md"] = struct{}{}
default:
t.Fatalf("unexpected planned report %s", resolved.Definition.ID)
} }
wantBundlePaths[fmt.Sprintf("archive/%s/%s/%s.md", resolved.ValidPeriod.Start.In(mustLoadTestLocation(t, cfg.WeatherAPI.Timezone)).Format(timeutil.DateLayout), resolved.Definition.ArtifactGroup, runID)] = struct{}{}
wantBundlePaths["latest/"+outputName] = struct{}{}
} }
gotBundlePaths := map[string]struct{}{} gotBundlePaths := map[string]struct{}{}
gotSourcePaths := map[string]struct{}{} gotSourcePaths := map[string]struct{}{}
@@ -2409,10 +2666,67 @@ func TestBuildBatchNotificationRequestIncludesEveningReports(t *testing.T) {
} }
} }
func TestBuildBatchNotificationRequestUsesReportOverridesAndDefaults(t *testing.T) {
server := dailyBundleServer(t)
cfg := dailyNotificationConfig(t, server)
applyReportDistributorPathOverrides(t, &cfg, `
reports:
daily:
distributor:
path_templates:
- "custom-daily/{valid_start_date}/{run_id}.md"
`)
startedAt := mustParse("2026-05-29T18:00:00-05:00")
planned, reports := plannedBatchNotificationReports(t, cfg, BatchEvening, startedAt, "2026-05-31")
runID := batchRunID(startedAt, BatchEvening)
req, err := buildBatchNotificationRequest(cfg, BatchEvening, runID, startedAt, reports, planned)
if err != nil {
t.Fatalf("buildBatchNotificationRequest() error = %v", err)
}
gotBundlePaths := map[string]struct{}{}
for _, file := range req.Files {
gotBundlePaths[file.BundlePath] = struct{}{}
}
for _, plannedReport := range planned {
reportRunID := plannedReport.Resolved.Metadata().RunID
switch plannedReport.Resolved.Definition.ID {
case report.Tomorrow:
for _, want := range []string{
"daily/2026-05-30/" + reportRunID + ".md",
"daily/2026-05-30/index.md",
"tomorrow/index.md",
} {
if _, ok := gotBundlePaths[want]; !ok {
t.Fatalf("bundle paths = %#v, missing default path %q", gotBundlePaths, want)
}
}
case report.Daily:
want := "custom-daily/2026-05-31/" + reportRunID + ".md"
if _, ok := gotBundlePaths[want]; !ok {
t.Fatalf("bundle paths = %#v, missing override path %q", gotBundlePaths, want)
}
if _, ok := gotBundlePaths["daily/2026-05-31/index.md"]; ok {
t.Fatalf("bundle paths = %#v, want daily defaults replaced by override", gotBundlePaths)
}
}
}
}
func TestBuildBatchNotificationRequestRejectsDuplicateBundlePaths(t *testing.T) { func TestBuildBatchNotificationRequestRejectsDuplicateBundlePaths(t *testing.T) {
server := dailyBundleServer(t) server := dailyBundleServer(t)
cfg := dailyNotificationConfig(t, server) cfg := dailyNotificationConfig(t, server)
cfg.Notify.Distributor.ReportPathTemplates = []string{"index.md"} applyReportDistributorPathOverrides(t, &cfg, `
reports:
tomorrow:
distributor:
path_templates:
- "index.md"
daily:
distributor:
path_templates:
- "index.md"
`)
startedAt := mustParse("2026-05-29T18:00:00-05:00") startedAt := mustParse("2026-05-29T18:00:00-05:00")
planned, reports := plannedBatchNotificationReports(t, cfg, BatchEvening, startedAt, "2026-05-31") planned, reports := plannedBatchNotificationReports(t, cfg, BatchEvening, startedAt, "2026-05-31")
@@ -2537,16 +2851,13 @@ func TestRunBatchMorningSendsOneBatchNotification(t *testing.T) {
if req.Batch != BatchMorning || req.RunID != "20260529T100000.000000000Z_morning" { if req.Batch != BatchMorning || req.RunID != "20260529T100000.000000000Z_morning" {
t.Fatalf("batch request identity = %s/%s, want morning run id", req.Batch, req.RunID) t.Fatalf("batch request identity = %s/%s, want morning run id", req.Batch, req.RunID)
} }
if len(req.IncludedReports) != 3 || len(req.Files) != 3 { if len(req.IncludedReports) != 3 || len(req.Files) != 8 {
t.Fatalf("batch request reports/files = %d/%d, want 3/3", len(req.IncludedReports), len(req.Files)) t.Fatalf("batch request reports/files = %d/%d, want 3/8", len(req.IncludedReports), len(req.Files))
} }
for _, file := range req.Files { for _, file := range req.Files {
if file.SourcePath == "" || file.BundlePath == "" { if file.SourcePath == "" || file.BundlePath == "" {
t.Fatalf("batch file = %#v, want source and bundle path", file) t.Fatalf("batch file = %#v, want source and bundle path", file)
} }
if !strings.Contains(file.BundlePath, file.RunID) {
t.Fatalf("bundle path %q does not include report run id %q", file.BundlePath, file.RunID)
}
} }
if result.Notification == nil || result.Notification.Status != "succeeded" || result.Notification.RunID != "batch-distributor-run" || result.Notification.Path == "" { if result.Notification == nil || result.Notification.Status != "succeeded" || result.Notification.RunID != "batch-distributor-run" || result.Notification.Path == "" {
t.Fatalf("batch notification = %#v, want succeeded result with artifact path", result.Notification) t.Fatalf("batch notification = %#v, want succeeded result with artifact path", result.Notification)
@@ -2998,6 +3309,36 @@ func dailyNotificationConfig(t *testing.T, server *httptest.Server) config.Confi
return cfg return cfg
} }
func notificationMetadataForTest(resolved report.Resolved) state.Metadata {
metadata := resolved.Metadata()
return state.Metadata{
RunID: metadata.RunID,
ReportID: metadata.ReportID,
PromptID: metadata.PromptID,
GeneratedAt: metadata.GeneratedAt,
Timezone: metadata.Timezone,
ValidPeriod: metadata.ValidPeriod,
}
}
func applyReportDistributorPathOverrides(t *testing.T, cfg *config.Config, data string) {
t.Helper()
path := filepath.Join(t.TempDir(), "config.yml")
if err := os.WriteFile(path, []byte(data), 0o600); err != nil {
t.Fatalf("write config fixture: %v", err)
}
loaded, err := config.LoadFile(path)
if err != nil {
t.Fatalf("LoadFile() error = %v", err)
}
if cfg.Reports == nil {
cfg.Reports = map[string]config.ReportConfig{}
}
for key, reportCfg := range loaded.Reports {
cfg.Reports[key] = reportCfg
}
}
func plannedBatchNotificationReports(t *testing.T, cfg config.Config, batch BatchKind, now time.Time, futureDailyDates ...string) ([]plannedBatchReport, []BatchReportResult) { func plannedBatchNotificationReports(t *testing.T, cfg config.Config, batch BatchKind, now time.Time, futureDailyDates ...string) ([]plannedBatchReport, []BatchReportResult) {
t.Helper() t.Helper()
collection := collectionWithFutureDailyForTest(t, cfg, futureDailyDates...) collection := collectionWithFutureDailyForTest(t, cfg, futureDailyDates...)
@@ -3140,7 +3481,6 @@ func applyHourlyGeneratedTextSettings(cfg *config.Config, t *testing.T, server *
cfg.Notify.Distributor.PipelineIDTemplate = "weatherreporter.{artifact_group}" cfg.Notify.Distributor.PipelineIDTemplate = "weatherreporter.{artifact_group}"
cfg.Notify.Distributor.BundleIDTemplate = "weatherreporter.{location_id}.{report_id}" cfg.Notify.Distributor.BundleIDTemplate = "weatherreporter.{location_id}.{report_id}"
cfg.Notify.Distributor.IdempotencyKeyTemplate = "weatherreporter.{location_id}.{report_id}.{run_id}" cfg.Notify.Distributor.IdempotencyKeyTemplate = "weatherreporter.{location_id}.{report_id}.{run_id}"
cfg.Notify.Distributor.ReportPathTemplates = []string{"{valid_start_date}/{artifact_group}/{batch_output_name}"}
} }
func resolveHourlyGeneratedTextFixture(t *testing.T, cfg config.Config) (report.Resolved, *recordingStore, *recordingNotifier, string) { func resolveHourlyGeneratedTextFixture(t *testing.T, cfg config.Config) (report.Resolved, *recordingStore, *recordingNotifier, string) {

View File

@@ -161,9 +161,9 @@ func buildBatchNotificationRequest(cfg config.Config, batch BatchKind, runID str
if err != nil { if err != nil {
return batchNotificationRequest{}, fmt.Errorf("batch notification report %q run %q source path %q: %w", item.ReportID, item.RunID, item.ReportPath, err) return batchNotificationRequest{}, fmt.Errorf("batch notification report %q run %q source path %q: %w", item.ReportID, item.RunID, item.ReportPath, err)
} }
bundlePaths, err := config.RenderDistributorReportPaths(cfg.Notify.Distributor.ReportPathTemplates, values) bundlePaths, err := renderDistributorReportBundlePaths(cfg, plannedReport.Resolved, item.RunID, item.ReportPath, values)
if err != nil { if err != nil {
return batchNotificationRequest{}, fmt.Errorf("batch notification report %q run %q source path %q: %w", item.ReportID, item.RunID, item.ReportPath, err) return batchNotificationRequest{}, err
} }
included := BatchNotificationReport{ included := BatchNotificationReport{

View File

@@ -66,7 +66,6 @@ type DistributorNotifyConfig struct {
PipelineIDTemplate string `yaml:"pipeline_id_template"` PipelineIDTemplate string `yaml:"pipeline_id_template"`
BundleIDTemplate string `yaml:"bundle_id_template"` BundleIDTemplate string `yaml:"bundle_id_template"`
IdempotencyKeyTemplate string `yaml:"idempotency_key_template"` IdempotencyKeyTemplate string `yaml:"idempotency_key_template"`
ReportPathTemplates []string `yaml:"report_path_templates"`
Batch DistributorBatchNotifyConfig `yaml:"batch"` Batch DistributorBatchNotifyConfig `yaml:"batch"`
} }
@@ -114,9 +113,15 @@ type RecentChangeConfig struct {
type ReportConfig struct { type ReportConfig struct {
DeterministicModules []ModuleConfigItem `yaml:"deterministic_modules"` DeterministicModules []ModuleConfigItem `yaml:"deterministic_modules"`
Distributor ReportDistributorConfig `yaml:"distributor"`
deterministicModulesSet bool deterministicModulesSet bool
} }
type ReportDistributorConfig struct {
PathTemplates []string `yaml:"path_templates"`
pathTemplatesSet bool
}
type ModuleConfigItem struct { type ModuleConfigItem struct {
ID module.ID `yaml:"id"` ID module.ID `yaml:"id"`
Options any `yaml:"options,omitempty"` Options any `yaml:"options,omitempty"`
@@ -135,6 +140,10 @@ func (c *ReportConfig) UnmarshalYAML(value *yaml.Node) error {
return err return err
} }
c.deterministicModulesSet = true c.deterministicModulesSet = true
case "distributor":
if err := node.Decode(&c.Distributor); err != nil {
return err
}
default: default:
return fmt.Errorf("unknown report entry field %q", key) return fmt.Errorf("unknown report entry field %q", key)
} }
@@ -142,6 +151,81 @@ func (c *ReportConfig) UnmarshalYAML(value *yaml.Node) error {
return nil return nil
} }
func (c *DistributorNotifyConfig) UnmarshalYAML(value *yaml.Node) error {
if value.Kind != yaml.MappingNode {
return fmt.Errorf("notify distributor entry must be a mapping")
}
for i := 0; i < len(value.Content); i += 2 {
key := value.Content[i].Value
node := value.Content[i+1]
switch key {
case "enabled":
if err := node.Decode(&c.Enabled); err != nil {
return err
}
case "endpoint":
if err := node.Decode(&c.Endpoint); err != nil {
return err
}
case "token_env":
if err := node.Decode(&c.TokenEnv); err != nil {
return err
}
case "timeout":
if err := node.Decode(&c.Timeout); err != nil {
return err
}
case "failure_policy":
if err := node.Decode(&c.FailurePolicy); err != nil {
return err
}
case "pipeline_id_template":
if err := node.Decode(&c.PipelineIDTemplate); err != nil {
return err
}
case "bundle_id_template":
if err := node.Decode(&c.BundleIDTemplate); err != nil {
return err
}
case "idempotency_key_template":
if err := node.Decode(&c.IdempotencyKeyTemplate); err != nil {
return err
}
case "batch":
if err := node.Decode(&c.Batch); err != nil {
return err
}
default:
return fmt.Errorf("unknown notify distributor field %q", key)
}
}
return nil
}
func (c *ReportDistributorConfig) UnmarshalYAML(value *yaml.Node) error {
if value.Kind != yaml.MappingNode {
return fmt.Errorf("report distributor entry must be a mapping")
}
for i := 0; i < len(value.Content); i += 2 {
key := value.Content[i].Value
node := value.Content[i+1]
switch key {
case "path_templates":
if err := node.Decode(&c.PathTemplates); err != nil {
return err
}
c.pathTemplatesSet = true
default:
return fmt.Errorf("unknown report distributor field %q", key)
}
}
return nil
}
func (c ReportDistributorConfig) PathTemplatesSet() bool {
return c.pathTemplatesSet
}
func (m *ModuleConfigItem) UnmarshalYAML(value *yaml.Node) error { func (m *ModuleConfigItem) UnmarshalYAML(value *yaml.Node) error {
switch value.Kind { switch value.Kind {
case yaml.ScalarNode: case yaml.ScalarNode:

View File

@@ -10,6 +10,7 @@ import (
"gitea.maximumdirect.net/eric/weatherreporter/internal/module" "gitea.maximumdirect.net/eric/weatherreporter/internal/module"
"gitea.maximumdirect.net/eric/weatherreporter/internal/report" "gitea.maximumdirect.net/eric/weatherreporter/internal/report"
"gopkg.in/yaml.v3"
) )
func TestDefaults(t *testing.T) { func TestDefaults(t *testing.T) {
@@ -57,12 +58,6 @@ func TestDefaults(t *testing.T) {
if cfg.Notify.Distributor.IdempotencyKeyTemplate != "{bundle_id}.{run_id}" { if cfg.Notify.Distributor.IdempotencyKeyTemplate != "{bundle_id}.{run_id}" {
t.Fatalf("Notify.Distributor.IdempotencyKeyTemplate = %q, want default", cfg.Notify.Distributor.IdempotencyKeyTemplate) t.Fatalf("Notify.Distributor.IdempotencyKeyTemplate = %q, want default", cfg.Notify.Distributor.IdempotencyKeyTemplate)
} }
wantReportPaths := []string{
"{valid_start_date}/{artifact_group}/{valid_start_date}-{artifact_group}-{run_id}.md",
}
if strings.Join(cfg.Notify.Distributor.ReportPathTemplates, "\n") != strings.Join(wantReportPaths, "\n") {
t.Fatalf("Notify.Distributor.ReportPathTemplates = %#v, want %#v", cfg.Notify.Distributor.ReportPathTemplates, wantReportPaths)
}
if !cfg.Notify.Distributor.Batch.Enabled { if !cfg.Notify.Distributor.Batch.Enabled {
t.Fatalf("Notify.Distributor.Batch.Enabled = false, want true") t.Fatalf("Notify.Distributor.Batch.Enabled = false, want true")
} }
@@ -101,9 +96,6 @@ func TestLoadExampleConfig(t *testing.T) {
if cfg.Notify.Distributor.PipelineIDTemplate != "weatherreporter.{report_id}" { if cfg.Notify.Distributor.PipelineIDTemplate != "weatherreporter.{report_id}" {
t.Fatalf("PipelineIDTemplate = %q, want example pipeline template", cfg.Notify.Distributor.PipelineIDTemplate) t.Fatalf("PipelineIDTemplate = %q, want example pipeline template", cfg.Notify.Distributor.PipelineIDTemplate)
} }
if len(cfg.Notify.Distributor.ReportPathTemplates) != 1 {
t.Fatalf("ReportPathTemplates = %#v, want example archive path", cfg.Notify.Distributor.ReportPathTemplates)
}
if !cfg.Notify.Distributor.Batch.Enabled { if !cfg.Notify.Distributor.Batch.Enabled {
t.Fatalf("Notify.Distributor.Batch.Enabled = false, want true") t.Fatalf("Notify.Distributor.Batch.Enabled = false, want true")
} }
@@ -313,6 +305,102 @@ reports:
} }
} }
func TestLoadReportDistributorPathOverrides(t *testing.T) {
path := writeConfig(t, `
reports:
daily:
distributor:
path_templates:
- "daily/{valid_start_date}/{run_id}.md"
- "daily/{valid_start_date}/index.md"
today:
deterministic_modules:
- metadata
`)
cfg, err := LoadFile(path)
if err != nil {
t.Fatalf("LoadFile() error = %v", err)
}
daily := cfg.Reports["daily"].Distributor
if !daily.PathTemplatesSet() {
t.Fatal("daily distributor path_templates set = false, want true")
}
want := []string{
"daily/{valid_start_date}/{run_id}.md",
"daily/{valid_start_date}/index.md",
}
if !reflect.DeepEqual(daily.PathTemplates, want) {
t.Fatalf("daily path templates = %#v, want %#v", daily.PathTemplates, want)
}
if cfg.Reports["today"].Distributor.PathTemplatesSet() {
t.Fatal("today distributor path_templates set = true, want false")
}
overrides, err := cfg.ReportDistributorPathOverrides()
if err != nil {
t.Fatalf("ReportDistributorPathOverrides() error = %v", err)
}
if !reflect.DeepEqual(overrides[report.Daily], want) {
t.Fatalf("daily distributor override = %#v, want %#v", overrides[report.Daily], want)
}
if _, ok := overrides[report.Today]; ok {
t.Fatalf("today distributor override = %#v, want omitted override absent", overrides[report.Today])
}
}
func TestReportDistributorPathTemplatesSetTracksExplicitEmptyList(t *testing.T) {
var cfg Config
if err := yaml.Unmarshal([]byte(`
reports:
daily:
distributor:
path_templates: []
today:
distributor: {}
`), &cfg); err != nil {
t.Fatalf("yaml.Unmarshal() error = %v", err)
}
if !cfg.Reports["daily"].Distributor.PathTemplatesSet() {
t.Fatal("daily distributor path_templates set = false, want true")
}
if len(cfg.Reports["daily"].Distributor.PathTemplates) != 0 {
t.Fatalf("daily path templates = %#v, want empty explicit list", cfg.Reports["daily"].Distributor.PathTemplates)
}
if cfg.Reports["today"].Distributor.PathTemplatesSet() {
t.Fatal("today distributor path_templates set = true, want false")
}
}
func TestLoadReportDistributorPathOverrideAliases(t *testing.T) {
path := writeConfig(t, `
reports:
three-day-outlook:
distributor:
path_templates:
- "three-day/{valid_start_date}/index.md"
weekend_outlook:
distributor:
path_templates:
- "weekend/{valid_start_date}/index.md"
`)
cfg, err := LoadFile(path)
if err != nil {
t.Fatalf("LoadFile() error = %v", err)
}
overrides, err := cfg.ReportDistributorPathOverrides()
if err != nil {
t.Fatalf("ReportDistributorPathOverrides() error = %v", err)
}
if !reflect.DeepEqual(overrides[report.ThreeDay], []string{"three-day/{valid_start_date}/index.md"}) {
t.Fatalf("three-day distributor override = %#v, want alias override", overrides[report.ThreeDay])
}
if !reflect.DeepEqual(overrides[report.Weekend], []string{"weekend/{valid_start_date}/index.md"}) {
t.Fatalf("weekend distributor override = %#v, want alias override", overrides[report.Weekend])
}
}
func TestValidateReportModuleKeysWithoutMutatingOptions(t *testing.T) { func TestValidateReportModuleKeysWithoutMutatingOptions(t *testing.T) {
cfg := Defaults() cfg := Defaults()
rawOptions := map[string]any{ rawOptions := map[string]any{
@@ -565,6 +653,168 @@ reports:
} }
} }
func TestReportDistributorPathOverrideValidation(t *testing.T) {
tests := []struct {
name string
yaml string
wantErr string
}{
{
name: "UnknownReportField",
yaml: `
reports:
daily:
distributor_paths:
- latest.md
`,
wantErr: `unknown report entry field "distributor_paths"`,
},
{
name: "UnknownDistributorField",
yaml: `
reports:
daily:
distributor:
paths:
- latest.md
`,
wantErr: `unknown report distributor field "paths"`,
},
{
name: "DuplicateReportAlias",
yaml: `
reports:
three-day:
distributor:
path_templates:
- "three-day/{valid_start_date}/index.md"
three_day:
distributor:
path_templates:
- "three-day/latest.md"
`,
wantErr: "duplicates report override",
},
{
name: "UnknownTemplateVariable",
yaml: `
reports:
daily:
distributor:
path_templates:
- "{unknown}.md"
`,
wantErr: `reports.daily.distributor.path_templates[0] contains unknown template variable "unknown"`,
},
{
name: "AbsolutePath",
yaml: `
reports:
daily:
distributor:
path_templates:
- "/daily.md"
`,
wantErr: "reports.daily.distributor.path_templates[0] must render a relative path",
},
{
name: "ParentSegment",
yaml: `
reports:
daily:
distributor:
path_templates:
- "daily/../index.md"
`,
wantErr: "reports.daily.distributor.path_templates[0] must not render . or .. path segments",
},
{
name: "Manifest",
yaml: `
reports:
daily:
distributor:
path_templates:
- "daily/manifest.json"
`,
wantErr: `reports.daily.distributor.path_templates[0] must not render reserved path segment "manifest.json"`,
},
{
name: "DuplicateRenderedPath",
yaml: `
reports:
daily:
distributor:
path_templates:
- "daily/index.md"
- "daily/index.md"
`,
wantErr: `reports.daily.distributor.path_templates renders duplicate path "daily/index.md"`,
},
{
name: "EmptyOverrideList",
yaml: `
reports:
daily:
distributor:
path_templates: []
`,
wantErr: "reports.daily.distributor.path_templates must contain at least one entry",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := LoadFile(writeConfig(t, tt.yaml))
if err == nil {
t.Fatal("LoadFile() error = nil, want validation error")
}
if !strings.Contains(err.Error(), tt.wantErr) {
t.Fatalf("error = %q, want %q", err.Error(), tt.wantErr)
}
})
}
}
func TestReportDistributorPathOverridesConsistentForLoadedAndConstructedConfig(t *testing.T) {
yaml := `
reports:
daily:
distributor:
path_templates:
- "daily/{valid_start_date}/index.md"
`
reports := map[string]ReportConfig{
"daily": {
Distributor: ReportDistributorConfig{
PathTemplates: []string{"daily/{valid_start_date}/index.md"},
pathTemplatesSet: true,
},
},
}
cfg, err := LoadFile(writeConfig(t, yaml))
if err != nil {
t.Fatalf("LoadFile() error = %v", err)
}
loaded, err := cfg.ReportDistributorPathOverrides()
if err != nil {
t.Fatalf("loaded ReportDistributorPathOverrides() error = %v", err)
}
cfg = Defaults()
cfg.Reports = reports
if err := Validate(cfg); err != nil {
t.Fatalf("Validate() error = %v", err)
}
constructed, err := cfg.ReportDistributorPathOverrides()
if err != nil {
t.Fatalf("constructed ReportDistributorPathOverrides() error = %v", err)
}
if !reflect.DeepEqual(loaded, constructed) {
t.Fatalf("loaded overrides = %#v, constructed = %#v", loaded, constructed)
}
}
func TestReportModuleValidationConsistentForLoadedAndConstructedConfig(t *testing.T) { func TestReportModuleValidationConsistentForLoadedAndConstructedConfig(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
@@ -806,6 +1056,37 @@ func TestDisabledDistributorNotifyAcceptsOmittedFields(t *testing.T) {
} }
} }
func TestDistributorNotifyRejectsRemovedGlobalReportPaths(t *testing.T) {
removedField := "report_path" + "_templates"
_, err := LoadFile(writeConfig(t, `
notify:
distributor:
`+removedField+`:
- index.md
`))
if err == nil {
t.Fatal("LoadFile() error = nil, want removed global path field error")
}
if !strings.Contains(err.Error(), `unknown notify distributor field "`+removedField+`"`) {
t.Fatalf("error = %q, want removed global path field rejection", err.Error())
}
}
func TestDistributorNotifyRejectsUnknownFields(t *testing.T) {
_, err := LoadFile(writeConfig(t, `
notify:
distributor:
paths:
- index.md
`))
if err == nil {
t.Fatal("LoadFile() error = nil, want unknown distributor field error")
}
if !strings.Contains(err.Error(), `unknown notify distributor field "paths"`) {
t.Fatalf("error = %q, want unknown field rejection", err.Error())
}
}
func TestDisabledDistributorNotifyAcceptsMalformedBatchTemplates(t *testing.T) { func TestDisabledDistributorNotifyAcceptsMalformedBatchTemplates(t *testing.T) {
cfg := Defaults() cfg := Defaults()
cfg.Notify.Distributor.Enabled = false cfg.Notify.Distributor.Enabled = false
@@ -895,32 +1176,11 @@ func TestEnabledDistributorNotifyValidation(t *testing.T) {
wantErr: "notify.distributor.idempotency_key_template", wantErr: "notify.distributor.idempotency_key_template",
}, },
{ {
name: "ReportPathTemplatesEmpty", name: "BatchTemplate",
mutate: func(cfg *Config) { mutate: func(cfg *Config) {
cfg.Notify.Distributor.ReportPathTemplates = nil cfg.Notify.Distributor.Batch.BundleIDTemplate = "{run_id}"
}, },
wantErr: "notify.distributor.report_path_templates", wantErr: "notify.distributor.batch.bundle_id_template",
},
{
name: "ReportPathTemplateUnknown",
mutate: func(cfg *Config) {
cfg.Notify.Distributor.ReportPathTemplates = []string{"{unknown}"}
},
wantErr: "notify.distributor.report_path_templates",
},
{
name: "ReportPathTemplateInvalidPath",
mutate: func(cfg *Config) {
cfg.Notify.Distributor.ReportPathTemplates = []string{"/{batch_output_name}"}
},
wantErr: "notify.distributor.report_path_templates",
},
{
name: "ReportPathTemplateDuplicatePath",
mutate: func(cfg *Config) {
cfg.Notify.Distributor.ReportPathTemplates = []string{"latest.md", "latest.md"}
},
wantErr: "notify.distributor.report_path_templates",
}, },
} }
@@ -1058,35 +1318,38 @@ func TestDistributorTemplateRendering(t *testing.T) {
ValidEndTime: "0600", ValidEndTime: "0600",
ValidStartStamp: "2026-06-07T1800", ValidStartStamp: "2026-06-07T1800",
ValidEndStamp: "2026-06-08T0600", ValidEndStamp: "2026-06-08T0600",
StormID: "2026-06-07T1800-2026-06-08T0600",
BundleID: "weatherreporter.home.daily", BundleID: "weatherreporter.home.daily",
} }
bundleID, err := RenderDistributorBundleID("weatherreporter.{location_id}.{report_id}", values) bundleID, err := RenderDistributorBundleID("weatherreporter.{location_id}.{report_id}.{storm_id}", values)
if err != nil { if err != nil {
t.Fatalf("RenderDistributorBundleID() error = %v", err) t.Fatalf("RenderDistributorBundleID() error = %v", err)
} }
if bundleID != "weatherreporter.home.daily" { if bundleID != "weatherreporter.home.daily.2026-06-07T1800-2026-06-08T0600" {
t.Fatalf("bundleID = %q, want rendered value", bundleID) t.Fatalf("bundleID = %q, want rendered value", bundleID)
} }
values.BundleID = bundleID
pipelineID, err := RenderDistributorPipelineID("weatherreporter.{artifact_group}.{bundle_id}", values) pipelineID, err := RenderDistributorPipelineID("weatherreporter.{artifact_group}.{storm_id}.{bundle_id}", values)
if err != nil { if err != nil {
t.Fatalf("RenderDistributorPipelineID() error = %v", err) t.Fatalf("RenderDistributorPipelineID() error = %v", err)
} }
if pipelineID != "weatherreporter.daily.weatherreporter.home.daily" { if pipelineID != "weatherreporter.daily.2026-06-07T1800-2026-06-08T0600.weatherreporter.home.daily.2026-06-07T1800-2026-06-08T0600" {
t.Fatalf("pipelineID = %q, want rendered pipeline ID", pipelineID) t.Fatalf("pipelineID = %q, want rendered pipeline ID", pipelineID)
} }
idempotencyKey, err := RenderDistributorIdempotencyKey("{bundle_id}.{run_id}", values) idempotencyKey, err := RenderDistributorIdempotencyKey("{bundle_id}.{storm_id}.{run_id}", values)
if err != nil { if err != nil {
t.Fatalf("RenderDistributorIdempotencyKey() error = %v", err) t.Fatalf("RenderDistributorIdempotencyKey() error = %v", err)
} }
if idempotencyKey != "weatherreporter.home.daily.20260607T120000Z" { if idempotencyKey != "weatherreporter.home.daily.2026-06-07T1800-2026-06-08T0600.2026-06-07T1800-2026-06-08T0600.20260607T120000Z" {
t.Fatalf("idempotencyKey = %q, want rendered run key", idempotencyKey) t.Fatalf("idempotencyKey = %q, want rendered run key", idempotencyKey)
} }
reportPaths, err := RenderDistributorReportPaths([]string{ reportPaths, err := RenderDistributorReportPaths("reports.daily.distributor.path_templates", []string{
"{valid_start_date}/{artifact_group}/{valid_start_stamp}-{valid_end_stamp}-{run_id}.md", "{valid_start_date}/{artifact_group}/{valid_start_stamp}-{valid_end_stamp}-{run_id}.md",
"storm/{storm_id}/index.md",
"{valid_start_date}/{artifact_group}/latest.md", "{valid_start_date}/{artifact_group}/latest.md",
}, values) }, values)
if err != nil { if err != nil {
@@ -1094,6 +1357,7 @@ func TestDistributorTemplateRendering(t *testing.T) {
} }
wantPaths := []string{ wantPaths := []string{
"2026-06-07/daily/2026-06-07T1800-2026-06-08T0600-20260607T120000Z.md", "2026-06-07/daily/2026-06-07T1800-2026-06-08T0600-20260607T120000Z.md",
"storm/2026-06-07T1800-2026-06-08T0600/index.md",
"2026-06-07/daily/latest.md", "2026-06-07/daily/latest.md",
} }
if strings.Join(reportPaths, "\n") != strings.Join(wantPaths, "\n") { if strings.Join(reportPaths, "\n") != strings.Join(wantPaths, "\n") {
@@ -1101,6 +1365,50 @@ func TestDistributorTemplateRendering(t *testing.T) {
} }
} }
func TestDistributorReportPathRenderingUsesCallerName(t *testing.T) {
values := DistributorTemplateValues{
BatchOutputName: "report.md",
}
tests := []struct {
name string
templates []string
wantErr string
}{
{
name: "UnknownVariable",
templates: []string{"{unknown}.md"},
wantErr: `report.daily.distributor_path_templates[0] contains unknown template variable "unknown"`,
},
{
name: "InvalidPath",
templates: []string{"/{batch_output_name}"},
wantErr: "report.daily.distributor_path_templates[0] must render a relative path",
},
{
name: "DuplicatePath",
templates: []string{"latest.md", "latest.md"},
wantErr: `report.daily.distributor_path_templates renders duplicate path "latest.md"`,
},
{
name: "Empty",
templates: nil,
wantErr: "report.daily.distributor_path_templates must contain at least one entry",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := RenderDistributorReportPaths("report.daily.distributor_path_templates", tt.templates, values)
if err == nil {
t.Fatal("RenderDistributorReportPaths() error = nil, want error")
}
if !strings.Contains(err.Error(), tt.wantErr) {
t.Fatalf("error = %q, want %q", err.Error(), tt.wantErr)
}
})
}
}
func TestDistributorBatchTemplateRendering(t *testing.T) { func TestDistributorBatchTemplateRendering(t *testing.T) {
values := DistributorBatchTemplateValues{ values := DistributorBatchTemplateValues{
LocationID: "home", LocationID: "home",
@@ -1233,12 +1541,15 @@ func TestDistributorReportPathRenderingRejectsInvalidValues(t *testing.T) {
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
_, err := RenderDistributorReportPaths([]string{"{batch_output_name}"}, DistributorTemplateValues{ _, err := RenderDistributorReportPaths("report.daily.distributor_path_templates", []string{"{batch_output_name}"}, DistributorTemplateValues{
BatchOutputName: tt.batchOutputName, BatchOutputName: tt.batchOutputName,
}) })
if err == nil { if err == nil {
t.Fatal("RenderDistributorReportPaths() error = nil, want error") t.Fatal("RenderDistributorReportPaths() error = nil, want error")
} }
if !strings.Contains(err.Error(), "report.daily.distributor_path_templates[0]") {
t.Fatalf("error = %q, want caller path name", err.Error())
}
}) })
} }
} }

View File

@@ -31,9 +31,6 @@ func Defaults() Config {
PipelineIDTemplate: "", PipelineIDTemplate: "",
BundleIDTemplate: "weatherreporter.{location_id}.{report_id}", BundleIDTemplate: "weatherreporter.{location_id}.{report_id}",
IdempotencyKeyTemplate: "{bundle_id}.{run_id}", IdempotencyKeyTemplate: "{bundle_id}.{run_id}",
ReportPathTemplates: []string{
"{valid_start_date}/{artifact_group}/{valid_start_date}-{artifact_group}-{run_id}.md",
},
Batch: DistributorBatchNotifyConfig{ Batch: DistributorBatchNotifyConfig{
Enabled: true, Enabled: true,
PipelineIDTemplate: "weatherreporter", PipelineIDTemplate: "weatherreporter",

View File

@@ -18,6 +18,7 @@ type DistributorTemplateValues struct {
ValidEndTime string ValidEndTime string
ValidStartStamp string ValidStartStamp string
ValidEndStamp string ValidEndStamp string
StormID string
BundleID string BundleID string
} }
@@ -41,6 +42,7 @@ var distributorTemplateVariables = map[string]struct{}{
"valid_end_time": {}, "valid_end_time": {},
"valid_start_stamp": {}, "valid_start_stamp": {},
"valid_end_stamp": {}, "valid_end_stamp": {},
"storm_id": {},
} }
var distributorIdempotencyTemplateVariables = map[string]struct{}{ var distributorIdempotencyTemplateVariables = map[string]struct{}{
@@ -55,6 +57,7 @@ var distributorIdempotencyTemplateVariables = map[string]struct{}{
"valid_end_time": {}, "valid_end_time": {},
"valid_start_stamp": {}, "valid_start_stamp": {},
"valid_end_stamp": {}, "valid_end_stamp": {},
"storm_id": {},
"bundle_id": {}, "bundle_id": {},
} }
@@ -129,23 +132,23 @@ func RenderDistributorBatchIdempotencyKey(template string, values DistributorBat
return rendered, nil return rendered, nil
} }
func RenderDistributorReportPaths(templates []string, values DistributorTemplateValues) ([]string, error) { func RenderDistributorReportPaths(name string, templates []string, values DistributorTemplateValues) ([]string, error) {
if len(templates) == 0 { if len(templates) == 0 {
return nil, fmt.Errorf("notify.distributor.report_path_templates must contain at least one entry") return nil, fmt.Errorf("%s must contain at least one entry", name)
} }
paths := make([]string, 0, len(templates)) paths := make([]string, 0, len(templates))
seen := make(map[string]struct{}, len(templates)) seen := make(map[string]struct{}, len(templates))
for i, template := range templates { for i, template := range templates {
name := fmt.Sprintf("notify.distributor.report_path_templates[%d]", i) itemName := fmt.Sprintf("%s[%d]", name, i)
rendered, err := renderDistributorTemplate(name, template, values, distributorTemplateVariables) rendered, err := renderDistributorTemplate(itemName, template, values, distributorTemplateVariables)
if err != nil { if err != nil {
return nil, err return nil, err
} }
if err := ValidateDistributorReportPath(name, rendered); err != nil { if err := ValidateDistributorReportPath(itemName, rendered); err != nil {
return nil, err return nil, err
} }
if _, ok := seen[rendered]; ok { if _, ok := seen[rendered]; ok {
return nil, fmt.Errorf("notify.distributor.report_path_templates renders duplicate path %q", rendered) return nil, fmt.Errorf("%s renders duplicate path %q", name, rendered)
} }
seen[rendered] = struct{}{} seen[rendered] = struct{}{}
paths = append(paths, rendered) paths = append(paths, rendered)
@@ -243,6 +246,8 @@ func distributorTemplateValue(variable string, values DistributorTemplateValues)
return values.ValidStartStamp return values.ValidStartStamp
case "valid_end_stamp": case "valid_end_stamp":
return values.ValidEndStamp return values.ValidEndStamp
case "storm_id":
return values.StormID
case "bundle_id": case "bundle_id":
return values.BundleID return values.BundleID
default: default:

View File

@@ -17,6 +17,10 @@ func (cfg Config) ReportModuleOverrides() (map[report.ID][]module.ConfigItem, er
}) })
} }
func (cfg Config) ReportDistributorPathOverrides() (map[report.ID][]string, error) {
return traverseReportDistributorPathOverrides(cfg)
}
func normalizeReportModules(cfg *Config) error { func normalizeReportModules(cfg *Config) error {
if cfg.Reports == nil { if cfg.Reports == nil {
cfg.Reports = map[string]ReportConfig{} cfg.Reports = map[string]ReportConfig{}
@@ -35,6 +39,11 @@ func validateReportModules(cfg Config) error {
return err return err
} }
func validateReportDistributorPathOverrides(cfg Config) error {
_, err := traverseReportDistributorPathOverrides(cfg)
return err
}
type reportModuleTraversalOptions struct { type reportModuleTraversalOptions struct {
normalizeOptions bool normalizeOptions bool
updateConfig bool updateConfig bool
@@ -82,6 +91,59 @@ func traverseReportModules(cfg *Config, opts reportModuleTraversalOptions) (map[
return overrides, nil return overrides, nil
} }
func traverseReportDistributorPathOverrides(cfg Config) (map[report.ID][]string, error) {
overrides := map[report.ID][]string{}
if cfg.Reports == nil {
return overrides, nil
}
reportRegistry := report.DefaultRegistry()
seenReports := map[report.ID]string{}
for key, reportCfg := range cfg.Reports {
reportID, err := report.IDForConfigKey(key)
if err != nil {
return nil, fmt.Errorf("reports.%s: %w", key, err)
}
if previous, ok := seenReports[reportID]; ok {
return nil, fmt.Errorf("reports.%s duplicates report override %q", key, previous)
}
seenReports[reportID] = key
if _, err := reportRegistry.Lookup(reportID); err != nil {
return nil, fmt.Errorf("reports.%s: %w", key, err)
}
if !reportCfg.Distributor.pathTemplatesSet {
continue
}
if err := validateReportDistributorPathTemplates(key, reportCfg.Distributor.PathTemplates); err != nil {
return nil, err
}
overrides[reportID] = append([]string(nil), reportCfg.Distributor.PathTemplates...)
}
return overrides, nil
}
func validateReportDistributorPathTemplates(reportKey string, templates []string) error {
name := fmt.Sprintf("reports.%s.distributor.path_templates", reportKey)
_, err := RenderDistributorReportPaths(name, templates, sampleDistributorTemplateValues())
return err
}
func sampleDistributorTemplateValues() DistributorTemplateValues {
return DistributorTemplateValues{
LocationID: "location",
ReportID: "report",
RunID: "run",
ArtifactGroup: "artifact",
BatchOutputName: "report.md",
ValidStartDate: "2026-05-29",
ValidEndDate: "2026-05-30",
ValidStartTime: "0000",
ValidEndTime: "0000",
ValidStartStamp: "2026-05-29T0000",
ValidEndStamp: "2026-05-30T0000",
StormID: "2026-05-29T0000-2026-05-30T0000",
}
}
func moduleItemsFromConfig(registry briefing.ModuleRegistry, reportKey string, items []ModuleConfigItem, normalizeOptions bool) ([]module.ConfigItem, []ModuleConfigItem, error) { func moduleItemsFromConfig(registry briefing.ModuleRegistry, reportKey string, items []ModuleConfigItem, normalizeOptions bool) ([]module.ConfigItem, []ModuleConfigItem, error) {
out := make([]module.ConfigItem, 0, len(items)) out := make([]module.ConfigItem, 0, len(items))
normalizedItems := append([]ModuleConfigItem(nil), items...) normalizedItems := append([]ModuleConfigItem(nil), items...)

View File

@@ -12,6 +12,9 @@ func Validate(cfg Config) error {
if err := validateReportModules(cfg); err != nil { if err := validateReportModules(cfg); err != nil {
return err return err
} }
if err := validateReportDistributorPathOverrides(cfg); err != nil {
return err
}
if cfg.WeatherAPI.BaseURL != "" { if cfg.WeatherAPI.BaseURL != "" {
parsed, err := url.Parse(cfg.WeatherAPI.BaseURL) parsed, err := url.Parse(cfg.WeatherAPI.BaseURL)
if err != nil || parsed.Scheme == "" || parsed.Host == "" { if err != nil || parsed.Scheme == "" || parsed.Host == "" {
@@ -121,22 +124,7 @@ func validateDistributorNotify(cfg DistributorNotifyConfig) error {
if err := validateDistributorTemplate("notify.distributor.idempotency_key_template", cfg.IdempotencyKeyTemplate, distributorIdempotencyTemplateVariables); err != nil { if err := validateDistributorTemplate("notify.distributor.idempotency_key_template", cfg.IdempotencyKeyTemplate, distributorIdempotencyTemplateVariables); err != nil {
return err return err
} }
if len(cfg.ReportPathTemplates) == 0 { values := sampleDistributorTemplateValues()
return fmt.Errorf("notify.distributor.report_path_templates must contain at least one entry when enabled")
}
values := DistributorTemplateValues{
LocationID: "location",
ReportID: "report",
RunID: "run",
ArtifactGroup: "artifact",
BatchOutputName: "report.md",
ValidStartDate: "2026-05-29",
ValidEndDate: "2026-05-30",
ValidStartTime: "0000",
ValidEndTime: "0000",
ValidStartStamp: "2026-05-29T0000",
ValidEndStamp: "2026-05-30T0000",
}
bundleID, err := RenderDistributorBundleID(cfg.BundleIDTemplate, values) bundleID, err := RenderDistributorBundleID(cfg.BundleIDTemplate, values)
if err != nil { if err != nil {
return err return err
@@ -145,9 +133,6 @@ func validateDistributorNotify(cfg DistributorNotifyConfig) error {
if _, err := RenderDistributorPipelineID(cfg.PipelineIDTemplate, values); err != nil { if _, err := RenderDistributorPipelineID(cfg.PipelineIDTemplate, values); err != nil {
return err return err
} }
if _, err := RenderDistributorReportPaths(cfg.ReportPathTemplates, values); err != nil {
return err
}
if err := validateDistributorBatchNotify(cfg.Batch); err != nil { if err := validateDistributorBatchNotify(cfg.Batch); err != nil {
return err return err
} }

View File

@@ -18,6 +18,10 @@ func dailyDefinition() Definition {
ComparisonStrategy: CompareSameValidDate, ComparisonStrategy: CompareSameValidDate,
ArtifactGroup: "daily", ArtifactGroup: "daily",
BatchOutputName: "daily.md", BatchOutputName: "daily.md",
DistributorPathTemplates: []string{
"daily/{valid_start_date}/{run_id}.md",
"daily/{valid_start_date}/index.md",
},
Generated: true, Generated: true,
CompatiblePriorIDs: []ID{Daily}, CompatiblePriorIDs: []ID{Daily},
Modules: dailyModules(), Modules: dailyModules(),

View File

@@ -55,6 +55,7 @@ type Definition struct {
ComparisonStrategy ComparisonStrategy ComparisonStrategy ComparisonStrategy
ArtifactGroup string ArtifactGroup string
BatchOutputName string BatchOutputName string
DistributorPathTemplates []string
Generated bool Generated bool
CompatiblePriorIDs []ID CompatiblePriorIDs []ID
Modules []module.ConfigItem Modules []module.ConfigItem

View File

@@ -20,6 +20,9 @@ func hourlyDefinition() Definition {
ComparisonStrategy: CompareRollingWindow, ComparisonStrategy: CompareRollingWindow,
ArtifactGroup: "hourly", ArtifactGroup: "hourly",
BatchOutputName: "hourly.md", BatchOutputName: "hourly.md",
DistributorPathTemplates: []string{
"hourly/index.md",
},
Generated: true, Generated: true,
CompatiblePriorIDs: []ID{Hourly}, CompatiblePriorIDs: []ID{Hourly},
Modules: hourlyModules(), Modules: hourlyModules(),

View File

@@ -520,6 +520,88 @@ func TestRegistryDefinitionsDeclarePathAndCompatibilityPolicy(t *testing.T) {
} }
} }
func TestGeneratedRegistryDefinitionsDeclareDistributorPathDefaults(t *testing.T) {
for _, definition := range DefaultRegistry().All() {
if !definition.Generated {
continue
}
if len(definition.DistributorPathTemplates) == 0 {
t.Fatalf("%s DistributorPathTemplates is empty", definition.ID)
}
}
}
func TestRegistryDefinitionsDeclareDefaultDistributorPathTemplates(t *testing.T) {
tests := []struct {
id ID
want []string
}{
{
id: Hourly,
want: []string{
"hourly/index.md",
},
},
{
id: Daily,
want: []string{
"daily/{valid_start_date}/{run_id}.md",
"daily/{valid_start_date}/index.md",
},
},
{
id: Today,
want: []string{
"daily/{valid_start_date}/{run_id}.md",
"daily/{valid_start_date}/index.md",
"today/index.md",
},
},
{
id: Tomorrow,
want: []string{
"daily/{valid_start_date}/{run_id}.md",
"daily/{valid_start_date}/index.md",
"tomorrow/index.md",
},
},
{
id: ThreeDay,
want: []string{
"three-day/{valid_start_date}/{run_id}.md",
"three-day/{valid_start_date}/index.md",
},
},
{
id: Weekend,
want: []string{
"weekend/{valid_start_date}/{run_id}.md",
"weekend/{valid_start_date}/index.md",
},
},
{
id: Storm,
want: []string{
"storm/{storm_id}/{run_id}.md",
"storm/{storm_id}/index.md",
},
},
}
registry := DefaultRegistry()
for _, tt := range tests {
t.Run(string(tt.id), func(t *testing.T) {
definition, err := registry.Lookup(tt.id)
if err != nil {
t.Fatalf("Lookup() error = %v", err)
}
if !reflect.DeepEqual(definition.DistributorPathTemplates, tt.want) {
t.Fatalf("DistributorPathTemplates = %#v, want %#v", definition.DistributorPathTemplates, tt.want)
}
})
}
}
func TestRegistryDefinitionsDeclareDefaultModules(t *testing.T) { func TestRegistryDefinitionsDeclareDefaultModules(t *testing.T) {
tests := []struct { tests := []struct {
id ID id ID
@@ -682,6 +764,9 @@ func TestRegistryAppliesModuleOverridesWithoutChangingDefaults(t *testing.T) {
if len(defaultDefinition.ModuleIDs()) <= len(definition.ModuleIDs()) { if len(defaultDefinition.ModuleIDs()) <= len(definition.ModuleIDs()) {
t.Fatalf("default ModuleIDs() = %#v, want original defaults unchanged", defaultDefinition.ModuleIDs()) t.Fatalf("default ModuleIDs() = %#v, want original defaults unchanged", defaultDefinition.ModuleIDs())
} }
if !reflect.DeepEqual(definition.DistributorPathTemplates, defaultDefinition.DistributorPathTemplates) {
t.Fatalf("overridden DistributorPathTemplates = %#v, want %#v", definition.DistributorPathTemplates, defaultDefinition.DistributorPathTemplates)
}
} }
func TestRegistryRejectsModuleOverrideForUnknownReport(t *testing.T) { func TestRegistryRejectsModuleOverrideForUnknownReport(t *testing.T) {

View File

@@ -30,8 +30,7 @@ func DefaultRegistry() Registry {
func (r Registry) WithModuleOverrides(overrides map[ID][]module.ConfigItem) (Registry, error) { func (r Registry) WithModuleOverrides(overrides map[ID][]module.ConfigItem) (Registry, error) {
next := Registry{definitions: map[ID]Definition{}} next := Registry{definitions: map[ID]Definition{}}
for id, definition := range r.definitions { for id, definition := range r.definitions {
definition.Modules = append([]module.ConfigItem(nil), definition.Modules...) next.definitions[id] = cloneDefinition(definition)
next.definitions[id] = definition
} }
for id, items := range overrides { for id, items := range overrides {
definition, ok := next.definitions[id] definition, ok := next.definitions[id]
@@ -44,6 +43,12 @@ func (r Registry) WithModuleOverrides(overrides map[ID][]module.ConfigItem) (Reg
return next, nil return next, nil
} }
func cloneDefinition(definition Definition) Definition {
definition.Modules = cloneModuleItems(definition.Modules)
definition.DistributorPathTemplates = append([]string(nil), definition.DistributorPathTemplates...)
return definition
}
func moduleItems(ids ...module.ID) []module.ConfigItem { func moduleItems(ids ...module.ID) []module.ConfigItem {
items := make([]module.ConfigItem, 0, len(ids)) items := make([]module.ConfigItem, 0, len(ids))
for _, id := range ids { for _, id := range ids {

View File

@@ -17,6 +17,10 @@ func stormDefinition() Definition {
ComparisonStrategy: CompareExplicitWindow, ComparisonStrategy: CompareExplicitWindow,
ArtifactGroup: "storm", ArtifactGroup: "storm",
BatchOutputName: "storm.md", BatchOutputName: "storm.md",
DistributorPathTemplates: []string{
"storm/{storm_id}/{run_id}.md",
"storm/{storm_id}/index.md",
},
Generated: true, Generated: true,
CompatiblePriorIDs: []ID{Storm}, CompatiblePriorIDs: []ID{Storm},
Modules: stormModules(), Modules: stormModules(),

View File

@@ -16,6 +16,10 @@ func threeDayDefinition() Definition {
ComparisonStrategy: CompareSameValidDate, ComparisonStrategy: CompareSameValidDate,
ArtifactGroup: "three-day", ArtifactGroup: "three-day",
BatchOutputName: "three-day.md", BatchOutputName: "three-day.md",
DistributorPathTemplates: []string{
"three-day/{valid_start_date}/{run_id}.md",
"three-day/{valid_start_date}/index.md",
},
Generated: true, Generated: true,
CompatiblePriorIDs: []ID{ThreeDay}, CompatiblePriorIDs: []ID{ThreeDay},
Modules: threeDayModules(), Modules: threeDayModules(),

View File

@@ -16,6 +16,11 @@ func todayDefinition() Definition {
ComparisonStrategy: CompareSameValidDate, ComparisonStrategy: CompareSameValidDate,
ArtifactGroup: "today", ArtifactGroup: "today",
BatchOutputName: "today.md", BatchOutputName: "today.md",
DistributorPathTemplates: []string{
"daily/{valid_start_date}/{run_id}.md",
"daily/{valid_start_date}/index.md",
"today/index.md",
},
Generated: true, Generated: true,
CompatiblePriorIDs: []ID{Today}, CompatiblePriorIDs: []ID{Today},
Modules: todayModules(), Modules: todayModules(),

View File

@@ -16,6 +16,11 @@ func tomorrowDefinition() Definition {
ComparisonStrategy: CompareSameValidDate, ComparisonStrategy: CompareSameValidDate,
ArtifactGroup: "tomorrow", ArtifactGroup: "tomorrow",
BatchOutputName: "tomorrow.md", BatchOutputName: "tomorrow.md",
DistributorPathTemplates: []string{
"daily/{valid_start_date}/{run_id}.md",
"daily/{valid_start_date}/index.md",
"tomorrow/index.md",
},
Generated: true, Generated: true,
CompatiblePriorIDs: []ID{Tomorrow}, CompatiblePriorIDs: []ID{Tomorrow},
Modules: tomorrowModules(), Modules: tomorrowModules(),

View File

@@ -17,6 +17,10 @@ func weekendDefinition() Definition {
ComparisonStrategy: CompareWeekendWindow, ComparisonStrategy: CompareWeekendWindow,
ArtifactGroup: "weekend", ArtifactGroup: "weekend",
BatchOutputName: "weekend.md", BatchOutputName: "weekend.md",
DistributorPathTemplates: []string{
"weekend/{valid_start_date}/{run_id}.md",
"weekend/{valid_start_date}/index.md",
},
Generated: true, Generated: true,
CompatiblePriorIDs: []ID{Weekend}, CompatiblePriorIDs: []ID{Weekend},
Modules: weekendModules(), Modules: weekendModules(),