Implemented debug artifacts for the distributor notification adapter

This commit is contained in:
2026-06-07 20:21:26 -05:00
parent 138bc4e7e4
commit 183b23cf5a
18 changed files with 601 additions and 77 deletions

View File

@@ -134,6 +134,7 @@ stub source slots use the missing-source policy.
- `reports_dir`: managed Markdown report directory under `workspace.root`. Default: `reports`.
- `data_packages_dir`: prompt input package directory under `workspace.root`. Default: `data-packages`.
- `preflight_dir`: Scriptorium render output directory under `workspace.root`. Default: `preflight`.
- `notifications_dir`: distributor notification debug artifact directory under `workspace.root`. Default: `notifications`.
Workspace subdirectories must be relative paths that stay inside
`workspace.root`.

View File

@@ -81,12 +81,17 @@ Single-report generation follows this order:
14. Save metadata with the managed report path.
15. If distributor notification is enabled, notify using the managed report
path as the source file.
16. Save a distributor notification debug artifact and update metadata with its
path.
If render preflight returns both a result and an error, preflight JSON and
metadata are persisted before the error is returned. If Scriptorium report
generation returns an error after writing output, the managed report and
metadata remain inspectable. Notification is not attempted after Weather API,
briefing, prompt input, render, Scriptorium run, or metadata-save failures.
When notification is attempted, the debug artifact records request identity,
accepted upload fields, distributor status fields, raw status report JSON when
available, and redacted failure context.
`--out` copies are never used as notification source files.
## Batch Workflow

View File

@@ -8,7 +8,8 @@ This document describes the distributor upload adapter in
The adapter submits generated weatherreporter Markdown reports to a configured
distributor HTTP upload endpoint. It isolates distributor package types,
token-env lookup, upload client construction, source-bundle file mapping,
timeout handling, and upload error wrapping from app orchestration.
timeout handling, status polling, and upload error wrapping from app
orchestration.
## Inputs And Outputs
@@ -21,12 +22,14 @@ Inputs:
- idempotency key
- source Markdown report path
- bundle-relative Markdown path
- bundle created timestamp
- context for cancellation
Outputs:
- accepted distributor run ID
- accepted distributor status
- accepted distributor upload status
- distributor run status, status polling error, and raw run report JSON when available
- weatherreporter-owned idempotency conflict error when applicable
## Boundaries
@@ -66,11 +69,21 @@ The adapter calls distributor `UploadFiles` with exactly one file:
- source path: the managed Markdown report path selected by app orchestration
- bundle path: the rendered bundle-relative report path
- created: the report generation timestamp
The adapter creates a distributor upload client with the configured endpoint,
bearer token, and timeout-backed HTTP client. It also wraps the upload context
with the configured timeout when the timeout is greater than zero.
After upload acceptance, the adapter polls distributor `Status` for the accepted
run ID until the run reaches `succeeded` or `failed`, or until the configured
timeout expires. It returns the latest status, error text, and raw report JSON in
weatherreporter-owned types so app orchestration can persist them in the
notification debug artifact. Status lookup failures or timeout before a terminal
state are kept as debug status errors on an otherwise accepted upload. A
terminal distributor run status of `failed` is returned as a notification failure
with the status report preserved.
## Failure Behavior
The adapter validates required endpoint, token env name, token value, bundle ID,

View File

@@ -47,6 +47,7 @@ converts adapter render results into that shape before saving.
- `workspace.reports_dir`
- `workspace.data_packages_dir`
- `workspace.preflight_dir`
- `workspace.notifications_dir`
Workspace subdirectories must be relative paths that stay under
`workspace.root`.
@@ -62,12 +63,14 @@ valid-period start date for JSON artifacts, and the RunID.
snapshots/<artifact_group>/<YYYY-MM-DD>/<run_id>.metadata.json
data-packages/<artifact_group>/<YYYY-MM-DD>/<run_id>.data_package.json
preflight/<artifact_group>/<YYYY-MM-DD>/<run_id>.render.json
notifications/<artifact_group>/<YYYY-MM-DD>/<run_id>.distributor.json
reports/<artifact_group>/<run_id>.md
```
Metadata is stored beside briefing snapshots and links the briefing, data
package, preflight, report paths, and configured prompt location. Report
listing walks metadata files under the snapshots directory.
package, preflight, report paths, notification path when attempted, and
configured prompt location. Report listing walks metadata files under the
snapshots directory.
## Prior Lookup
@@ -89,6 +92,8 @@ current report definition.
Durable JSON writes use shared atomic file helpers. Managed Markdown reports are
prepared by creating their parent directory; Scriptorium writes the report body
to the prepared path. Extra Markdown copies are handled by app orchestration.
Distributor notification debug artifacts are written atomically when
notification is attempted.
Inspection helpers read existing metadata, briefing, and data package files.
Missing metadata directories return no inspection records or no prior snapshot

View File

@@ -91,6 +91,19 @@ workspace/
storm/
YYYY-MM-DD/
<run_id>.render.json
notifications/
daily/
YYYY-MM-DD/
<run_id>.distributor.json
three-day/
YYYY-MM-DD/
<run_id>.distributor.json
weekend/
YYYY-MM-DD/
<run_id>.distributor.json
storm/
YYYY-MM-DD/
<run_id>.distributor.json
reports/
daily/
<run_id>.md
@@ -122,6 +135,7 @@ Each generated report writes metadata that links:
- prompt input data package path
- preflight output path
- managed Markdown report path
- distributor notification debug artifact path, when notification is attempted
Batch summaries include report status, error text when applicable, notification
outcome when attempted, valid period, and known artifact paths for each
@@ -153,6 +167,16 @@ not trigger notification. A notification failure fails that report. In a batch,
other reports continue, the failed report includes notification fields in the
JSON summary, and the batch returns nonzero.
Each notification attempt writes a debug artifact under `notifications/`. The
artifact records the rendered bundle ID, idempotency key, managed source path,
bundle-relative path, bundle created timestamp, accepted upload response, and
the latest distributor run status response when available. Weatherreporter polls
status until distributor reports `succeeded` or `failed`, or until the configured
notification timeout expires. The run status includes the distributor status,
error text, and raw run report JSON, which can show actions such as
`replace_older`, `skip_same`, `skip_destination_newer`, or `failed`. Token values
are not written.
Weatherreporter is responsible for selecting the managed Markdown report,
constructing a source bundle, and submitting it to the configured distributor
HTTP endpoint. Distributor remains responsible for destination routing,

View File

@@ -248,6 +248,10 @@ Diagnostic: inspect the failed batch JSON or stderr line for bundle and
idempotency context. Compare the configured templates with the report RunID and
report path.
Also inspect the notification artifact linked from metadata. It records the
rendered bundle ID, idempotency key, upload result, distributor run status,
status error, and raw run report JSON when available.
Safe fix: keep idempotency templates stable for retries of the same generated
report, but do not reuse the same rendered key for different generated report
content.
@@ -268,6 +272,11 @@ Diagnostic: inspect stdout JSON or stderr status lines for
`notify.distributor.report_path_template`, and token configuration. Token values
are redacted from weatherreporter errors.
If the upload was accepted but destination output did not change, inspect the
notification artifact's `runStatus.report`. Distributor actions such as
`replace_older`, `skip_same`, `skip_destination_newer`, or `failed` explain how
the destination handled the uploaded bundle.
Safe fix: fix the endpoint, token, templates, or distributor-side upload
configuration. The weatherreporter upload source is the managed Markdown report,
not `--out` or `--out-dir` copies.

View File

@@ -40,6 +40,7 @@ workspace:
reports_dir: reports
data_packages_dir: data-packages
preflight_dir: preflight
notifications_dir: notifications
dayparts:
- name: overnight

View File

@@ -3,6 +3,7 @@ package distributor
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
@@ -28,11 +29,26 @@ type UploadRequest struct {
IdempotencyKey string
SourcePath string
BundlePath string
CreatedAt time.Time
}
type UploadResult struct {
RunID string
Status string
RunID string
Status string
UploadStatus string
StatusError string
RunStatus *RunStatus
}
type RunStatus struct {
RunID string
PipelineID string
Status string
AcceptedAt time.Time
StartedAt *time.Time
FinishedAt *time.Time
Report json.RawMessage
Error string
}
type IdempotencyConflictError struct {
@@ -57,6 +73,7 @@ type uploadClientFactory func(endpoint, token string, timeout time.Duration) (up
type uploadClient interface {
UploadFiles(ctx context.Context, opts uploadFilesOptions) (uploadFilesResult, error)
Status(ctx context.Context, runID string) (runStatus, error)
}
type uploadFilesOptions struct {
@@ -64,6 +81,7 @@ type uploadFilesOptions struct {
IdempotencyKey string
SourcePath string
BundlePath string
CreatedAt time.Time
}
type uploadFilesResult struct {
@@ -71,6 +89,19 @@ type uploadFilesResult struct {
Status string
}
type runStatus struct {
RunID string
PipelineID string
Status string
AcceptedAt time.Time
StartedAt *time.Time
FinishedAt *time.Time
Report json.RawMessage
Error string
}
const statusPollInterval = 250 * time.Millisecond
func New(cfg config.DistributorNotifyConfig) *Client {
return newClient(cfg, newDistributorUploadClient)
}
@@ -138,6 +169,7 @@ func (c *Client) Upload(ctx context.Context, req UploadRequest) (UploadResult, e
IdempotencyKey: req.IdempotencyKey,
SourcePath: req.SourcePath,
BundlePath: req.BundlePath,
CreatedAt: req.CreatedAt,
})
if err != nil {
return UploadResult{}, wrapUploadError(err, uploadErrorContext{
@@ -150,10 +182,65 @@ func (c *Client) Upload(ctx context.Context, req UploadRequest) (UploadResult, e
})
}
return UploadResult{
RunID: result.RunID,
Status: result.Status,
}, nil
uploadResult := UploadResult{
RunID: result.RunID,
Status: result.Status,
UploadStatus: result.Status,
}
status, statusErr := waitForRunStatus(runCtx, uploadClient, result.RunID, c.Timeout > 0)
if status.RunID != "" || status.Status != "" {
uploadResult.RunStatus = &RunStatus{
RunID: status.RunID,
PipelineID: status.PipelineID,
Status: status.Status,
AcceptedAt: status.AcceptedAt,
StartedAt: status.StartedAt,
FinishedAt: status.FinishedAt,
Report: append(json.RawMessage(nil), status.Report...),
Error: redactTokenString(status.Error, token),
}
if status.Status != "" {
uploadResult.Status = status.Status
}
}
if statusErr != nil {
uploadResult.StatusError = redactTokenString(statusErr.Error(), token)
return uploadResult, nil
}
if status.Status == "failed" {
return uploadResult, fmt.Errorf("distributor run %q failed: %s", status.RunID, uploadResult.RunStatus.Error)
}
return uploadResult, nil
}
func waitForRunStatus(ctx context.Context, client uploadClient, runID string, poll bool) (runStatus, error) {
status, err := client.Status(ctx, runID)
if err != nil || terminalRunStatus(status.Status) || !poll {
return status, err
}
for {
timer := time.NewTimer(statusPollInterval)
select {
case <-ctx.Done():
timer.Stop()
return status, fmt.Errorf("distributor run %q did not reach terminal status before timeout: %w", runID, ctx.Err())
case <-timer.C:
}
next, err := client.Status(ctx, runID)
if err != nil {
return status, err
}
status = next
if terminalRunStatus(status.Status) {
return status, nil
}
}
}
func terminalRunStatus(status string) bool {
return status == "succeeded" || status == "failed"
}
type distributorUploadClient struct {
@@ -179,6 +266,7 @@ func newDistributorUploadClient(endpoint, token string, timeout time.Duration) (
func (c distributorUploadClient) UploadFiles(ctx context.Context, opts uploadFilesOptions) (uploadFilesResult, error) {
result, err := c.client.UploadFiles(ctx, distributorupload.UploadFilesOptions{
ID: opts.BundleID,
Created: opts.CreatedAt,
IdempotencyKey: opts.IdempotencyKey,
Files: []distributorbundle.BundleFile{
{SourcePath: opts.SourcePath, Path: opts.BundlePath},
@@ -193,6 +281,23 @@ func (c distributorUploadClient) UploadFiles(ctx context.Context, opts uploadFil
}, nil
}
func (c distributorUploadClient) Status(ctx context.Context, runID string) (runStatus, error) {
status, err := c.client.Status(ctx, runID)
if err != nil {
return runStatus{}, err
}
return runStatus{
RunID: status.RunID,
PipelineID: status.PipelineID,
Status: status.Status,
AcceptedAt: status.AcceptedAt,
StartedAt: status.StartedAt,
FinishedAt: status.FinishedAt,
Report: append(json.RawMessage(nil), status.Report...),
Error: status.Error,
}, nil
}
type uploadErrorContext struct {
Endpoint string
BundleID string

View File

@@ -2,6 +2,7 @@ package distributor
import (
"context"
"encoding/json"
"errors"
"fmt"
"strings"
@@ -23,6 +24,7 @@ func TestUploadUsesConfiguredClientAndSingleFile(t *testing.T) {
factory := &fakeUploadFactory{
client: &fakeUploadClient{
result: uploadFilesResult{RunID: "run-123", Status: "accepted"},
status: runStatus{RunID: "run-123", PipelineID: "reports", Status: "succeeded", Report: json.RawMessage(`{"actions":[{"action":"replace_older"}]}`)},
},
}
client := newClient(cfg, factory.newClient)
@@ -32,13 +34,17 @@ func TestUploadUsesConfiguredClientAndSingleFile(t *testing.T) {
IdempotencyKey: "weatherreporter.home.daily.run",
SourcePath: "/tmp/report.md",
BundlePath: "daily.md",
CreatedAt: time.Date(2026, 6, 7, 12, 0, 0, 123, time.UTC),
})
if err != nil {
t.Fatalf("Upload() error = %v", err)
}
if result.RunID != "run-123" || result.Status != "accepted" {
if result.RunID != "run-123" || result.Status != "succeeded" || result.UploadStatus != "accepted" {
t.Fatalf("result = %#v, want accepted run", result)
}
if result.RunStatus == nil || result.RunStatus.PipelineID != "reports" || !strings.Contains(string(result.RunStatus.Report), "replace_older") {
t.Fatalf("RunStatus = %#v, want parsed run report", result.RunStatus)
}
if factory.endpoint != cfg.Endpoint {
t.Fatalf("factory endpoint = %q, want %q", factory.endpoint, cfg.Endpoint)
}
@@ -61,6 +67,12 @@ func TestUploadUsesConfiguredClientAndSingleFile(t *testing.T) {
if got.BundlePath != "daily.md" {
t.Fatalf("BundlePath = %q, want daily.md", got.BundlePath)
}
if got.CreatedAt.IsZero() {
t.Fatal("CreatedAt is zero, want generated report timestamp")
}
if factory.client.statusRunID != "run-123" {
t.Fatalf("Status runID = %q, want run-123", factory.client.statusRunID)
}
}
func TestUploadRejectsMissingInputs(t *testing.T) {
@@ -168,6 +180,109 @@ func TestUploadWrapsUploadFailureWithContextWithoutToken(t *testing.T) {
}
}
func TestUploadReturnsAcceptedWhenStatusLookupFails(t *testing.T) {
cfg := config.Defaults().Notify.Distributor
cfg.Endpoint = "https://distributor.example.test"
t.Setenv(cfg.TokenEnv, "secret-token")
factory := &fakeUploadFactory{
client: &fakeUploadClient{
result: uploadFilesResult{RunID: "run-123", Status: "accepted"},
statusErr: fmt.Errorf("status rejected secret-token"),
},
}
client := newClient(cfg, factory.newClient)
result, err := client.Upload(context.Background(), validUploadRequest())
if err != nil {
t.Fatalf("Upload() error = %v, want accepted upload despite status lookup failure", err)
}
if result.Status != "accepted" || result.StatusError == "" {
t.Fatalf("result = %#v, want accepted status with status error", result)
}
if strings.Contains(result.StatusError, "secret-token") {
t.Fatalf("StatusError = %q, want token redacted", result.StatusError)
}
}
func TestUploadPollsUntilTerminalStatus(t *testing.T) {
cfg := config.Defaults().Notify.Distributor
cfg.Endpoint = "https://distributor.example.test"
cfg.Timeout = 2 * time.Second
t.Setenv(cfg.TokenEnv, "secret-token")
factory := &fakeUploadFactory{
client: &fakeUploadClient{
result: uploadFilesResult{RunID: "run-123", Status: "accepted"},
statuses: []runStatus{
{RunID: "run-123", Status: "accepted"},
{RunID: "run-123", Status: "succeeded", Report: json.RawMessage(`{"actions":[{"action":"replace_older"}]}`)},
},
},
}
client := newClient(cfg, factory.newClient)
result, err := client.Upload(context.Background(), validUploadRequest())
if err != nil {
t.Fatalf("Upload() error = %v", err)
}
if result.Status != "succeeded" || result.RunStatus == nil || !strings.Contains(string(result.RunStatus.Report), "replace_older") {
t.Fatalf("result = %#v, want terminal succeeded status with run report", result)
}
if factory.client.statusCalls != 2 {
t.Fatalf("status calls = %d, want 2", factory.client.statusCalls)
}
}
func TestUploadReturnsLatestStatusWhenPollingTimesOut(t *testing.T) {
cfg := config.Defaults().Notify.Distributor
cfg.Endpoint = "https://distributor.example.test"
cfg.Timeout = time.Millisecond
t.Setenv(cfg.TokenEnv, "secret-token")
factory := &fakeUploadFactory{
client: &fakeUploadClient{
result: uploadFilesResult{RunID: "run-123", Status: "accepted"},
status: runStatus{RunID: "run-123", Status: "running"},
},
}
client := newClient(cfg, factory.newClient)
result, err := client.Upload(context.Background(), validUploadRequest())
if err != nil {
t.Fatalf("Upload() error = %v, want accepted upload with status timeout recorded", err)
}
if result.Status != "running" || result.StatusError == "" {
t.Fatalf("result = %#v, want latest status and status timeout", result)
}
}
func TestUploadFailsWhenDistributorRunFailed(t *testing.T) {
cfg := config.Defaults().Notify.Distributor
cfg.Endpoint = "https://distributor.example.test"
t.Setenv(cfg.TokenEnv, "secret-token")
factory := &fakeUploadFactory{
client: &fakeUploadClient{
result: uploadFilesResult{RunID: "run-123", Status: "accepted"},
status: runStatus{
RunID: "run-123",
Status: "failed",
Error: "destination rejected secret-token",
Report: json.RawMessage(`{"actions":[{"action":"failed"}]}`),
},
},
}
client := newClient(cfg, factory.newClient)
result, err := client.Upload(context.Background(), validUploadRequest())
if err == nil {
t.Fatal("Upload() error = nil, want failed distributor run error")
}
if result.RunStatus == nil || result.RunStatus.Status != "failed" || !strings.Contains(string(result.RunStatus.Report), "failed") {
t.Fatalf("result = %#v, want failed run status report", result)
}
if strings.Contains(err.Error(), "secret-token") || strings.Contains(result.RunStatus.Error, "secret-token") {
t.Fatalf("error/result leaked token: err=%q result=%#v", err.Error(), result)
}
}
func TestUploadPreservesIdempotencyConflictDiagnosis(t *testing.T) {
cfg := config.Defaults().Notify.Distributor
cfg.Endpoint = "https://distributor.example.test"
@@ -207,6 +322,7 @@ func validUploadRequest() UploadRequest {
IdempotencyKey: "weatherreporter.home.daily.run",
SourcePath: "/tmp/report.md",
BundlePath: "daily.md",
CreatedAt: time.Date(2026, 6, 7, 12, 0, 0, 123, time.UTC),
}
}
@@ -229,9 +345,14 @@ func (f *fakeUploadFactory) newClient(endpoint, token string, timeout time.Durat
}
type fakeUploadClient struct {
opts uploadFilesOptions
result uploadFilesResult
err error
opts uploadFilesOptions
statusRunID string
statusCalls int
result uploadFilesResult
status runStatus
statuses []runStatus
err error
statusErr error
}
func (c *fakeUploadClient) UploadFiles(ctx context.Context, opts uploadFilesOptions) (uploadFilesResult, error) {
@@ -241,3 +362,19 @@ func (c *fakeUploadClient) UploadFiles(ctx context.Context, opts uploadFilesOpti
}
return c.result, nil
}
func (c *fakeUploadClient) Status(ctx context.Context, runID string) (runStatus, error) {
c.statusRunID = runID
c.statusCalls++
if c.statusErr != nil {
return runStatus{}, c.statusErr
}
if len(c.statuses) > 0 {
index := c.statusCalls - 1
if index >= len(c.statuses) {
index = len(c.statuses) - 1
}
return c.statuses[index], nil
}
return c.status, nil
}

View File

@@ -86,20 +86,21 @@ type BriefingResult struct {
}
type ReportResult struct {
Briefing briefing.Package
BriefingPath string
DataPackage promptinput.Package
DataPackagePath string
PreflightPath string
ReportPath string
OutputPath string
Metadata state.Metadata
MetadataPath string
PriorSnapshot *state.PriorSnapshot
RecentChanges []changes.Change
RenderResult *scriptorium.RenderResult
RunResult *scriptorium.RunResult
Notification *NotificationResult
Briefing briefing.Package
BriefingPath string
DataPackage promptinput.Package
DataPackagePath string
PreflightPath string
ReportPath string
OutputPath string
NotificationPath string
Metadata state.Metadata
MetadataPath string
PriorSnapshot *state.PriorSnapshot
RecentChanges []changes.Change
RenderResult *scriptorium.RenderResult
RunResult *scriptorium.RunResult
Notification *NotificationResult
}
type BatchResult struct {
@@ -122,6 +123,7 @@ type BatchReportResult struct {
NotificationStatus string `json:"notificationStatus,omitempty"`
NotificationRunID string `json:"notificationRunId,omitempty"`
NotificationError string `json:"notificationError,omitempty"`
NotificationPath string `json:"notificationPath,omitempty"`
GeneratedAt time.Time `json:"generatedAt"`
ValidPeriod timeutil.Period `json:"validPeriod"`
BriefingPath string `json:"briefingPath,omitempty"`
@@ -159,6 +161,7 @@ type NotificationRequest struct {
IdempotencyKey string
ReportPath string
BundlePath string
CreatedAt time.Time
}
type NotificationResult struct {
@@ -166,6 +169,14 @@ type NotificationResult struct {
IdempotencyKey string
RunID string
Status string
UploadStatus string
StatusError string
PipelineID string
AcceptedAt time.Time
StartedAt *time.Time
FinishedAt *time.Time
Report []byte
Error string
}
type NotificationError struct {
@@ -269,6 +280,9 @@ func RunBatchDetailed(ctx context.Context, req BatchRequest) (*BatchResult, erro
if errors.As(err, &notificationErr) {
item.NotificationStatus = "failed"
item.NotificationError = notificationErr.Error()
if paths, pathErr := store.Paths(resolved); pathErr == nil {
item.NotificationPath = paths.Notification
}
}
result.Failed++
} else {
@@ -279,6 +293,7 @@ func RunBatchDetailed(ctx context.Context, req BatchRequest) (*BatchResult, erro
item.ReportPath = reportResult.ReportPath
item.OutputPath = reportResult.OutputPath
item.MetadataPath = reportResult.MetadataPath
item.NotificationPath = reportResult.NotificationPath
if reportResult.Notification != nil {
item.NotificationStatus = reportResult.Notification.Status
item.NotificationRunID = reportResult.Notification.RunID
@@ -540,46 +555,62 @@ func GenerateReport(ctx context.Context, req ReportRequest) (*ReportResult, erro
return nil, runErr
}
notification, err := notifyReport(ctx, req.Config, req.Resolved, reportPath, metadata, req.Notifier)
notification, notificationPath, err := notifyReport(ctx, req.Config, req.Resolved, reportPath, metadata, req.Notifier, store)
if notificationPath != "" {
metadata.NotificationPath = notificationPath
metadataPath, metadataErr = store.SaveMetadata(ctx, metadata)
if metadataErr != nil {
return nil, metadataErr
}
}
if err != nil {
return nil, err
}
return &ReportResult{
Briefing: briefingPackage,
BriefingPath: briefingPath,
DataPackage: dataPackage,
DataPackagePath: dataPackagePath,
PreflightPath: preflightPath,
ReportPath: reportPath,
OutputPath: outputPath,
Metadata: metadata,
MetadataPath: metadataPath,
PriorSnapshot: priorSnapshot,
RecentChanges: recentChanges,
RenderResult: renderResult,
RunResult: runResult,
Notification: notification,
Briefing: briefingPackage,
BriefingPath: briefingPath,
DataPackage: dataPackage,
DataPackagePath: dataPackagePath,
PreflightPath: preflightPath,
ReportPath: reportPath,
OutputPath: outputPath,
NotificationPath: notificationPath,
Metadata: metadata,
MetadataPath: metadataPath,
PriorSnapshot: priorSnapshot,
RecentChanges: recentChanges,
RenderResult: renderResult,
RunResult: runResult,
Notification: notification,
}, nil
}
func notifyReport(ctx context.Context, cfg config.Config, resolved report.Resolved, reportPath string, metadata state.Metadata, notifier Notifier) (*NotificationResult, error) {
func notifyReport(ctx context.Context, cfg config.Config, resolved report.Resolved, reportPath string, metadata state.Metadata, notifier Notifier, store state.Store) (*NotificationResult, string, error) {
notifier, enabled := reportNotifier(cfg, notifier)
if !enabled {
return nil, nil
return nil, "", nil
}
notificationRequest, err := buildNotificationRequest(cfg, resolved, reportPath, metadata)
if err != nil {
return nil, err
notificationPath, saveErr := saveNotificationArtifact(ctx, store, resolved, cfg, metadata, NotificationRequest{}, nil, err)
if saveErr != nil {
return nil, "", saveErr
}
return nil, notificationPath, err
}
result, err := notifier.Notify(ctx, notificationRequest)
notificationPath, saveErr := saveNotificationArtifact(ctx, store, resolved, cfg, metadata, notificationRequest, result, err)
if saveErr != nil {
return nil, "", saveErr
}
if err != nil {
return nil, &NotificationError{
return result, notificationPath, &NotificationError{
Request: notificationRequest,
Err: fmt.Errorf("notify report %q run %q from managed report %q: %w", resolved.Definition.ID, metadata.RunID, reportPath, err),
}
}
return result, nil
return result, notificationPath, nil
}
func reportNotifier(cfg config.Config, notifier Notifier) (Notifier, bool) {
@@ -622,9 +653,57 @@ func buildNotificationRequest(cfg config.Config, resolved report.Resolved, repor
IdempotencyKey: idempotencyKey,
ReportPath: reportPath,
BundlePath: bundlePath,
CreatedAt: metadata.GeneratedAt,
}, nil
}
func saveNotificationArtifact(ctx context.Context, store state.Store, resolved report.Resolved, cfg config.Config, metadata state.Metadata, req NotificationRequest, result *NotificationResult, notifyErr error) (string, error) {
if store == nil {
return "", fmt.Errorf("state store is required")
}
artifact := state.DistributorNotificationArtifact{
SchemaVersion: state.DistributorNotificationSchemaVersion,
RunID: metadata.RunID,
ReportID: resolved.Definition.ID,
AttemptedAt: time.Now(),
Endpoint: cfg.Notify.Distributor.Endpoint,
BundleID: req.BundleID,
IdempotencyKey: req.IdempotencyKey,
SourcePath: req.ReportPath,
BundlePath: req.BundlePath,
BundleCreated: req.CreatedAt,
Status: "attempted",
}
if result != nil {
artifact.Status = result.Status
artifact.Upload = &state.DistributorUploadResult{
RunID: result.RunID,
Status: result.UploadStatus,
}
if result.PipelineID != "" || !result.AcceptedAt.IsZero() || result.StartedAt != nil || result.FinishedAt != nil || len(result.Report) > 0 || result.Error != "" {
artifact.RunStatus = &state.DistributorRunStatus{
RunID: result.RunID,
PipelineID: result.PipelineID,
Status: result.Status,
AcceptedAt: result.AcceptedAt,
StartedAt: result.StartedAt,
FinishedAt: result.FinishedAt,
Report: append([]byte(nil), result.Report...),
Error: result.Error,
}
}
artifact.StatusError = result.StatusError
}
if notifyErr != nil {
artifact.Status = "failed"
artifact.Error = notifyErr.Error()
}
if artifact.Status == "" {
artifact.Status = "unknown"
}
return store.SaveDistributorNotification(ctx, resolved, artifact)
}
type noopNotifier struct{}
func (noopNotifier) Notify(context.Context, NotificationRequest) (*NotificationResult, error) {
@@ -641,16 +720,28 @@ func (n distributorNotifier) Notify(ctx context.Context, req NotificationRequest
IdempotencyKey: req.IdempotencyKey,
SourcePath: req.ReportPath,
BundlePath: req.BundlePath,
CreatedAt: req.CreatedAt,
})
if err != nil {
return nil, err
}
return &NotificationResult{
notification := &NotificationResult{
BundleID: req.BundleID,
IdempotencyKey: req.IdempotencyKey,
RunID: result.RunID,
Status: result.Status,
}, nil
UploadStatus: result.UploadStatus,
StatusError: result.StatusError,
}
if result.RunStatus != nil {
notification.PipelineID = result.RunStatus.PipelineID
notification.AcceptedAt = result.RunStatus.AcceptedAt
notification.StartedAt = result.RunStatus.StartedAt
notification.FinishedAt = result.RunStatus.FinishedAt
notification.Report = append([]byte(nil), result.RunStatus.Report...)
notification.Error = result.RunStatus.Error
}
if err != nil {
return notification, err
}
return notification, nil
}
func BuildBriefing(req BriefingRequest, bundle *forecast.Bundle) (briefing.Package, error) {

View File

@@ -297,7 +297,13 @@ func TestGenerateReportNotifiesManagedReportPath(t *testing.T) {
t.Fatalf("ResolveGenerate() error = %v", err)
}
notifier := &recordingNotifier{
result: &NotificationResult{RunID: "distributor-run", Status: "accepted"},
result: &NotificationResult{
RunID: "distributor-run",
Status: "succeeded",
UploadStatus: "accepted",
PipelineID: "reports",
Report: []byte(`{"actions":[{"action":"replace_older"}]}`),
},
}
outputPath := filepath.Join(t.TempDir(), "daily-copy.md")
@@ -314,8 +320,18 @@ func TestGenerateReportNotifiesManagedReportPath(t *testing.T) {
if result.Notification == nil {
t.Fatal("Notification = nil, want notification result")
}
if result.Notification.RunID != "distributor-run" || result.Notification.Status != "accepted" {
t.Fatalf("Notification = %#v, want accepted distributor run", result.Notification)
if result.Notification.RunID != "distributor-run" || result.Notification.Status != "succeeded" {
t.Fatalf("Notification = %#v, want succeeded distributor run", result.Notification)
}
if result.NotificationPath == "" || result.Metadata.NotificationPath != result.NotificationPath {
t.Fatalf("NotificationPath result=%q metadata=%q, want linked artifact", result.NotificationPath, result.Metadata.NotificationPath)
}
notificationData, err := os.ReadFile(result.NotificationPath)
if err != nil {
t.Fatalf("read notification artifact: %v", err)
}
if !strings.Contains(string(notificationData), `"replace_older"`) || !strings.Contains(string(notificationData), `"bundleCreated"`) {
t.Fatalf("notification artifact missing status report or created timestamp:\n%s", string(notificationData))
}
if len(notifier.requests) != 1 {
t.Fatalf("notification requests = %d, want 1", len(notifier.requests))
@@ -339,6 +355,9 @@ func TestGenerateReportNotifiesManagedReportPath(t *testing.T) {
if req.RunID != result.Metadata.RunID {
t.Fatalf("notification RunID = %q, want report run id %q", req.RunID, result.Metadata.RunID)
}
if !req.CreatedAt.Equal(result.Metadata.GeneratedAt) {
t.Fatalf("notification CreatedAt = %s, want generated at %s", req.CreatedAt, result.Metadata.GeneratedAt)
}
}
func TestGenerateReportNotificationFailureFailsReport(t *testing.T) {
@@ -356,10 +375,15 @@ func TestGenerateReportNotificationFailureFailsReport(t *testing.T) {
}
notifier := &recordingNotifier{err: errors.New("upload rejected")}
store, err := state.NewFilesystemStore(cfg.Workspace)
if err != nil {
t.Fatalf("NewFilesystemStore() error = %v", err)
}
_, err = GenerateReport(context.Background(), ReportRequest{
Config: cfg,
Resolved: resolved,
Renderer: successfulRenderer("# Daily Report\n"),
Store: store,
Notifier: notifier,
})
if err == nil {
@@ -371,6 +395,21 @@ func TestGenerateReportNotificationFailureFailsReport(t *testing.T) {
if len(notifier.requests) != 1 {
t.Fatalf("notification requests = %d, want one attempted notification", len(notifier.requests))
}
paths, pathErr := store.Paths(resolved)
if pathErr != nil {
t.Fatalf("Paths() error = %v", pathErr)
}
notificationData, readErr := os.ReadFile(paths.Notification)
if readErr != nil {
t.Fatalf("read notification artifact after failure: %v", readErr)
}
var notification state.DistributorNotificationArtifact
if err := json.Unmarshal(notificationData, &notification); err != nil {
t.Fatalf("decode notification artifact: %v", err)
}
if notification.Status != "failed" || !strings.Contains(notification.Error, "upload rejected") {
t.Fatalf("notification failure artifact = %+v, want failed upload context", notification)
}
}
func TestGenerateReportDoesNotNotifyAfterRenderOrRunFailure(t *testing.T) {
@@ -1467,6 +1506,7 @@ func (n *recordingNotifier) Notify(_ context.Context, req NotificationRequest) (
BundleID: req.BundleID,
IdempotencyKey: req.IdempotencyKey,
Status: "accepted",
UploadStatus: "accepted",
}, nil
}

View File

@@ -75,11 +75,12 @@ type ScriptoriumConfig struct {
}
type WorkspaceConfig struct {
Root string `yaml:"root"`
SnapshotsDir string `yaml:"snapshots_dir"`
ReportsDir string `yaml:"reports_dir"`
DataPackagesDir string `yaml:"data_packages_dir"`
PreflightDir string `yaml:"preflight_dir"`
Root string `yaml:"root"`
SnapshotsDir string `yaml:"snapshots_dir"`
ReportsDir string `yaml:"reports_dir"`
DataPackagesDir string `yaml:"data_packages_dir"`
PreflightDir string `yaml:"preflight_dir"`
NotificationsDir string `yaml:"notifications_dir"`
}
type DaypartConfig struct {

View File

@@ -96,6 +96,9 @@ func TestLoadMinimalExampleConfig(t *testing.T) {
if cfg.Workspace.Root != "workspace" {
t.Fatalf("Workspace.Root = %q, want default workspace", cfg.Workspace.Root)
}
if cfg.Workspace.NotificationsDir != "notifications" {
t.Fatalf("Workspace.NotificationsDir = %q, want notifications", cfg.Workspace.NotificationsDir)
}
if cfg.Location.Name != "Brentwood" {
t.Fatalf("Location.Name = %q, want default Brentwood", cfg.Location.Name)
}

View File

@@ -42,11 +42,12 @@ func Defaults() Config {
Timeout: 2 * time.Minute,
},
Workspace: WorkspaceConfig{
Root: "workspace",
SnapshotsDir: "snapshots",
ReportsDir: "reports",
DataPackagesDir: "data-packages",
PreflightDir: "preflight",
Root: "workspace",
SnapshotsDir: "snapshots",
ReportsDir: "reports",
DataPackagesDir: "data-packages",
PreflightDir: "preflight",
NotificationsDir: "notifications",
},
Dayparts: []DaypartConfig{
{Name: "overnight", Start: "00:00", End: "06:00"},

View File

@@ -18,11 +18,12 @@ import (
)
type FilesystemStore struct {
root string
snapshotsDir string
reportsDir string
dataPackagesDir string
preflightDir string
root string
snapshotsDir string
reportsDir string
dataPackagesDir string
preflightDir string
notificationsDir string
}
type ArtifactPaths struct {
@@ -30,6 +31,7 @@ type ArtifactPaths struct {
Metadata string `json:"metadata"`
DataPackage string `json:"dataPackage"`
Preflight string `json:"preflight"`
Notification string `json:"notification,omitempty"`
RenderedReport string `json:"renderedReport,omitempty"`
}
@@ -57,17 +59,19 @@ func NewFilesystemStore(cfg config.WorkspaceConfig) (*FilesystemStore, error) {
"reports_dir": cfg.ReportsDir,
"data_packages_dir": cfg.DataPackagesDir,
"preflight_dir": cfg.PreflightDir,
"notifications_dir": cfg.NotificationsDir,
} {
if err := validateRelativeDir(name, value); err != nil {
return nil, err
}
}
return &FilesystemStore{
root: filepath.Clean(cfg.Root),
snapshotsDir: filepath.Clean(cfg.SnapshotsDir),
reportsDir: filepath.Clean(cfg.ReportsDir),
dataPackagesDir: filepath.Clean(cfg.DataPackagesDir),
preflightDir: filepath.Clean(cfg.PreflightDir),
root: filepath.Clean(cfg.Root),
snapshotsDir: filepath.Clean(cfg.SnapshotsDir),
reportsDir: filepath.Clean(cfg.ReportsDir),
dataPackagesDir: filepath.Clean(cfg.DataPackagesDir),
preflightDir: filepath.Clean(cfg.PreflightDir),
notificationsDir: filepath.Clean(cfg.NotificationsDir),
}, nil
}
@@ -90,6 +94,7 @@ func (s *FilesystemStore) Paths(resolved report.Resolved) (ArtifactPaths, error)
Metadata: s.join(s.snapshotsDir, group, validDate, filenameBase+".metadata.json"),
DataPackage: s.join(s.dataPackagesDir, group, validDate, filenameBase+".data_package.json"),
Preflight: s.join(s.preflightDir, group, validDate, filenameBase+".render.json"),
Notification: s.join(s.notificationsDir, group, validDate, filenameBase+".distributor.json"),
RenderedReport: s.join(s.reportsDir, group, filenameBase+".md"),
}, nil
}
@@ -130,6 +135,20 @@ func (s *FilesystemStore) SavePreflight(_ context.Context, resolved report.Resol
return paths.Preflight, nil
}
func (s *FilesystemStore) SaveDistributorNotification(_ context.Context, resolved report.Resolved, artifact DistributorNotificationArtifact) (string, error) {
paths, err := s.Paths(resolved)
if err != nil {
return "", err
}
if artifact.SchemaVersion == "" {
artifact.SchemaVersion = DistributorNotificationSchemaVersion
}
if err := fileutil.WriteJSONAtomic(paths.Notification, artifact); err != nil {
return "", err
}
return paths.Notification, nil
}
func (s *FilesystemStore) PrepareRenderedReport(_ context.Context, resolved report.Resolved) (string, error) {
paths, err := s.Paths(resolved)
if err != nil {

View File

@@ -30,6 +30,7 @@ func TestPathsUseRunIDAndWorkspace(t *testing.T) {
filepath.Join("snapshots", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.metadata.json"),
filepath.Join("data-packages", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.data_package.json"),
filepath.Join("preflight", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.render.json"),
filepath.Join("notifications", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.distributor.json"),
filepath.Join("reports", "daily", "20260529T100000.000000000Z_daily_today.md"),
} {
if !strings.Contains(pathsString(paths), want) {
@@ -59,6 +60,22 @@ func TestSaveArtifactsAndMetadataRoundTrip(t *testing.T) {
if err != nil {
t.Fatalf("SavePreflight() error = %v", err)
}
notificationPath, err := store.SaveDistributorNotification(context.Background(), resolved, DistributorNotificationArtifact{
RunID: resolved.Metadata().RunID,
ReportID: resolved.Definition.ID,
AttemptedAt: resolved.GeneratedAt,
Endpoint: "https://distributor.example.test",
BundleID: "weatherreporter.home.daily.run",
IdempotencyKey: "weatherreporter.home.daily.run",
SourcePath: "/tmp/report.md",
BundlePath: "daily.md",
BundleCreated: resolved.GeneratedAt,
Status: "succeeded",
RunStatus: &DistributorRunStatus{RunID: "distributor-run", Status: "succeeded"},
})
if err != nil {
t.Fatalf("SaveDistributorNotification() error = %v", err)
}
renderedReportPath, err := store.PrepareRenderedReport(context.Background(), resolved)
if err != nil {
t.Fatalf("PrepareRenderedReport() error = %v", err)
@@ -77,6 +94,17 @@ func TestSaveArtifactsAndMetadataRoundTrip(t *testing.T) {
if preflight.Stdout != `{"ok":true}` {
t.Fatalf("preflight stdout = %q, want render stdout", preflight.Stdout)
}
var notification DistributorNotificationArtifact
notificationData, err := os.ReadFile(notificationPath)
if err != nil {
t.Fatalf("read notification: %v", err)
}
if err := json.Unmarshal(notificationData, &notification); err != nil {
t.Fatalf("decode notification: %v", err)
}
if notification.SchemaVersion != DistributorNotificationSchemaVersion || notification.RunStatus == nil || notification.RunStatus.Status != "succeeded" {
t.Fatalf("notification = %#v, want persisted distributor status", notification)
}
paths, err := store.Paths(resolved)
if err != nil {
t.Fatalf("Paths() error = %v", err)
@@ -93,7 +121,7 @@ func TestSaveArtifactsAndMetadataRoundTrip(t *testing.T) {
t.Fatalf("SaveMetadata() error = %v", err)
}
for _, path := range []string{briefingPath, dataPackagePath, preflightPath, renderedReportPath, metadataPath} {
for _, path := range []string{briefingPath, dataPackagePath, preflightPath, notificationPath, renderedReportPath, metadataPath} {
if _, err := os.Stat(path); err != nil {
t.Fatalf("expected artifact %q: %v", path, err)
}
@@ -494,6 +522,7 @@ func pathsString(paths ArtifactPaths) string {
paths.Metadata,
paths.DataPackage,
paths.Preflight,
paths.Notification,
paths.RenderedReport,
}, "\n")
}

View File

@@ -29,6 +29,7 @@ type Metadata struct {
BriefingPath string `json:"briefingPath"`
DataPackagePath string `json:"dataPackagePath"`
PreflightPath string `json:"preflightPath"`
NotificationPath string `json:"notificationPath,omitempty"`
RenderedReportPath string `json:"renderedReportPath,omitempty"`
}

View File

@@ -3,6 +3,8 @@ package state
import (
"context"
"encoding/json"
"time"
"gitea.maximumdirect.net/eric/weatherreporter/internal/briefing"
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
@@ -14,6 +16,7 @@ type Store interface {
SaveBriefing(context.Context, report.Resolved, briefing.Package) (string, error)
SaveDataPackage(context.Context, report.Resolved, promptinput.Package) (string, error)
SavePreflight(context.Context, report.Resolved, PreflightArtifact) (string, error)
SaveDistributorNotification(context.Context, report.Resolved, DistributorNotificationArtifact) (string, error)
PrepareRenderedReport(context.Context, report.Resolved) (string, error)
SaveMetadata(context.Context, Metadata) (string, error)
FindPriorSnapshot(context.Context, report.Resolved) (*PriorSnapshot, error)
@@ -33,3 +36,39 @@ type PreflightArtifact struct {
StderrTruncated bool `json:"stderrTruncated,omitempty"`
ExitCode int `json:"exitCode"`
}
const DistributorNotificationSchemaVersion = "weatherreporter.distributor_notification.v1"
type DistributorNotificationArtifact struct {
SchemaVersion string `json:"schemaVersion"`
RunID string `json:"runId"`
ReportID report.ID `json:"reportId"`
AttemptedAt time.Time `json:"attemptedAt"`
Endpoint string `json:"endpoint"`
BundleID string `json:"bundleId,omitempty"`
IdempotencyKey string `json:"idempotencyKey,omitempty"`
SourcePath string `json:"sourcePath,omitempty"`
BundlePath string `json:"bundlePath,omitempty"`
BundleCreated time.Time `json:"bundleCreated,omitempty"`
Status string `json:"status"`
Upload *DistributorUploadResult `json:"upload,omitempty"`
RunStatus *DistributorRunStatus `json:"runStatus,omitempty"`
StatusError string `json:"statusError,omitempty"`
Error string `json:"error,omitempty"`
}
type DistributorUploadResult struct {
RunID string `json:"runId,omitempty"`
Status string `json:"status,omitempty"`
}
type DistributorRunStatus struct {
RunID string `json:"runId,omitempty"`
PipelineID string `json:"pipelineId,omitempty"`
Status string `json:"status,omitempty"`
AcceptedAt time.Time `json:"acceptedAt,omitempty"`
StartedAt *time.Time `json:"startedAt,omitempty"`
FinishedAt *time.Time `json:"finishedAt,omitempty"`
Report json.RawMessage `json:"report,omitempty"`
Error string `json:"error,omitempty"`
}