Implemented debug artifacts for the distributor notification adapter

This commit is contained in:
2026-06-07 20:21:26 -05:00
parent 138bc4e7e4
commit 183b23cf5a
18 changed files with 601 additions and 77 deletions

View File

@@ -134,6 +134,7 @@ stub source slots use the missing-source policy.
- `reports_dir`: managed Markdown report directory under `workspace.root`. Default: `reports`. - `reports_dir`: managed Markdown report directory under `workspace.root`. Default: `reports`.
- `data_packages_dir`: prompt input package directory under `workspace.root`. Default: `data-packages`. - `data_packages_dir`: prompt input package directory under `workspace.root`. Default: `data-packages`.
- `preflight_dir`: Scriptorium render output directory under `workspace.root`. Default: `preflight`. - `preflight_dir`: Scriptorium render output directory under `workspace.root`. Default: `preflight`.
- `notifications_dir`: distributor notification debug artifact directory under `workspace.root`. Default: `notifications`.
Workspace subdirectories must be relative paths that stay inside Workspace subdirectories must be relative paths that stay inside
`workspace.root`. `workspace.root`.

View File

@@ -81,12 +81,17 @@ Single-report generation follows this order:
14. Save metadata with the managed report path. 14. Save metadata with the managed report path.
15. If distributor notification is enabled, notify using the managed report 15. If distributor notification is enabled, notify using the managed report
path as the source file. path as the source file.
16. Save a distributor notification debug artifact and update metadata with its
path.
If render preflight returns both a result and an error, preflight JSON and If render preflight returns both a result and an error, preflight JSON and
metadata are persisted before the error is returned. If Scriptorium report metadata are persisted before the error is returned. If Scriptorium report
generation returns an error after writing output, the managed report and generation returns an error after writing output, the managed report and
metadata remain inspectable. Notification is not attempted after Weather API, metadata remain inspectable. Notification is not attempted after Weather API,
briefing, prompt input, render, Scriptorium run, or metadata-save failures. briefing, prompt input, render, Scriptorium run, or metadata-save failures.
When notification is attempted, the debug artifact records request identity,
accepted upload fields, distributor status fields, raw status report JSON when
available, and redacted failure context.
`--out` copies are never used as notification source files. `--out` copies are never used as notification source files.
## Batch Workflow ## Batch Workflow

View File

@@ -8,7 +8,8 @@ This document describes the distributor upload adapter in
The adapter submits generated weatherreporter Markdown reports to a configured The adapter submits generated weatherreporter Markdown reports to a configured
distributor HTTP upload endpoint. It isolates distributor package types, distributor HTTP upload endpoint. It isolates distributor package types,
token-env lookup, upload client construction, source-bundle file mapping, token-env lookup, upload client construction, source-bundle file mapping,
timeout handling, and upload error wrapping from app orchestration. timeout handling, status polling, and upload error wrapping from app
orchestration.
## Inputs And Outputs ## Inputs And Outputs
@@ -21,12 +22,14 @@ Inputs:
- idempotency key - idempotency key
- source Markdown report path - source Markdown report path
- bundle-relative Markdown path - bundle-relative Markdown path
- bundle created timestamp
- context for cancellation - context for cancellation
Outputs: Outputs:
- accepted distributor run ID - accepted distributor run ID
- accepted distributor status - accepted distributor upload status
- distributor run status, status polling error, and raw run report JSON when available
- weatherreporter-owned idempotency conflict error when applicable - weatherreporter-owned idempotency conflict error when applicable
## Boundaries ## Boundaries
@@ -66,11 +69,21 @@ The adapter calls distributor `UploadFiles` with exactly one file:
- source path: the managed Markdown report path selected by app orchestration - source path: the managed Markdown report path selected by app orchestration
- bundle path: the rendered bundle-relative report path - bundle path: the rendered bundle-relative report path
- created: the report generation timestamp
The adapter creates a distributor upload client with the configured endpoint, The adapter creates a distributor upload client with the configured endpoint,
bearer token, and timeout-backed HTTP client. It also wraps the upload context bearer token, and timeout-backed HTTP client. It also wraps the upload context
with the configured timeout when the timeout is greater than zero. with the configured timeout when the timeout is greater than zero.
After upload acceptance, the adapter polls distributor `Status` for the accepted
run ID until the run reaches `succeeded` or `failed`, or until the configured
timeout expires. It returns the latest status, error text, and raw report JSON in
weatherreporter-owned types so app orchestration can persist them in the
notification debug artifact. Status lookup failures or timeout before a terminal
state are kept as debug status errors on an otherwise accepted upload. A
terminal distributor run status of `failed` is returned as a notification failure
with the status report preserved.
## Failure Behavior ## Failure Behavior
The adapter validates required endpoint, token env name, token value, bundle ID, The adapter validates required endpoint, token env name, token value, bundle ID,

View File

@@ -47,6 +47,7 @@ converts adapter render results into that shape before saving.
- `workspace.reports_dir` - `workspace.reports_dir`
- `workspace.data_packages_dir` - `workspace.data_packages_dir`
- `workspace.preflight_dir` - `workspace.preflight_dir`
- `workspace.notifications_dir`
Workspace subdirectories must be relative paths that stay under Workspace subdirectories must be relative paths that stay under
`workspace.root`. `workspace.root`.
@@ -62,12 +63,14 @@ valid-period start date for JSON artifacts, and the RunID.
snapshots/<artifact_group>/<YYYY-MM-DD>/<run_id>.metadata.json snapshots/<artifact_group>/<YYYY-MM-DD>/<run_id>.metadata.json
data-packages/<artifact_group>/<YYYY-MM-DD>/<run_id>.data_package.json data-packages/<artifact_group>/<YYYY-MM-DD>/<run_id>.data_package.json
preflight/<artifact_group>/<YYYY-MM-DD>/<run_id>.render.json preflight/<artifact_group>/<YYYY-MM-DD>/<run_id>.render.json
notifications/<artifact_group>/<YYYY-MM-DD>/<run_id>.distributor.json
reports/<artifact_group>/<run_id>.md reports/<artifact_group>/<run_id>.md
``` ```
Metadata is stored beside briefing snapshots and links the briefing, data Metadata is stored beside briefing snapshots and links the briefing, data
package, preflight, report paths, and configured prompt location. Report package, preflight, report paths, notification path when attempted, and
listing walks metadata files under the snapshots directory. configured prompt location. Report listing walks metadata files under the
snapshots directory.
## Prior Lookup ## Prior Lookup
@@ -89,6 +92,8 @@ current report definition.
Durable JSON writes use shared atomic file helpers. Managed Markdown reports are Durable JSON writes use shared atomic file helpers. Managed Markdown reports are
prepared by creating their parent directory; Scriptorium writes the report body prepared by creating their parent directory; Scriptorium writes the report body
to the prepared path. Extra Markdown copies are handled by app orchestration. to the prepared path. Extra Markdown copies are handled by app orchestration.
Distributor notification debug artifacts are written atomically when
notification is attempted.
Inspection helpers read existing metadata, briefing, and data package files. Inspection helpers read existing metadata, briefing, and data package files.
Missing metadata directories return no inspection records or no prior snapshot Missing metadata directories return no inspection records or no prior snapshot

View File

@@ -91,6 +91,19 @@ workspace/
storm/ storm/
YYYY-MM-DD/ YYYY-MM-DD/
<run_id>.render.json <run_id>.render.json
notifications/
daily/
YYYY-MM-DD/
<run_id>.distributor.json
three-day/
YYYY-MM-DD/
<run_id>.distributor.json
weekend/
YYYY-MM-DD/
<run_id>.distributor.json
storm/
YYYY-MM-DD/
<run_id>.distributor.json
reports/ reports/
daily/ daily/
<run_id>.md <run_id>.md
@@ -122,6 +135,7 @@ Each generated report writes metadata that links:
- prompt input data package path - prompt input data package path
- preflight output path - preflight output path
- managed Markdown report path - managed Markdown report path
- distributor notification debug artifact path, when notification is attempted
Batch summaries include report status, error text when applicable, notification Batch summaries include report status, error text when applicable, notification
outcome when attempted, valid period, and known artifact paths for each outcome when attempted, valid period, and known artifact paths for each
@@ -153,6 +167,16 @@ not trigger notification. A notification failure fails that report. In a batch,
other reports continue, the failed report includes notification fields in the other reports continue, the failed report includes notification fields in the
JSON summary, and the batch returns nonzero. JSON summary, and the batch returns nonzero.
Each notification attempt writes a debug artifact under `notifications/`. The
artifact records the rendered bundle ID, idempotency key, managed source path,
bundle-relative path, bundle created timestamp, accepted upload response, and
the latest distributor run status response when available. Weatherreporter polls
status until distributor reports `succeeded` or `failed`, or until the configured
notification timeout expires. The run status includes the distributor status,
error text, and raw run report JSON, which can show actions such as
`replace_older`, `skip_same`, `skip_destination_newer`, or `failed`. Token values
are not written.
Weatherreporter is responsible for selecting the managed Markdown report, Weatherreporter is responsible for selecting the managed Markdown report,
constructing a source bundle, and submitting it to the configured distributor constructing a source bundle, and submitting it to the configured distributor
HTTP endpoint. Distributor remains responsible for destination routing, HTTP endpoint. Distributor remains responsible for destination routing,

View File

@@ -248,6 +248,10 @@ Diagnostic: inspect the failed batch JSON or stderr line for bundle and
idempotency context. Compare the configured templates with the report RunID and idempotency context. Compare the configured templates with the report RunID and
report path. report path.
Also inspect the notification artifact linked from metadata. It records the
rendered bundle ID, idempotency key, upload result, distributor run status,
status error, and raw run report JSON when available.
Safe fix: keep idempotency templates stable for retries of the same generated Safe fix: keep idempotency templates stable for retries of the same generated
report, but do not reuse the same rendered key for different generated report report, but do not reuse the same rendered key for different generated report
content. content.
@@ -268,6 +272,11 @@ Diagnostic: inspect stdout JSON or stderr status lines for
`notify.distributor.report_path_template`, and token configuration. Token values `notify.distributor.report_path_template`, and token configuration. Token values
are redacted from weatherreporter errors. are redacted from weatherreporter errors.
If the upload was accepted but destination output did not change, inspect the
notification artifact's `runStatus.report`. Distributor actions such as
`replace_older`, `skip_same`, `skip_destination_newer`, or `failed` explain how
the destination handled the uploaded bundle.
Safe fix: fix the endpoint, token, templates, or distributor-side upload Safe fix: fix the endpoint, token, templates, or distributor-side upload
configuration. The weatherreporter upload source is the managed Markdown report, configuration. The weatherreporter upload source is the managed Markdown report,
not `--out` or `--out-dir` copies. not `--out` or `--out-dir` copies.

View File

@@ -40,6 +40,7 @@ workspace:
reports_dir: reports reports_dir: reports
data_packages_dir: data-packages data_packages_dir: data-packages
preflight_dir: preflight preflight_dir: preflight
notifications_dir: notifications
dayparts: dayparts:
- name: overnight - name: overnight

View File

@@ -3,6 +3,7 @@ package distributor
import ( import (
"context" "context"
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"net/http" "net/http"
@@ -28,11 +29,26 @@ type UploadRequest struct {
IdempotencyKey string IdempotencyKey string
SourcePath string SourcePath string
BundlePath string BundlePath string
CreatedAt time.Time
} }
type UploadResult struct { type UploadResult struct {
RunID string RunID string
Status string Status string
UploadStatus string
StatusError string
RunStatus *RunStatus
}
type RunStatus struct {
RunID string
PipelineID string
Status string
AcceptedAt time.Time
StartedAt *time.Time
FinishedAt *time.Time
Report json.RawMessage
Error string
} }
type IdempotencyConflictError struct { type IdempotencyConflictError struct {
@@ -57,6 +73,7 @@ type uploadClientFactory func(endpoint, token string, timeout time.Duration) (up
type uploadClient interface { type uploadClient interface {
UploadFiles(ctx context.Context, opts uploadFilesOptions) (uploadFilesResult, error) UploadFiles(ctx context.Context, opts uploadFilesOptions) (uploadFilesResult, error)
Status(ctx context.Context, runID string) (runStatus, error)
} }
type uploadFilesOptions struct { type uploadFilesOptions struct {
@@ -64,6 +81,7 @@ type uploadFilesOptions struct {
IdempotencyKey string IdempotencyKey string
SourcePath string SourcePath string
BundlePath string BundlePath string
CreatedAt time.Time
} }
type uploadFilesResult struct { type uploadFilesResult struct {
@@ -71,6 +89,19 @@ type uploadFilesResult struct {
Status string Status string
} }
type runStatus struct {
RunID string
PipelineID string
Status string
AcceptedAt time.Time
StartedAt *time.Time
FinishedAt *time.Time
Report json.RawMessage
Error string
}
const statusPollInterval = 250 * time.Millisecond
func New(cfg config.DistributorNotifyConfig) *Client { func New(cfg config.DistributorNotifyConfig) *Client {
return newClient(cfg, newDistributorUploadClient) return newClient(cfg, newDistributorUploadClient)
} }
@@ -138,6 +169,7 @@ func (c *Client) Upload(ctx context.Context, req UploadRequest) (UploadResult, e
IdempotencyKey: req.IdempotencyKey, IdempotencyKey: req.IdempotencyKey,
SourcePath: req.SourcePath, SourcePath: req.SourcePath,
BundlePath: req.BundlePath, BundlePath: req.BundlePath,
CreatedAt: req.CreatedAt,
}) })
if err != nil { if err != nil {
return UploadResult{}, wrapUploadError(err, uploadErrorContext{ return UploadResult{}, wrapUploadError(err, uploadErrorContext{
@@ -150,10 +182,65 @@ func (c *Client) Upload(ctx context.Context, req UploadRequest) (UploadResult, e
}) })
} }
return UploadResult{ uploadResult := UploadResult{
RunID: result.RunID, RunID: result.RunID,
Status: result.Status, Status: result.Status,
}, nil UploadStatus: result.Status,
}
status, statusErr := waitForRunStatus(runCtx, uploadClient, result.RunID, c.Timeout > 0)
if status.RunID != "" || status.Status != "" {
uploadResult.RunStatus = &RunStatus{
RunID: status.RunID,
PipelineID: status.PipelineID,
Status: status.Status,
AcceptedAt: status.AcceptedAt,
StartedAt: status.StartedAt,
FinishedAt: status.FinishedAt,
Report: append(json.RawMessage(nil), status.Report...),
Error: redactTokenString(status.Error, token),
}
if status.Status != "" {
uploadResult.Status = status.Status
}
}
if statusErr != nil {
uploadResult.StatusError = redactTokenString(statusErr.Error(), token)
return uploadResult, nil
}
if status.Status == "failed" {
return uploadResult, fmt.Errorf("distributor run %q failed: %s", status.RunID, uploadResult.RunStatus.Error)
}
return uploadResult, nil
}
func waitForRunStatus(ctx context.Context, client uploadClient, runID string, poll bool) (runStatus, error) {
status, err := client.Status(ctx, runID)
if err != nil || terminalRunStatus(status.Status) || !poll {
return status, err
}
for {
timer := time.NewTimer(statusPollInterval)
select {
case <-ctx.Done():
timer.Stop()
return status, fmt.Errorf("distributor run %q did not reach terminal status before timeout: %w", runID, ctx.Err())
case <-timer.C:
}
next, err := client.Status(ctx, runID)
if err != nil {
return status, err
}
status = next
if terminalRunStatus(status.Status) {
return status, nil
}
}
}
func terminalRunStatus(status string) bool {
return status == "succeeded" || status == "failed"
} }
type distributorUploadClient struct { type distributorUploadClient struct {
@@ -179,6 +266,7 @@ func newDistributorUploadClient(endpoint, token string, timeout time.Duration) (
func (c distributorUploadClient) UploadFiles(ctx context.Context, opts uploadFilesOptions) (uploadFilesResult, error) { func (c distributorUploadClient) UploadFiles(ctx context.Context, opts uploadFilesOptions) (uploadFilesResult, error) {
result, err := c.client.UploadFiles(ctx, distributorupload.UploadFilesOptions{ result, err := c.client.UploadFiles(ctx, distributorupload.UploadFilesOptions{
ID: opts.BundleID, ID: opts.BundleID,
Created: opts.CreatedAt,
IdempotencyKey: opts.IdempotencyKey, IdempotencyKey: opts.IdempotencyKey,
Files: []distributorbundle.BundleFile{ Files: []distributorbundle.BundleFile{
{SourcePath: opts.SourcePath, Path: opts.BundlePath}, {SourcePath: opts.SourcePath, Path: opts.BundlePath},
@@ -193,6 +281,23 @@ func (c distributorUploadClient) UploadFiles(ctx context.Context, opts uploadFil
}, nil }, nil
} }
func (c distributorUploadClient) Status(ctx context.Context, runID string) (runStatus, error) {
status, err := c.client.Status(ctx, runID)
if err != nil {
return runStatus{}, err
}
return runStatus{
RunID: status.RunID,
PipelineID: status.PipelineID,
Status: status.Status,
AcceptedAt: status.AcceptedAt,
StartedAt: status.StartedAt,
FinishedAt: status.FinishedAt,
Report: append(json.RawMessage(nil), status.Report...),
Error: status.Error,
}, nil
}
type uploadErrorContext struct { type uploadErrorContext struct {
Endpoint string Endpoint string
BundleID string BundleID string

View File

@@ -2,6 +2,7 @@ package distributor
import ( import (
"context" "context"
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"strings" "strings"
@@ -23,6 +24,7 @@ func TestUploadUsesConfiguredClientAndSingleFile(t *testing.T) {
factory := &fakeUploadFactory{ factory := &fakeUploadFactory{
client: &fakeUploadClient{ client: &fakeUploadClient{
result: uploadFilesResult{RunID: "run-123", Status: "accepted"}, result: uploadFilesResult{RunID: "run-123", Status: "accepted"},
status: runStatus{RunID: "run-123", PipelineID: "reports", Status: "succeeded", Report: json.RawMessage(`{"actions":[{"action":"replace_older"}]}`)},
}, },
} }
client := newClient(cfg, factory.newClient) client := newClient(cfg, factory.newClient)
@@ -32,13 +34,17 @@ func TestUploadUsesConfiguredClientAndSingleFile(t *testing.T) {
IdempotencyKey: "weatherreporter.home.daily.run", IdempotencyKey: "weatherreporter.home.daily.run",
SourcePath: "/tmp/report.md", SourcePath: "/tmp/report.md",
BundlePath: "daily.md", BundlePath: "daily.md",
CreatedAt: time.Date(2026, 6, 7, 12, 0, 0, 123, time.UTC),
}) })
if err != nil { if err != nil {
t.Fatalf("Upload() error = %v", err) t.Fatalf("Upload() error = %v", err)
} }
if result.RunID != "run-123" || result.Status != "accepted" { if result.RunID != "run-123" || result.Status != "succeeded" || result.UploadStatus != "accepted" {
t.Fatalf("result = %#v, want accepted run", result) t.Fatalf("result = %#v, want accepted run", result)
} }
if result.RunStatus == nil || result.RunStatus.PipelineID != "reports" || !strings.Contains(string(result.RunStatus.Report), "replace_older") {
t.Fatalf("RunStatus = %#v, want parsed run report", result.RunStatus)
}
if factory.endpoint != cfg.Endpoint { if factory.endpoint != cfg.Endpoint {
t.Fatalf("factory endpoint = %q, want %q", factory.endpoint, cfg.Endpoint) t.Fatalf("factory endpoint = %q, want %q", factory.endpoint, cfg.Endpoint)
} }
@@ -61,6 +67,12 @@ func TestUploadUsesConfiguredClientAndSingleFile(t *testing.T) {
if got.BundlePath != "daily.md" { if got.BundlePath != "daily.md" {
t.Fatalf("BundlePath = %q, want daily.md", got.BundlePath) t.Fatalf("BundlePath = %q, want daily.md", got.BundlePath)
} }
if got.CreatedAt.IsZero() {
t.Fatal("CreatedAt is zero, want generated report timestamp")
}
if factory.client.statusRunID != "run-123" {
t.Fatalf("Status runID = %q, want run-123", factory.client.statusRunID)
}
} }
func TestUploadRejectsMissingInputs(t *testing.T) { func TestUploadRejectsMissingInputs(t *testing.T) {
@@ -168,6 +180,109 @@ func TestUploadWrapsUploadFailureWithContextWithoutToken(t *testing.T) {
} }
} }
func TestUploadReturnsAcceptedWhenStatusLookupFails(t *testing.T) {
cfg := config.Defaults().Notify.Distributor
cfg.Endpoint = "https://distributor.example.test"
t.Setenv(cfg.TokenEnv, "secret-token")
factory := &fakeUploadFactory{
client: &fakeUploadClient{
result: uploadFilesResult{RunID: "run-123", Status: "accepted"},
statusErr: fmt.Errorf("status rejected secret-token"),
},
}
client := newClient(cfg, factory.newClient)
result, err := client.Upload(context.Background(), validUploadRequest())
if err != nil {
t.Fatalf("Upload() error = %v, want accepted upload despite status lookup failure", err)
}
if result.Status != "accepted" || result.StatusError == "" {
t.Fatalf("result = %#v, want accepted status with status error", result)
}
if strings.Contains(result.StatusError, "secret-token") {
t.Fatalf("StatusError = %q, want token redacted", result.StatusError)
}
}
func TestUploadPollsUntilTerminalStatus(t *testing.T) {
cfg := config.Defaults().Notify.Distributor
cfg.Endpoint = "https://distributor.example.test"
cfg.Timeout = 2 * time.Second
t.Setenv(cfg.TokenEnv, "secret-token")
factory := &fakeUploadFactory{
client: &fakeUploadClient{
result: uploadFilesResult{RunID: "run-123", Status: "accepted"},
statuses: []runStatus{
{RunID: "run-123", Status: "accepted"},
{RunID: "run-123", Status: "succeeded", Report: json.RawMessage(`{"actions":[{"action":"replace_older"}]}`)},
},
},
}
client := newClient(cfg, factory.newClient)
result, err := client.Upload(context.Background(), validUploadRequest())
if err != nil {
t.Fatalf("Upload() error = %v", err)
}
if result.Status != "succeeded" || result.RunStatus == nil || !strings.Contains(string(result.RunStatus.Report), "replace_older") {
t.Fatalf("result = %#v, want terminal succeeded status with run report", result)
}
if factory.client.statusCalls != 2 {
t.Fatalf("status calls = %d, want 2", factory.client.statusCalls)
}
}
func TestUploadReturnsLatestStatusWhenPollingTimesOut(t *testing.T) {
cfg := config.Defaults().Notify.Distributor
cfg.Endpoint = "https://distributor.example.test"
cfg.Timeout = time.Millisecond
t.Setenv(cfg.TokenEnv, "secret-token")
factory := &fakeUploadFactory{
client: &fakeUploadClient{
result: uploadFilesResult{RunID: "run-123", Status: "accepted"},
status: runStatus{RunID: "run-123", Status: "running"},
},
}
client := newClient(cfg, factory.newClient)
result, err := client.Upload(context.Background(), validUploadRequest())
if err != nil {
t.Fatalf("Upload() error = %v, want accepted upload with status timeout recorded", err)
}
if result.Status != "running" || result.StatusError == "" {
t.Fatalf("result = %#v, want latest status and status timeout", result)
}
}
func TestUploadFailsWhenDistributorRunFailed(t *testing.T) {
cfg := config.Defaults().Notify.Distributor
cfg.Endpoint = "https://distributor.example.test"
t.Setenv(cfg.TokenEnv, "secret-token")
factory := &fakeUploadFactory{
client: &fakeUploadClient{
result: uploadFilesResult{RunID: "run-123", Status: "accepted"},
status: runStatus{
RunID: "run-123",
Status: "failed",
Error: "destination rejected secret-token",
Report: json.RawMessage(`{"actions":[{"action":"failed"}]}`),
},
},
}
client := newClient(cfg, factory.newClient)
result, err := client.Upload(context.Background(), validUploadRequest())
if err == nil {
t.Fatal("Upload() error = nil, want failed distributor run error")
}
if result.RunStatus == nil || result.RunStatus.Status != "failed" || !strings.Contains(string(result.RunStatus.Report), "failed") {
t.Fatalf("result = %#v, want failed run status report", result)
}
if strings.Contains(err.Error(), "secret-token") || strings.Contains(result.RunStatus.Error, "secret-token") {
t.Fatalf("error/result leaked token: err=%q result=%#v", err.Error(), result)
}
}
func TestUploadPreservesIdempotencyConflictDiagnosis(t *testing.T) { func TestUploadPreservesIdempotencyConflictDiagnosis(t *testing.T) {
cfg := config.Defaults().Notify.Distributor cfg := config.Defaults().Notify.Distributor
cfg.Endpoint = "https://distributor.example.test" cfg.Endpoint = "https://distributor.example.test"
@@ -207,6 +322,7 @@ func validUploadRequest() UploadRequest {
IdempotencyKey: "weatherreporter.home.daily.run", IdempotencyKey: "weatherreporter.home.daily.run",
SourcePath: "/tmp/report.md", SourcePath: "/tmp/report.md",
BundlePath: "daily.md", BundlePath: "daily.md",
CreatedAt: time.Date(2026, 6, 7, 12, 0, 0, 123, time.UTC),
} }
} }
@@ -229,9 +345,14 @@ func (f *fakeUploadFactory) newClient(endpoint, token string, timeout time.Durat
} }
type fakeUploadClient struct { type fakeUploadClient struct {
opts uploadFilesOptions opts uploadFilesOptions
result uploadFilesResult statusRunID string
err error statusCalls int
result uploadFilesResult
status runStatus
statuses []runStatus
err error
statusErr error
} }
func (c *fakeUploadClient) UploadFiles(ctx context.Context, opts uploadFilesOptions) (uploadFilesResult, error) { func (c *fakeUploadClient) UploadFiles(ctx context.Context, opts uploadFilesOptions) (uploadFilesResult, error) {
@@ -241,3 +362,19 @@ func (c *fakeUploadClient) UploadFiles(ctx context.Context, opts uploadFilesOpti
} }
return c.result, nil return c.result, nil
} }
func (c *fakeUploadClient) Status(ctx context.Context, runID string) (runStatus, error) {
c.statusRunID = runID
c.statusCalls++
if c.statusErr != nil {
return runStatus{}, c.statusErr
}
if len(c.statuses) > 0 {
index := c.statusCalls - 1
if index >= len(c.statuses) {
index = len(c.statuses) - 1
}
return c.statuses[index], nil
}
return c.status, nil
}

View File

@@ -86,20 +86,21 @@ type BriefingResult struct {
} }
type ReportResult struct { type ReportResult struct {
Briefing briefing.Package Briefing briefing.Package
BriefingPath string BriefingPath string
DataPackage promptinput.Package DataPackage promptinput.Package
DataPackagePath string DataPackagePath string
PreflightPath string PreflightPath string
ReportPath string ReportPath string
OutputPath string OutputPath string
Metadata state.Metadata NotificationPath string
MetadataPath string Metadata state.Metadata
PriorSnapshot *state.PriorSnapshot MetadataPath string
RecentChanges []changes.Change PriorSnapshot *state.PriorSnapshot
RenderResult *scriptorium.RenderResult RecentChanges []changes.Change
RunResult *scriptorium.RunResult RenderResult *scriptorium.RenderResult
Notification *NotificationResult RunResult *scriptorium.RunResult
Notification *NotificationResult
} }
type BatchResult struct { type BatchResult struct {
@@ -122,6 +123,7 @@ type BatchReportResult struct {
NotificationStatus string `json:"notificationStatus,omitempty"` NotificationStatus string `json:"notificationStatus,omitempty"`
NotificationRunID string `json:"notificationRunId,omitempty"` NotificationRunID string `json:"notificationRunId,omitempty"`
NotificationError string `json:"notificationError,omitempty"` NotificationError string `json:"notificationError,omitempty"`
NotificationPath string `json:"notificationPath,omitempty"`
GeneratedAt time.Time `json:"generatedAt"` GeneratedAt time.Time `json:"generatedAt"`
ValidPeriod timeutil.Period `json:"validPeriod"` ValidPeriod timeutil.Period `json:"validPeriod"`
BriefingPath string `json:"briefingPath,omitempty"` BriefingPath string `json:"briefingPath,omitempty"`
@@ -159,6 +161,7 @@ type NotificationRequest struct {
IdempotencyKey string IdempotencyKey string
ReportPath string ReportPath string
BundlePath string BundlePath string
CreatedAt time.Time
} }
type NotificationResult struct { type NotificationResult struct {
@@ -166,6 +169,14 @@ type NotificationResult struct {
IdempotencyKey string IdempotencyKey string
RunID string RunID string
Status string Status string
UploadStatus string
StatusError string
PipelineID string
AcceptedAt time.Time
StartedAt *time.Time
FinishedAt *time.Time
Report []byte
Error string
} }
type NotificationError struct { type NotificationError struct {
@@ -269,6 +280,9 @@ func RunBatchDetailed(ctx context.Context, req BatchRequest) (*BatchResult, erro
if errors.As(err, &notificationErr) { if errors.As(err, &notificationErr) {
item.NotificationStatus = "failed" item.NotificationStatus = "failed"
item.NotificationError = notificationErr.Error() item.NotificationError = notificationErr.Error()
if paths, pathErr := store.Paths(resolved); pathErr == nil {
item.NotificationPath = paths.Notification
}
} }
result.Failed++ result.Failed++
} else { } else {
@@ -279,6 +293,7 @@ func RunBatchDetailed(ctx context.Context, req BatchRequest) (*BatchResult, erro
item.ReportPath = reportResult.ReportPath item.ReportPath = reportResult.ReportPath
item.OutputPath = reportResult.OutputPath item.OutputPath = reportResult.OutputPath
item.MetadataPath = reportResult.MetadataPath item.MetadataPath = reportResult.MetadataPath
item.NotificationPath = reportResult.NotificationPath
if reportResult.Notification != nil { if reportResult.Notification != nil {
item.NotificationStatus = reportResult.Notification.Status item.NotificationStatus = reportResult.Notification.Status
item.NotificationRunID = reportResult.Notification.RunID item.NotificationRunID = reportResult.Notification.RunID
@@ -540,46 +555,62 @@ func GenerateReport(ctx context.Context, req ReportRequest) (*ReportResult, erro
return nil, runErr return nil, runErr
} }
notification, err := notifyReport(ctx, req.Config, req.Resolved, reportPath, metadata, req.Notifier) notification, notificationPath, err := notifyReport(ctx, req.Config, req.Resolved, reportPath, metadata, req.Notifier, store)
if notificationPath != "" {
metadata.NotificationPath = notificationPath
metadataPath, metadataErr = store.SaveMetadata(ctx, metadata)
if metadataErr != nil {
return nil, metadataErr
}
}
if err != nil { if err != nil {
return nil, err return nil, err
} }
return &ReportResult{ return &ReportResult{
Briefing: briefingPackage, Briefing: briefingPackage,
BriefingPath: briefingPath, BriefingPath: briefingPath,
DataPackage: dataPackage, DataPackage: dataPackage,
DataPackagePath: dataPackagePath, DataPackagePath: dataPackagePath,
PreflightPath: preflightPath, PreflightPath: preflightPath,
ReportPath: reportPath, ReportPath: reportPath,
OutputPath: outputPath, OutputPath: outputPath,
Metadata: metadata, NotificationPath: notificationPath,
MetadataPath: metadataPath, Metadata: metadata,
PriorSnapshot: priorSnapshot, MetadataPath: metadataPath,
RecentChanges: recentChanges, PriorSnapshot: priorSnapshot,
RenderResult: renderResult, RecentChanges: recentChanges,
RunResult: runResult, RenderResult: renderResult,
Notification: notification, RunResult: runResult,
Notification: notification,
}, nil }, nil
} }
func notifyReport(ctx context.Context, cfg config.Config, resolved report.Resolved, reportPath string, metadata state.Metadata, notifier Notifier) (*NotificationResult, error) { func notifyReport(ctx context.Context, cfg config.Config, resolved report.Resolved, reportPath string, metadata state.Metadata, notifier Notifier, store state.Store) (*NotificationResult, string, error) {
notifier, enabled := reportNotifier(cfg, notifier) notifier, enabled := reportNotifier(cfg, notifier)
if !enabled { if !enabled {
return nil, nil return nil, "", nil
} }
notificationRequest, err := buildNotificationRequest(cfg, resolved, reportPath, metadata) notificationRequest, err := buildNotificationRequest(cfg, resolved, reportPath, metadata)
if err != nil { if err != nil {
return nil, err notificationPath, saveErr := saveNotificationArtifact(ctx, store, resolved, cfg, metadata, NotificationRequest{}, nil, err)
if saveErr != nil {
return nil, "", saveErr
}
return nil, notificationPath, err
} }
result, err := notifier.Notify(ctx, notificationRequest) result, err := notifier.Notify(ctx, notificationRequest)
notificationPath, saveErr := saveNotificationArtifact(ctx, store, resolved, cfg, metadata, notificationRequest, result, err)
if saveErr != nil {
return nil, "", saveErr
}
if err != nil { if err != nil {
return nil, &NotificationError{ return result, notificationPath, &NotificationError{
Request: notificationRequest, Request: notificationRequest,
Err: fmt.Errorf("notify report %q run %q from managed report %q: %w", resolved.Definition.ID, metadata.RunID, reportPath, err), Err: fmt.Errorf("notify report %q run %q from managed report %q: %w", resolved.Definition.ID, metadata.RunID, reportPath, err),
} }
} }
return result, nil return result, notificationPath, nil
} }
func reportNotifier(cfg config.Config, notifier Notifier) (Notifier, bool) { func reportNotifier(cfg config.Config, notifier Notifier) (Notifier, bool) {
@@ -622,9 +653,57 @@ func buildNotificationRequest(cfg config.Config, resolved report.Resolved, repor
IdempotencyKey: idempotencyKey, IdempotencyKey: idempotencyKey,
ReportPath: reportPath, ReportPath: reportPath,
BundlePath: bundlePath, BundlePath: bundlePath,
CreatedAt: metadata.GeneratedAt,
}, nil }, nil
} }
func saveNotificationArtifact(ctx context.Context, store state.Store, resolved report.Resolved, cfg config.Config, metadata state.Metadata, req NotificationRequest, result *NotificationResult, notifyErr error) (string, error) {
if store == nil {
return "", fmt.Errorf("state store is required")
}
artifact := state.DistributorNotificationArtifact{
SchemaVersion: state.DistributorNotificationSchemaVersion,
RunID: metadata.RunID,
ReportID: resolved.Definition.ID,
AttemptedAt: time.Now(),
Endpoint: cfg.Notify.Distributor.Endpoint,
BundleID: req.BundleID,
IdempotencyKey: req.IdempotencyKey,
SourcePath: req.ReportPath,
BundlePath: req.BundlePath,
BundleCreated: req.CreatedAt,
Status: "attempted",
}
if result != nil {
artifact.Status = result.Status
artifact.Upload = &state.DistributorUploadResult{
RunID: result.RunID,
Status: result.UploadStatus,
}
if result.PipelineID != "" || !result.AcceptedAt.IsZero() || result.StartedAt != nil || result.FinishedAt != nil || len(result.Report) > 0 || result.Error != "" {
artifact.RunStatus = &state.DistributorRunStatus{
RunID: result.RunID,
PipelineID: result.PipelineID,
Status: result.Status,
AcceptedAt: result.AcceptedAt,
StartedAt: result.StartedAt,
FinishedAt: result.FinishedAt,
Report: append([]byte(nil), result.Report...),
Error: result.Error,
}
}
artifact.StatusError = result.StatusError
}
if notifyErr != nil {
artifact.Status = "failed"
artifact.Error = notifyErr.Error()
}
if artifact.Status == "" {
artifact.Status = "unknown"
}
return store.SaveDistributorNotification(ctx, resolved, artifact)
}
type noopNotifier struct{} type noopNotifier struct{}
func (noopNotifier) Notify(context.Context, NotificationRequest) (*NotificationResult, error) { func (noopNotifier) Notify(context.Context, NotificationRequest) (*NotificationResult, error) {
@@ -641,16 +720,28 @@ func (n distributorNotifier) Notify(ctx context.Context, req NotificationRequest
IdempotencyKey: req.IdempotencyKey, IdempotencyKey: req.IdempotencyKey,
SourcePath: req.ReportPath, SourcePath: req.ReportPath,
BundlePath: req.BundlePath, BundlePath: req.BundlePath,
CreatedAt: req.CreatedAt,
}) })
if err != nil { notification := &NotificationResult{
return nil, err
}
return &NotificationResult{
BundleID: req.BundleID, BundleID: req.BundleID,
IdempotencyKey: req.IdempotencyKey, IdempotencyKey: req.IdempotencyKey,
RunID: result.RunID, RunID: result.RunID,
Status: result.Status, Status: result.Status,
}, nil UploadStatus: result.UploadStatus,
StatusError: result.StatusError,
}
if result.RunStatus != nil {
notification.PipelineID = result.RunStatus.PipelineID
notification.AcceptedAt = result.RunStatus.AcceptedAt
notification.StartedAt = result.RunStatus.StartedAt
notification.FinishedAt = result.RunStatus.FinishedAt
notification.Report = append([]byte(nil), result.RunStatus.Report...)
notification.Error = result.RunStatus.Error
}
if err != nil {
return notification, err
}
return notification, nil
} }
func BuildBriefing(req BriefingRequest, bundle *forecast.Bundle) (briefing.Package, error) { func BuildBriefing(req BriefingRequest, bundle *forecast.Bundle) (briefing.Package, error) {

View File

@@ -297,7 +297,13 @@ func TestGenerateReportNotifiesManagedReportPath(t *testing.T) {
t.Fatalf("ResolveGenerate() error = %v", err) t.Fatalf("ResolveGenerate() error = %v", err)
} }
notifier := &recordingNotifier{ notifier := &recordingNotifier{
result: &NotificationResult{RunID: "distributor-run", Status: "accepted"}, result: &NotificationResult{
RunID: "distributor-run",
Status: "succeeded",
UploadStatus: "accepted",
PipelineID: "reports",
Report: []byte(`{"actions":[{"action":"replace_older"}]}`),
},
} }
outputPath := filepath.Join(t.TempDir(), "daily-copy.md") outputPath := filepath.Join(t.TempDir(), "daily-copy.md")
@@ -314,8 +320,18 @@ func TestGenerateReportNotifiesManagedReportPath(t *testing.T) {
if result.Notification == nil { if result.Notification == nil {
t.Fatal("Notification = nil, want notification result") t.Fatal("Notification = nil, want notification result")
} }
if result.Notification.RunID != "distributor-run" || result.Notification.Status != "accepted" { if result.Notification.RunID != "distributor-run" || result.Notification.Status != "succeeded" {
t.Fatalf("Notification = %#v, want accepted distributor run", result.Notification) t.Fatalf("Notification = %#v, want succeeded distributor run", result.Notification)
}
if result.NotificationPath == "" || result.Metadata.NotificationPath != result.NotificationPath {
t.Fatalf("NotificationPath result=%q metadata=%q, want linked artifact", result.NotificationPath, result.Metadata.NotificationPath)
}
notificationData, err := os.ReadFile(result.NotificationPath)
if err != nil {
t.Fatalf("read notification artifact: %v", err)
}
if !strings.Contains(string(notificationData), `"replace_older"`) || !strings.Contains(string(notificationData), `"bundleCreated"`) {
t.Fatalf("notification artifact missing status report or created timestamp:\n%s", string(notificationData))
} }
if len(notifier.requests) != 1 { if len(notifier.requests) != 1 {
t.Fatalf("notification requests = %d, want 1", len(notifier.requests)) t.Fatalf("notification requests = %d, want 1", len(notifier.requests))
@@ -339,6 +355,9 @@ func TestGenerateReportNotifiesManagedReportPath(t *testing.T) {
if req.RunID != result.Metadata.RunID { if req.RunID != result.Metadata.RunID {
t.Fatalf("notification RunID = %q, want report run id %q", req.RunID, result.Metadata.RunID) t.Fatalf("notification RunID = %q, want report run id %q", req.RunID, result.Metadata.RunID)
} }
if !req.CreatedAt.Equal(result.Metadata.GeneratedAt) {
t.Fatalf("notification CreatedAt = %s, want generated at %s", req.CreatedAt, result.Metadata.GeneratedAt)
}
} }
func TestGenerateReportNotificationFailureFailsReport(t *testing.T) { func TestGenerateReportNotificationFailureFailsReport(t *testing.T) {
@@ -356,10 +375,15 @@ func TestGenerateReportNotificationFailureFailsReport(t *testing.T) {
} }
notifier := &recordingNotifier{err: errors.New("upload rejected")} notifier := &recordingNotifier{err: errors.New("upload rejected")}
store, err := state.NewFilesystemStore(cfg.Workspace)
if err != nil {
t.Fatalf("NewFilesystemStore() error = %v", err)
}
_, err = GenerateReport(context.Background(), ReportRequest{ _, err = GenerateReport(context.Background(), ReportRequest{
Config: cfg, Config: cfg,
Resolved: resolved, Resolved: resolved,
Renderer: successfulRenderer("# Daily Report\n"), Renderer: successfulRenderer("# Daily Report\n"),
Store: store,
Notifier: notifier, Notifier: notifier,
}) })
if err == nil { if err == nil {
@@ -371,6 +395,21 @@ func TestGenerateReportNotificationFailureFailsReport(t *testing.T) {
if len(notifier.requests) != 1 { if len(notifier.requests) != 1 {
t.Fatalf("notification requests = %d, want one attempted notification", len(notifier.requests)) t.Fatalf("notification requests = %d, want one attempted notification", len(notifier.requests))
} }
paths, pathErr := store.Paths(resolved)
if pathErr != nil {
t.Fatalf("Paths() error = %v", pathErr)
}
notificationData, readErr := os.ReadFile(paths.Notification)
if readErr != nil {
t.Fatalf("read notification artifact after failure: %v", readErr)
}
var notification state.DistributorNotificationArtifact
if err := json.Unmarshal(notificationData, &notification); err != nil {
t.Fatalf("decode notification artifact: %v", err)
}
if notification.Status != "failed" || !strings.Contains(notification.Error, "upload rejected") {
t.Fatalf("notification failure artifact = %+v, want failed upload context", notification)
}
} }
func TestGenerateReportDoesNotNotifyAfterRenderOrRunFailure(t *testing.T) { func TestGenerateReportDoesNotNotifyAfterRenderOrRunFailure(t *testing.T) {
@@ -1467,6 +1506,7 @@ func (n *recordingNotifier) Notify(_ context.Context, req NotificationRequest) (
BundleID: req.BundleID, BundleID: req.BundleID,
IdempotencyKey: req.IdempotencyKey, IdempotencyKey: req.IdempotencyKey,
Status: "accepted", Status: "accepted",
UploadStatus: "accepted",
}, nil }, nil
} }

View File

@@ -75,11 +75,12 @@ type ScriptoriumConfig struct {
} }
type WorkspaceConfig struct { type WorkspaceConfig struct {
Root string `yaml:"root"` Root string `yaml:"root"`
SnapshotsDir string `yaml:"snapshots_dir"` SnapshotsDir string `yaml:"snapshots_dir"`
ReportsDir string `yaml:"reports_dir"` ReportsDir string `yaml:"reports_dir"`
DataPackagesDir string `yaml:"data_packages_dir"` DataPackagesDir string `yaml:"data_packages_dir"`
PreflightDir string `yaml:"preflight_dir"` PreflightDir string `yaml:"preflight_dir"`
NotificationsDir string `yaml:"notifications_dir"`
} }
type DaypartConfig struct { type DaypartConfig struct {

View File

@@ -96,6 +96,9 @@ func TestLoadMinimalExampleConfig(t *testing.T) {
if cfg.Workspace.Root != "workspace" { if cfg.Workspace.Root != "workspace" {
t.Fatalf("Workspace.Root = %q, want default workspace", cfg.Workspace.Root) t.Fatalf("Workspace.Root = %q, want default workspace", cfg.Workspace.Root)
} }
if cfg.Workspace.NotificationsDir != "notifications" {
t.Fatalf("Workspace.NotificationsDir = %q, want notifications", cfg.Workspace.NotificationsDir)
}
if cfg.Location.Name != "Brentwood" { if cfg.Location.Name != "Brentwood" {
t.Fatalf("Location.Name = %q, want default Brentwood", cfg.Location.Name) t.Fatalf("Location.Name = %q, want default Brentwood", cfg.Location.Name)
} }

View File

@@ -42,11 +42,12 @@ func Defaults() Config {
Timeout: 2 * time.Minute, Timeout: 2 * time.Minute,
}, },
Workspace: WorkspaceConfig{ Workspace: WorkspaceConfig{
Root: "workspace", Root: "workspace",
SnapshotsDir: "snapshots", SnapshotsDir: "snapshots",
ReportsDir: "reports", ReportsDir: "reports",
DataPackagesDir: "data-packages", DataPackagesDir: "data-packages",
PreflightDir: "preflight", PreflightDir: "preflight",
NotificationsDir: "notifications",
}, },
Dayparts: []DaypartConfig{ Dayparts: []DaypartConfig{
{Name: "overnight", Start: "00:00", End: "06:00"}, {Name: "overnight", Start: "00:00", End: "06:00"},

View File

@@ -18,11 +18,12 @@ import (
) )
type FilesystemStore struct { type FilesystemStore struct {
root string root string
snapshotsDir string snapshotsDir string
reportsDir string reportsDir string
dataPackagesDir string dataPackagesDir string
preflightDir string preflightDir string
notificationsDir string
} }
type ArtifactPaths struct { type ArtifactPaths struct {
@@ -30,6 +31,7 @@ type ArtifactPaths struct {
Metadata string `json:"metadata"` Metadata string `json:"metadata"`
DataPackage string `json:"dataPackage"` DataPackage string `json:"dataPackage"`
Preflight string `json:"preflight"` Preflight string `json:"preflight"`
Notification string `json:"notification,omitempty"`
RenderedReport string `json:"renderedReport,omitempty"` RenderedReport string `json:"renderedReport,omitempty"`
} }
@@ -57,17 +59,19 @@ func NewFilesystemStore(cfg config.WorkspaceConfig) (*FilesystemStore, error) {
"reports_dir": cfg.ReportsDir, "reports_dir": cfg.ReportsDir,
"data_packages_dir": cfg.DataPackagesDir, "data_packages_dir": cfg.DataPackagesDir,
"preflight_dir": cfg.PreflightDir, "preflight_dir": cfg.PreflightDir,
"notifications_dir": cfg.NotificationsDir,
} { } {
if err := validateRelativeDir(name, value); err != nil { if err := validateRelativeDir(name, value); err != nil {
return nil, err return nil, err
} }
} }
return &FilesystemStore{ return &FilesystemStore{
root: filepath.Clean(cfg.Root), root: filepath.Clean(cfg.Root),
snapshotsDir: filepath.Clean(cfg.SnapshotsDir), snapshotsDir: filepath.Clean(cfg.SnapshotsDir),
reportsDir: filepath.Clean(cfg.ReportsDir), reportsDir: filepath.Clean(cfg.ReportsDir),
dataPackagesDir: filepath.Clean(cfg.DataPackagesDir), dataPackagesDir: filepath.Clean(cfg.DataPackagesDir),
preflightDir: filepath.Clean(cfg.PreflightDir), preflightDir: filepath.Clean(cfg.PreflightDir),
notificationsDir: filepath.Clean(cfg.NotificationsDir),
}, nil }, nil
} }
@@ -90,6 +94,7 @@ func (s *FilesystemStore) Paths(resolved report.Resolved) (ArtifactPaths, error)
Metadata: s.join(s.snapshotsDir, group, validDate, filenameBase+".metadata.json"), Metadata: s.join(s.snapshotsDir, group, validDate, filenameBase+".metadata.json"),
DataPackage: s.join(s.dataPackagesDir, group, validDate, filenameBase+".data_package.json"), DataPackage: s.join(s.dataPackagesDir, group, validDate, filenameBase+".data_package.json"),
Preflight: s.join(s.preflightDir, group, validDate, filenameBase+".render.json"), Preflight: s.join(s.preflightDir, group, validDate, filenameBase+".render.json"),
Notification: s.join(s.notificationsDir, group, validDate, filenameBase+".distributor.json"),
RenderedReport: s.join(s.reportsDir, group, filenameBase+".md"), RenderedReport: s.join(s.reportsDir, group, filenameBase+".md"),
}, nil }, nil
} }
@@ -130,6 +135,20 @@ func (s *FilesystemStore) SavePreflight(_ context.Context, resolved report.Resol
return paths.Preflight, nil return paths.Preflight, nil
} }
func (s *FilesystemStore) SaveDistributorNotification(_ context.Context, resolved report.Resolved, artifact DistributorNotificationArtifact) (string, error) {
paths, err := s.Paths(resolved)
if err != nil {
return "", err
}
if artifact.SchemaVersion == "" {
artifact.SchemaVersion = DistributorNotificationSchemaVersion
}
if err := fileutil.WriteJSONAtomic(paths.Notification, artifact); err != nil {
return "", err
}
return paths.Notification, nil
}
func (s *FilesystemStore) PrepareRenderedReport(_ context.Context, resolved report.Resolved) (string, error) { func (s *FilesystemStore) PrepareRenderedReport(_ context.Context, resolved report.Resolved) (string, error) {
paths, err := s.Paths(resolved) paths, err := s.Paths(resolved)
if err != nil { if err != nil {

View File

@@ -30,6 +30,7 @@ func TestPathsUseRunIDAndWorkspace(t *testing.T) {
filepath.Join("snapshots", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.metadata.json"), filepath.Join("snapshots", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.metadata.json"),
filepath.Join("data-packages", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.data_package.json"), filepath.Join("data-packages", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.data_package.json"),
filepath.Join("preflight", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.render.json"), filepath.Join("preflight", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.render.json"),
filepath.Join("notifications", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_today.distributor.json"),
filepath.Join("reports", "daily", "20260529T100000.000000000Z_daily_today.md"), filepath.Join("reports", "daily", "20260529T100000.000000000Z_daily_today.md"),
} { } {
if !strings.Contains(pathsString(paths), want) { if !strings.Contains(pathsString(paths), want) {
@@ -59,6 +60,22 @@ func TestSaveArtifactsAndMetadataRoundTrip(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("SavePreflight() error = %v", err) t.Fatalf("SavePreflight() error = %v", err)
} }
notificationPath, err := store.SaveDistributorNotification(context.Background(), resolved, DistributorNotificationArtifact{
RunID: resolved.Metadata().RunID,
ReportID: resolved.Definition.ID,
AttemptedAt: resolved.GeneratedAt,
Endpoint: "https://distributor.example.test",
BundleID: "weatherreporter.home.daily.run",
IdempotencyKey: "weatherreporter.home.daily.run",
SourcePath: "/tmp/report.md",
BundlePath: "daily.md",
BundleCreated: resolved.GeneratedAt,
Status: "succeeded",
RunStatus: &DistributorRunStatus{RunID: "distributor-run", Status: "succeeded"},
})
if err != nil {
t.Fatalf("SaveDistributorNotification() error = %v", err)
}
renderedReportPath, err := store.PrepareRenderedReport(context.Background(), resolved) renderedReportPath, err := store.PrepareRenderedReport(context.Background(), resolved)
if err != nil { if err != nil {
t.Fatalf("PrepareRenderedReport() error = %v", err) t.Fatalf("PrepareRenderedReport() error = %v", err)
@@ -77,6 +94,17 @@ func TestSaveArtifactsAndMetadataRoundTrip(t *testing.T) {
if preflight.Stdout != `{"ok":true}` { if preflight.Stdout != `{"ok":true}` {
t.Fatalf("preflight stdout = %q, want render stdout", preflight.Stdout) t.Fatalf("preflight stdout = %q, want render stdout", preflight.Stdout)
} }
var notification DistributorNotificationArtifact
notificationData, err := os.ReadFile(notificationPath)
if err != nil {
t.Fatalf("read notification: %v", err)
}
if err := json.Unmarshal(notificationData, &notification); err != nil {
t.Fatalf("decode notification: %v", err)
}
if notification.SchemaVersion != DistributorNotificationSchemaVersion || notification.RunStatus == nil || notification.RunStatus.Status != "succeeded" {
t.Fatalf("notification = %#v, want persisted distributor status", notification)
}
paths, err := store.Paths(resolved) paths, err := store.Paths(resolved)
if err != nil { if err != nil {
t.Fatalf("Paths() error = %v", err) t.Fatalf("Paths() error = %v", err)
@@ -93,7 +121,7 @@ func TestSaveArtifactsAndMetadataRoundTrip(t *testing.T) {
t.Fatalf("SaveMetadata() error = %v", err) t.Fatalf("SaveMetadata() error = %v", err)
} }
for _, path := range []string{briefingPath, dataPackagePath, preflightPath, renderedReportPath, metadataPath} { for _, path := range []string{briefingPath, dataPackagePath, preflightPath, notificationPath, renderedReportPath, metadataPath} {
if _, err := os.Stat(path); err != nil { if _, err := os.Stat(path); err != nil {
t.Fatalf("expected artifact %q: %v", path, err) t.Fatalf("expected artifact %q: %v", path, err)
} }
@@ -494,6 +522,7 @@ func pathsString(paths ArtifactPaths) string {
paths.Metadata, paths.Metadata,
paths.DataPackage, paths.DataPackage,
paths.Preflight, paths.Preflight,
paths.Notification,
paths.RenderedReport, paths.RenderedReport,
}, "\n") }, "\n")
} }

View File

@@ -29,6 +29,7 @@ type Metadata struct {
BriefingPath string `json:"briefingPath"` BriefingPath string `json:"briefingPath"`
DataPackagePath string `json:"dataPackagePath"` DataPackagePath string `json:"dataPackagePath"`
PreflightPath string `json:"preflightPath"` PreflightPath string `json:"preflightPath"`
NotificationPath string `json:"notificationPath,omitempty"`
RenderedReportPath string `json:"renderedReportPath,omitempty"` RenderedReportPath string `json:"renderedReportPath,omitempty"`
} }

View File

@@ -3,6 +3,8 @@ package state
import ( import (
"context" "context"
"encoding/json"
"time"
"gitea.maximumdirect.net/eric/weatherreporter/internal/briefing" "gitea.maximumdirect.net/eric/weatherreporter/internal/briefing"
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput" "gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
@@ -14,6 +16,7 @@ type Store interface {
SaveBriefing(context.Context, report.Resolved, briefing.Package) (string, error) SaveBriefing(context.Context, report.Resolved, briefing.Package) (string, error)
SaveDataPackage(context.Context, report.Resolved, promptinput.Package) (string, error) SaveDataPackage(context.Context, report.Resolved, promptinput.Package) (string, error)
SavePreflight(context.Context, report.Resolved, PreflightArtifact) (string, error) SavePreflight(context.Context, report.Resolved, PreflightArtifact) (string, error)
SaveDistributorNotification(context.Context, report.Resolved, DistributorNotificationArtifact) (string, error)
PrepareRenderedReport(context.Context, report.Resolved) (string, error) PrepareRenderedReport(context.Context, report.Resolved) (string, error)
SaveMetadata(context.Context, Metadata) (string, error) SaveMetadata(context.Context, Metadata) (string, error)
FindPriorSnapshot(context.Context, report.Resolved) (*PriorSnapshot, error) FindPriorSnapshot(context.Context, report.Resolved) (*PriorSnapshot, error)
@@ -33,3 +36,39 @@ type PreflightArtifact struct {
StderrTruncated bool `json:"stderrTruncated,omitempty"` StderrTruncated bool `json:"stderrTruncated,omitempty"`
ExitCode int `json:"exitCode"` ExitCode int `json:"exitCode"`
} }
const DistributorNotificationSchemaVersion = "weatherreporter.distributor_notification.v1"
type DistributorNotificationArtifact struct {
SchemaVersion string `json:"schemaVersion"`
RunID string `json:"runId"`
ReportID report.ID `json:"reportId"`
AttemptedAt time.Time `json:"attemptedAt"`
Endpoint string `json:"endpoint"`
BundleID string `json:"bundleId,omitempty"`
IdempotencyKey string `json:"idempotencyKey,omitempty"`
SourcePath string `json:"sourcePath,omitempty"`
BundlePath string `json:"bundlePath,omitempty"`
BundleCreated time.Time `json:"bundleCreated,omitempty"`
Status string `json:"status"`
Upload *DistributorUploadResult `json:"upload,omitempty"`
RunStatus *DistributorRunStatus `json:"runStatus,omitempty"`
StatusError string `json:"statusError,omitempty"`
Error string `json:"error,omitempty"`
}
type DistributorUploadResult struct {
RunID string `json:"runId,omitempty"`
Status string `json:"status,omitempty"`
}
type DistributorRunStatus struct {
RunID string `json:"runId,omitempty"`
PipelineID string `json:"pipelineId,omitempty"`
Status string `json:"status,omitempty"`
AcceptedAt time.Time `json:"acceptedAt,omitempty"`
StartedAt *time.Time `json:"startedAt,omitempty"`
FinishedAt *time.Time `json:"finishedAt,omitempty"`
Report json.RawMessage `json:"report,omitempty"`
Error string `json:"error,omitempty"`
}